Azure Data Engineer Lead

Creative Synergies Group OverviewCreative is a leading Global Technology Services firm delivering innovative solutions that build upon synergies between our Deep Engineering Domain Expertise and Digital Engineering Services We work with clients across the Global Transportation, Energy/Process, Industrial Products, Hi-Tech, and EdTech verticals Creative’s global team of 1,500+ “Can-Do” engineers collaborate with our 40+ Fortune 500 clients to help them achieve their strategic and operational objectives (Including Google, GM, Samsung, Tesla, Mercedes, Sony, Mitsubishi) 95% of our revenue comes from US, Europe, Japan based customersService Offerings:Data, Analytics, and AI Engineering Services Digital Product and Platform Engineering Intelligent Connected Product Engineering. Vehicle Centric Electronics Connected Car Solutions Mobile Robotics Industrial IoT Digital Manufacturing Digital Plant EngineeringFounder / CEO: Dr. Mukesh Gandhi Former Michigan State University Professor (1984 - 1999) Published Ground-Breaking Technical Publications and Books Supervised PhD, Masters and Undergraduate Research Students - Who Are Now Leaders and Decision Makers In Industry And Academia Awarded Over $15 Million For Pioneering Research from National Science Foundation (NSF), US Army Research Office (ARO), Defense Advanced Research Projects Agency (DARPA), US National Institute of Standards and Technology (NIST)Founder / CEO of Quantech Global Services Pioneering US-Indian Services company to serve major global automotive OEMs (GM, Ford, Nissan, Peugeot) from India-based delivery centers in the early 2000s Acquired by Wipro (NYSE:WIT)Former Wipro, EDS Chief Executive Founded Creative Synergies Group In 2011Locations:Headquartered in the U.S., multiple delivery centers in India (Bengaluru, Pune), branch offices in Germany, U.K, Netherlands and Japan Culture:Creative has a flat organization and an agile culture of positivity, entrepreneurial spirit, customer centricity, teamwork, and meritocracyCreative Synergies Group 3 Year VisionContinue growing overall business at 40% per year for the next 3 years Establish Leadership in AI/ML Engineering Services Substantially expand revenues in Electronics, Connected Vehicle Solutions, Mobile Robots, Industrial IoT Meaningfully expand into EdTech Explore niche acquisitions in the U.S.Job Title:Azure Data Engineer Lead -(Availability to join within 4 weeks is highly preferable) Experience:8-12 yearsJob Summary:We are seeking a skilled and motivated Senior Azure Data Engineer with 8-12 years of relevant experience. Candidate will play a pivotal role in designing, implementing, and optimizing large-scale data solutions on the Microsoft Azure platform. Candidate will work closely with other cross-functional teams to ensure data solutions are scalable, secure, and high-performance. The ideal candidate will have a strong background in data engineering, deep expertise in Azure Databricks and proven leadership skills to guide and mentor the team.Key Responsibilities:Architect and Design Data Solutions: Lead the design and implementation of scalable, secure and high-performing data solutions on Azure, ensuring alignment with business goals and technical requirements. Data Pipeline Development: Build and optimize end-to-end data pipelines to collect, process and analyze data from various sources using Azure Databricks, Azure Data Factory and other Azure data services. Cloud Infrastructure Management: Architect and maintain cloud data infrastructure using Azure Data Lake, Azure Synapse Analytics, Azure SQL Data Warehouse and Azure Blob Storage. Collaborate with Cross-functional Teams: Work closely with data architects, business analysts and developers to gather requirements, define data models and ensure smooth integration of data across systems. Performance Tuning and Optimization: Continuously monitor and optimize the performance of data workflows and pipelines to ensure efficient data processing. Data Governance & Security: Implement best practices for data governance, security and compliance, ensuring that data solutions adhere to company and regulatory standards. Mentor and Lead Teams: Provide technical leadership and mentorship to junior data engineers, guiding them in the best practices of data engineering and cloud technologies.Required QualificationsBachelor's or Master’s degree in Computer Science, Engineering, or a related field8-12 years of experience in data engineering, with at least 5+ years of hands-on experience with Microsoft Azure cloud technologies. Strong experience with Azure Databricks, Azure Data Factory, Azure Synapse Analytics, Azure SQL Database, Azure Blob Storage, Azure Data Lake and other related Azure services. Hands-on experience using these services: PySpark, Notebooks, Structured Streaming, Unity Catalog, Delta Live Tables, Workflows and SQL Warehouse In-depth knowledge of data modeling, ETL/ELT processes, data warehousing and data pipeline architecture. Proven experience in building and optimizing large-scale data pipelines in a cloud environment. Strong experience with SQL, NoSQL databases and big data processing technologies (e.g., Spark, Hadoop, Kafka). Hands-on experience with data transformation, data integration and data quality frameworks. Expertise in data security and compliance standards in a cloud-based environment. Ability to design and implement data governance strategies for large datasets. Strong leadership, communication and collaboration skills, with a proven ability to guide teams and manage complex projects.Good to have:Microsoft Azure Data Engineer Certification Databricks Certified Data Engineer Professional Familiarity with GenAI Concepts Familiar with DevOps Practices