Azure Data Engineer Lead

Creative Synergies Group OverviewCreative is a leading Global Technology Services firm delivering innovative solutions that build upon synergies between our Deep Engineering Domain Expertise and Digital Engineering ServicesWe work with clients across the Global Transportation, Energy/Process, Industrial Products, Hi-Tech, and EdTech verticalsCreative’s global team of 1,500+ “Can-Do” engineers collaborate with our 40+ Fortune 500 clients to help them achieve their strategic and operational objectives (Including Google, GM, Samsung, Tesla, Mercedes, Sony, Mitsubishi)95% of our revenue comes from US, Europe, Japan based customersService Offerings:Data, Analytics, and AI Engineering ServicesDigital Product and Platform EngineeringIntelligent Connected Product Engineering.Vehicle Centric ElectronicsConnected Car SolutionsMobile RoboticsIndustrial IoTDigital ManufacturingDigital Plant EngineeringFounder / CEO: Dr. Mukesh GandhiFormer Michigan State University Professor (1984 - 1999)Published Ground-Breaking Technical Publications and BooksSupervised PhD, Masters and Undergraduate Research Students - Who Are Now Leaders and Decision Makers In Industry And AcademiaAwarded Over $15 Million For Pioneering Research from National Science Foundation (NSF), US Army Research Office (ARO), Defense Advanced Research Projects Agency (DARPA), US National Institute of Standards and Technology (NIST)Founder / CEO of Quantech Global ServicesPioneering US-Indian Services company to serve major global automotive OEMs (GM, Ford, Nissan, Peugeot) from India-based delivery centers in the early 2000sAcquired by Wipro (NYSE:WIT)Former Wipro, EDS Chief Executive Founded Creative Synergies Group In 2011Locations: Headquartered in the U.S., multiple delivery centers in India (Bengaluru, Pune), branch offices in Germany, U.K, Netherlands and JapanCulture: Creative has a flat organization and an agile culture of positivity, entrepreneurial spirit, customer centricity, teamwork, and meritocracyCreative Synergies Group 3 Year VisionContinue growing overall business at 40% per year for the next 3 yearsEstablish Leadership in AI/ML Engineering ServicesSubstantially expand revenues in Electronics, Connected Vehicle Solutions, Mobile Robots, Industrial IoTMeaningfully expand into EdTechExplore niche acquisitions in the U.S.Job Title: Azure Data Engineer Lead - (Availability to join within 4 weeks is highly preferable)Experience: 8-12 yearsJob Summary:We are seeking a skilled and motivated Senior Azure Data Engineer with 8-12 years of relevant experience. Candidate will play a pivotal role in designing, implementing, and optimizing large-scale data solutions on the Microsoft Azure platform. Candidate will work closely with other cross-functional teams to ensure data solutions are scalable, secure, and high-performance. The ideal candidate will have a strong background in data engineering, deep expertise in Azure Databricks and proven leadership skills to guide and mentor the team.Key Responsibilities:Architect and Design Data Solutions: Lead the design and implementation of scalable, secure and high-performing data solutions on Azure, ensuring alignment with business goals and technical requirements.Data Pipeline Development: Build and optimize end-to-end data pipelines to collect, process and analyze data from various sources using Azure Databricks, Azure Data Factory and other Azure data services.Cloud Infrastructure Management: Architect and maintain cloud data infrastructure using Azure Data Lake, Azure Synapse Analytics, Azure SQL Data Warehouse and Azure Blob Storage.Collaborate with Cross-functional Teams: Work closely with data architects, business analysts and developers to gather requirements, define data models and ensure smooth integration of data across systems.Performance Tuning and Optimization: Continuously monitor and optimize the performance of data workflows and pipelines to ensure efficient data processing.Data Governance & Security: Implement best practices for data governance, security and compliance, ensuring that data solutions adhere to company and regulatory standards.Mentor and Lead Teams: Provide technical leadership and mentorship to junior data engineers, guiding them in the best practices of data engineering and cloud technologies.Required QualificationsBachelor's or Master’s degree in Computer Science, Engineering, or a related field8-12 years of experience in data engineering, with at least 5+ years of hands-on experience with Microsoft Azure cloud technologies.Strong experience with Azure Databricks, Azure Data Factory, Azure Synapse Analytics, Azure SQL Database, Azure Blob Storage, Azure Data Lake and other related Azure services.Hands-on experience using these services: PySpark, Notebooks, Structured Streaming, Unity Catalog, Delta Live Tables, Workflows and SQL WarehouseIn-depth knowledge of data modeling, ETL/ELT processes, data warehousing and data pipeline architecture.Proven experience in building and optimizing large-scale data pipelines in a cloud environment.Strong experience with SQL, NoSQL databases and big data processing technologies (e.g., Spark, Hadoop, Kafka).Hands-on experience with data transformation, data integration and data quality frameworks.Expertise in data security and compliance standards in a cloud-based environment.Ability to design and implement data governance strategies for large datasets.Strong leadership, communication and collaboration skills, with a proven ability to guide teams and manage complex projects.Good to have:Microsoft Azure Data Engineer CertificationDatabricks Certified Data Engineer ProfessionalFamiliarity with GenAI ConceptsFamiliar with DevOps Practices