Azure Data Engineer Lead

Job Description Creative Synergies Group Overview - Creative is a leading Global Technology Services firm delivering innovative solutions that build upon synergies between our Deep Engineering Domain Expertise and Digital Engineering Services - We work with clients across the Global Transportation, Energy/Process, Industrial Products, Hi-Tech, and EdTech verticals - Creative's global team of 1,500+ Can-Do engineers collaborate with our 40+ Fortune 500 clients to help them achieve their strategic and operational objectives (Including Google, GM, Samsung, Tesla, Mercedes, Sony, Mitsubishi) - 95% of our revenue comes from US, Europe, Japan based customers Service Offerings: Data, Analytics, and AI Engineering Services Digital Product and Platform Engineering Intelligent Connected Product Engineering. - Vehicle Centric Electronics - Connected Car Solutions - Mobile Robotics - Industrial IoT Digital Manufacturing Digital Plant Engineering Founder / CEO: Dr. Mukesh Gandhi Former Michigan State University Professor (1984 - 1999) - Published Ground-Breaking Technical Publications and Books - Supervised PhD, Masters and Undergraduate Research Students - Who Are Now Leaders and Decision Makers In Industry And Academia - Awarded Over $15 Million For Pioneering Research from National Science Foundation (NSF), US Army Research Office (ARO), Defense Advanced Research Projects Agency (DARPA), US National Institute of Standards and Technology (NIST) Founder / CEO of Quantech Global Services - Pioneering US-Indian Services company to serve major global automotive OEMs (GM, Ford, Nissan, Peugeot) from India-based delivery centers in the early 2000s - Acquired by Wipro (NYSE:WIT) Former Wipro, EDS Chief Executive Founded Creative Synergies Group In 2011 Locations: Headquartered in the U.S., multiple delivery centers in India (Bengaluru, Pune), branch offices in Germany, U.K, Netherlands and Japan Culture: Creative has a flat organization and an agile culture of positivity, entrepreneurial spirit, customer centricity, teamwork, and meritocracy Creative Synergies Group 3 Year Vision - Continue growing overall business at 40% per year for the next 3 years - Establish Leadership in AI/ML Engineering Services - Substantially expand revenues in Electronics, Connected Vehicle Solutions, Mobile Robots, Industrial IoT - Meaningfully expand into EdTech - Explore niche acquisitions in the U.S. Job Title: Azure Data Engineer Lead - (Availability to join within 4 weeks is highly preferable) Experience: 8-12 years Job Summary: We are seeking a skilled and motivated Senior Azure Data Engineer with 8-12 years of relevant experience. Candidate will play a pivotal role in designing, implementing, and optimizing large-scale data solutions on the Microsoft Azure platform. Candidate will work closely with other cross-functional teams to ensure data solutions are scalable, secure, and high-performance. The ideal candidate will have a strong background in data engineering, deep expertise in Azure Databricks and proven leadership skills to guide and mentor the team. Key Responsibilities: - Architect and Design Data Solutions: Lead the design and implementation of scalable, secure and high-performing data solutions on Azure, ensuring alignment with business goals and technical requirements. - Data Pipeline Development: Build and optimize end-to-end data pipelines to collect, process and analyze data from various sources using Azure Databricks, Azure Data Factory and other Azure data services. - Cloud Infrastructure Management: Architect and maintain cloud data infrastructure using Azure Data Lake, Azure Synapse Analytics, Azure SQL Data Warehouse and Azure Blob Storage. - Collaborate with Cross-functional Teams: Work closely with data architects, business analysts and developers to gather requirements, define data models and ensure smooth integration of data across systems. - Performance Tuning and Optimization: Continuously monitor and optimize the performance of data workflows and pipelines to ensure efficient data processing. - Data Governance & Security: Implement best practices for data governance, security and compliance, ensuring that data solutions adhere to company and regulatory standards. - Mentor and Lead Teams: Provide technical leadership and mentorship to junior data engineers, guiding them in the best practices of data engineering and cloud technologies. Required Qualifications - Bachelor's or Master's degree in Computer Science, Engineering, or a related field - 8-12 years of experience in data engineering, with at least 5+ years of hands-on experience with Microsoft Azure cloud technologies. - Strong experience with Azure Databricks, Azure Data Factory, Azure Synapse Analytics, Azure SQL Database, Azure Blob Storage, Azure Data Lake and other related Azure services. - Hands-on experience using these services: PySpark, Notebooks, Structured Streaming, Unity Catalog, Delta Live Tables, Workflows and SQL Warehouse - In-depth knowledge of data modeling, ETL/ELT processes, data warehousing and data pipeline architecture. - Proven experience in building and optimizing large-scale data pipelines in a cloud environment. - Strong experience with SQL, NoSQL databases and big data processing technologies (e.g., Spark, Hadoop, Kafka). - Hands-on experience with data transformation, data integration and data quality frameworks. - Expertise in data security and compliance standards in a cloud-based environment. - Ability to design and implement data governance strategies for large datasets. - Strong leadership, communication and collaboration skills, with a proven ability to guide teams and manage complex projects. Good to have: - Microsoft Azure Data Engineer Certification - Databricks Certified Data Engineer Professional - Familiarity with GenAI Concepts - Familiar with DevOps Practices