Azure Data Lead Engineer

About Position:

We are looking for Lead Azure Data engineer who have experience in Azure data bricks, Data factory, to join our development team.

• Role: Azure Data Lead Engineer
• Location: Pune/ Bangalore
• Experience: 8 to 10 years
• Job Type: Full Time Employment

What You'll Do:

• Design and implement complex, scalable data pipelines for ingestion, processing, and transformation using Azure Databricks technologies.
• Collaborate with Architects, Data Analysts, and Business Analysts to understand data requirements and develop efficient workflows.
• Develop and manage data storage solutions including Azure SQL Database, Data Lake, and Blob Storage.
• Leverage Azure Data Factory and other cloud-native tools to build and maintain ETL processes.
• Conduct unit testing and ensure quality of data pipelines; mentor junior engineers and review their deliverables.
• Monitor pipeline performance, troubleshoot issues, and provide regular status updates.
• Optimise data workflows for performance and cost-efficiency; implement automation to reduce manual effort.

Expertise You'll Bring:

• Around 8+ years of experience in Databricks and Azure cloud Data migration
• Experience on migrating and re-modeling large enterprise data from legacy warehouse to Lakehouse (Delta Lake) on MS Fabric or Databricks.
• Strong Data Engineering Skills: Proficiency in data extraction, transformation, and loading (ETL) processes, data modeling, and database management.
• Also experience around setting up pipelines in Notebooks and ADF, setting up monitoring and alert notifications.
• Experience with Data Lake Technologies: MS Fabric, Azure, Databricks, Python, Orchestration tool like Apache Airflow or Azure Data Factory, Azure Synapse along with stored procedures, Azure data lake storage.
• Data Integration Knowledge: Familiarity with data integration techniques, including batch processing, streaming, and real-time data ingestion, auto-loader, change data capture, creation of fact and dimension tables.
• Programming Skills: Proficiency in PySQL, SQL, Python, Pyspark for data manipulation and transformation.
• Collaboration and Communication: Strong interpersonal skills to collaborate effectively with cross-functional teams and communicate technical concepts to non-technical stakeholders.

Benefits:

• Competitive salary and benefits package
• Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications
• Opportunity to work with cutting-edge technologies
• Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards
• Annual health check-ups
• Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents

Values-Driven, People-Centric & Inclusive Work Environment:

Persistent Ltd. is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds.

• We support hybrid work and flexible hours to fit diverse lifestyles.
• Our office is accessibility-friendly, with ergonomic setups and assistive technologies to support employees with physical disabilities.
• If you are a person with disabilities and have specific requirements, please inform us during the application process or at any time during your employment

Let's unleash your full potential at Persistent -

"Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind."