Azure Data Engineer

2 weeks ago

Gurgaon, Haryana, India Infogain Full time ₹ 8,00,000 - ₹ 12,00,000 per year

ROLES & RESPONSIBILITIES

Key Responsibilities

  • Analyze existing Hadoop, Pig, and Spark scripts from Dataproc and refactor them into Databricks-native PySpark.
  • Implement data ingestion and transformation pipelines using Delta Lake best practices.
  • Apply conversion rules and templates for automated code migration and testing.
  • Conduct data validation between legacy and migrated environments (schema, count, and data-level checks).
  • Collaborate on developing AI-driven tools for code conversion, dependency extraction, and error remediation.
  • Ensure best practices for code versioning, error handling, and performance optimization.
  • Participate in UAT, troubleshooting, and post-migration validation activities.

Technical Skills

  • Core: Python, PySpark, SQL
  • Databricks: Delta Lake, Unity Catalog, Databricks Workflows, MLflow (basic understanding)
  • GCP: Dataproc, BigQuery, GCS, Composer/Airflow, Cloud Functions
  • Data Engineering: Hadoop, Hive, Pig, Spark SQL
  • Automation: Experience with migration utilities or AI-assisted code transformation tools
  • CI/CD: Git, Jenkins, Terraform (preferred)
  • Validation: Data comparison utilities (Delta-to-Delta, DataFrame diffing, schema validation)

Preferred Experience

  • 5–8 years in data engineering or big data application development.
  • Hands-on experience migrating Spark or Hadoop workloads to Databricks.
  • Familiarity with Delta architecture, data quality frameworks, and GCP cloud integration.
  • Exposure to GenAI-based tools for automation or code refactoring is a plus.

EXPERIENCE

6-8 Years

SKILLS

Primary Skill: Data Engineering

Sub Skill(s): Data Engineering

Additional Skill(s): Python, Apache Hadoop, Apache Hive, Apache Airflow, synapse, databricks, SQL, Apache Spark, Azure Data Factory, Pyspark, GenAI Fundamentals, Cloud Pub/Sub, BigQuery

ABOUT THE COMPANY

Infogain is a human-centered digital platform and software engineering company based out of Silicon Valley. We engineer business outcomes for Fortune 500 companies and digital natives in the technology, healthcare, insurance, travel, telecom, and retail & CPG industries using technologies such as cloud, microservices, automation, IoT, and artificial intelligence. We accelerate experience-led transformation in the delivery of digital platforms. Infogain is also a Microsoft (NASDAQ: MSFT) Gold Partner and Azure Expert Managed Services Provider (MSP).

Infogain, an Apax Funds portfolio company, has offices in California, Washington, Texas, the UK, the UAE, and Singapore, with delivery centers in Seattle, Houston, Austin, Kraków, Noida, Gurgaon, Mumbai, Pune, and Bengaluru.

  • Azure Data Engineer

    7 days ago

    Gurgaon, Haryana, India Metaphor Infotech Full time ₹ 8,00,000 - ₹ 12,00,000 per year

    Azure Data Engineer- Immediate joiners onlyPlease find the JD below for the same :We are looking for an experienced Data Engineer to drive the development of scalable, secure, and high-performance data solutions. This role requires deep technical expertise in Python, Apache Spark, Delta Lake, and orchestration tools like Databricks Workflows or Azure Data...

  • Lead Azure Data Engineer

    2 weeks ago

    Gurgaon, Haryana, India Naukri Healthcare Jobs Full time ₹ 1,50,00,000 - ₹ 3,00,00,000 per year

    We are looking for a skilled Lead Azure Data Engineer with 8 to 15 years of experience in the Pharmaceutical & Life Sciences industry, specifically at Syneos Health. The ideal candidate will have expertise in designing and implementing data pipelines using Azure.Roles and ResponsibilityDesign and implement scalable data pipelines using Azure.Collaborate with...

  • Azure Data Engineer

    4 days ago

    Gurgaon, Haryana, India INADEV Full time

    Essential Duties and ResponsibilitiesMinimum 3 years of experience working with Azure servicesExperience with Azure SQL Databases, Azure Data Factory, Azure Synapse, Azure Logic Apps, Azure Service Bus and Storage Accounts/Data Lake Gen 23+ years of experience working in an Azure CI/CD environmentDeep understanding of ADF parameterization and movement of...

  • Azure Data Engineer

    2 weeks ago

    Gurgaon, Haryana, India Infogain Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    ROLES & RESPONSIBILITIESKey ResponsibilitiesLead design and execution of Dataproc ? Databricks PySpark migration roadmap.Define modernization strategy, including data ingestion, transformation, orchestration, and governance.Architect scalable Delta Lake and Unity Catalog–based solutions.Manage and guide teams on code conversion, dependency mapping, and...

  • Azure Data Engineer

    2 weeks ago

    Gurgaon, Haryana, India Antal TECH jobs Full time ₹ 1,00,00,000 - ₹ 3,00,00,000 per year

    We are seeking an experienced Data Developer with expertise in Microsoft Fabric, Azure Synapse Analytics, Databricks, and strong SQL development skills. The ideal candidate will work on end-to-end data solutions supporting analytics initiatives across clinical, regulatory, and commercial domains in the Life Sciences industry.Key Responsibilities:• Design,...

  • Azure AI Engineer

    1 day ago

    Gurgaon, Haryana, India OTS Solutions Full time

    Hiring: Azure AI Engineer (Immediate Joiners Only)Location:Pune / Gurgaon (Hybrid)Employment Type:Full-timeWe are looking for an experiencedAzure AI Engineerto design and build enterprise-grade AI solutions and multi-agent systems on Azure.Mandatory Skills:• Azure AI Foundry• Agentic AI / Multi-Agent Systems• Azure AI Search• RAG Architecture (3–4...

  • Senior Data Engineer

    5 days ago

    Gurgaon, Haryana, India Pacific Data Integrators Full time

    Role: Senior Data EngineerLocation: RemoteJob Type: Full-timeShift time: Open to work in EST shift (5PM to 2AM IST)Key ResponsibilitiesLead the design, development, and implementation of complex data integration solutions using Informatica Intelligent Data Management Cloud (IDMC).Develop, document, unit test, and maintain high-quality ETL applications that...

  • Data Engineer

    1 day ago

    Gurgaon, Haryana, India EXL Full time

    DescriptionData Engineers design and build data systems and pipelines. Responsibilities include developing data processing workflows, optimizing data storage, and ensuring data accuracy. You will collaborate with data scientists and analysts to meet data requirements and resolve data issues. Strong experience in data engineering and problem-solving skills...

  • Data Engineer

    5 days ago

    Gurgaon, Haryana, India Senpiper PTY LTD Full time

    About the RoleWe are looking for a Data Engineer with strong hands-on expertise in Databricks and the Azure Data ecosystem. This role focuses on building and optimizing large-scale data pipelines, troubleshooting performance issues, and ensuring high data quality and governance across our cloud data platforms.Key ResponsibilitiesDesign, develop, and optimize...

  • Senior Data Engineer

    1 day ago

    Gurgaon, Haryana, India EXL Full time

    DescriptionSenior Data Engineers lead the development and optimization of complex data systems and pipelines. Responsibilities include managing data projects, ensuring data quality, and mentoring junior engineers. You will work with stakeholders to align data solutions with business goals and drive improvements. Extensive experience in data engineering and...