Senior Data Engineer

5 days ago

Greater Chennai Area, India Toppan Merrill Full time ₹ 12,00,000 - ₹ 36,00,000 per year

Job Description:

Responsibilities:

  • Develop & Optimize Data Pipelines

  • Build, test, and maintain ETL/ELT data pipelines using Azure Databricks & Apache Spark (PySpark).

  • Optimize performance and cost-efficiency of Spark jobs.
  • Ensure data quality through validation, monitoring, and alerting mechanisms.

  • Understand cluster types, configuration, and use-case for serverless

  • Implement Unity Catalog for Data Governance

  • Design and enforce access control policies using Unity Catalog.

  • Manage data lineage, auditing, and metadata governance.

  • Enable secure data sharing across teams and external stakeholders.

  • Integrate with Cloud Data Platforms

  • Work with Azure Data Lake Storage / Azure Blob Storage/ Azure Event Hub to integrate Databricks with cloud-based data lakes, data warehouses, and event streams.

  • Implement Delta Lake for scalable, ACID-compliant storage.

  • Automate & Orchestrate Workflows

  • Develop CI/CD pipelines for data workflows using Azure Databricks Workflows or Azure Data Factory.

  • Monitor and troubleshoot failures in job execution and cluster performance.

  • Collaborate with Stakeholders

  • Work with Data Analysts, Scientists, and Business Teams to understand requirements.

  • Translate business needs into scalable data engineering solutions.

  • API expertise

  • Ability to pull data from a wide variety of APIs using different strategies and methods

Required Skills & Experience:

  • Azure Databricks & Apache Spark (PySpark) – Strong experience in building distributed data pipelines.
  • Python – Proficiency in writing optimized and maintainable Python code for data engineering.
  • Unity Catalog – Hands-on experience implementing data governance, access controls, and lineage tracking.
  • SQL – Strong knowledge of SQL for data transformations and optimizations.
  • Delta Lake – Understanding of time travel, schema evolution, and performance tuning.
  • Workflow Orchestration – Experience with Azure Databricks Jobs or Azure Data Factory.
  • CI/CD & Infrastructure as Code (IaC) – Familiarity with Databricks CLI, Databricks DABs, and DevOps principles.
  • Security & Compliance – Knowledge of IAM, role-based access control (RBAC), and encryption.

Preferred Qualifications:

  • Experience with MLflow for model tracking & deployment in Databricks.
  • Familiarity with streaming technologies (Kafka, Delta Live Tables, Azure Event Hub, Azure Event Grid).
  • Hands-on experience with dbt (Data Build Tool) for modular ETL development.
  • Certification in Databricks, Azure is a plus.
  • Experience with Azure Databricks Lakehouse connectors for SalesForce and SQL Server
  • Experience with Azure Synapse Link for Dynamics, dataverse
  • Familiarity with other data pipeline strategies, like Azure Functions, Fabric, ADF, etc

Soft Skills:

  • Strong problem-solving and debugging skills.
  • Ability to work independently and in teams.
  • Excellent communication and documentation skills.

  • Lead Data Engineer

    2 weeks ago

    Greater Kolkata Area, India Eucloid Data Solutions Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job DescriptionEucloid is looking for a senior/ lead Data Engineer to join our Data Platform team supporting various business applications. The ideal candidate will support development of data infrastructure on Databricks for our clients by participating in activities which may include starting from up- stream and down-stream technology selection to...

  • Senior Data Engineer

    7 days ago

    Greater Chennai Area, India Tiger Analytics Full time

    Job Title: Senior Data Engineer - Power BITiger Analytics is a global AI and analytics consulting firm. With data and technology at the core of our solutions, our 4000+ tribe is solving problems that eventually impact the lives of millions globally. Our culture is modeled around expertise and respect with a team-first mindset. Headquartered in Silicon...

  • Senior Data Engineer

    6 days ago

    Chennai, Tamil Nadu, India AABM Cloud Data Solution Full time

    Senior Data Engineer (Remote, India)10+ Years Data Engineer Experience RequiredSQL, Python, Snowflake - 8+ years Mandatory hands-on experienceExperience with any ETL Tools (SSIS Preferred), Cloud Environments, and DB2/DBT is a plus.

  • Senior Data Engineer

    6 days ago

    Greater Delhi Area, India Dhurin Full time

    About the Company Dhurin is a fastgrowing data science firm serving financial services industry. We work with banks, NBFCs, fintech and other financial institutions. Our work is in the space of risk, fraud, collections, marketing, sales, and operations through it we directly influence how financial institutions make smarter decisions. Dhurin is ‘the...

  • Senior Data Engineer

    6 days ago

    Greater Delhi Area, India Dhurin Full time

    About the Company Dhurin is a fastgrowing data science firm serving financial services industry. We work with banks, NBFCs, fintech and other financial institutions. Our work is in the space of risk, fraud, collections, marketing, sales, and operations through it we directly influence how financial institutions make smarter decisions. Dhurin is ‘the...

  • Senior Data Engineer

    2 days ago

    Greater Delhi Area, India Dhurin Full time

    About the CompanyDhurin is a fastgrowing data science firm serving financial services industry. We work with banks, NBFCs, fintech and other financial institutions. Our work is in the space of risk, fraud, collections, marketing, sales, and operations through it we directly influence how financial institutions make smarter decisions. Dhurin is ‘the...

  • Senior Data Engineer

    1 week ago

    Greater Delhi Area, India Dhurin Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    About the CompanyDhurin is a fastgrowing data science firm serving financial services industry. We work with banks, NBFCs, fintech and other financial institutions. Our work is in the space of risk, fraud, collections, marketing, sales, and operations through it we directly influence how financial institutions make smarter decisions. Dhurin is 'the place'...

  • Senior Data Engineer

    2 weeks ago

    Greater Delhi Area, IN Dhurin Full time

    About the CompanyDhurin is a fastgrowing data science firm serving financial services industry. We work with banks, NBFCs, fintech and other financial institutions. Our work is in the space of risk, fraud, collections, marketing, sales, and operations through it we directly influence how financial institutions make smarter decisions. Dhurin is ‘the...

  • Senior Data Engineer

    3 weeks ago

    Greater Hyderabad Area, India Logic Pursuits Full time

    Job Title: Senior Data Engineer – Snowflake + dbt Location: Hyderabad, India Job Type: Full-time Job Description: We are seeking a highly skilled and experienced Senior Data Engineer to join our growing data team at Logic Pursuits. In this role, you will lead the design and implementation of scalable, high-performance data pipelines using Snowflake and...

  • Data Engineer

    1 week ago

    Greater Chennai Area, India iLink Digital Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    RequirementsWe are seeking an experienced and passionate Senior Data Engineer with hands-on expertise in Microsoft Fabric to join our growing data & AI team. The ideal candidate will be responsible for designing, developing, and maintaining scalable data solutions using Fabrics unified platform, integrating capabilities across Azure Data Factory, Synapse,...