Data Engineer – Databricks

2 days ago

tamil nadu, India Tata Consultancy Services Full time

Role Summary:

We are looking for a Data Engineer with expertise in Azure Databricks, data transformation, and integration pipelines to support forecasting, scheduling, and analytics flows in the WFM Re-Platform program.


Key Responsibilities

  • Design and implement data pipelines in Azure Databricks for ingesting and transforming data from upstream systems (SAP, PIPS, POS) into WFM and UKG.
  • Optimize ETL/ELT workflows for performance and scalability.
  • Collaborate with Java/API developers to integrate event-driven triggers into data pipelines.
  • Implement data quality checks, schema validation, and error handling.
  • Support batch and near-real-time data flows for operational and analytics use cases.
  • Work with Boomi and WFM teams to ensure data contracts and canonical models are enforced.


Required Skills

  • Strong experience with Azure Databricks (PySpark, Delta Lake).
  • Proficiency in SQL and data modeling.
  • Experience with Azure Data Lake, and Event Hub.
  • Familiarity with data governance and security best practices.
  • Ability to work with large datasets and optimize for performance.
  • Familiarity with CI/CD (Jenkins and Terraform preferred).
  • Knowledge of JSON, XML.

Nice-to-Have

  • Experience with UKG Data Hub or similar HR/payroll data platforms.
  • Knowledge of API integration and microservices.
  • Familiarity with Power BI or other reporting tools.

  • Databricks

    6 days ago

    Chennai, Tamil Nadu, India Cognizant Full time

    **Job Summary** **Responsibilities** - Develop and optimize data solutions using Databricks SQL to ensure efficient data processing and analysis. - Implement and manage Databricks Workflows to automate data pipelines and improve operational efficiency. - Collaborate with cross-functional teams to understand business requirements and translate them into...

  • Azure Databricks Engineer

    10 hours ago

    tamil nadu, India Tata Consultancy Services Full time

    Role**Databricks Engineer (Individual Contributor)Required Technical Skill Set**Azure Databricks, PysparkExperience: 4 to 10 YearsLocation: Hyd,Kolkata, Noida, Chennai, BangaloreDesired Competencies (Technical/Behavioral Competency)Must-Have**1. Azure Data Bricks (Python)2. Pyspark3. ADLS4. complex problem-solving skillGood-to-HaveAzure DevopsAzure Data...

  • Databrick

    7 days ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Key Responsibilities: Design, develop, and maintain scalable data pipelines using Apache Spark on Databricks. Write efficient and production-ready PySpark or Scala code for data transformation and ETL processes. Integrate data from various structured and unstructured sources into a unified platform. Implement Delta Lake and manage data versioning, updates,...

  • Cloud Data Architect

    2 weeks ago

    Chennai, Tamil Nadu, India Cognizant Full time

    **Skill: Cloud Data Architect - Azure Databricks** **Role: Solution Architect** Solutioning lead for NA Data Engineering - Azure and Databicks as primary stack **Roles and responsibilities**: 1) Architecture and Solutioning on Azure, databricks data platforms, expertise on architecture patterns - data warehosue, lakehouse, data fabric and datamesh 2)...

  • Data Engineer – Databricks

    2 hours ago

    Chennai, Tamil Nadu, India, Tamil Nadu Tata Consultancy Services Full time

    Role Summary:We are looking for a Data Engineer with expertise in Azure Databricks, data transformation, and integration pipelines to support forecasting, scheduling, and analytics flows in the WFM Re-Platform program.Key ResponsibilitiesDesign and implement data pipelines in Azure Databricks for ingesting and transforming data from upstream systems (SAP,...

  • Azure Databricks Engineer

    2 hours ago

    Chennai, Tamil Nadu, India, Tamil Nadu Tata Consultancy Services Full time

    Role**Databricks Engineer (Individual Contributor)Required Technical Skill Set**Azure Databricks, PysparkExperience: 4 to 10 YearsLocation: Hyd,Kolkata, Noida, Chennai, BangaloreDesired Competencies (Technical/Behavioral Competency)Must-Have**1. Azure Data Bricks (Python)2. Pyspark3. ADLS4. complex problem-solving skillGood-to-HaveAzure DevopsAzure Data...

  • Databricks + Pyspark

    2 weeks ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Data Pipeline Development: Design, implement, and maintain scalable and efficient data pipelines using PySpark and Databricks for ETL processing of large volumes of data. Cloud Integration: Develop solutions leveraging Databricks on cloud platforms (AWS/Azure/GCP) to process and analyze data in a distributed computing environment. Data Modeling: Build robust...

  • Data Engineer

    2 weeks ago

    tamil nadu, India HCLTech Full time

    Azure Data Lead with Azure Data Factory (ADF),Azure Databricks,Job SummaryThe Technical Lead will be responsible for overseeing and leading projects related to Azure Data Factory (ADF), Azure Databricks, SQL, Oracle PL/SQL, and Python. The role involves designing, developing, and implementing data solutions while ensuring they meet the business requirements...

  • Data Engineer

    10 hours ago

    tamil nadu, India KaarTech Full time

    Key Responsibilities· Data Pipeline Development: Design, develop, and maintain scalable data pipelines to support various data analytics and reporting needs.· Data Integration: Integrate data from multiple sources and ensure data quality and consistency.· Data Modelling: Build and optimize data models using Spark to support complex data transformations...

  • Pyspark/databricks

    2 weeks ago

    Chennai, Tamil Nadu, India Virtusa Full time

    Bachelors degree or equivalent in Computer Engineering, Computer Science, or a related field. 5+ years of experience in data / software engineering role. 3+ years of experience with building AWS or Azure cloud-based data pipelines and AI solutions. 3+ years of experience with Python and Spark. Strong experience with Databricks, including Spark-based...