Databricks Data Engineer

3 days ago


Chennai Tamil Nadu India, Tamil Nadu Tata Consultancy Services Full time

About the Role


The Databricks Data Engineer will play a crucial role in migrating and optimizing data workflows, ensuring high performance and reliability in our data operations.


Experience- 3 to 6 years


Location - Kolkata /Chennai/Trivandrum


Required Technical Skill Set ------ Databricks, AWS, Python & PySpark


Good to have- Experience in AWS platform (S3, EC2, EMR, MWAA, and Redshift).


Responsibilities


  • Collaborate with stakeholders to gather requirements and design solutions using the Databricks platform.
  • Migrate Python, Pyspark, MWAA DAG to Databricks Notebook and Workflow.
  • Work with BJs Computer operations team and repoint Talend/MWAA Autosys jobs to Databricks workflow.
  • Implement medallion architecture in Databricks Enterprise Data Platform with industry best practices and recommend the best framework for Redshift to Delta tables.
  • Provide technical expertise and guidance to the team members and stakeholders.
  • Contribute to the development of documentation, training materials, and user guides.
  • Participate in testing and quality assurance activities to ensure reliability and performance.
  • Optimize long-running Databricks workflows.
  • Development experience in Python, Pyspark.
  • Hands-on experience in writing complex SQLs and performance optimization of very complex SQL.


  • Chennai, Tamil Nadu, India, Tamil Nadu Tata Consultancy Services Full time

    Role Summary:We are looking for a Data Engineer with expertise in Azure Databricks, data transformation, and integration pipelines to support forecasting, scheduling, and analytics flows in the WFM Re-Platform program.Key ResponsibilitiesDesign and implement data pipelines in Azure Databricks for ingesting and transforming data from upstream systems (SAP,...


  • Chennai, Tamil Nadu, India, Tamil Nadu Tata Consultancy Services Full time

    Role**Databricks Engineer (Individual Contributor)Required Technical Skill Set**Azure Databricks, PysparkExperience: 4 to 10 YearsLocation: Hyd,Kolkata, Noida, Chennai, BangaloreDesired Competencies (Technical/Behavioral Competency)Must-Have**1. Azure Data Bricks (Python)2. Pyspark3. ADLS4. complex problem-solving skillGood-to-HaveAzure DevopsAzure Data...

  • Databricks Admin

    3 days ago


    Chennai, Tamil Nadu, India, Tamil Nadu Tata Consultancy Services Full time

    Databricks AdminExp- 5 to 8 yearsLocation- Chennai/ KolkataRequired SkillsDatabricks AdministrationTerraformAWS cloud servicesMLflowWorkflowsDatabricks Asset BundlesAWS IAMVPCPrivate endpointsFirewallsS3SQL + PySparkdbx CLIResponsibilitiesImplement, and maintain the Databricks platform, including workspace setup, user and group management, access control,...


  • Chennai, Tamil Nadu, India Zillion Technologies Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Senior Data Engineer – DatabricksLocation: Hyderabad/ChennaiWork Schedule: Timings 2:30 PM to 11:30 PMRole Overview:We are seeking a highly skilled Senior Data Engineer with deep expertise in Databricks to design, build, and optimize scalable data pipelines and architectures. You will collaborate with cross-functional teams including data scientists,...

  • Data Engineer

    2 days ago


    Chennai, Tamil Nadu, India CX Data Labs Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    HiRole - Azure Data EngineerLocation - Chennai, Gurugram (5days Office)Full timeShift timing 2pm to 11pmKey Responsibilities:· Develop, maintain, and optimize scalable ETL/ELT pipelines using PySpark and Databricks.· Collaborate with cross-functional teams to design and implement data models and data integration solutions.· Create and maintain robust SQL...

  • Databricks Admin

    2 days ago


    Chennai, Tamil Nadu, India, Tamil Nadu Tata Consultancy Services Full time

    TCS present an excellent opportunity for Databricks AdminJob Location: Chennai , KolkataExperience required : 6-12 yrsInterview date: 17th Oct (Friday)Required Technical Skill Set**Databricks Administration, Terraform, AWS cloud services, MLflow, Workflows, Databricks Asset Bundles, AWS IAM, VPC, private endpoints, firewalls, S3, SQL + PySpark, dbx...

  • Databricks Admin

    2 days ago


    Chennai, Tamil Nadu, India, Tamil Nadu Tata Consultancy Services Full time

    TCS is Hiring For Databricks AdminJob Location : Chennai, KolkataExperience : 6 - 12 yearsRequired Skillset : Databricks Administration, Terraform, AWS cloud services, MLflow, Workflows, Databricks Asset Bundles, AWS IAM, VPC, private endpoints, firewalls, S3, SQL + PySpark, DBX CLIMust have skills :Databricks Platform Administration including environment...

  • Data Engineer

    2 weeks ago


    Chennai, Tamil Nadu, India, Tamil Nadu KaarTech Full time

    Key Responsibilities· Data Pipeline Development: Design, develop, and maintain scalable data pipelines to support various data analytics and reporting needs.· Data Integration: Integrate data from multiple sources and ensure data quality and consistency.· Data Modelling: Build and optimize data models using Spark to support complex data transformations...

  • Data Engineering

    2 weeks ago


    Chennai, Tamil Nadu, India, Tamil Nadu EXL Full time

    Responsibilities:Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes.Create the data integration and data diagram documentation. Lead the data validation, UAT and regression test for new data asset creation.Create and maintain data models, including schema design and optimization.Create and manage...


  • Chennai, Tamil Nadu, India InfoCepts Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Position: Data Architect (Databricks, Snowflake)Location: Nagpur, Pune, Chennai, BangaloreType of Employment: Full time EmploymentPurpose of the Position:Design and optimize cloud-native data architectures on platforms like Databricks and Snowflake, enabling scalable data engineering, advanced analytics, and AI/ML solutions aligned with business needs.Key...