Senior Databricks Data Engineer

3 weeks ago

Madurai, Tamil Nadu, India CodeVyasa Full time

Looking for Senior Databricks Data Engineer | Pan India to join a team of rockstar developers. The candidate should have a minimum of 10+ yrs. of experience.

About CodeVyasa: CodeVyasa is a mid-sized product engineering company that works with top-tier product/solutions companies such as McKinsey, Walmart, RazorPay, Swiggy, and others. We are about 550+ people strong and we cater to Product & Data Engineering use-cases around Agentic AI, RPA, Full-stack and various other GenAI areas.

Job Summary:

We are seeking a highly skilled Senior Databricks Data Engineer with strong hands-on experience in Databricks and AWS to join our dynamic data engineering team. The ideal candidate will be well-versed in PySpark, CI/CD pipelines, and Terraform for infrastructure as code. Experience with DAB (Data Access Broker) is highly preferred.

Key Responsibilities:

  • Design, develop, and maintain scalable and high-performance data pipelines using Databricks and PySpark.
  • Implement and manage data solutions on AWS, including services like S3, Glue, Lambda, EMR, and Redshift.
  • Use CI/CD tools for automated deployment and testing of data workflows.
  • Develop infrastructure as code using Terraform to provision and manage cloud resources.
  • Collaborate with data scientists, analysts, and other engineering teams to support end-to-end data solutions.
  • Monitor, troubleshoot, and optimize performance of data pipelines and cloud infrastructure.
  • Work on data governance, security, and compliance practices, preferably leveraging DAB if applicable.

Must-Have Skills:

  • Strong hands-on experience with Databricks (Mandatory)
  • Solid experience with AWS cloud services (Mandatory)
  • Proficiency in PySpark and distributed data processing
  • Experience in CI/CD implementation for data workflows
  • Hands-on with Terraform for infrastructure management

Why Join CodeVyasa?

  • Work on innovative, high-impact projects with a team of top-tier professionals.
  • Continuous learning opportunities and professional growth.
  • Flexible work environment with a supportive company culture.
  • Competitive salary and comprehensive benefits package.
  • Free healthcare coverage.

  • Senior Data Integration Specialist

    2 weeks ago

    Madurai, Tamil Nadu, India beBeeDeveloper Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Senior ETL DeveloperJob OverviewWe are seeking a highly experienced and skilled Senior ETL Developer to join our team.Key Responsibilities:Collaborate with stakeholders to understand data requirements and translate them into technical solutions.Design, develop, and deploy scalable data pipelines and ETL processes using Databricks.Optimize Spark...

  • Senior Principal Data Architect

    2 weeks ago

    Madurai, Tamil Nadu, India beBeeData Full time ₹ 22,00,000 - ₹ 31,00,000

    Job Opportunity in Healthcare Revenue Cycle ManagementWe are seeking an experienced Senior/Principal Data Engineer to join our technology team. As a key member of the data engineering team, you will be responsible for developing and maintaining scalable data solutions using Databricks, SQL, Python, Spark, and Delta Lake on Azure.Key Responsibilities:Lead the...

  • Lead Data Infrastructure Specialist

    2 weeks ago

    Madurai, Tamil Nadu, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Senior Data Engineer Job OverviewWe are seeking an experienced Senior Data Engineer to design, develop and maintain data pipelines that extract data from various sources via APIs, process and store it in a cloud-based lakehouse platform.Key Responsibilities:Design and implement end-to-end pipelines from API extraction into the Databricks Lakehouse...

  • Senior Data Engineer

    2 weeks ago

    Madurai, Tamil Nadu, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Cloud Data Architect RoleOur organization is seeking a highly experienced Cloud Data Information Architect to lead the implementation of cloud data pipelines. The ideal candidate will have expertise in AWS Native data Architecture and services, including S3, Lambda, Glue, EMR, and Databricks Spark.The successful architect must be able to recommend data...

  • Chief Data Architect

    2 weeks ago

    Madurai, Tamil Nadu, India beBeeDatabricks Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Data Engineering Leadership RoleWe are seeking a highly experienced Senior Data Engineering Lead with a strong focus on Databricks and cloud-based data engineering to drive our data engineering strategy.Lead and manage a team of data engineers, providing mentorship, technical guidance, and fostering a culture of collaboration and innovation.Design, develop,...

  • Senior Cloud Data Architect

    2 weeks ago

    Madurai, Tamil Nadu, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Azure Data Engineer RoleWe are seeking a skilled Azure data engineer to join our team. In this role, you will work on designing and implementing data pipelines using Azure Data Factory, Databricks, and PySpark.Your key responsibilities will include:Designing and implementing data ingestion and processing workflows using ELT, data modeling, and data...

  • Data Engineer Architect

    2 weeks ago

    Madurai, Tamil Nadu, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    About Data ArchitectureInfoCepts offers a global solution to end-to-end data and analytics needs with nearly 20 years of expertise. We operate at the scale of a global consulting firm yet possess niche expertise.We have been recognized as a Gartner Peer Insights Customers' Choice for two consecutive years, which highlights our best-in-class services that...

  • Highly Experienced Azure Data Solutions Architect

    2 weeks ago

    Madurai, Tamil Nadu, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Our team seeks a highly skilled Lead Azure Data Engineer with deep expertise in Azure and Databricks.This senior technical role demands a strong problem solver who can optimize data pipelines, design scalable ELT/ETL workflows, and adopt new features to accelerate delivery.The ideal candidate has hands-on experience with:Azure services like ADF and Delta...

  • Azure Data Engineer-Technical Lead

    2 days ago

    Madurai, Tamil Nadu, India HCLTech Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Azure Data Lead with Azure Data Factory (ADF),Azure Databricks,SQL,Oracle PL/SQL,PythonJob SummaryThe Technical Lead will be responsible for overseeing and leading projects related to Azure Data Factory (ADF), Azure Databricks, SQL, Oracle PL/SQL, and Python. The role involves designing, developing, and implementing data solutions while ensuring they meet...

  • Cloud Data Engineer

    2 weeks ago

    Madurai, Tamil Nadu, India beBeeData Full time ₹ 15,00,000 - ₹ 20,00,000

    Cloud Data Engineering Specialist RoleWe are seeking a skilled professional to support our cloud-based data engineering platform. The ideal candidate will have expertise in maintaining and monitoring complex data pipelines, implementing continuous integration and deployment (CI/CD) pipelines, and conducting regular system health checks.Key...