Senior Databricks Engineer

2 weeks ago


Chennai, Tamil Nadu, India Finding Candidates Full time
Job Title: Senior Databricks Engineer

About the Role:
We are seeking a highly skilled Senior Databricks Engineer to join our team at Vidpro Consultancy Services. As a Senior Databricks Engineer, you will be responsible for designing and implementing modern data warehouse solutions using Databricks and AWS/Azure Stack. You will work closely with our DW/BI leads to develop new ETL pipeline requirements, troubleshoot existing pipelines, and address gaps. Your strong understanding of data management principles and proficiency in Databricks Data & AI platform and Databricks Delta Lake Architecture will be essential in this role.

Key Responsibilities:
• Creating modern data warehouse solutions using Databricks and AWS/Azure Stack
• Providing forward-thinking solutions in data engineering and analytics
• Collaborating with DW/BI leads for new ETL pipeline development requirements
• Troubleshooting issues in existing pipelines and addressing gaps
• Understanding reporting layer needs and developing data models
• Assisting team members with problem resolution and technical challenges
• Leading technical discussions with client architects and team members
• Orchestrating data pipelines in a scheduler via Airflow

Requirements:
• Bachelor's and/or master's degree in computer science or equivalent experience
• Minimum 3 years of experience in data warehouse/ETL projects
• Deep understanding of Star and Snowflake dimensional modeling
• Strong knowledge of data management principles
• Proficiency in Databricks Data & AI platform and Databricks Delta Lake Architecture
• Hands-on experience in SQL, Python, and Spark (PySpark)
• Experience with AWS/Azure stack and ETL with batch and streaming (Kinesis)
• Familiarity with Apache Kafka for use with streaming data/event-based data
• Working knowledge of Hadoop (incl. Hive, Pig, Impala) and non-relational/NoSQL data repositories (incl. MongoDB, Cassandra, Neo4J)
• Experience with structured and unstructured data including imaging & geospatial data
• Proficiency in RDBMS complex SQL, PL/SQL, Unix Shell Scripting, performance tuning, and troubleshooting
• Databricks Certified Data Engineer Associate/Professional Certification (Desirable)
• Comfortable working in a dynamic, fast-paced, innovative environment with several ongoing concurrent projects
• Experience working in Agile methodology
• Strong verbal and written communication skills
• Strong analytical and problem-solving skills with a high attention to detail

Skills and Qualifications:
• ETL, CircleCI, RDBMS, MongoDB, Hadoop, PySpark, PL/SQL, Databricks Data & AI platform, Azure, Azure Databricks, Git, Data Warehouse, Databricks Delta Lake Architecture, SQL, AWS, Pig, Impala, Apache Kafka, Unix Shell Scripting, Data Management, Spark, Neo4J, Extract, Transform, Load (ETL), Hive, Microsoft Azure, Azure Data Factory, Cassandra, Python, Pipelines, Terraform

  • Chennai, Tamil Nadu, India Finding Candidates Full time

    Job Title: Senior Databricks EngineerAbout the Role:We are seeking a highly skilled Senior Databricks Engineer to join our team. As a Senior Databricks Engineer, you will be responsible for designing and implementing modern data warehouse solutions using Databricks and AWS/Azure Stack. You will work closely with our DW/BI leads to develop new ETL pipeline...


  • Chennai, Tamil Nadu, India Breezy Talent Full time

    Job Title: Senior Databricks EngineerJob Summary:Breezy Talent is seeking a highly skilled Senior Databricks Engineer to join our team. As a Senior Databricks Engineer, you will be responsible for designing and implementing modern data warehouse solutions using Databricks and AWS/Azure Stack. You will work closely with our DW/BI leads to develop new ETL...


  • Chennai, Tamil Nadu, India Databricks Full time

    About the RoleAs a Senior Big Data Migration Engineer at Databricks, you will work closely with customers on short to medium-term engagements on their big data challenges using our platform. Your expertise will be invaluable in providing data engineering, data science, and cloud technology projects that require integrating with client systems, training, and...


  • Chennai, Tamil Nadu, India Databricks Full time

    The Incident Response team at Databricks is dedicated to protecting our customers, employees, and enterprise data from security threats. We're looking for a skilled Senior Security Engineer who will lead our incident response efforts and help us scale our security capabilities.Key Responsibilities:Lead incidents, investigations, and security initiatives from...


  • Chennai, Tamil Nadu, India Finding Candidates Full time

    Job Title: Senior Databricks EngineerJob Summary: Vidpro Consultancy Services is seeking a highly skilled Senior Databricks Engineer to join our team. The ideal candidate will have 510 years of experience in data engineering and analytics, with a strong background in Python, PySpark, and Spark with Azure and Azure Databricks.Key Responsibilities:• Creating...


  • Chennai, Tamil Nadu, India Databricks Full time

    The Incident Response team at Databricks is responsible for responding to security threats, incidents, and investigations to protect our customers, employees, and enterprise data in an efficient and standardized manner. We're a tight-knit team of security incident responders and incident handlers doing "Security for Databricks on Databricks", using our own...


  • Chennai, Tamil Nadu, India Databricks Full time

    Join Our Team as a Senior Security EngineerWe're seeking a highly skilled Senior Security Engineer to join our Incident Response team at Databricks. As a key member of our team, you will be responsible for leading incidents, investigations, and security initiatives from postmortems. Your expertise will help us scale security incident response at Databricks...


  • Chennai, Tamil Nadu, India Databricks Full time

    The Incident Response team at Databricks is looking for a Senior Security Specialist to join our team. As a key member of our security group, you will be responsible for leading incidents, investigations, and security initiatives. You will be a security multiplier, helping the team scale security incident response at Databricks.Responsibilities...


  • Chennai, Tamil Nadu, India Breezy Talent Full time

    Breezy Talent is seeking a Senior Databricks Engineer / Data Architect with 510 years of experience.Location: Hybrid Work Type: Mandatory Skills: Python/ PySpark / Spark with Azure/ Azure Databricks / AzureDataFactoryPrimary Roles and Responsibilities:• Creating Modern Data Warehouse solutions using Databricks and AWS/Azure Stack• Providing...


  • Chennai, Tamil Nadu, India Databricks Full time

    We are seeking an experienced Data Engineering Lead to join our Professional Services team at Databricks. As a key member of our team, you will work closely with customers to deliver short to medium-term customer engagements on their big data challenges using the Databricks platform.Your responsibilities will include providing data engineering, data science,...


  • Chennai, Tamil Nadu, India Databricks Full time

    Job Title: Senior Security EngineerAt Databricks, we're building a team of security experts to help us protect our customers, employees, and enterprise data. We're looking for a Senior Security Engineer to join our Incident Response team.Key Responsibilities:Lead incidents, investigations, and security initiatives from postmortems.Communicate technical...

  • Data Engineering Lead

    4 weeks ago


    Chennai, Tamil Nadu, India Databricks Full time

    Job Title: Solutions ArchitectAs a Solutions Architect at Databricks, you will play a key role in helping our customers overcome their big data challenges using our platform. You will work closely with customers on short to medium-term engagements, providing data engineering, data science, and cloud technology projects that require integrating with client...


  • Chennai, Tamil Nadu, India IT Source Global Full time

    Role : Senior Databricks Engineer Job Description :- Lead the design and development of big data pipelines and ETL/ELT workflows using Databricks and Azure Data Factory (ADF).- Develop scalable, high-performance data processing and transformation pipelines using Apache Spark within Databricks.- Create, schedule, and orchestrate data workflows and pipelines...


  • Chennai, Tamil Nadu, India Databricks Full time

    The Incident Response team at Databricks is responsible for responding to security threats, incidents, and investigations to protect our customers, employees, and enterprise data in an efficient and standardized manner. We're a tight-knit team of security incident responders and handlers using our own platform to create near-real-time log analytics,...


  • Chennai, Tamil Nadu, India Find Next Hire Full time

    Sought after by Vidpro Consultancy Services is a talented Data Architect who will lead the development of modern data warehouse solutions. Key responsibilities include:Designing and implementing data pipelines using Databricks and AWS/Azure StackProviding forward-thinking solutions in data engineering and analyticsCollaborating with DW/BI leads for new ETL...


  • Chennai, Tamil Nadu, India Databricks Full time

    Job SummaryAs a Migrations Architect at Databricks, you will work with customers on short to medium-term engagements on their big data challenges using the Databricks platform. You will provide data engineering, data science, and cloud technology projects that require integrating with client systems, training, and other technical tasks to help customers get...


  • Chennai, Tamil Nadu, India Databricks Full time

    Job Summary:As a Solutions Architect (Big Data) at Databricks, you will work with customers on short to medium-term customer engagements on their big data challenges using the Databricks platform. You will provide data engineering, data science, and cloud technology projects that require integrating with client systems, training, and other technical tasks to...


  • Chennai, Tamil Nadu, India Databricks Full time

    Job Title: Resident Data Architect, Migrations SpecialistAs a key member of our Professional Services team, you will work closely with customers on short to medium-term engagements to address their big data challenges using the Databricks platform. Your expertise will be instrumental in providing data engineering, data science, and cloud technology projects...


  • Chennai, Tamil Nadu, India Databricks Full time

    Job Title: Resident Solution Architect, Migrations SpecialistAs a key member of our Professional Services team, you will work closely with customers on short to medium-term engagements to address their big data challenges using the Databricks platform. Your expertise will be instrumental in providing data engineering, data science, and cloud technology...


  • Chennai, Tamil Nadu, India Databricks Full time

    Job Title: Migrations ArchitectAs a Migrations Architect at Databricks, you will play a key role in helping our customers overcome their big data challenges. You will work closely with our Professional Services team to deliver data engineering, data science, and cloud technology projects that require integrating with client systems, training, and other...