Senior Data Engineer

3 weeks ago

Navi Mumbai, Maharashtra, India Celebal Technologies Full time

Dear Applicants,

Please find following Job Description for Senior Data Engineer role at Celebal Technologies, Please let us know your level of comfort along with most recent copy of your resume. You can directly apply or share your details at

Designation – Senior Data Engineer

Experience – 4 -9 Years

Location Navi Mumbai (onsite)

Job Summary:

We are seeking a highly skilled Data Engineer with deep expertise in Apache Kafka integration with Databricks, structured streaming, and large-scale data pipeline design using the Medallion Architecture. The ideal candidate will demonstrate strong hands-on experience in building and optimizing real-time and batch pipelines, and will be expected to solve real coding problems during the interview.

Job Description:


•Design, develop, and maintain real-time and batch data pipelines in Databricks.


•Integrate Apache Kafka with Databricks using Structured Streaming.


•Implement robust data ingestion frameworks using Databricks Autoloader.


•Build and maintain Medallion Architecture pipelines across Bronze, Silver, and Gold layers.


•Implement checkpointing, output modes, and appropriate processing modes in structured streaming jobs.


•Design and implement Change Data Capture (CDC) workflows and Slowly Changing Dimensions (SCD) Type 1 and Type 2 logic.


•Develop reusable components for merge/upsert operations and window function-based transformations.


•Handle large volumes of data efficiently through proper partitioning, caching, and cluster tuning techniques.


•Collaborate with cross-functional teams to ensure data availability, reliability, and consistency.

Must Have:


•Apache Kafka: Integration, topic management, schema registry (Avro/JSON).


•Databricks & Spark Structured Streaming:

Processing Modes: Append, Update, Complete

Output Modes: Memory, Console, File, Kafka, Delta

Checkpointing and fault tolerance


•Databricks Autoloader: Schema inference, schema evolution, incremental loads.


•Medallion Architecture implementation expertise.


•Performance Optimization:

Data partitioning strategies

Caching and persistence

Adaptive query execution and cluster configuration tuning


•SQL & Spark SQL: Proficiency in writing efficient queries and transformations.


•Data Governance: Schema enforcement, data quality checks, and monitoring.

Good to Have:


•Strong coding skills in Python and PySpark.


•Experience working in CI/CD environments for data pipelines.


•Exposure to cloud platforms (AWS/Azure/GCP).


•Understanding of Delta Lake, time travel, and data versioning.


•Familiarity with orchestration tools like Airflow or Azure Data Factory.

  • Senior Data Engineer

    3 weeks ago

    Navi Mumbai, Maharashtra, India Celebal Technologies Full time

    Dear Applicants,Please find following Job Description for Senior Data Engineer role at Celebal Technologies, Please let us know your level of comfort along with most recent copy of your resume. You can directly apply or share your details at shailendra.sharma@celebaltech.comDesignation – Senior Data EngineerExperience – 4 -9 YearsLocation Navi Mumbai...

  • Senior Application Engineer

    3 days ago

    Navi Mumbai, Maharashtra, India beBeeEngineering Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Title: Senior Engineering ArchitectWe are seeking a seasoned Senior Engineering Architect to join our team. As an Engineering Lead, you will provide technical expertise to plan, analyze, define, and support the delivery of future functional and technical capabilities for applications.Responsibilities:Provide solutions to business problems that apply...

  • Senior Data Engineer

    10 hours ago

    Navi Mumbai, Maharashtra, India Seven Consultancy Full time

    JOB DETAILS Highly proficient working with Scala/Java or PythonComfort working with large data sets and high-velocity data streamsHighly experience at data storage and processing (e.g. Kafka, Flink, Spark)Good relational and NoSQL database working knowledgeAn interest in healthcare and medical sectorsTechnical degree with 5 plus years backend data-heavy...

  • Senior Data Architect

    30 minutes ago

    Navi Mumbai, Maharashtra, India beBeeData Full time ₹ 1,80,00,000 - ₹ 2,00,00,000

    Job OverviewWe are seeking a highly skilled professional to fill the role of Senior Data Engineer.The ideal candidate will possess strong technical skills, including proficiency in working with Scala, Java, or Python, as well as experience with large data sets and high-velocity data streams.Candidates should also have in-depth knowledge of data storage and...

  • Senior Software Engineer

    3 weeks ago

    Navi Mumbai, Maharashtra, India HATCHTRA INNOTECH PRIVATE LIMITED Full time

    Senior Software Engineer - Data ScienceLocation : Airoli, Navi MumbaiExperience : 3-5 YearsEmployment Type : Full-timeJob Overview :We are seeking a Senior Software Engineer - Data Science with 3-5 years of experience to join our team. This role is ideal for a hands-on engineer with a strong background in software development and a passion for applying data...

  • Senior Data Engineer

    4 weeks ago

    Mumbai, Maharashtra, India Periscope-Tech Full time

    We're seeking a Senior Data Engineer to join our data engineering team. In this role, you'll be responsible for maintaining and extending existing data pipelines and partnering with our implementation team to design and deliver custom ETL workflows for new customers. This hands-on role requires strong coding skills, experience with modern data tools, and...

  • Senior Data Engineer

    1 week ago

    Mumbai, Maharashtra, India Straive Full time

    Dear Candidate,Straive is hiring for Senior Data Engineer, Interested candidate can apply to below link. Description:-Design- Analyze relevant internally and externally sourced data (raw data) to generate BI and Advanced Analytics- datasets based on your stakeholders' requirements- Design data pipelines to curate sourced data into the inhouse data warehouse-...

  • Senior Data Engineer and Scientist

    4 days ago

    Navi Mumbai, Maharashtra, India beBeeDataEngineer Full time ₹ 12,00,000 - ₹ 30,00,000

    Job TitleA talented Data Engineer and Scientist is needed to spearhead the development of software solutions, applications, platforms, products, and services in Data Engineering, Warehousing, and Analysis domain.Key ResponsibilitiesDesign and develop data engineering based data pipelines and implement algorithms in a distributed environment.Experience in R&D...

  • Senior Data Platform Specialist

    4 days ago

    Navi Mumbai, Maharashtra, India beBeeDataEngineer Full time ₹ 19,12,418 - ₹ 24,38,096

    Key Roles and Responsibilities:The ideal Integration Data Engineer is a dynamic contributor passionate about software development and dedicated to making a meaningful impact within a team of smart, creative individuals.We seek a candidate experienced in data integrations, data transformations, query database optimization, consuming web services, and working...

  • Senior Data Engineer

    4 weeks ago

    Mumbai, Maharashtra, India Straive Full time

    Dear Candidate, Straive is hiring for Senior Data Engineer, Interested candidate can apply to below link. Job Description:- Design Analyze relevant internally and externally sourced data (raw data) to generate BI and Advanced Analytics datasets based on your stakeholders' requirements Design data pipelines to curate sourced data into the inhouse data...