Big Data Engineer

2 days ago

Pune, Maharashtra, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000
Big Data Engineer">

We are seeking a highly skilled Big Data Engineer to drive the design and implementation of scalable data pipelines using Apache Spark, Databricks, or equivalent big data frameworks. This is an excellent opportunity for experienced professionals to join our organization and leverage their expertise to drive business outcomes through innovative technology solutions.


Key Responsibilities:
  1. Design and build robust data pipelines using Apache Spark, Databricks, or equivalent big data frameworks.
  2. Develop expertise with streaming and messaging systems such as Apache Kafka (publish subscribe architecture), Confluent Cloud, RabbitMQ, or Azure Event Hub.
  3. Capture change events using Debezium or native CDC tools and transform them for downstream consumption.
  4. Implement cloud platform expertise using Azure or AWS services for data storage, compute, and orchestration.
  5. Data modelling and warehousing using knowledge of data Lakehouse architectures, Delta Lake, partitioning strategies, and performance optimisation.

The ideal candidate will possess hands-on expertise with the following technologies:

  • Apache Spark
  • Databricks
  • Apache Kafka
  • RabbitMQ
  • Azure Event Hub
  • Debezium
  • Native CDC tools
  • Azure or AWS services
  • Delta Lake
  • Git
  • CI/CD pipelines

Benefits:

This role offers a unique opportunity to work with cutting-edge technologies and contribute to the growth and success of our organization. The ideal candidate will be motivated by the prospect of working in a dynamic environment and will be driven by a passion for innovation and excellence.


Requirements:

To be successful in this role, you must have at least 5 years of experience in designing and building big data pipelines using Apache Spark, Databricks, or equivalent big data frameworks. Additionally, you must have hands-on expertise with the following technologies:

  • Apache Spark
  • Databricks
  • Apache Kafka
  • RabbitMQ
  • Azure Event Hub
  • Debezium
  • Native CDC tools
  • Azure or AWS services
  • Delta Lake
  • Git
  • CI/CD pipelines

  • Big Data Engineer

    13 hours ago

    Pune, Maharashtra, India Nice Software Solutions Pvt. Ltd. Full time

    Big Data Engineer (PySpark) Location: Pune/Nagpur (WFO) Experience: 8 - 12 Years Employment Type: Full-time Job Overview We are looking for an experienced Big Data Engineer with strong expertise in PySpark and Big Data ecosystems. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high...

  • Big Data Engineer

    2 weeks ago

    Pune, Maharashtra, India LION AND ELEPHANTS CONSULTANCY PRIVATE LIMITED Full time

    Job Description : We are looking for skilled Big Data Engineers using Java Spark with 5-10 years of experience in Big Data / legacy platforms, who can join immediately. Desired candidates should have design, development and optimization of real-time & batch data pipelines experience in a Big Data environment at an enterprise scale application. You will work...

  • Big Data Engineer

    3 weeks ago

    Pune, Maharashtra, India WorkiFicient Full time

    Roles and Responsibility :- Collaborate closely with Product Management and Engineering leadership to devise and build the right solution.- Participate in Design discussions and brainstorming sessions to select, integrate, and maintain Big Data tools and frameworks required to solve Big Data problems at scale.- Design and implement systems to cleanse,...

  • Senior Big Data Engineer

    3 weeks ago

    Pune, Maharashtra, India Devlats Pvt Ltd Full time

    About the Role : We are seeking a highly skilled and experienced Big Data Engineer to join our team. The ideal candidate will have strong expertise in designing, building, and optimizing data pipelines and architectures, as well as a solid understanding of big data tools and Responsibilities : - Design, develop, and maintain scalable and high-performance...

  • Big Data Engineer

    2 days ago

    Pune, Maharashtra, India beBeeData Full time ₹ 13,70,400 - ₹ 25,10,800

    Job Title: Data ArchitectWe are seeking an experienced Data Architect to lead the development of our big data strategy.About the RoleThis is a highly visible and rewarding role for a seasoned professional with expertise in designing, implementing, and maintaining large-scale data systems.Key ResponsibilitiesLeverage deep technical expertise to drive...

  • Senior Big Data Engineer

    4 weeks ago

    Pune, Maharashtra, India Citigroup Full time

    The Applications Development Senior Programmer Analyst is an intermediate level position responsible for participation in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team The overall objective of this role is to contribute to applications systems analysis and programming...

  • Highly Skilled Big Data Engineer Required

    2 days ago

    Pune, Maharashtra, India beBeeData Full time ₹ 12,00,000 - ₹ 18,00,000

    Job Title:Big Data EngineerWe are seeking a skilled Big Data Engineer to design, develop, and maintain large-scale data systems. This role requires expertise in big data technologies, data integration, and data architecture.Key Responsibilities:Design and develop scalable big data architectures and pipelines for efficient data processing and...

  • Big Data Technology Specialist

    10 hours ago

    Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 1,04,000 - ₹ 1,30,878

    Big Data Engineer PositionOur organization is looking for a highly skilled Big Data Engineer to join our team.Required Skills:Must have 3-8 years of strong experience in Java J2EE technologies with expertise in big data technology stacks such as Hadoop, Spark, Kafka, HBase, and Hive.Strong fundamentals including data structures, algorithms, object-oriented...

  • Big Data Architect

    9 hours ago

    Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 9,00,000 - ₹ 12,00,000

    Job Opportunity for Data EngineerWe are seeking an experienced data engineer to join our team in Pune. The ideal candidate will have a strong background in big data processing and a proven track record of designing and implementing scalable data architectures.Responsibilities:

  • Senior Big Data Developer

    4 hours ago

    Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    We are seeking a highly skilled Big Data Engineer, to join our team. As a key member of our engineering team, you will be responsible for designing and developing large-scale data processing systems.Your primary focus will be on working with Big Data technologies such as Hadoop, Java, and Spark, to ensure the efficient processing and storage of vast amounts...