Distributed Data Processing Engineer

2 weeks ago

Pune, Maharashtra, India beBeeBigData Full time ₹ 25,00,000 - ₹ 35,00,000

Job Overview

This role focuses on developing large-scale data processing pipelines utilizing Apache Spark and related technologies.

Key Responsibilities

  • Design, develop, and maintain complex distributed data processing systems using Apache Spark and its ecosystem.
  • Implement efficient algorithms and data structures to process large datasets.
  • Collaborate with cross-functional teams to identify and resolve data-related issues.
  • Develop and implement ETL processes to transform and load data into warehouses.
  • Stay up-to-date with industry trends and emerging technologies in data processing.

Requirements

  • At least 8+ years of experience in Scala programming language.
  • Strong expertise in designing and developing large-scale data processing pipelines using Apache Spark.
  • Expertise in Spark Core, Spark SQL, and Spark Streaming.
  • Familiarity with Data warehousing and ETL concepts and techniques.
  • Experience with Hadoop, HDFS, Hive, and other BigData technologies.
  • UNIX shell scripting knowledge is an added advantage.
  • Excellent problem-solving skills and analytical thinking.
  • Strong communication and collaboration skills.
  • Ability to work in a team environment and adapt to changing project requirements.

About the Opportunity

A highly skilled data processing engineer is sought after to join our team. The ideal candidate will have extensive experience in designing and developing large-scale data processing pipelines using Apache Spark and related technologies.

  • Data Engineer

    2 weeks ago

    Pune, Maharashtra, India Data Axle Full time ₹ 15,00,000 - ₹ 28,00,000 per year

    About Data Axle:Data Axle Inc. has been an industry leader in data, marketing solutions, sales and research for 50 years in the US. Data Axle has set a strategic global centre of excellence in Pune. This centre delivers mission critical data services to its global customers powered by its proprietary cloud-based technology platform and leveraging proprietary...

  • Distribution Automation Engineer

    2 weeks ago

    Pune, Maharashtra, India beBeeEngineer Full time ₹ 8,00,000 - ₹ 15,00,000

    Job DescriptionWe are seeking an experienced professional to support design, implementation, integration, and lifecycle support for Distribution Automation projects.The ideal candidate will have hands-on knowledge of the GE PowerOn platform and experience with PowerOn configuration, SCADA/DMS/OMS workflows, GIS integration, and utility operational...

  • Distribution Engineering Lead

    2 weeks ago

    Pune, Maharashtra, India Leidos Full time

    Distribution Engineering Lead Based in Pune hybrid working At Leidos we challenge you with interesting assignments that provide you with an opportunity to thrive professionally and personally For us helping you grow your career is good business Leidos is seeking an experienced Distribution Engineering Lead in Pune India for electric distribution...

  • Distribution Engineer

    18 hours ago

    Pune, Maharashtra, India Leidos Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    DescriptionAt Leidos, we challenge you with interesting assignments that provide you with an opportunity to thrive, professionally and personally. For us, helping you grow your career is good business.Leidos is seeking Distribution Engineers in Pune, India, who are passionate about electric utility design engineering. We're looking for someone who is...

  • Distribution Engineer

    24 hours ago

    Pune, Maharashtra, India Leidos Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    At Leidos, we challenge you with interesting assignments that provide you with an opportunity to thrive, professionally and personally. For us, helping you grow your career is good business.Leidos is seeking Distribution Engineers in Pune, India, who are passionate about electric utility design engineering. We're looking for someone who is dedicated to...

  • Distribution Planning Engineer

    2 weeks ago

    Pune, Maharashtra, India Leidos Full time ₹ 15,00,000 - ₹ 28,00,000 per year

    DescriptionDistribution Planning and Analysis EngineerBased in our Pune officeAtLeidos, we challenge you with interesting assignments that provide you with an opportunity to thrive, professionally and personally.For us, helping you grow your career is good business.Leidos Engineering is seeking aDistribution Planning and Analysis Engineerto join our...

  • Distribution Planning Engineer

    2 weeks ago

    Pune, Maharashtra, India Leidos Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    Distribution Planning and Analysis EngineerBased in our Pune officeAt Leidos, we challenge you with interesting assignments that provide you with an opportunity to thrive, professionally and personally. For us, helping you grow your career is good business.Leidos Engineering is seeking a Distribution Planning and Analysis Engineer to join our distribution...

  • Senior Distributed Systems Specialist

    2 weeks ago

    Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Big Data Engineer PositionWe are seeking an experienced Big Data Engineer to lead the design, development, and maintenance of large-scale data processing systems. The ideal candidate will have hands-on expertise in building, optimizing, and managing distributed data pipelines using Apache Spark, Kafka, and/or Apache NiFi.

  • Cloud Data Engineer

    2 weeks ago

    Pune, Maharashtra, India beBeeSpark Full time US$ 50,000 - US$ 1,00,000

    Job Title: Azure Databricks DeveloperAzure Databricks is a fast, easy, and collaborative Apache Spark-based analytics platform optimized for cloud services. As an Apache Spark Engineer, you will work on designing, developing, and deploying large-scale data engineering pipelines using distributed systems.The ideal candidate will have experience working with...

  • Data Engineering

    1 day ago

    Pune, Maharashtra, India Verdantas Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Join Verdantas – A Top #ENR 81 Firm Driving Sustainable ProgressWe are seeking an Engineering Lead in Pune to architect and implement scalable, secure AI/ML data solutions aligned with business objectives. The role involves designing robust batch and real-time data architectures, leading ETL/ELT pipeline development, managing data lakes and warehouses,...