High-Performance Data Engineer

1 week ago

Bengaluru Bangalore Ahmedabad Pune, India beBeeData Full time US$ 1,50,000 - US$ 2,00,000
Big Data Engineer - Design and Deliver High-Performance Analytics Solutions

As a Big Data Engineer, you will play a crucial role in designing, developing, and deploying high-performance analytics solutions using Apache Spark and Scala. Your expertise in big data processing frameworks, distributed computing, and data modeling will enable our organization to unlock new insights and drive business growth.



Key Responsibilities:
  • Design and Develop: Advanced analytics workloads using Apache Spark and Scala for big data processing.
  • Create and Optimize: Data transformation pipelines using Spark or Apache Flink to ensure seamless data flow.
  • Migrate Workloads: From cloud platforms to open-source Apache Spark infrastructure on Kubernetes to optimize resource utilization.
  • Implement Optimization: Techniques for large-scale data processing to enhance performance and scalability.

Required Skills and Qualifications:
  • Scala Programming: Proficiency in functional programming paradigm with a strong focus on code quality and maintainability.
  • Apache Spark: Extensive experience with core concepts and APIs, including Spark SQL and DataFrame APIs, Spark Structured Streaming, and Spark MLlib for analytics.
  • Distributed Computing: Strong understanding of big data processing frameworks and their applications in real-world scenarios.
  • Data Modeling: Expertise in optimization techniques for large-scale datasets and ability to design efficient data models.
  • Performance Tuning: Proficiency in optimizing Spark jobs for improved performance and scalability.
  • Lakehouse Storage: Good understanding of technologies like Delta Lake and Apache Iceberg for efficient data storage and retrieval.

  • High-Performance Data Engineer

    7 days ago

    Ahmedabad, Gujarat, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    We are seeking a Data Engineer to join our team. This is a critical role in ensuring our data infrastructure can handle increasing volumes of data while maintaining exceptional performance standards.Job Description:The successful candidate will design, build, and maintain high-performance data pipelines that process terabytes of data. They will work on...

  • High-Performance Data Engineer

    5 days ago

    Pune, Bengaluru / Bangalore, Hyderabad / Secunderabad, Telangana, India beBeeData Full time US$ 90,000 - US$ 1,20,000

    Job OverviewWe are seeking a seasoned Ab Initio Lead Developer to spearhead the creation of data integration and ETL solutions using the Ab Initio platform.This role entails leading the design, development, and implementation of scalable ETL workflows and data integration solutions utilizing Ab Initio tools such as GDE, Conduct and Express.The successful...

  • High-Performance Data Engineer

    5 days ago

    Ahmedabad, Gujarat, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Are you a data engineering expert looking for a new challenge? As a Senior Data Engineer, you will be responsible for designing and implementing scalable data pipelines to support our organization's advanced analytics and machine learning initiatives.The ideal candidate will have a strong background in cloud data warehouses, dimensional modeling, and...

  • High-Performance Data Engineer

    3 days ago

    Pune, Maharashtra, India beBeeData Full time ₹ 17,95,200 - ₹ 24,74,800

    As a seasoned data professional, you will design and implement high-performance data pipelines that process vast amounts of information. Your work will be critical in ensuring our data infrastructure can handle increasing volumes of data while maintaining exceptional performance standards.Key Responsibilities• Design and implement scalable batch processing...

  • High-Performance Data Engineer

    7 days ago

    Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Job OpeningWe are seeking a skilled PySpark Data Engineer to drive the development of robust data processing and transformation solutions within our data platform.Key Responsibilities:Data Engineering Development:Design, develop, and test PySpark-based applications to process, transform, and analyze large-scale datasets from various sources.Implement...

  • High-Performance Data Architect

    3 days ago

    Ahmedabad, Gujarat, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Senior Data Engineer PositionWe are seeking a highly skilled Senior Data Engineer to join our global engineering team. As a key member of our data engineering team, you will design and build high-performance data pipelines using AWS and Azure services.

  • High-Performance Data Engineer

    5 days ago

    Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 24,00,000

    Job Title: Data EngineerWe are seeking a skilled data engineer to take on a challenging role in designing, building, and maintaining large-scale data systems.Collaborate with team members and stakeholders to understand data requirementsPull data from various sources into a central data platformStructure datasets and calculate metrics based on...

  • High Performance Data Engineer

    5 days ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    About Data Engineering RolesData Engineers design, build, and maintain large-scale data systems. They are responsible for ensuring that data is accurate, complete, and delivered on time to meet business needs.Key Responsibilities:Developing and implementing data pipelines using Big Data technologies like Spark, PySpark, and HiveImplementing performance...

  • High-Performance Data Engineer

    7 days ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Senior Data Engineer RoleWe are seeking a highly skilled and experienced data engineer to play a critical role in building, optimizing, and maintaining data pipelines and architectures.Key Responsibilities:Design, develop, and optimize scalable data pipelines and workflows using Azure Data Factory, Synapse Pipelines, and Microsoft Fabric.Transform and...

  • High-Performance Data Engineer

    1 day ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 22,50,000

    Job DescriptionWe are seeking a highly skilled data engineer to design, build, and maintain high-performance, scalable data pipelines that support our consulting and analytics solutions.The ideal candidate will work collaboratively with cross-functional teams to develop and implement data models, ETL processes, and data warehousing solutions. Key areas of...