High Performance Data Pipeline Specialist

1 day ago


India beBeeDataEngineer Full time ₹ 9,00,000 - ₹ 12,00,000
Big Data Engineer

We are looking for a talented professional to design, build, and maintain high-performance data pipelines that process terabytes of data. This role is critical in ensuring our data infrastructure can handle increasing volumes of data while maintaining exceptional performance standards.

About the Job
This role involves designing and implementing scalable batch processing systems using Python and big data technologies. The ideal candidate will have experience with big data technologies (Spark, Hadoop, etc.) and proven experience optimizing database performance (SQL or NoSQL).

Responsibilities
  • ">
  • Design and implement efficient ETL pipelines that can handle large-scale data processing ">
  • Collaborate with data scientists and product teams to understand data requirements ">
  • Optimize database performance, focusing on slow-running queries and latency improvements ">
  • Use Python profilers and performance monitoring tools to identify bottlenecks ">
  • Monitor and troubleshoot data pipeline issues in production ">

Requirements
  • ">
  • 4+ years of experience in data engineering roles ">
  • Strong Python programming skills with focus on data processing libraries ">
  • Experience with cloud platforms (AWS, GCP, or Azure) ">
  • Bachelor's degree in Computer Science, Engineering, or related technical field ">

Bonus Points
  • ">
  • Master's degree in Computer Science or related field ">
  • Experience with SEO data or web crawling systems ">
  • Familiarity with container orchestration (Kubernetes, Docker) ">
  • Experience with real-time data processing ">


  • India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Data Engineering RoleWe are seeking a talented data engineer to design and build high-performance data pipelines that process large datasets. As a key member of our team, you will optimize existing systems, identify performance bottlenecks, and implement solutions to improve efficiency.


  • India beBeeDeveloper Full time US$ 1,04,000 - US$ 1,30,878

    ETL Data Engineer - Scalable Pipeline DeveloperWe are seeking an experienced ETL data engineer to build and maintain scalable data pipelines using IBM DataStage and AWS Glue. As a key member of our data engineering team, you will work collaboratively with architects, business analysts, and data modelers to ensure timely and accurate delivery of critical data...


  • India beBeeDatabase Full time ₹ 29,40,000 - ₹ 31,20,000

    Job Title: Data EngineerDescription:We are seeking a skilled professional to fill the role of Data Engineer. This is a contract-to-hire opportunity that requires strong experience in ETL testing, database testing, and manual testing.Key Responsibilities:- Design and develop data pipelines using ETL concepts and methodologies- Implement monitoring scripts and...


  • India beBeeData Full time US$ 1,50,000 - US$ 1,75,000

    Senior Data Engineer PositionWe are seeking a highly skilled Senior Data Engineer with expertise in designing, building, and operating scalable data pipelines that support real-time clinical workflows.The ideal candidate will have 5+ years of experience building and maintaining production-grade data pipelines, strong background in workflow orchestration and...


  • India beBeeDataEngineer Full time US$ 1,04,000 - US$ 1,30,878

    Senior Data EngineerAbout the RoleWe are seeking an experienced Senior Data Engineer with in-depth expertise in building data pipelines and integrating healthcare data. You will report to the Head of AI Engineering and be responsible for designing, building, and operating the backbone of our AI-powered applications. This is a critical role where you will...


  • India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

    Data Engineering PositionWe are seeking an experienced Data Engineer to design and build scalable data pipelines, manage large datasets using Spark and SQL, and deploy real-time streaming applications.Key Responsibilities:Design and optimize data pipelines and ETL workflows.Manage and process large datasets using Spark, PySpark, and SQL.Build and maintain...


  • India beBeeInfrastructure Full time ₹ 1,00,00,000 - ₹ 1,50,00,000

    Site Reliability EngineerWe are seeking a skilled Site Reliability Engineer with expertise in infrastructure automation, cloud scalability, and performance optimization to join our high-performance team. In this role, you will be responsible for designing, managing, and monitoring large-scale AWS environments to ensure high availability, security, and...


  • India Weekday AI Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    This role is for one of Weekday's clientsMin Experience: 6 yearsLocation: IndiaJobType: full-timeWe are looking for an experienced and motivated Senior Pipeline Engineer to join our engineering team and take ownership of designing, building, and managing high-performance data pipelines. This role demands strong expertise in orchestration frameworks such as...


  • India beBeeSoftware Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job OpportunityWe strive to push the limits of innovation to solve complex challenges in high-performance computing.As a seasoned software engineering lead, you will be part of a core team driving cutting-edge technology advancements. Your expertise will play a vital role in enriching our industry and communities worldwide.This role involves collaborating...

  • Senior Data Engineer

    3 weeks ago


    India Aptus Data Labs Full time

    Job Title: Senior Data EngineerLocation: RemoteExperience: 4+Employment Type: Full-TimeNotice- Looking for immediate joiners or who can join within 15-20 days.About the RoleAptus Data Labs is looking for a talented and proactive Senior Data Engineer to help build the backbone of our enterprise data and AI initiatives. You'll work on modern data lake...