High-Performance Data Pipeline Specialist

23 hours ago

Amrāvati, Maharashtra, India beBeeDataEngineering Full time US$ 1,25,000 - US$ 1,75,000
Job Title: Senior Data Engineer

We are seeking a highly skilled Data Engineering Expert to design and implement high-performance data pipelines.

The ideal candidate will have experience with big data technologies, such as Spark and Hadoop, and be proficient in Python programming. Strong database performance optimization skills are also required.

In this role, you will work closely with data scientists and product teams to understand data requirements and collaborate on the development of efficient ETL pipelines.

  • Main Responsibilities:
  • Create scalable batch processing systems using Python and big data technologies.
  • Optimize database performance by identifying slow-running queries and implementing latency improvements.
  • Use Python profilers and performance monitoring tools to identify bottlenecks.
  • Build efficient ETL pipelines that can handle large-scale data processing.
  • Collaborate with cross-functional teams to develop data architecture and engineering processes.

Requirements:

  • Bachelor's degree in Computer Science, Engineering, or related technical field.
  • 4+ years of experience in data engineering roles.
  • Strong Python programming skills with focus on data processing libraries.
  • Experience with big data technologies (Spark, Hadoop, etc.).
  • Proven experience optimizing database performance (SQL or NoSQL).
  • Knowledge of data pipeline orchestration tools (Airflow, Luigi, etc.).
  • Understanding of performance optimization techniques and profiling tools.
  • Experience with cloud platforms (AWS, GCP, or Azure).

Preferred Qualifications:

  • Master's degree in Computer Science or related field.
  • Experience with SEO data or web crawling systems.
  • Experience with Clickhouse Database.
  • Knowledge of distributed systems and microservices architecture.
  • Familiarity with container orchestration (Kubernetes, Docker).
  • Experience with real-time data processing.

About This Role:

This is an excellent opportunity for a talented data engineer to join our team and contribute to the development of high-quality data solutions.

We offer a competitive remuneration package and opportunities for growth and development.

  • Data Pipeline Quality Assurance Specialist

    3 days ago

    Amrāvati, Maharashtra, India beBeePipeline Full time ₹ 1,50,00,000 - ₹ 2,01,00,000

    Job Title:Data Pipeline Quality Assurance SpecialistJob Description:We are seeking an experienced Data Pipeline Quality Assurance Specialist to join our team. As a key member of our data operations team, you will play a vital role in ensuring the integrity and quality of our data pipelines.This is an excellent opportunity for a motivated and detail-oriented...

  • Data Pipeline Specialist

    6 days ago

    Amrāvati, Maharashtra, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Big Data Expertise Job DescriptionWe seek a highly skilled Big Data Expert to create and maintain scalable data systems.Job Summary:• Design, develop, and implement large-scale data pipelines using Apache Spark and Python.• Collaborate with cross-functional teams to gather requirements and deliver high-quality solutions.• Ensure data accuracy, quality,...

  • Data Pipeline Specialist

    6 days ago

    Amrāvati, Maharashtra, India beBeeDataEngineer Full time ₹ 25,00,000 - ₹ 30,00,000

    Unlock Data ExcellenceKey Responsibilities:Design and build scalable data pipelines handling structured and unstructured data using cloud services.Own end-to-end ETL/ELT processes ensuring reliability, scalability, and high availability.Collaborate closely with AI teams to support model training, retrieval-augmented generation, and analytics...

  • High-Performance Data Architect

    2 weeks ago

    Amrāvati, Maharashtra, India beBeeDataEngineer Full time US$ 2,00,000 - US$ 2,50,000

    About Us:We are a leading provider of innovative telenutrition and foodcare solutions. Our platform is guided by a robust network of Registered Dietitians, empowering members to make informed decisions about their diet and lifestyle.Our mission is to make nutritious food accessible and affordable for everyone, regardless of economic status. We achieve this...

  • Data Engineer for High Performance Systems

    2 weeks ago

    Amrāvati, Maharashtra, India beBeeDataEngineer Full time US$ 1,20,000 - US$ 1,70,000

    **Big Data Expert Wanted**We're looking for a talented individual to join our team as a Big Data Engineer. The ideal candidate will have experience designing and implementing high-performance data pipelines that process massive amounts of data.The role involves working with big data technologies such as Spark, Hadoop, and Clickhouse Database to optimize...

  • Senior Data Pipeline Specialist

    2 weeks ago

    Amrāvati, Maharashtra, India beBeeData Full time ₹ 10,00,000 - ₹ 20,00,000

    Job Title: Data EngineerRole Overview:We are seeking a skilled professional to design, develop, and maintain scalable data pipelines that process large volumes of structured and unstructured data.The ideal candidate will have a strong background in data engineering, cloud technologies, and data architecture, collaborating closely with cross-functional teams...

  • AI Data Pipeline Specialist

    1 week ago

    Amrāvati, Maharashtra, India beBeeDataEngineer Full time ₹ 12,00,000 - ₹ 18,00,000

    We're seeking an experienced Data Engineer to join our team.The role involves designing, building, and maintaining production-level data pipelines that handle terabytes of structured and unstructured data.Own end-to-end ETL/ELT processes, ensuring reliability, scalability, and high availability.Work with AWS services (S3, EC2, RDS, DynamoDB, Athena,...

  • Data Pipeline Specialist

    7 days ago

    Amrāvati, Maharashtra, India beBeeMachineLearning Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job Title:As a senior machine learning engineer, you will be responsible for developing and optimizing complex data pipelines using machine learning engineering principles.Key Responsibilities:Designing and implementing data pipelines using AWS services such as Glue and SageMaker.Collaborating with data science teams to review model-ready datasets and...

  • Mastering Azure Data Pipeline Architectures

    1 week ago

    Amrāvati, Maharashtra, India beBeeData Full time ₹ 10,00,000 - ₹ 15,00,000

    Azure Data Factory SpecialistWe seek an experienced professional to support data integration and ETL needs. The ideal candidate will have expertise in ADF pipelines, data orchestration, and transformations for smooth, scalable, and secure data flows.Design, build, and maintain Azure Data Factory pipelines for ETL and ELT processes.Integrate data from...

  • Data Pipeline Specialist

    6 days ago

    Amrāvati, Maharashtra, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Data Engineering RoleWe are seeking a skilled professional to develop and optimize data pipelines, process large datasets, and work with relational databases. The ideal candidate will have experience in ETL tools, data processing technologies, data orchestration, and Python scripting.