Cloud Data Pipeline Specialist

1 day ago

Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

We are seeking a seasoned expert to spearhead the design and implementation of large-scale data pipelines on Amazon Web Services (AWS) cloud infrastructure.

The ideal candidate will possess strong technical expertise in AWS services such as Amazon S3, AWS Glue, AWS Lambda, and Amazon Redshift, as well as hands-on experience with Apache Spark for data processing and analytics.

They will be responsible for collaborating closely with analysts, business stakeholders, and other engineers to deliver scalable data solutions that meet the evolving needs of our organization.

A key aspect of this role is the ability to optimize data pipelines for performance, reliability, and cost-effectiveness, leveraging best practices and cloud-native technologies.

Responsibilities
  • Design and develop end-to-end data pipelines on AWS cloud infrastructure.
  • Implement data processing and transformation workflows using Apache Spark and SQL.
  • Build and maintain orchestration workflows to automate data pipeline execution, scheduling, and monitoring.
  • Collaborate with analysts and business stakeholders to understand data requirements and deliver scalable data solutions.
  • Optimize data pipelines for performance, reliability, and cost-effectiveness.

Requirements:

  • Minimum 8+ years of experience building and deploying large-scale data processing pipelines in a production environment.
  • Hands-on experience in designing and building data pipelines on AWS cloud infrastructure.
  • Strong proficiency in AWS services such as Amazon S3, AWS Glue, AWS Lambda, and Amazon Redshift.
  • Strong experience with Apache Spark for data processing and analytics.
  • Hands-on experience on orchestrating and scheduling data pipelines using AppFlow, Event Bridge and Lambda.
  • Solid understanding of data modeling, database design principles, and SQL and Spark SQL.

This is an excellent opportunity for a motivated and experienced professional to join our team and contribute to the success of our organization.

We offer a dynamic work environment, opportunities for growth and development, and a competitive compensation package.

  • Cloud Data Pipeline Specialist

    3 days ago

    Bengaluru, Karnataka, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    AWS Data Engineer PositionJob Title: AWS Data EngineerKey Responsibilities:To design, build and maintain scalable data pipelines for the cloud-based platform AWS.Implement data ingestion and transformation processes to facilitate efficient data warehousing using AWS services.We are looking for an expert in data engineering with experience in cloud...

  • Data Pipeline Specialist

    1 day ago

    Bengaluru, Karnataka, India beBeeDataPipeline Full time US$ 1,00,000 - US$ 1,50,000

    Job Overview- We are seeking a highly skilled Data Pipeline Specialist to join our team. The ideal candidate will have experience building and productionizing data pipelines using industry-leading technologies.- A strong background in working with complex SQL datasets is required, as well as expertise in building efficient SQL scripts using BigQuery Syntax-...

  • Data Pipeline Engineer

    1 day ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 1,00,00,000 - ₹ 2,00,00,000

    The Role of Senior Data Integration SpecialistJob Overview:We are seeking an experienced Senior Data Integration Specialist to join our team. This is a key position that will play a vital role in the smooth operation of our enterprise data integration platform.As a Senior Data Integration Specialist, you will be responsible for designing, implementing, and...

  • Data Pipeline Architect

    8 hours ago

    Bengaluru, Karnataka, India beBeeAzure Full time ₹ 15,00,000 - ₹ 25,00,000

    Freelance Azure Data Factory SpecialistWe are seeking an experienced Freelance Azure Data Factory (ADF) Specialist to support our data integration and ETL needs.The ideal consultant will have strong hands-on expertise in Azure ADF pipelines, data orchestration, and transformations to ensure smooth, scalable, and secure data flows across multiple sources and...

  • Data Pipeline Specialist

    6 days ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time

    Job Title: Azure Databricks EngineerAs a skilled Data Engineer, you will play a pivotal role in our dynamic team.High Impact Required: Immediate joiners only. Please note that candidates with less than 12 months of experience will not be considered.Key Responsibilities:Our ideal candidate will have strong expertise in building and optimizing data pipelines...

  • Lead Data Pipeline Specialist

    2 days ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time US$ 1,04,000 - US$ 1,30,878

    Job Title: Senior Data EngineerAbout the Role:This is an exceptional opportunity to join our organization as a Senior Data Engineer. As a key member of our data team, you will be responsible for designing, building, and optimizing scalable data pipelines using cloud-native technologies.Key Responsibilities:Design and architect high-throughput ETL pipelines...

  • Senior Data Pipeline Developer

    19 hours ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 30,00,000

    Data Engineering SpecialistWe are seeking a highly skilled Data Engineer to join our organization. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining scalable data pipelines to support our growing business needs.You will collaborate with analytics and business teams to improve data models that...

  • Data Pipeline Specialist

    3 days ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    We're seeking a skilled Data Engineer to support our team.Key Responsibilities:Design, build, and maintain robust, scalable data pipelines for large volume data from various sources.Integrate data from multiple sources, ensuring consistency and reliability.Develop and maintain efficient data models and schemas.Manage and optimize databases for performance,...

  • Chief Data Pipeline Specialist

    3 days ago

    Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 25,00,000 - ₹ 50,00,000

    Job Overview:We are seeking a skilled Data Architect with strong expertise in designing and building scalable, efficient data flows and pipelines in a cloud environment.The ideal candidate will have hands-on experience with real-time data ingestion using Kafka, proficiency in Spark for data processing, and familiarity with AWS cloud services such as Glue and...

  • AWS Cloud Data Specialist

    4 days ago

    Bengaluru, Karnataka, India beBeeCloudData Full time US$ 90,000 - US$ 1,20,000

    Job Title: AWS Cloud Data SpecialistLocation: Bangalore, Gurgaon and PuneExperience: 5 to 10 yearsEmployment Type: Full-timeAbout the RoleWe are seeking a skilled professional to design and build scalable data pipelines and platforms. The ideal candidate will have expertise in AWS cloud services, Snowflake, Python, Airflow, Glue, Redshift, and Kafka and will...