Senior Cloud Data Pipeline Specialist

3 days ago


Salem, Tamil Nadu, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

About Our Data Engineering Role

We are looking for an experienced Freelance Data Engineer with strong expertise in Python and AWS services to design, develop, and optimize data pipelines. Proficiency in pytest for testing, Git/GitHub for version control, and clear technical documentation is required.

Key Responsibilities:

  • Design and develop data pipelines using AWS Glue, PySpark, and Step Functions.
  • Implement ETL/ELT processes for ingesting, transforming, and delivering data from multiple sources.
  • Manage and optimize storage solutions with Amazon S3.
  • Write reusable and scalable Python code for data engineering workflows.
  • Implement unit and integration tests using pytest to ensure data quality and pipeline reliability.
  • Collaborate using Git/GitHub for version control, code reviews, and CI/CD integration.
  • Create clear technical documentation for pipelines, workflows, and design decisions.
  • Troubleshoot and resolve issues related to data pipelines and AWS services.

Qualifications

  • 5–10 years of data engineering experience, with at least 5 years in AWS ecosystem.
  • Strong coding skills in Python with experience in PySpark.
  • Hands-on experience with AWS Glue, Step Functions, and S3.
  • Proficiency in pytest for test automation.
  • Solid understanding of Git/GitHub workflows.
  • Strong communication skills with a focus on documentation and knowledge sharing.


  • Salem, Tamil Nadu, India beBeePipeline Full time ₹ 80,00,000 - ₹ 1,40,00,000

    Senior Pipeline SpecialistWe are seeking a skilled Senior Pipeline Specialist to join our team and oversee the management of our end-to-end release process.This role involves designing, implementing, and managing DevOps pipelines to streamline software development and deployment.You will work closely with developers and admins to ensure smooth release...


  • Salem, Tamil Nadu, India beBeeIngestion Full time US$ 1,00,000 - US$ 1,50,000

    Job DescriptionWe are seeking a Data Ingestion Specialist with expertise in continuous integration and deployment (CI/CD) and container orchestration to design, develop, and optimize data ingestion pipelines for integrating multiple sources into Databricks.The ideal candidate will have hands-on experience with data pipelines, automation, and infrastructure...


  • Salem, Tamil Nadu, India beBeeDataEngineer Full time ₹ 90,00,000 - ₹ 1,50,00,000

    Data Engineering is a vital role that involves designing, building and maintaining production-level data pipelines to handle structured and unstructured data. We are seeking a skilled Data Engineer to join our team.The ideal candidate has 1–2 years of experience in data engineering and possesses strong knowledge of AWS cloud services and big data best...


  • Salem, Tamil Nadu, India beBeeCloudDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: AWS Cloud Data Engineer SpecialistWe are seeking an experienced AWS Cloud Data Engineer Specialist to join our team. In this role, you will be responsible for designing and developing end-to-end data pipelines on AWS cloud infrastructure using services such as Amazon S3, AWS Glue, AWS Lambda, and Amazon Redshift.You will implement data processing...


  • Salem, Tamil Nadu, India beBeeDataEngineer Full time ₹ 12,56,775 - ₹ 25,13,550

    Job TitleSenior Data EngineerWe are seeking an experienced data engineer with expertise in cloud-based platforms and big data tools to design, develop, and maintain scalable data pipelines. The ideal candidate will have a strong background in AWS and Databricks services, as well as experience in building and optimizing ETL/ELT workflows for large-scale...


  • Salem, Tamil Nadu, India beBeeData Full time ₹ 1,51,00,000 - ₹ 2,01,00,000

    Job OverviewWe are seeking an experienced Data Engineer to design, develop, and maintain scalable data pipelines on Databricks.The ideal candidate will have a strong background in data engineering, experience with cloud platforms, and expertise in Databricks (including Databricks SQL, Delta Lake, and MLflow).ResponsibilitiesDesign and implement data...


  • Salem, Tamil Nadu, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job OpportunityWe are seeking a highly skilled Senior Data Engineer to join our team and play a key role in scaling our analytics and reporting capabilities.This role is perfect for someone with deep technical expertise in ETL pipeline design, data modeling, and data infrastructure who thrives in a fast-paced, engineering-driven environment.You'll work...


  • Salem, Tamil Nadu, India beBeeEngineering Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Senior Data Engineering Specialist   We are seeking an experienced data engineering specialist to join our team.   Key Responsibilities:Design and develop end-to-end data integration solutions using Informatica IICS.Integrate IICS solutions with cloud-based data platforms such as Google Cloud Platform (GCP) and Google BigQuery (GBQ).Optimize...


  • Salem, Tamil Nadu, India beBeeTest Full time ₹ 15,00,000 - ₹ 21,00,000

    Job TitleA senior test engineer is needed to lead the testing strategy and execution for a project using Databricks, Data Factory pipelines, and Power BI.5+ years of experience in data pipeline/ETL testing with at least 2+ years as a Test Lead.Key responsibilities include:Driving the testing strategy and execution for the project.Leading a team of test...


  • Salem, Tamil Nadu, India beBeeData Full time ₹ 80,00,000 - ₹ 1,50,00,000

    Job Title: Technical SpecialistThis is a challenging and exciting role that offers the opportunity to work on designing, implementing, and managing data pipelines using Azure Data Factory. You will implement scalable data processing solutions within Azure Synapse Analytics, and have the chance to work with data modeling, ETL processes, and query...