
Senior Cloud Data Pipeline Specialist
3 days ago
About Our Data Engineering Role
We are looking for an experienced Freelance Data Engineer with strong expertise in Python and AWS services to design, develop, and optimize data pipelines. Proficiency in pytest for testing, Git/GitHub for version control, and clear technical documentation is required.
Key Responsibilities:
- Design and develop data pipelines using AWS Glue, PySpark, and Step Functions.
- Implement ETL/ELT processes for ingesting, transforming, and delivering data from multiple sources.
- Manage and optimize storage solutions with Amazon S3.
- Write reusable and scalable Python code for data engineering workflows.
- Implement unit and integration tests using pytest to ensure data quality and pipeline reliability.
- Collaborate using Git/GitHub for version control, code reviews, and CI/CD integration.
- Create clear technical documentation for pipelines, workflows, and design decisions.
- Troubleshoot and resolve issues related to data pipelines and AWS services.
Qualifications
- 5–10 years of data engineering experience, with at least 5 years in AWS ecosystem.
- Strong coding skills in Python with experience in PySpark.
- Hands-on experience with AWS Glue, Step Functions, and S3.
- Proficiency in pytest for test automation.
- Solid understanding of Git/GitHub workflows.
- Strong communication skills with a focus on documentation and knowledge sharing.
-
Senior Pipeline Specialist
4 days ago
Salem, Tamil Nadu, India beBeePipeline Full time ₹ 80,00,000 - ₹ 1,40,00,000Senior Pipeline SpecialistWe are seeking a skilled Senior Pipeline Specialist to join our team and oversee the management of our end-to-end release process.This role involves designing, implementing, and managing DevOps pipelines to streamline software development and deployment.You will work closely with developers and admins to ensure smooth release...
-
Senior Data Pipeline Engineer
6 days ago
Salem, Tamil Nadu, India beBeeIngestion Full time US$ 1,00,000 - US$ 1,50,000Job DescriptionWe are seeking a Data Ingestion Specialist with expertise in continuous integration and deployment (CI/CD) and container orchestration to design, develop, and optimize data ingestion pipelines for integrating multiple sources into Databricks.The ideal candidate will have hands-on experience with data pipelines, automation, and infrastructure...
-
Senior Data Pipeline Specialist
1 day ago
Salem, Tamil Nadu, India beBeeDataEngineer Full time ₹ 90,00,000 - ₹ 1,50,00,000Data Engineering is a vital role that involves designing, building and maintaining production-level data pipelines to handle structured and unstructured data. We are seeking a skilled Data Engineer to join our team.The ideal candidate has 1–2 years of experience in data engineering and possesses strong knowledge of AWS cloud services and big data best...
-
AWS Cloud Data Engineer Specialist
4 days ago
Salem, Tamil Nadu, India beBeeCloudDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000Job Title: AWS Cloud Data Engineer SpecialistWe are seeking an experienced AWS Cloud Data Engineer Specialist to join our team. In this role, you will be responsible for designing and developing end-to-end data pipelines on AWS cloud infrastructure using services such as Amazon S3, AWS Glue, AWS Lambda, and Amazon Redshift.You will implement data processing...
-
Chief Data Pipeline Architect
3 days ago
Salem, Tamil Nadu, India beBeeDataEngineer Full time ₹ 12,56,775 - ₹ 25,13,550Job TitleSenior Data EngineerWe are seeking an experienced data engineer with expertise in cloud-based platforms and big data tools to design, develop, and maintain scalable data pipelines. The ideal candidate will have a strong background in AWS and Databricks services, as well as experience in building and optimizing ETL/ELT workflows for large-scale...
-
Senior Data Pipelines Architect
6 days ago
Salem, Tamil Nadu, India beBeeData Full time ₹ 1,51,00,000 - ₹ 2,01,00,000Job OverviewWe are seeking an experienced Data Engineer to design, develop, and maintain scalable data pipelines on Databricks.The ideal candidate will have a strong background in data engineering, experience with cloud platforms, and expertise in Databricks (including Databricks SQL, Delta Lake, and MLflow).ResponsibilitiesDesign and implement data...
-
Senior Data Engineer
4 days ago
Salem, Tamil Nadu, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000Job OpportunityWe are seeking a highly skilled Senior Data Engineer to join our team and play a key role in scaling our analytics and reporting capabilities.This role is perfect for someone with deep technical expertise in ETL pipeline design, data modeling, and data infrastructure who thrives in a fast-paced, engineering-driven environment.You'll work...
-
Senior Data Engineering Specialist
3 days ago
Salem, Tamil Nadu, India beBeeEngineering Full time ₹ 20,00,000 - ₹ 25,00,000Job Title: Senior Data Engineering Specialist We are seeking an experienced data engineering specialist to join our team. Key Responsibilities:Design and develop end-to-end data integration solutions using Informatica IICS.Integrate IICS solutions with cloud-based data platforms such as Google Cloud Platform (GCP) and Google BigQuery (GBQ).Optimize...
-
Data Pipeline Test Engineer
6 days ago
Salem, Tamil Nadu, India beBeeTest Full time ₹ 15,00,000 - ₹ 21,00,000Job TitleA senior test engineer is needed to lead the testing strategy and execution for a project using Databricks, Data Factory pipelines, and Power BI.5+ years of experience in data pipeline/ETL testing with at least 2+ years as a Test Lead.Key responsibilities include:Driving the testing strategy and execution for the project.Leading a team of test...
-
Data Pipeline Engineer
3 days ago
Salem, Tamil Nadu, India beBeeData Full time ₹ 80,00,000 - ₹ 1,50,00,000Job Title: Technical SpecialistThis is a challenging and exciting role that offers the opportunity to work on designing, implementing, and managing data pipelines using Azure Data Factory. You will implement scalable data processing solutions within Azure Synapse Analytics, and have the chance to work with data modeling, ETL processes, and query...