
Expert Python Data Pipeline Specialist
3 days ago
As a Freelance Data Engineer with Python expertise, you will design and develop data pipelines using AWS Glue, PySpark, and Step Functions. Your primary responsibility will be to create efficient and scalable data engineering workflows that meet the needs of our organization.
In this role, you will work closely with cross-functional teams to implement ETL/ELT processes for ingesting, transforming, and delivering data from multiple sources. You will also be responsible for managing and optimizing storage solutions with Amazon S3, writing reusable and scalable Python code, and implementing unit and integration tests using pytest.
Additionally, you will collaborate with team members on Git/GitHub for version control, code reviews, and CI/CD integration. Clear technical documentation is essential in this role, as you will need to create comprehensive documents for pipelines, workflows, and design decisions.
Troubleshooting and resolving issues related to data pipelines and AWS services are also crucial responsibilities in this position.
Required Skills and Qualifications- 5–10 years of data engineering experience, with at least 5 years in the AWS ecosystem.
- Strong coding skills in Python, with experience in PySpark.
- Hands-on experience with AWS Glue, Step Functions, and S3.
- Proficiency in pytest for test automation.
- Solid understanding of Git/GitHub workflows.
- Strong communication skills with a focus on documentation and knowledge sharing.
-
Data Pipelines Specialist
4 days ago
Gandhinagar, Gujarat, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000Senior Data EngineerAs a seasoned data professional, you will design and implement high-quality, secure, and scalable data pipelines and systems. The ideal candidate has 4–5 years of experience in designing, building, and maintaining complex data flows.Key Responsibilities:Design and build scalable data pipelines using Snowflake, Fivetran, DBT, and...
-
Data Pipeline Assurance Specialist
4 days ago
Gandhinagar, Gujarat, India beBeeSeniorTestEngineer Full time ₹ 15,00,000 - ₹ 25,00,000Job Title: Senior Test EngineerAbout the Role:We are seeking an experienced Senior Test Engineer to join our team. As a key member of our engineering department, you will play a critical role in ensuring the accuracy, integrity, and performance of our ETL pipelines deployed into Azure cloud environments.Responsibilities:Evaluate and Validate Data Flows:...
-
Senior Data Pipeline Specialist
6 days ago
Gandhinagar, Gujarat, India beBeeTalent Full time ₹ 18,00,000 - ₹ 20,00,000Talend ETL Developer Job OpeningWe are looking for a highly skilled and experienced Talend ETL developer to join our team on a contractual basis. The ideal candidate will have expertise in Talend Open Studio, BigQuery, PostgreSQL, Python, and GCP.The role requires strong analytical skills, attention to detail, and the ability to design, optimize, and deploy...
-
Data Expert
2 days ago
Gandhinagar, Gujarat, India beBeeDataexpert Full time ₹ 10,00,000 - ₹ 17,50,000Our team is seeking a Data Expert. The ideal candidate will be responsible for designing end-to-end data pipelines and developing advanced propensity models to drive business growth. We are looking for someone with strong analytical skills, including statistical analysis, data integrity, and operational performance tracking.Key Responsibilities:Design and...
-
Data Pipeline Architect
4 days ago
Gandhinagar, Gujarat, India beBeeAdministration Full time ₹ 1,50,00,000 - ₹ 2,00,00,000Job OverviewAs a Kafka expert, you will be responsible for designing and implementing scalable data pipelines to optimize message processing efficiency.You will work on automating the installation and configuration of production-grade Kafka clusters using tools such as Ansible and Chef, and troubleshoot critical issues to ensure high availability and fault...
-
Senior Cloud Pipeline Specialist
6 days ago
Gandhinagar, Gujarat, India beBeeAzureEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000Job Title:Azure Databricks Engineer Description: We are seeking a seasoned professional to fill the role of Azure Databrick Engineer. The ideal candidate will have at least 7 years of experience in creating, modifying and optimizing Azure Data Factory pipelines. This is a contract position lasting approximately 3-4 months. Key Responsibilities:...
-
Data Ingestion Specialist
1 week ago
Gandhinagar, Gujarat, India beBeeIngestion Full time ₹ 20,00,000 - ₹ 25,00,000We are seeking a highly skilled Data Ingestion Specialist to join our team.This role requires a strong understanding of data ingestion processes, focusing on integrating various data sources into Databricks.The ideal candidate will have hands-on experience with data pipelines, automation, and infrastructure management to support scalable and efficient...
-
Data Engineering Expert
3 days ago
Gandhinagar, Gujarat, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000We are seeking an experienced Data Engineer to fill a key role in our organization. As a core member of the team, you will be responsible for designing, building, and optimizing large-scale data pipelines using Python, SQL, PySpark, AWS, and Databricks.Job Responsibilities:Data Pipeline Development: Design, develop, and maintain ETL/ELT pipelines using...
-
Data Scientist
4 days ago
Gandhinagar, Gujarat, India beBeeData Full time ₹ 20,00,000 - ₹ 25,40,000">Business Intelligence Data SpecialistWe're looking for a skilled Business Intelligence Data Specialist to join our team. As a key member of our data analytics group, you will be responsible for developing and maintaining automated data pipelines that support our business intelligence initiatives.">">Pipeline Development: We require an expert in designing,...
-
Senior Software Engineer
4 days ago
Gandhinagar, Gujarat, India beBeeEngineering Full time ₹ 24,42,000 - ₹ 25,90,000Python Engineering Lead Position The successful candidate will lead a team of experienced Python engineers and develop scalable software solutions using modern engineering techniques.Key responsibilities include designing reusable libraries, developing Apache Beam pipelines and Databricks jobs, and creating pluggable interfaces for multiple teams to...