
Data Engineer for Efficient Pipeline Development
7 days ago
Job Title: Data Engineering Lead
We are seeking a motivated and skilled ETL developer to join our data engineering team. As an ETL lead, you will be responsible for designing, developing, and maintaining efficient data pipelines using IBM DataStage (on Cloud Pak for Data), AWS Glue, and Snowflake.
Key Responsibilities:
- Design, develop, and maintain ETL pipelines using IBM DataStage (CP4D) and AWS Glue/Lambda for ingestion from varied sources.
- Build and optimize data flows for loading curated datasets into Snowflake, leveraging best practices for schema design, partitioning, and transformation logic.
- Participate in code reviews, performance tuning, and defect triage sessions.
- Work closely with data governance teams to ensure lineage, privacy tagging, and quality controls are embedded within pipelines.
- Contribute to CI/CD integration of ETL components using Git, Jenkins, and parameterized job configurations.
Requirements:
- 4+ years of experience in ETL development with at least 1–2 years in IBM DataStage (preferably CP4D version).
- Hands-on experience with AWS Glue (PySpark or Spark) and AWS Lambda for event-based processing.
- Experience working with Snowflake: loading strategies, stream-task, zero-copy cloning, and performance tuning.
- Proficiency in SQL, Unix scripting, and basic Python for data handling or automation.
-
Data Pipeline Developer
2 days ago
Allahabad, Uttar Pradesh, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000Job Overview:As a data pipeline developer, you will be responsible for designing and implementing efficient data processing systems. You will collaborate with teams to meet business data needs and ensure data accuracy.Key Responsibilities:Develop and maintain data pipelines using Python and AWS services such as S3, Redshift, Lambda, and Glue.Implement and...
-
Data Pipeline Engineer
5 days ago
Allahabad, Uttar Pradesh, India beBeeDatabaseEngineer Full time ₹ 18,90,000 - ₹ 25,20,000Database Engineering and Testing SpecialistThis role presents an excellent opportunity to leverage your expertise in database engineering and testing.Key Responsibilities:Design efficient data pipelines utilizing ETL concepts and data warehouse testing methodologies.Maintain and monitor scripts for data integrity and quality.Proficiently work with RDBMS:...
-
Data Pipeline Specialist
2 days ago
Allahabad, Uttar Pradesh, India beBeeSenior Full time ₹ 12,00,000 - ₹ 30,00,000Job Title: Senior Test Engineer">The role of a senior test engineer is multifaceted, requiring expertise in designing and executing manual and automated test cases for complex data pipelines.">Key Responsibilities:">">Design and develop test cases for Azure Data Factory ETL pipelines using Python and Pytest.">Validate data transformations, mappings, and...
-
Senior Data Pipeline Developer
7 days ago
Allahabad, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000Highly skilled data professionals are sought after to spearhead the development of complex data pipelines. This role involves leveraging expertise in streaming data environments and frameworks such as Apache Spark, Streamsets, or Apache NIFI to ensure high-quality data delivery.Mandatory Skills:Experience with Kafka Framework, including kSQL and Mirror...
-
Data Pipeline Specialist
6 days ago
Allahabad, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 10,00,000 - ₹ 12,00,000Job Overview:We are seeking an accomplished Data Engineer to spearhead our data pipeline initiatives.About the Role:The ideal candidate will possess experience in designing and implementing data pipelines utilizing Google Cloud Platform and Azure Databricks. Proficiency in DBT and a solid understanding of secure coding practices are essential.In addition,...
-
BFSI Domain Data Pipeline Architect
2 days ago
Allahabad, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000Job OpportunityWe are seeking a seasoned Data Engineer to spearhead our BFSI project. In this pivotal role, you will collaborate with data science teams to design and develop scalable data pipelines, leveraging advanced analytics to empower business decision-making.Key Responsibilities:Collaborate with cross-functional teams to implement complex algorithms...
-
#Top Data Pipeline Architect
3 days ago
Allahabad, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000Senior Data EngineerAbout the RoleWe are seeking a highly skilled Senior Data Engineer with deep expertise in designing, building, and operating data pipelines that support real-time clinical workflows. This is a critical role where you will create scalable pipelines that process clinical encounter data, retrieve associated medical documents, and deliver...
-
Building Scalable Data Pipelines
3 days ago
Allahabad, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000Azure Data Engineer OpportunityDear candidate, we are seeking a skilled professional to design, build, and maintain scalable data pipelines using cloud-based data platforms.The ideal candidate will have hands-on experience with Azure Data Factory (ADF), Azure Databricks (ADB), PySpark, and SQL. They will work closely with cross-functional teams to develop...
-
Data Engineering Specialist
2 days ago
Allahabad, Uttar Pradesh, India beBeeBigData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000Job Title: Data Engineering SpecialistJob Overview:As a Data Engineering Specialist, you will play a pivotal role in designing and developing large-scale data processing systems using cutting-edge technologies.This challenging position requires expertise in big data technologies, including PySpark and Hive. Familiarity with Spark, HBase, and data quality...
-
Data Solutions Engineer
5 days ago
Allahabad, Uttar Pradesh, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000Unlocking Data PotentialWe are dedicated to helping clients build scalable data platforms and AI solutions that drive real business impact.Solution Architecture: Design and implement efficient, secure, and scalable data solutions that meet client requirements.Engineering Expertise: Develop data pipelines, architect data lakes, and implement data warehousing...