Data Pipeline Engineer

3 days ago

Mysore, Karnataka, India beBeeETLPipeline Full time ₹ 10,00,000 - ₹ 16,00,000

Job Summary

We are seeking a talented and motivated professional to join our data engineering team. As an ETL Developer, you will work on building scalable and efficient data pipelines using IBM DataStage (on Cloud Pak for Data), AWS Glue, and Snowflake.

Key Responsibilities

  • Design, develop, and maintain ETL pipelines using IBM DataStage (CP4D) and AWS Glue/Lambda for ingestion from varied sources like flat files, APIs, Oracle, DB2, etc.
  • Build and optimize data flows for loading curated datasets into Snowflake, leveraging best practices for schema design, partitioning, and transformation logic.
  • Participate in code reviews, performance tuning, and defect triage sessions.
  • Work closely with data governance teams to ensure lineage, privacy tagging, and quality controls are embedded within pipelines.
  • Contribute to CI/CD integration of ETL components using Git, Jenkins, and parameterized job configurations.
  • Troubleshoot and resolve issues in QA/UAT/Production environments as needed.
  • Adhere to agile delivery practices, sprint planning, and documentation requirements.

Required Skills and Experience

  • 4+ years of experience in ETL development with at least 1–2 years in IBM DataStage (preferably CP4D version).
  • Hands-on experience with AWS Glue (PySpark or Spark) and AWS Lambda for event-based processing.
  • Experience working with Snowflake: loading strategies, stream-task, zero-copy cloning, and performance tuning.
  • Proficiency in SQL, Unix scripting, and basic Python for data handling or automation.
  • Familiarity with S3, version control systems (Git), and job orchestration tools.
  • Experience with data profiling, cleansing, and quality validation routines.
  • Understanding of data lake/data warehouse architectures and DevOps practices.

Good to Have

  • Experience with Collibra, BigID, or other metadata/governance tools
  • Exposure to Data Mesh/Data Domain models
  • Experience with agile/Scrum delivery and Jira/Confluence tools
  • AWS or Snowflake certification is a plus

About the Role

This position offers innovative opportunities to work with cutting-edge technologies and contribute to the success of our organization.

What We Offer

  • Opportunities to work with innovative projects
  • Collaborative and dynamic work environment
  • Professional growth and development opportunities
  • Competitive salary and benefits package
  • Recognition and rewards for outstanding performance

How to Apply

Interested candidates should submit their resume and cover letter to us.

  • AWS Data Pipeline Engineer

    2 days ago

    Mysore, Karnataka, India beBeeData Full time ₹ 18,00,000 - ₹ 20,00,000

    Dear Candidate, we are seeking a highly skilled professional to join our data engineering team as an AWS Data Architect.The ideal candidate will have deep expertise in building scalable data pipelines using Apache Spark, PySpark, SQL, and Python, along with hands-on experience in the AWS ecosystem.Our requirements include:Designing, Developing, and...

  • Data Pipeline Architect

    1 week ago

    Mysore, Karnataka, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Senior Data EngineerAbout the RoleWe are seeking a seasoned professional to build data pipelines and integrate healthcare data.This role is critical in creating scalable pipelines that process clinical encounter data, retrieve associated medical documents, and deliver validated information to machine learning systems for inference.You will work...

  • Data Pipeline Specialist

    3 days ago

    Mysore, Karnataka, India beBeeDataEngineer Full time ₹ 75,00,000 - ₹ 1,20,00,000

    Data Engineering is a vital function within our organization, driving business growth and informing strategic decision-making.We're on a mission to revolutionize the food and beverage industry with our powerful cloud-based platform.As a Data Engineer, you'll design and optimize distributed data pipelines, infrastructure, and tools that power insights across...

  • Azure Data Pipeline Architect

    1 week ago

    Mysore, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 21,00,000

    Job Overview:We are seeking a skilled professional to design and deploy robust data pipelines that support data governance and analytics initiatives. This role will focus on building production-ready workflows for ingestion, transformation, quality checks, lineage capture, access auditing, cost usage analysis, retention tracking, and metadata integration,...

  • Chief Data Pipeline Architect

    5 days ago

    Mysore, Karnataka, India beBeeDataEngineer Full time US$ 1,80,000 - US$ 2,20,000

    Cloud Data Engineer Job DescriptionWe are seeking an experienced Cloud Data Engineer to design and develop scalable ETL pipelines using cloud-native tools. The ideal candidate will have extensive experience with cloud platforms (AWS, Azure, GCP), data lakes, and warehouses.Key Responsibilities:Design and Development: Design scalable ETL pipelines, architect...

  • Chief Data Pipeline Architect

    5 days ago

    Mysore, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Key Data Engineer RoleCraft efficient data pipelines and collaborate with cross-functional teams to empower decision-making through data-driven insights.

  • Senior Data Pipeline Specialist

    23 hours ago

    Mysore, Karnataka, India beBeeData Full time ₹ 7,00,000 - ₹ 14,00,000

    Job DescriptionWe are seeking a skilled professional to lead the design, development, and deployment of data pipelines on AWS. As a Data Engineer, you will collaborate with cross-functional teams to identify business requirements and implement data-driven solutions.Main Responsibilities:Design, develop, and deploy data pipelines on AWSCollaborate with...

  • Advanced Data Pipeline Architect

    1 week ago

    Mysore, Karnataka, India beBeeDataEngineer Full time US$ 1,00,000 - US$ 1,40,000

    **Job Summary**As a skilled data engineer, you will be responsible for designing and implementing efficient data pipelines using Python and AWS technologies. Your primary goal will be to ensure seamless data flow across various systems.",

  • Data Pipeline Architect

    2 weeks ago

    Mysore, Karnataka, India beBeeDataPipelineDeveloper Full time ₹ 37,50,000 - ₹ 52,50,000

    Senior Data Pipeline DeveloperJob OverviewAs a Senior Data Pipeline Developer, you will be responsible for designing and implementing high-performance data pipelines using advanced ETL concepts and methodologies.Key ResponsibilitiesDesign and implement complex data pipelines utilizing ETL frameworks and tools such as SSIS and AWS GlueDevelop and maintain...

  • Experienced Data Pipeline Developer

    3 days ago

    Mysore, Karnataka, India beBeeDataPipelineDeveloper Full time ₹ 10,00,000 - ₹ 20,00,000

    Job Title: Experienced Data Pipeline DeveloperWe are seeking a skilled data pipeline developer to join our team. As a key member of our development team, you will be responsible for designing, building, and optimizing data pipelines using IBM DataStage, along with strong SQL skills in Teradata.Key Responsibilities:Design and Develop ETL JobsWrite Optimized...