Senior Data Pipeline Specialist

7 days ago

Pune, Maharashtra, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

The Role of a Data Pipeline Developer.

We are seeking an experienced Data Pipeline Developer to join our team. The ideal candidate will have a strong background in designing, developing and maintaining data pipelines using IBM DataStage (on Cloud Pak for Data) and AWS Glue/Lambda.

Key Responsibilities

  • Design, develop and maintain ETL pipelines using IBM DataStage (CP4D) and AWS Glue/Lambda for ingestion from varied sources like flat files, APIs, Oracle, DB2 etc.
  • Build and optimize data flows for loading curated datasets into Snowflake leveraging best practices for schema design, partitioning and transformation logic.
  • Participate in code reviews, performance tuning and defect triage sessions.
  • Work closely with data governance teams to ensure lineage, privacy tagging and quality controls are embedded within pipelines.
  • Contribute to CI/CD integration of ETL components using Git, Jenkins and parameterized job configurations.
  • Troubleshoot and resolve issues in QA/UAT/Production environments as needed.

Requirements

  • 4+ years of experience in ETL development with at least 1–2 years in IBM DataStage (preferably CP4D version).
  • Hands-on experience with AWS Glue (PySpark or Spark) and AWS Lambda for event-based processing.
  • Experience working with Snowflake: loading strategies, stream-task, zero-copy cloning and performance tuning.
  • Proficiency in SQL, Unix scripting and basic Python for data handling or automation.
  • Familiarity with S3, version control systems (Git), and job orchestration tools.
  • Experience with data profiling, cleansing and quality validation routines.
  • Understanding of data lake/data warehouse architectures and DevOps practices.

Bonus Skills

  • Experience with Collibra, BigID or other metadata/governance tools
  • Exposure to Data Mesh/Data Domain models
  • Experience with agile/Scrum delivery and Jira/Confluence tools
  • AWS or Snowflake certification is a plus

  • Senior Data Pipeline Expert

    2 days ago

    Pune, Maharashtra, India beBeeData Full time ₹ 1,80,00,000 - ₹ 2,00,00,000

    Job Title: Data Architect SpecialistDescription:We are looking for a skilled Data Architect Specialist to craft and maintain robust data pipelines, utilizing Python and SQL, to ensure efficient extraction, transformation, and loading (ETL) of data. This role involves building and optimizing data pipelines to facilitate seamless data flow across systems and...

  • Expert Data Pipeline Developer

    2 days ago

    Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,30,00,000

    Senior Data Engineer PositionWe are seeking a highly skilled Senior Data Engineer to join our team. As a critical member of our AI-powered applications, you will design and build scalable data pipelines that process clinical encounter data.You will work closely with software engineers, machine learning specialists, and our clinical partners to bring our...

  • Data Pipeline Automation Specialist

    3 days ago

    Pune, Maharashtra, India beBeeAutomation Full time ₹ 9,00,000 - ₹ 12,00,000

    We are seeking a skilled Data Pipeline Automation Specialist to join our team. The ideal candidate will have extensive experience in designing, developing and maintaining automated testing frameworks for large-scale data pipelines.The successful candidate will be responsible for ensuring the stability and quality of our data pipeline solutions.This is an...

  • Senior Data Pipeline Architect

    17 hours ago

    Pune, Maharashtra, India beBeeData Full time US$ 1,80,000 - US$ 2,20,000

    Senior Data Engineer RoleWe are seeking a talented and passionate Senior Data Engineer to join our team.This role involves building and scaling our data infrastructure, enabling business-driven decision-making across the organization.The ideal candidate will design, develop, and maintain efficient and reliable data pipelines for both ELT (Extract, Load,...

  • Senior Data Pipeline Architect

    2 days ago

    Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

    Unlock your potential with a career in data engineering.Job RoleOur organization is one of the largest banking and financial services providers globally, operating in 64 countries and territories. Our mission is to empower businesses and communities by facilitating growth and prosperity.We are currently seeking an experienced professional to join our team as...

  • Senior Data Pipeline Specialist

    1 day ago

    Pune, Maharashtra, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job OverviewWe are seeking a seasoned Data Engineer to join our team. This role involves crafting and maintaining robust data pipelines using Python and SQL to ensure efficient extraction, transformation, and loading (ETL) of data.The ideal candidate will have expertise in building and optimizing data pipelines to facilitate seamless data flow across systems...

  • Data Pipeline Architect

    3 days ago

    Pune, Maharashtra, India beBeeData Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Senior Data Engineer - AWS ExpertWe are seeking a seasoned professional to spearhead the design, development, and deployment of end-to-end data pipelines on AWS cloud infrastructure.Key Responsibilities:Design scalable data processing workflows using Apache Spark and SQL for analytics and reporting requirements.Build and maintain orchestration workflows to...

  • Senior Data Specialist

    2 days ago

    Pune, Maharashtra, India beBeeData Full time ₹ 18,00,000 - ₹ 24,00,000

    Senior Data SpecialistWe are seeking a highly skilled and motivated professional to fill the position of Senior Data Specialist. The ideal candidate will have a strong foundation in SQL, experience with Python development, and a background in Computer Engineering or Computer Science.The successful applicant will design, develop, and maintain efficient,...

  • Pipeline Operations Specialist

    2 days ago

    Pune, Maharashtra, India beBeePipeline Full time ₹ 19,99,999 - ₹ 24,99,999

    Pipeline SpecialistAs a key member of our Pipelines team, you will be responsible for delivering pipeline engineering expertise to support the safe and reliable operation of our pipelines globally.Key Responsibilities:Regional Pipeline Support: Provide execution and assessment of subsea inspection campaigns and define repair requirements.Subsea and Pipeline...

  • Data Architecture Specialist

    3 days ago

    Pune, Maharashtra, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Senior Data Engineer RoleWe are seeking an experienced professional to join our team in the role of Senior Consultant Specialist.About the Role:This is a senior-level position responsible for designing, building, and managing the data infrastructure and data pipeline processes for our organization. The ideal candidate will have a strong background in data...