Senior Data Pipeline Developer

6 days ago

India beBeeDataEngineer Full time ₹ 9,00,000 - ₹ 12,00,000

We are seeking a skilled ETL Developer / Senior Developer to join our data engineering team in a full-time capacity.

The successful candidate will work on building scalable and efficient data pipelines using IBM DataStage (on Cloud Pak for Data), AWS Glue, and Snowflake.

Key Responsibilities

  • Design, develop, and maintain ETL pipelines using IBM DataStage (CP4D) and AWS Glue/Lambda for ingestion from varied sources like flat files, APIs, Oracle, DB2, etc.
  • Build and optimize data flows for loading curated datasets into Snowflake, leveraging best practices for schema design, partitioning, and transformation logic.
  • Participate in code reviews, performance tuning, and defect triage sessions.
  • Work closely with data governance teams to ensure lineage, privacy tagging, and quality controls are embedded within pipelines.
  • Contribute to CI/CD integration of ETL components using Git, Jenkins, and parameterized job configurations.
  • Troubleshoot and resolve issues in QA/UAT/Production environments as needed.
  • Adhere to agile delivery practices, sprint planning, and documentation requirements.

Required Skills and Experience

  • ~4+ years of experience in ETL development with at least 1–2 years in IBM DataStage (preferably CP4D version).
  • ~ Hands-on experience with AWS Glue (PySpark or Spark) and AWS Lambda for event-based processing.
  • ~ Experience working with Snowflake: loading strategies, stream-task, zero-copy cloning, and performance tuning.
  • ~ Proficiency in SQL, Unix scripting, and basic Python for data handling or automation.
  • ~ Familiarity with S3, version control systems (Git), and job orchestration tools.
  • ~ Experience with data profiling, cleansing, and quality validation routines.
  • ~ Understanding of data lake/data warehouse architectures and DevOps practices.

Good to Have

  • Experience with Collibra, BigID, or other metadata/governance tools.
  • Exposure to Data Mesh/Data Domain models.
  • Experience with agile/Scrum delivery and Jira/Confluence tools.
  • AWS or Snowflake certification is a plus.

  • Lead Data Pipeline Developer

    5 days ago

    India beBeeData Full time ₹ 6,00,000 - ₹ 8,00,000

    Senior Web Engineer Job DescriptionWe are seeking an experienced Senior Web Engineer to build a high-throughput product data ingestion pipeline across hundreds of domains. The successful candidate will design, develop, and implement scalable web crawlers using Python.The role involves collaborating with the data team to integrate vendor/public APIs...

  • Senior Data Pipeline Engineer

    2 weeks ago

    India Remote YipitData (Alternative) Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    About YipitData: YipitData is the leading market research and analytics firm for the disruptive economy and recently raised up to $475M from The Carlyle Group at a valuation over $1B. We analyze billions of alternative data points every day to provide accurate, detailed insights on ridesharing, e-commerce marketplaces, payments, and more. Our on-demand...

  • Scalable Data Pipelines Developer

    2 weeks ago

    India beBeeDataEngineer Full time ₹ 9,00,000 - ₹ 12,00,000

    Data Engineer - Build Scalable Data PipelinesWe're seeking an experienced Data Engineer to design, build and maintain production-level data pipelines handling terabytes of structured and unstructured data. The ideal candidate has a strong background in data engineering and hands-on experience with AWS services.### Job Responsibilities:Design and implement...

  • Senior Developer

    2 weeks ago

    India Pacific Data Integrators Full time

    Job Title: Senior Data Engineer (IDMC-CDI) Primary Skills: SQL, IDMC (CDI), ETL Concepts Base Location: Remote Shift: US EST (5 PM to 2 AM IST)Experience: 5-12 Years We are PDI (Pacific Data Integrators), a leading provider of Data Management, Analytics, and Cloud solutions. As an Informatica Platinum Partner and a reseller of various market-leading...

  • Chief Data Pipeline Architect

    2 weeks ago

    India beBeeDataEngineer Full time US$ 1,00,000 - US$ 1,75,000

    Job SummaryWe are seeking an experienced senior Data Engineer to design, build and maintain robust data pipelines for processing channel activity data from multiple sources.Utilize AWS cloud infrastructure services like Pyspark, Glue, EMR, Lambda, Redshift, Athena and other services like Iceberg to implement scalable data solutions.Collaborate with...

  • Advanced Data Pipeline Developer

    1 week ago

    India beBeeDatastage Full time ₹ 1,50,000 - ₹ 28,00,000

    Teradata with DataStage Developer Job DescriptionWe are seeking an experienced Teradata with DataStage Developer to join our team. This individual will be responsible for designing, building, and optimizing data pipelines using IBM DataStage, as well as writing optimized SQL queries (Teradata) for data extraction, transformation, and validation.The ideal...

  • Data Pipeline Optimization Expert

    2 weeks ago

    India beBeeEtl Full time ₹ 15,00,000 - ₹ 18,00,000

    Senior ETL SpecialistWe are seeking a highly skilled ETL specialist to design and optimize complex data pipelines.Key Skills:Talend Data Integration: Design & optimize complex ETL pipelines, debugging, and deployment using TMCBig Query: Partitioning, clustering, federated queries, advanced SQL (CTEs, arrays, window functions)Python: ETL scripting,...

  • Senior Informatica Developer

    2 weeks ago

    India Pacific Data Integrators Full time

    Job Title: Senior Data Engineer (IDMC-CDI)Primary Skills: SQL, IDMC (CDI), ETL Concepts BaseLocation: RemoteShift: US EST (5 PM to 2 AM IST)Experience: 5-12 YearsAbout UsWe are PDI (Pacific Data Integrators), a leading provider of Data Management, Analytics, and Cloud solutions. As an Informatica Platinum Partner and a reseller of various market-leading...

  • Senior Informatica Developer

    2 weeks ago

    India Pacific Data Integrators Full time

    Job Title: Senior Data Engineer (IDMC-CDI) Primary Skills: SQL, IDMC (CDI), ETL Concepts Base Location: Remote Shift: US EST (5 PM to 2 AM IST) Experience: 5-12 Years About Us We are PDI (Pacific Data Integrators), a leading provider of Data Management, Analytics, and Cloud solutions. As an Informatica Platinum Partner and a reseller of various...

  • Data Pipeline Architect

    2 weeks ago

    India beBeeETLDeveloper Full time ₹ 15,00,000 - ₹ 25,00,000

    About this RoleAs a skilled ETL Developer, you will play a pivotal role in designing, developing, and maintaining scalable and efficient data pipelines using IBM DataStage (on Cloud Pak for Data), AWS Glue, and Snowflake.The ideal candidate will have 4+ years of experience in ETL development with at least 1-2 years in IBM DataStage (preferably CP4D version)...