Senior Data Pipeline Developer

4 days ago

Bellary, Karnataka, India beBeeetl Full time ₹ 1,20,00,000 - ₹ 2,00,00,000

Job Title: ETL Specialist - Data Pipeline Developer

Experience: 5–7 Years

Location: Remote

Job Type: Full-time

About the Role

We are seeking an experienced ETL specialist to join our data engineering team. The ideal candidate will have a strong background in designing, developing, and maintaining large-scale data pipelines using IBM DataStage, AWS Glue, and Snowflake.

Key Responsibilities

  • Design, develop, and maintain ETL pipelines using IBM DataStage (CP4D) and AWS Glue/Lambda for ingestion from varied sources like flat files, APIs, Oracle, DB2, etc.
  • Build and optimize data flows for loading curated datasets into Snowflake, leveraging best practices for schema design, partitioning, and transformation logic.
  • Participate in code reviews, performance tuning, and defect triage sessions.
  • Work closely with data governance teams to ensure lineage, privacy tagging, and quality controls are embedded within pipelines.
  • Contribute to CI/CD integration of ETL components using Git, Jenkins, and parameterized job configurations.
  • Troubleshoot and resolve issues in QA/UAT/Production environments as needed.
  • Adhere to agile delivery practices, sprint planning, and documentation requirements.

Required Skills and Experience

  • 4+ years of experience in ETL development with at least 1–2 years in IBM DataStage (preferably CP4D version).
  • Hands-on experience with AWS Glue (PySpark or Spark) and AWS Lambda for event-based processing.
  • Experience working with Snowflake: loading strategies, stream-task, zero-copy cloning, and performance tuning.
  • Proficiency in SQL, Unix scripting, and basic Python for data handling or automation.
  • Familiarity with S3, version control systems (Git), and job orchestration tools.
  • Experience with data profiling, cleansing, and quality validation routines.
  • Understanding of data lake/data warehouse architectures and DevOps practices.

Good to Have

  • Experience with Collibra, BigID, or other metadata/governance tools
  • Exposure to Data Mesh/Data Domain models
  • Experience with agile/Scrum delivery and Jira/Confluence tools
  • AWS or Snowflake certification is a plus

  • Data Pipeline Tester

    5 days ago

    Bellary, Karnataka, India beBeeSenior Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Summary:We are seeking a skilled Senior Test Engineer to join our team. The ideal candidate will have experience in ETL and Power BI testing, with a strong background in Azure Data Factory, Python, and Pytest.The successful candidate will be responsible for designing and developing comprehensive test cases to validate data flows, transformations, and...

  • Senior Data Pipeline Developer

    13 hours ago

    Bellary, Karnataka, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title:Data Solutions EngineerAbout the JobAs a key member of our team, you will play a pivotal role in designing and developing scalable data pipelines and ETL processes to support downstream analytics and AI applications. Your expertise will help ensure seamless integration with machine learning models and workflows.Key Responsibilities...

  • Data Pipeline Architect

    1 week ago

    Bellary, Karnataka, India beBeeDataPipeline Full time ₹ 8,00,000 - ₹ 15,00,000

    Job Title: Data Pipeline ArchitectKey Responsibilities:We're seeking an experienced Data Pipeline Architect to design, develop and optimize data ingestion pipelines for integrating multiple sources into Databricks.The successful candidate will implement and maintain CI/CD pipelines for data workflows.Deploy and manage containerized applications using...

  • Data Pipeline Specialist

    6 days ago

    Bellary, Karnataka, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Data Engineering Role OverviewWe are seeking a skilled Data Engineer to join our organization.As a Data Engineer, you will be responsible for designing, developing, and maintaining efficient data pipelines to ingest data from various sources into a data warehouse.Design and develop optimized data pipelines to extract data from databases and flat...

  • Data Pipelines Specialist

    1 week ago

    Bellary, Karnataka, India beBeeTechnicalConsultant Full time ₹ 80,00,000 - ₹ 1,20,00,000

    Job OverviewThe role involves configuring, integrating, and optimizing financial data workflows within a platform. Collaboration with cross-functional teams is essential for designing reusable and scalable solutions, implementing automation, and enhancing system performance across key investment operations.This position demands hands-on expertise in tools...

  • Data Pipeline Specialist

    3 days ago

    Bellary, Karnataka, India beBeeDataEngineer Full time ₹ 18,00,000 - ₹ 22,00,000

    Job SummaryWe are seeking a talented Data Engineer to join our team. As a Data Engineer, you will design, build, and maintain production-level data pipelines that handle large amounts of structured and unstructured data.Design and implement scalable data pipelines using AWS services such as S3, EC2, RDS, DynamoDB, Athena, and Redshift.Collaborate with...

  • Building Scalable Data Pipelines

    19 hours ago

    Bellary, Karnataka, India beBeeDataEngineer Full time ₹ 12,00,000 - ₹ 20,00,000

    Cloud Data EngineerWe are seeking a highly skilled Cloud Data Engineer to join our team. In this role, you will design and build scalable data pipelines using Google Cloud Platform (GCP) services.The ideal candidate will have experience with GCP data processing technologies such as Cloud Functions, Cloud Run, Pub/Sub, BigQuery, and Cloud Storage. You will...

  • Senior Data Solutions Developer

    3 days ago

    Bellary, Karnataka, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    **Job Opportunity:** Senior Data EngineerWe are seeking a skilled and experienced data engineer to join our dynamic team. In this role, you will design, develop, and maintain scalable data pipelines and data warehouse solutions on Google Cloud Platform (GCP). You will play a critical role in ensuring the seamless flow of data across systems, enabling...

  • Scalable Data Pipeline Specialist

    1 week ago

    Bellary, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

    As a Data Engineer, you will be responsible for designing and developing scalable data pipelines that fuel real-time insights.We are seeking an experienced professional with a strong understanding of data lake and warehouse design principles to join our team.You will be responsible for creating and maintaining cloud-based data lakes, warehouses, and ETL/ELT...

  • Expert Large-Scale Data Pipeline Specialist

    4 days ago

    Bellary, Karnataka, India beBeeDataEngineer Full time ₹ 10,00,000 - ₹ 25,00,000

    Data Engineering OpportunityWe are seeking a seasoned Freelance Data Engineer with expertise in designing, building, and optimizing large-scale data pipelines.This role involves developing and maintaining ETL/ELT pipelines using PySpark, AWS, and Databricks. Additionally, the successful candidate will perform data profiling, validation, and transformation...