Senior Data Pipeline Specialist

15 hours ago

Belgaum, Karnataka, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000
ETL Developer Job Description

We are seeking a talented ETL Developer to join our data engineering team. As an ETL Developer, you will be responsible for designing, developing, and maintaining scalable and efficient data pipelines using IBM DataStage (on Cloud Pak for Data), AWS Glue, and Snowflake.

Key Responsibilities:

  • Design, develop, and maintain ETL pipelines using IBM DataStage (CP4D) and AWS Glue/Lambda for ingestion from varied sources like flat files, APIs, Oracle, DB2, etc.
  • Build and optimize data flows for loading curated datasets into Snowflake, leveraging best practices for schema design, partitioning, and transformation logic.
  • Participate in code reviews, performance tuning, and defect triage sessions.
  • Work closely with data governance teams to ensure lineage, privacy tagging, and quality controls are embedded within pipelines.
  • Contribute to CI/CD integration of ETL components using Git, Jenkins, and parameterized job configurations.
  • Troubleshoot and resolve issues in QA/UAT/Production environments as needed.
Required Skills and Qualifications

To be successful as an ETL Developer, you should have the following skills and qualifications:

  • 4+ years of experience in ETL development with at least 1–2 years in IBM DataStage (preferably CP4D version).
  • Hands-on experience with AWS Glue (PySpark or Spark) and AWS Lambda for event-based processing.
  • Experience working with Snowflake: loading strategies, stream-task, zero-copy cloning, and performance tuning.
  • Proficiency in SQL, Unix scripting, and basic Python for data handling or automation.
  • Familiarity with S3, version control systems (Git), and job orchestration tools.
  • Experience with data profiling, cleansing, and quality validation routines.
  • Understanding of data lake/data warehouse architectures and DevOps practices.
Benefits

As an ETL Developer, you will have the opportunity to work on complex data engineering projects and contribute to the growth and success of our organization.

You will also have access to ongoing training and professional development opportunities to help you advance your career.

Additionally, you will be part of a dynamic and collaborative team that values innovation, creativity, and teamwork.

Others

We offer a competitive salary and benefits package, including health insurance, retirement plan, and paid time off.

We are an equal opportunity employer and welcome applications from diverse candidates.

  • Building Scalable Clinical Data Pipelines

    15 hours ago

    Belgaum, Karnataka, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Senior Data Engineer - Clinical Data Pipeline LeadAbout the RoleWe're seeking a highly experienced Senior Data Engineer to lead the design and operation of our AI-powered applications' backbone. This critical role involves creating scalable pipelines that process clinical data, retrieve medical documents, and deliver validated information to machine learning...

  • Senior Data Pipeline Specialist

    22 minutes ago

    Belgaum, Karnataka, India beBeeData Full time ₹ 25,00,000 - ₹ 30,00,000

    Job Overview:We are seeking a skilled Tester to join our team. As a Tester, you will play a critical role in ensuring the quality and reliability of our data pipelines.The ideal candidate will have a strong background in Data Migration Testing, with experience in source-to-target validation, record count verification, data integrity and validation,...

  • Chief Data Pipeline Architect

    37 minutes ago

    Belgaum, Karnataka, India beBeeDataEngineer Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Senior Data Engineer PositionWe are seeking an accomplished Senior Data Engineer to spearhead the development, maintenance, and optimization of our data infrastructure. This critical role requires a seasoned professional with expertise in designing and implementing large-scale data pipelines, leveraging cloud services, and ensuring seamless integration with...

  • Senior Data Pipeline Architect

    51 minutes ago

    Belgaum, Karnataka, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    We are seeking a highly skilled Data Engineer to join our team. This role involves designing, developing, and optimizing large-scale data pipelines and distributed data processing systems.Key Responsibilities:Design, develop, and optimize large-scale data pipelines and distributed data processing systems.Work extensively with Scala, Spark (PySpark), and...

  • Chief Data Pipeline Architect

    2 days ago

    Belgaum, Karnataka, India beBeeDataEngineer Full time ₹ 12,00,000 - ₹ 25,00,000

    Job OverviewAs a data engineer, you will be responsible for designing and building scalable data pipelines to ingest, process, and store data.The ideal candidate will have expertise in data integration, processing, and storage, as well as knowledge of data optimization architecture, data processes, and data flow.You will work closely with cross-functional...

  • Senior Data Engineer

    14 hours ago

    Belgaum, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    We are seeking a senior Databricks data engineer to design and develop high-performance data pipelines.Job Overview:The ideal candidate will have expertise in PySpark, CI/CD pipelines, and Terraform for infrastructure as code.Key Responsibilities:Develop scalable data pipelines using Databricks and PySparkImplement and manage data solutions on AWS, including...

  • Data Engineering Specialist

    17 hours ago

    Belgaum, Karnataka, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Data Engineering SpecialistJob Description:We are seeking an experienced Data Engineering Specialist to develop, optimize and maintain automated data pipelines supporting business intelligence initiatives. This role will focus on building production-ready workflows for data ingestion, transformation, quality checks and metadata integration,...

  • Data Integration Specialist

    4 days ago

    Belgaum, Karnataka, India beBeeIntegration Full time ₹ 30,00,000 - ₹ 40,00,000

    Job TitleWe are seeking an experienced Data Integration Specialist with a proven track record of delivering high-quality data solutions to join our team.Mastery in ETL tools, specifically Informatica Intelligent Cloud Services (IICS).Design, develop, test, and deploy data pipelines and transformations using Informatica PowerCenter and IICS.Migrate data...

  • Senior Data Architecture Specialist

    2 days ago

    Belgaum, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

    We are seeking a highly skilled Senior Data Engineer to join our team of consultants.This role will involve designing, constructing, and optimizing large-scale data processing systems that enable advanced analytics and business intelligence.The successful candidate will collaborate closely with data scientists, analysts, and other engineers to ensure the...

  • Senior Informatica Developer

    5 days ago

    Belgaum, Karnataka, India Pacific Data Integrators Full time

    Job Title: Senior Data Engineer (IDMC-CDI)Primary Skills: SQL, IDMC (CDI), ETL Concepts BaseLocation: RemoteShift: US EST (5 PM to 2 AM IST)Experience: 5-12 YearsAbout UsWe are PDI (Pacific Data Integrators), a leading provider of Data Management, Analytics, and Cloud solutions. As an Informatica Platinum Partner and a reseller of various market-leading...