Data Pipelining Specialist

5 days ago

Vellore, Tamil Nadu, India beBeeETLPipeliningEngineer Full time ₹ 18,00,000 - ₹ 22,00,000
Job Title:
Data Pipelining Engineer.

We are seeking a skilled Data Pipelining Engineer to join our data engineering team. The successful candidate will be responsible for designing, developing, and maintaining scalable and efficient data pipelines using IBM DataStage (on Cloud Pak for Data), AWS Glue, and Snowflake.

Key Responsibilities:
  1. Design and develop ETL pipelines using IBM DataStage (CP4D) and AWS Glue/Lambda for ingestion from varied sources like flat files, APIs, Oracle, DB2, etc.
  2. Bulk and optimize data flows for loading curated datasets into Snowflake, leveraging best practices for schema design, partitioning, and transformation logic.
  3. Participate in code reviews, performance tuning, and defect triage sessions.
  4. Work closely with data governance teams to ensure lineage, privacy tagging, and quality controls are embedded within pipelines.
  5. Contribute to CI/CD integration of ETL components using Git, Jenkins, and parameterized job configurations.
  6. Troubleshoot and resolve issues in QA/UAT/Production environments as needed.

Required Skills and Experience:
  1. 4+ years of experience in ETL development with at least 1–2 years in IBM DataStage (preferably CP4D version).
  2. Hands-on experience with AWS Glue (PySpark or Spark) and AWS Lambda for event-based processing.
  3. Experience working with Snowflake: loading strategies, stream-task, zero-copy cloning, and performance tuning.
  4. Proficiency in SQL, Unix scripting, and basic Python for data handling or automation.
  5. Familiarity with S3, version control systems (Git), and job orchestration tools.
  6. Experience with data profiling, cleansing, and quality validation routines.
  7. Understanding of data lake/data warehouse architectures and DevOps practices.

Good to Have:
  1. Experience with Collibra, BigID, or other metadata/governance tools
  2. Exposure to Data Mesh/Data Domain models
  3. Experience with agile/Scrum delivery and Jira/Confluence tools
  4. AWS or Snowflake certification is a plus


This role offers an exciting opportunity to work on cutting-edge data engineering projects and collaborate with a talented team of professionals. If you have the necessary skills and experience, we encourage you to apply for this position.

About the Role:
The ideal candidate will have a strong background in data pipelining and experience working with IBM DataStage, AWS Glue, and Snowflake. They will be able to design, develop, and maintain scalable and efficient data pipelines, as well as collaborate with the data governance team to ensure high-quality data assets. In addition, they will participate in code reviews, performance tuning, and defect triage sessions, and contribute to CI/CD integration of ETL components.

  • Data Pipeline Specialist

    2 days ago

    Vellore, Tamil Nadu, India beBeeEtl Full time ₹ 15,90,000 - ₹ 19,90,000

    We are seeking a skilled ETL developer to join our team. The ideal candidate will have experience in designing and optimizing complex data pipelines, debugging, and deployment using Talend.**Position Details:**- **Role:** ETL Developer- **Experience:** 8 to 10 years- **Work Timings:** Flexible schedule with evening shifts- **Duration:** 3 Months (Contract)-...

  • Building Scalable Data Pipelines

    13 hours ago

    Vellore, Tamil Nadu, India beBeeDataEngineer Full time ₹ 16,00,000 - ₹ 22,00,000

    About this JobWe are seeking a Senior Data Engineer to lead our data infrastructure initiatives. This critical role involves designing and operating scalable data pipelines that support real-time clinical workflows.Key Responsibilities:Create event-driven data pipelines integrating healthcare data from various sources.Collaborate with software engineers and...

  • Chief Data Pipeline Architect

    13 hours ago

    Vellore, Tamil Nadu, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Azure Data EngineerWe are seeking a skilled Azure Data Engineer to fill this key role in our organization.Key Responsibilities:Design and implement data pipelines using PySpark, SQL, and Azure Data Factory (ADF)Collaborate with stakeholders to deliver scalable data-driven outcomesWork in Agile teams to handle data validation, transformation, and...

  • Data Transformation Specialist

    15 hours ago

    Vellore, Tamil Nadu, India beBeeData Full time ₹ 12,00,000 - ₹ 18,00,000

    Job DescriptionWe are seeking a skilled Data Specialist to design and implement robust data pipelines that transform large, complex datasets into high-quality structured information.The primary focus is to:Extract key attributes from unstructured/semi-structured documents using advanced automation techniques.Validate and check data quality with automated...

  • Software Professional: Data Pipeline Engineer

    14 hours ago

    Vellore, Tamil Nadu, India beBeeBigDataDeveloper Full time ₹ 90,00,000 - ₹ 1,20,00,000

    Senior Big Data DeveloperWe are looking for a seasoned Senior Big Data Developer to design and implement robust data pipelines using Python and SQL.Develop, test, and maintain large-scale data processing systems in Databricks.Configure and manage GitHub branch protection rules and workflows to ensure smooth code integration and deployment.Implement unit...

  • Azure Data Engineer: Expertise in Data Pipeline Design and Implementation

    15 hours ago

    Vellore, Tamil Nadu, India beBeeDataEngineering Full time ₹ 1,20,00,000 - ₹ 1,50,00,000

    Job TitleAzure Data Engineer: Expertise in Data Pipeline Design and ImplementationJob DescriptionWe are seeking a skilled professional with expertise in designing, implementing, and maintaining data pipelines using ADF, ADB, PySpark, and SQL.The ideal candidate will have strong passion for data engineering and keen eye for detail.Required Skills and...

  • Data Engineering Specialist

    2 days ago

    Vellore, Tamil Nadu, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job OverviewWe are seeking a highly skilled professional to design and develop efficient ETL pipelines using FiveTran, DBT Cloud, and custom frameworks.Key ResponsibilitiesETL Pipeline Development: Develop scalable and maintainable ETL pipelines to ensure seamless data integration.Data Transformation: Implement client-specific data transformations for...

  • Advanced Data Architecture Specialist

    12 hours ago

    Vellore, Tamil Nadu, India beBeeData Full time ₹ 80,00,000 - ₹ 1,50,00,000

    Job DescriptionAs a Senior Data Engineer, you will play a key role in designing, building, and optimizing scalable data pipelines and Lakehouse architectures using Microsoft Azure.Our ideal candidate will have expertise in Azure Data Services, strong proficiency in SQL for complex transformations, and advanced Python skills for data pipeline development and...

  • Strategic Data Insights Professional

    4 days ago

    Vellore, Tamil Nadu, India beBeeAnalytics Full time ₹ 1,00,00,000 - ₹ 1,50,00,000

    Advanced Data Analytics SpecialistWe are seeking a skilled Advanced Data Analytics Specialist to drive strategic growth in the pharmaceutical and healthcare sectors. This role involves leveraging cutting-edge data analytics and predictive modeling techniques to deliver actionable insights that inform product development, marketing strategies, and business...

  • Lead Data Extraction Specialist

    1 day ago

    Vellore, Tamil Nadu, India beBeeData Full time ₹ 15,00,000 - ₹ 20,00,000

    We are seeking an exceptional Data Extraction Specialist to lead our team of skilled engineers. As a seasoned professional, you will be responsible for designing and developing scalable data extraction solutions using tools like Scrapy and Python libraries.Key Responsibilities:Data Extraction: Develop advanced data extraction techniques using Scrapy,...