Data Pipeline Engineer

1 day ago

Kanpur, Uttar Pradesh, India beBeeETL Full time ₹ 10,00,000 - ₹ 15,00,000
Senior ETL Developer

We are seeking an accomplished Senior ETL Developer to join our Data Engineering team. The ideal candidate will have a proven track record of designing, developing, and maintaining scalable and efficient data pipelines using IBM DataStage, AWS Glue, and Snowflake.

The role involves collaborating with architects, business analysts, and data modelers to ensure timely and accurate delivery of critical data assets supporting analytics and AI/ML use cases.

About the Role:
  • Design and develop ETL pipelines using IBM DataStage and AWS Glue/Lambda for ingestion from varied sources.
  • Build and optimize data flows for loading curated datasets into Snowflake, leveraging best practices for schema design, partitioning, and transformation logic.
  • Participate in code reviews, performance tuning, and defect triage sessions.
  • Work closely with data governance teams to ensure lineage, privacy tagging, and quality controls are embedded within pipelines.
  • Contribute to CI/CD integration of ETL components using Git, Jenkins, and parameterized job configurations.
  • Troubleshoot and resolve issues in QA/UAT/Production environments as needed.
Required Skills and Experience:
  • 4+ years of experience in ETL development with at least 1–2 years in IBM DataStage.
  • Hands-on experience with AWS Glue (PySpark or Spark) and AWS Lambda for event-based processing.
  • Experience working with Snowflake: loading strategies, stream-task, zero-copy cloning, and performance tuning.
  • Proficiency in SQL, Unix scripting, and basic Python for data handling or automation.
  • Familiarity with S3, version control systems (Git), and job orchestration tools.
  • Experience with data profiling, cleansing, and quality validation routines.

  • Data Pipeline Specialist

    1 day ago

    Kanpur, Uttar Pradesh, India beBeeData Full time ₹ 10,00,000 - ₹ 20,00,000

    Big Data EngineerWe are seeking a highly skilled professional to join our team as a Big Data Engineer. In this role, you will be responsible for designing, building, and maintaining robust data pipelines that handle high-volume financial data.Key Responsibilities:Design, develop, and manage end-to-end data pipelines for stocks, crypto, and other financial...

  • Data Pipeline Specialist

    2 days ago

    Kanpur, Uttar Pradesh, India beBeeData Full time ₹ 15,00,000 - ₹ 17,50,000

    About the RoleWe're seeking a highly skilled Data Engineer to build robust pipelines that transform large, messy technical datasets into high-quality structured data.

  • Data Pipeline Architect

    7 days ago

    Kanpur, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    About the Role:The ideal candidate will have a solid foundation in data engineering with a minimum of 7 years of experience in development on data-centric projects.Mandatory Skills:Expertise in working with real-time data and Kafka framework, including kSQL/Mirror Maker etc.Proficiency in at least one programming language: Groovy or Java.Thorough...

  • Data Pipeline Specialist

    3 days ago

    Kanpur, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

    Job OpportunityWe are seeking a skilled professional to take on the role of Data Engineer, responsible for designing, developing, and maintaining data pipelines that ingest data from various sources into our enterprise-wide data warehouse system.Utilize cutting-edge technologies like AWS services (AWS Lambda, AWS DMS, and AWS Glue) for data ingestion and...

  • Engineering Pipeline Expert

    3 days ago

    Kanpur, Uttar Pradesh, India beBeeAutomation Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job SummaryPure Storage seeks a hands-on Engineering Manager to lead the Tools & Frameworks Team in Observability & Reliability, responsible for scaling automated test infrastructure for all-flash storage solutions.This role requires a passionate leader who improves efficiency, streamlines build-test pipelines, and drives faster development cycles.Key...

  • High-Performance Data Pipelines Expert

    3 days ago

    Kanpur, Uttar Pradesh, India beBeeDataEngineering Full time ₹ 18,00,000 - ₹ 25,00,000

    Are you a data engineering expert looking for a new challenge? This is an excellent opportunity to leverage your skills and experience in designing, developing, and maintaining scalable and high-performance data pipelines using Databricks and PySpark.About the RoleThis is a Senior Data Engineer position where you will be working on the design, development,...

  • Senior Data Pipeline Architect

    4 days ago

    Kanpur, Uttar Pradesh, India beBeeEngineeringManager Full time ₹ 20,00,000 - ₹ 25,00,000

    About the RoleWe are seeking an experienced Engineering Manager to lead a team of data engineers in designing and delivering scalable data pipelines and analytics platforms using Big Data platforms, Microsoft Azure, and Databricks.Main ResponsibilitiesLead, mentor, and grow a team of data engineers working on large-scale distributed data systems.Architect...

  • High-Performance Data Pipeline Specialist

    4 days ago

    Kanpur, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    About Us We're looking for a talented Data Engineer to help us scale and optimize our data processing capabilities. Role Overview You will design, build, and maintain high-performance data pipelines that process terabytes of data. Key Responsibilities Design and implement scalable batch processing systems using Python and big data technologies Optimize...

  • Advanced Data Engineer

    3 days ago

    Kanpur, Uttar Pradesh, India beBeeData Full time ₹ 15,00,000 - ₹ 22,50,000

    About the RoleWe're seeking a seasoned Data Engineering professional to spearhead the design, development, and operation of scalable data pipelines.As a critical member of our team, you will create event-driven data pipelines that support real-time clinical workflows. Your work will involve integrating healthcare data with EHR/EMR systems using standards...

  • Data Engineering Expert

    3 days ago

    Kanpur, Uttar Pradesh, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title: Data Engineering ExpertAbout the Role:We are seeking a skilled Data Engineer to build and maintain large-scale data processing workflows using leading-edge technologies like Azure Databricks, PySpark, and Apache Spark.The ideal candidate will be responsible for performing rigorous data pre-processing tasks such as cleaning, transformation,...