Senior Data Engineer for Scalable Pipelines

4 days ago

Malappuram, Kerala, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000
ETL Developer/Senior Developer Role

We are seeking a highly skilled ETL developer to join our data engineering team.

About the Job

The successful candidate will be responsible for designing, developing, and maintaining scalable and efficient data pipelines using IBM DataStage, AWS Glue, and Snowflake.

Main Responsibilities

  • Design, develop, and maintain ETL pipelines using IBM DataStage and AWS Glue/Lambda for ingestion from varied sources.
  • Build and optimize data flows for loading curated datasets into Snowflake, leveraging best practices for schema design, partitioning, and transformation logic.
  • Participate in code reviews, performance tuning, and defect triage sessions.
  • Work closely with data governance teams to ensure lineage, privacy tagging, and quality controls are embedded within pipelines.
  • Contribute to CI/CD integration of ETL components using Git, Jenkins, and parameterized job configurations.
  • Troubleshoot and resolve issues in QA/UAT/Production environments as needed.

Required Skills and Qualifications

  • 4+ years of experience in ETL development with at least 1–2 years in IBM DataStage (preferably CP4D version).
  • Hands-on experience with AWS Glue (PySpark or Spark) and AWS Lambda for event-based processing.
  • Experience working with Snowflake: loading strategies, stream-task, zero-copy cloning, and performance tuning.
  • Proficiency in SQL, Unix scripting, and basic Python for data handling or automation.
  • Familiarity with S3, version control systems (Git), and job orchestration tools.
  • Experience with data profiling, cleansing, and quality validation routines.
  • Understanding of data lake/data warehouse architectures and DevOps practices.

Bonus Points

  • Experience with Collibra, BigID, or other metadata/governance tools
  • Exposure to Data Mesh/Data Domain models
  • Experience with agile/Scrum delivery and Jira/Confluence tools
  • AWS or Snowflake certification is a plus

Key Performance Indicators

The ideal candidate will have a strong understanding of data engineering principles and practices, excellent problem-solving skills, and a passion for delivering high-quality results under tight deadlines.

],

  • Building Scalable Data Pipelines

    1 week ago

    Malappuram, Kerala, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Business Intelligence ArchitectWe are seeking a skilled professional to design, build, and optimize scalable data pipelines using Snowflake, AWS (Lambda, Glue), DBT, and SQL.The ideal candidate will be responsible for enabling seamless data integration, transformation, and analytics to support business intelligence and advanced analytics...

  • Senior Data Pipeline Specialist

    1 week ago

    Malappuram, Kerala, India beBeeDataEngineer Full time ₹ 25,00,000 - ₹ 35,00,000

    Are you a data expert looking to leverage your skills in designing and building robust data pipelines?We are seeking an experienced Azure Data Engineer to join our team.The ideal candidate will have strong SQL skills, experience with Fabric, and proficiency in Python (PySpark).Additionally, they will have hands-on experience with Azure Data Services, CI/CD...

  • Cloud Data Engineer

    1 day ago

    Malappuram, Kerala, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job Title: Azure Data EngineerAbout this role:We are seeking a skilled data engineer to design, develop and maintain scalable ETL/ELT pipelines using Apache Airflow, Apache Spark, Azure Data Factory and Databricks. This role requires integration and management of structured and unstructured data from diverse sources, including Azure Cosmos DB, Azure Data...

  • Data Pipeline Architect

    1 week ago

    Malappuram, Kerala, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,10,000

    Job DescriptionAs a data engineer, you will play a crucial role in designing and implementing scalable data pipelines to support our business growth.

  • Data Engineer: Designing Scalable Data Solutions

    4 days ago

    Malappuram, Kerala, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job Title: Senior Data ArchitectJob Summary:We are seeking a seasoned data engineer to design and optimize data solutions that power analytics, reporting, and business intelligence initiatives.The ideal candidate will have deep expertise in ETL development, data modeling, and financial data domains, particularly within credit unions and banking systems.Key...

  • Building Scalable Data Platforms

    6 days ago

    Malappuram, Kerala, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title: Data EngineerWe are seeking a skilled professional to design, implement, and maintain scalable data platforms using open-source technologies.The ideal candidate will have expertise in PostgreSQL and hands-on experience in building data warehouses and data lakes.Responsibilities:Designing and implementing a PostgreSQL-based data warehouse for...

  • Data Ingestion Engineer

    3 days ago

    Malappuram, Kerala, India beBeeDataIngestion Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Data Ingestion SpecialistJob DescriptionWe are seeking an experienced Data Ingestion Specialist to design, develop and optimize data ingestion pipelines for integrating multiple sources into Databricks. The ideal candidate will have hands-on experience with data pipelines, automation, and infrastructure management to support scalable and efficient...

  • Chief Data Engineering Strategist

    7 days ago

    Malappuram, Kerala, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Senior Data Engineer PositionWe are seeking an experienced Senior Data Engineer to lead the design and implementation of scalable, high-performance data pipelines. The role involves building and optimizing ETL/ELT workflows, managing big data environments, and ensuring seamless data availability for analytics, AI/ML, and business intelligence use cases.About...

  • Senior Data Engineer

    1 week ago

    Malappuram, Kerala, India beBeeData Full time ₹ 20,00,000 - ₹ 30,00,000

    Data Engineer - Cloud Expert We are seeking a seasoned and proficient Senior Data Engineer with substantial experience in cloud technologies to join our dynamic team. As a pivotal member of our data engineering team, you will play a crucial role in designing, implementing, and optimizing data pipelines, ensuring seamless integration with cloud platforms. Job...

  • Data Pipeline Architect

    1 day ago

    Malappuram, Kerala, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,25,000

    Data Engineer PositionWe are seeking a skilled professional to design and implement scalable data pipelines using modern cloud and big data platforms. Key Responsibilities: Design, develop, and maintain large-scale data processing systems Implement efficient data ingestion, storage, and retrieval solutions Collaborate with cross-functional teams to ensure...