Data Pipeline Engineer

3 days ago

Allahabad, Uttar Pradesh, India beBeeDatabaseEngineer Full time ₹ 18,90,000 - ₹ 25,20,000
Database Engineering and Testing Specialist

This role presents an excellent opportunity to leverage your expertise in database engineering and testing.

  • Key Responsibilities:
  • Design efficient data pipelines utilizing ETL concepts and data warehouse testing methodologies.
  • Maintain and monitor scripts for data integrity and quality.
  • Proficiently work with RDBMS: MySQL, Oracle SQL.
  • Familiarity with SSIS (SQL Server Integration Services) and UNIX commands for log analysis and job monitoring.
  • Develop advanced SQL skills: joins, group functions, analytical functions, subqueries.
  • AWS services: CloudWatch, EventBridge, QuickSight, Batch Jobs, Lambda, S3, Redshift, Glue.
  • Testing Expertise:
  • Strong background in Data Migration Testing:
  • Source-to-target validation
  • Record count verification
  • Data integrity and validation
  • Transformation logic and job run verification
  • Experience in system integration and functional testing across sprints/iterations.
  • Additional Skills:
  • Requirement analysis and work effort estimation.
  • Quality management: defect identification, root cause analysis, corrective measures.
  • Strong analytical, problem-solving, and communication skills to collaborate effectively.
Nice to Have
  • Experience automating data testing using AI/ML tools
  • CI/CD pipeline integration for test automation

As a key member of the team, you will have the opportunity to develop and implement efficient data pipelines, maintain and monitor scripts, and work with various RDBMS systems.

  • Data Pipeline Specialist

    1 day ago

    Allahabad, Uttar Pradesh, India beBeeSenior Full time ₹ 12,00,000 - ₹ 30,00,000

    Job Title: Senior Test Engineer">The role of a senior test engineer is multifaceted, requiring expertise in designing and executing manual and automated test cases for complex data pipelines.">Key Responsibilities:">">Design and develop test cases for Azure Data Factory ETL pipelines using Python and Pytest.">Validate data transformations, mappings, and...

  • Data Pipeline Developer

    1 day ago

    Allahabad, Uttar Pradesh, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Overview:As a data pipeline developer, you will be responsible for designing and implementing efficient data processing systems. You will collaborate with teams to meet business data needs and ensure data accuracy.Key Responsibilities:Develop and maintain data pipelines using Python and AWS services such as S3, Redshift, Lambda, and Glue.Implement and...

  • Data Engineer for Efficient Pipeline Development

    6 days ago

    Allahabad, Uttar Pradesh, India beBeeETL Full time ₹ 14,45,838 - ₹ 23,13,723

    Job Title: Data Engineering LeadWe are seeking a motivated and skilled ETL developer to join our data engineering team. As an ETL lead, you will be responsible for designing, developing, and maintaining efficient data pipelines using IBM DataStage (on Cloud Pak for Data), AWS Glue, and Snowflake.Key Responsibilities:Design, develop, and maintain ETL...

  • #Top Data Pipeline Architect

    2 days ago

    Allahabad, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Senior Data EngineerAbout the RoleWe are seeking a highly skilled Senior Data Engineer with deep expertise in designing, building, and operating data pipelines that support real-time clinical workflows. This is a critical role where you will create scalable pipelines that process clinical encounter data, retrieve associated medical documents, and deliver...

  • Data Pipeline Specialist

    5 days ago

    Allahabad, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 10,00,000 - ₹ 12,00,000

    Job Overview:We are seeking an accomplished Data Engineer to spearhead our data pipeline initiatives.About the Role:The ideal candidate will possess experience in designing and implementing data pipelines utilizing Google Cloud Platform and Azure Databricks. Proficiency in DBT and a solid understanding of secure coding practices are essential.In addition,...

  • Building Scalable Data Pipelines

    2 days ago

    Allahabad, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Azure Data Engineer OpportunityDear candidate, we are seeking a skilled professional to design, build, and maintain scalable data pipelines using cloud-based data platforms.The ideal candidate will have hands-on experience with Azure Data Factory (ADF), Azure Databricks (ADB), PySpark, and SQL. They will work closely with cross-functional teams to develop...

  • BFSI Domain Data Pipeline Architect

    9 hours ago

    Allahabad, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job OpportunityWe are seeking a seasoned Data Engineer to spearhead our BFSI project. In this pivotal role, you will collaborate with data science teams to design and develop scalable data pipelines, leveraging advanced analytics to empower business decision-making.Key Responsibilities:Collaborate with cross-functional teams to implement complex algorithms...

  • Senior Data Pipeline Developer

    6 days ago

    Allahabad, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Highly skilled data professionals are sought after to spearhead the development of complex data pipelines. This role involves leveraging expertise in streaming data environments and frameworks such as Apache Spark, Streamsets, or Apache NIFI to ensure high-quality data delivery.Mandatory Skills:Experience with Kafka Framework, including kSQL and Mirror...

  • Fabric Data Engineer

    2 weeks ago

    Allahabad, Uttar Pradesh, India DHJ TECHNOLOGIES Full time

    Job Title: Fabric Data EngineerLocation: [Remote]Type: [Contract]Project Duration: 3 monthsAbout the Role:We are seeking a skilled Fabric Data Engineer to support a key modernization initiative: migrating an on-premises SQL Server 2016 data warehouse to Microsoft Fabric. This role involves designing and building new data pipelines, as well as testing and...

  • Senior Data Engineer

    5 hours ago

    Allahabad, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 12,00,000 - ₹ 18,00,000

    Job OpportunityAptimized seeks a Data Engineering & Business Intelligence professional to support enterprise data initiatives. This role combines hands-on data engineering and ETL development with business intelligence and reporting delivery, ensuring timely and accurate insights for stakeholders.The ideal candidate will be technically proficient in SQL,...