Senior Data Engineer

3 weeks ago

Meerut, Uttar Pradesh, India Talent Corner HR Services Pvt Ltd Full time

We are looking for an experienced Senior Data Engineer to lead the development of scalable AWS-native data lake pipelines with a strong focus on time series forecasting and upsert-ready architectures. This role requires end-to-end ownership of the data lifecycle, from ingestion to partitioning, versioning, and BI delivery. The ideal candidate must be highly proficient in AWS data services, PySpark, versioned storage formats like Apache Hudi/Iceberg, and must understand the nuances of data quality and observability in large-scale analytics systems.

Responsibilities

● Design and implement data lake zoning (Raw → Clean → Modeled) using Amazon S3, AWS Glue, and Athena.

● Ingest structured and unstructured datasets including POS, USDA, Circana, and internal sales data.

● Build versioned and upsert-friendly ETL pipelines using Apache Hudi or Iceberg.

● Create forecast-ready datasets with lagged, rolling, and trend features for revenue and occupancy modeling.

● Optimize Athena datasets with partitioning, CTAS queries, and metadata tagging.

● Implement S3 lifecycle policies, intelligent file partitioning, and audit logging.

● Build reusable transformation logic using dbt-core or PySpark to support KPIs and time series outputs.

● Integrate robust data quality checks using custom logs, AWS CloudWatch, or other DQ tooling.

● Design and manage a forecast feature registry with metrics versioning and traceability.

● Collaborate with BI and business teams to finalize schema design and deliverables for dashboard consumption.

Essential Skills Job

● Deep hands-on experience with AWS Glue, Athena, S3, Step Functions, and Glue Data Catalog.

● Strong command over PySpark, dbt-core, CTAS query optimization, and partition strategies.

● Working knowledge of Apache Hudi, Iceberg, or Delta Lake for versioned ingestion.

● Experience in S3 metadata tagging and scalable data lake design patterns.

● Expertise in feature engineering and forecasting dataset preparation (lags, trends, windows).

● Proficiency in Git-based workflows (Bitbucket), CI/CD, and deployment automation.

● Strong understanding of time series KPIs, such as revenue forecasts, occupancy trends, or demand volatility.

● Data observability best practices including field-level logging, anomaly alerts, and classification tagging.

Mandatory Skills - Iceberg , QA Frameworks , Governance , Lineage , Open Metadata , Great Expectations, Atlan

Regards

Sahiba

8296043355

  • Senior Data Architect

    24 hours ago

    Meerut, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job OpportunityWe are seeking a senior data engineer to play a pivotal part in designing, developing, and optimizing data ingestion, transformation, and governance frameworks.This role will enable real-time and batch data analytics across our data platform.Deep expertise in building scalable data pipelines is required.Implementing enterprise-grade data...

  • Senior Data Engineer

    2 days ago

    Meerut, Uttar Pradesh, India Mitra AI Full time

    We are seeking a candidate who is comfortable working on B shift (3PM - 11PM IST).JOB SPECIFIC DUTIES AND RESPONSIBILITIESDevelop and maintain ETL pipelines using FiveTran, DBT Cloud, and custom frameworksImplement client-specific data transformations for HighTouch integrationSupport data ingestion and quality checks during client onboardingOptimize...

  • Chief Data Engineering Specialist

    5 hours ago

    Meerut, Uttar Pradesh, India beBeeDataArchitecture Full time ₹ 6,00,000 - ₹ 12,00,000

    Senior Data Architect PositionJob Summary:The ideal candidate will have 5+ years of experience in designing and implementing data architecture solutions, with a strong background in real-time and close-to-real-time data integration using Kafka, MS Event Hubs, etc.Experience in developing Business Intelligence Dashboards and Data Models using PowerBI, Azure...

  • Senior Data Engineering Specialist

    4 days ago

    Meerut, Uttar Pradesh, India beBeeDataEngineer Full time

    Job Title: Senior Data Engineer (IDMC-CDI)Primary Skills: SQL, IDMC (CDI), ETL Concepts BaseLocation: RemoteShift: US EST (5 PM to 2 AM IST)Experience: 5-12 YearsAbout the Role:We are seeking a highly skilled and experienced Informatica IDMC ETL Developer with a deep understanding of ETL design, data pipeline development, and Informatica Cloud Data...

  • Senior Data Extraction Engineer

    24 hours ago

    Meerut, Uttar Pradesh, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Advanced Web Scraping Specialist">We are seeking a highly skilled and experienced web scraping professional to drive our data extraction operations forward. The ideal candidate will have a minimum of 4 years of hands-on experience in IT scraping, with at least 2 years leading a team of 5+ developers.">About the Role:">Design and develop scalable data...

  • Data Engineering Professional

    1 day ago

    Meerut, Uttar Pradesh, India beBeedataengineering Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Data Engineering PositionWe are seeking a skilled Data Engineer to fill this challenging role.

  • Senior AI Engineer

    4 hours ago

    Meerut, Uttar Pradesh, India beBeeArtificial Full time ₹ 1,80,00,000 - ₹ 2,30,00,000

    AI Engineering OpportunityOverview:As a senior AI engineer, you will be responsible for designing and implementing scalable data intelligence platforms that integrate structured and unstructured data sources.Key Responsibilities:Design and implement data pipelines, feature stores, and model monitoring systems to optimize AI performance.Develop and fine-tune...

  • Senior Microservices Engineer

    5 hours ago

    Meerut, Uttar Pradesh, India beBeeEngineering Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job Title:Senior Microservices Engineer Role DescriptionWe are seeking an experienced and skilled Senior Microservices Engineer to join our R&D group.This is a unique opportunity to be part of a dynamic team building the enablement platform for current and next-generation media experiences in personalization, content interaction, and social interaction.Key...

  • Strategic Cloud Data Architect

    7 hours ago

    Meerut, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    **Senior Databricks Data Engineer Position OverviewJob DescriptionThis is a senior-level position in our data engineering team, focusing on the design, development, and maintenance of scalable and high-performance data pipelines using Databricks and PySpark.The ideal candidate will be well-versed in PySpark, CI/CD pipelines, and Terraform for infrastructure...

  • Senior Data Engineer

    4 hours ago

    Meerut, Uttar Pradesh, India beBeeDataEngineer Full time ₹ 1,20,00,000 - ₹ 1,80,00,000

    About the RoleAt a leading healthcare technology company, we seek an experienced professional to design and build scalable data pipelines that process clinical encounter data and deliver validated information to machine learning systems.This is a critical role where you will:Design, build, and operate event-driven data pipelines that support real-time...