Senior Data Engineer

2 weeks ago

India Talent Corner HR Services Pvt Ltd Full time ₹ 1,50,00,000 - ₹ 2,00,00,000 per year

We are looking for an experienced Senior Data Engineer to lead the development of scalable AWS-native data lake pipelines with a strong focus on time series forecasting and upsert-ready architectures. This role requires end-to-end ownership of the data lifecycle, from ingestion to partitioning, versioning, and BI delivery. The ideal candidate must be highly proficient in AWS data services, PySpark, versioned storage formats like Apache Hudi/Iceberg, and must understand the nuances of data quality and observability in large-scale analytics systems.

Responsibilities

● Design and implement data lake zoning (Raw → Clean → Modeled) using Amazon S3, AWS Glue, and Athena.

● Ingest structured and unstructured datasets including POS, USDA, Circana, and internal sales data.

● Build versioned and upsert-friendly ETL pipelines using Apache Hudi or Iceberg.

● Create forecast-ready datasets with lagged, rolling, and trend features for revenue and occupancy modeling.

● Optimize Athena datasets with partitioning, CTAS queries, and metadata tagging.

● Implement S3 lifecycle policies, intelligent file partitioning, and audit logging.

● Build reusable transformation logic using dbt-core or PySpark to support KPIs and time series outputs.

● Integrate robust data quality checks using custom logs, AWS CloudWatch, or other DQ tooling.

● Design and manage a forecast feature registry with metrics versioning and traceability.

● Collaborate with BI and business teams to finalize schema design and deliverables for dashboard consumption.

Essential Skills Job

● Deep hands-on experience with AWS Glue, Athena, S3, Step Functions, and Glue Data Catalog.

● Strong command over PySpark, dbt-core, CTAS query optimization, and partition strategies.

● Working knowledge of Apache Hudi, Iceberg, or Delta Lake for versioned ingestion.

● Experience in S3 metadata tagging and scalable data lake design patterns.

● Expertise in feature engineering and forecasting dataset preparation (lags, trends, windows).

● Proficiency in Git-based workflows (Bitbucket), CI/CD, and deployment automation.

● Strong understanding of time series KPIs, such as revenue forecasts, occupancy trends, or demand volatility.

● Data observability best practices including field-level logging, anomaly alerts, and classification tagging.

Mandatory Skills -
Iceberg , QA Frameworks , Governance , Lineage , Open Metadata , Great Expectations, Atlan

Regards

Sahiba

  • Senior Associate Data Engineer

    3 days ago

    Bengaluru, India NTT DATA, Inc. Full time

    Job Description Make an impact with NTT DATA Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion its a place where you can grow, belong and thrive. Your day at NTT...

  • Senior GenAI Engineers

    3 days ago

    Pune, India NTT Data Full time

    Job Description NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Senior GenAI Engineers to join our team in Pune, Mahrshtra (IN-MH), India (IN). Location: | Experience: 8+...

  • Senior GenAI Engineers

    3 days ago

    Mumbai, India NTT Data Full time

    Job Description NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Senior GenAI Engineers to join our team in Mumbai, Mahrshtra (IN-MH), India (IN). Location: | Experience: 8+...

  • Senior Azure Stack Engineer

    2 weeks ago

    Remote, India NTT Data Full time

    Job DescriptionNTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Senior Azure Stack Engineer to join our team in Remote, Uttar Pradesh (IN-UP), India (IN).Job Title: Senior Azure Stack...

  • Senior Engineer, Data Engineering

    1 week ago

    India People Prime Worldwide Full time

    About Client: Our Client is a multinational IT services and consulting company headquartered in USA, With revenues 19.7 Billion USD, with Global work force of 3,50,000 and Listed in NASDAQ, It is one of the leading IT services firms globally, known for its work in digital transformation, technology consulting, and business process outsourcing, Business...

  • Senior Data Engineer

    2 weeks ago

    India ADP Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Senior Data Engineer with MLAbout the RoleWe are seeking an experienced Senior Data Engineer with 5-10 years of experience to join our data team. In this role, you will design, build, and maintain robust data pipelines that support both training and inference for our machine learning systems. You will work closely with data scientists to ensure seamless...

  • Senior Data Engineer

    2 weeks ago

    India Kinara Capital Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    1 About Company Kinara Capital is a FinTech NBFC dedicated to driving Financial Inclusion in the MSME sector. Our mission is to transform lives, livelihoods, and local economies by providing fast and flexible loans without property collateral to small business entrepreneurs. Led by a women-majority management team, Kinara Capital values diversity and...

  • Senior Data Engineer

    2 weeks ago

    India Recruber Full time US$ 1,04,000 - US$ 1,30,878 per year

    Position: Senior Data Engineer / Data EngineerDesired Experience: 5-8 yearsLocation - RemoteSalary: Best-in-industry You will act as a key member of the Data consulting team, working directly with the partners and senior stakeholders of the clients designing and implementing big data and analytics solutions. Communication and organisation skills are keys for...

  • Senior Data Engineer

    13 hours ago

    India Recruber Full time

    Position: Senior Data Engineer / Data Engineer Desired Experience: 5-8 years Location - Remote Salary: Best-in-industry You will act as a key member of the Data consulting team, working directly with the partners and senior stakeholders of the clients designing and implementing big data and analytics solutions. Communication and organisation skills are keys...

  • Senior Data Engineer

    4 days ago

    India Collabera Full time

    Senior Data Engineer Arrangement: Fully Remote Shift: US Time Zone (Night Shift)Industry: Financial Institution (FI) - Credit Union Job Summary Collabera is seeking a seasoned Senior Data Engineer for its leading financial customer to design and optimize data solutions that power our analytics, reporting, and Member 360 initiatives.This role is ideal for...