Senior Data Engineer

5 days ago

Delhi Division, India Markovate Full time

Job Description :

We are looking for an experienced Senior Data Engineer to lead the development of scalable AWS-native data lake pipelines, with a strong focus on time series forecasting, upsert-ready architectures, and enterprise-grade data governance.

This role demands end-to-end ownership of the data lifecycle from ingestion to partitioning, versioning, QA, lineage tracking, and BI delivery.

The ideal candidate will be highly proficient in AWS data services, PySpark, and versioned storage formats such as Apache Hudi or Iceberg.

A strong understanding of data quality, observability, governance, and metadata management in large-scale analytical systems is & Responsibilities :


- Design and implement data lake zoning (Raw Clean Modeled) using Amazon S3, AWS Glue, and Athena.


- Ingest structured and unstructured datasets including POS, USDA, Circana, and internal sales data.

- Build versioned and upsert-ready ETL pipelines using Apache Hudi or Iceberg.

- Create forecast-ready datasets with lagged, rolling, and trend features for revenue and occupancy modeling.

- Optimize Athena datasets with partitioning, CTAS queries, and S3 metadata tagging.

- Implement S3 lifecycle policies, intelligent file partitioning, and audit logging for performance and compliance.

- Build reusable transformation logic using dbt-core or PySpark to support KPIs and time series outputs.

- Integrate data quality frameworks such as Great Expectations, custom logs, and AWS CloudWatch for field-level validation and anomaly detection.

- Apply data governance practices using tools like OpenMetadata or Atlan, enabling lineage tracking, data cataloging, and impact analysis.

- Establish QA automation frameworks for pipeline validation, data regression testing, and UAT handoff.

- Collaborate with BI, QA, and business teams to finalize schema design and deliverables for dashboard consumption.

- Ensure compliance with enterprise data governance policies and enable discovery and collaboration through metadata Candidate Profile :

- 9-12 years of experience in data engineering.

- Deep hands-on experience with AWS Glue, Athena, S3, Step Functions, and Glue, Data Catalog.

- Strong command over PySpark, dbt-core, CTAS query optimization, and advanced partition strategies.

- Proven experience with versioned ingestion using Apache Hudi, Iceberg, or Delta Lake.

- Experience in data lineage, metadata tagging, and governance tooling using OpenMetadata, Atlan, or similar platforms.

- Proficiency in feature engineering for time series forecasting (lags, rolling windows, trends).

- Expertise in Git-based workflows, CI/CD, and deployment automation (Bitbucket or similar).

- Strong understanding of time series KPIs: revenue forecasts, occupancy trends, demand volatility, etc.

- Knowledge of statistical forecasting frameworks (e.g., Prophet, GluonTS, Scikit-learn).

- Experience with Superset or Streamlit for QA visualization and UAT testing.

- Experience building data QA frameworks and embedding data validation checks at each stage of the ETL lifecycle.

- Independent thinker capable of designing systems that scale with evolving business logic and compliance requirements.

- Excellent communication skills for collaboration with BI, QA, data governance, and business stakeholders.

- High attention to detail, especially around data accuracy, documentation, traceability, and auditability.
(ref:hirist.tech)

  • Senior Data Engineer

    3 weeks ago

    Delhi, Delhi, India Aptus Data Labs Full time

    Job Title : Senior Data EngineerLocation : : 58 YearsEmployment Type : Full-TimeAbout the Role :Aptus Data Labs is looking for a talented and proactive Senior Data Engineer to help build the backbone of our enterprise data and AI initiatives. Youll work on modern data lake architectures and high-performance pipelines in AWS, enabling real-time insights and...

  • Senior Data Engineer

    5 days ago

    Delhi Division, India Findem Full time

    Job Description :Key Responsibilities :- Design, develop, and maintain scalable data pipelines and ETL processes to support data integration from multiple sources.- Perform data modeling for both structured and semi-structured datasets.- Implement and manage data warehousing solutions to store and organize large volumes of data efficiently.- Optimize ETL...

  • Tatras Data

    1 week ago

    Delhi Division, India Tatras Data Services Pvt Ltd Full time

    About the job :Job Title :Automation QA EngineerAbout Us :We are building high-quality, scalable, and intelligent platforms, and quality is central to everything we do. We are looking for an Automation QA Engineer who doesn't just test software, but helps build it betterwith a mindset to break things early, code automation robustly, and ensure...

  • Kroll - Senior Consultant - Data Strategy & Engineering

    7 days ago

    Delhi Division, India Kroll LLC Full time

    In a world of disruption and increasingly complex business challenges, our professionals bring truth into focus with the Kroll Lens. Our sharp analytical skills, paired with the latest technology, allow us to give our clients clarity-not just answers-in all areas of business. We embrace diverse backgrounds and global perspectives, and we cultivate diversity...

  • Senior Data Engineer

    3 weeks ago

    Delhi, Delhi, India Vikash Technologies Full time

    Senior Data EngineerSenior Data Engineer with strong expertise in SQL, Python, Azure Synapse, Azure Data Factory, Snowflake, and Databricks. The ideal candidate should have a solid understanding of SQL (DDL, DML, query optimization) and ETL pipelines while demonstrating a learning mindset to adapt to evolving Responsibilities : - Design and implement...

  • Kroll - Consultant - Data Engineering

    5 days ago

    Delhi Division, India Kroll LLC Full time

    In a world of disruption and increasingly complex business challenges, our professionals bring truth into focus with the Kroll Lens. Our sharp analytical skills, paired with the latest technology, allow us to give our clients claritynot just answersin all areas of business. We embrace diverse backgrounds and global perspectives, and we cultivate diversity by...

  • Senior Data Engineer

    1 week ago

    Delhi, Delhi, India Hunarstreet Technologies Pvt Ltd Full time

    Role : Senior Data Engineer - Remote (Pan India)Are you passionate about building scalable data pipelines and optimizing data architecture? Were looking for an experienced Senior Data Engineer (10+ years) to join our team and play a key role in shaping next-gen data systems.What youll do :- Design & develop robust data pipelines (ETL) using the latest Big...

  • Senior Engineer, Data Engineering

    4 days ago

    Delhi, India Dhurin Full time

    About the Company Dhurin is a fastgrowing data science firm serving financial services industry. We work with banks, NBFCs, fintech and other financial institutions. Our work is in the space of risk, fraud, collections, marketing, sales, and operations through it we directly influence how financial institutions make smarter decisions. Dhurin is ‘the...

  • Software Engineering Head

    1 day ago

    Delhi Division, India Ixceed Solutions Full time

    Role : Head of Software Engineering.Location : Delhi NCR (Hybrid).Type : Permanent.Job Description : Lead Technology Architect / Head of Engineering. We are seeking an experienced Lead Technology Architect to define and deliver innovative technology solutions across our products and platforms. This role blends hands-on engineering with strategic leadership -...

  • AI/ML Engineer

    1 day ago

    Delhi Division, India MantraCare Full time

    Key Responsibilities :- Design, train, and fine-tune machine learning and deep learning models, with a focus on LLMs for conversational and generative tasks.- Perform feature engineering, data preprocessing, and model evaluation to ensure high accuracy, robustness, and scalability.- Optimize models for real-world performance and production deployment.- Work...