Senior Data Engineer

3 weeks ago

Delhi, Delhi, India Markovate Full time

Job Description :

We are looking for an experienced Senior Data Engineer to lead the development of scalable AWS-native data lake pipelines, with a strong focus on time series forecasting, upsert-ready architectures, and enterprise-grade data governance.

This role demands end-to-end ownership of the data lifecycle from ingestion to partitioning, versioning, QA, lineage tracking, and BI delivery.

The ideal candidate will be highly proficient in AWS data services, PySpark, and versioned storage formats such as Apache Hudi or Iceberg.

A strong understanding of data quality, observability, governance, and metadata management in large-scale analytical systems is & Responsibilities :

- Design and implement data lake zoning (Raw Clean Modeled) using Amazon S3, AWS Glue, and Athena.

- Ingest structured and unstructured datasets including POS, USDA, Circana, and internal sales data.

- Build versioned and upsert-ready ETL pipelines using Apache Hudi or Iceberg.

- Create forecast-ready datasets with lagged, rolling, and trend features for revenue and occupancy modeling.

- Optimize Athena datasets with partitioning, CTAS queries, and S3 metadata tagging.

- Implement S3 lifecycle policies, intelligent file partitioning, and audit logging for performance and compliance.

- Build reusable transformation logic using dbt-core or PySpark to support KPIs and time series outputs.

- Integrate data quality frameworks such as Great Expectations, custom logs, and AWS CloudWatch for field-level validation and anomaly detection.

- Apply data governance practices using tools like OpenMetadata or Atlan, enabling lineage tracking, data cataloging, and impact analysis.

- Establish QA automation frameworks for pipeline validation, data regression testing, and UAT handoff.

- Collaborate with BI, QA, and business teams to finalize schema design and deliverables for dashboard consumption.

- Ensure compliance with enterprise data governance policies and enable discovery and collaboration through metadata Candidate Profile :

- 9-12 years of experience in data engineering.

- Deep hands-on experience with AWS Glue, Athena, S3, Step Functions, and Glue, Data Catalog.

- Strong command over PySpark, dbt-core, CTAS query optimization, and advanced partition strategies.

- Proven experience with versioned ingestion using Apache Hudi, Iceberg, or Delta Lake.

- Experience in data lineage, metadata tagging, and governance tooling using OpenMetadata, Atlan, or similar platforms.

- Proficiency in feature engineering for time series forecasting (lags, rolling windows, trends).

- Expertise in Git-based workflows, CI/CD, and deployment automation (Bitbucket or similar).

- Strong understanding of time series KPIs: revenue forecasts, occupancy trends, demand volatility, etc.

- Knowledge of statistical forecasting frameworks (e.g., Prophet, GluonTS, Scikit-learn).

- Experience with Superset or Streamlit for QA visualization and UAT testing.

- Experience building data QA frameworks and embedding data validation checks at each stage of the ETL lifecycle.

- Independent thinker capable of designing systems that scale with evolving business logic and compliance requirements.

- Excellent communication skills for collaboration with BI, QA, data governance, and business stakeholders.

- High attention to detail, especially around data accuracy, documentation, traceability, and auditability. (ref:hirist.tech)

  • Senior Data Engineer

    3 weeks ago

    Delhi, Delhi, India SWAN Full time

    Job Description : Role : Senior Data EngineerLocation : Doha, QatarDuration : Full-time / 12 Months (Subcon)Experience : 8 - 12 YearsWorking Days : SundayThursday (8 hrs/day)Languages : English (Mandatory), Arabic (Preferred)Company Overview : Quantiphi is an award-winning Data Science and Machine Learning company and a global launch partner for Google in...

  • Data Engineer

    3 days ago

    Delhi, Delhi, India Marktine Technology Solutions Pvt Ltd Full time

    Position: Senior Data EngineerExp: 12-15 YearsLocation: RemoteWe are seeking a Senior Data Engineer. The ideal candidate will be based in India and work remotely. This role requires a blend of design and implementation expertise.Key Responsibilities:Design and bootstrap data solutions, including setting up Snowflake for a large, complex conglomerate.Utilize...

  • Senior Salesforce Qa Automation Engineer

    4 weeks ago

    Delhi, Delhi, India NTT DATA Full time

    Req ID 322395 NTT DATA strives to hire exceptional innovative and passionate individuals who want to grow with us If you want to be part of an inclusive adaptable and forward-thinking organization apply now We are currently seeking a Senior Salesforce QA Automation Engineer to join our team in Delhi Delhi IN-DL India IN Salesforce ...

  • Senior Data Engineer

    3 weeks ago

    Delhi, Delhi, India Hunarstreet Technologies Full time

    Job Title : Senior Data EngineerLocation : Remote (Pan India)Shift Timings : 2 : 00 PM - 11 : 00 PM ISTEmployment Type : Full-Time, PermanentReporting To : CEO or assigned LeadResponsibilities :- Design and build parts of data pipeline architecture for ETL from diverse data sources using latest Big Data technologies.- Identify, design, and implement...

  • Senior Azure Data Engineer

    4 weeks ago

    Delhi, Delhi, India Shadow Placements Full time

    Job Title : Senior Azure Data EngineerLocation : RemoteTimings : UK Shift & US Shift timingsiO Associates is seeking a Senior Data Engineer skilled in building and managing robust ETL/data pipelines across diverse data sources and formats using Databricks, Python, and Scala.Strong expertise in data modelling, data warehousing, and CI/CD practices.Proficient...

  • Data Engineer

    3 days ago

    Delhi, Delhi, India Indsafri Full time

    Hiring Now: Senior Data Engineers (Banking Domain – 2 Positions)Location: RemoteExperience: 7+ Years | Banking sector experience preferredType: ContractAbout the OpportunityWe're looking for highly skilled Senior Data Engineers to join a mission-critical banking project. This is your chance to work on cutting-edge data pipelines, streaming...

  • Senior AI Data Engineer

    4 weeks ago

    Delhi, Delhi, India Teamified Full time

    About the job :We focus on uplifting the way organisations build and manage their remote teams through great working relationships, trust, integrity, culture and hiring processes. Additional to this we are building our own technology product offerings. We strive to deliver the best outcomes for our customers, our partners and our people.We are seeking a...

  • Senior Azure Data Engineer

    2 weeks ago

    Delhi, Delhi, India Programmers Full time

    Job Title: Senior Azure Data EngineerJob Location: RemoteSkills Required: Power BI, Synapse, Azure Data Factory, Azure Databricks, and ADLS Gen2, PySpark, SQLShift Timings: General ShiftExperience Range: 12–15 YearsJob Summary:We are seeking a highly skilled and experienced Senior Azure Data Engineer to join our team. The ideal candidate will have deep...

  • Senior Big Data Engineer

    4 weeks ago

    Delhi, Delhi, India huns info pvt ltd Full time

    Senior Data :- Skilled and experienced Sr. Data Engineer with proven experience in Spark, Kafka, Hadoop, Data Management.Skills :- Proficiency in programming languages Scala, Java, Python.- Experience with design and development of ELT workflows and data pipelines for high-volume data systems e.g. Datastage, Informatica, Talend, etc.- Experience in Data...

  • Senior Data Engineer – GCP

    4 weeks ago

    Delhi, Delhi, India Xebia Full time

    We're Hiring: Senior Data Engineer – GCP | Databricks | E-commerce DomainWork Locations : Chennai | Bangalore | Hyderabad | Gurugram | Jaipur | Pune | BhopalExperience : 7–8+ YearsShift Timing : 2 PM – 10 PM ISTWork Mode : Hybrid – 3 days/week from officeOnly Immediate Joiners (0–15 Days Notice)Are you a seasoned Data Engineer passionate about...