Senior Data Engineer

2 weeks ago

Ahmedabad, Gujarat, India Talent Corner HR Services Pvt Ltd Full time

We are looking for an experienced Senior Data Engineer to lead the development of scalable AWS-native data lake pipelines with a strong focus on time series forecasting and upsert-ready architectures. This role requires end-to-end ownership of the data lifecycle, from ingestion to partitioning, versioning, and BI delivery. The ideal candidate must be highly proficient in AWS data services, PySpark, versioned storage formats like Apache Hudi/Iceberg, and must understand the nuances of data quality and observability in large-scale analytics systems.

Responsibilities

● Design and implement data lake zoning (Raw → Clean → Modeled) using Amazon S3, AWS Glue, and Athena.

● Ingest structured and unstructured datasets including POS, USDA, Circana, and internal sales data.

● Build versioned and upsert-friendly ETL pipelines using Apache Hudi or Iceberg.

● Create forecast-ready datasets with lagged, rolling, and trend features for revenue and occupancy modeling.

● Optimize Athena datasets with partitioning, CTAS queries, and metadata tagging.

● Implement S3 lifecycle policies, intelligent file partitioning, and audit logging.

● Build reusable transformation logic using dbt-core or PySpark to support KPIs and time series outputs.

● Integrate robust data quality checks using custom logs, AWS CloudWatch, or other DQ tooling.

● Design and manage a forecast feature registry with metrics versioning and traceability.

● Collaborate with BI and business teams to finalize schema design and deliverables for dashboard consumption.

Essential Skills Job

● Deep hands-on experience with AWS Glue, Athena, S3, Step Functions, and Glue Data Catalog.

● Strong command over PySpark, dbt-core, CTAS query optimization, and partition strategies.

● Working knowledge of Apache Hudi, Iceberg, or Delta Lake for versioned ingestion.

● Experience in S3 metadata tagging and scalable data lake design patterns.

● Expertise in feature engineering and forecasting dataset preparation (lags, trends, windows).

● Proficiency in Git-based workflows (Bitbucket), CI/CD, and deployment automation.

● Strong understanding of time series KPIs, such as revenue forecasts, occupancy trends, or demand volatility.

● Data observability best practices including field-level logging, anomaly alerts, and classification tagging.

Mandatory Skills - Iceberg , QA Frameworks , Governance , Lineage , Open Metadata , Great Expectations, Atlan

Regards

Sahiba

8296043355

  • Senior Data Engineer

    4 days ago

    Ahmedabad, Gujarat, India eInfochips (An Arrow Company) Full time

    Role: Senior Data EngineerYears of Experience: 5-8 YearsLocation: Ahmedabad, Pune, IndoreNotice Period: Immediate to 45 Days MaxJob Description: We are seeking a highly motivated and enthusiastic Senior Data Engineer with 5- 8 years of experience to join our dynamic team. The ideal candidate will have a strong background in Data Engineering/Data analytics...

  • Senior Data Engineer

    2 days ago

    Ahmedabad, Gujarat, India eInfochips (An Arrow Company) Full time

    Role: Senior Data EngineerYears of Experience: 5-8 YearsLocation: Ahmedabad, Pune, IndoreNotice Period: Immediate to 45 Days MaxJob Description: We are seeking a highly motivated and enthusiastic Senior Data Engineer with 5- 8 years of experience to join our dynamic team. The ideal candidate will have a strong background in Data Engineering/Data analytics...

  • Senior Software Engineer

    4 weeks ago

    Ahmedabad, Gujarat, India Crest Data Full time

    Job DescriptionWe are looking for a strong ReactJS Lead Engineers to join our engineering team in Ahmedabad. The primary responsibility will be to lead the design and develop Enterprise Software for our Global Fortune 500 clients in Data Center and Big Data segments.Key Responsibilities -- Responsible for providing expertise in software development life...

  • Senior Big Data Engineer

    1 day ago

    Ahmedabad, Gujarat, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 30,00,000

    Job Title: Senior Big Data Engineer">We are seeking a highly skilled Senior Big Data Engineer to join our team.">About the Role:">This is an exciting opportunity for an experienced engineer to lead the development of large-scale data processing systems using Apache Spark, Python and AWS services.">Key Responsibilities:">- Design and implement scalable data...

  • Data Engineer

    4 weeks ago

    Ahmedabad, Gujarat, India Indsafri Full time

    Hiring Now: Senior Data Engineers (Banking Domain – 2 Positions)Location: RemoteExperience: 7+ Years | Banking sector experience preferred Type: Contract About the OpportunityWe're looking for highly skilled Senior Data Engineers to join a mission-critical banking project. This is your chance to work on cutting-edge data pipelines, streaming platforms,...

  • Senior Data Engineer

    4 weeks ago

    Ahmedabad, Gujarat, India ConsultMeIn Full time

    PLEASE READ THE JOB DESCRIPTION BEFORE APPLYING. Mandatory – Active Python Coding Experience of 5+ Years.Role: Data Engineering ManagerExperience: 6 – 12 YearsCTC Budget: 20 – 35 LPA + 20% VariableLocation: Ahmedabad (work from office)Are you passionate about building modern data platforms and leading high-performing engineering teams? We're seeking a...

  • Senior Data Engineer

    4 weeks ago

    Ahmedabad, Gujarat, India ConsultMeIn Private Limited Full time

    Mandatory Active Python Coding Experience of 5+ Years.Role : Data Engineering Manager.Experience : 6 12 Years.Location : Ahmedabad (work from office).Are you passionate about building modern data platforms and leading high-performing engineering teams? We're seeking a Data Engineering Manager to drive the design, development, and deployment of scalable...

  • Data Engineering Lead

    3 days ago

    Ahmedabad, Gujarat, India beBeeData Full time ₹ 9,00,000 - ₹ 12,00,000

    We are seeking a highly motivated Senior Data Engineer with 5-8 years of experience to lead our data engineering initiatives. The ideal candidate will have a strong background in data analytics and a passion for leveraging data to drive business insights.Bachelor's degree in Computer Science, Data Science, or a related field.6-8 years of experience in...

  • Senior Data Engineering Lead

    2 days ago

    Ahmedabad, Gujarat, India beBeeDataEngineering Full time ₹ 90,00,000 - ₹ 1,75,00,000

    Job Title: Senior Data Engineering LeadWe are seeking an accomplished and experienced professional to spearhead our data integration efforts. As a trusted partner to top-tier clients, we implement cutting-edge data management solutions using industry-leading tools.About the Role:This is an exceptional opportunity for a highly skilled Informatica IDMC ETL...

  • Senior Data Engineer

    2 weeks ago

    Ahmedabad, Gujarat, India RK HR Management Private Limited Full time

    Key Responsibilities • Design, deliver and maintain the appropriate data solution to provide the correct data for analytical development to address key issues within the organization. • Collaborate with a cross-functional team to deliver high quality results (such as Tableau developers and analysts). • Develop and operate modern data architecture...