Senior Data Engineer

2 weeks ago

Gurugram, India Gemini Solutions Pvt Ltd Full time

We are seeking an 10+ yrs experienced ETL Tech Lead to oversee and drive data migration, integration, and pipeline development projects. This role involves leading the migration of on-premise databases (Oracle, SQL Server) to AWS cloud platforms such as Redshift and S3, ensuring seamless integration across diverse systems. The ideal candidate will design and optimize robust ETL/ELT workflows, leveraging tools like Informatica, AWS Glue, and custom Python solutions, while maintaining data quality and governance. Additionally, the role requires expertise in building scalable data architectures using AWS services, managing automated workflows with Apache Airflow, and mentoring junior engineers. A strong background in SQL, Python, and AWS cloud infrastructure is essential, with a focus on driving high-performance data operations aligned with business objectives.


About the Role


  • Data Migration & Integration: Lead data migration projects to move on-premise data (Oracle, SQL Server) to AWS cloud-based solutions (Redshift, S3, RDS). Coordinate end-to-end processes for seamless data transfers and integrations across platforms.
  • Data Ingestion & Pipelines: Design, implement, and maintain robust, scalable data ingestion pipelines to integrate structured and unstructured data from various sources into cloud-based platforms like AWS. Use tools like Informatica, Apache Airflow, or custom Python solutions.
  • Cloud Infrastructure (AWS): Develop cloud-native data architectures using AWS services (e.g., Redshift, S3, Glue, Lambda, RDS, and Athena). Optimize data storage, processing, and retrieval in AWS to meet performance and cost efficiency goals.
  • ETL/ELT Development: Build and optimize ETL/ELT processes using tools like Informatica, AWS Glue, AWS Transfer Family and custom Python-based solutions. Ensure data flows are automated, efficient, and scalable.
  • Automation & Workflow Orchestration: Implement and manage workflow orchestration with Apache Airflow to automate and schedule ETL tasks, ensuring reliable data delivery to target systems.
  • Collaborate with Stakeholders: Work closely with business users, analysts, and other engineering teams to understand data requirements, propose data solutions, and ensure alignment with business goals. Translate business needs into technical solutions.
  • Data Quality & Governance: Ensure data quality, integrity, and compliance with governance policies. Implement monitoring and logging to ensure pipeline health and detect anomalies or issues proactively.
  • Mentoring & Leadership: Mentor and guide junior data engineers, helping them grow in their technical skills and best practices. Promote a culture of continuous learning and high performance.
  • Performance Tuning & Optimization: Continuously monitor and optimize data pipeline performance, troubleshoot issues, and apply best practices for improving query performance and data processing times.


Skills (Must have) :


  • Experience with Data Migrations: Proven track record of handling large-scale data migrations, particularly from on-premise databases (e.g., Oracle, SQL Server) to cloud platforms like AWS.
  • Cloud Platforms (AWS): Strong hands-on experience with AWS services including S3, Redshift, RDS, Lambda, Glue, and Athena. Familiarity with AWs Transfer Family, AWS Data Pipeline and AWS DMS (Database Migration Service) is a plus.
  • ETL/ELT Tools: Extensive experience with ETL/ELT tools like Informatica, AWS Glue, or Apache NiFi, as well as custom Python scripts for data transformations and data workflows.
  • Python Programming: Advanced proficiency in Python, including experience with libraries such as Pandas, NumPy, and PySpark for data processing and transformations. Experience in writing efficient, scalable code is essential.
  • Apache Airflow: Proficiency in Apache Airflow for building, scheduling, and managing data workflows and pipeline automation.
  • SQL & Database Skills: Strong experience with relational databases like Oracle, SQL Server, and cloud databases like Amazon Redshift or RDS. Familiarity with writing complex SQL queries and optimizing them for large-scale data processing.
  • Data Modeling & Warehousing: Solid understanding of data modeling techniques (e.g., star schema, snowflake schema) and experience with data warehousing solutions, particularly in the cloud (AWS Redshift, Snowflake, etc.).
  • Data Integration: Experience integrating data from various sources (databases, APIs, flat files, etc.), and managing large-scale data ingestion pipelines.
  • Data Quality & Governance: Experience implementing data quality checks, error handling, and monitoring for data pipelines. Familiarity with data governance best practices for compliance and security.
  • Version Control & CI/CD: Experience with version control (e.g., Git) and CI/CD practices for managing code and automating data workflows. Familiarity with Jenkins, GitLab CI, or other tools is a plus.


Skills (Good to Have) :


  • Financial Services/Capital Markets Experience: Experience working with financial data, including market data, transaction data, risk data, or financial reporting and working with financial data vendors such as Bloomberg, Refinitiv, Morningstar, Barclays.
  • Big Data Technologies: Experience with Big Data tools and frameworks like Hadoop, Spark, or Kafka is a plus, particularly in a cloud-based environment (AWS EMR, AWS Kinesis, etc.).
  • Data Security & Compliance: Knowledge of best practices for securing data, particularly in financial services, including encryption, data masking, and managing sensitive information in cloud environments.
  • Leadership Experience: For Lead roles, experience managing or mentoring a team of data engineers, guiding architecture decisions, and driving process improvements.


Education and Certifications:


  • Bachelor’s degree in Computer Science, Engineering, Data Science, or a related field (or equivalent experience).
  • Certifications in AWS (e.g., AWS Certified Data Analytics – Specialty, AWS Certified Solutions Architect – Associate/Professional) are highly desirable.
  • Additional certifications in data engineering, databases, or cloud platforms would be beneficial.

  • Senior Data Engineer

    2 weeks ago

    Gurugram, India True Tech Professionals Full time

    Senior Data EngineerLocation: Gurgaon (Hybrid Mode)Experience: 5 years and above. Immediate Joiners Preferred (Notice Period: 7 to 10 days)We are seeking a skilled Senior Data Engineer to join our dynamic team. The ideal candidate should have a strong background in data engineering with expertise in Python, Pyspark, AWS Glue, Athena, Redshift, and SQL. The...

  • Senior Data Engineer

    2 weeks ago

    Gurugram, India True Tech Professionals Full time

    Senior Data EngineerLocation: Gurgaon (Hybrid Mode)Experience: 5 years and above. Immediate Joiners Preferred (Notice Period: 7 to 10 days)We are seeking a skilled Senior Data Engineer to join our dynamic team. The ideal candidate should have a strong background in data engineering with expertise in Python, Pyspark, AWS Glue, Athena, Redshift, and SQL. The...

  • Senior Data Engineer

    2 weeks ago

    Gurugram, India True Tech Professionals Full time

    Senior Data Engineer Location: Gurgaon (Hybrid Mode) Experience: 5 years and above. Immediate Joiners Preferred (Notice Period: 7 to 10 days) We are seeking a skilled Senior Data Engineer to join our dynamic team. The ideal candidate should have a strong background in data engineering with expertise in Python, Pyspark, AWS Glue, Athena, Redshift, and SQL....

  • Senior Data Engineer

    2 months ago

    Gurugram, India Invokhr Full time

    Position: Senior Data Engineer / Data Engineer Desired Experience: 3-8 years Salary: Best-in-industry You will act as a key member of the Data consulting team, working directly with the partners and senior stakeholders of the clients designing and implementing big data and analytics solutions. Communication and organisation skills are keys for this position,...

  • Senior Data Engineer

    2 months ago

    Gurugram, India Admin Looks Full time

    Urgently looking for the Senior Data Engineer for onsite at the location of GurgaonJob description:We are seeking a passionate, hands-on and experienced Senior Data Engineer to join our team. You will play a key role in implementing large scale Data platforms with our customers. The candidate will collaborate with internal and customer teams on multiple...

  • Senior Data Engineer

    2 months ago

    gurugram, India MakeMyTrip Full time

    Position: Senior Data Engineer IIExperience: Must have 4+ years of experienceAbout Role:We are looking for experienced Data engineers with excellent problem-solving skills to develop machine-learning powered Data Products design to enhance customer experiences.About us:Nurtured from the seed of a single great idea - to empower the traveler - MakeMyTrip went...

  • Senior Data Engineer

    4 months ago

    Gurugram, India Invokhr Full time

    Desired Experience: 3-8 years Salary: Best-in-industry Location: Gurgaon ( 5 days onsite) Overview: You will act as a key member of the Data consulting team, working directly with the partners and senior stakeholders of the clients designing and implementing big data and analytics solutions. Communication and organisation skills are keys for this position,...

  • Senior Data Engineer

    4 weeks ago

    gurugram, India NCR Voyix Full time

    About NCR Voyix CorporationNCR Voyix (NYSE: VYX) is a global technology company leading how the world connects, interacts and transacts with business. NCR Voyix’s assisted and self-service solutions and comprehensive support services address the needs of retail & restaurants organizations in more than 100 countries. NCR Voyix (www.ncr.com) is headquartered...

  • Senior Data Engineer

    4 weeks ago

    Gurugram, India Partech Full time

    Hi there! Have you noticed that the restaurant industry is changing?We are PAR and our purpose is:To deliver solutions that connect people to the restaurants, meals, and moments they love.At PAR Technology, our relentless drive for innovation and unwavering commitment to customer success are at the heart of everything we do. We lead the restaurant and retail...

  • Senior Data Engineer

    3 months ago

    gurugram, India MakeMyTrip Full time

    Position: Senior Data Engineer IIExperience: Must have 4+ years of experienceAbout Role:We are looking for experienced Data engineers with excellent problem-solving skills to develop machine-learning powered Data Products design to enhance customer experiences.About us:Nurtured from the seed of a single great idea - to empower the traveler - MakeMyTrip went...

  • Senior Data Engineer

    3 months ago

    gurugram, India MakeMyTrip Full time

    Position: Senior Data Engineer II Experience: Must have 4+ years of experience About Role: We are looking for experienced Data engineers with excellent problem-solving skills to develop machine-learning powered Data Products design to enhance customer experiences. About us: Nurtured from the seed of a single great idea - to empower the traveler -...

  • Senior Data Engineer

    3 months ago

    gurugram, India MakeMyTrip Full time

    Position: Senior Data Engineer IIExperience: Must have 4+ years of experienceAbout Role:We are looking for experienced Data engineers with excellent problem-solving skills to develop machine-learning powered Data Products design to enhance customer experiences.About us:Nurtured from the seed of a single great idea - to empower the traveler - MakeMyTrip went...

  • Senior Data Engineer

    3 months ago

    Gurugram, India MakeMyTrip Full time

    Position: Senior Data Engineer IIExperience: Must have 4+ years of experienceAbout Role:We are looking for experienced Data engineers with excellent problem-solving skills to develop machine-learning powered Data Products design to enhance customer experiences.About us:Nurtured from the seed of a single great idea - to empower the traveler - MakeMyTrip went...

  • Senior Data Engineer

    3 months ago

    Gurugram, India MakeMyTrip Full time

    Position: Senior Data Engineer II Experience: Must have 4+ years of experience About Role: We are looking for experienced Data engineers with excellent problem-solving skills to develop machine-learning powered Data Products design to enhance customer experiences. About us: Nurtured from the seed of a single great idea - to empower the traveler -...

  • Senior Data Engineer

    3 months ago

    Gurugram, India MakeMyTrip Full time

    Position: Senior Data Engineer IIExperience: Must have 4+ years of experienceAbout Role:We are looking for experienced Data engineers with excellent problem-solving skills to develop machine-learning powered Data Products design to enhance customer experiences.About us:Nurtured from the seed of a single great idea - to empower the traveler - MakeMyTrip went...

  • Senior Data Engineer

    2 weeks ago

    Gurugram, India True Tech Professionals Full time

    Role : Senior Data Engineer. Location : Gurgaon (Hybrid Mode). Experience : 5 years and above. Immediate Joiners Preferred (Notice Period: 7 to 10 days). We are seeking a skilled Senior Data Engineer to join our dynamic team. The ideal candidate should have a strong background in data engineering with expertise in Python, Pyspark, AWS Glue, Athena, Redshift,...

  • Senior Data Engineer

    1 month ago

    gurugram, India NCR Voyix Full time

    About NCR Voyix Corporation NCR Voyix (NYSE: VYX) is a global technology company leading how the world connects, interacts and transacts with business. NCR Voyix’s assisted and self-service solutions and comprehensive support services address the needs of retail & restaurants organizations in more than 100 countries. NCR Voyix ( ) is headquartered in...

  • Senior Data Engineer

    1 month ago

    Gurugram, India NCR Voyix Full time

    About NCR Voyix CorporationNCR Voyix (NYSE: VYX) is a global technology company leading how the world connects, interacts and transacts with business. NCR Voyix’s assisted and self-service solutions and comprehensive support services address the needs of retail & restaurants organizations in more than 100 countries. NCR Voyix (www.ncr.com) is headquartered...

  • Senior Data Engineer

    5 months ago

    Gurugram, India Circle K Full time

    JOB DESCRIPTION Job Description Alimentation Couche-Tard Inc., (ACT) is a global Fortune 200 company. A leader in the convenience store and fuel space with over 16,700 stores in 31 countries, serving more than 9 million customers each day. It is an exciting time to be a part of the growing Data Engineering team at Circle K. We are driving a...

  • Senior Data Engineer

    6 months ago

    Gurugram, India Expedia Group Full time

    Senior Data Engineer We are looking for a highly motivated, self-driven , talented and experienced Data Engineer to join our dynamic team and drive the development and maintenance of our data platform. You have at least 9 years of experience as a Data Engineer. An independent thinker, who works around problems and who isn’t shy of trying new technologies....