Senior Data Engineer(Python, AWS)

3 weeks ago

Gurgaon, Haryana, India S&P Global Market Intelligence Full time
Job Description

Responsibilities

- Design, develop, and maintain scalable ETL/ELT pipelines.
- Optimize and automate data ingestion, transformation, and storage processes.
- Work with structured and unstructured data sources, ensuring data quality and consistency.
- Develop and maintain data models, warehouses, and databases.
- Collaborate with cross-functional teams to support data-driven decision-making.
- Ensure data security, privacy, and compliance with industry standards.
- Troubleshoot and resolve data-related issues in a timely manner.
- Monitor and improve system performance, reliability, and scalability.
- Stay up-to-date with emerging data technologies and recommend improvements to our data architecture and engineering practices.

What you will need:

- Strong programming skills using python.
- 5+ years of experience in data engineering, ETL development, or a related role.
- Proficiency in SQL and experience with relational (PostgreSQL, MySQL, etc.) and NoSQL (DynamoDB, MongoDB etc) databases.
- Proficiency building data pipelines in AWS using services like Glue, Batch, Step Functions, Lambda, SQS, SNS, DynamoDb etc..
- Strong understanding of data modeling, data warehousing, and data governance principles.
- Should be capable of mentoring junior data engineers and assisting them with technical challenges.
- Familiarity with orchestration tools like Apache Airflow.
- Familiarity with containerization and orchestration (Docker, Kubernetes).
- Experience with version control systems (Git) and CI/CD pipelines.
- Excellent problem-solving skills and ability to work in a fast-paced environment.
- Excellent communication skills.
- Should be able to convert business queries into technical documentation.
- Hands-on experience with snowflake is a plus.
- Experience with big data technologies (Hadoop, Spark, Kafka, etc.) is a plus.
- Experience in GCP is a plus.

Education and Experience

- Bachelors degree in Computer Science, Information Systems, Information Technology, or a similar major or Certified Development Program
- 5+ years of experience building data pipelines using python & AWS.

  • AWS Data Engineer

    6 days ago

    Gurgaon, Haryana, India Workforce Next Pvt Ltd. Full time ₹ 22,00,000 - ₹ 48,00,000 per year

    Position: Senior AWS Data EngineerLocation: Remote (India)Job Type: Full-TimeStart Date: ImmediateAbout the Role:We are looking for a Senior AWS Data Engineer with extensive experience in architecting, building, and optimizing large-scale data pipelines on AWS. The ideal candidate will have deep expertise in AWS Glue, Amazon Redshift, and advanced data...

  • Python Data Engineer

    2 weeks ago

    Gurgaon, Haryana, India Digitrix Software LLP Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Location: Bangalore / Pune / Kolkata / Hyderabad / GurugramData Engineer4+ experiencePython, AWS Python (core language skill) -- Backend, Pandas, PySpark (DataFrame API), interacting with AWS (e.g., boto3 for S3, Glue, Lambda)Data Processing: Spark (PySpark), Glue, EMR AWS Core Services: S3, Glue, Athena, Lambda, Step Functions, EMRContainerization:...

  • AWS Data Engineer

    1 day ago

    Gurgaon, Haryana, India Incedo Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Role DescriptionAs a Technical Lead - Cloud Data Platform (AWS) at Incedo, you will be responsible for designing, deploying and maintaining cloud-based data platforms on the AWS platform. You will work with data engineers, data scientists and business analysts to understand business requirements and design scalable, reliable and cost-effective solutions that...

  • AWS Data Engineering Manager

    2 days ago

    Gurgaon, Haryana, India Recro Full time ₹ 15,00,000 - ₹ 30,00,000 per year

    We are seeking an experienced Senior AWS Data Engineer or Data Engineering Manager with 12 years of hands-on expertise in designing, building, and managing scalable data platforms on AWS. The ideal candidate will have deep knowledge of AWS services including Redshift, Glue, Lambda, Athena, S3, and strong proficiency in SQL and PySpark. Key...

  • AWS Data Engineer

    6 days ago

    Gurgaon, Haryana, India Deqode Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Profile :AWS Data EngineerMandate Skills :AWS + Databricks + Pyspark + SQL roleLocation: Bangalore /Pune /Hyderabad /Chennai /GurgaonNotice Period :ImmediateKey ResponsibilitiesDesign, build, and maintain scalable data pipelines to collect, process, and store data from multiple datasetsOptimize data storage solutions for performance, scalability, and...

  • Senior Data Engineer

    1 day ago

    Gurgaon, Haryana, India Freecharge Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job Title: Senior Engineering  (Data Engineering)Location: GurgaonExperience Required: 3+ YearsEmployment Type: Full-TimeDepartment: Data & AnalyticsAbout the RoleWe are looking for a passionate and experienced Big Data Engineer for driving the design anddevelopment of core platform frameworks with the mega knowledge of Spark, Python/Scalaand Kafka Streams...

  • Manager -Data Engineer (AWS, Python, Spark, Databricks for ETL - Agentic AI

    4 weeks ago

    Gurgaon, Haryana, India Genpact Full time

    Job DescriptionInviting applications for the role of Manager -Data Engineer (AWS, Python, Spark, Databricks for ETL - Agentic AIIn this role, you%27ll be part of Genpact%27s transformation under GenpactNext, as we lead the shift to Agentic AI Solutions-domain-specific, autonomous systems that redefine how we deliver value to clients. You%27ll help drive the...

  • Consultant -Data Engineer ( AWS, Python, Spark, Databricks for ETL -Agentic AI

    4 weeks ago

    Gurgaon, Haryana, India Genpact Full time

    Job DescriptionInviting applications for the role of Consultant -Data Engineer ( AWS, Python, Spark, Databricks for ETL -Agentic AIIn this role, you%27ll be part of Genpact%27s transformation under GenpactNext, as we lead the shift to Agentic AI Solutions-domain-specific, autonomous systems that redefine how we deliver value to clients. You%27ll help drive...

  • Senior Data Engineer

    6 days ago

    Gurgaon, Haryana, India Weekday AI Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    This role is for one of the Weekday's clientsMin Experience: 5 yearsLocation: GurgaonJobType: full-timeWe are seeking an experienced Senior Data Engineer with strong expertise in building and managing large-scale data pipelines within the AWS ecosystem. The ideal candidate will have a solid background in SQL, cloud-native data platforms, and orchestration...

  • Senior Data Engineer

    6 days ago

    Gurgaon, Haryana, India Weekday AI Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    This role is for one of the Weekday's clientsMin Experience: 5 yearsLocation: GurgaonJobType: full-timeWe are seeking an experienced Senior Data Engineer with strong expertise in building and managing large-scale data pipelines within the AWS ecosystem. The ideal candidate will have a solid background in SQL, cloud-native data platforms, and orchestration...