AWS Glue PySpark Developer

1 week ago

India CIGNEX Full time ₹ 15,00,000 - ₹ 25,00,000 per year

We are looking for an experienced AWS Glue PySpark Developer to design, develop, and optimize ETL pipelines and data processing solutions on AWS. The ideal candidate will have deep expertise in PySpark, AWS Glue, and data engineering best practices, along with hands-on experience in building scalable, high-performance data solutions in the cloud.

Key Responsibilities:

  • Design, build, and maintain scalable ETL pipelines using AWS Glue and PySpark.
  • Work with stakeholders to gather and analyse data requirements and translate them into technical solutions.
  • Develop efficient and reusable PySpark scripts to process large-scale structured and unstructured datasets.
  • Optimize ETL jobs for performance, scalability, and cost-effectiveness in AWS environments.
  • Integrate AWS Glue with other AWS services such as S3, Redshift, RDS, Lambda, Step Functions, and Athena.
  • Implement data quality checks, validation frameworks, and error-handling mechanisms within ETL pipelines.
  • Collaborate with data engineers, analysts, and business teams to ensure data accuracy and consistency.
  • Monitor, debug, and resolve production issues related to Glue jobs and data workflows.
  • Ensure compliance with security, governance, and regulatory requirements for data pipelines.
  • Stay current with AWS and big data ecosystem advancements to continuously improve solutions.

Required Skills:

  • 5-6 years of experience in data engineering/ETL development, with at least 3 years in AWS Glue & PySpark.
  • Strong proficiency in PySpark, Spark SQL, and distributed data processing.
  • Hands-on experience with AWS services: S3, Glue Catalog, Redshift, RDS, Lambda, Step Functions, CloudWatch.
  • Expertise in designing data models, partitioning strategies, and optimizing large datasets.
  • Proficiency in SQL and working with relational as well as NoSQL databases.
  • Experience with version control (Git), CI/CD pipelines, and Agile methodologies.
  • Strong problem-solving skills and ability to debug complex data issues.
  • Excellent communication and collaboration skills.

  • FlyonIT-AWS Glue Developer

    1 day ago

    india Nexthire Full time

    Job Title: AWS Glue Developer  Location: Remote Employment Type: (Full-Time/Contract) Experience Level: 5 to 12 years   Job Summary:   We are seeking a skilled and experienced AWS Glue Developer to join our dynamic team. The ideal candidate will have a strong background in data integration, ETL (Extract, Transform, Load) processes, and cloud-based data...

  • AWS Pyspark Data Engineer

    1 week ago

    Pune City, Maharashtra, , India Dataeconomy Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    We are seeking a highly skilled and experienced Senior Data Engineer to lead the end -to -end development of complex models for compliance and supervision. The ideal candidate will have deep expertise in cloud -based infrastructure, ETL pipeline development, and financial domains, with a strong focus on creating robust, scalable, and efficient...

  • [Apply in 3 Minutes] Part Time-Sr. AWS Glue ETL Developer

    3 weeks ago

    India Abidott Solutions (OPC) Private Limited Full time

    Job Description We are seeking a Sr. AWS Glue ETL Developer with strong systems, software, and AWS cloud experience to support a major mainframe systems modernization initiative for federal customer. Based in DevOps framework, participate in and/or direct major deliverables of projects through all aspects of the software development lifecycle including scope...

  • Python AWS Data Engineer

    3 weeks ago

    India Digitrix Software LLP Full time

    Experience: 5 to 8 yearsJob description: Python AWS Data EngineerMandatory skills - python , pyspark, who can write codes , any cloud exp - aws/ gcp / azure. Python, AWS Python (core language skill) -- Backend, Pandas, PySpark (DataFrame API), interacting with AWS (e.g., boto3 for S3, Glue, Lambda)Data Processing: Spark (PySpark), Glue, EMR AWS Core...

  • Senior Data Engineer

    2 weeks ago

    India All European Careers Full time

    For an international organization in Geneva, Switzerland, we are urgently looking for a Full Remote Senior Data Engineer. The Senior Data Engineer is responsible for the implementation of the set of automated ETL pipelines, implementation of the analytics of the platform operations and importing new data sources and its maintenance. The Analytics Data...

  • Pyspark Data Engineer

    1 week ago

    Gurugaon, Haryana, India Dataeconomy Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    JobTitle: PySpark Data Engineer Experience: 5 – 8 Years Location: Hyderabad Employment Type: Full -Time JobSummary:We are looking for a skilledand experienced PySpark Data Engineer to join our growing dataengineering team. The ideal candidate will have 5–8 years of experience indesigning and implementing data pipelines using PySpark, AWS Glue,and Apache...

  • (15h Left) Python AWS Data Engineer

    3 weeks ago

    India Digitrix Software LLP Full time

    Experience: 5 to 8 years Job description: Python AWS Data Engineer Mandatory skills - python , pyspark, who can write codes , any cloud exp - aws/ gcp / azure. - Python, AWS Python (core language skill) -- Backend, Pandas, PySpark (DataFrame API), interacting with AWS (e.g., boto3 for S3, Glue, Lambda) - Data Processing: Spark (PySpark), Glue, EMR AWS Core...

  • Senior Pyspark Data Engineer

    7 days ago

    Hyderabad, Telangana, , India Dataeconomy Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Title: PySpark Data EngineerExperience: 8+ YearsLocation: HyderabadEmployment Type: Full -Time Job Summary:We are looking for a skilled and experienced PySpark Data Engineer to join our growing data engineering team. The ideal candidate will have 8+ years of experience in designing and implementing data pipelines using PySpark, AWS Glue,...

  • AWS Python

    4 days ago

    India Virtusa Full time

    Job Description Title: Developer (AWS Engineer) Requirements Candidate must have 5-6 years of IT working experience with at least 3 years of experience on AWS Cloud environment is preferred Strong hands-on experience Proficient in Node.js and Python Seasoned developers capable of independently driving development tasks Ability to understand the existing...

  • AWS Data Engineer

    4 days ago

    Chennai, India Virtusa Full time

    Job Description P2-C3-STS AWS Data Engineer Design and build scalable data pipelines using AWS services like AWS Glue, Amazon Redshift, and QS/SNS/Cloudwatch/Step function/CDK. Develop efficient ETL processes for data extraction, transformation, and loading into data warehouses and lakes. Create and manage applications using Python, Pyspark, SQL, Databricks,...