Lead Pyspark Data Engineer

1 day ago

Hyderabad, Telangana, India DATAECONOMY Full time ₹ 12,00,000 - ₹ 36,00,000 per year

Job Title: PySpark Data Engineer
Experience: 10+ Years
Location: Hyderabad/ Pune
Employment Type: Full-Time

Job Summary:

We are looking for a skilled and experienced PySpark Data Engineer to join our growing data engineering team. The ideal candidate will have 10+ years of experience in designing and implementing data pipelines using PySpark, AWS Glue, and Apache Airflow, with strong proficiency in SQL. You will be responsible for building scalable data processing solutions, optimizing data workflows, and collaborating with cross-functional teams to deliver high-quality data assets.

Requirements

Key Responsibilities:

  • Design, develop, and maintain large-scale ETL pipelines using PySpark and AWS Glue.
  • Orchestrate and schedule data workflows using Apache Airflow.
  • Optimize data processing jobs for performance and cost-efficiency.
  • Work with large datasets from various sources, ensuring data quality and consistency.
  • Collaborate with Data Scientists, Analysts, and other Engineers to understand data requirements and deliver solutions.
  • Write efficient, reusable, and well-documented code following best practices.
  • Monitor data pipeline health and performance; resolve data-related issues proactively.
  • Participate in code reviews, architecture discussions, and performance tuning.
Requirements

  • 10+ years of experience in data engineering roles.
  • Strong expertise in PySpark for distributed data processing.
  • Hands-on experience with AWS Glue and other AWS data services (S3, Athena, Lambda, etc.).
  • Experience with Apache Airflow for workflow orchestration.
  • Strong proficiency in SQL for data extraction, transformation, and analysis.
  • Familiarity with data modeling concepts and data lake/data warehouse architectures.
  • Experience with version control systems (e.g., Git) and CI/CD processes.
  • Ability to write clean, scalable, and production-grade code.
BenefitsAs per company standards.

  • Senior Pyspark Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India DATAECONOMY Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    About UsAbout DATAECONOMY: We are a fast-growing data & analytics company headquartered in Dublin with offices inDublin, OH, Providence, RI, and an advanced technology center in Hyderabad,India. We are clearly differentiated in the data & analytics space via our suite of solutions, accelerators, frameworks, and thought leadership.Job DescriptionJob...

  • Senior Pyspark Data Engineer

    3 days ago

    Hyderabad, Telangana, India DataEconomy Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job InformationDate Opened10/13/2025Job TypeFull timeIndustryIT ServicesCityHyderabadState/ProvinceTelanganaCountryIndiaZip/Postal Code500081About UsAbout DATAECONOMY: We are a fast-growing data & analytics company headquartered in Dublin with offices inDublin, OH, Providence, RI, and an advanced technology center in Hyderabad,India. We are clearly...

  • Pyspark Developer

    2 weeks ago

    Hyderabad, Telangana, India NTT DATA Business Solutions Full time ₹ 8,00,000 - ₹ 12,00,000 per year

    Job SummaryWe are looking for a Senior PySpark Developer with 3 to 6 years of experience in building and optimizing data pipelines using PySpark on Databricks, within AWS cloud environments. This role focuses on the modernization of legacy domains, involving integration with systems like Kafka and collaboration across cross-functional teams.Key...

  • Senior PySpark Engineer

    1 day ago

    Hyderabad, Telangana, India Rapsys Technologies Pte. Ltd Full time ₹ 6,00,000 - ₹ 27,00,000 per year

    Role: Senior PySpark EngineerExperience Required: Minimum 8+ YearsWork Location: Hyderabad (5 Days Work from Office)Job Type: Contract to Hire (1 Year/ Renewable)Notice Period: Immediate to 15 Days maxMode of Interview: VirtualWe are seeking a highly skilled PySpark Data Engineer to design, build, and optimize large-scale data pipelines and distributed...

  • Lead Data Engineer

    3 days ago

    Hyderabad, Telangana, India Zorba AI Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Primary Job Title:Data Engineering LeadAbout The OpportunityWe are seeking a highly skilledLead Data Engineerwith strong expertise inPython, Pandas, PySpark, AWS, and SQLto design, build, and manage scalable data solutions. The ideal candidate will lead a team of data engineers, develop robust ETL pipelines, and collaborate with analytics, data science, and...

  • Lead Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Zorba Consulting Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    About The Opportunity : We are seeking a highly skilled Lead Data Engineer with strong expertise in Python, Pandas, PySpark, AWS, and SQL to design, build, and manage scalable data solutions. The ideal candidate will lead a team of data engineers, develop robust ETL pipelines, and collaborate with analytics, data science, and business teams to ensure high...

  • PySpark Engineer

    1 week ago

    Hyderabad, Telangana, India Rapsys Technologies Pte. Ltd Full time ₹ 6,00,000 - ₹ 25,00,000 per year

    Role: PySpark EngineerExperience Required: Minimum 5-10 YearsWork Location: Hyderabad (5 Days Work from Office)Job Type: Contract to Hire (1 Year/ Renewable)Notice Period: Immediate to 30 Days maxRequired skill sets and expertise:5+ years of experience in PySpark, Big Data technologies, Data Warehousing.Strong Python programming experience.Domain: Experience...

  • Databricks + Pyspark

    6 days ago

    Hyderabad, Telangana, India Cognizant Full time ₹ 1,00,00,000 - ₹ 3,00,00,000 per year

    Skills- Databricks+ PysparkExperience: 4 to 13 yearsLocation: AIA-PuneWe are looking for a highly skilled Data Engineer with expertise in PySpark and Databricks to design, build, and optimize scalable data pipelines for processing massive datasets.Key Responsibilities:Build & Optimize Pipelines: Develop high-throughput ETL workflows using PySpark on...

  • Senior Data Engineer

    6 days ago

    Hyderabad, Telangana, India Enable Data Incorporated Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Experience Required: 8+YearsMode of work: RemoteSkills Required: Azure DataBricks, Eventhub, Kafka, Architecture, Azure Data Factory, Pyspark, Python, SQL, SparkNotice Period : Immediate Joiners/ Permanent/Contract role (Can join within September 29th 2025)Translate business rules into technical specifications and implement scalable data solutions.Manage a...

  • Pyspark

    2 weeks ago

    Hyderabad, Telangana, India Cirruslabs Full time

    Led the data integration efforts for enterprise-scale analytics and warehousing projects.Designed and developed PySpark scriptsboth generic frameworks and project-specific solutions—to handle complex ETL workflows.Created and managed process schedules to automate data ingestion, transformation, and loading pipelines.Developed and optimized data integration...