PySpark Developer

7 days ago


Hyderabad, Telangana, India algoleap Full time ₹ 15,00,000 - ₹ 25,00,000 per year

Job Summary
We are looking for a Senior PySpark Developer with 3 to 6 years of experience in building and optimizing data pipelines using PySpark on Databricks, within AWS cloud environments. This role focuses on the modernization of legacy domains, involving integration with systems like Kafka and collaboration across cross-functional teams.

Key Responsibilities

  • Develop and optimize scalable PySpark applications on Databricks.
  • Work with AWS services (S3, EMy, Lambda, Dlue) for cloud-native data processing.
  • Integrate streaming and batch data sources, especially using Kafka.
  • Tune Spark jobs for performance, memory, and compute efficiency.
  • Collaborate with DevOps, product, and analytics teams on delivery and deployment.
  • Ensure data governance, lineage, and quality compliance across all pipelines.

Required Skills

  • 3–6 years of hands-on development in PySpark.
  • Experience with Databricks and performance tuning using Spark UI.
  • Strong understanding of AWS services, Kafka, and distributed data processing.
  • Proficient in partitioning, caching, join optimization, and resource configuration.
  • Familiarity with data formats like Parquet, Avro, and OyC.
  • Exposure to orchestration tools (Airflow, Databricks Workflows).
  • Scala experience is a strong plus.

  • Pyspark Developer

    2 days ago


    Hyderabad, Telangana, India KloudPortal Technology Solutions PVT. LTD Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Job DescriptionJob Summary We are hiring a Senior PySpark Developer with 4-7 years of experience in building and optimising data pipelines using PySpark on Databricks, within AWS cloud environments. This role involves modernising legacy systems, integrating with Kafka, and collaborating across cross-functional teams.Key ResponsibilitiesDevelop and optimise...

  • PySpark Developer

    3 hours ago


    Hyderabad, Telangana, India Algoleap Technologies Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    SUMMARY Job SummaryWe are looking for a Senior PySpark Developer with 3 to 6 years of experience in building and optimizing data pipelines using PySpark on Databricks, within AWS cloud environments. This role focuses on the modernization of legacy domains, involving integration with systems like Kafka and collaboration across cross-functional teams.Key...

  • Pyspark Developer

    2 weeks ago


    Hyderabad, Telangana, India, Telangana Vista Applied Solutions Group Inc Full time

    Job Summary:A PySpark Developer is responsible for designing, developing, and optimizing large-scale data processing applications and pipelines using Apache Spark and Python. This role involves leveraging PySpark to handle, transform, and analyze vast datasets in distributed computing environments, often integrating with other big data technologies and cloud...

  • Pyspark

    4 minutes ago


    Hyderabad, Telangana, India Cirruslabs Full time

    Led the data integration efforts for enterprise-scale analytics and warehousing projects.Designed and developed PySpark scriptsboth generic frameworks and project-specific solutions—to handle complex ETL workflows.Created and managed process schedules to automate data ingestion, transformation, and loading pipelines.Developed and optimized data integration...


  • Hyderabad, Telangana, India Nisum Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Company DescriptionNisum is a global consulting partner specializing in digital commerce and evolution, offering AI-powered platforms and tailor-made solutions to unlock growth, optimize operations, and create long-term value. With over 25 years of proven experience, Nisum has a consistent track record of delivering complex digital transformations for...

  • Pyspark Developer

    2 days ago


    Hyderabad, Telangana, India Infosys Full time ₹ 5,00,000 - ₹ 12,00,000 per year

    Educational RequirementsBachelor of EngineeringService LineData & Analytics UnitResponsibilitiesA day in the life of an Infoscion- As part of the Infosys delivery team, your primary role would be to interface with the client for quality assurance, issue resolution and ensuring high customer satisfaction.- You will understand requirements, create and review...


  • Hyderabad, Telangana, India CirrusLabs Full time

    We areCirrusLabs. Our vision is to become the world's most sought-after niche digital transformation company that helps customers realize value through innovation. Our mission is to co-create success with our customers, partners and community. Our goal is to enable employees to dream, grow and make things happen. We are committed to excellence. We are a...


  • Hyderabad, Telangana, India Zorba AI Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Key Skills Required-Minimum 6 years of working experience in Python, Pyspark, AWS and SQL.Programming & Frameworks: Python, PySparkDatabases & Querying: Strong SQL (query optimization, joins, window functions)Big Data & Processing: PySpark for distributed data processing, ETL pipelinesCloud Platform: AWS (S3, Glue, Lambda, EMR, Redshift, Athena)Data...


  • Hyderabad, Telangana, India DATAECONOMY Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Title: PySpark Data EngineerExperience: 8 YearsLocation: HyderabadEmployment Type: Full-Time Job Summary: We are looking for a skilled and experienced PySpark Data Engineer to join our growing data engineering team. The ideal candidate will have 8 years of experience in designing and implementing data pipelines using PySpark, AWS Glue, and Apache...


  • Hyderabad, Telangana, India IDESLABS PRIVATE LIMITED Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Looking to onboard a skilled professional with 6-8 years of experience in Pyspark, SQL, and AWS Glue. The ideal candidate will have a strong background in these technologies and excellent problem-solving skills. This position is located across Pan India.Roles and ResponsibilityDesign, develop, and implement data processing pipelines using Pyspark and AWS...