Pyspark Developer

3 weeks ago


Hyderabad, India KloudPortal Technology Solutions PVT. LTD Full time

Job Description

Job Summary
We are hiring a Senior PySpark Developer with 4-7 years of experience in building and optimising data pipelines using PySpark on Databricks, within AWS cloud environments. This role involves modernising legacy systems, integrating with Kafka, and collaborating across cross-functional teams.

Key Responsibilities

  • Develop and optimise scalable PySpark applications on Databricks.
  • Work with AWS services (S3, EMR, Glue, Lambda) for data processing.
  • Integrate streaming and batch data sources using Kafka.
  • Tune Spark jobs for performance and cost efficiency.
  • Collaborate with DevOps, product, and analytics teams.
  • Ensure data governance, lineage, and quality compliance.

Required Skills

  • 3-7 years of strong PySpark development experience.
  • Hands-on Databricks (Spark UI, performance tuning).
  • Good understanding of AWS services (S3, EMR, Glue, Lambda).
  • Experience with Kafka for streaming/batch processing.
  • Spark optimisation (partitioning, caching, joins).
  • Data formats: Parquet, Avro, ORC.
  • Orchestration: Airflow / Databricks Workflows.
  • Scala is a strong plus.

Qualifications

  • Bachelors or Master's in Computer Science / IT.

  • PySpark Developer

    1 week ago


    Hyderabad, India Anicalls (Pty) Ltd Full time

    • PySpark Developer / PySpark Data Engineer • Should have at least three years of strong experience in PySpark • Strong SQL Expertise • Strong communication and client-facing skills

  • PySpark Developer

    3 weeks ago


    Hyderabad, India Tata Consultancy Services Full time

    Job Title: PySpark Developer Experience: 6 to 8 Years Location: Hyderabad ( Onsite ) Job Description Must-Have Strong knowledge of Pyspark and SQL 2 + years of experience in PySpark. 4+Years of experience in DataStage, Terada. 6+ Years of experience in Python. Experience in GCP cloud Services Delta Lake Good-to-Have Pyspark GCP Google Cloud / Google...

  • PySpark Developer

    1 week ago


    Hyderabad, Telangana, India Tata Consultancy Services Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Job Title: PySpark DeveloperExperience: 6 to 8 YearsLocation: Hyderabad ( Onsite )Job DescriptionMust-HaveStrong knowledge of Pyspark and SQL2 + years of experience in PySpark.4+Years of experience in DataStage, Terada.6+ Years of experience in Python.Experience in GCP cloud ServicesDelta LakeGood-to-HavePysparkGCPGoogle Cloud / Google...

  • PySpark Developer

    3 weeks ago


    Hyderabad, India Algoleap Technologies Full time

    SUMMARY Job Summary We are looking for a Senior PySpark Developer with 3 to 6 years of experience in building and optimizing data pipelines using PySpark on Databricks, within AWS cloud environments. This role focuses on the modernization of legacy domains, involving integration with systems like Kafka and collaboration across cross-functional teams. Key...

  • Pyspark Developer

    2 weeks ago


    Hyderabad, Telangana, India KloudPortal Technology Solutions PVT. LTD Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Job DescriptionJob Summary We are hiring a Senior PySpark Developer with 4-7 years of experience in building and optimising data pipelines using PySpark on Databricks, within AWS cloud environments. This role involves modernising legacy systems, integrating with Kafka, and collaborating across cross-functional teams.Key ResponsibilitiesDevelop and optimise...

  • PySpark Developer

    1 week ago


    Hyderabad, Telangana, India Algoleap Technologies Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    SUMMARY Job SummaryWe are looking for a Senior PySpark Developer with 3 to 6 years of experience in building and optimizing data pipelines using PySpark on Databricks, within AWS cloud environments. This role focuses on the modernization of legacy domains, involving integration with systems like Kafka and collaboration across cross-functional teams.Key...

  • Pyspark Developer

    5 days ago


    Hyderabad, India Vista Applied Solutions Group Inc Full time

    Job Summary:A PySpark Developer is responsible for designing, developing, and optimizing large-scale data processing applications and pipelines using Apache Spark and Python. This role involves leveraging PySpark to handle, transform, and analyze vast datasets in distributed computing environments, often integrating with other big data technologies and cloud...

  • Pyspark Developer

    6 days ago


    Hyderabad, India Vista Applied Solutions Group Inc Full time

    Job Summary:A PySpark Developer is responsible for designing, developing, and optimizing large-scale data processing applications and pipelines using Apache Spark and Python. This role involves leveraging PySpark to handle, transform, and analyze vast datasets in distributed computing environments, often integrating with other big data technologies and cloud...

  • Pyspark Developer

    4 days ago


    Hyderabad, India Vista Applied Solutions Group Inc Full time

    Job Summary: A PySpark Developer is responsible for designing, developing, and optimizing large-scale data processing applications and pipelines using Apache Spark and Python. This role involves leveraging PySpark to handle, transform, and analyze vast datasets in distributed computing environments, often integrating with other big data technologies and...

  • Pyspark Developer

    5 days ago


    Hyderabad, India Vista Applied Solutions Group Inc Full time

    Job Summary:A PySpark Developer is responsible for designing, developing, and optimizing large-scale data processing applications and pipelines using Apache Spark and Python. This role involves leveraging PySpark to handle, transform, and analyze vast datasets in distributed computing environments, often integrating with other big data technologies and cloud...