PySpark Developer

16 hours ago

Pune, Maharashtra, India 636ca7a0-36ec-40d4-b141-86628a2f9f4a Full time ₹ 5,00,000 - ₹ 25,00,000 per year

Experience: 2-10 Years

Qualification: BE / B. Tech

Location: Pune, Manila, Remote

Technical Know -How: Pyspark, Python programming, ETL Pipeline, Spark SQL, DataFrames, Azure, Big data ecosystem

We are looking for skilled PySpark Developers to design and develop scalable data processing solutions. The role involves working with big data platforms, building ETL pipelines, and collaborating with cross-functional teams to ensure data availability, performance, and quality.

Key Responsibilities

  • Design, develop, and optimize ETL/ELT pipelines using PySpark.
  • Ingest, clean, transform, and process structured/semi-structured/unstructured data.
  • Work with large-scale datasets on distributed computing platforms (Hadoop, Spark, Databricks).
  • Integrate data from multiple sources including Delta Lake, Data Lake, RDBMS, APIs.
  • Ensure high-performance, scalability, and reliability of data pipelines.
  • Collaborate with data engineers, analysts, and data scientists to deliver business-ready datasets.
  • Implement monitoring, logging, and error-handling for data pipelines.
  • Contribute to data quality frameworks and best practices.

Required Skills & Experience

  • Bachelor's degree in computer science, computer engineering or similar.
  • Excellent communication skills.
  • 2–10 years of experience in data engineering with strong hands-on expertise in PySpark.
  • Proficiency in Python programming for data manipulation and pipeline development.
  • Strong knowledge of Spark SQL, DataFrames, and RDDs.
  • Experience with big data ecosystems – Hadoop, Hive, or Databricks.
  • Good understanding of SQL and relational databases (PostgreSQL, MySQL, Oracle).
  • Familiarity with cloud platforms (AWS, Azure, GCP) and services (S3, ADLS, BigQuery).
  • Experience with version control (Git) and CI/CD tools.
  • Strong debugging, performance optimization, and troubleshooting skills.
  • Experience with containerization (Docker, Kubernetes).

  • pyspark developer

    2 weeks ago

    Pune, Maharashtra, India Capgemini Engineering Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    At Capgemini Engineering, the world leader in engineering services, we bring together a global team of engineers, scientists, and architects to help the world's most innovative companies unleash their potential. From autonomous cars to life-saving robots, our digital and software technology experts think outside the box as they provide unique R&D and...

  • pyspark developer

    2 weeks ago

    Pune, Maharashtra, India Capgemini Full time ₹ 8,00,000 - ₹ 12,00,000 per year

    At Capgemini Engineering, the world leader in engineering services, we bring together a global team of engineers, scientists, and architects to help the world's most innovative companies unleash their potential. From autonomous cars to life-saving robots, our digital and software technology experts think outside the box as they provide unique R&D and...

  • Pyspark Developer

    2 weeks ago

    Pune, Maharashtra, India Capgemini Full time ₹ 5,00,000 - ₹ 12,00,000 per year

    Choosing Capgemini means choosing a company where you will be empowered to shape your career in the way you'd like, where you'll be supported and inspired by a collaborative community of colleagues around the world, and where you'll be able to reimagine what's possible. Join us and help the world's leading organizations unlock the value of technology and...

  • Pyspark Developer

    5 days ago

    Pune, Maharashtra, India Capgemini Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Choosing Capgemini means choosing a company where you will be empowered to shape your career in the way you'd like, where you'll be supported and inspired by a collaborative community of colleagues around the world, and where you'll be able to reimagine what's possible. Join us and help the world's leading organizations unlock the value of technology and...

  • Pyspark Developer

    3 days ago

    Pune, Maharashtra, India Capgemini Engineering Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Choosing Capgemini means choosing a company where you will be empowered to shape your career in the way you'd like, where you'll be supported and inspired by a collaborative community of colleagues around the world, and where you'll be able to reimagine what's possible. Join us and help the world's leading organizations unlock the value of technology and...

  • Pyspark Lead

    7 days ago

    Pune, Maharashtra, India Wipro Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Position OverviewWe are seeking a skilled and experienced Senior PySpark Developer with expertise in Apache spark, Spark Batch, and Spark Streaming to join our dynamic team. The ideal candidate will design, develop, and maintain high-performance, scalable applications for processing large-scale data in batch and real-time environments.Required Skills and...

  • Python with pyspark

    2 weeks ago

    Pune, Maharashtra, India Natobotics Full time ₹ 2,04,000 per year

    We're Hiring: Python with PySpark Developer (5–7 Years Experience)Skills Required:PythonPySparkSQLAirflowLocations: Bangalore | Hyderabad | PuneBudget: Up to 17 LPANote: Only immediate joiners or candidates serving notice period (who can join within a week) will be considered.

  • Bigdata PySpark Developer

    2 weeks ago

    Pune, Maharashtra, India Citi Full time ₹ 1,20,000 - ₹ 2,60,000 per year

    The Digital Software Engineer Intermediate Programmer Analyst is an intermediate level position responsible for participation in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team. The overall objective of this role is to contribute to applications systems analysis and programming...

  • ETL QA with Pyspark

    1 week ago

    Pune, Maharashtra, India Live Connections Full time ₹ 8,00,000 - ₹ 12,00,000 per year

    Role - ETL QA with Pyspark & PythonExperience - 6 to 10 yearsWork Location - PuneRequired Notice Period - Immedaite Joiners or 30 days maxJob DescriptionAs an ETL Tester,you will play a critical role in validating ETL processes and ensuring the accuracy of data extraction, transformation, and loading procedures. You will work closely with ETL developers,...

  • MarsDevs - AI/ML Engineer - ETL/PySpark

    2 weeks ago

    Pune, Maharashtra, India Marsdevs Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Company Description : MarsDevs is a remote-first, offshore digital product development company, committed to delivering high-quality code and long-lasting technical architectures. We specialize in building MVPs, mobile and web applications, and SaaS products for startups and SMBs. Our services include systematic project analysis, UI/UX design,...