Pyspark Developer

2 days ago

Bengaluru Chennai Pune, India Maveric Full time ₹ 6,00,000 - ₹ 12,00,000 per year

Job Summary:

We are seeking a skilled and experienced PySpark Developer with 3 to 5 years of hands-on experience in building and optimizing distributed data pipelines, data processing workflows, and data validation using Apache Spark with Python (PySpark). The ideal candidate should have strong analytical and problem-solving skills, and be proficient in handling large-scale datasets, performing transformations, and ensuring data quality across distributed systems.

Key Responsibilities:

  • Develop, maintain, and optimize PySpark scripts for distributed data processing, cleansing, and validation.
  • Work extensively with Spark Data Frames for data wrangling, aggregation, and transformation.
  • Design and implement scalable ETL pipelines to process large datasets from diverse sources (e.g., HDFS, S3, Kafka).
  • Collaborate with data analysts, data scientists, and business teams to understand data requirements and deliver reliable solutions.
  • Ensure performance optimization through partitioning, caching, and efficient Spark job tuning.
  • Write unit tests and implement robust error handling and logging mechanisms.
  • Debug and troubleshoot Spark jobs and data-related issues to ensure accuracy and consistency.
  • Document data workflows, pipeline architecture, and transformation logic for future reference.
  • Contribute to process improvements and propose new approaches for distributed data handling and performance enhancement.

  • Data Engineer- 1

    7 days ago

    Bengaluru, Karnataka, India Databricks PySpark Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Position: Senior Data Engineer We are seeking an experienced Senior Data Engineer to design and implement high-performance data pipelines, ensuring the smooth flow of large datasets across systems. In this role, you will leverage your expertise in Databricks, Pyspark, and ETL design to build efficient data solutions. You will also contribute to...

  • Pyspark Developer

    4 weeks ago

    Chennai, India Tata Consultancy Services Full time

    Dear Candidate, Greetings from TCS !!! We are looking for Pyspark developers. Interested candidates share your resume to Role: Pyspark developer Experience : 5 to 8 years Location: Chennai / Hyderabad / Kolkata / Delhi / Pune / Bangalore

  • Pyspark Developer

    4 days ago

    chennai, India Tata Consultancy Services Full time

    Dear Candidate, Greetings from TCS !!! We are looking for Pyspark developers. Interested candidates share your resume to Role: Pyspark developer Experience : 5 to 8 years Location: Chennai / Hyderabad / Kolkata / Delhi / Pune / Bangalore

  • Pyspark Developer

    4 weeks ago

    Chennai, India Tata Consultancy Services Full time

    Dear Candidate,Greetings from TCS !!!We are looking for Pyspark developers. Interested candidates share your resume to : Pyspark developerExperience : 5 to 8 yearsLocation: Chennai / Hyderabad / Kolkata / Delhi / Pune / Bangalore

  • Pyspark Developer

    1 week ago

    Chennai, India Tata Consultancy Services Full time

    Dear Candidate,Greetings from TCS !!!We are looking for Pyspark developers. Interested candidates share your resume to : Pyspark developerExperience : 5 to 8 yearsLocation: Chennai / Hyderabad / Kolkata / Delhi / Pune / Bangalore

  • Pyspark Developer

    1 week ago

    chennai, India Tata Consultancy Services Full time

    Dear Candidate, Greetings from TCS !!! We are looking for Pyspark developers. Interested candidates share your resume to s.janani4@tcs.comRole: Pyspark developerExperience : 5 to 8 yearsLocation: Chennai / Hyderabad / Kolkata / Delhi / Pune / Bangalore

  • Pyspark Developer

    1 week ago

    Chennai, Tamil Nadu, India viraaj hr solutions Full time ₹ 5,00,000 - ₹ 15,00,000 per year

    Primary Job Title: PySpark DeveloperRole & ResponsibilitiesDesign, develop and maintain scalable PySpark ETL pipelines for batch and streaming workloads across data lakes and warehouses.Ingest and transform data from diverse sources (Kafka, S3, HDFS, RDBMS) into analytics-ready formats using PySpark and Spark SQL.Optimize Spark jobs for performance and...

  • PySpark Developer

    6 days ago

    Chennai, Tamil Nadu, India Risk Resources LLP Full time ₹ 10,000 - ₹ 80,000 per year

    Overview The PySpark Developer plays a crucial role in leveraging big data technologies to analyze and transform data, thereby driving informed business decisions. This position focuses on the design, development, and optimization of data processing applications using PySpark, a Python API for Apache Spark. In the fast-paced tech environment, the PySpark...

  • Pyspark Developer

    12 hours ago

    Chennai, India Citi Full time

    At Citi we’re not just building technology, we’re building the future of banking. Encompassing a broad range of specialties, roles, and cultures, our teams are creating innovations used across the globe. Citi is constantly growing and progressing through our technology, with laser focused on evolving the ways of doing things. As one of the world’s most...

  • PySpark Developers

    12 hours ago

    Pune, India Techno Wise Full time

    We are looking for a PySpark developer ( with ETL background ) to be able to design and build solution on one of our customer programs. This to build a data standardized and curation layer that will integrate data across internal and external sources, provide analytical insights and integrate with customer’s critical systems. Roles and Responsibilities...