Pyspark Data Engineer

4 weeks ago


Chennai India Prowesstics Full time

Job Description

We are looking for a skilled Data Engineer with hands-on experience in PySpark and Databricks to design, develop, and optimize large-scale data pipelines. The ideal candidate should have strong problem-solving skills, experience working in cloud-based environments, and the ability to collaborate effectively with cross-functional teams.

Key Responsibilities:

- Develop and maintain ETL/ELT pipelines using PySpark on Databricks.
- Optimize data processing workflows for performance and scalability.
- Work with structured, semi-structured, and unstructured data from multiple sources.
- Collaborate with data scientists, analysts, and business teams to deliver clean, reliable datasets.
- Implement data quality checks, validations, and monitoring.
- Support migration of existing pipelines to cloud platforms (Azure/AWS/GCP).
- Document technical solutions, workflows, and best practices.

Required Skills:

- 35 years of experience in data engineering roles.
- Strong expertise in PySpark (DataFrames, RDDs, Spark SQL).
- Hands-on experience with Databricks (Delta Lake, Workflows, Jobs, Clusters).
- Solid understanding of data warehousing concepts and SQL.
- Experience with cloud platforms (Azure Data Lake, AWS S3, or GCP BigQuery).
- Familiarity with CI/CD pipelines and version control (Git).
- Good understanding of performance tuning and optimization in Spark.
- Strong problem-solving and debugging skills.



  • Chennai, Tamil Nadu, India Prowesstics Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    We are looking for a skilledData Engineerwith hands-on experience inPySparkandDatabricksto design, develop, and optimize large-scale data pipelines. The ideal candidate should have strong problem-solving skills, experience working in cloud-based environments, and the ability to collaborate effectively with cross-functional teams.Key Responsibilities:Develop...

  • Data Engineer

    3 weeks ago


    Pune, Maharashtra, India, Maharashtra Tata Consultancy Services Full time

    Job Title :- Data Engineer - PysparkExperience: 5 to 8 YearsLocation: Pune/HyderabadJob DescriptionRequired Skills:5+ years of experience in Big data and pysparkMust-HaveGood work experience on Big Data Platforms like Hadoop, Spark, Scala, Hive, Impala, SQLGood-to-HaveGood Spark, Pyspark,Big Data experienceSpark UI/Optimization/debugging techniquesGood...

  • PySpark Data Engineer

    2 weeks ago


    India EXTRAGIG Full time

    🚀 Contract Assistant – Data Engineer Support (Remote, EST Hours) 🚀 📅 Start Date: Sept 10, 2025 ⏳ Duration: 6 months (extendable) 💰 Pay: $1,000/month 🕗 Work Hours: 8:00 AM – 5:30 PM EST We’re looking for a Contract Assistant to support a PySpark Data Engineer with daily activities. This is a remote contract role (not formal employment)....


  • India EXTRAGIG Full time

    🚀 Contract Assistant – Data Engineer Support (Remote, EST Hours) 🚀📅 Start Date: Sept 10, 2025⏳ Duration: 6 months (extendable)💰 Pay: $1,000/month🕗 Work Hours: 8:00 AM – 5:30 PM ESTWe’re looking for a Contract Assistant to support a PySpark Data Engineer with daily activities. This is a remote contract role (not formal employment).What...


  • Chennai, Tamil Nadu, India Cynosure Corporate Solutions Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Build up data pipelines for consumption by the data science team.Clear understanding and experience with Python and PySpark.Experience in writing Python programs and SQL queries.Experience in SQL Query tuning.Build and maintain data pipelines in Pyspark with SQL and Python.Knowledge of Cloud (Azure/AWS) technologies is additional. Suggest and implement best...

  • PySpark Data Engineer

    2 weeks ago


    india, IN EXTRAGIG Full time

    Contract Assistant – Data Engineer Support (Remote, EST Hours) Start Date: Sept 10, 2025⏳ Duration: 6 months (extendable) Pay: $1,000/month Work Hours: 8:00 AM – 5:30 PM ESTWe’re looking for a Contract Assistant to support a PySpark Data Engineer with daily activities. This is a remote contract role (not formal employment).What You’ll Do:Execute...

  • Data Engineer

    4 days ago


    Chennai, India Ixceed Solutions Full time

    Role : Data Engineer Location : Chennai (Hybrid).Job Description : Key Responsibilities : - Lead the design, development, and deployment of scalable data pipelines and ETL processes using Scala and Apache Spark (Databricks). - Develop and maintain data processing workflows using PySpark and Python. - Collaborate with data scientists, analysts, and business...


  • Bengaluru, Karnataka, India, Karnataka Capgemini Engineering Full time

    Data Engineer – Databricks & PySparkLocation: BangaloreExperience: 6–9 yearsChoosing Capgemini means choosing a place where you’ll be empowered to shape your career, supported by a collaborative global community, and inspired to reimagine what’s possible. Join us in helping leading Consumer Products and Retail Services (CPRS) organizations unlock the...


  • Chennai, Tamil Nadu, India NatWest Group Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Join us as a Data EngineerThis is an exciting opportunity to use your technical expertise to collaborate with colleagues and build effortless, digital first customer experiencesYou'll be simplifying the bank by developing innovative data driven solutions, using insight to be commercially successful, and keeping our customers' and the bank's data safe and...


  • Chennai, India Pan Asia HR Solutions Full time

    Job Title : Senior Data EngineerExperience : 6+ YearsLocation : Chennai, IndiaWork Mode : Hybrid (2-3 days/week onsite)About the Role :We are looking for a highly skilled Senior Data Engineer with a strong background in Python, Java, PySpark, and Google Cloud Platform (GCP) to join our growing data team in Chennai. The ideal candidate should have a deep...