Pyspark Data Engineer

22 hours ago

Chennai, Tamil Nadu, India Prowesstics Full time ₹ 15,00,000 - ₹ 25,00,000 per year

We are looking for a skilled
Data Engineer
with hands-on experience in
PySpark
and
Databricks
to design, develop, and optimize large-scale data pipelines. The ideal candidate should have strong problem-solving skills, experience working in cloud-based environments, and the ability to collaborate effectively with cross-functional teams.

Key Responsibilities:

  • Develop and maintain
    ETL/ELT pipelines
    using
    PySpark
    on Databricks.
  • Optimize data processing workflows for
    performance and scalability
    .
  • Work with structured, semi-structured, and unstructured data from multiple sources.
  • Collaborate with data scientists, analysts, and business teams to deliver
    clean, reliable datasets
    .
  • Implement
    data quality checks, validations, and monitoring
    .
  • Support migration of existing pipelines to
    cloud platforms
    (Azure/AWS/GCP).
  • Document technical solutions, workflows, and best practices.

Required Skills:

  • 3–5 years
    of experience in data engineering roles.
  • Strong expertise in
    PySpark
    (DataFrames, RDDs, Spark SQL).
  • Hands-on experience with
    Databricks
    (Delta Lake, Workflows, Jobs, Clusters).
  • Solid understanding of
    data warehousing concepts
    and
    SQL
    .
  • Experience with
    cloud platforms
    (Azure Data Lake, AWS S3, or GCP BigQuery).
  • Familiarity with
    CI/CD pipelines
    and version control (Git).
  • Good understanding of
    performance tuning and optimization
    in Spark.
  • Strong problem-solving and debugging skills.

  • AWS Data Engineer + Pyspark

    5 days ago

    Chennai, Tamil Nadu, India Tata Consultancy Services (TCS) Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    TCS Inviting ApplicationRole : AWS Data engineer+ PysparkExperience : 6-10 YearsLocation : ChennaiMust: PySpark, AWS[ETL Concepts, S3, Glue, EMR, Redshift, DMS, AppFlow] ,Qlik Replicate, Data TestingNice To Have: Hadoop, Teradata Background, IaC[Cloud Formation / Terraform], GitLocationChennaiJob FunctionTECHNOLOGYRoleEngineerJob Id380170Desired...

  • Data Engineer Pyspark

    5 days ago

    Chennai, Tamil Nadu, India NatWest Group Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Join us as a Data EngineerThis is an exciting opportunity to use your technical expertise to collaborate with colleagues and build effortless, digital first customer experiencesYou'll be simplifying the bank by developing innovative data driven solutions, using insight to be commercially successful, and keeping our customers' and the bank's data safe and...

  • Data Engineer

    6 days ago

    Chennai, Tamil Nadu, India CX Data Labs Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    HiRole - Azure Data EngineerLocation - Chennai, Gurugram (5days Office)Full timeShift timing 2pm to 11pmKey Responsibilities:· Develop, maintain, and optimize scalable ETL/ELT pipelines using PySpark and Databricks.· Collaborate with cross-functional teams to design and implement data models and data integration solutions.· Create and maintain robust SQL...

  • Data engineer(Databricks,pyspark, ADF,SQL)

    5 days ago

    Chennai, Tamil Nadu, India Risk Resources LLP Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Overview: The Data Engineer is a crucial member of our data team, responsible for designing, building, and maintaining robust data infrastructure. Utilizing platforms such as Databricks and Azure Data Factory, this role focuses on creating efficient data pipelines and managing large datasets for analysis. By employing tools like PySpark and SQL, the Data...

  • Pyspark Developer

    5 days ago

    Chennai, Tamil Nadu, India viraaj hr solutions Full time ₹ 5,00,000 - ₹ 15,00,000 per year

    Primary Job Title: PySpark DeveloperRole & ResponsibilitiesDesign, develop and maintain scalable PySpark ETL pipelines for batch and streaming workloads across data lakes and warehouses.Ingest and transform data from diverse sources (Kafka, S3, HDFS, RDBMS) into analytics-ready formats using PySpark and Spark SQL.Optimize Spark jobs for performance and...

  • PySpark Developer

    3 days ago

    Chennai, Tamil Nadu, India Risk Resources LLP Full time ₹ 10,000 - ₹ 80,000 per year

    Overview The PySpark Developer plays a crucial role in leveraging big data technologies to analyze and transform data, thereby driving informed business decisions. This position focuses on the design, development, and optimization of data processing applications using PySpark, a Python API for Apache Spark. In the fast-paced tech environment, the PySpark...

  • Data Engineer

    7 days ago

    Chennai, Tamil Nadu, India Crayon Data Full time ₹ 5,00,000 - ₹ 15,00,000 per year

    Data Engineer – Chennai Build the foundation of data intelligence — design, develop, and optimize data pipelines that power AI-driven insights. Location: Chennai, India Experience: 2–3 years in data engineering or data-driven product development Role Overview As a Data Engineer, you will be responsible for building and...

  • AWS + Pyspark

    2 weeks ago

    Chennai, Tamil Nadu, India Cognizant Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Skills- AWS +PysparkExperience: 4 to 9 yearsLocation: AIA-PuneAs an AWS Data Engineer, you will design, build, and manage robust data pipelines on the AWS cloud platform. You will leverage your expertise in AWS data services and programming languages to ingest, process, transform, and store data efficiently, enabling data-driven insights and...

  • Pyspark

    2 weeks ago

    Chennai, Tamil Nadu, India Cognizant Technology Solutions Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job SummaryAs a Sr. Developer you will play a pivotal role in optimizing and enhancing our data processing capabilities using PySpark and AWS technologies. Your expertise in Python and domain knowledge in Cards & Payments will drive impactful solutions. This hybrid role offers the flexibility of day shifts ensuring a balanced work-life...

  • Pyspark Developer

    1 week ago

    Chennai, Tamil Nadu, India Citi Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Discover your future at CitiWorking at Citi is far more than just a job. A career with us means joining a team of more than 230,000 dedicated people from around the globe. At Citi, you'll have the opportunity to grow your career, give back to your community and make a real impact.Job OverviewAt Citi we're not just building technology, we're building the...