Pyspark Data Engineer
4 weeks ago
Job Description
We are looking for a skilled Data Engineer with hands-on experience in PySpark and Databricks to design, develop, and optimize large-scale data pipelines. The ideal candidate should have strong problem-solving skills, experience working in cloud-based environments, and the ability to collaborate effectively with cross-functional teams.
Key Responsibilities:
- Develop and maintain ETL/ELT pipelines using PySpark on Databricks.
- Optimize data processing workflows for performance and scalability.
- Work with structured, semi-structured, and unstructured data from multiple sources.
- Collaborate with data scientists, analysts, and business teams to deliver clean, reliable datasets.
- Implement data quality checks, validations, and monitoring.
- Support migration of existing pipelines to cloud platforms (Azure/AWS/GCP).
- Document technical solutions, workflows, and best practices.
Required Skills:
- 35 years of experience in data engineering roles.
- Strong expertise in PySpark (DataFrames, RDDs, Spark SQL).
- Hands-on experience with Databricks (Delta Lake, Workflows, Jobs, Clusters).
- Solid understanding of data warehousing concepts and SQL.
- Experience with cloud platforms (Azure Data Lake, AWS S3, or GCP BigQuery).
- Familiarity with CI/CD pipelines and version control (Git).
- Good understanding of performance tuning and optimization in Spark.
- Strong problem-solving and debugging skills.
-
Pyspark Data Engineer
5 days ago
Chennai, Tamil Nadu, India Prowesstics Full time ₹ 15,00,000 - ₹ 25,00,000 per yearWe are looking for a skilledData Engineerwith hands-on experience inPySparkandDatabricksto design, develop, and optimize large-scale data pipelines. The ideal candidate should have strong problem-solving skills, experience working in cloud-based environments, and the ability to collaborate effectively with cross-functional teams.Key Responsibilities:Develop...
-
Data Engineer
3 weeks ago
Pune, Maharashtra, India, Maharashtra Tata Consultancy Services Full timeJob Title :- Data Engineer - PysparkExperience: 5 to 8 YearsLocation: Pune/HyderabadJob DescriptionRequired Skills:5+ years of experience in Big data and pysparkMust-HaveGood work experience on Big Data Platforms like Hadoop, Spark, Scala, Hive, Impala, SQLGood-to-HaveGood Spark, Pyspark,Big Data experienceSpark UI/Optimization/debugging techniquesGood...
-
PySpark Data Engineer
2 weeks ago
India EXTRAGIG Full time🚀 Contract Assistant – Data Engineer Support (Remote, EST Hours) 🚀 📅 Start Date: Sept 10, 2025 ⏳ Duration: 6 months (extendable) 💰 Pay: $1,000/month 🕗 Work Hours: 8:00 AM – 5:30 PM EST We’re looking for a Contract Assistant to support a PySpark Data Engineer with daily activities. This is a remote contract role (not formal employment)....
-
PySpark Data Engineer
1 week ago
India EXTRAGIG Full time🚀 Contract Assistant – Data Engineer Support (Remote, EST Hours) 🚀📅 Start Date: Sept 10, 2025⏳ Duration: 6 months (extendable)💰 Pay: $1,000/month🕗 Work Hours: 8:00 AM – 5:30 PM ESTWe’re looking for a Contract Assistant to support a PySpark Data Engineer with daily activities. This is a remote contract role (not formal employment).What...
-
Data Engineer- PySpark
5 days ago
Chennai, Tamil Nadu, India Cynosure Corporate Solutions Full time ₹ 9,00,000 - ₹ 12,00,000 per yearBuild up data pipelines for consumption by the data science team.Clear understanding and experience with Python and PySpark.Experience in writing Python programs and SQL queries.Experience in SQL Query tuning.Build and maintain data pipelines in Pyspark with SQL and Python.Knowledge of Cloud (Azure/AWS) technologies is additional. Suggest and implement best...
-
PySpark Data Engineer
2 weeks ago
india, IN EXTRAGIG Full timeContract Assistant – Data Engineer Support (Remote, EST Hours) Start Date: Sept 10, 2025⏳ Duration: 6 months (extendable) Pay: $1,000/month Work Hours: 8:00 AM – 5:30 PM ESTWe’re looking for a Contract Assistant to support a PySpark Data Engineer with daily activities. This is a remote contract role (not formal employment).What You’ll Do:Execute...
-
Data Engineer
4 days ago
Chennai, India Ixceed Solutions Full timeRole : Data Engineer Location : Chennai (Hybrid).Job Description : Key Responsibilities : - Lead the design, development, and deployment of scalable data pipelines and ETL processes using Scala and Apache Spark (Databricks). - Develop and maintain data processing workflows using PySpark and Python. - Collaborate with data scientists, analysts, and business...
-
Data Engineer – Databricks
2 weeks ago
Bengaluru, Karnataka, India, Karnataka Capgemini Engineering Full timeData Engineer – Databricks & PySparkLocation: BangaloreExperience: 6–9 yearsChoosing Capgemini means choosing a place where you’ll be empowered to shape your career, supported by a collaborative global community, and inspired to reimagine what’s possible. Join us in helping leading Consumer Products and Retail Services (CPRS) organizations unlock the...
-
Data Engineer Pyspark
1 week ago
Chennai, Tamil Nadu, India NatWest Group Full time ₹ 15,00,000 - ₹ 25,00,000 per yearJoin us as a Data EngineerThis is an exciting opportunity to use your technical expertise to collaborate with colleagues and build effortless, digital first customer experiencesYou'll be simplifying the bank by developing innovative data driven solutions, using insight to be commercially successful, and keeping our customers' and the bank's data safe and...
-
Senior Data Engineer
2 days ago
Chennai, India Pan Asia HR Solutions Full timeJob Title : Senior Data EngineerExperience : 6+ YearsLocation : Chennai, IndiaWork Mode : Hybrid (2-3 days/week onsite)About the Role :We are looking for a highly skilled Senior Data Engineer with a strong background in Python, Java, PySpark, and Google Cloud Platform (GCP) to join our growing data team in Chennai. The ideal candidate should have a deep...