
Pyspark Data Engineer
4 days ago
Job Description
We are looking for a skilled Data Engineer with hands-on experience in PySpark and Databricks to design, develop, and optimize large-scale data pipelines. The ideal candidate should have strong problem-solving skills, experience working in cloud-based environments, and the ability to collaborate effectively with cross-functional teams.
Key Responsibilities:
- Develop and maintain ETL/ELT pipelines using PySpark on Databricks.
- Optimize data processing workflows for performance and scalability.
- Work with structured, semi-structured, and unstructured data from multiple sources.
- Collaborate with data scientists, analysts, and business teams to deliver clean, reliable datasets.
- Implement data quality checks, validations, and monitoring.
- Support migration of existing pipelines to cloud platforms (Azure/AWS/GCP).
- Document technical solutions, workflows, and best practices.
Required Skills:
- 35 years of experience in data engineering roles.
- Strong expertise in PySpark (DataFrames, RDDs, Spark SQL).
- Hands-on experience with Databricks (Delta Lake, Workflows, Jobs, Clusters).
- Solid understanding of data warehousing concepts and SQL.
- Experience with cloud platforms (Azure Data Lake, AWS S3, or GCP BigQuery).
- Familiarity with CI/CD pipelines and version control (Git).
- Good understanding of performance tuning and optimization in Spark.
- Strong problem-solving and debugging skills.
-
Pyspark Data Engineer
3 days ago
Chennai, Tamil Nadu, India Prowesstics Full time ₹ 15,00,000 - ₹ 25,00,000 per yearWe are looking for a skilledData Engineerwith hands-on experience inPySparkandDatabricksto design, develop, and optimize large-scale data pipelines. The ideal candidate should have strong problem-solving skills, experience working in cloud-based environments, and the ability to collaborate effectively with cross-functional teams.Key Responsibilities:Develop...
-
(15h Left) Data Engineer
2 weeks ago
Bengaluru, India NTT Data Full timeJob Description Req ID:334009 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer (Talend &Pyspark) to join our team in Bangalore, Karntaka (IN-KA), India (IN). Job...
-
Data Engineer- PySpark
2 weeks ago
Chennai, India Cynosure Corporate Solutions Full timeBuild up data pipelines for consumption by the data science team. Clear understanding and experience with Python and PySpark. Experience in writing Python programs and SQL queries. Experience in SQL Query tuning. Build and maintain data pipelines in Pyspark with SQL and Python. Knowledge of Cloud (Azure/AWS) technologies is additional. Suggest and implement...
-
Data Engineer- PySpark
3 days ago
Chennai, Tamil Nadu, India Cynosure Corporate Solutions Full time ₹ 9,00,000 - ₹ 12,00,000 per yearBuild up data pipelines for consumption by the data science team.Clear understanding and experience with Python and PySpark.Experience in writing Python programs and SQL queries.Experience in SQL Query tuning.Build and maintain data pipelines in Pyspark with SQL and Python.Knowledge of Cloud (Azure/AWS) technologies is additional. Suggest and implement best...
-
Data Engineer
1 week ago
Ambala, India Shiv Kanti Infosystems Full timeJob Description Senior Data Engineer Azure, Airflow & PySpark We're hiring a Senior Data Engineer (69 years of experience) to lead the modernization of our data ecosystem from legacy platforms (Azure Synapse, SQL Server) to a next-gen Microsoft Fabric lakehouse. If you bring deep Azure expertise, strong Airflow orchestration skills, and at least 4 years of...
-
Pyspark Developer
22 hours ago
india Tata Consultancy Services Full timeGreetings from TCS!!TCS is hiring forPyspark DeveloperRequired Skill Set:Pyspark, Python, SQL and relational databases, SparkSQL, Spark Scripting, UNIX Shell Scripting, ETL, Data Warehousing, CI/CDDesired Experience Range:4 to 10 Years Job Location:Hyderabad, Bangalore, Chennai, Kolkata, PuneMust Have: Data Engineer, Python developer with specialty in Pandas...
-
Data Engineer
20 hours ago
Pune, India Barclays Full timeJob Description Join us as a Data Engineer - Pyspark,SQL at Barclays, where you'll spearhead the evolution of our digital landscape, driving innovation and excellence. You'll harness cutting-edge technology to revolutionise our digital offerings, ensuring unparalleled customer experiences. As a part of team of developers, you will deliver technology stack,...
-
Azure Data Engineer
20 hours ago
Hyderabad, India Fragma Data Systems Full timeJob Description Must-Have Skills - Good experience in Pyspark - Including Dataframe core functions and Spark SQL - Good experience in SQL DBs - Be able to write queries including fair complexity. - Should have excellent experience in Big Data programming for data transformation and aggregations - Good at ELT architecture. Business rules processing and data...
-
Pune, India NTT Data Full timeJob Description Req ID:332236 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Business Consulting-Technical analyst with ETL,GCP using Pyspark to join our team in Pune, Mahrshtra...
-
Snowflake Developer With PySpark
18 hours ago
India DigiHelic Solutions Pvt. Ltd. Full timeJob Role: Snowflake Developer Experience: 6-10 Years Location: Trivandrum/Kochi/Bangalore/Chennai/Pune/Noida/Hyderabad Work Model: Hybrid Mandatory Skills: Snowflake, PySpark, ETL, SQL Must Have Skills Data Warehouse: · Design, implement, and optimize data warehouses on the Snowflake platform. · Ensure effective utilization of Snowflake features for...