
PySpark Developer
6 days ago
Job Summary
We are looking for a Senior PySpark Developer with 3 to 6 years of experience in building and optimizing data pipelines using PySpark on Databricks, within AWS cloud environments. This role focuses on the modernization of legacy domains, involving integration with systems like Kafka and collaboration across cross-functional teams.
Key Responsibilities
- Develop and optimize scalable PySpark applications on Databricks.
- Work with AWS services (S3, EMy, Lambda, Dlue) for cloud-native data processing.
- Integrate streaming and batch data sources, especially using Kafka.
- Tune Spark jobs for performance, memory, and compute efficiency.
- Collaborate with DevOps, product, and analytics teams on delivery and deployment.
- Ensure data governance, lineage, and quality compliance across all pipelines.
Required Skills
- 3–6 years of hands-on development in PySpark.
- Experience with Databricks and performance tuning using Spark UI.
- Strong understanding of AWS services, Kafka, and distributed data processing.
- Proficient in partitioning, caching, join optimization, and resource configuration.
- Familiarity with data formats like Parquet, Avro, and OyC.
- Exposure to orchestration tools (Airflow, Databricks Workflows).
- Scala experience is a strong plus.
-
Pyspark Developer
24 hours ago
Hyderabad, Telangana, India KloudPortal Technology Solutions PVT. LTD Full time ₹ 15,00,000 - ₹ 25,00,000 per yearJob DescriptionJob Summary We are hiring a Senior PySpark Developer with 4-7 years of experience in building and optimising data pipelines using PySpark on Databricks, within AWS cloud environments. This role involves modernising legacy systems, integrating with Kafka, and collaborating across cross-functional teams.Key ResponsibilitiesDevelop and optimise...
-
eSoftLabs - PySpark Developer
3 weeks ago
Hyderabad, Telangana, India ENTERPRISE SOFTLABS PRIVATE LINITED Full timeJob Title : Pyspark DeveloperLocation : HyderabadExperience Required : 4- 8Keywords : AWS, Pyspark, DatabricksSkills and experiences required :- 3- 6 years of hands-on development in PySpark.- Experience with Databricks and performance tuning using Spark UI.- Strong understanding of AWS services, Kafka, and distributed data processing.- Proficient in...
-
Pyspark Developer
1 day ago
Hyderabad, Telangana, India Infosys Full time ₹ 5,00,000 - ₹ 12,00,000 per yearEducational RequirementsBachelor of EngineeringService LineData & Analytics UnitResponsibilitiesA day in the life of an Infoscion- As part of the Infosys delivery team, your primary role would be to interface with the client for quality assurance, issue resolution and ensuring high customer satisfaction.- You will understand requirements, create and review...
-
Date Engineer(Python, Pyspark and AWS)
1 week ago
Hyderabad, Telangana, India Zorba AI Full time ₹ 15,00,000 - ₹ 25,00,000 per yearKey Skills Required-Minimum 6 years of working experience in Python, Pyspark, AWS and SQL.Programming & Frameworks: Python, PySparkDatabases & Querying: Strong SQL (query optimization, joins, window functions)Big Data & Processing: PySpark for distributed data processing, ETL pipelinesCloud Platform: AWS (S3, Glue, Lambda, EMR, Redshift, Athena)Data...
-
Data Engineer Architect- Pyspark
4 weeks ago
Hyderabad, Telangana, India Mount Talent Consulting Pvt Ltd. Full timeProfessional & Technical Skills: - Must To Have Skills: Proficiency in PySpark, Amazon Web Services (AWS), Python (Programming Language). - Strong understanding of data processing and analysis which includes relational database - Experience in designing and implementing scalable applications. - Proficient in troubleshooting and problem-solving in application...
-
Data Engineer Architect- Pyspark
2 weeks ago
Hyderabad, Telangana, India Mount Talent Consulting Pvt Ltd. Full timeProfessional & Technical Skills:- Must To Have Skills: Proficiency in PySpark, Amazon Web Services (AWS), Python (Programming Language).- Strong understanding of data processing and analysis which includes relational database- Experience in designing and implementing scalable applications.- Proficient in troubleshooting and problem-solving in application...
-
Pyspark,sql & aws glue Professional
4 days ago
Hyderabad, Telangana, India IDESLABS PRIVATE LIMITED Full time ₹ 12,00,000 - ₹ 36,00,000 per yearLooking to onboard a skilled professional with 6-8 years of experience in Pyspark, SQL, and AWS Glue. The ideal candidate will have a strong background in these technologies and excellent problem-solving skills. This position is located across Pan India.Roles and ResponsibilityDesign, develop, and implement data processing pipelines using Pyspark and AWS...
-
Pyspark Lead Developer
1 week ago
Hyderabad, Telangana, India Wipro Full time US$ 80,000 - US$ 1,20,000 per yearRole Purpose The purpose of the role is to support process delivery by ensuring daily performance of the Production Specialists, resolve technical escalations and develop technical capability within the Production Specialists.*Do*Oversee and support process by reviewing daily transactions on performance parametersReview performance dashboard and the scores...
-
Hyderabad, Telangana, India Disa Consulting Services Full time ₹ 15,00,000 - ₹ 25,00,000 per year2 rounds of coding sessions is must for every candidate.Position: Pyspark, Python, SQL, Palantir Data EngineerLocation: Hyderabad, India. Onsite from Day1. Need local Hyderabad, India or close by locations candidatesResponsibilitiesDevelop and enhance data-processing, orchestration, monitoring, and more by leveraging popular open-source software, AWS, and...
-
Hyderabad, Telangana, India Rapsys Tech Solutions Full time ₹ 15,00,000 - ₹ 25,00,000 per year7+ Years of expertise required in below skillsets: Data Engineering, Python, SQL, PySpark, Cloud (Any), Airflow, DevOps & Orchestration, Agile, Scrum, Data Pipeline Development, Stakeholder Management, Banking/ Telecom Domain Experience.