
Pyspark Developer
12 hours ago
Job Description
Job Summary
We are hiring a Senior PySpark Developer with 4-7 years of experience in building and optimising data pipelines using PySpark on Databricks, within AWS cloud environments. This role involves modernising legacy systems, integrating with Kafka, and collaborating across cross-functional teams.
Key Responsibilities
- Develop and optimise scalable PySpark applications on Databricks.
- Work with AWS services (S3, EMR, Glue, Lambda) for data processing.
- Integrate streaming and batch data sources using Kafka.
- Tune Spark jobs for performance and cost efficiency.
- Collaborate with DevOps, product, and analytics teams.
- Ensure data governance, lineage, and quality compliance.
Required Skills
- 3-7 years of strong PySpark development experience.
- Hands-on Databricks (Spark UI, performance tuning).
- Good understanding of AWS services (S3, EMR, Glue, Lambda).
- Experience with Kafka for streaming/batch processing.
- Spark optimisation (partitioning, caching, joins).
- Data formats: Parquet, Avro, ORC.
- Orchestration: Airflow / Databricks Workflows.
- Scala is a strong plus.
Qualifications
- Bachelors or Master's in Computer Science / IT.
-
Pyspark Developer
2 weeks ago
Hyderabad, Telangana, India Tata Consultancy Services Full time ₹ 15,00,000 - ₹ 20,00,000 per yearJob Title : PySpark SMEJob Location – HyderabadExperience: 4-8 Years of experience in IT industry with exposure to Python, Python , PySpark , Delta Lake , DataStage , Terada ( Bteq ,)Strong in Pyspark and SQL Requirement:Strong knowledge of Pyspark and SQL2 + years of experience in PySpark.4+Years of experience in DataStage, Terada.6+ Years of experience...
-
PySpark Developer
1 week ago
Hyderabad, Telangana, India Tata Consultancy Services Full time ₹ 1,04,000 - ₹ 1,30,878 per yearJob Title: PySpark DeveloperExperience: 6 to 8 YearsLocation: Hyderabad ( Onsite )Job DescriptionMust-HaveStrong knowledge of Pyspark and SQL2 + years of experience in PySpark.4+Years of experience in DataStage, Terada.6+ Years of experience in Python.Experience in GCP cloud ServicesDelta LakeGood-to-HavePysparkGCPGoogle Cloud / Google...
-
PySpark Developer
11 hours ago
Hyderabad, India Tata Consultancy Services Full timeJob Title: PySpark Developer Experience: 6 to 8 Years Location: Hyderabad ( Onsite ) Job Description Must-Have Strong knowledge of Pyspark and SQL 2 + years of experience in PySpark. 4+Years of experience in DataStage, Terada. 6+ Years of experience in Python. Experience in GCP cloud Services Delta Lake Good-to-Have Pyspark GCP Google Cloud / Google...
-
Pyspark Developer
2 weeks ago
Hyderabad, Telangana, India KloudPortal Technology Solutions PVT. LTD Full time ₹ 15,00,000 - ₹ 25,00,000 per yearJob DescriptionJob Summary We are hiring a Senior PySpark Developer with 4-7 years of experience in building and optimising data pipelines using PySpark on Databricks, within AWS cloud environments. This role involves modernising legacy systems, integrating with Kafka, and collaborating across cross-functional teams.Key ResponsibilitiesDevelop and optimise...
-
PySpark Developer
1 week ago
Hyderabad, Telangana, India Algoleap Technologies Full time ₹ 1,04,000 - ₹ 1,30,878 per yearSUMMARY Job SummaryWe are looking for a Senior PySpark Developer with 3 to 6 years of experience in building and optimizing data pipelines using PySpark on Databricks, within AWS cloud environments. This role focuses on the modernization of legacy domains, involving integration with systems like Kafka and collaboration across cross-functional teams.Key...
-
PySpark Developer
12 hours ago
Hyderabad, India Algoleap Technologies Full timeSUMMARY Job Summary We are looking for a Senior PySpark Developer with 3 to 6 years of experience in building and optimizing data pipelines using PySpark on Databricks, within AWS cloud environments. This role focuses on the modernization of legacy domains, involving integration with systems like Kafka and collaboration across cross-functional teams. Key...
-
PySpark Developer
3 days ago
Hyderabad, Bengaluru, Chennai, India Coders Brain Technology Private Limited Full timeJob Description ROLE RESPONSIBILITIES Data Engineering and Processing: Develop and manage data pipelines using PySpark on Databricks. Implement ETL/ELT processes to process structured and unstructured data at scale. Optimize data pipelines for performance, scalability, and cost-efficiency in Databricks. Databricks Platform Expertise: Experience in...
-
eSoftLabs - PySpark Developer
2 weeks ago
Hyderabad, Telangana, India ENTERPRISE SOFTLABS PRIVATE LINITED Full timeJob Title : Pyspark DeveloperLocation : HyderabadExperience Required : 4- 8Keywords : AWS, Pyspark, DatabricksSkills and experiences required :- 3- 6 years of hands-on development in PySpark.- Experience with Databricks and performance tuning using Spark UI.- Strong understanding of AWS services, Kafka, and distributed data processing.- Proficient in...
-
Pyspark Developer
2 days ago
Chennai, Hyderabad, Pune, India Tata Consultancy Services Full time ₹ 15,00,000 - ₹ 25,00,000 per yearTCS is Hiring Location :Bangalore, Chennai, Kolkata, Hyderabad, PuneExp : 4-8 YrsFunctional Skills: Experience in Credit Risk/Regulatory risk domainTechnical Skills: Spark ,PySpark, Python, Hive, Scala, MapReduce, Unix shell scriptingGood to Have Skills: Exposure to Machine Learning TechniquesJob Description:4+ Years of experience with Developing/Fine tuning...
-
Pyspark Developer
2 weeks ago
Bengaluru, Hyderabad, India Sonata Software Full time US$ 90,000 - US$ 1,20,000 per yearWe are seeking a skilled and experienced PySpark Developer to join our dynamic team. The ideal candidate will have a strong background in software development, with a focus on data transformation, processing, and performance optimization using PySpark. Expertise in SQL and its functions is required. Developer will be responsible for designing, developing,...