Pyspark Developer

1 week ago

Greater Kolkata Area, India Viraaj HR Solutions Private Limited Full time ₹ 18,00,000 - ₹ 22,50,000 per year

Primary Title:
PySpark Developer (Big Data Engineer)

About The Opportunity
Recruiting for the Information Technology & Big Data Analytics sector, supporting enterprise-grade data platform and analytics initiatives. This on-site role in India places a PySpark Developer into engineering teams building scalable ETL/streaming pipelines, data lakes, and real-time analytics for mission-critical systems.

Role & Responsibilities

  • Design, develop and optimise PySpark-based ETL and streaming jobs to transform large-scale datasets for analytics and ML feature stores.
  • Implement robust data ingestion and processing pipelines across batch and streaming sources (Kafka, S3/HDFS, RDBMS), ensuring data quality and low-latency delivery.
  • Collaborate with data engineers and data scientists to productionise models and create reusable data schemas, partitioning and performance tuning strategies.
  • Author unit and integration tests, apply CI/CD practices, and deploy Spark jobs to production environments (on-premise clusters, EMR, or Databricks).
  • Monitor pipeline health, troubleshoot performance issues, and implement observability (metrics, logging, alerting) to maintain SLAs.
  • Document pipeline design, data contracts and runbooks; mentor junior engineers on Spark best practices and coding standards.

Skills & Qualifications
Must-Have

  • PySpark
  • Apache Spark
  • Python
  • SQL
  • Apache Hive
  • Hadoop
  • Apache Kafka
  • Airflow

Preferred

  • Databricks
  • AWS EMR
  • Apache NiFi

Other qualifications
: On-site in India; candidates should demonstrate proven delivery of production Spark pipelines and an ability to optimise for throughput and cost. Strong debugging and profiling experience with Spark UI and JVM metrics is highly valued.

Benefits & Culture Highlights

  • Work on high-impact, enterprise data platforms and real-time analytics use-cases.
  • Collaborative engineering teams with emphasis on code quality, observability, and continuous improvement.
  • Opportunity to upskill on cloud data services and modern data engineering patterns.

Location:
India — On-site. Recruitment partner handling the hiring process for our client. To apply, provide an updated CV highlighting PySpark projects, pipeline architectures, and performance tuning achievements.

Skills: python,sql,apache kafka,pyspark,hadoop,airflow,apache spark

  • Pyspark Developer

    3 weeks ago

    Kolkata, India Tata Consultancy Services Full time

    TCS is Hiring Pyspark Developer For Chennai, Hyderabad, Kolkata locationExperience : 6 - 10 yearsMode of interview: VirtualLocation : Chennai, Hyderabad, KolkataJOB DESCRIPTION:Must have:- Minimum 3 Years of experience in design, development and deployment of big data applications and ETL jobs using PySpark APIs/sparkSQL.- Experience in design, build and...

  • Pyspark

    2 weeks ago

    Kolkata, West Bengal, India Cognizant Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Skill: PysparkExperience: 6 to 9 yearsLocation: KolkataJob description: ResponsibilitiesDevelop and maintain scalable data pipelines using Python and PySpark.Collaborate with data engineers and data scientists to understand and fulfill data processing needs.Optimize and troubleshoot existing PySpark applications for performance improvements.Write clean,...

  • Pyspark Architect

    4 weeks ago

    Kolkata, India Talent Worx Full time

    Job Description: PySpark Architect Location: Bangalore,Pune,Kolkata Experience Required: 7 to 13 Years Role Overview: We are seeking a highly skilled and motivated PySpark Manager to lead our data engineering initiatives. The ideal candidate will have deep expertise in PySpark, Hadoop, AWS, Teradata, Scala, and data visualization tools. You will be...

  • Databricks QA Engineer – Python, PySpark

    1 week ago

    Greater Hyderabad Area, India ValueMomentum Full time ₹ 1,20,000 - ₹ 2,40,000 per year

    Job Title: Data QA Engineer – Databricks & PythonExperience:3 – 8 YearsLocation:Hyderabad / Pune / CoimbatoreNotice Period:Immediate to 30 DaysAbout The JobWe are seeking a Data QA Engineer with strong experience in Databricks and Python to ensure the quality, reliability, and accuracy of data pipelines and analytical solutions. The ideal candidate will...

  • Azure Synapse, Azure Databricks with Pyspark

    6 days ago

    Kolkata, West Bengal, India Cognizant Technology Solutions Full time ₹ 6,00,000 - ₹ 15,00,000 per year

    Job Title: - Azure Synapse, Azure Databricks with PysparkLocation: AIA-KolkataJob Type 3 days from office (Hybrid)Minimum 6 to 13 years of experienceMandatory Skills: Azure Databricks with PYSPARK/ Azure SynapseAt least 4 years of experience in Big Data\Distributed computing implementation as a developer.Working knowledge of Data Warehousing concepts and...

  • Python + Pyspark + SQL

    5 days ago

    Kolkata, West Bengal, India Talent Worx Full time ₹ 40,00,000 - ₹ 80,00,000 per year

    Our Client is a professional services firm, is the Indian member firm affiliated with International and wasestablished in September 1993. Our professionals leverage the global network of firms, providing detailedknowledge of local laws, regulations, markets, and competition. Our client has offices across India in Ahmedabad,Bengaluru, Chandigarh, Chennai,...

  • Data Engineer

    2 weeks ago

    Greater Kolkata Area, India Codesmith Full time ₹ 5,00,000 - ₹ 25,00,000 per year

    DescriptionWe are seeking a strong Data Engineer with advanced expertise in Databricks and PySpark. The successful candidate will be a key contributor to critical projects, including migrating Palantir data transformation pipelines to Databricks Notebooks, designing and implementing incremental data pipelines, and orchestrating workflows in Azure...

  • Python + Pyspark + SQL

    4 weeks ago

    Kolkata, India Talent Worx Full time

    Our Client is a professional services firm, is the Indian member firm affiliated with International and was established in September 1993. Our professionals leverage the global network of firms, providing detailed knowledge of local laws, regulations, markets, and competition. Our client has offices across India in Ahmedabad, Bengaluru, Chandigarh, Chennai,...

  • Azure Databricks with Pyspark

    6 days ago

    Kolkata, West Bengal, India Cognizant Technology Solutions Full time ₹ 5,00,000 - ₹ 12,00,000 per year

    Skill: Azure Databricks with PysparkExperience: 6 to 13 YrsLocation: AIA KolkataJob descriptionResponsibilities:Designing, building, and deploying data pipelines and workflows using Databricks.Developing and optimizing Apache Spark applications for data processing and analytics.Integrating Databricks with various data sources and platforms.Implementing data...

  • Hadoop Developer

    4 weeks ago

    Greater Bengaluru Area, India Tata Consultancy Services Full time

    Greetings from TCS!!! TCS has been a great pioneer in feeding the fire of young Techies like you. We are a global leader in the technology arena and there’s nothing that can stop us from growing together.Your role is of key importance, as it lays down the foundation for the entire project. Job Role- Bigdata & Hadoop DeveloperLocation- BangaloreTotal IT...