Developer - PySpark

3 days ago

Noida, Uttar Pradesh, India Compunnel Full time ₹ 15,00,000 - ₹ 25,00,000 per year

We are looking for skilled PySparkDevelopers to design and develop scalable data processing solutions. Therole involves working with big data platforms, building ETL pipelines,and collaborating with cross-functional teams to ensure data availability,performance, and quality.

Qualification:BE / B. TechLocation:Pune,

Key Responsibilities

  • Design, develop, and optimize ETL/ELT pipelines using PySpark.
  • Ingest, clean, transform, and process structured/semi-structured/unstructured data.
  • Work with large-scale datasets on distributed computing platforms (Hadoop, Spark, Databricks).
  • Integrate data from multiple sources including Delta Lake, Data Lake, RDBMS, APIs.
  • Ensure high-performance, scalability, and reliability of data pipelines.
  • Collaborate with data engineers, analysts, and data scientists to deliver business-ready datasets.
  • Implement monitoring, logging, and error-handling for data pipelines.
  • Contribute to data quality frameworks and best practices

Required Skills & Experience

  • Bachelors degree in computer science, computer engineering or similar.
  • Excellent communication skills.
  • 2-10 years of experience in data engineering with strong hands-on expertise in PySpark.
  • Proficiency in Python programming for data manipulation and pipeline development.
  • Strong knowledge of Spark SQL, DataFrames, and RDDs.
  • Experience with big data ecosystems Hadoop, Hive, or Databricks.
  • Good understanding of SQL and relational databases (PostgreSQL, MySQL, Oracle).
  • Familiarity with cloud platforms (AWS, Azure, GCP) and services (S3, ADLS, BigQuery).
  • Experience with version control (Git) and CI/CD tools.
  • Strong debugging, performance optimization, and troubleshooting skills.
  • Experience with containerization (Docker, Kubernetes).

  • Pyspark Developer

    1 day ago

    Noida, Uttar Pradesh, India Zigsaw Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Roles & Responsibilities: Design development and implementation of performant ETL pipelines using python API (pySpark) of Apache Spark on AWS EMR Writing reusable testable and efficient code Integration of data storage solutions in spark & ndash especially with AWS S3 object storage. Performance tuning of pySpark scripts. Need to ensure overall build...

  • Python Pyspark Developer

    1 day ago

    Noida, Uttar Pradesh, India Zigsaw Full time ₹ 1,00,00,000 - ₹ 2,00,00,000 per year

    Roles & Responsibilities: Design, development and implementation of performant ETL pipelines using python API (pySpark) of Apache Spark on AWS EMR Writing reusable, testable, and efficient code Integration of data storage solutions in spark – especially with AWS S3 object storage. Performance tuning of pySpark scripts. Need to ensure overall build delivery...

  • Business Consulting-Technical analyst with ETL,GCP using Pyspark

    6 days ago

    Noida, Uttar Pradesh, India NTT DATA North America Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Req ID: NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Business Consulting-Technical analyst with ETL,GCP using Pyspark to join our team in Noida, Uttar Pradesh (IN-UP), India...

  • Data Engineer

    2 weeks ago

    Noida, Uttar Pradesh, India Source-Right Full time ₹ 5,00,000 - ₹ 12,00,000 per year

    Position: Data Engineer (AWS QuickSight, Glue, PySpark) (Noida) (CE46SF RM 3386)Education Required: Bachelor's / Masters / PhD:Bachelor's or master's in computer science, Statistics, Mathematics, Data Science, EngineeringAWS certification (e.g., AWS Certified Data Analytics – Specialty, AWS Certified Developer)Must Have SkillsProficiency in AWS cloud...

  • Sr. Databricks Developer

    1 day ago

    Noida, Uttar Pradesh, India Topcoder Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Primary skill: Azure Databricks, SQL, PySparkAdditional skills: Power BIDetailed JD: -Extensive experience in designing and building data pipelines within an Azure Databricks Lakehouse environment.Strong understanding and practical experience with Spark and other big data technologies.Integrated spark jobs in Databricks to read and write structured, batch...

  • Associate - Business Analyst-Application Development-Cloud Implementation

    2 weeks ago

    Noida, Uttar Pradesh, India EXL Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    DescriptionJob DescriptionJob Title: Data Engineer - PySpark, Python, SQL, Git, AWS Services – Glue, Lambda, Step Functions, S3, Athena. Job Description:We are seeking a talented Data Engineer with expertise in PySpark, Python, SQL, Git, and AWS to join our dynamic team. The ideal candidate will have a strong background in data engineering, data...

  • ETL/Automation Test Lead

    4 days ago

    Noida, Uttar Pradesh, India Product Company Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    We're Hiring: Senior ETL/Automation Test lead in Pharmacovigilance ITWe are looking for anexperienced Senior ETL/Automation TesterKey Requirements:Developer Skills– Strong experience in writing SQL codeETL Testing Expertise– Ability to design test cases, scenarios, and scripts; execute and analyze results; and contribute to analysis, design,...

  • Associate- BIM

    1 week ago

    Noida, Uttar Pradesh, India Axtria - Ingenious Insights Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Position SummaryTo be a driven business analyst who can work on complex Analytical problems and help the customer in better business decision making especially in the area of pharma/life sciences (domain).Job ResponsibilitiesEngage with Client to participate in requirement gathering, Status update on work, UAT and be the key partner in the overall...

  • Data Engineer

    1 week ago

    Noida, Uttar Pradesh, India EICE Technology Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Role Overview:We are looking for a highly skilledSenior Data Engineer 7+ years of experience as a Microsoft Fabric Specialistto lead the design, development, and optimization of modern data solutions. The role involves working extensively withAzure Fabric, ADF, Databricks, SSIS, Power BI, and PySparkto build scalable, high-performance data pipelines and...

  • Senior Data Engineer

    2 weeks ago

    Noida, Uttar Pradesh, India Eice Technology Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Job Title: Senior Data Engineer, Microsoft Fabric + SSIS SpecialistLocation: Noida, Sector 62 (Onsite, Full-Time, Long-Term Project)Experience Required: 7 + YearsAvailability: Immediate Joiners OnlyRole Overview:We are looking for a highly skilled Senior Data Engineer 7+ years of experience as a Microsoft Fabric Specialist to lead the design, development,...