Developer - PySpark

4 days ago

Noida, Uttar Pradesh, India Compunnel Full time ₹ 15,00,000 - ₹ 25,00,000 per year

We are looking for skilled PySparkDevelopers to design and develop scalable data processing solutions. Therole involves working with big data platforms, building ETL pipelines,and collaborating with cross-functional teams to ensure data availability,performance, and quality.

Qualification:BE / B. TechLocation:Pune,

Key Responsibilities

  • Design, develop, and optimize ETL/ELT pipelines using PySpark.
  • Ingest, clean, transform, and process structured/semi-structured/unstructured data.
  • Work with large-scale datasets on distributed computing platforms (Hadoop, Spark, Databricks).
  • Integrate data from multiple sources including Delta Lake, Data Lake, RDBMS, APIs.
  • Ensure high-performance, scalability, and reliability of data pipelines.
  • Collaborate with data engineers, analysts, and data scientists to deliver business-ready datasets.
  • Implement monitoring, logging, and error-handling for data pipelines.
  • Contribute to data quality frameworks and best practices

Required Skills & Experience

  • Bachelors degree in computer science, computer engineering or similar.
  • Excellent communication skills.
  • 2-10 years of experience in data engineering with strong hands-on expertise in PySpark.
  • Proficiency in Python programming for data manipulation and pipeline development.
  • Strong knowledge of Spark SQL, DataFrames, and RDDs.
  • Experience with big data ecosystems Hadoop, Hive, or Databricks.
  • Good understanding of SQL and relational databases (PostgreSQL, MySQL, Oracle).
  • Familiarity with cloud platforms (AWS, Azure, GCP) and services (S3, ADLS, BigQuery).
  • Experience with version control (Git) and CI/CD tools.
  • Strong debugging, performance optimization, and troubleshooting skills.
  • Experience with containerization (Docker, Kubernetes).

  • Business Consulting-Technical analyst with ETL,GCP using Pyspark

    6 days ago

    Noida, Uttar Pradesh, India NTT DATA North America Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Req ID: NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Business Consulting-Technical analyst with ETL,GCP using Pyspark to join our team in Noida, Uttar Pradesh (IN-UP), India...

  • Python Full Stack Developer

    4 days ago

    Noida, Uttar Pradesh, India Tata Consultancy Services Full time ₹ 18,00,000 - ₹ 25,00,000 per year

    Greetings from TCS Walk-in Interview for Python Full Stack DeveloperExperience Range : 4- 6 yearsJob Location: BangaloreJob Description:Must-Have Skills· Strong hands-on experience with Databricks and Apache Airflow for data orchestration.· Expertise in PySpark, SQL, and Python for large-scale data processing.· Proven experience in building and...

  • Azure Data Engineer – Microsoft Fabric Specialist

    4 days ago

    Noida, Uttar Pradesh, India Eice Technology Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Job Opening: Azure Data Engineer Microsoft Fabric SpecialistLocation: Noida, Sector 62 (Hybrid)Experience: 5+ YearsJoin: ImmediateAbout the Role:We are looking for an experienced Azure Data Engineer with strong expertise in Microsoft Fabric and modern data engineering practices. The ideal candidate will be skilled in developing scalable data pipelines,...

  • Cloud Data Engineer

    4 days ago

    Noida, Uttar Pradesh, India SWITS DIGITAL Private Limited Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Job Title:Cloud Data EngineerLocation:Noida, IndiaExperience:3–6 YearsAbout The RoleWe are seeking a talentedData Engineerto design, develop, and optimize enterprise-scale data platforms. The role requires strong expertise inDatabricks, PySpark, ETL pipelines, GCP services, Unity Catalog, and Terraform. The ideal candidate will collaborate with architects...

  • Data Engineer

    2 weeks ago

    Noida, Uttar Pradesh, India EICE Technology Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Role Overview:We are looking for a highly skilledSenior Data Engineer 7+ years of experience as a Microsoft Fabric Specialistto lead the design, development, and optimization of modern data solutions. The role involves working extensively withAzure Fabric, ADF, Databricks, SSIS, Power BI, and PySparkto build scalable, high-performance data pipelines and...

  • Senior Data Engineer

    2 weeks ago

    Noida, Uttar Pradesh, India Eice Technology Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Job Title: Senior Data Engineer, Microsoft Fabric + SSIS SpecialistLocation: Noida, Sector 62 (Onsite, Full-Time, Long-Term Project)Experience Required: 7 + YearsAvailability: Immediate Joiners OnlyRole Overview:We are looking for a highly skilled Senior Data Engineer 7+ years of experience as a Microsoft Fabric Specialist to lead the design, development,...

  • Databricks Architect

    2 weeks ago

    Noida, Uttar Pradesh, India Jellyfish Technologies Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Position - Databricks ArchitectExperience yearsPosition Summary:We are looking for a Databricks Architect with 8-10 years of experience in designing, building, and optimizing data solutions on the Databricks platform. The ideal candidate will have a deep understanding of Databricks and its core functionalities, including Apache Spark, Delta Lake, and MLflow....

  • Azure Data Engineer – MS Fabric Specialist

    2 weeks ago

    Noida, Uttar Pradesh, India EICE Technology Full time ₹ 1,50,000 - ₹ 28,00,000 per year

    Job Title: Azure Data Engineer – MS Fabric SpecialistLocation: Noida, Sector 62 (Onsite)Experience: 6+ YearsAbout the Role:We are seeking anexperienced Azure Data Engineerwith strong expertise inMicrosoft Fabricto design and deliver modern data solutions. The ideal candidate will be responsible for building and optimizingdata pipelines, integrating...

  • Azure Databricks

    2 days ago

    Noida, Uttar Pradesh, India WNS Global Services Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Company DescriptionWNS (Holdings) Limited (NYSE: WNS), is a leading Business Process Management (BPM) company. We combine our deep industry knowledge with technology and analytics expertise to co-create innovative, digital-led transformational solutions with clients across 10 industries. We enable businesses in Travel, Insurance, Banking and Financial...

  • Azure Databricks

    2 days ago

    Noida, Uttar Pradesh, India WNS Global Services Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Company Description WNS (Holdings) Limited (NYSE: WNS), is a leading Business Process Management (BPM) company. We combine our deep industry knowledge with technology and analytics expertise to co-create innovative, digital-led transformational solutions with clients across 10 industries. We enable businesses in Travel, Insurance, Banking and Financial...