Developer - PySpark

4 days ago

Noida, Uttar Pradesh, India Compunnel Full time ₹ 15,00,000 - ₹ 25,00,000 per year

We are looking for skilled PySparkDevelopers to design and develop scalable data processing solutions. Therole involves working with big data platforms, building ETL pipelines,and collaborating with cross-functional teams to ensure data availability,performance, and quality.

Qualification:BE / B. TechLocation:Pune,

Key Responsibilities

  • Design, develop, and optimize ETL/ELT pipelines using PySpark.
  • Ingest, clean, transform, and process structured/semi-structured/unstructured data.
  • Work with large-scale datasets on distributed computing platforms (Hadoop, Spark, Databricks).
  • Integrate data from multiple sources including Delta Lake, Data Lake, RDBMS, APIs.
  • Ensure high-performance, scalability, and reliability of data pipelines.
  • Collaborate with data engineers, analysts, and data scientists to deliver business-ready datasets.
  • Implement monitoring, logging, and error-handling for data pipelines.
  • Contribute to data quality frameworks and best practices

Required Skills & Experience

  • Bachelors degree in computer science, computer engineering or similar.
  • Excellent communication skills.
  • 2-10 years of experience in data engineering with strong hands-on expertise in PySpark.
  • Proficiency in Python programming for data manipulation and pipeline development.
  • Strong knowledge of Spark SQL, DataFrames, and RDDs.
  • Experience with big data ecosystems Hadoop, Hive, or Databricks.
  • Good understanding of SQL and relational databases (PostgreSQL, MySQL, Oracle).
  • Familiarity with cloud platforms (AWS, Azure, GCP) and services (S3, ADLS, BigQuery).
  • Experience with version control (Git) and CI/CD tools.
  • Strong debugging, performance optimization, and troubleshooting skills.
  • Experience with containerization (Docker, Kubernetes).

  • Pyspark Developer

    2 days ago

    Noida, Uttar Pradesh, India Zigsaw Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Roles & Responsibilities: Design development and implementation of performant ETL pipelines using python API (pySpark) of Apache Spark on AWS EMR Writing reusable testable and efficient code Integration of data storage solutions in spark & ndash especially with AWS S3 object storage. Performance tuning of pySpark scripts. Need to ensure overall build...

  • Python Pyspark Developer

    2 days ago

    Noida, Uttar Pradesh, India Zigsaw Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Roles & Responsibilities: Design, development and implementation of performant ETL pipelines using python API (pySpark) of Apache Spark on AWS EMR Writing reusable, testable, and efficient code Integration of data storage solutions in spark – especially with AWS S3 object storage. Performance tuning of pySpark scripts. Need to ensure overall build delivery...

  • Technical Analyst with ETL, GCP using Pyspark

    6 days ago

    Noida, Uttar Pradesh, India NTT DATA Global Delivery Services Ltd Full time ₹ 1,00,00,000 - ₹ 2,00,00,000 per year

    Business Consulting-Technical analyst with ETL,GCP using Pyspark Req ID: 335235 NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Business Consulting-Technical analyst with...

  • Business Consulting-Technical analyst with ETL,GCP using Pyspark

    6 days ago

    Noida, Uttar Pradesh, India NTT DATA North America Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Req ID: NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Business Consulting-Technical analyst with ETL,GCP using Pyspark to join our team in Noida, Uttar Pradesh (IN-UP), India...

  • Sr. Databricks Developer

    2 days ago

    Noida, Uttar Pradesh, India Topcoder Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Primary skill: Azure Databricks, SQL, PySparkAdditional skills: Power BIDetailed JD: -Extensive experience in designing and building data pipelines within an Azure Databricks Lakehouse environment.Strong understanding and practical experience with Spark and other big data technologies.Integrated spark jobs in Databricks to read and write structured, batch...

  • Python Full Stack Developer

    4 days ago

    Noida, Uttar Pradesh, India Tata Consultancy Services Full time ₹ 18,00,000 - ₹ 25,00,000 per year

    Greetings from TCS Walk-in Interview for Python Full Stack DeveloperExperience Range : 4- 6 yearsJob Location: BangaloreJob Description:Must-Have Skills· Strong hands-on experience with Databricks and Apache Airflow for data orchestration.· Expertise in PySpark, SQL, and Python for large-scale data processing.· Proven experience in building and...

  • Data Scientist

    2 days ago

    Noida, Uttar Pradesh, India Growel Softech Pvt. Ltd. Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    About the Role : We are looking for an experienced Data Scientist to lead the design and development of AI/ML solutions leveraging Azure Machine Learning and Generative AI frameworks. The ideal candidate will have hands-on expertise in Python, PySpark, Azure ML, and OpenAI, with a proven track record of building, deploying, and optimizing data-driven...

  • Group Manager

    2 weeks ago

    Noida, Uttar Pradesh, India WNS Global Services Pvt Ltd Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Azure Databricks (Python/PySpark, SQL), and GCP Group Manager-Analytics Company Description WNS (Holdings) Limited (NYSE: WNS), is a leading Business Process Management (BPM) company. We combine our deep industry knowledge with technology and analytics expertise to co-create innovative, digital-led transformational solutions with clients across 10...

  • Associate- BIM

    2 weeks ago

    Noida, Uttar Pradesh, India Axtria - Ingenious Insights Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Position SummaryTo be a driven business analyst who can work on complex Analytical problems and help the customer in better business decision making especially in the area of pharma/life sciences (domain).Job ResponsibilitiesEngage with Client to participate in requirement gathering, Status update on work, UAT and be the key partner in the overall...

  • Azure Data Engineer – Microsoft Fabric Specialist

    4 days ago

    Noida, Uttar Pradesh, India Eice Technology Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Job Opening: Azure Data Engineer Microsoft Fabric SpecialistLocation: Noida, Sector 62 (Hybrid)Experience: 5+ YearsJoin: ImmediateAbout the Role:We are looking for an experienced Azure Data Engineer with strong expertise in Microsoft Fabric and modern data engineering practices. The ideal candidate will be skilled in developing scalable data pipelines,...