PySpark Developer

2 months ago

gurugram, India Waytogo Consultants Full time

Responsibilities :

  • Design and develop PySpark applications using Spark DataFrames and SQL to extract, transform, and load (ETL) data.
  • Read data from various sources such as relational databases, data lakes, and cloud storage using PySpark connectors.
  • Clean, transform, and enrich data using PySpark functions for data wrangling and manipulation.
  • Build and maintain robust data pipelines using PySpark for efficient data processing.
  • Optimize PySpark applications for performance and scalability to handle large datasets effectively.
  • Collaborate with engineers and data scientists to understand data requirements and develop data-driven solutions.
  • Write unit tests for PySpark applications to ensure code quality and maintainability.
  • Document PySpark code and applications clearly and concisely for future reference and knowledge sharing.

Qualifications :

  • Bachelor's degree in Computer Science, Information Technology, or a related field (or equivalent experience).
  • 2+ years of experience in developing big data applications using PySpark.
  • Strong proficiency in Python programming language and object-oriented programming concepts.
  • In-depth understanding of Apache Spark architecture, including Spark DataFrames, Spark SQL, and distributed processing.
  • Experience working with big data platforms such as Hadoop, YARN, and data lakes (e.g., AWS S3, Azure Data Lake Storage).
  • Experience with cloud platforms (AWS, Azure, GCP) is a plus.
(ref:hirist.tech)

  • PySpark Developer

    2 months ago

    Gurgaon/Gurugram, India Waytogo Consultants Full time

    Responsibilities : Design and develop PySpark applications using Spark DataFrames and SQL to extract, transform, and load (ETL) data. Read data from various sources such as relational databases, data lakes, and cloud storage using PySpark connectors. Clean, transform, and enrich data using PySpark functions for data wrangling and manipulation. Build and...

  • PySpark Developer

    1 week ago

    Gurgaon/Gurugram, India Waytogo Consultants Full time

    Responsibilities : Design and develop PySpark applications using Spark DataFrames and SQL to extract, transform, and load (ETL) data. Read data from various sources such as relational databases, data lakes, and cloud storage using PySpark connectors. Clean, transform, and enrich data using PySpark functions for data wrangling and manipulation. Build and...

  • Data Analyst

    2 weeks ago

    Gurugram, India KPMG India Full time

    Experience: 7-9 yearsKey Skills: Python, SQL, PySparkJob Description:Overview:We are seeking a skilled and experienced Data Analyst to join our dynamic team. The ideal candidate will have 7-9 years of experience in data analysis, with a strong background in Python, SQL, and PySpark. This role is open for candidates in Bangalore, Gurgaon, and...

  • Data Analyst

    3 weeks ago

    Gurugram, India KPMG India Full time

    Experience: 5-9 years Key Skills: Python, SQL, PySpark Job Description: Overview: We are seeking a skilled and experienced Data Analyst to join our dynamic team. The ideal candidate will have 5-9 years of experience in data analysis, with a strong background in Python, SQL, and PySpark. This role is open for candidates in Bangalore, Gurgaon, and Kolkata....

  • Data Analyst

    3 weeks ago

    Gurugram, India KPMG India Full time

    Experience: 5-9 years Key Skills: Python, SQL, PySpark Job Description: Overview: We are seeking a skilled and experienced Data Analyst to join our dynamic team. The ideal candidate will have 5-9 years of experience in data analysis, with a strong background in Python, SQL, and PySpark. This role is open for candidates in Bangalore, Gurgaon, and Kolkata....

  • Data Analyst

    2 weeks ago

    Gurugram, India KPMG India Full time

    Experience: 7-9 yearsKey Skills: Python, SQL, PySparkJob Description:Overview:We are seeking a skilled and experienced Data Analyst to join our dynamic team. The ideal candidate will have 7-9 years of experience in data analysis, with a strong background in Python, SQL, and PySpark. This role is open for candidates in Bangalore, Gurgaon, and...

  • Data Analyst

    1 week ago

    Gurugram, India KPMG India Full time

    Experience: 7-9 yearsKey Skills: Python, SQL, PySparkJob Description:Overview:We are seeking a skilled and experienced Data Analyst to join our dynamic team. The ideal candidate will have 7-9 years of experience in data analysis, with a strong background in Python, SQL, and PySpark. This role is open for candidates in Bangalore, Gurgaon, and...

  • Data Analyst

    2 weeks ago

    Gurugram, India KPMG India Full time

    Experience: 7-9 yearsKey Skills: Python, SQL, PySparkJob Description:Overview:We are seeking a skilled and experienced Data Analyst to join our dynamic team. The ideal candidate will have 7-9 years of experience in data analysis, with a strong background in Python, SQL, and PySpark. This role is open for candidates in Bangalore, Gurgaon, and...

  • Data Analyst

    1 week ago

    Gurugram, India KPMG India Full time

    Experience: 7-9 years Key Skills: Python, SQL, PySpark Job Description: Overview: We are seeking a skilled and experienced Data Analyst to join our dynamic team. The ideal candidate will have 7-9 years of experience in data analysis, with a strong background in Python, SQL, and PySpark. This role is open for candidates in Bangalore, Gurgaon, and Kolkata....

  • Bigdata + Pyspark

    3 days ago

    gurugram, India Impetus Technologies Full time

    BIgdata + Pyspark :  Big Data; Hadoop / HDFS,Pyspark,Python   Experience with design and coding across one or more platforms and languages (e.g. , Python/Pyspark/SQL) as appropriate Hands-on expertise with application design, software development  Proficient in Big Data technologies Designs, codes, tests, corrects and documents large and/or complex...

  • AWS Data Engineer

    2 months ago

    Gurugram, India True Tech Professionals Full time

    We are seeking a skilled Senior Data Engineer to join our dynamic team.- The ideal candidate should have a strong background in data engineering with expertise in Python, Pyspark, AWS Glue, Athena, Redshift, and SQL.- The successful candidate will play a key role in designing, developing, and maintaining our data infrastructure, ensuring optimal performance...

  • AWS Data Engineer

    1 week ago

    Gurugram, India True Tech Professionals Full time

    We are seeking a skilled Senior Data Engineer to join our dynamic team.- The ideal candidate should have a strong background in data engineering with expertise in Python, Pyspark, AWS Glue, Athena, Redshift, and SQL.- The successful candidate will play a key role in designing, developing, and maintaining our data infrastructure, ensuring optimal performance...

  • Python Pyspark

    3 weeks ago

    Gurugram, Haryana, India Virtusa Full time

    Data engineer 1) Overall 7+ in IT Experience with relevant 5+ years of experience in Data Engineering. Required technical skills: Python, PySpark, SQL, AWS 2) Required Domain: Healthcare Insurance 4) Extensive data analysis skills and Good communication skills 5) Experience working in Agile **About Virtusa** Teamwork, quality of life, professional and...

  • Coforge - Machine Learning Engineer - PySpark/Python

    1 month ago

    Gurugram, India COFORGE Full time

    Responsibilities :Model Deployment & Management :- Ensure seamless operation of deployed machine learning models within the RCP platform.- Monitor and troubleshoot model performance for accuracy and efficiency.- Collaborate with data scientists and software engineers to optimize model pipelines for production.Machine Learning Operations (MLOps) :- Utilize...

  • Coforge - Machine Learning Engineer - PySpark/Python

    1 week ago

    Gurugram, India COFORGE Full time

    Responsibilities :Model Deployment & Management :- Ensure seamless operation of deployed machine learning models within the RCP platform.- Monitor and troubleshoot model performance for accuracy and efficiency.- Collaborate with data scientists and software engineers to optimize model pipelines for production.Machine Learning Operations (MLOps) :- Utilize...

  • AWS Data Engineer

    2 months ago

    Gurgaon,Gurugram, India True Tech Professionals Full time

    We are seeking a skilled Senior Data Engineer to join our dynamic team.- The ideal candidate should have a strong background in data engineering with expertise in Python, Pyspark, AWS Glue, Athena, Redshift, and SQL.- The successful candidate will play a key role in designing, developing, and maintaining our data infrastructure, ensuring optimal performance...

  • AWS Data Engineer

    2 weeks ago

    Gurgaon/Gurugram, IN True Tech Professionals Full time

    We are seeking a skilled Senior Data Engineer to join our dynamic team.- The ideal candidate should have a strong background in data engineering with expertise in Python, Pyspark, AWS Glue, Athena, Redshift, and SQL.- The successful candidate will play a key role in designing, developing, and maintaining our data infrastructure, ensuring optimal performance...

  • AWS Data Engineer

    2 months ago

    Gurgaon/Gurugram, IN True Tech Professionals Full time

    We are seeking a skilled Senior Data Engineer to join our dynamic team.- The ideal candidate should have a strong background in data engineering with expertise in Python, Pyspark, AWS Glue, Athena, Redshift, and SQL.- The successful candidate will play a key role in designing, developing, and maintaining our data infrastructure, ensuring optimal performance...

  • AWS Data Engineer

    1 week ago

    Gurgaon/Gurugram, India True Tech Professionals Full time

    We are seeking a skilled Senior Data Engineer to join our dynamic team.- The ideal candidate should have a strong background in data engineering with expertise in Python, Pyspark, AWS Glue, Athena, Redshift, and SQL.- The successful candidate will play a key role in designing, developing, and maintaining our data infrastructure, ensuring optimal performance...

  • ETL Data Engineer

    2 months ago

    Gurgaon,Gurugram, India Coders Brain Pvt Ltd Full time

    Job Description: ETL - Data Design, Develop and maintain ETL pipelines using Pyspark in Azure Databricks using delta tables.- Create build from Github and release pipeline for Ingestion and Databricks using Azure Devops / Harness- Monitor Performance of ETL Jobs, resolve any issue that arose and improve the performance metrics as needed.- Diagnose system...