PySpark Developer

3 weeks ago

GurgaonGurugram, India Waytogo Consultants Full time

Responsibilities :

  • Design and develop PySpark applications using Spark DataFrames and SQL to extract, transform, and load (ETL) data.
  • Read data from various sources such as relational databases, data lakes, and cloud storage using PySpark connectors.
  • Clean, transform, and enrich data using PySpark functions for data wrangling and manipulation.
  • Build and maintain robust data pipelines using PySpark for efficient data processing.
  • Optimize PySpark applications for performance and scalability to handle large datasets effectively.
  • Collaborate with engineers and data scientists to understand data requirements and develop data-driven solutions.
  • Write unit tests for PySpark applications to ensure code quality and maintainability.
  • Document PySpark code and applications clearly and concisely for future reference and knowledge sharing.

Qualifications :

  • Bachelor's degree in Computer Science, Information Technology, or a related field (or equivalent experience).
  • 2+ years of experience in developing big data applications using PySpark.
  • Strong proficiency in Python programming language and object-oriented programming concepts.
  • In-depth understanding of Apache Spark architecture, including Spark DataFrames, Spark SQL, and distributed processing.
  • Experience working with big data platforms such as Hadoop, YARN, and data lakes (e.g., AWS S3, Azure Data Lake Storage).
  • Experience with cloud platforms (AWS, Azure, GCP) is a plus.
(ref:hirist.tech)

  • PySpark Developer

    5 days ago

    Gurgaon, Haryana, India Waytogo Consultants Full time

    Responsibilities : Design and develop PySpark applications using Spark DataFrames and SQL to extract, transform, and load (ETL) data. Read data from various sources such as relational databases, data lakes, and cloud storage using PySpark connectors. Clean, transform, and enrich data using PySpark functions for data wrangling and manipulation. Build and...

  • Python +pyspark

    2 months ago

    Gurgaon, India Nityo Infotech Full time

    Mandatory Skills – Python , Pyspark, DataBricks, SQL Primary Skills: •Hands-on Python, PySpark, Databricks, SQL, AWS (S3, Lambda, EC2, RDS), and CI-CD tools. Job description: •6 or more years of experience developing, testing, and implementing major Information Technology programs or projects that are broad in scope or multi-unit/cross-divisional •3...

  • Python +pyspark

    2 months ago

    Gurgaon, India Nityo Infotech Full time

    Mandatory Skills – Python , Pyspark, DataBricks, SQL Primary Skills: •Hands-on Python, PySpark, Databricks, SQL, AWS (S3, Lambda, EC2, RDS), and CI-CD tools. Job description: •6 or more years of experience developing, testing, and implementing major Information Technology programs or projects that are broad in scope or multi-unit/cross-divisional •3...

  • Python +pyspark

    3 weeks ago

    Gurgaon, India Nityo Infotech Full time

    Mandatory Skills – Python , Pyspark, DataBricks, SQL Primary Skills: •Hands-on Python, PySpark, Databricks, SQL, AWS (S3, Lambda, EC2, RDS), and CI-CD tools. Job description: •6 or more years of experience developing, testing, and implementing major Information Technology programs or projects that are broad in scope or multi-unit/cross-divisional •3...

  • Python +pyspark

    3 weeks ago

    gurgaon, India Nityo Infotech Full time

    Mandatory Skills – Python , Pyspark, DataBricks, SQL Primary Skills: •Hands-on Python, PySpark, Databricks, SQL, AWS (S3, Lambda, EC2, RDS), and CI-CD tools. Job description: •6 or more years of experience developing, testing, and implementing major Information Technology programs or projects that are broad in scope or multi-unit/cross-divisional •3...

  • Python +pyspark

    2 months ago

    gurgaon, India Nityo Infotech Full time

    Mandatory Skills – Python , Pyspark, DataBricks, SQL Primary Skills: •Hands-on Python, PySpark, Databricks, SQL, AWS (S3, Lambda, EC2, RDS), and CI-CD tools. Job description: •6 or more years of experience developing, testing, and implementing major Information Technology programs or projects that are broad in scope or multi-unit/cross-divisional •3...

  • Python +pyspark

    5 days ago

    Gurgaon, Haryana, India Nityo Infotech Full time

    Mandatory Skills – Python , Pyspark, DataBricks, SQL Primary Skills:Hands-on Python, PySpark, Databricks, SQL, AWS (S3, Lambda, EC2, RDS), and CI-CD toolsJob description:6 or more years of experience developing, testing, and implementing major Information Technology programs or projects that are broad in scope or multi-unit/cross-divisional 3 or more years...

  • AWS Data Engineer

    3 weeks ago

    Gurgaon/Gurugram, IN True Tech Professionals Full time

    We are seeking a skilled Senior Data Engineer to join our dynamic team.- The ideal candidate should have a strong background in data engineering with expertise in Python, Pyspark, AWS Glue, Athena, Redshift, and SQL.- The successful candidate will play a key role in designing, developing, and maintaining our data infrastructure, ensuring optimal performance...

  • AWS Data Engineer

    2 months ago

    Gurgaon/Gurugram, IN True Tech Professionals Full time

    We are seeking a skilled Senior Data Engineer to join our dynamic team.- The ideal candidate should have a strong background in data engineering with expertise in Python, Pyspark, AWS Glue, Athena, Redshift, and SQL.- The successful candidate will play a key role in designing, developing, and maintaining our data infrastructure, ensuring optimal performance...

  • AWS Data Engineer

    3 weeks ago

    Gurgaon/Gurugram, India True Tech Professionals Full time

    We are seeking a skilled Senior Data Engineer to join our dynamic team.- The ideal candidate should have a strong background in data engineering with expertise in Python, Pyspark, AWS Glue, Athena, Redshift, and SQL.- The successful candidate will play a key role in designing, developing, and maintaining our data infrastructure, ensuring optimal performance...

  • AWS Data Engineer

    2 months ago

    Gurgaon,Gurugram, India True Tech Professionals Full time

    We are seeking a skilled Senior Data Engineer to join our dynamic team.- The ideal candidate should have a strong background in data engineering with expertise in Python, Pyspark, AWS Glue, Athena, Redshift, and SQL.- The successful candidate will play a key role in designing, developing, and maintaining our data infrastructure, ensuring optimal performance...

  • ETL Data Engineer

    2 months ago

    Gurgaon,Gurugram, India Coders Brain Pvt Ltd Full time

    Job Description: ETL - Data Design, Develop and maintain ETL pipelines using Pyspark in Azure Databricks using delta tables.- Create build from Github and release pipeline for Ingestion and Databricks using Azure Devops / Harness- Monitor Performance of ETL Jobs, resolve any issue that arose and improve the performance metrics as needed.- Diagnose system...

  • Data Analyst

    3 weeks ago

    gurgaon, India KPMG India Full time

    Experience: 7-9 yearsKey Skills: Python, SQL, PySparkJob Description:Overview:We are seeking a skilled and experienced Data Analyst to join our dynamic team. The ideal candidate will have 7-9 years of experience in data analysis, with a strong background in Python, SQL, and PySpark. This role is open for candidates in Bangalore, Gurgaon, and...

  • Coforge - Machine Learning Engineer - PySpark/Python

    2 months ago

    Gurgaon/Gurugram, IN COFORGE Full time

    Responsibilities :Model Deployment & Management :- Ensure seamless operation of deployed machine learning models within the RCP platform.- Monitor and troubleshoot model performance for accuracy and efficiency.- Collaborate with data scientists and software engineers to optimize model pipelines for production.Machine Learning Operations (MLOps) :- Utilize...

  • Coforge - Machine Learning Engineer - PySpark/Python

    2 months ago

    Gurgaon/Gurugram, India COFORGE Full time

    Responsibilities :Model Deployment & Management :- Ensure seamless operation of deployed machine learning models within the RCP platform.- Monitor and troubleshoot model performance for accuracy and efficiency.- Collaborate with data scientists and software engineers to optimize model pipelines for production.Machine Learning Operations (MLOps) :- Utilize...

  • Coforge - Machine Learning Engineer - PySpark/Python

    3 weeks ago

    Gurgaon/Gurugram, IN COFORGE Full time

    Responsibilities :Model Deployment & Management :- Ensure seamless operation of deployed machine learning models within the RCP platform.- Monitor and troubleshoot model performance for accuracy and efficiency.- Collaborate with data scientists and software engineers to optimize model pipelines for production.Machine Learning Operations (MLOps) :- Utilize...

  • Coforge - Machine Learning Engineer - PySpark/Python

    3 weeks ago

    Gurgaon/Gurugram, India COFORGE Full time

    Responsibilities :Model Deployment & Management :- Ensure seamless operation of deployed machine learning models within the RCP platform.- Monitor and troubleshoot model performance for accuracy and efficiency.- Collaborate with data scientists and software engineers to optimize model pipelines for production.Machine Learning Operations (MLOps) :- Utilize...

  • Data Engineer

    2 months ago

    Gurgaon, Haryana, India Cognizant Technology Solutions Full time

    Position Summary: The Data engineer is responsible for development & maintenance of the Data Lake within Cognizant. You should be able to collaborate with technology cross-commits and business teams to code, test and deploy scalable solution and adhering to required quality standards. You must be a self-starter, think outside the box and enjoy coming up with...

  • Data Analyst

    4 weeks ago

    Gurugram, India KPMG India Full time

    Experience: 5-9 years Key Skills: Python, SQL, PySpark Job Description: Overview: We are seeking a skilled and experienced Data Analyst to join our dynamic team. The ideal candidate will have 5-9 years of experience in data analysis, with a strong background in Python, SQL, and PySpark. This role is open for candidates in Bangalore, Gurgaon, and Kolkata....

  • Data Analyst

    3 weeks ago

    Gurugram, India KPMG India Full time

    Experience: 7-9 yearsKey Skills: Python, SQL, PySparkJob Description:Overview:We are seeking a skilled and experienced Data Analyst to join our dynamic team. The ideal candidate will have 7-9 years of experience in data analysis, with a strong background in Python, SQL, and PySpark. This role is open for candidates in Bangalore, Gurgaon, and...