PySpark Developer

1 week ago


Gurgaon, Haryana, India Waytogo Consultants Full time

Responsibilities :

  • Design and develop PySpark applications using Spark DataFrames and SQL to extract, transform, and load (ETL) data.
  • Read data from various sources such as relational databases, data lakes, and cloud storage using PySpark connectors.
  • Clean, transform, and enrich data using PySpark functions for data wrangling and manipulation.
  • Build and maintain robust data pipelines using PySpark for efficient data processing.
  • Optimize PySpark applications for performance and scalability to handle large datasets effectively.
  • Collaborate with engineers and data scientists to understand data requirements and develop data-driven solutions.
  • Write unit tests for PySpark applications to ensure code quality and maintainability.
  • Document PySpark code and applications clearly and concisely for future reference and knowledge sharing.

Qualifications :

  • Bachelor's degree in Computer Science, Information Technology, or a related field (or equivalent experience).
  • 2+ years of experience in developing big data applications using PySpark.
  • Strong proficiency in Python programming language and object-oriented programming concepts.
  • In-depth understanding of Apache Spark architecture, including Spark DataFrames, Spark SQL, and distributed processing.
  • Experience working with big data platforms such as Hadoop, YARN, and data lakes (e.g., AWS S3, Azure Data Lake Storage).
  • Experience with cloud platforms (AWS, Azure, GCP) is a plus.
)
  • Python +pyspark

    1 week ago


    Gurgaon, Haryana, India Nityo Infotech Full time

    Mandatory Skills – Python , Pyspark, DataBricks, SQL Primary Skills:Hands-on Python, PySpark, Databricks, SQL, AWS (S3, Lambda, EC2, RDS), and CI-CD toolsJob description:6 or more years of experience developing, testing, and implementing major Information Technology programs or projects that are broad in scope or multi-unit/cross-divisional 3 or more years...

  • Data Engineer

    2 months ago


    Gurgaon, Haryana, India Cognizant Technology Solutions Full time

    Position Summary: The Data engineer is responsible for development & maintenance of the Data Lake within Cognizant. You should be able to collaborate with technology cross-commits and business teams to code, test and deploy scalable solution and adhering to required quality standards. You must be a self-starter, think outside the box and enjoy coming up with...

  • Python Pyspark

    1 week ago


    Gurgaon, Haryana, India Virtusa Full time

    Data engineer1) Overall 7+ in IT Experience with relevant 5+ years of experience in Data EngineeringRequired technical skills:Python, PySpark, SQL, AWS2)Required Domain: Healthcare Insurance4) Extensive data analysis skills and Good communication skills5) Experience working in AgileAbout VirtusaTeamwork, quality of life, professional and personal...


  • Gurgaon, Haryana, India COFORGE Full time

    Responsibilities :Model Deployment & Management : Ensure seamless operation of deployed machine learning models within the RCP platform. Monitor and troubleshoot model performance for accuracy and efficiency. Collaborate with data scientists and software engineers to optimize model pipelines for production.Machine Learning Operations (MLOps) : Utilize...


  • Gurgaon, Haryana, India NatWest Digital X Full time

    Join us as a Software EngineerThis is an opportunity for a driven Software Engineer to take on an exciting new career challengeDay-to-day, you'll build a wide network of stakeholders of varying levels of seniorityIt's a chance to hone your existing technical skills and advance your careerWe're offering this role at associate levelWhat you'll do In your new...


  • Gurgaon, Haryana, India NatWest Digital X Full time

    Join us as a Software EngineerThis is an opportunity for a driven Software Engineer to design and engineer software with the customer or user experience as the primary objectiveWe'll look to you to engineer and maintain innovative, customer centric, high performance, secure and robust solutionsIt's a chance to hone your existing technical skills and advance...

  • Data Engineer

    2 months ago


    Gurgaon, Haryana, India Cognizant Technology Solutions Full time

    Technical LeadQualification:Bachelors in science , engineering or equivalentResponsibility:Project Planning and Setup: Understand the project scope, identify activities/ tasks, task level estimates, schedule, dependencies, risks and provide inputs to Module Lead for review. Provide inputs to testing strategy, configuration, deployment, hardware/software...

  • Big Data Developer

    1 week ago


    Gurgaon, Haryana, India Impetus Full time

    About ImpetusImpetus Technologies is a digital engineering company focused on delivering expert services and products to help enterprises achieve their transformation goals. We solve the analytics, AI, and cloud puzzle, enabling businesses to drive unmatched innovation and growth.Founded in 1991, we are cloud and data engineering leaders providing solutions...

  • Big Data Developer

    1 week ago


    Gurgaon, Haryana, India Impetus Full time

    Location: Gurgaon, Experience: 4- 6 years Description: We are looking for Big Data developers having minimum experience of 4 year in Hadoop Ecosystem, passionate about technology, motivated for continuous learning and an individual who views every client interaction as an opportunity to create an exceptional customer experience. Qualifications: Must have:...


  • Gurgaon, Haryana, India exl Full time

    Job Description : We are looking for a Pyspark developer who can analyze and transform from various system and has experience with ADF. Design and implement end-to-end data solutions (storage, integration, processing, visualization) in Azure Needs to lead, guide team members by experience Skills and Responsibilities : Create a data transformation...

  • Big Data Engineer

    1 week ago


    Gurgaon, Haryana, India Virtusa Full time

    Big Data Engineer - CREQ190572 Description Qualifications Bachelors degree in Engineering or Computer Science or equivalent OR Masters in Computer Applications or equivalent. 6+ years of software development experience and leading teams of engineers and scrum teams 3+ years of hands-on experience of working with Map-Reduce, Hive, Spark (core, SQL and...

  • Data bricks(Jaipur

    1 week ago


    Gurgaon, Haryana, India HyreFox Consultants Full time

    Overview:As a Data Engineer, you will work with multiple teams to deliver solutions on the Azure Cloud using corecloud data warehouse tools (Azure Data Factory, Azure Databricks, Azure SQL DW and other Big Datarelated technologies). In addition to building the next generation of application data platforms (notinfrastructure) and/or improving recent...

  • Big Data Developer

    1 week ago


    Gurgaon, Haryana, India Impetus Full time

    Years of experience : 4 to 6 Years Location: gurugram Notice period : Immediate to 30 Days Experience with design and coding across one or more platforms and languages (e.g. , Python/Pyspark/SQL, Scala) as appropriateHands-on expertise with application design, software development Proficient in Big Data technologies Designs, codes, tests, corrects and...

  • Big Data Developer

    1 week ago


    Gurgaon, Haryana, India Impetus Full time

    Years of experience : 4 to 6 Years Location: gurugram Notice period : Immediate to 30 Days Experience with design and coding across one or more platforms and languages (e.g. , Python/Pyspark/SQL, Scala) as appropriate Hands-on expertise with application design, software development Proficient in Big Data technologies Designs, codes, tests, corrects and...

  • Python Developer

    1 week ago


    Gurgaon, Haryana, India Huquo Full time

    Job Description : Role - Deputy Manager, Business Performance Management and AnalyticsBand - 5BTeam Size - Individual ContributorWhat will you do in this role? Python application development, testing, and maintenance Working with other developers, designers, and stakeholders to satisfy project needs. Developing and maintaining code and application...


  • Gurgaon, Haryana, India exl Full time

    Job Description Job Summary: Data Engineer (DE) Consultant is responsible for designing, developing, and maintaining data assets and data related products by liaising with multiple stakeholders. Responsibilities: Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes. Create the data...


  • Gurgaon, Haryana, India EXL Full time

    Data Engineer Consultant – Job DescriptionJob Summary:Data Engineer (DE) Consultant is responsible for designing, developing, and maintaining data assets and data related products by liaising with multiple stakeholders.Responsibilities:Work with stakeholders to understand the data requirements to design, develop, and maintain complex ETL processes.Create...

  • AWS Data Engineer

    1 week ago


    Gurgaon, Haryana, India True Tech Professionals Full time

    We are seeking a skilled Senior Data Engineer to join our dynamic team. The ideal candidate should have a strong background in data engineering with expertise in Python, Pyspark, AWS Glue, Athena, Redshift, and SQL. The successful candidate will play a key role in designing, developing, and maintaining our data infrastructure, ensuring optimal performance...


  • Gurgaon, Haryana, India True Tech Professionals Full time

    Senior Data Engineer Location: Gurgaon (Hybrid Mode) Experience: 5 years and above. Immediate Joiners Preferred (Notice Period: days) We are seeking a skilled Senior Data Engineer to join our dynamic team. The ideal candidate should have a strong background in data engineering with expertise in Python, Pyspark, AWS Glue, Athena, Redshift, and SQL. The...

  • Big Data Developer

    1 week ago


    Gurgaon, Haryana, India Impetus Full time

    Location: Gurgaon,Experience: 4- 6 yearsDescription:We are looking for Big Data developers having minimum experience of 4 year in Hadoop Ecosystem, passionate about technology, motivated for continuous learning and an individual who views every client interaction as an opportunity to create an exceptional customer experience.Qualifications:Must have:...