Pyspark

2 months ago


Bengaluru, India Riverforest Connections Full time

Position Purpose
- For the projects which are implemented on Data SSC platform, we need senior developer for python
- pyspark development to migrate project from spark/scala to python-pyspark._

**Responsibilities**

**Direct Responsibilities**

Design high quality deliverables adhering to business requirement with defined standards and design principles, patterns

Develop and maintain highly scalable, high performance code

Develop/Integrate the code adhering to CI/CD

Provide solutions relevant to existing technology architecture.

Manage & Support the storage platform for the project

Create tests and ensure code coverage is met as per the agreed standards

**Contributing Responsibilities**

To work with members of DataSSC in establishing the best practises related to python-pyspark development

**Technical & Behavioral Competencies**

1. Deep experience in developing data processing task using pyspark such as reading data from external sources, merge data, perform data enrichment and load into target destinations

2. Experience to Big Data ecosystem components (Hive, ZooKeeper, Oozie, HDFS etc.)

3. Produce unit tests and document for pyspark transformations and helper methods

4. Preferred to have ETL background

5. Good knowledge in Unix Shell scripting

6. CI/CD with knowledge of Git Hub and Jenkins

7. Ability to learn new concepts, technologies and tools quickly

8. Solid understanding of Agile Sprint methodologies

9. Ability to communicate effectively in both a technical and non-technical manner

**Skills Required**:
pyspark

**Location**

Mumbai / Chennai / Bangalore/Bengaluru

**Years Of Experience**

4 to 9 Years



  • Bengaluru, India LTIMindtree Full time

    Job Description: Primary Skill Set: Lead PySpark Developer Seconday Skill Set: SAS tools Lead a team of data engineers in the design and implementation of big data solutions using PySpark. Collaborate with cross-functional teams to understand business needs and develop data-driven solutions. Design and implement scalable and robust data pipelines using...


  • Bengaluru, India LTIMindtree Full time

    Job Description: Primary Skill Set: Lead PySpark Developer Seconday Skill Set: SAS tools Lead a team of data engineers in the design and implementation of big data solutions using PySpark. Collaborate with cross-functional teams to understand business needs and develop data-driven solutions. Design and implement scalable and robust data pipelines using...

  • Python/pyspark

    1 week ago


    Bengaluru, India Cognizant Technology Solutions Full time

    **Python/Pyspark Engineer** **Job Location: Bangalore/Willing to relocate to Bangalore** **Exp: 4 to 12 years** **Introduction**: We work with customers who need to improve their capacity to respond on computerized world and empower adaptability, evacuate execution boundaries, empower advancement, and modernize center frameworks and rethink their business...

  • Pyspark Developer

    4 weeks ago


    Bengaluru, India Manvision Technologies Full time

    SNRequired InformationDetails1RolePySpark Developer2Required Technical Skill SetPySpark, Python, HDFS, Hadoop, SQL3No of Requirements024Desired Experience Range5+ yearsLocation of RequirementIndiaDesired Competencies (Technical/Behavioral Competency)Must-HaveSound programming knowledge on PySpark & SQL in terms of processing large amount of semi structured &...

  • Python+Pyspark

    2 weeks ago


    Bengaluru, India Wissen Full time

    Wissen Technology is now hiringPython+PysparkWe are seeking a skilled Python Developer with a strong background in PySpark to develop and optimize data processing applications. The ideal candidate will be responsible for building robust and scalable data processing solutions using Python and...

  • Python, Pyspark

    2 months ago


    Bengaluru, India Ziniosedge Full time

    Python, Pyspark 5 Technology Lead (Data Engineer) BE - 8+ years of industry experience as Lead Developer - Experience in implementing ETL and ELT data pipelines with PySpark - Spark Structured API, SPARK SQL & Spark Performance tuning are highly preferred. - Experience in building data pipelines on data lake or Lakehouse(AWS Databricks) & handling...


  • Bengaluru, India Natobotics Technologies Private Limited Full time

    **INFOSYS IS HIRING PYTHON,PYSPARK WITH AWS FOR BANGALORE & MUMBAI** Exp : 5+ Yrs Role : Python,Pyspark With AWS Location : Bangalore & Mumbai Payroll : Natobotics Client : Infosys Below is JD. Skill Set (any 1 as primary pls mention) 1. Python 2. PySpark 3. AWS 4. SQL 5. Snowflake, 6. DevOps JD details 1. Design and develop data integration...

  • Adb + Pyspark + SQL

    2 months ago


    Bengaluru, India RARR Technologies Full time

    **Job Details - ADB + Pyspark + SQL**: **Job ID**: - RARR Job 3323**Job Title**: - ADB + Pyspark + SQL**Job Type**: - Permanent**Job Location**: - Bangalore**Total Experience**: - 5 - 8 Years**Skills**: - Adb, Sql, Pyspark, Azure Databricks, Synapse Analytics**Posted - **23/05/2023 **Job Description**: **Responsibilities**: - Design, develop, and...

  • Big Data

    2 months ago


    Bengaluru, India AR TECHNOLOGY Full time

    **Pyspark Developer** Exp : 6-9 yrs Location - Bangalore /Chennai **Job Description**: **Key Responsibilities**: MongoDB - Build Machine Learning models - Deployment on Cloud platforms **Must have**: - Total IT / development experience of minimum 4+ years of Big Data. - Deep knowledge of Pyspark and libraries to develop and debug complex data...

  • Pyspark Developer

    2 months ago


    Bengaluru, India Corporate Resources Full time

    Job DescriptionDesigning and developing big data applications using the PySpark framework to meet the needs of the businessWriting and optimizing Spark SQL statements to extract and manipulate large datasetsDeveloping and deploying Spark algorithms to perform data processing and analytics tasks, such as machine learning and graph processingDebugging and...

  • Python+Pyspark

    2 weeks ago


    Bengaluru, India Wissen Full time

    Wissen Technology is now hiringPython+Pyspark We are seeking a skilled Python Developer with a strong background in PySpark to develop and optimize data processing applications. The ideal candidate will be responsible for building robust and scalable data processing solutions using Python and PySpark. Experience–4-8years Location:Bangalore ...

  • Aws Pyspark

    2 weeks ago


    Bengaluru, India Cognizant Full time

    Mandatory skills AWS, Python and PySpark, Spark SQL, NoSQL Data Bricks Experience on AWS and its web service offering S3, Redshift, EC2, EMR, Lambda, CloudWatch, RDS, Step functions, Spark streaming etc. Good knowledge of Configuring and working on Multi node clusters and distributed data processing framework Spark. Hands on experience with EMR Apache Spark...


  • Bengaluru, Karnataka, India Black & White Business Solutions Full time

    **Job Information**: Industry **IT Services*** Province **Karnataka*** City **Bangalore North*** Postal Code **560001*** Country **India*** Apache Pyspark, python, sql JD: Key Skills/Experience: SQL, Python and Pyspark are required with AWS Cloud exposure preferable - Working experience and strong knowledge in Databricks.Knowledge in Cloud...

  • AM-Pyspark Developer

    4 weeks ago


    Bengaluru, India Genpact Full time

    With a startup spirit and 115,000+ curious and courageous minds, we have the expertise to go deep with the world’s biggest brands—and we have fun doing it. We dream in digital, dare in reality, and reinvent the ways companies work to make an impact far bigger than just our bottom line. We’re harnessing the power of technology and humanity to create...

  • AM-Pyspark Developer

    2 weeks ago


    Bengaluru, India Genpact Full time

    With a startup spirit and 115,000+ curious and courageous minds, we have the expertise to go deep with the world’s biggest brands—and we have fun doing it. We dream in digital, dare in reality, and reinvent the ways companies work to make an impact far bigger than just our bottom line. We’re harnessing the power of technology and humanity to create...

  • AM-Pyspark Developer

    4 weeks ago


    Bengaluru, India Genpact Full time

    With a startup spirit and 115,000+ curious and courageous minds, we have the expertise to go deep with the world’s biggest brands—and we have fun doing it. We dream in digital, dare in reality, and reinvent the ways companies work to make an impact far bigger than just our bottom line. We’re harnessing the power of technology and humanity to create...

  • AM-Pyspark Developer

    2 weeks ago


    Bengaluru, India Genpact Full time

    With a startup spirit and 115,000+ curious and courageous minds, we have the expertise to go deep with the world’s biggest brands—and we have fun doing it. We dream in digital, dare in reality, and reinvent the ways companies work to make an impact far bigger than just our bottom line. We’re harnessing the power of technology and humanity to create...

  • Pyspark QA

    7 days ago


    Bengaluru, India Virtusa Full time

    **Skill: Pyspark QA** **Role / Tier: T2/Senior Consultant*** **Required Skills**: Experience with BIG Data technology mentioned below Hadoop Big Data HDFS,PYTHON,SPARK SQL MapReduce with PYSpark. Hands on experience in build CI CD pipelines is required Outstanding coding, debugging and analytical skills,Core problem solving skills and data migration from...

  • Pyspark Lead

    2 weeks ago


    Bengaluru, India CIEL HR Services Full time

    We are hiring for our client-An International IT service providerRole- Pyspark LeadMandatory skill set- Python+Spark+Scala(experience in all 3 skill sets should be strong & mandatory) ,Data Engineering+ AWSExp. Level- 5-12 yrsLocation- Bangalore & HyderabadNotice Period-Immediate to 15daysJob DescriptionDesign and implement scalable and robust ELT processes...

  • Data Engineer

    2 months ago


    Bengaluru, India HNM Solutions Full time

    Note : Only immediate joinerRole : PysparkExperience : 8+ yearsLocation : Hyderabad, ChenniaInterview mode is Walkin driveRequirements :Mandatory skills : (8+ Years of experience in Data engineering with 5+ Years on Pyspark/NoSQL is Mandatory)1. Person should be strong in Pyspark2. Should have hands on in MWAA (Airflow) / AWS EMR(Hadoop, Hive) framework3....