PySpark Developer

2 weeks ago

Bangalore, India Rigel networks Full time

Job Title - Pyspark Developer

Location : Bangalore G.P.O., Karnataka

Job Description :

Lead Data Engineer :

Top Skills :

- Python

- SQL

- AWS

- Spark

- 7+ years relevant work experience in the Data Engineering field

- 5+ years of experience working with Hadoop and Big Data processing frameworks (Hadoop, Spark, Hive, Flink, Airflow etc.)

- 5+ years of Strong experience with relational SQL and at least one programming language such as Python, Scala, or Java

- Experience working in AWS environment primarily EMR, S3, Kinesis, Redshift, Athena, etc.

- Experience building scalable, real-time and high-performance cloud data lake solutions

- Experience with source control tools such as GitHub and related CI/CD processes.

- Experience working with Big Data streaming services such as Kinesis, Kafka, etc.

- Experience working with NoSQL data stores such as HBase, DynamoDB, etc.

- Experience with data warehouses/RDBMS like Snowflake & Teradata

- Ability to debug issue and assist team to resolve the challenges

- Ability to translate to functional requirement to design, develop and deploy (From Data Engg perspective)

Mandatory: Primary skill : Pyspark

Education : B. Tech/M. Tech/MCA/MS/MBA

- 7+ years of experience in the design and implementation of Big Data systems using PySpark, database migration, transformation and integration solutions for any Data warehousing project.

- Must have excellent knowledge in Apache Spark and Python programming experience

- Deep experience in developing data processing tasks using PySpark such as reading data from external sources, merging data, performing data enrichment and loading into target data destinations.

- Experience in deployment and operationalizing the code, knowledge of scheduling tools like Airflow, Control-M etc. is preferred

- Working experience on Cloud technology architecture like AWS ecosystem, Google Cloud, BigQuery etc. is an added advantage

- Understanding of Unix/Linux + Shell Scripting

- Data modeling experience using advanced statistical analysis, unstructured data processing

- Experience with building APIs for provisioning data to downstream systems by leveraging different frameworks.

- Hands-on project experience on Jupyter Notebook/ Zeppelin/ PyCharm etc. IDEs

- Hands-on experience with AWS S3 Filesystem operations

- Good knowledge of Hadoop, Hive, and Cloudera/ Hortonworks Data Platform

- Experience handling CDC operations for huge volumes of data

- Should understand and have operating experience with Agile delivery methodologies

- Should have hands-on experience in the following: data validation, writing unit test cases

- Should have experience in integrating PySpark with downstream and upstream applications through a batch/real-time interface

- Should have experience in fine-tuning process and troubleshooting performance issues

- Should have demonstrated expertise in the development of design documents like HLD, LLD etc.

- Should have experience in leading requirements gathering and developing solution architecture for Data migration/integration initiatives

- Should have experience in handling client interactions at different phases of the projects

- Should have experience in leading a team in a project or a module

- Should be well versed with onsite/offshore model and its challenges

(ref:hirist.tech)

  • Pyspark Developer

    2 months ago

    bangalore, India Tata Consultancy Services Full time

    Greetings from TCS!! TCS is hiring for Data Engineer (Pyspark Developer) Required Skill Set: Design and implementation of Big Data pipelines using PySpark Desired Experience Range: 5 to 8 Years Job Location: Bangalore / Pune Must Have: Mandatory: Primary skill: Pyspark. 5+ years of experience in the design and implementation of Big Data pipelines...

  • Pyspark Developer

    3 weeks ago

    Bangalore, India Tata Consultancy Services Full time

    Greetings from TCS!! TCS is hiring for Data Engineer (Pyspark Developer) Required Skill Set: Design and implementation of Big Data pipelines using PySpark Desired Experience Range: 5 to 8 Years Job Location: Bangalore / Pune Must Have: Mandatory: Primary skill: Pyspark. 5+ years of experience in the design and implementation of Big Data pipelines using...

  • PySpark Developer

    3 weeks ago

    Bangalore, India Rigel networks Full time

    Job Title: PySpark DeveloperRigel Networks is seeking a highly skilled PySpark Developer to join our team. As a key member of our data engineering team, you will be responsible for designing, developing, and deploying scalable, real-time, and high-performance cloud data lake solutions using PySpark.Key Responsibilities:Design and develop data processing...

  • Pyspark Developer

    2 months ago

    bangalore, India Tata Consultancy Services Full time

    Greetings from TCS!!TCS is hiring for Data Engineer (Pyspark Developer)Required Skill Set: Design and implementation of Big Data pipelines using PySparkDesired Experience Range: 5 to 8 YearsJob Location: Bangalore / PuneMust Have:Mandatory: Primary skill: Pyspark.5+ years of experience in the design and implementation of Big Data pipelines using PySpark,...

  • Pyspark Developer

    2 months ago

    bangalore, India Tata Consultancy Services Full time

    Greetings from TCS!!TCS is hiring for Data Engineer (Pyspark Developer)Required Skill Set: Design and implementation of Big Data pipelines using PySparkDesired Experience Range: 5 to 8 YearsJob Location: Bangalore / PuneMust Have:Mandatory: Primary skill: Pyspark.5+ years of experience in the design and implementation of Big Data pipelines using PySpark,...

  • Pyspark Developer

    3 weeks ago

    bangalore, India Tata Consultancy Services Full time

    Greetings from TCS!!TCS is hiring for Data Engineer (Pyspark Developer)Required Skill Set: Design and implementation of Big Data pipelines using PySparkDesired Experience Range: 5 to 8 YearsJob Location: Bangalore / PuneMust Have:Mandatory: Primary skill: Pyspark.5+ years of experience in the design and implementation of Big Data pipelines using PySpark,...

  • Pyspark Developer

    3 days ago

    bangalore, India Tata Consultancy Services Full time

    POSITION :Pyspark developer JOB LOCATION : Bangalore / Chennai / Pune / Delhi EXPERIENCE REQUIREMENT : 4 to 12 Years Mandatory: Primary skill: Pyspark. - 3-6 years of experience in the design and implementation of Big Data pipelines using PySpark, database migration, transformation, and integration solutions for any Data warehousing project. - Must have...

  • Pyspark Developer

    3 weeks ago

    Bangalore, India Tata Consultancy Services Full time

    POSITION :Pyspark developer JOB LOCATION : Bangalore / Chennai / Pune / Delhi EXPERIENCE REQUIREMENT : 4 to 12 Years Mandatory: Primary skill: Pyspark. 3-6 years of experience in the design and implementation of Big Data pipelines using PySpark, database migration, transformation, and integration solutions for any Data warehousing project. Must have...

  • Pyspark Developer

    3 weeks ago

    bangalore, India Tata Consultancy Services Full time

    POSITION :Pyspark developerJOB LOCATION : Bangalore / Chennai / Pune / DelhiEXPERIENCE REQUIREMENT : 4 to 12 YearsMandatory: Primary skill: Pyspark. 3-6 years of experience in the design and implementation of Big Data pipelines using PySpark, database migration, transformation, and integration solutions for any Data warehousing project.Must have excellent...

  • Pyspark Developer

    4 days ago

    bangalore, India Tata Consultancy Services Full time

    POSITION :Pyspark developerJOB LOCATION : Bangalore / Chennai / Pune / DelhiEXPERIENCE REQUIREMENT : 4 to 12 YearsMandatory: Primary skill: Pyspark.- 3-6 years of experience in the design and implementation of Big Data pipelines using PySpark, database migration, transformation, and integration solutions for any Data warehousing project.- Must have excellent...

  • Pyspark Developer

    2 months ago

    bangalore, India Tata Consultancy Services Full time

    Greetings from TCS!! TCS is hiring for Data Engineer (Pyspark Developer) Required Skill Set: Design and implementation of Big Data pipelines using PySpark Desired Experience Range: 5 to 8 Years Job Location: Bangalore / Pune Must Have: Mandatory: Primary skill: Pyspark. 5+ years of experience in the design and implementation of Big Data pipelines using...

  • Pyspark Developer

    3 days ago

    bangalore, India Tata Consultancy Services Full time

    POSITION :Pyspark developer JOB LOCATION : Bangalore / Chennai / Pune / Delhi EXPERIENCE REQUIREMENT : 4 to 12 Years Mandatory: Primary skill: Pyspark. 3-6 years of experience in the design and implementation of Big Data pipelines using PySpark, database migration, transformation, and integration solutions for any Data warehousing project. Must have...

  • Pyspark Developer

    3 weeks ago

    bangalore, India Tata Consultancy Services Full time

    POSITION :Pyspark developerJOB LOCATION : Bangalore / Chennai / Pune / DelhiEXPERIENCE REQUIREMENT : 4 to 12 YearsMandatory: Primary skill: Pyspark. 3-6 years of experience in the design and implementation of Big Data pipelines using PySpark, database migration, transformation, and integration solutions for any Data warehousing project.Must have excellent...

  • PySpark Developer

    3 weeks ago

    Pune/Bangalore/Chennai/Hyderabad/Mysore, India Vtricks Technologies Full time

    Job Title: PySpark DeveloperWe are seeking an experienced PySpark Developer to join our team at Vtricks Technologies. As a PySpark Developer, you will be responsible for designing, developing, and maintaining large-scale data pipelines using PySpark and Azure services.Key Responsibilities:Develop and maintain PySpark-based data pipelines using Azure...

  • Pyspark Developer

    4 weeks ago

    Bangalore City, India Tata Consultancy Services Full time

    POSITION :Pyspark developerJOB LOCATION : Bangalore / Chennai / Pune / DelhiEXPERIENCE REQUIREMENT : 4 to 12 YearsMandatory: Primary skill: Pyspark. 3-6 years of experience in the design and implementation of Big Data pipelines using PySpark, database migration, transformation, and integration solutions for any Data warehousing project.Must have excellent...

  • PySpark Developer

    3 weeks ago

    Pune/Bangalore/Chennai/Hyderabad/Mysore, India Vtricks Technologies Full time

    Job Title: PySpark DeveloperWe are seeking an experienced PySpark Developer to join our team at Vtricks Technologies. As a key member of our Agile team, you will be responsible for designing, developing, and maintaining large-scale PySpark-based data pipelines using Azure services.Key Responsibilities:Develop and maintain PySpark-based data pipelines using...

  • Pyspark Hadoop Developer

    3 days ago

    bangalore, India Tata Consultancy Services Full time

    Dear Candidate, Greetings from TCS Human Resources Team!! Job Role: Pyspark Hadoop Developer Job Locations: Bengaluru Experience Range: 5 to 10 years - 5+yrs of hands on experience in Hadoop Scala, Pyspark, Hive, Spark , Hive/Impala/SQL with Hadoop eco system - Good Data background. Hadoop is the primary skill and PySpark is secondary experience. -...

  • Pyspark Hadoop Developer

    9 hours ago

    bangalore, India Tata Consultancy Services Full time

    Dear Candidate,Greetings from TCS Human Resources Team!!Job Role: Pyspark Hadoop DeveloperJob Locations: BengaluruExperience Range: 5 to 10 years- 5+yrs of hands on experience in Hadoop Scala, Pyspark, Hive, Spark , Hive/Impala/SQL with Hadoop eco system- Good Data background. Hadoop is the primary skill and PySpark is secondary experience.- Experience in...

  • Pyspark Hadoop Developer

    2 days ago

    bangalore, India Tata Consultancy Services Full time

    Dear Candidate,Greetings from TCS Human Resources Team!!Job Role: Pyspark Hadoop DeveloperJob Locations: BengaluruExperience Range: 5 to 10 years5+yrs of hands on experience in Hadoop Scala, Pyspark, Hive, Spark , Hive/Impala/SQL with Hadoop eco systemGood Data background. Hadoop is the primary skill and PySpark is secondary experience.Experience in design...

  • Pyspark Hadoop Developer

    4 days ago

    bangalore, India Tata Consultancy Services Full time

    Dear Candidate,Greetings from TCS Human Resources Team!!Job Role: Pyspark Hadoop DeveloperJob Locations: BengaluruExperience Range: 5 to 10 years- 5+yrs of hands on experience in Hadoop Scala, Pyspark, Hive, Spark , Hive/Impala/SQL with Hadoop eco system- Good Data background. Hadoop is the primary skill and PySpark is secondary experience.- Experience in...