PySpark Developer

3 weeks ago

Bangalore, India Rigel networks Full time
Job Title: PySpark Developer

Rigel Networks is seeking a highly skilled PySpark Developer to join our team. As a key member of our data engineering team, you will be responsible for designing, developing, and deploying scalable, real-time, and high-performance cloud data lake solutions using PySpark.

Key Responsibilities:
  • Design and develop data processing tasks using PySpark, including reading data from external sources, merging data, performing data enrichment, and loading into target data destinations.
  • Deploy and operationalize code, with knowledge of scheduling tools like Airflow and Control-M.
  • Work on cloud technology architecture, including AWS ecosystem, Google Cloud, and BigQuery.
  • Develop data modeling experience using advanced statistical analysis and unstructured data processing.
  • Build APIs for provisioning data to downstream systems using different frameworks.
  • Work with Jupyter Notebook, Zeppelin, and PyCharm IDEs.
  • Experience with AWS S3 Filesystem operations and handling CDC operations for huge volumes of data.
  • Understand and operate Agile delivery methodologies.
  • Develop data validation and writing unit test cases.
  • Integrate PySpark with downstream and upstream applications through batch/real-time interfaces.
  • Fine-tune processes and troubleshoot performance issues.
  • Develop design documents like HLD and LLD.
  • Lead requirements gathering and developing solution architecture for data migration/integration initiatives.
  • Handle client interactions at different phases of projects.
  • Lead a team in a project or module.
  • Be well-versed with onsite/offshore models and their challenges.
Requirements:
  • 7+ years of experience in designing and implementing Big Data systems using PySpark, database migration, transformation, and integration solutions for data warehousing projects.
  • Excellent knowledge of Apache Spark and Python programming experience.
  • Deep experience in developing data processing tasks using PySpark.
  • Experience in deployment and operationalizing code.
  • Working experience on cloud technology architecture.
  • Understanding of Unix/Linux and Shell Scripting.
  • Data modeling experience using advanced statistical analysis and unstructured data processing.
  • Experience with building APIs for provisioning data to downstream systems.
  • Hands-on experience with AWS S3 Filesystem operations and handling CDC operations for huge volumes of data.
  • Good knowledge of Hadoop, Hive, and Cloudera/Hortonworks Data Platform.
  • Experience handling client interactions at different phases of projects.
  • Experience leading a team in a project or module.
What We Offer:
  • A competitive salary and benefits package.
  • A dynamic and supportive work environment.
  • Opportunities for professional growth and development.
  • A collaborative and innovative team culture.

  • Pyspark Developer

    2 months ago

    bangalore, India Tata Consultancy Services Full time

    Greetings from TCS!! TCS is hiring for Data Engineer (Pyspark Developer) Required Skill Set: Design and implementation of Big Data pipelines using PySpark Desired Experience Range: 5 to 8 Years Job Location: Bangalore / Pune Must Have: Mandatory: Primary skill: Pyspark. 5+ years of experience in the design and implementation of Big Data pipelines...

  • Pyspark Developer

    3 weeks ago

    Bangalore, India Tata Consultancy Services Full time

    Greetings from TCS!! TCS is hiring for Data Engineer (Pyspark Developer) Required Skill Set: Design and implementation of Big Data pipelines using PySpark Desired Experience Range: 5 to 8 Years Job Location: Bangalore / Pune Must Have: Mandatory: Primary skill: Pyspark. 5+ years of experience in the design and implementation of Big Data pipelines using...

  • Pyspark Developer

    2 months ago

    bangalore, India Tata Consultancy Services Full time

    Greetings from TCS!!TCS is hiring for Data Engineer (Pyspark Developer)Required Skill Set: Design and implementation of Big Data pipelines using PySparkDesired Experience Range: 5 to 8 YearsJob Location: Bangalore / PuneMust Have:Mandatory: Primary skill: Pyspark.5+ years of experience in the design and implementation of Big Data pipelines using PySpark,...

  • Pyspark Developer

    3 weeks ago

    bangalore, India Tata Consultancy Services Full time

    Greetings from TCS!!TCS is hiring for Data Engineer (Pyspark Developer)Required Skill Set: Design and implementation of Big Data pipelines using PySparkDesired Experience Range: 5 to 8 YearsJob Location: Bangalore / PuneMust Have:Mandatory: Primary skill: Pyspark.5+ years of experience in the design and implementation of Big Data pipelines using PySpark,...

  • Pyspark Developer

    2 months ago

    bangalore, India Tata Consultancy Services Full time

    Greetings from TCS!!TCS is hiring for Data Engineer (Pyspark Developer)Required Skill Set: Design and implementation of Big Data pipelines using PySparkDesired Experience Range: 5 to 8 YearsJob Location: Bangalore / PuneMust Have:Mandatory: Primary skill: Pyspark.5+ years of experience in the design and implementation of Big Data pipelines using PySpark,...

  • Pyspark Developer

    2 days ago

    bangalore, India Tata Consultancy Services Full time

    POSITION :Pyspark developer JOB LOCATION : Bangalore / Chennai / Pune / Delhi EXPERIENCE REQUIREMENT : 4 to 12 Years Mandatory: Primary skill: Pyspark. - 3-6 years of experience in the design and implementation of Big Data pipelines using PySpark, database migration, transformation, and integration solutions for any Data warehousing project. - Must have...

  • Pyspark Developer

    3 weeks ago

    Bangalore, India Tata Consultancy Services Full time

    POSITION :Pyspark developer JOB LOCATION : Bangalore / Chennai / Pune / Delhi EXPERIENCE REQUIREMENT : 4 to 12 Years Mandatory: Primary skill: Pyspark. 3-6 years of experience in the design and implementation of Big Data pipelines using PySpark, database migration, transformation, and integration solutions for any Data warehousing project. Must have...

  • Pyspark Developer

    3 weeks ago

    bangalore, India Tata Consultancy Services Full time

    POSITION :Pyspark developerJOB LOCATION : Bangalore / Chennai / Pune / DelhiEXPERIENCE REQUIREMENT : 4 to 12 YearsMandatory: Primary skill: Pyspark. 3-6 years of experience in the design and implementation of Big Data pipelines using PySpark, database migration, transformation, and integration solutions for any Data warehousing project.Must have excellent...

  • Pyspark Developer

    4 days ago

    bangalore, India Tata Consultancy Services Full time

    POSITION :Pyspark developerJOB LOCATION : Bangalore / Chennai / Pune / DelhiEXPERIENCE REQUIREMENT : 4 to 12 YearsMandatory: Primary skill: Pyspark.- 3-6 years of experience in the design and implementation of Big Data pipelines using PySpark, database migration, transformation, and integration solutions for any Data warehousing project.- Must have excellent...

  • Pyspark Developer

    2 months ago

    bangalore, India Tata Consultancy Services Full time

    Greetings from TCS!! TCS is hiring for Data Engineer (Pyspark Developer) Required Skill Set: Design and implementation of Big Data pipelines using PySpark Desired Experience Range: 5 to 8 Years Job Location: Bangalore / Pune Must Have: Mandatory: Primary skill: Pyspark. 5+ years of experience in the design and implementation of Big Data pipelines using...

  • Pyspark Developer

    2 days ago

    bangalore, India Tata Consultancy Services Full time

    POSITION :Pyspark developer JOB LOCATION : Bangalore / Chennai / Pune / Delhi EXPERIENCE REQUIREMENT : 4 to 12 Years Mandatory: Primary skill: Pyspark. 3-6 years of experience in the design and implementation of Big Data pipelines using PySpark, database migration, transformation, and integration solutions for any Data warehousing project. Must have...

  • Pyspark Developer

    3 weeks ago

    bangalore, India Tata Consultancy Services Full time

    POSITION :Pyspark developerJOB LOCATION : Bangalore / Chennai / Pune / DelhiEXPERIENCE REQUIREMENT : 4 to 12 YearsMandatory: Primary skill: Pyspark. 3-6 years of experience in the design and implementation of Big Data pipelines using PySpark, database migration, transformation, and integration solutions for any Data warehousing project.Must have excellent...

  • PySpark Developer

    3 weeks ago

    Pune/Bangalore/Chennai/Hyderabad/Mysore, India Vtricks Technologies Full time

    Job Title: PySpark DeveloperWe are seeking an experienced PySpark Developer to join our team at Vtricks Technologies. As a PySpark Developer, you will be responsible for designing, developing, and maintaining large-scale data pipelines using PySpark and Azure services.Key Responsibilities:Develop and maintain PySpark-based data pipelines using Azure...

  • Pyspark Developer

    4 weeks ago

    Bangalore City, India Tata Consultancy Services Full time

    POSITION :Pyspark developerJOB LOCATION : Bangalore / Chennai / Pune / DelhiEXPERIENCE REQUIREMENT : 4 to 12 YearsMandatory: Primary skill: Pyspark. 3-6 years of experience in the design and implementation of Big Data pipelines using PySpark, database migration, transformation, and integration solutions for any Data warehousing project.Must have excellent...

  • PySpark Developer

    3 weeks ago

    Pune/Bangalore/Chennai/Hyderabad/Mysore, India Vtricks Technologies Full time

    Job Title: PySpark DeveloperWe are seeking an experienced PySpark Developer to join our team at Vtricks Technologies. As a key member of our Agile team, you will be responsible for designing, developing, and maintaining large-scale PySpark-based data pipelines using Azure services.Key Responsibilities:Develop and maintain PySpark-based data pipelines using...

  • Pyspark Hadoop Developer

    2 days ago

    bangalore, India Tata Consultancy Services Full time

    Dear Candidate, Greetings from TCS Human Resources Team!! Job Role: Pyspark Hadoop Developer Job Locations: Bengaluru Experience Range: 5 to 10 years - 5+yrs of hands on experience in Hadoop Scala, Pyspark, Hive, Spark , Hive/Impala/SQL with Hadoop eco system - Good Data background. Hadoop is the primary skill and PySpark is secondary experience. -...

  • Pyspark developer

    3 weeks ago

    Bangalore, India Tata Consultancy Services Full time

    POSITION : Pyspark developer JOB LOCATION : Bangalore / Chennai / Pune / Delhi EXPERIENCE REQUIREMENT : 4 to 12 Years Mandatory: Primary skill: Pyspark. 3-6 years of experience in the design and implementation of Big Data pipelines using Py Spark, database migration, transformation, and integration solutions for any Data warehousing project. Must...

  • Pyspark Hadoop Developer

    2 days ago

    bangalore, India Tata Consultancy Services Full time

    Dear Candidate,Greetings from TCS Human Resources Team!!Job Role: Pyspark Hadoop DeveloperJob Locations: BengaluruExperience Range: 5 to 10 years5+yrs of hands on experience in Hadoop Scala, Pyspark, Hive, Spark , Hive/Impala/SQL with Hadoop eco systemGood Data background. Hadoop is the primary skill and PySpark is secondary experience.Experience in design...

  • Pyspark Hadoop Developer

    5 hours ago

    bangalore, India Tata Consultancy Services Full time

    Dear Candidate,Greetings from TCS Human Resources Team!!Job Role: Pyspark Hadoop DeveloperJob Locations: BengaluruExperience Range: 5 to 10 years- 5+yrs of hands on experience in Hadoop Scala, Pyspark, Hive, Spark , Hive/Impala/SQL with Hadoop eco system- Good Data background. Hadoop is the primary skill and PySpark is secondary experience.- Experience in...

  • Pyspark Hadoop Developer

    4 days ago

    bangalore, India Tata Consultancy Services Full time

    Dear Candidate,Greetings from TCS Human Resources Team!!Job Role: Pyspark Hadoop DeveloperJob Locations: BengaluruExperience Range: 5 to 10 years- 5+yrs of hands on experience in Hadoop Scala, Pyspark, Hive, Spark , Hive/Impala/SQL with Hadoop eco system- Good Data background. Hadoop is the primary skill and PySpark is secondary experience.- Experience in...