Data Streaming

2 days ago

Pune, India ACE Talent Consulting Full time

ob Description:
Spark/Scala/PySpark developer who knows how to fully exploit the potential of our Spark cluster.

Should have ability to clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data.

**Responsibilities**:
Create Scala/Spark/Pyspark jobs for data transformation and aggregation

Produce unit tests for Spark transformations and helper methods

Write Scaladoc-style documentation with all code

Design data processing pipelines

**Skills**:Pyspark

Scala (with a focus on the functional programming paradigm)

Apache Spark 2.x, 3.x
- Apache Spark RDD API
- Apache Spark SQL DataFrame API
- Apache Spark Streaming API

Spark query tuning and performance optimization

SQL database integration (Postgres, and/or MySQL)

Experience working with HDFS, AWS ( S3, Redshift, EMR, IAM, Polices, Routing)

CI-CD Pipleline, Jenkins, Gitlab /Bitbucket

Deep understanding of distributed systems (e.g. partitioning, replication, consistency, and consensus)

**Salary**: ₹300,000.00 - ₹1,500,000.00 per year

Schedule:

- Day shift
- Monday to Friday

Ability to commute/relocate:

- Pune, Maharashtra: Reliably commute or planning to relocate before starting work (required)

**Experience**:

- total work: 3 years (preferred)

Work Location: One location

  • Data Engineer

    2 weeks ago

    Pune, Maharashtra, India NTT DATA Full time

    Req ID 324612NTT DATA strives to hire exceptional innovative and passionate individuals who want to grow with us If you want to be part of an inclusive adaptable and forward-thinking organization apply now We are currently seeking a Data Engineer to join our team in Pune Mahxc4x81rxc4x81shtra IN-MH India IN Key Responsibilities Design and...

  • Data Streaming

    2 days ago

    Pune, India ACE Talent Consulting Full time

    Job Description: Spark/Scala/PySpark developer who knows how to fully exploit the potential of our Spark cluster. Should have ability to clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data. **Responsibilities**: Create Scala/Spark/Pyspark jobs for data transformation and aggregation Produce...

  • Big Data Senior Developer

    13 hours ago

    Pune, India NTT DATA Full time

    Job Description Req ID:     NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Big Data Senior Developer to join our team in Pune, Mahārāshtra (IN-MH), India (IN).  •...

  • Big Data Senior Developer

    18 hours ago

    Pune, India NTT DATA Full time

    Job Description Req ID:     NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Big Data Senior Developer to join our team in Pune, Mahārāshtra (IN-MH), India (IN).  • Bachelor...

  • Cloud Solution Delivery Senior Consultant

    3 weeks ago

    Pune, Maharashtra, India NTT Data Full time

    Job DescriptionNTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Cloud Solution Delivery Senior Consultant to join our team in Pune, Mahrshtra (IN-MH), India (IN).AWS DevOps EngineerJob...

  • Cloud Solution Delivery Senior Consultant

    2 weeks ago

    Pune, India NTT DATA Full time

    Req ID: NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Cloud Solution Delivery Senior Consultant to join our team in Pune, Mahārāshtra (IN-MH), India (IN). AWS DevOps...

  • Cloud Solution Delivery Senior Consultant

    2 days ago

    Pune, Maharashtra, India NTT DATA Full time US$ 1,04,000 - US$ 1,30,878 per year

    Req ID: 339869NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Cloud Solution Delivery Senior Consultant to join our team in Pune, Mahārāshtra (IN-MH), India (IN). AWS DevOps...

  • Urgent: Cloud Solution Delivery Senior Consultant

    2 weeks ago

    Pune, India NTT Data Full time

    Job Description NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Cloud Solution Delivery Senior Consultant to join our team in Pune, Mahrshtra (IN-MH), India (IN). AWS DevOps...

  • Data Engineer

    4 weeks ago

    Pune, Maharashtra, India Maveric Systems Limited Full time

    Real-Time Data Engineer is responsible for designing and implementing scalable, real time data processing systems in Kafka. This role involves build of KAFKA cluster, ensuring high availability, and integrating with other data processing tools and platformsAs part of the CTO Data Ingestion Service, the incumbent will be required to:Building scalable,...

  • Big Data Developer

    2 weeks ago

    Pune, India Emergys Full time

    Experience: 5 to 7 years Location: Pune, Kothrud Job Description: Must Have: - Proficiency in at least one of the following programming languages: Java/Scala/Python - Good understanding of SQL - Experience of development and deployment of at least one end-to-end data storage/processing pipeline - Strong Experience in Spark development with batch and...