Tcs data Streaming

2 days ago

Pashan Pune Maharashtra, India Response Informatics Full time

JOB DESCRIPTION:
Spark/Scala/PySpark developer who knows how to fully exploit the potential of
our Spark cluster.
Should have ability to clean, transform, and analyze vast amounts of raw data
from various systems using Spark to provide ready-to-use data.

SKILLS:
Pyspark

Scala (with a focus on the functional programming paradigm)

Apache Spark 2.x, 3.x
- Apache Spark RDD API
- Apache Spark SQL DataFrame API
- Apache Spark Streaming API

Spark query tuning and performance optimization

SQL database integration (Postgres, and/or MySQL)

Experience working with HDFS, AWS ( S3, Redshift, EMR, IAM, Polices, Routing)

CI-CD Pipleline, Jenkins, Gitlab /Bitbucket

Deep understanding of distributed systems (e.g. partitioning, replication, consistency,

and consensus)

  • Tcs datastage

    2 days ago

    Pashan, Pune, Maharashtra, India Response Informatics Full time

    Data modelling, Database design, Data ware housing ecosystem. - ETL Platform

  • Data Streaming Engineer_6+ Years_ PAN India

    7 days ago

    Pune, Maharashtra, India Innova ESI Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    We are looking for a Data Streaming Engineer to design, develop, and test real-time data pipelines leveraging Solace, Kafka, Databricks Delta Live Tables (DLT), and Apache Spark Structured Streaming. The role involves building ingestion frameworks, streaming data readers, raw data parsers, transformation pipelines from raw to bronze layers, and publishing...

  • Data Entry Executive

    1 week ago

    Pashan, Pune, Maharashtra, India IFAS PUBLICATIONS Full time

    **Job Description | IFAS Publications | Data Entry Executive** **ABOUT COMPANY**: IFAS Publications is India's No. 1 publisher for reference books in graduate and postgraduate college exams, as well as competitive examinations like **CSIR NET, UGC NET, GATE, SET, PSC, CUET PG, IIT JAM,**and more. IFAS Publications is your trusted companion in the quest for...

  • Data Entry Executive

    7 days ago

    Pashan, Pune, Maharashtra, India IFAS PUBLICATIONS Full time ₹ 2,40,000 - ₹ 3,60,000 per year

    Job Description | IFAS Publications | Data Entry ExecutiveABOUT COMPANY:IFAS Publications is India's No. 1 publisher for reference books in graduate and postgraduate college exams, as well as competitive examinations like CSIR NET, UGC NET, GATE, SET, PSC, CUET PG, IIT JAM, and more. IFAS Publications is your trusted companion in the quest for...

  • Event Streaming

    1 week ago

    Pune, India NMS Consultant Full time

    Job SummaryWe are seeking an experienced Event Streaming & Cloud Integration Developer to design, build, and optimize real-time data streaming and integration solutions across cloud environments. The ideal candidate will have strong hands-on expertise in Kafka, AWS integration services, and cloud-native microservices architectures, with a focus on delivering...

  • Event Streaming

    1 week ago

    pune, India NMS Consultant Full time

    Job SummaryWe are seeking an experienced Event Streaming & Cloud Integration Developer to design, build, and optimize real-time data streaming and integration solutions across cloud environments. The ideal candidate will have strong hands-on expertise in Kafka, AWS integration services, and cloud-native microservices architectures, with a focus on delivering...

  • Event Streaming

    1 week ago

    Pune, India NMS Consultant Full time

    Job SummaryWe are seeking an experienced Event Streaming & Cloud Integration Developer to design, build, and optimize real-time data streaming and integration solutions across cloud environments. The ideal candidate will have strong hands-on expertise in Kafka, AWS integration services, and cloud-native microservices architectures, with a focus on delivering...

  • Event Streaming

    1 week ago

    Pune, India NMS Consultant Full time

    Job SummaryWe are seeking an experienced Event Streaming & Cloud Integration Developer to design, build, and optimize real-time data streaming and integration solutions across cloud environments. The ideal candidate will have strong hands-on expertise in Kafka, AWS integration services, and cloud-native microservices architectures, with a focus on delivering...

  • Event Streaming

    1 week ago

    Pune, India NMS Consultant Full time

    Job Summary We are seeking an experienced Event Streaming & Cloud Integration Developer to design, build, and optimize real-time data streaming and integration solutions across cloud environments. The ideal candidate will have strong hands-on expertise in Kafka , AWS integration services , and cloud-native microservices architectures, with a focus on...

  • Event Streaming

    1 week ago

    Pune, India NMS Consultant Full time

    Job Summary We are seeking an experienced Event Streaming & Cloud Integration Developer to design, build, and optimize real-time data streaming and integration solutions across cloud environments. The ideal candidate will have strong hands-on expertise in Kafka , AWS integration services , and cloud-native microservices architectures, with a focus on...