Data Streaming

1 week ago

Chennai, India ACE Talent Consulting Full time

ob Description:
Spark/Scala/PySpark developer who knows how to fully exploit the potential of our Spark cluster.

Should have ability to clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data.

**Responsibilities**:
Create Scala/Spark/Pyspark jobs for data transformation and aggregation

Produce unit tests for Spark transformations and helper methods

Write Scaladoc-style documentation with all code

Design data processing pipelines

**Skills**:Pyspark

Scala (with a focus on the functional programming paradigm)

Apache Spark 2.x, 3.x
- Apache Spark RDD API
- Apache Spark SQL DataFrame API
- Apache Spark Streaming API

Spark query tuning and performance optimization

SQL database integration (Postgres, and/or MySQL)

Experience working with HDFS, AWS ( S3, Redshift, EMR, IAM, Polices, Routing)

CI-CD Pipleline, Jenkins, Gitlab /Bitbucket

Deep understanding of distributed systems (e.g. partitioning, replication, consistency, and consensus)

**Salary**: ₹300,000.00 - ₹1,500,000.00 per year

Schedule:

- Day shift
- Monday to Friday

Ability to commute/relocate:

- Chennai, Tamil Nadu: Reliably commute or planning to relocate before starting work (required)

**Experience**:

- total work: 3 years (preferred)

Work Location: One location

  • Data Streaming Specialist

    1 week ago

    Chennai, Bengaluru, Pune, India beBeeKafka Full time ₹ 15,00,000 - ₹ 25,00,000

    Kafka Professional About the Role We are seeking an experienced Kafka professional to join our team. As a key member of our data streaming solutions group, you will be responsible for managing and supporting the Kafka ecosystem. Key Responsibilities Ecosystem Management: Manage and support the Kafka ecosystem, focusing primarily on Level 2 (L2) support...

  • Data Streaming Specialist

    1 week ago

    Pune, Bengaluru / Bangalore, Chennai, India beBeeStreaming Full time ₹ 1,04,000 - ₹ 1,30,878

    Job Title: Data Streaming SpecialistWe are seeking a skilled Data Streaming Specialist to join our team. As a Data Streaming Specialist, you will be responsible for designing and implementing data streaming pipelines using Spark and Flink.The ideal candidate will have experience with Spark and Flink, as well as strong knowledge of data processing concepts...

  • Streaming Data Architect

    7 days ago

    Chennai, Bengaluru, Pune, India beBeeData Full time ₹ 10,00,000 - ₹ 15,00,000

    Job Posting for Data EngineerAbout the Role:We are looking for a highly skilled data engineer to join our team. The ideal candidate will have experience with streaming data and be able to design, develop, and implement data pipelines using Spark.Key Responsibilities:Design, develop, and maintain large-scale data processing systems using Apache...

  • Kafka Data Streaming

    6 days ago

    Chennai, India Vega Intellisoft Full time

    "- Managing Kafka real-time data streaming scenarios in a productive environment - Working with Confluent Kafka Control center - Supporting Kafka components like Broker, Zookeeper, Schema registry - Building complex Kafka KSQL Streams - Setup of Kafka scenarios on different hyperscaler - Design and build Data Flows using different Kafka connectors - Develop...

  • Senior Data Streaming Specialist

    7 days ago

    Pune, Bengaluru / Bangalore, Chennai, India beBeeDataStreaming Full time ₹ 1,04,000 - ₹ 1,30,878

    **Job Overview**We are seeking an experienced professional to join our team as a Data Streaming Expert. As a key member of our organization, you will be responsible for managing and supporting the Kafka ecosystem.

  • Streaming Solutions Architect

    5 days ago

    Chennai, Tamil Nadu, India beBeeKafka Full time ₹ 18,00,000 - ₹ 24,00,000

    Event Streaming Engineer PositionJob SummaryWe seek a skilled Event Streaming Engineer to join our organization. The ideal candidate will design, implement, and deploy event-driven architecture solutions using Kafka.The selected candidate will play a crucial role in ensuring seamless integration with client applications and providing expert support for the...

  • Data Engineer

    4 days ago

    Chennai, Tamil Nadu, India NTT DATA Full time US$ 80,000 - US$ 1,20,000 per year

    Req ID: 336026NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer - Azure to join our team in Chennai, Tamil Nādu (IN-TN), India (IN). "Job Duties: Key Responsibilities:...

  • Senior Data Engineer

    3 days ago

    Chennai, Tamil Nadu, India Eucloid Data Solutions Full time US$ 1,50,000 - US$ 2,00,000 per year

    About The Role:Eucloid is looking for a senior Data Engineer with hands-on expertise in Databricks to join our Data Platform team supporting various business applications. The ideal candidate will support development of data infrastructure on Databricks for our clients by participating in activities which may include starting from up- stream and down-stream...

  • Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India NTT DATA North America Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Req ID: NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer - Azure to join our team in Chennai, Tamil Nādu (IN-TN), India (IN)."Job Duties: Key Responsibilities:•  ...

  • Data Engineer

    4 days ago

    Chennai, Tamil Nadu, India NTT DATA North America Full time US$ 80,000 - US$ 1,20,000 per year

    Req ID:336025NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.We are currently seeking a Data Engineer - Azure to join our team in Chennai, Tamil Nādu (IN-TN), India (IN)."Job Duties: Key...