Data Streaming

7 days ago

Bengaluru, India ACE Talent Consulting Full time

ob Description:
Spark/Scala/PySpark developer who knows how to fully exploit the potential of our Spark cluster.

Should have ability to clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data.

**Responsibilities**:
Create Scala/Spark/Pyspark jobs for data transformation and aggregation

Produce unit tests for Spark transformations and helper methods

Write Scaladoc-style documentation with all code

Design data processing pipelines

**Skills**:Pyspark

Scala (with a focus on the functional programming paradigm)

Apache Spark 2.x, 3.x
- Apache Spark RDD API
- Apache Spark SQL DataFrame API
- Apache Spark Streaming API

Spark query tuning and performance optimization

SQL database integration (Postgres, and/or MySQL)

Experience working with HDFS, AWS ( S3, Redshift, EMR, IAM, Polices, Routing)

CI-CD Pipleline, Jenkins, Gitlab /Bitbucket

Deep understanding of distributed systems (e.g. partitioning, replication, consistency, and consensus)

**Salary**: ₹300,000.00 - ₹1,500,000.00 per year

Schedule:

- Day shift
- Monday to Friday

Ability to commute/relocate:

- Bengaluru, Karnataka: Reliably commute or planning to relocate before starting work (required)

**Experience**:

- total work: 3 years (preferred)

Work Location: One location

  • Data Streaming Architect

    2 days ago

    Bengaluru, Karnataka, India beBeeKafka Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Kafka Integration Specialist Our organization is seeking a skilled Kafka Integration Specialist to play a pivotal role in our data streaming architecture. As a key member of our team, you will be responsible for integrating Kafka with other systems such as databases, APIs, and cloud services. Key Responsibilities: Integration: Integrate Kafka with other...

  • Data Streaming Specialist

    7 days ago

    Chennai, Bengaluru, Pune, India beBeeKafka Full time ₹ 15,00,000 - ₹ 25,00,000

    Kafka Professional About the Role We are seeking an experienced Kafka professional to join our team. As a key member of our data streaming solutions group, you will be responsible for managing and supporting the Kafka ecosystem. Key Responsibilities Ecosystem Management: Manage and support the Kafka ecosystem, focusing primarily on Level 2 (L2) support...

  • Data Streaming Specialist

    5 days ago

    Bengaluru, Karnataka, India beBeeIntegration Full time ₹ 1,04,000 - ₹ 1,30,878

    Job DescriptionAs a highly skilled professional in data streaming and integration, you will be responsible for designing, developing, and implementing robust Kafka-based solutions. You will work closely with cross-functional teams to understand integration requirements and deliver high-quality solutions.Key responsibilities include:Designing, implementing,...

  • Data Streaming Specialist

    7 days ago

    Pune, Bengaluru / Bangalore, Chennai, India beBeeStreaming Full time ₹ 1,04,000 - ₹ 1,30,878

    Job Title: Data Streaming SpecialistWe are seeking a skilled Data Streaming Specialist to join our team. As a Data Streaming Specialist, you will be responsible for designing and implementing data streaming pipelines using Spark and Flink.The ideal candidate will have experience with Spark and Flink, as well as strong knowledge of data processing concepts...

  • Senior Data Streaming Specialist

    7 hours ago

    Bengaluru, Karnataka, India beBeeKafka Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Kafka Integration ExpertJob Profile:We are seeking a seasoned Kafka expert to spearhead our data streaming efforts. The successful candidate will have extensive experience with Kafka, including Kafka Connect, Kafka Streams and related technologies.Key Responsibilities:Integrate Kafka with other systems such as databases, APIs, and cloud services, KSQL.Manage...

  • Streaming Data Architect

    6 days ago

    Chennai, Bengaluru, Pune, India beBeeData Full time ₹ 10,00,000 - ₹ 15,00,000

    Job Posting for Data EngineerAbout the Role:We are looking for a highly skilled data engineer to join our team. The ideal candidate will have experience with streaming data and be able to design, develop, and implement data pipelines using Spark.Key Responsibilities:Design, develop, and maintain large-scale data processing systems using Apache...

  • Cloud Data Streaming Specialist

    1 week ago

    Bengaluru, Karnataka, India beBeeKafka Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Expert Data Streamer SoughtWe are on the hunt for a seasoned expert in data streaming who can harness their skills to develop innovative, low-latency solutions.Key Responsibilities:Design and implement scalable software components utilizing Java, .NET, or Node.js.Collaborate with cross-functional teams to design, develop, and deploy cloud-based...

  • Azure Data Architect with Azure Streaming Analytics

    24 hours ago

    Bengaluru, India Black & White Business Solutions Full time

    **Job Information**: Industry **IT Services** *** Province **Karnataka** *** City **Bangalore** *** Postal Code **560001** *** Country **India** - **Azure Data Architect with Azure Streaming Analytics (10-15 Yrs)** Azure Data Architect with streaming experience Total experience in data management area for 10 + years with Azure cloud data platform...

  • Senior Data Streaming Engineer

    1 week ago

    Bengaluru, Karnataka, India beBeeKafka Full time ₹ 9,00,000 - ₹ 12,00,000

    Job DescriptionWe are seeking a highly skilled Kafka Specialist to manage and support our Kafka ecosystem.The ideal candidate will collaborate closely with the Kafka team and suppliers to ensure the reliability and performance of our data streaming solutions.

  • Streaming Systems Architect

    5 days ago

    Bengaluru, Karnataka, India beBeeKafka Full time ₹ 12,50,000 - ₹ 17,50,000

    Job OpportunityWe're seeking a highly skilled Streaming Systems Architect to oversee our data streaming infrastructure. This role demands strong expertise in managing and developing low-latency solutions with Confluent Kafka.Key Responsibilities:Monitor, maintain, and optimize the performance of our Kafka ecosystem.Collaborate with development teams to...