Data Streaming

4 days ago

Bengaluru, India ACE Talent Consulting Full time

Job Description:
Spark/Scala/PySpark developer who knows how to fully exploit the potential of our Spark cluster.

Should have ability to clean, transform, and analyze vast amounts of raw data from various systems using Spark to provide ready-to-use data.

**Responsibilities**:
Create Scala/Spark/Pyspark jobs for data transformation and aggregation

Produce unit tests for Spark transformations and helper methods

Write Scaladoc-style documentation with all code

Design data processing pipelines

**Skills**:Pyspark

Scala (with a focus on the functional programming paradigm)

Apache Spark 2.x, 3.x
- Apache Spark RDD API
- Apache Spark SQL DataFrame API
- Apache Spark Streaming API

Spark query tuning and performance optimization

SQL database integration (Postgres, and/or MySQL)

Experience working with HDFS, AWS ( S3, Redshift, EMR, IAM, Polices, Routing)

CI-CD Pipleline, Jenkins, Gitlab /Bitbucket

Deep understanding of distributed systems (e.g. partitioning, replication, consistency, and consensus)

**Salary**: ₹300,000.00 - ₹1,500,000.00 per year

Schedule:

- Day shift

Ability to commute/relocate:

- Bengaluru, Karnataka: Reliably commute or planning to relocate before starting work (required)

**Experience**:

- total work: 3 years (preferred)

Work Location: One location

  • Data Streaming Specialist

    3 days ago

    Chennai, Bengaluru, Pune, India beBeeKafka Full time ₹ 15,00,000 - ₹ 25,00,000

    Kafka Professional About the Role We are seeking an experienced Kafka professional to join our team. As a key member of our data streaming solutions group, you will be responsible for managing and supporting the Kafka ecosystem. Key Responsibilities Ecosystem Management: Manage and support the Kafka ecosystem, focusing primarily on Level 2 (L2) support...

  • Data Streaming Specialist

    21 hours ago

    Bengaluru, Karnataka, India beBeeIntegration Full time ₹ 1,04,000 - ₹ 1,30,878

    Job DescriptionAs a highly skilled professional in data streaming and integration, you will be responsible for designing, developing, and implementing robust Kafka-based solutions. You will work closely with cross-functional teams to understand integration requirements and deliver high-quality solutions.Key responsibilities include:Designing, implementing,...

  • Data Streaming Specialist

    3 days ago

    Pune, Bengaluru / Bangalore, Chennai, India beBeeStreaming Full time ₹ 1,04,000 - ₹ 1,30,878

    Job Title: Data Streaming SpecialistWe are seeking a skilled Data Streaming Specialist to join our team. As a Data Streaming Specialist, you will be responsible for designing and implementing data streaming pipelines using Spark and Flink.The ideal candidate will have experience with Spark and Flink, as well as strong knowledge of data processing concepts...

  • Streaming Data Architect

    2 days ago

    Chennai, Bengaluru, Pune, India beBeeData Full time ₹ 10,00,000 - ₹ 15,00,000

    Job Posting for Data EngineerAbout the Role:We are looking for a highly skilled data engineer to join our team. The ideal candidate will have experience with streaming data and be able to design, develop, and implement data pipelines using Spark.Key Responsibilities:Design, develop, and maintain large-scale data processing systems using Apache...

  • Cloud Data Streaming Specialist

    5 days ago

    Bengaluru, Karnataka, India beBeeKafka Full time ₹ 1,80,00,000 - ₹ 2,50,00,000

    Expert Data Streamer SoughtWe are on the hunt for a seasoned expert in data streaming who can harness their skills to develop innovative, low-latency solutions.Key Responsibilities:Design and implement scalable software components utilizing Java, .NET, or Node.js.Collaborate with cross-functional teams to design, develop, and deploy cloud-based...

  • Senior Data Streaming Engineer

    4 days ago

    Bengaluru, Karnataka, India beBeeKafka Full time ₹ 9,00,000 - ₹ 12,00,000

    Job DescriptionWe are seeking a highly skilled Kafka Specialist to manage and support our Kafka ecosystem.The ideal candidate will collaborate closely with the Kafka team and suppliers to ensure the reliability and performance of our data streaming solutions.

  • Streaming Systems Architect

    1 day ago

    Bengaluru, Karnataka, India beBeeKafka Full time ₹ 12,50,000 - ₹ 17,50,000

    Job OpportunityWe're seeking a highly skilled Streaming Systems Architect to oversee our data streaming infrastructure. This role demands strong expertise in managing and developing low-latency solutions with Confluent Kafka.Key Responsibilities:Monitor, maintain, and optimize the performance of our Kafka ecosystem.Collaborate with development teams to...

  • Senior Streaming Data Specialist

    3 days ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 1,04,000 - ₹ 1,30,878

    Job OpportunityWe are seeking a skilled professional with working knowledge of streaming data environments and pipelines.Strong programming skills in at least one language, such as Groovy or Java, are required.Familiarity with data structures, ETL design, and storage solutions is necessary for this role.Experience working with Apache Spark, Streamsets, or...

  • Senior Data Streaming Specialist

    2 days ago

    Pune, Bengaluru / Bangalore, Chennai, India beBeeDataStreaming Full time ₹ 1,04,000 - ₹ 1,30,878

    **Job Overview**We are seeking an experienced professional to join our team as a Data Streaming Expert. As a key member of our organization, you will be responsible for managing and supporting the Kafka ecosystem.

  • Senior Data Streaming Specialist

    23 hours ago

    Bengaluru, Karnataka, India beBeeInnovative Full time ₹ 1,04,000 - ₹ 1,30,878

    Job OpportunitySeeking a skilled professional to take on the role of Confluent Kafka Developer in our organization.As a Confluent Kafka Developer, you will be responsible for:Administering and developing low-latency data streaming solutions using Confluent Kafka.Managing and supporting the entire Kafka ecosystem.Ensuring performance via real-time monitoring...