Streaming Data Architect

2 days ago

Hyderabad Secunderabad Telangana Delhi, India beBeeDataEngineer Full time ₹ 1,04,000 - ₹ 1,30,878
Streaming Data Engineer

We are seeking a highly skilled Streaming Data Engineer to join our team. As a key member of our engineering team, you will be responsible for designing, developing, and deploying scalable and efficient data processing systems.

Responsibilities:

  • Design and develop high-performance data processing pipelines using Apache Spark and other big data technologies.
  • Develop and maintain real-time data streaming applications using Apache Kafka, Apache Flink, and other related tools.
  • Collaborate with cross-functional teams to integrate data processing systems with other enterprise applications.
  • Develop and maintain automated testing frameworks to ensure data quality and integrity.
  • Perform data analysis and provide insights to inform business decisions.
  • Stay up-to-date with industry trends and emerging technologies in the field of big data and streaming analytics.

Required Skills and Qualifications:

  • Expert-level knowledge of Apache Spark, Apache Flink, and Apache Kafka.
  • Strong programming skills in Java, Python, or Scala.
  • Familiarity with Hadoop and NoSQL databases.
  • Experience with containerization using Docker.
  • Knowledge of cloud-based services such as AWS or Azure.
  • Excellent communication and teamwork skills.

Benefits:

  • Competitive salary and benefits package.
  • Ongoing training and professional development opportunities.
  • Collaborative and dynamic work environment.
  • Flexible working hours and remote work options.

Others:

  • Opportunity to work on cutting-edge projects and technologies.
  • Collaborate with experienced engineers and data scientists.
  • Contribute to the growth and success of our company.
  • Work-life balance and flexible working arrangements.

Some of the set of queries might be asked for Evaluation:

  • Difference between Spark Streaming session Vs Batch Sessions.
  • Spark Structured Streaming.
  • spark.readStream() method used to read data in Spark Streaming Application.
  • Use of writeStream method and what are different arguments supported by it to write data frame to provided sink, it has arguments like format, location, writeMode etc.
  • Action need to call for reading data from kafka queue start().
  • print output of streaming operation on Terminal Using console.

  • Data Streaming Specialist

    2 weeks ago

    Delhi, Hyderabad / Secunderabad, Telangana, India beBeeData Full time ₹ 9,00,000 - ₹ 12,00,000

    Job Title: Data Streaming SpecialistWe are seeking a skilled Data Streaming Specialist to join our team. As a key member of our organization, you will play a crucial role in designing and implementing scalable data streaming solutions.

  • Data Streaming Specialist

    2 weeks ago

    Hyderabad / Secunderabad, Telangana, India beBeeDataStreaming Full time

    Job Title: Data Streaming SpecialistAbout the RoleWe are seeking a highly skilled Data Streaming Specialist to join our team. The successful candidate will be responsible for designing, developing, and maintaining scalable data streaming architectures using Apache Kafka.Key ResponsibilitiesDesign and implement data pipelines using Confluent/Apache...

  • Data Streaming Engineer

    1 week ago

    Hyderabad / Secunderabad, Telangana, Delhi, India beBeeData Full time ₹ 1,04,000 - ₹ 1,30,878

    Job Title: Data Streaming EngineerWe are seeking a skilled Data Streaming Engineer to join our team. As a key member of our engineering team, you will be responsible for designing and implementing real-time data processing systems using technologies such as Apache Spark and Flink.

  • Data Streaming Expert

    3 days ago

    Hyderabad / Secunderabad, Telangana, India beBeeKafka Full time ₹ 9,00,000 - ₹ 12,00,000

    Job Summary:We are seeking a highly skilled Data Streaming Expert to join our team. In this role, you will be responsible for managing Kafka Clusters and Apache Zookeeper, transforming data into Kafka topics, and building streaming processors using Kafka.Key Responsibilities:Design and implement scalable and reliable data pipelines using Confluent-specific...

  • Kafka Data Streaming Specialist

    1 week ago

    Hyderabad / Secunderabad, Telangana, Delhi, India beBeeCompliance Full time ₹ 1,04,000 - ₹ 1,30,878

    Job OverviewAs a key member of our team, you will be responsible for managing and supporting the Kafka ecosystem, with a primary focus on Level 2 (L2) support activities.You will monitor Kafka performance and reliability through proactive monitoring tools and dashboards, ensuring that our data streaming operations run smoothly.Collaboration with development...

  • Data Streaming Engineer

    7 days ago

    Delhi, Delhi, India beBeeKafka Full time ₹ 20,00,000 - ₹ 25,00,000

    Confluent Kafka ArchitectThis position involves designing and developing scalable, real-time data streaming platforms using Apache Kafka and Confluent Platform.We are seeking a skilled professional with hands-on experience in building, managing, and scaling distributed systems to play a pivotal role in our company's success.The ideal candidate will have a...

  • Data Architect

    5 days ago

    Delhi, India iXceed Solutions Full time

    **Job Title**:Data Architect **Job Location**:WFH from India **Job Mode**: Remote (need to follow Europe Time zone) Primarily experts of **Spark 3.X, SCALA,** Delta lake implementation, streaming solution implementation for IOT in Spark streaming. Kafka expertise. Any experience with MFG BI, DWH and Datalake implementation would be a bonus. **Job Type**:...

  • Chief Data Architect

    2 weeks ago

    Hyderabad / Secunderabad, Telangana, India beBeeDataArchitect Full time

    Job SummaryWe are seeking a Chief Data Architect to lead our data engineering initiatives and align them with business goals. The ideal candidate will have a strong background in data engineering, excellent leadership skills, and experience with modern data tools and platforms.Key ResponsibilitiesDesign, build, and optimize scalable, robust, and secure data...

  • Data Streaming Specialist

    7 days ago

    Hyderabad, Telangana, India beBeeFlink Full time ₹ 20,00,000 - ₹ 25,00,000

    Apache Flink EngineerThis role will be instrumental in building and maintaining robust, scalable, and reliable data pipelines using Confluent Kafka, ksqlDB, Kafka Connect, and Apache Flink.The ideal candidate will have a strong understanding of data streaming concepts, experience with real-time data processing, and a passion for building high-performance...

  • Data Architect

    1 week ago

    Hyderabad, India Prodapt SA Full time

    **Roles and Responsibilities - Senior Data Architect who will be responsible for leading Big Data Platform Development Programs from Client location working closely with Business and existing vendor teams. - In collaboration with Product Engineering, and colleagues in Service Management and Operations, work to reduce or eliminate manual or repetitive tasks,...