Kafka Architect

3 weeks ago

pune, India Ultrabot Innovations Full time

Job Description :

As a Kafka Architect specializing in Spark and Apache Server, you will play a key role in designing, architecting, and implementing real-time data streaming solutions using Apache Kafka, Apache Spark, and related technologies.

You will work closely with our clients and internal teams to understand business requirements, design robust architectures, and implement scalable and reliable data pipelines to enable real-time data processing and analytics.

This role offers an exciting opportunity to work on cutting-edge projects and shape the future of data streaming and analytics solutions.

Key Responsibilities :

Architectural Design :


- Design and architect end-to-end data streaming solutions using Apache Kafka, Apache Spark, and other relevant technologies, considering scalability, reliability, performance, and security requirements.

Platform Setup and Configuration :


- Set up and configure Kafka clusters, Spark clusters, and Apache Servers, including installation, configuration, tuning, and optimization for optimal performance and resource utilization.

Data Ingestion and Processing :


- Implement data ingestion pipelines to capture, collect, and ingest data from various sources into Kafka topics, and design and develop Spark streaming applications for real-time data processing, transformation, and analysis.

Integration :


- Integrate Kafka and Spark with other systems and technologies in the data ecosystem, including data lakes, databases, messaging systems, and analytics platforms, to facilitate seamless data exchange and interoperability.

Monitoring and Performance Tuning :


- Implement monitoring and alerting solutions to track the health and performance of Kafka and Spark clusters, and perform performance tuning and optimization to improve throughput, latency, and reliability.

Security and Compliance :


- Implement security controls and best practices to ensure data privacy, confidentiality, and compliance with regulatory requirements, including encryption, authentication, authorization, and auditing.

Documentation and Best Practices :


- Create and maintain comprehensive documentation, architectural diagrams, and best practices for Kafka and Spark implementations, and provide guidance and training to internal teams and clients.

Technical Leadership :


- Provide technical leadership and guidance to development teams, mentor junior architects and engineers, and collaborate with cross-functional teams to drive successful project delivery.

Qualifications :

- Bachelor's degree in Computer Science, Information Technology, or related field.

- 6-10 years of experience in architecting and implementing data streaming solutions using Apache Kafka, Apache Spark, and related technologies.

- Deep understanding of Kafka architecture, components, and ecosystem, including Kafka brokers, producers, consumers, topics, partitions, and replication.

- Hands-on experience with Apache Spark, including Spark Streaming, Spark SQL, and Spark MLlib, for real-time data processing, analytics, and machine learning.

- Proficiency in setting up and configuring Kafka clusters and Spark clusters, including cluster sizing, deployment, configuration management, and performance tuning.

- Experience with Apache Server setup and configuration for serving web applications and APIs.

- Strong knowledge of data integration and interoperability concepts, including data formats (Avro, Parquet, JSON), serialization (Apache Avro, Apache Thrift), and messaging protocols (Kafka protocol, RESTful APIs).

- Excellent problem-solving skills, with the ability to analyze complex technical challenges and design scalable and reliable solutions.

- Strong communication and interpersonal skills, with the ability to effectively communicate technical concepts to non-technical stakeholders and collaborate with cross-functional teams.

(ref:hirist.tech)

  • Kafka Architect

    4 weeks ago

    Pune, India Ultrabot Innovations Full time

    Job Description :As a Kafka Architect specializing in Spark and Apache Server, you will play a key role in designing, architecting, and implementing real-time data streaming solutions using Apache Kafka, Apache Spark, and related technologies. You will work closely with our clients and internal teams to understand business requirements, design robust...

  • Kafka Architect

    4 weeks ago

    Bangalore/Pune/Kolkata/Chennai, India Ultrabot Innovations Full time

    Job Description :As a Kafka Architect specializing in Spark and Apache Server, you will play a key role in designing, architecting, and implementing real-time data streaming solutions using Apache Kafka, Apache Spark, and related technologies. You will work closely with our clients and internal teams to understand business requirements, design robust...

  • Kafka DeveloperCBS

    1 month ago

    Pune, India Coders Brain Technology Pvt. Ltd. Full time

    Position Name : KafkaDeveloperCollaboratewith software developers architects and data engineers to designdevelop and maintain Kafkabased solutions for our microservicesecosystem.Integrate Kafka and Kafka Streamsinto our Kuberneteshosted environment on Azure ensuring seamlesscommunication and data flow amongmicroservices.Utilize your strong Java andSpring...

  • Kafka Expert

    1 month ago

    Pune, Maharashtra, India growel softech Full time

    **Position : Kafka Expert Exp-4-6 yrs Budget - 7 LPA to 11 LPA (slightly negotiable) Notice : Immediate joiner or 10 Days **Location - Remote Mandatory: **Kafka, confluent Kafka, AWS, KUBERNETES, kafka architect ( **in-depth **knowledge), python ,terraform **Good communication skills Mandatory Skills**: - In Depth knowledge of Kafka architecture** - MSK,...

  • ServiceNow Kafka Architect

    2 months ago

    pune, India Jade Global Full time

    * Strong experience in different integration technologies, data streaming and messaging technologies primarily Kafka * Solid understanding of Kafka architecture and various components in kafka ecosystem such as Zookeeper, Kafka cluster, Kafka broker, producer, consumer, Connectors, KSQL, Topic, kPow etc * Excellent knowledge on deployment architecture,...

  • Servicenow Kafka Architect

    1 month ago

    Pune, India Jade Global Full time

    ID: 6572 | 8-10 yrs | Pune | careers - Strong experience in different integration technologies, data streaming and messaging technologies primarily Kafka - Solid understanding of Kafka architecture and various components in kafka ecosystem such as Zookeeper, Kafka cluster, Kafka broker, producer, consumer, Connectors, KSQL, Topic, kPow etc - Excellent...

  • KAFKA Developer- OOM

    1 month ago

    Pune, India MICHELIN Full time

    KAFKA Developer- OOM- - - - - - - - - - - -Responsibilities:Collaborate with software developers, architects, and data engineers to design, develop, and maintain Kafka-based solutions for our microservices ecosystem.Integrate Kafka and Kafka Streams into our Kubernetes-hosted environment on Azure, ensuring seamless communication and data flow among...

  • KAFKA Developer- OOM

    1 month ago

    Pune, India MICHELIN Full time

    KAFKA Developer- OOM - - - - - - - - - - - - Responsibilities: Collaborate with software developers, architects, and data engineers to design, develop, and maintain Kafka-based solutions for our microservices ecosystem. Integrate Kafka and Kafka Streams into our Kubernetes-hosted environment on Azure, ensuring seamless communication and data flow among...

  • KAFKA Developer- OOM

    1 month ago

    pune, India MICHELIN Full time

    KAFKA Developer- OOM - - - - - - - - - - - - Responsibilities: Collaborate with software developers, architects, and data engineers to design, develop, and maintain Kafka-based solutions for our microservices ecosystem. Integrate Kafka and Kafka Streams into our Kubernetes-hosted environment on Azure, ensuring seamless communication and data flow...

  • Kafka Developer

    4 weeks ago

    Pune, India Codersbrain technology pvt ltd Full time

    Position Name : Kafka DeveloperExperience Required : 4- 9Notice Period : Immediate - 15 daysJD's :- Collaborate with software developers, architects, and data engineers to design, develop, and maintain Kafka-based solutions for our microservices ecosystem.- Integrate Kafka and Kafka Streams into our Kubernetes-hosted environment on Azure, ensuring...

  • Kafka Developer

    4 weeks ago

    Pune/Bangalore, India Codersbrain technology pvt ltd Full time

    Position Name : Kafka DeveloperExperience Required : 4- 9Notice Period : Immediate - 15 daysJD's :- Collaborate with software developers, architects, and data engineers to design, develop, and maintain Kafka-based solutions for our microservices ecosystem.- Integrate Kafka and Kafka Streams into our Kubernetes-hosted environment on Azure, ensuring...

  • Cloud Architect

    1 week ago

    Pune, India VOLANSYS (An ACL Digital Company) Full time

    Dear Connections, We have looking Cloud Architect for Ahmedabad OR Pune Location Location. Experience: 8 years to 18 Years Location: Ahmedabad/Pune Work Type: Permanent Work Nature: Work from office only Notice Period: Immediate Only Job Description: Technical Skills Must Have: • 15+ years overall IT industry experience, with 5+ years in a solution or...

  • Enterprise Integration Architect

    3 weeks ago

    Mumbai/Bangalore/Hyderabad/Pune/Gurgaon/Gurugram, India Sigma Consultant Full time

    Enterprise Integration ArchitectRole- Consultant, Manager & Senior ManagerCompany- Leading MNC in Consulting SpaceExp - 5 to 15 YearsLocation - Bangalore. Hyderabad, Mumbai, Pune & GurgaonJob Description : Project Role : Enterprise Integration Solution & Strategy Architect Project Role Description : Defines Integration technology transformation, strategies...

  • Cloud Architect-Immediate

    3 weeks ago

    pune, India VOLANSYS (An ACL Digital Company) Full time

    Dear Connections,We have looking Cloud Architect for Ahmedabad OR Pune Location Location.Experience: 8 years to 18 YearsLocation: Ahmedabad/PuneWork Type: PermanentWork Nature: Work from office onlyNotice Period: Immediate OnlyJob Description:Technical Skills Must Have:• 15+ years overall IT industry experience, with 5+ years in a solution or technical...

  • Cloud Architect-Immediate

    3 weeks ago

    Pune, India VOLANSYS (An ACL Digital Company) Full time

    Dear Connections,We have looking Cloud Architect for Ahmedabad OR Pune Location Location.Experience: 8 years to 18 Years Location: Ahmedabad/PuneWork Type: PermanentWork Nature: Work from office onlyNotice Period: Immediate OnlyJob Description:Technical Skills Must Have:• 15+ years overall IT industry experience, with 5+ years in a solution or technical...

  • Cloud Architect-Immediate

    3 weeks ago

    pune, India VOLANSYS (An ACL Digital Company) Full time

    Dear Connections, We have looking Cloud Architect for Ahmedabad OR Pune Location Location. Experience: 8 years to 18 Years Location: Ahmedabad/Pune Work Type: Permanent Work Nature: Work from office only Notice Period: Immediate Only Job Description: Technical Skills Must Have: • 15+ years overall IT industry experience, with 5+ years in a solution or...

  • Cloud Architect-Immediate

    3 weeks ago

    Pune, India VOLANSYS (An ACL Digital Company) Full time

    Dear Connections,We have looking Cloud Architect for Ahmedabad OR Pune Location Location.Experience: 8 years to 18 Years Location: Ahmedabad/PuneWork Type: PermanentWork Nature: Work from office onlyNotice Period: Immediate OnlyJob Description:Technical Skills Must Have:• 15+ years overall IT industry experience, with 5+ years in a solution or technical...

  • Technical Architect

    2 weeks ago

    Pune, India Allianz Services Full time

    Designation - Senior Manager / Technical Architect – Data EngineeringExperience Range – 10 to 16 YearsEducation - Bachelor's/master’s degree in computer science, Data Science, or a related fieldLocation – Trivandrum or PuneRoles & responsibility -• Architecting, designing, and implementing scalable data solutions in the data engineering space•...

  • Technical Architect

    2 weeks ago

    Pune, India Allianz Services Full time

    Designation - Senior Manager / Technical Architect – Data EngineeringExperience Range – 10 to 16 YearsEducation - Bachelor's/master’s degree in computer science, Data Science, or a related fieldLocation – Trivandrum or PuneRoles & responsibility -• Architecting, designing, and implementing scalable data solutions in the data engineering space•...

  • Cloud Assessment, Cloud Architect, Cloud Migration

    1 month ago

    Pune, India Zensar Technologies Full time

    **CLOUD ASSESSMENT, CLOUD ARCHITECT, CLOUD MIGRATION, CLOUD INTEGRATION** **-** **(** **0095698** **)** **Description** Pepkor Integration Architect - 25% WE ARE LOKKING FOR INTERGRATION ARCHITECT ROLE NOT TECHNOLOGY SPECIFICALLY EXP 12 TO 15 YEARS. LOCATION-INDIA BAND: - D AND BELOW Experience in integration patterns, standards/protocols, and...