Kafka Architect

2 weeks ago


BangalorePuneKolkataChennai, IN Ultrabot Innovations Full time

Job Description :

As a Kafka Architect specializing in Spark and Apache Server, you will play a key role in designing, architecting, and implementing real-time data streaming solutions using Apache Kafka, Apache Spark, and related technologies.

You will work closely with our clients and internal teams to understand business requirements, design robust architectures, and implement scalable and reliable data pipelines to enable real-time data processing and analytics.

This role offers an exciting opportunity to work on cutting-edge projects and shape the future of data streaming and analytics solutions.

Key Responsibilities :

Architectural Design :


- Design and architect end-to-end data streaming solutions using Apache Kafka, Apache Spark, and other relevant technologies, considering scalability, reliability, performance, and security requirements.

Platform Setup and Configuration :


- Set up and configure Kafka clusters, Spark clusters, and Apache Servers, including installation, configuration, tuning, and optimization for optimal performance and resource utilization.

Data Ingestion and Processing :


- Implement data ingestion pipelines to capture, collect, and ingest data from various sources into Kafka topics, and design and develop Spark streaming applications for real-time data processing, transformation, and analysis.

Integration :


- Integrate Kafka and Spark with other systems and technologies in the data ecosystem, including data lakes, databases, messaging systems, and analytics platforms, to facilitate seamless data exchange and interoperability.

Monitoring and Performance Tuning :


- Implement monitoring and alerting solutions to track the health and performance of Kafka and Spark clusters, and perform performance tuning and optimization to improve throughput, latency, and reliability.

Security and Compliance :


- Implement security controls and best practices to ensure data privacy, confidentiality, and compliance with regulatory requirements, including encryption, authentication, authorization, and auditing.

Documentation and Best Practices :


- Create and maintain comprehensive documentation, architectural diagrams, and best practices for Kafka and Spark implementations, and provide guidance and training to internal teams and clients.

Technical Leadership :


- Provide technical leadership and guidance to development teams, mentor junior architects and engineers, and collaborate with cross-functional teams to drive successful project delivery.

Qualifications :

- Bachelor's degree in Computer Science, Information Technology, or related field.

- 6-10 years of experience in architecting and implementing data streaming solutions using Apache Kafka, Apache Spark, and related technologies.

- Deep understanding of Kafka architecture, components, and ecosystem, including Kafka brokers, producers, consumers, topics, partitions, and replication.

- Hands-on experience with Apache Spark, including Spark Streaming, Spark SQL, and Spark MLlib, for real-time data processing, analytics, and machine learning.

- Proficiency in setting up and configuring Kafka clusters and Spark clusters, including cluster sizing, deployment, configuration management, and performance tuning.

- Experience with Apache Server setup and configuration for serving web applications and APIs.

- Strong knowledge of data integration and interoperability concepts, including data formats (Avro, Parquet, JSON), serialization (Apache Avro, Apache Thrift), and messaging protocols (Kafka protocol, RESTful APIs).

- Excellent problem-solving skills, with the ability to analyze complex technical challenges and design scalable and reliable solutions.

- Strong communication and interpersonal skills, with the ability to effectively communicate technical concepts to non-technical stakeholders and collaborate with cross-functional teams.

(ref:hirist.tech)
  • Kafka Developer

    4 weeks ago


    Pune/Bangalore, IN Codersbrain technology pvt ltd Full time

    Position Name : Kafka DeveloperExperience Required : 4- 9Notice Period : Immediate - 15 daysJD's :- Collaborate with software developers, architects, and data engineers to design, develop, and maintain Kafka-based solutions for our microservices ecosystem.- Integrate Kafka and Kafka Streams into our Kubernetes-hosted environment on Azure, ensuring...


  • Mumbai/Bangalore/Hyderabad/Pune/Gurgaon/Gurugram, IN Sigma Consultant Full time

    Enterprise Integration ArchitectRole- Consultant, Manager & Senior ManagerCompany- Leading MNC in Consulting SpaceExp - 5 to 15 YearsLocation - Bangalore. Hyderabad, Mumbai, Pune & GurgaonJob Description :Project Role : Enterprise Integration Solution & Strategy Architect Project Role Description : Defines Integration technology transformation, strategies...


  • Bangalore/Anywhere in India/Multiple Locations, IN Leadway Resource Full time

    We are seeking a highly skilled and experienced Messaging Platform Administrator to join our team. You will be responsible for the administration, operations, and optimization of our on-premises and cloud-native Kafka and RabbitMQ clusters. You will play a key role in ensuring the high availability, performance, and security of these messaging platforms,...


  • Bangalore/Noida, IN Squareroot Consulting Pvt Ltd. Full time

    Hiring Java Architect or Senior Architect for SaaS Product Company in Fintech domain.Looking for Tech Architects who have built fault tolerant, Scalable and High Available Systems. Must be hands on coding/designing & have built for scale.Position: Software ArchitectExperience : 10+ YearsLocation: Bangalore, IndiaEducation: BE/BTech/MS/MTech (Preferably from...

  • Solution Architect

    4 weeks ago


    Pune/Chennai/Bangalore/Hyderabad/Mumbai/Kolkata, IN U-SET Full time

    Job Description:-Role : BFSI (Cards & Payments) Solution ArchitectExperience : 10+ yearsJob Location : Chennai, Pune (open for PAN India ready to relocate)Grade : P5/P6Primary Technical Skills :- Architecture & Design patterns - Microservices, Cloud-native, API-First, Event-driven architecture styles- Card (Credit/Debit/Pre-paid) Processing domain expertise-...

  • AWS Data Architect

    4 weeks ago


    Bangalore/Chennai/Gurgaon/Gurugram/Hyderabad/Pune, IN Huquo Full time

    Role Description: Data Engineer - Big Data The ideal candidate is a hands-on technology developer with experience in developing scalable applications and platforms. They must be at ease working in an agile environment with little supervision. The person should be a self-motivated person with a passion for problem solving and continuous learning. Role and...

  • Senior Java Developer

    11 hours ago


    Bangalore/Pune/Hyderabad, IN Hashone careers Pvt Ltd Full time

    Job Description : Founded in the year 2017, CoffeeBeans specializes in offering high end consulting services in technology, product, and processes. We help our clients attain significant improvement in quality of delivery through impactful product launches, process simplification, and help build competencies that drive business outcomes across industries....


  • Bangalore/Pune, IN TELL Jobs Full time

    The person appointed will be an integral member of the Architecture Team and will be responsible for the design and implementation of the enterprise-wide data strategy, ensuring the strategy supports the current and future business needs.The role will involve collaborating with Business and IT stakeholders at all levels to ensure the enterprise data strategy...


  • Bangalore/Pune, IN TELL Jobs Full time

    Job Description : The person appointed will be an integral member of the Architecture Team and will be responsible for the design and implementation of the enterprise-wide data strategy, ensuring the strategy supports the current and future business needs.The role will involve collaborating with Business and IT stakeholders at all levels to ensure the...


  • Pune/Chennai/Bangalore/Hyderabad, IN SavvyR Solutions Private Limited Full time

    Responsibilities :- Collaborate with software engineers, architects, and business analysts to understand user requirements and translate them into efficient database designs.- Design, develop, implement, and optimize complex database solutions using relational databases (e.g. Oracle, SQL Server).- Write clean, maintainable, and efficient SQL queries,...

  • Platform Architect

    4 weeks ago


    Bangalore/Gurgaon/Gurugram, IN Productiva Group Full time

    Job Description : Role : Platform Architect- Someone coming from payment domain. - Microservices Architecture, Scalability, Security, Java latest version, Security Engineering, Performance Engineering, DevSecOps Engineering, Cloud native engineering, Data Engineering, Automation Engineering.As Head of Platform Engineering - DFS Road map will manage the...

  • GCP Architect

    4 weeks ago


    Bangalore/Chennai/Pune/Gurgaon/Gurugram/Hyderabad/Kolkata, IN Change leaders Full time

    Desired Skills & Responsibilities :- Strong understanding of data warehousing and data modeling techniques.- Proficient understanding of distributed computing principles - Hadoop v2, MapReduce, HDFS- Strong data engineering skills on GCP cloud platforms Airflow, Cloud Composer, Data Fusion, Data Flow, Data Proc, Big Query- Experience with building...


  • Bangalore/Anywhere in India/Multiple Locations, IN Cephas Consultancy Services Private Limited Full time

    About the job :Job Responsibilities :- Architect, Design and develop application using java components, tools and frameworks- Perform the object oriented analysis, design and documentation of the project- Ensure component developed are reusable and extensible for future challenges- Improve performance by fine tuning parameters of application, database and...


  • Pune/Hyderabad/Gurgaon/Gurugram, IN Huquo Full time

    Data Architect/Data Modeler Role :- Data Modeling, Enterprise Data Warehouses(Structured and Unstructured data) , Relational Databases, ETL , Big Data, Cloud platforms.- Experienced in multiple domains like Financial domain (Royal Bank Of Scotland, NatWest), US Healthcare(United Health Group).- Hands-on experience in end to end life cycle of Enterprise Data...

  • Solution Architect

    3 weeks ago


    Bangalore/Any Location, IN VIPSA TALENT SOLUTIONS PRIVATE LIMITED Full time

    Tech Stack : AZURE/AWS Architect, AZURE Functions/AWS Lambda, AZURE/AWS DevOps, Event Hub, Kafka, Power Shell, GitLab, Jenkins, Terraform, Ansible, Artifactory, Kubernetes, Docker, Microservices, CI/CDResponsibility :- Creating TSA (Target State Architecture) & BOM (Bill Of Material)- Responsible for taking an app team from initial kickoff through all stages...

  • Database Architect

    4 weeks ago


    Bangalore/Mumbai/Delhi/Pune, IN CloudHire Full time

    About the job :What you are good at :- Integrate multiple databases together, Snowflake schema, Star schema, Network model, and others.- Work with multiple message buses, Kafka, IBM MQ to targets like Redshift, Postgres, MongoDB- Discovering appropriate workloads and use the appropriate database to deliver the performance and functionality needed- Adept at...

  • Java Architect

    4 weeks ago


    Chennai/Mumbai/Pune, IN Coders Brain Pvt Ltd Full time

    Key Responsibilities:- Strong coding, solution design and implementation skills.- Exposure in understanding business requirements and converting into solution design, architecture diagram, data models and workflow diagrams and implementations.- Lead the technical team to perform Proof of Concept (PoC) on solution and component.- Mentor and guide development...

  • Data Architect

    1 week ago


    Chandigarh/Bangalore, IN Maropost Full time

    Job Description : The Data Architect is an individual with a passion for designing and architecting complex systems in a cloud environment and who possesses strong leadership and technical skills. In this role, you will play a crucial role in shaping the architecture of our systems, ensuring their scalability, reliability, and efficiency. Collaborating...


  • Any Location/Bangalore, IN KARYA CONSULTANTS PRIVATE LIMITED Full time

    Job Description :We are in search of an experienced Senior Systems Architect (SSA) with proficiency in Pega 8, capable of identifying and resolving performance issues within live production applications.Preferred qualifications include a background in defect resolution and performance enhancement, especially within live production environments. The ideal...

  • Java Developer

    7 days ago


    Bangalore/Pune, IN Mobile Programming LLC Full time

    Job Description : We are seeking a skilled Java Developer experienced in building microservices architectures using Spring Boot. As a Java Developer, you will be responsible for designing, developing, and deploying robust and scalable microservices that power our cutting-edge applications. You will work closely with cross-functional teams to understand...