Senior Apache Kafka Engineer

3 days ago

BangaloreDelhiPune, India EDGESOFT Full time

About the Role

We are seeking a highly skilled Senior Apache Kafka Engineer to join our team at Edgesoft. As a key member of our technical staff, you will be responsible for designing, implementing, and managing a scalable and high-performance Apache Kafka infrastructure on Google Cloud Platform (GCP).

Job Description

The successful candidate will have extensive experience in leading migration projects to GCP, including migrating Apache Kafka workloads. You will work closely with cross-functional teams to optimize Kafka API integration and develop comprehensive performance metrics dashboards for the Kafka ecosystem.

Key Responsibilities

  • Lead the migration of Apache Kafka workloads from AWS to GCP utilizing Kubernetes and GCP-native services.
  • Architect, implement, and manage a scalable Apache Kafka infrastructure on Google Kubernetes Engine (GKE).
  • Develop and maintain comprehensive performance metrics dashboards for the Apache Kafka ecosystem.
  • Collaborate with cross-functional teams to optimize Kafka API integration.
  • Conduct in-depth assessments of existing AWS Apache Kafka deployments for migration planning.
  • Develop and execute a robust migration strategy encompassing data migration and consumer repointing.
  • Optimize Apache Kafka performance and scalability for high-throughput data processing.
  • Implement robust monitoring and alerting for system health and performance.
  • Guarantee data integrity and security throughout the migration process.
  • Collaborate with engineering and data teams for seamless integration and optimal performance.
  • Design and implement a multi-tenant GKE environment to accommodate diverse workloads.
  • Deploy and configure Apache Strimzi on the underlying infrastructure.
  • Identify and implement efficient data ingestion pipelines (e.g., Kafka Connect, Dataflow).
  • Conduct thorough fault simulation testing to mitigate potential risks.

Requirements

To be successful in this role, you will need to possess the following qualifications:

  • Bachelor's degree in Computer Science, Engineering, or related field (Master's degree preferred).
  • Proven hands-on experience as a Kafka Lead or similar role, with a minimum of 5 years of experience.
  • Deep knowledge of Apache Kafka internals and ecosystem, including Kafka Connect, Kafka Streams, and KSQL.
  • In-depth knowledge of Apache Kafka architecture, internals, and ecosystem components.
  • Proficiency in scripting and automation for Kafka management and migration.
  • Hands-on experience with Apache Kafka administration, including cluster setup, configuration, and tuning.
  • Proficiency in Kafka APIs, including Producer, Consumer, Streams, and Connect.
  • Strong programming skills in Java, Scala, or Python.
  • Experience with Kafka monitoring and management tools such as Confluent Control Center, Kafka Manager, or similar.
  • Solid understanding of distributed systems, data pipelines, and stream processing.
  • Experience leading migration projects to Google Cloud Platform (GCP), including migrating Apache Kafka workloads.
  • Familiarity with GCP services such as Google Kubernetes Engine (GKE), Google Cloud Storage, Google Cloud Pub/Sub, and BigQuery.
  • Excellent communication and collaboration skills.
  • Ability to work independently and manage multiple tasks in a fast-paced environment.
  • Knowledge of containerization technologies such as Docker and Kubernetes.
  • Familiarity with related technologies such as Apache ZooKeeper, Apache Flink, or Apache Spark.
  • Working knowledge of hands-on experience with Strimzi Kafka.

Salary Range

The salary range for this position is $150,000 - $200,000 per year, depending on experience.

  • Senior Apache Kafka Integration Specialist

    3 days ago

    Pune, Maharashtra, India Wipro Full time

    Company Overview: Wipro is a leading global information technology, consulting and business process services company.About the Role: We are seeking an experienced Senior Apache Kafka Integration Specialist to join our event streaming team.Salary Range: ₹ 12.50 lakhs - ₹ 20.00 lakhs per annum (approx.), depending on experience.Job Description:Kafka Code...

  • Apache Kafka Integration Specialist

    2 weeks ago

    Pune, Maharashtra, India Fulcrum Digital Full time

    About Fulcrum DigitalFulcrum Digital is an agile and next-generation digital accelerating company providing comprehensive digital transformation and technology services across various industries.The RoleWe are seeking a highly skilled Apache Kafka Integration Specialist to join our team. As a key member, you will be responsible for working on Apache Kafka...

  • Senior Apache Kafka Developer Advocate

    2 weeks ago

    Pune, Maharashtra, India Wipro Full time

    Wipro is seeking a seasoned Solutions IT Developer with a strong background in Apache Kafka to join our developer advocacy function in our event streaming team.The ideal candidate will be responsible for Kafka code reviews with clients, troubleshooting client connection issues with Kafka and supporting client onboarding to Confluent Cloud.This role requires...

  • Senior Data Integration Engineer

    2 weeks ago

    Delhi, India Senior Data Integration Engineer Full time

    Must Have Skills/Skill Requirement:Design and architect integration solutions to connect various enterprise applications, systems, and databases.Develop and implement integration workflows, APIs, and data pipelines to enable smooth communication and data exchange between different applications.Utilize Azure Integration Services such as Azure Logic Apps,...

  • Senior Data Integration Engineer

    2 weeks ago

    Delhi, India Senior Data Integration Engineer Full time

    Must Have Skills/Skill Requirement:- Design and architect integration solutions to connect various enterprise applications, systems, and databases.- Develop and implement integration workflows, APIs, and data pipelines to enable smooth communication and data exchange between different applications.- Utilize Azure Integration Services such as Azure Logic...

  • Senior Data Integration Engineer | Senior Data Integration Engineer | delhi

    2 weeks ago

    delhi, India Senior Data Integration Engineer Full time

    Must Have Skills/Skill Requirement:Design and architect integration solutions to connect various enterprise applications, systems, and databases.Develop and implement integration workflows, APIs, and data pipelines to enable smooth communication and data exchange between different applications.Utilize Azure Integration Services such as Azure Logic Apps,...

  • Senior data integration engineer

    1 week ago

    Delhi, India Senior Data Integration Engineer Full time

    Must Have Skills/Skill Requirement:Design and architect integration solutions to connect various enterprise applications, systems, and databases.Develop and implement integration workflows, APIs, and data pipelines to enable smooth communication and data exchange between different applications.Utilize Azure Integration Services such as Azure Logic Apps,...

  • Confluent Kafka Designer

    2 months ago

    bangalore, India Torry Harris Integration Solutions Full time

    Confluent Kafka DesignerJob DescriptionAre you an experienced Confluent Kafka Developer looking for a new opportunity? Look no further than Torry Harris! We are currently seeking a talented and experienced individual to join our team as a Senior Confluent Kafka Developer.The Senior Confluent Kafka Developer will be responsible for designing, developing, and...

  • Confluent kafka designer

    2 months ago

    Bangalore, India Torry Harris Integration Solutions Full time

    Confluent Kafka Designer Job Description Are you an experienced Confluent Kafka Developer looking for a new opportunity? Look no further than Torry Harris! We are currently seeking a talented and experienced individual to join our team as a Senior Confluent Kafka Developer. The Senior Confluent Kafka Developer will be responsible for designing,...

  • Confluent Kafka Designer

    2 months ago

    Bangalore, India Torry Harris Integration Solutions Full time

    Confluent Kafka Designer Job Description Are you an experienced Confluent Kafka Developer looking for a new opportunity? Look no further than Torry Harris! We are currently seeking a talented and experienced individual to join our team as a Senior Confluent Kafka Developer. The Senior Confluent Kafka Developer will be responsible for designing,...

  • Confluent Kafka Designer

    4 weeks ago

    bangalore, India Torry Harris Integration Solutions Full time

    Confluent Kafka Designer Job Description Are you an experienced Confluent Kafka Developer looking for a new opportunity? Look no further than Torry Harris! We are currently seeking a talented and experienced individual to join our team as a Senior Confluent Kafka Developer. The Senior Confluent Kafka Developer will be responsible for designing, developing,...

  • Confluent Kafka Designer

    2 months ago

    bangalore, India Torry Harris Integration Solutions Full time

    Confluent Kafka DesignerJob DescriptionAre you an experienced Confluent Kafka Developer looking for a new opportunity? Look no further than Torry Harris! We are currently seeking a talented and experienced individual to join our team as a Senior Confluent Kafka Developer.The Senior Confluent Kafka Developer will be responsible for designing, developing, and...

  • Confluent Kafka Designer

    5 months ago

    Delhi, India Torry Harris Integration Solutions Full time

    Confluent Kafka DesignerJob DescriptionAre you an experienced Confluent Kafka Developer looking for a new opportunity? Look no further than Torry Harris! We are currently seeking a talented and experienced individual to join our team as a Senior Confluent Kafka Developer.The Senior Confluent Kafka Developer will be responsible for designing, developing, and...

  • Confluent kafka designer

    1 month ago

    Bangalore City, India Torry Harris Integration Solutions Full time

    Confluent Kafka Designer Job Description Are you an experienced Confluent Kafka Developer looking for a new opportunity? Look no further than Torry Harris! We are currently seeking a talented and experienced individual to join our team as a Senior Confluent Kafka Developer. The Senior Confluent Kafka Developer will be responsible for designing, developing,...

  • Kafka Administrator

    1 week ago

    Bangalore/Delhi,Pune, India EDGESOFT Full time

    Responsibilities : - Lead the migration of Kafka workloads from AWS to GCP utilising Kubernetes and GCP-native services.- Architect, implement, and manage a scalable Kafka infrastructure on GKE.- Develop and maintain comprehensive performance metrics dashboards for the Kafka ecosystem.- Collaborate with cross-functional teams to optimise Kafka API...

  • Apache NiFi Engineer

    1 month ago

    Bangalore, India Minions Ventures Pvt Ltd Full time

    Job Title: Apache NiFi EngineerWe are seeking an experienced Apache NiFi developer to join our team at Minions Ventures Pvt Ltd. As a Data Integration Specialist, you will be responsible for designing, developing, and implementing data integration solutions using Apache NiFi.Key Responsibilities:Design and develop data integration flows using Apache...

  • Kafka Architect

    3 weeks ago

    bangalore, India Connectpro Management Consultants Private Limited Full time

    Location- Bangalore Years of experience - 8 to 15 Years Qualification - B.Tech / MCA Key Responsibilities : Design, deploy, and manage Kafka clusters in a production environment, ensuring high availability, scalability, and reliability. Collaborate with engineering teams to understand data processing requirements and design Kafka solutions that meet...

  • Kafka Architect

    3 weeks ago

    bangalore, India Connectpro Management Consultants Private Limited Full time

    Location- BangaloreYears of experience - 8 to 15 YearsQualification - B.Tech / MCAKey Responsibilities :Design, deploy, and manage Kafka clusters in a production environment, ensuring high availability, scalability, and reliability.Collaborate with engineering teams to understand data processing requirements and design Kafka solutions that meet their...

  • Apache Spark Expert

    6 months ago

    Pune, India Info Origin Full time

    Position Title: - Apache Spark Expert**Location**: - Pune, MH, OnsitePosition Type: - Full-TimeInterview Process: - Webcam/SkypeReq Id: - 3314Salary/Rate: - MarketPosition Duration: - Permanent**Key Responsibilities**: - Provide architectural consultancy to clients and develop implementation roadmaps for Apache Spark and Kafka solutions. - Translate...

  • Senior Kafka Solutions Architect

    5 days ago

    Pune, Maharashtra, India Wipro Full time

    Company Overview: We are Wipro, a leading global information technology consulting company. Our event streaming team is seeking a seasoned expert to join our developer advocacy function.Job Description: As a Senior Kafka Solutions Architect, you will be responsible for ensuring smooth integration of Apache Kafka for our Line of Business (LOB) users. This...