Highly Experienced Big Data Systems Architect

3 days ago

Tirupati, Andhra Pradesh, India beBeeAdministrator Full time ₹ 20,00,000 - ₹ 25,00,000
Job Overview

We are seeking a skilled Kafka Administrator to lead the deployment and management of our production-grade Kafka clusters. The ideal candidate will have extensive experience in designing, deploying, and maintaining high-availability and fault-tolerant Kafka infrastructure.

Key Responsibilities:

  • Deploy and upgrade production-grade Kafka clusters to ensure high availability and fault tolerance.
  • Automate the installation and configuration of Kafka clusters using tools such as Ansible and Chef.
  • Troubleshoot critical production issues, optimizing Kafka performance and minimizing downtime.
  • Configure MirrorMaker for data replication between two data centers, ensuring effective disaster recovery.
  • Deploy and manage Provectus Kafka UI on a Kubernetes cluster for real-time monitoring and management.
  • Integrate Schema Registry and deploy multiple Kafka Connectors to facilitate seamless data flow.
  • Onboard new users by managing ACL permissions, creating topics, and implementing end-to-end security with SSL/TLS.
  • Design and manage Kafka infrastructure to enhance message processing efficiency.
  • Conduct performance testing to optimize partitioning, replication, and retention policies.
  • Integrate Kafka with ELK Stack and Solr for scalable data storage and analysis.

Requirements:

  • BE, BTech, MCA, or MCS degree.
  • 3+ years of experience in Kafka administration.
  • Strong understanding of Kafka architecture and design principles.
  • Proficiency in programming languages such as Java, Python, and Scala.
  • Familiarity with containerization tools like Docker and Kubernetes.
  • Experience with DevOps tools such as Ansible, Terraform, and Jenkins.
  • Strong problem-solving skills and ability to work under pressure.
  • Excellent communication and collaboration skills.

Preferred Skills:

  • Experience with cloud platforms such as AWS or GCP.
  • Knowledge of distributed systems and big data processing.
  • Experience with CI/CD pipelines and automated testing.
  • Understanding of security best practices for Kafka deployments.

Benefits:

  • Competitive salary and benefits package.
  • Opportunity to work with cutting-edge technology.
  • Collaborative and dynamic work environment.

  • Senior Big Data Architect

    5 days ago

    Tirupati, Andhra Pradesh, India beBeeEngineering Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Senior Big Data ArchitectJob Summary:About this Role:We are seeking an experienced and strategic technology leader with expertise in designing and delivering scalable data solutions using Microsoft Azure, Databricks, and Big Data platforms.The ideal candidate will lead a team of talented engineers to architect and oversee the development of...

  • Senior Data Systems Architect

    2 days ago

    Tirupati, Andhra Pradesh, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Senior Data Systems ArchitectWe are excited to announce an opportunity for a skilled data architect to join our team.About the Role: As a senior data systems architect, you will play a crucial role in designing, developing, and deploying large-scale data processing systems using cutting-edge technologies like PySpark and...

  • Big Data Engineer Manager

    3 days ago

    Tirupati, Andhra Pradesh, India beBeeCloud Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title: Big Data Engineer Manager - Cloud ArchitectureOverviewWe are seeking a highly experienced and strategic engineering manager to lead our team of data engineers. The ideal candidate will have deep expertise in Big Data platforms, Microsoft Azure, and Databricks.The successful candidate will design and deliver scalable data pipelines and analytics...

  • Senior Big Data Architect

    2 days ago

    Tirupati, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 12,00,000 - ₹ 16,00,000

    We are seeking an experienced professional to design and implement scalable, high-performance data pipelines using AWS cloud services and Databricks.The ideal candidate will have strong expertise in big data environments, ETL/ELT workflows, and ensuring seamless data availability for analytics, AI/ML, and business intelligence use cases.

  • Big Data Database Administration Role

    2 days ago

    Tirupati, Andhra Pradesh, India beBeeDatabaseAdministrator Full time ₹ 15,00,000 - ₹ 20,00,000

    Database Administrator PositionThis is a senior-level role that requires in-depth knowledge of database systems and technologies. We are seeking an experienced professional to join our Big Data team, where they will be responsible for analyzing existing databases, recommending improvements, and ensuring high availability and performance.The ideal candidate...

  • Senior Data Solutions Architect

    4 days ago

    Tirupati, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Overview:Big Data Engineer PositionSummary: We are seeking a highly skilled and experienced Big Data Engineer to design, develop, and maintain scalable, secure, and high-performance data solutions on Databricks.Key Responsibilities include:

  • Data Platform Architect

    5 days ago

    Tirupati, Andhra Pradesh, India beBeeInnovation Full time US$ 1,80,000 - US$ 2,50,000

    Finance Innovation LeadWe're seeking a skilled Finance Innovation Lead to join our mission-driven team. This role involves architecting and developing platforms for financial data, providing technical structure to teams, and working closely with management and stakeholders to define strategic roadmaps.The ideal candidate will have 8+ years of experience in...

  • Cloud Data Architect with Big Data, Analytics, Cloud Computing

    5 hours ago

    Tirupati, Andhra Pradesh, India beBeeDataEngineer Full time ₹ 1,00,00,000 - ₹ 2,50,00,000

    Senior Data Engineer PositionWe are seeking a highly skilled Senior Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing and developing large-scale data systems using Databricks and AWS.Key Responsibilities:Analyze existing data warehouse implementations to support migration and consolidation...

  • Engineering Manager – Big Data, Azure

    3 days ago

    Tirupati, Andhra Pradesh, India DataE2E Technologies Full time

    Technical Manager – AzureLocation: 100% RemoteEmployment Type: Full-timeShift Timing : 3 pm – 11 pm ISTJob Summary:About the Role:We are seeking a highly experienced and strategic Engineering Manager with deep expertise in Big Data platforms, Microsoft Azure, and Databricks.The ideal candidate will lead a team of talented data engineers to design and...

  • Big Data Solutions Specialist

    3 days ago

    Tirupati, Andhra Pradesh, India beBeeData Full time ₹ 80,00,000 - ₹ 2,00,00,000

    Big Data ExpertWelcome to a role where you will be responsible for developing and implementing Big Data solutions using PySpark and Hive.This position is ideal for experts in data analytics who are passionate about working with large datasets and extracting valuable insights from them.Develop and implement Big Data solutions using PySpark and Hive.Work on...