Senior Data Engineer

1 week ago

Amrāvati, Maharashtra, India beBeeDataStreaming Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

Job Title: Data Streaming Architect

A skilled professional with 3+ years of experience is required to design and manage data streaming infrastructure, deploy and upgrade production-grade clusters, and integrate schema registry and connectors.

Key Responsibilities:

  • Data Infrastructure Design: Ensure high availability and fault tolerance in production-grade clusters by designing and managing data infrastructure.
  • Cluster Deployment and Upgrade: Deploy and upgrade clusters regularly to ensure seamless data flow.
  • Troubleshooting and Optimization: Optimize cluster performance and minimize downtime by troubleshooting critical issues.
  • Schema Registry Integration: Integrate schema registry and deploy multiple connectors to facilitate seamless data flow.
  • Data Replication and Backup: Ensure effective disaster recovery by configuring MirrorMaker for data replication between two data centers.
  • Kafka UI Management: Manage Kafka UI on a Kubernetes cluster for real-time monitoring and management.
  • User Onboarding: Manage ACL permissions, create topics, and implement end-to-end security to onboard new users.
  • Infrastructure Management: Design and manage data infrastructure to enhance message processing efficiency.
  • Performance Testing: Conduct performance testing to optimize partitioning, replication, and retention policies.
  • Integration with ELK Stack and Solr: Integrate Kafka with ELK Stack and Solr for scalable data storage and analysis.

Qualifications:

Bachelor's degree in Computer Science or related field.

Required Skills:

Data streaming, Kafka, MirrorMaker, Kafka Streams, schema registry, Kubernetes, Docker, Ansible, Terraform, monitoring and logging, programming and scripting, shell scripting.

  • Senior Data Engineering Professional

    7 days ago

    Amrāvati, Maharashtra, India beBeeDataEngineering Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title Senior Data Engineer Location: Remote (India) – Occasional travel to Riyadh, KSA We are seeking a Senior Data Engineer to join our product development and delivery team. This is a key role in building the intelligence infrastructure that powers enterprise transformation in complex telecommunications and IT Asset Management environments. You will...

  • Senior Data Engineering Leader

    1 week ago

    Amrāvati, Maharashtra, India beBeeDataEngineer Full time US$ 1,20,000 - US$ 1,60,000

    Job OverviewWe are seeking an experienced Senior Data Engineer to lead our data engineering team. Key Responsibilities:Design, develop, and optimize ETL/ELT data pipelines using Apache Spark, AWS Glue, and Azure Data FactoryLead a high-performing data engineering teamRequirements:Strong background in designing and developing ETL/ELT data pipelinesExperience...

  • Senior Cloud Data Engineer

    7 days ago

    Amrāvati, Maharashtra, India beBeeData Full time ₹ 1,20,00,000 - ₹ 2,50,00,000

    Job Title: Senior Cloud Data EngineerWe are seeking a highly skilled Senior Cloud Data Engineer to join our team.The ideal candidate will have 1-3 years of experience in cloud-based technologies, including design and implementation of automated data pipelines using Google Cloud Platform.They will be responsible for developing analytical algorithms to drive...

  • Senior Lead Data Engineer

    2 days ago

    Amrāvati, Maharashtra, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job TitleWe are seeking an experienced Senior Lead Engineer to join our Commercial Data team in Advertising. This role involves developing leading property and market data insights to power Real Commercial products.This senior engineer will lead the design, development, and optimization of features to ensure high-quality, scalable solutions that deliver...

  • Data Engineer

    3 days ago

    Amrāvati, Maharashtra, India beBeeSenior Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    A Senior ETL Developer will be responsible for designing, developing and deploying scalable data pipelines and ETL processes using Databricks. The ideal candidate will have 6+ years of experience in development and maintenance in a data engineering project. They will collaborate with stakeholders to understand data requirements and translate them into...

  • Senior Data Engineer Healthcare Data Solutions

    2 days ago

    Amrāvati, Maharashtra, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    We are seeking a seasoned Data Engineering expert to lead the development of our self-service data platform for reporting and analytics.Build scalable data solutions with Databricks, SQL, Python, Spark, and Delta Lake on Azure.The ideal candidate will have 8+ years of experience in data engineering, with at least 3 years in senior roles. They should be...

  • Senior Data Engineering Specialist

    3 days ago

    Amrāvati, Maharashtra, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job Title: Senior Data Engineering SpecialistWe are seeking a skilled and experienced professional to lead the design, development, and optimization of large-scale data projects.Key Responsibilities:Drive initiatives to develop and continuously improve our data engineering capabilitiesDesign, build, and optimize complex data pipelines for structured and...

  • Data Engineer

    2 weeks ago

    Amrāvati, Maharashtra, India beBeeDataEngineer Full time ₹ 1,20,00,000 - ₹ 1,80,00,000

    Unlock Your Potential as a Data EngineerWe're seeking an experienced and innovative Data Engineer to join our team. As a key member of our data analytics platforms, you will be responsible for developing and deploying scalable data processing pipelines using Azure Data Factory, Azure Databricks, and other relevant technologies.Your primary focus will be on...

  • Senior Cloud Data Engineering Expert

    6 days ago

    Amrāvati, Maharashtra, India beBeeDataEngineering Full time ₹ 15,60,000 - ₹ 25,12,000

    Seeking Senior Data Engineer with strong cloud-native data engineering background. Must have expertise in building scalable data pipelines, implementing enterprise-grade data governance, security, and AI-powered engineering automation.This role will play a pivotal part in designing, developing, and optimizing data ingestion, transformation, and governance...

  • Data Engineer

    3 weeks ago

    Amrāvati, Maharashtra, India Centrilogic Full time

    Data EngineerPurpose:Over 15 years, we have become a premier global provider of multi-cloud management, cloud-native application development solutions, and strategic end-to-end digital transformation services.Headquartered in Canada and with regional headquarters in the U.S. and the United Kingdom, Centrilogic delivers smart, streamlined solutions to clients...