Data Pipeline Engineer

2 days ago

Chennai, Tamil Nadu, India beBeeKafkaIntegration Full time ₹ 1,04,000 - ₹ 1,30,878
Kafka Integration Professional

We are seeking a highly skilled Kafka Integration Specialist with extensive experience in designing, developing, and integrating Apache Kafka solutions. You will be responsible for creating and maintaining robust, real-time integration pipelines, ensuring high availability, scalability, and performance.

Key Responsibilities:

  • Design and implement scalable data pipelines using Apache Kafka.
  • Develop integration solutions utilizing Kafka Connect, Kafka Streams, and other related technologies.
  • Manage Kafka clusters to guarantee high availability, scalability, and performance.
  • Collaborate with cross-functional teams to deliver customized integration solutions.
  • Implement best practices for data streaming, including message serialization, partitioning, and replication.
  • Maintain and optimize Kafka performance, latency, and security.
  • Evaluate and improve data integrity, implementing failover strategies for critical data pipelines.

Required Skills and Qualifications:

  • Substantial experience in Apache Kafka (Kafka Streams, Kafka Connect).
  • Proficiency in programming languages like Java, Python, or Scala.
  • Familiarity with distributed systems and data streaming concepts.
  • Expertise in managing topics, partitions, and consumer groups.
  • Knowledge of cloud platforms like AWS, Azure, or GCP for Kafka deployment.
  • Familiarity with monitoring tools like Prometheus, Grafana, or Datadog.
  • Exposure to DevOps practices, CI/CD pipelines, and infrastructure automation.
  • Bachelor's degree in Computer Science, Information Technology, or a related field.

About the Role:

This role offers an excellent opportunity to work on cutting-edge integration projects and contribute to the growth and development of our organization.

What We Offer:

  • A dynamic and collaborative work environment.
  • Opportunities for professional growth and development.
  • A competitive compensation package.

Contact Information:

  • Career Opportunity

    3 days ago

    Chennai, Tamil Nadu, India beBeeData Engineer Full time ₹ 1,04,000 - ₹ 1,30,878

    Job Summary">This role is responsible for participating as an individual contributor in project teams, troubleshooting operational issues, and providing technical solutions to operational problems.The ideal candidate will have experience with Python, SQL, and Bash Scripting, as well as knowledge of data pipeline fundamentals and general understanding of...

  • Data Pipeline Engineer

    9 hours ago

    Chennai, Tamil Nadu, India beBeeDataPipeline Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Title: Data Pipeline EngineerAbout the Role:Data Pipeline Engineer is a highly skilled technical professional responsible for designing, building, and maintaining complex data pipelines and workflows to move and transform data from various sources to destinations within the Azure ecosystem.Key Responsibilities:Design, build, and maintain complex data...

  • Data Engineer

    3 weeks ago

    Chennai, Tamil Nadu, India Quantrail Data Full time

    Location: Chennai, India (On-site)Duration: 3 monthsStipend: UnpaidEligibility: 2025 graduates onlyConversion: Opportunity for a full-time role based on performance Note: Please apply only if you are serious about joining us. This is a learning-focused internship, and we expect interns to be committed to upskilling themselves and contributing to real-world...

  • Data Engineer

    4 hours ago

    Chennai, Tamil Nadu, India Quantrail Data Full time US$ 60,000 - US$ 1,00,000 per year

    Location: Chennai, India (On-site)Duration: 3 monthsStipend: UnpaidEligibility:2025 graduates onlyConversion: Opportunity for a full-time role based on performance Note: Please apply only if you are serious about joining us. This is a learning-focused internship, and we expect interns to be committed to upskilling themselves and contributing to real-world...

  • Senior Data Engineer

    4 days ago

    Chennai, Tamil Nadu, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Key Responsibilities:We are seeking a seasoned data engineering professional to lead the design and development of robust data pipelines using Azure Data Factory.Collaborate with cross-functional teams to identify complex data requirements and deliver tailored solutions that meet business objectives.Stay updated on the latest Azure Data Factory features and...

  • Data Modeler and Pipeline Engineer

    1 day ago

    Chennai, Tamil Nadu, India beBeeDataScientist Full time ₹ 10,00,000 - ₹ 15,00,000

    About this role : You will be creating, maintaining and supporting data pipelines with information coming from various sources. This includes vessels, projects, campaigns, third-party data services and more. You will play a key role in organizing data, developing and maintaining data models and designing modern data solutions and products on our cloud...

  • Expert Data Pipeline Architect

    9 hours ago

    Chennai, Tamil Nadu, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Data EngineerWe are seeking a highly skilled and motivated Data Engineer to join our team. As a Data Engineer, you will be responsible for designing, developing, and managing scalable data pipelines.Key Responsibilities:Design and implement data pipelines using GCP services such as BigQuery, Dataflow, Pub/Sub, Cloud Storage, Composer, and...

  • Data Pipelines Specialist

    17 hours ago

    Chennai, Tamil Nadu, India beBeeDataPipelines Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Title: Data Pipelines SpecialistAbout the Role:We are seeking a skilled Data Pipelines Specialist to join our team. The ideal candidate will have hands-on experience designing, building, and maintaining production-grade pipelines at scale.Dream Job Responsibilities:Create end-to-end ETL/ELT processes that ensure reliability, scalability, and high...

  • GCP Data Engineer

    2 days ago

    Chennai, Tamil Nadu, India CustomerLabs 1P Data OPs Full time

    Position Overview:"Yesterday is history, tomorrow is a mystery, but today is a gift. That's why we call itthe present." - Master OogwayJoin CustomerLabs' dynamic data team as a Data Engineer and play a pivotal role intransforming raw marketing data into actionable insights that power our digitalmarketing platform. As a key member of our data infrastructure...

  • Senior Data Pipeline Specialist

    4 days ago

    Chennai, Tamil Nadu, India beBeeDataPipeline Full time ₹ 18,00,000 - ₹ 24,00,000

    Job Title:Senior Test Engineer - Data Pipeline Specialist About the Job We are seeking an experienced Senior Test Engineer to drive testing strategy and execution for a project leveraging Databricks, Data Factory pipelines and Power BI. The ideal candidate will have expertise in testing data pipelines using Python, SQL, etc. Key Responsibilities Develop and...