Distributed Data Engineer

1 week ago

Malappuram, Kerala, India beBeeDataScientist Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

Job Title: Data Architect

About the Role

We are seeking a skilled data architect to join our team. The ideal candidate will have expertise in building, scaling, and optimizing big data pipelines using Scala, Spark, Python, NiFi, and Apache Kafka.

The successful candidate will design, develop, and optimize large-scale data pipelines and distributed data processing systems. They will work extensively with Scala, Spark (PySpark), and Python for data processing and transformation.

Key Responsibilities:

  • Create scalable cloud-based data architectures that meet business requirements.
  • Develop and integrate streaming solutions using Apache Kafka and orchestration tools like NiFi / Airflow.
  • Design efficient data models and write effective queries using Jupyter Notebooks and SQL.
  • Collaborate with cross-functional teams to ensure delivery of high-quality code through code reviews, performance tuning, and best practices.
  • Build monitoring and alerting systems leveraging Splunk or equivalent tools.
  • Contribute to product development with a focus on scalability, maintainability, and performance.

Mandatory Skills:

  • Scala – Minimum 3+ years of hands-on experience.
  • Strong expertise in Spark (PySpark) and Python.
  • Hands-on experience with Apache Kafka.
  • Knowledge of NiFi / Airflow for orchestration.
  • Strong experience in Distributed Data Systems (5+ years).
  • Proficiency in SQL and query optimization.
  • Good understanding of Cloud Architecture.

Preferred Skills:

  • Exposure to messaging technologies like Apache Kafka or equivalent.
  • Experience in designing intuitive, responsive UIs for data analytics visualization.
  • Familiarity with Splunk or other monitoring/alerting solutions.
  • Hands-on experience with CI/CD tools (Git, Jenkins).
  • Strong grasp of software engineering concepts, data modeling, and optimization techniques.

  • Data Engineering Specialist

    2 weeks ago

    Malappuram, Kerala, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job DescriptionWe're seeking an expert in data engineering to join our high-impact team. You'll design, build, and operate scalable data infrastructure to support distributed computing and data orchestration for large language model research.

  • Key Data Engineer Role

    2 weeks ago

    Malappuram, Kerala, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

    Key Data Engineering RoleWe are seeking a highly skilled data engineer to fill a critical position within our organization.The ideal candidate will have extensive experience in designing and developing scalable data pipelines, implementing data warehousing solutions, and ensuring the smooth operation of distributed systems.Responsibilities include working...

  • Highly Skilled Distributed Systems Engineer

    1 week ago

    Malappuram, Kerala, India beBeeKafka Full time ₹ 18,00,000 - ₹ 25,00,000

    Job Title: Kafka Engineer">We are seeking a highly skilled Kafka Engineer to lead the design and implementation of our large-scale distributed systems infrastructure.">This role involves installing, configuring, and managing complex Apache Kafka environments for both on-premises and cloud deployments.">Responsibilities include:">Install, configure, and...

  • Data Engineering Lead

    1 week ago

    Malappuram, Kerala, India beBeeEngineering Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    About UsWe're on a mission to deliver connections that change the world.As a Data Engineer, you'll be a key member of our team, working closely with others to design, develop and operate data warehouses and platforms. We use an agile development methodology, prioritizing tasks quickly but staying ahead to delight our customers.5-8 years of experience in...

  • Senior Electrical Power Distribution Specialist

    1 week ago

    Malappuram, Kerala, India beBeeElectricalEngineering Full time ₹ 80,00,000 - ₹ 1,00,00,000

    Job Opportunity:Our organization is seeking an Electrical Power Distribution Panel Boards' Cost Estimation Specialist to join our team. This is a full-time role with on-site responsibilities located in Ahmedabad.The successful candidate will be responsible for designing electrical power distribution panels, estimating costs, and selecting appropriate...

  • Principal Software Engineer

    1 week ago

    Malappuram, Kerala, India beBeeBackend Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Senior Backend Engineering LeaderAbout the Role:We are seeking a seasoned technology executive with deep expertise in Golang and Python to lead our high-performing backend team.Key Responsibilities:Oversee design, development, and delivery of scalable backend systems.Guide the team in building transactional systems and behavioral systems.Partner with product...

  • AWS Cloud Data Engineer

    1 week ago

    Malappuram, Kerala, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job Title:A Data Engineer is needed to design, develop and support data pipelines and applications using AWS services.Key Responsibilities:Design and implement robust ETL pipelines focusing on performance, scalability and data quality.Develop and maintain data manipulation and analysis applications using Python and PySpark.Translate business requirements...

  • Senior Distributed Systems Specialist

    2 weeks ago

    Malappuram, Kerala, India beBeeDevOps Full time ₹ 24,00,000 - ₹ 32,00,000

    Job Title: Reliability Site EngineerThis role involves managing the end-to-end application and system stack, ensuring high reliability, scalability, and performance of distributed systems. As an SRE, you will combine software engineering and systems engineering to build and operate large-scale, fault-tolerant production environments.We are looking for a...

  • Senior Cloud Data Specialist

    1 week ago

    Malappuram, Kerala, India beBeeData Full time ₹ 13,43,238 - ₹ 25,16,920

    AWS Data Architect PositionJob Title: AWS Data ArchitectWe are seeking a highly skilled Senior AWS Data Engineer to join our data engineering team.The ideal candidate will have deep expertise in building scalable data pipelines using Apache Spark, PySpark, SQL, and Python, along with hands-on experience in the AWS ecosystem.Key Responsibilities:Design,...

  • Senior Data Architect

    2 weeks ago

    Malappuram, Kerala, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job TitleData Engineer - Scalable Data SolutionsAbout the RoleWe are seeking an experienced and skilled Data Engineer to design, build, and maintain robust data pipelines. The ideal candidate will have deep expertise in cloud-based data processing technologies using GCP, AWS, or Azure. You will play a critical role in ensuring data reliability, quality, and...