Big Data Engineer

3 weeks ago

Salem, Tamil Nadu, India Idyllic Services Full time

Job Title: Big Data Engineer – Scala

Location: Bangalore, Chennai, Gurgaon, Pune, Mumbai.

Experience: 7–10 Years (Minimum 3+ years in Scala)

Notice Period: Immediate to 30 Days

Mode of Work: Hybrid

Click the link below to learn more about the role and take the AI Interview to begin your application journey: https://sl1nk.com/XiPeN

Role Overview

We are looking for a highly skilled Big Data Engineer (Scala) with strong expertise in Scala, Spark, Python, NiFi, and Apache Kafka to join our data engineering team. The ideal candidate will have a proven track record in building, scaling, and optimizing big data pipelines, and hands-on experience in distributed data systems and cloud-based solutions.

Key Responsibilities

- Design, develop, and optimize large-scale data pipelines and distributed data processing systems.

- Work extensively with Scala, Spark (PySpark), and Python for data processing and transformation.

- Develop and integrate streaming solutions using Apache Kafka and orchestration tools like NiFi / Airflow.

- Write efficient queries and perform data analysis using Jupyter Notebooks and SQL.

- Collaborate with cross-functional teams to design scalable cloud-based data architectures.

- Ensure delivery of high-quality code through code reviews, performance tuning, and best practices.

- Build monitoring and alerting systems leveraging Splunk or equivalent tools.

- Participate in CI/CD workflows using Git, Jenkins, and other DevOps tools.

- Contribute to product development with a focus on scalability, maintainability, and performance.

Mandatory Skills

- Scala – Minimum 3+ years of hands-on experience.

- Strong expertise in Spark (PySpark) and Python.

- Hands-on experience with Apache Kafka.

- Knowledge of NiFi / Airflow for orchestration.

- Strong experience in Distributed Data Systems (5+ years).

- Proficiency in SQL and query optimization.

- Good understanding of Cloud Architecture.

Preferred Skills

- Exposure to messaging technologies like Apache Kafka or equivalent.

- Experience in designing intuitive, responsive UIs for data analytics visualization.

- Familiarity with Splunk or other monitoring/alerting solutions.

- Hands-on experience with CI/CD tools (Git, Jenkins).

- Strong grasp of software engineering concepts, data modeling, and optimization techniques.

  • Big Data Architect

    2 weeks ago

    Salem, Tamil Nadu, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Big Data ExpertWe are seeking a skilled Big Data professional to lead our infrastructure implementation via code and provide strategic advice on Microsoft Azure technologies.Key ResponsibilitiesImplement infrastructure via code and provide strategic advice on Microsoft Azure technologies based on previous research on trends in public cloud...

  • Data Engineer

    3 weeks ago

    Salem, Tamil Nadu, India Steps AI Full time

    Location: India (Hyderabad - Preferred/Remote Flexibility)Type: Full time, PaidThe RoleWe're looking for an AWS focused Data Engineer with hands-on experience building production-grade pipelines at scale (terabytes of data). The ideal candidate has 1–2 years of experience in data engineering and is excited to solve tough data problems in a fast-paced...

  • Cloud Data Engineering Professional

    2 weeks ago

    Salem, Tamil Nadu, India beBeeDataEngineer Full time ₹ 17,00,000 - ₹ 23,25,000

    Cloud Data EngineerWe are seeking a skilled Cloud professional to lead our data engineering and analytics efforts.Key Responsibilities:Design, develop, and maintain cloud-based data engineering and analytics solutions using Google Cloud Platform (GCP) tools.Develop and maintain ETL processes and data pipelines using Python or scripting languages.Collaborate...

  • Advanced Data Architect

    2 weeks ago

    Salem, Tamil Nadu, India beBeeBigDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    As a Big Data Engineer, you will play a pivotal role in driving technological transformation across various sectors. Our organization partners with enterprises to deliver cutting-edge solutions for healthcare, technology, communications, manufacturing, and finance.We focus on innovation and acceleration to empower clients to build, modernize, and scale...

  • Data Engineer

    3 weeks ago

    Salem, Tamil Nadu, India People Prime Worldwide Full time

    About Client:Our Client is a multinational IT services and consulting company headquartered in USA, With revenues 19.7 Billion USD, with Global work force of 3,50,000 and Listed in NASDAQ, It is one of the leading IT services firms globally, known for its work in digital transformation, technology consulting, and business process outsourcing, Business Focus...

  • Senior Data Systems Architect

    2 weeks ago

    Salem, Tamil Nadu, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Summary:We are seeking a highly skilled Data Engineer to lead our data systems and drive business outcomes.The ideal candidate will have strong experience in designing and implementing large-scale data pipelines using Python and GCP Cloud Run.Main Responsibilities:Data Engineering: Design, implement, and maintain scalable and efficient data pipelines...

  • Data Engineer

    3 weeks ago

    Salem, Tamil Nadu, India People Prime Worldwide Full time

    About Client:Our Client is a global IT services company headquartered in Southborough, Massachusetts, USA. Founded in 1996, with a revenue of $1.8B, with 35,000+ associates worldwide, specializes in digital engineering, and IT services company helping clients modernize their technology infrastructure, adopt cloud and AI solutions, and accelerate innovation....

  • Data Testing engineer

    3 weeks ago

    Salem, Tamil Nadu, India Innova ESI Full time

    Hello Greetings we are hiring for Data Testing engineer for one of the MNC company with INNOVA.JD for Data Testing About the Role:We are seeking a highly skilled and detail-oriented Senior Data Testing Engineer to join our data engineering team. The ideal candidate will have strong experience in Python, Pandas, REST API and testing, with a solid...

  • Machine Learning Engineer

    4 weeks ago

    Salem, Tamil Nadu, India SYNOpix Full time

    Join our team as a Machine Learning Engineer and build innovative solutions for our U.S. client. We're looking for someone with a strong background in machine learning who is ready to design, build, and deploy robust ML models in a production environment. Your expertise in AWS SageMaker will be key to creating scalable solutions that deliver real...

  • Intelligent Data Solutions Expert

    2 weeks ago

    Salem, Tamil Nadu, India beBeeBusiness Full time US$ 1,25,000 - US$ 1,75,000

    Business Intelligence Analyst PositionWe are seeking a highly skilled Business Intelligence Analyst to drive business growth through data-driven insights.Key Responsibilities:Develop and implement predictive models using machine learning techniques to extract value from business data.Perform large-scale experimentation and build data-driven models to answer...