Big Data Engineer

5 days ago

Thoothukudi, Tamil Nadu, India Idyllic Services Full time

Job Title: Big Data Engineer – Scala

Location: Bangalore, Chennai, Gurgaon, Pune, Mumbai.

Experience: 7–10 Years (Minimum 3+ years in Scala)

Notice Period: Immediate to 30 Days

Mode of Work: Hybrid

Click the link below to learn more about the role and take the AI Interview to begin your application journey: https://sl1nk.com/XiPeN

Role Overview

We are looking for a highly skilled Big Data Engineer (Scala) with strong expertise in Scala, Spark, Python, NiFi, and Apache Kafka to join our data engineering team. The ideal candidate will have a proven track record in building, scaling, and optimizing big data pipelines, and hands-on experience in distributed data systems and cloud-based solutions.

Key Responsibilities

- Design, develop, and optimize large-scale data pipelines and distributed data processing systems.

- Work extensively with Scala, Spark (PySpark), and Python for data processing and transformation.

- Develop and integrate streaming solutions using Apache Kafka and orchestration tools like NiFi / Airflow.

- Write efficient queries and perform data analysis using Jupyter Notebooks and SQL.

- Collaborate with cross-functional teams to design scalable cloud-based data architectures.

- Ensure delivery of high-quality code through code reviews, performance tuning, and best practices.

- Build monitoring and alerting systems leveraging Splunk or equivalent tools.

- Participate in CI/CD workflows using Git, Jenkins, and other DevOps tools.

- Contribute to product development with a focus on scalability, maintainability, and performance.

Mandatory Skills

- Scala – Minimum 3+ years of hands-on experience.

- Strong expertise in Spark (PySpark) and Python.

- Hands-on experience with Apache Kafka.

- Knowledge of NiFi / Airflow for orchestration.

- Strong experience in Distributed Data Systems (5+ years).

- Proficiency in SQL and query optimization.

- Good understanding of Cloud Architecture.

Preferred Skills

- Exposure to messaging technologies like Apache Kafka or equivalent.

- Experience in designing intuitive, responsive UIs for data analytics visualization.

- Familiarity with Splunk or other monitoring/alerting solutions.

- Hands-on experience with CI/CD tools (Git, Jenkins).

- Strong grasp of software engineering concepts, data modeling, and optimization techniques.

  • Senior Big Data Specialist

    3 days ago

    Thoothukudi, Tamil Nadu, India beBeeData Full time ₹ 23,40,000 - ₹ 30,14,000

    Job Title: Senior Data EngineerJob Summary:Highly skilled data engineer with 5+ years of experience in Python, PySpark, and SQL.Key Responsibilities:Expertise in big data processing, telemetry, and streaming analytics.Strong background in backend engineering and system architecture.Familiarity with GCP and experience working with IoT systems.Ability to...

  • Big Data Architect

    6 days ago

    Thoothukudi, Tamil Nadu, India beBeeData Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Title: Data ArchitectOverviewWe are seeking an experienced Data Architect to help design and implement a cutting-edge application for a large retailer. The ideal candidate will have a deep understanding of big data, IoT systems, and backend engineering.Key Responsibilities:Design and develop a scalable data architecture for the application.Collaborate...

  • Big Data Specialist

    6 days ago

    Thoothukudi, Tamil Nadu, India beBeeDataEngineering Full time ₹ 10,00,000 - ₹ 15,00,000

    Data Engineer Role SummaryWe're seeking a skilled Data Engineer to design and develop large-scale data infrastructures. This full-time role involves building scalable data pipelines using technologies like Kafka, Flink, and Spark.Main ResponsibilitiesDesigning Data Systems: Develop efficient data modeling processes and ETL systems to handle large volumes of...

  • Big Data Specialist

    6 days ago

    Thoothukudi, Tamil Nadu, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title: Data Engineer    We are seeking a skilled Data Engineer to join our organization. As a key member of our team, you will be responsible for designing, building, and maintaining large-scale data processing systems.Required Skills and Qualifications:• Proficiency in AWS technologies, including S3, Lambda, EC2, and EMR• Hands-on experience...

  • Enterprise Big Data Architect

    2 days ago

    Thoothukudi, Tamil Nadu, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 30,00,000

    Job TitleSeeking a Data Engineer to join our team, responsible for designing and building large-scale data pipelines.">Design, build, maintain production-level data pipelines handling structured/unstructured data.Own end-to-end ETL/ELT processes ensuring reliability/scalability/high availability.Work with AWS services (S3, EC2, RDS, DynamoDB, Athena,...

  • Senior Big Data Professional

    9 hours ago

    Thoothukudi, Tamil Nadu, India beBeeData Full time ₹ 18,00,000 - ₹ 25,00,000

    Job Description:As a seasoned Big Data professional, you will be responsible for designing and implementing large-scale data processing systems.The ideal candidate must have expertise in handling vast amounts of data, leveraging technologies like Scala, Python, or Java to drive business outcomes.A high level of collaboration with cross-functional teams is...

  • High-Level Big Data Expert

    9 hours ago

    Thoothukudi, Tamil Nadu, India beBeeData Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Senior Data Engineer Role The company is seeking a skilled Senior Data Engineer to join their team. This role involves designing, building, and maintaining high-performance data pipelines to support the organization's consulting and analytics solutions. Key responsibilities will include working with cross-functional teams to develop and implement data...

  • Cloud Big Data Architecture Expert

    9 hours ago

    Thoothukudi, Tamil Nadu, India beBeeBigData Full time ₹ 1,20,00,000 - ₹ 2,01,20,000

    Big Data Architect PositionWe are seeking a skilled Big Data Architect to join our team. As a key member of our digital and data solutions group, you will play a critical role in designing and implementing cloud-based big data management systems.This is a full-time hybrid role offering flexibility for remote work. You will be responsible for developing and...

  • Big Data Engineer

    4 days ago

    Thoothukudi, Tamil Nadu, India beBeeData Full time ₹ 15,00,000 - ₹ 25,00,000

    Senior Data Systems DeveloperWe are seeking a highly skilled Senior Data Systems Developer to design, build, and maintain large-scale data systems.The ideal candidate will collaborate with cross-functional teams to deliver high-quality data systems that power analytics, insights, and data-driven decision-making.Design and develop robust data pipelines and...

  • Data Engineering Specialist

    7 days ago

    Thoothukudi, Tamil Nadu, India beBeeDataEngineer Full time ₹ 25,00,000 - ₹ 35,00,000

    Senior Data Engineer RoleWe are seeking an experienced Senior Data Engineer to fill a key position in our data engineering team. As a core member of the team, you will be responsible for designing and building large-scale data platforms on Google Cloud (GCP).Key Responsibilities:Data Pipeline DesignDesign and build robust data pipelines for ingestion,...