Big Data Engineer

7 days ago

Pushkar, Rajasthan, India Idyllic Services Full time

Job Title: Big Data Engineer – Scala

Location: Bangalore, Chennai, Gurgaon, Pune, Mumbai.

Experience: 7–10 Years (Minimum 3+ years in Scala)

Notice Period: Immediate to 30 Days

Mode of Work: Hybrid

Role Overview

We are looking for a highly skilled Big Data Engineer (Scala) with strong expertise in Scala, Spark, Python, NiFi, and Apache Kafka to join our data engineering team. The ideal candidate will have a proven track record in building, scaling, and optimizing big data pipelines, and hands-on experience in distributed data systems and cloud-based solutions.

Key Responsibilities

- Design, develop, and optimize large-scale data pipelines and distributed data processing systems.

- Work extensively with Scala, Spark (PySpark), and Python for data processing and transformation.

- Develop and integrate streaming solutions using Apache Kafka and orchestration tools like NiFi / Airflow.

- Write efficient queries and perform data analysis using Jupyter Notebooks and SQL.

- Collaborate with cross-functional teams to design scalable cloud-based data architectures.

- Ensure delivery of high-quality code through code reviews, performance tuning, and best practices.

- Build monitoring and alerting systems leveraging Splunk or equivalent tools.

- Participate in CI/CD workflows using Git, Jenkins, and other DevOps tools.

- Contribute to product development with a focus on scalability, maintainability, and performance.

Mandatory Skills

- Scala – Minimum 3+ years of hands-on experience.

- Strong expertise in Spark (PySpark) and Python.

- Hands-on experience with Apache Kafka.

- Knowledge of NiFi / Airflow for orchestration.

- Strong experience in Distributed Data Systems (5+ years).

- Proficiency in SQL and query optimization.

- Good understanding of Cloud Architecture.

Preferred Skills

- Exposure to messaging technologies like Apache Kafka or equivalent.

- Experience in designing intuitive, responsive UIs for data analytics visualization.

- Familiarity with Splunk or other monitoring/alerting solutions.

- Hands-on experience with CI/CD tools (Git, Jenkins).

- Strong grasp of software engineering concepts, data modeling, and optimization techniques.

  • Big Data Engineering Lead

    2 weeks ago

    Pushkar, Rajasthan, India beBeeEngineering Full time ₹ 1,50,00,000 - ₹ 3,00,00,000

    We are seeking an experienced Big Data Engineering Lead to join our team.About the RoleLead a team of data engineers across multiple geographies and time zones.Collaborate with cross-functional teams, including architects, product managers, and operations.Design and implement scalable big data solutions using Apache Spark and Python.Develop data pipelines...

  • Senior Big Data Developer

    2 weeks ago

    Pushkar, Rajasthan, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Job DescriptionWe are seeking an experienced Big Data Engineer to join our team. As a Cloud Engineer, you will be responsible for designing and implementing scalable big data architectures, developing and deploying data pipelines for structured and unstructured data, and collaborating with cross-functional teams to deliver high-quality solutions.Required...

  • Big Data Engineering Lead

    1 week ago

    Pushkar, Rajasthan, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Lead Data Engineer PositionWe are seeking an experienced data engineer to deliver roadmap features for our Enterprise TruRisk Platform. This platform helps customers measure, communicate, and eliminate cyber risks.You will work closely with engineers and architects to prototype, design, develop, and support a highly scalable, distributed SaaS-based Security...

  • Expert Big Data Developer

    1 week ago

    Pushkar, Rajasthan, India beBeeBigData Full time ₹ 90,00,000 - ₹ 1,50,00,000

    Job OverviewWe are seeking a skilled expert in Big Data to lead high-impact projects and drive business growth through hands-on development, mentoring junior engineers, and implementing scalable solutions.Main Responsibilities

  • Big Data Product Developer

    6 days ago

    Pushkar, Rajasthan, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Unlocking Data InsightsWe empower businesses to transform through intelligent product development.Our approach focuses on innovation and acceleration, enabling clients to build and modernize products that drive meaningful outcomes.With expertise in big data analytics platforms and extensive experience in Python/Java, CI/CD, infrastructure & cloud, Terraform,...

  • Big Data Architect

    2 weeks ago

    Pushkar, Rajasthan, India beBeeData Full time ₹ 17,29,400 - ₹ 21,43,700

    Job Title: Big Data ArchitectAt our organization, you will play a key role in designing and managing large-scale data systems.The team is responsible for building and managing platforms that handle high-scale services and host millions of unique topics. To achieve this, we utilize cutting-edge Machine Learning and AI technologies on a large Hadoop...

  • Senior Big Data Architect

    1 week ago

    Pushkar, Rajasthan, India beBeeData Full time ₹ 10,00,000 - ₹ 15,00,000

    Big Data Engineer Job OverviewThe role of Big Data Engineer involves designing, developing, and optimizing large-scale data pipelines and distributed data processing systems. This position requires strong expertise in Scala, Spark, Python, NiFi, and Apache Kafka to join our data engineering team.Key Responsibilities:Data Pipeline Development: Design and...

  • Big Data Solutions Architect

    1 week ago

    Pushkar, Rajasthan, India beBeeData Full time ₹ 15,40,400 - ₹ 25,42,200

    Cloud Data ArchitectWe are seeking a seasoned professional with extensive experience in Big Data technologies and Google Cloud Platform services to design and implement scalable data solutions.Design, develop, and optimize data pipelines using GCP BigQuery, Dataflow, and Apache Airflow to support large-scale data analytics.Utilize the Big Data Hadoop...

  • Big Data Analysis Expert

    2 weeks ago

    Pushkar, Rajasthan, India beBeeDataInsight Full time ₹ 11,50,000 - ₹ 2,01,50,000

    Job Title: Data Insight SpecialistWe are seeking an experienced Data Analyst to deliver high-quality data solutions. This role involves working on end-to-end project lifecycles, handling complex queries, and providing timely results.Main Responsibilities:Analyze business requirements and understand technical specifications.Coding using Spark and Scala based...

  • Data Engineering Professional

    2 weeks ago

    Pushkar, Rajasthan, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 25,00,000

    About Data Engineering:We are looking for an expert professional to help organizations accelerate their transition to a digital and sustainable world.Our ideal candidate will have extensive knowledge of big data technologies, including HDFS, Hive, Spark architecture, and data lakehouse concepts.Job Requirements:Expertise in Programming LanguagesDetailed...