Big Data Engineer

1 week ago

Davangere, Karnataka, India Idyllic Services Full time

Job Title: Big Data Engineer – Scala

Location: Bangalore, Chennai, Gurgaon, Pune, Mumbai.

Experience: 7–10 Years (Minimum 3+ years in Scala)

Notice Period: Immediate to 30 Days

Mode of Work: Hybrid

Click the link below to learn more about the role and take the AI Interview to begin your application journey:

Role Overview

We are looking for a highly skilled Big Data Engineer (Scala) with strong expertise in Scala, Spark, Python, NiFi, and Apache Kafka to join our data engineering team. The ideal candidate will have a proven track record in building, scaling, and optimizing big data pipelines , and hands-on experience in distributed data systems and cloud-based solutions.

Key Responsibilities

- Design, develop, and optimize large-scale data pipelines and distributed data processing systems.

- Work extensively with Scala, Spark (PySpark), and Python for data processing and transformation.

- Develop and integrate streaming solutions using Apache Kafka and orchestration tools like NiFi / Airflow .

- Write efficient queries and perform data analysis using Jupyter Notebooks and SQL .

- Collaborate with cross-functional teams to design scalable cloud-based data architectures .

- Ensure delivery of high-quality code through code reviews, performance tuning, and best practices .

- Build monitoring and alerting systems leveraging Splunk or equivalent tools .

- Participate in CI/CD workflows using Git, Jenkins, and other DevOps tools.

- Contribute to product development with a focus on scalability, maintainability, and performance.

Mandatory Skills

- Scala – Minimum 3+ years of hands-on experience.

- Strong expertise in Spark (PySpark) and Python .

- Hands-on experience with Apache Kafka .

- Knowledge of NiFi / Airflow for orchestration.

- Strong experience in Distributed Data Systems (5+ years) .

- Proficiency in SQL and query optimization.

- Good understanding of Cloud Architecture .

Preferred Skills

- Exposure to messaging technologies like Apache Kafka or equivalent.

- Experience in designing intuitive, responsive UIs for data analytics visualization.

- Familiarity with Splunk or other monitoring/alerting solutions .

- Hands-on experience with CI/CD tools (Git, Jenkins).

- Strong grasp of software engineering concepts, data modeling, and optimization techniques .

  • Senior Big Data Engineer

    5 days ago

    Davangere, Karnataka, India beBeeDataEngineer Full time ₹ 20,00,000 - ₹ 35,00,000

    Big Data Engineer Job DescriptionWe are seeking a highly skilled Big Data Engineer to join our team. The ideal candidate will have experience in designing, executing, and managing large and complex distributed data systems.The successful candidate will have expertise in big data technologies (Apache Hadoop) and relational databases (MS SQL...

  • Senior Big Data Engineer

    3 days ago

    Davangere, Karnataka, India beBeeBigData Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Senior Big Data Engineer">We are seeking a highly skilled and experienced Senior Big Data Engineer with expertise in designing, architecting, and implementing scalable data engineering solutions using AWS Cloud Services.">Key Responsibilities:">Design, develop, and optimize Big Data architectures leveraging AWS services for large-scale, complex...

  • Senior Big Data Engineer

    3 days ago

    Davangere, Karnataka, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 25,00,000

    Data Engineer Position\We are seeking a skilled Data Engineer to lead our data systems team. The ideal candidate will have 3-5 years of experience in data modeling, ETL, and Snowflake.\The successful candidate will be responsible for:\Designing and developing complex data pipelines using Snowflake DBImplementing machine learning methods using linear...

  • Big Data Developer

    5 days ago

    Davangere, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 22,00,000

    We are seeking a skilled Data Engineer to support the growth of our organization's data management capabilities.Key ResponsibilitiesOur ideal candidate will design, develop, and maintain large-scale data processing systems using Big Data technologies such as Hadoop, Scala, and Spark. They will also be responsible for developing and optimizing complex SQL...

  • Advanced Big Data Engineer

    3 days ago

    Davangere, Karnataka, India beBeeBigDataDeveloper Full time ₹ 15,00,000 - ₹ 25,00,000

    Job TitleWe are seeking a skilled Big Data Developer to join our team. The ideal candidate will have a strong background in big data technologies and be able to design, develop, and implement scalable solutions.",

  • Senior Big Data Engineer

    3 days ago

    Davangere, Karnataka, India beBeeBigDataDeveloper Full time ₹ 80,00,000 - ₹ 1,50,00,000

    Big Data Development OpportunityWe are seeking an experienced professional to design and implement big data solutions utilizing Java, Scala, and Apache Spark Frameworks.Develop and deploy scalable data processing systems using HDFS commands.Utilize query languages such as Oracle SQL, Hive SQL, Spark SQL, Impala, and HBase DB to extract valuable insights from...

  • Unlock Big Data Potential

    1 week ago

    Davangere, Karnataka, India beBeeData Full time US$ 70,000 - US$ 1,50,000

    Big Data Engineer OpportunityWe are seeking a highly skilled professional to leverage big data and drive business growth.About the Role:Design, develop, and implement large-scale data processing systems using big data technologies such as PySpark, Hive, Spark, HBase, and DQ tools.Expertise in data ingestion from diverse sources onto big data platforms and...

  • Big Data Architect

    2 days ago

    Davangere, Karnataka, India beBeeDataScientist Full time ₹ 1,75,00,000 - ₹ 2,25,00,000

    About People Prime Worldwide:We are a company that enables enterprises to reimagine business models and accelerate innovation through digital technologies. We cater to clients with domain and technology expertise to drive competitive differentiation, customer experiences, and business outcomes.Job Title: Data Scientist & EngineerJob Description:We are...

  • Big Data Migration Specialist

    5 days ago

    Davangere, Karnataka, India beBeeData Full time ₹ 4,50,000 - ₹ 9,00,000

    Big Data Migration SpecialistJob Summary:Project OverviewThis role supports a critical enterprise-wide data migration initiative, ensuring seamless transition from legacy to modernized platforms. The engineer will be the sole Data QA resource, guaranteeing data integrity and dashboard accuracy throughout the migration.Key Responsibilities

  • Big Data Expert

    4 days ago

    Davangere, Karnataka, India beBeeData Full time ₹ 17,50,000 - ₹ 22,50,000

    Senior Big Data EngineerThe ideal candidate will possess a strong background in Python and PySpark programming languages, along with a solid understanding of HDFS, Hive, and Spark architecture. This expertise will enable them to design, develop, and maintain large-scale data processing systems.">Strong knowledge of Python and PySpark programming...