Big Data Engineer

4 days ago

Bengaluru, Karnataka, India Idyllic Services Full time

Job Title: Big Data Engineer – Scala

Location: Bangalore, Chennai, Gurgaon, Pune, Mumbai.

Experience: 7–10 Years (Minimum 3+ years in Scala)

Notice Period: Immediate to 30 Days

Mode of Work: Hybrid

Click the link below to learn more about the role and take the AI Interview to begin your application journey: https://sl1nk.com/XiPeN

Role Overview

We are looking for a highly skilled Big Data Engineer (Scala) with strong expertise in Scala, Spark, Python, NiFi, and Apache Kafka to join our data engineering team. The ideal candidate will have a proven track record in building, scaling, and optimizing big data pipelines, and hands-on experience in distributed data systems and cloud-based solutions.

Key Responsibilities

- Design, develop, and optimize large-scale data pipelines and distributed data processing systems.

- Work extensively with Scala, Spark (PySpark), and Python for data processing and transformation.

- Develop and integrate streaming solutions using Apache Kafka and orchestration tools like NiFi / Airflow.

- Write efficient queries and perform data analysis using Jupyter Notebooks and SQL.

- Collaborate with cross-functional teams to design scalable cloud-based data architectures.

- Ensure delivery of high-quality code through code reviews, performance tuning, and best practices.

- Build monitoring and alerting systems leveraging Splunk or equivalent tools.

- Participate in CI/CD workflows using Git, Jenkins, and other DevOps tools.

- Contribute to product development with a focus on scalability, maintainability, and performance.

Mandatory Skills

- Scala – Minimum 3+ years of hands-on experience.

- Strong expertise in Spark (PySpark) and Python.

- Hands-on experience with Apache Kafka.

- Knowledge of NiFi / Airflow for orchestration.

- Strong experience in Distributed Data Systems (5+ years).

- Proficiency in SQL and query optimization.

- Good understanding of Cloud Architecture.

Preferred Skills

- Exposure to messaging technologies like Apache Kafka or equivalent.

- Experience in designing intuitive, responsive UIs for data analytics visualization.

- Familiarity with Splunk or other monitoring/alerting solutions.

- Hands-on experience with CI/CD tools (Git, Jenkins).

- Strong grasp of software engineering concepts, data modeling, and optimization techniques.

  • Big Data Engineer

    1 week ago

    Bengaluru, Karnataka, India Coders Brain Full time

    Job DescriptionRoles and Responsibilities:- Develop scalable infrastructure and platform to collect, analyze and process large amounts of structured and unstructured data with real-time data interpretation.- Work closely across an array of various teams and organizations in the company (including partners, customers)Basic Qualifications:- Bachelor's degree...

  • Big Data Engineer

    17 hours ago

    Bengaluru, Karnataka, India Reveille Technologies,Inc Full time ₹ 1,04,000 - ₹ 13,39,200 per year

    We're Hiring: Big Data Engineer – Full-TimeLooking for a skilledBig Data Engineerto join our team inBangaloreRole: Big Data EngineerGrade: G4 |TSRLocation: Bangalore (Onsite/Hybrid – specify if needed)Budget: ₹13,39,200Experience: 4–6 yearsSkills:Azure Databricks (ADB)PySparkSQLSpark (Scala/Python)TerraformAirflow & KubernetesData Warehousing &...

  • Big Data Engineer

    4 days ago

    Bengaluru, Karnataka, India Xebia Full time

    We're Hiring – Big Data Engineer (Scala + Spark)&## ; Location: Bangalore – Whitefield (Hybrid: 3 Days Office / Week)&##Availability: Immediate to Max 2 Weeks Notice Big Data & Hadoop Ecosystem Python Data Structures & Algorithms Design & Coding Skills Experience in distributed data processing & performance optimization At Xebia, you'll be part of a...

  • Big Data Engineer

    5 days ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job Title: Big Data EngineerOverviewWe are seeking a highly skilled professional to design, develop and implement large-scale data engineering solutions.The ideal candidate will be part of a fast-paced team focused on delivering cutting-edge technology and product roadmaps.The team is responsible for developing marketing campaign tracking capabilities using...

  • Big Data Engineer

    1 day ago

    Bengaluru, Karnataka, India IT Firm Full time

    Role Overview :As a Big Data Engineer, you will be responsible for building and maintaining scalable data pipelines and architectures that support data analytics, reporting, and real-time processing. You will collaborate with data scientists, business analysts, and software engineers to design robust solutions that drive data-driven decisions.Mandatory Skill...

  • Big Data Solutions Engineer

    1 day ago

    Bengaluru, Karnataka, India beBeeDataEngineering Full time ₹ 15,00,000 - ₹ 25,00,000

    Big Data Solutions EngineerWe are seeking a highly skilled and experienced Big Data Solutions Engineer to lead and manage our team of data engineers. As a key member of our organization, you will be responsible for designing, building, and maintaining scalable big data platforms and pipelines that meet the needs of our business.Your primary focus will be on...

  • Big Data Engineer

    1 week ago

    Bengaluru, Karnataka, India Xebia Full time

    We're Hiring – Big Data Engineer (Scala + Spark)Location: Bangalore – Whitefield (Hybrid: 3 Days Office / Week)Budget: Up to 30 LPAAvailability: Immediate to Max 2 Weeks NoticeMust Have Skills:Scala & Spark (Mandatory)Big Data & Hadoop EcosystemHive, Kafka, OoziePythonData Structures & AlgorithmsDesign & Coding SkillsCI/CD PipelinesProblem Solving...

  • Big Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India Xebia Full time

    ???? We're Hiring – Big Data Engineer (Scala + Spark)???? Location: Bangalore – Whitefield (Hybrid: 3 Days Office / Week)???? Budget: Up to 30 LPA???? Availability: Immediate to Max 2 Weeks NoticeMust Have Skills:Scala & Spark (Mandatory)Big Data & Hadoop EcosystemHive, Kafka, OoziePythonData Structures & AlgorithmsDesign & Coding SkillsCI/CD...

  • Big Data Engineer

    2 days ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 1,04,000 - ₹ 1,30,878

    Job Title: Big Data EngineerWe are seeking a highly skilled Big Data Engineer to join our team. As a key member of our data engineering team, you will play a critical role in designing, building, and maintaining large-scale data processing systems.This is an exceptional opportunity for a talented engineer who is passionate about working with big data...

  • Big Data Engineer

    7 days ago

    Bengaluru, Karnataka, India RiskInsight Consulting Pvt Ltd Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    ResponsibilitiesDesign, develop, and implement robust Big Data solutions using technologies such as Hadoop, Spark, and NoSQL databases. Build and maintain scalable data pipelines for effective data ingestion, transformation, and analysis. Collaborate with data scientists, analysts, and cross-functional teams to understand business requirements and translate...