Big Data Engineer

5 days ago

Bengaluru Chennai Pune, India Brillio Full time ₹ 15,00,000 - ₹ 20,00,000 per year

Job Title: Big Data Engineer

Role Overview

We are seeking a highly skilled Big Data Engineer to join our team.The ideal candidate will have strong experience in building, maintaining, and optimizing large-scale data pipelines and distributed data processing systems. This role involves working closely with cross-functional teams to ensure the reliability, scalability, and performance of data solutions.

Key Responsibilities

  • Design, develop, and maintain scalable data pipelines and ETL processes.
  • Work with large datasets using Hadoop ecosystem tools (Hive, Spark).
  • Build and optimize real-time and batch data processing solutions using Kafka and Spark Streaming.
  • Write efficient, high-performance SQL queries to extract, transform, and load data.
  • Develop reusable data frameworks and utilities in Python.
  • Collaborate with data scientists, analysts, and product teams to deliver reliable data solutions.
  • Monitor, troubleshoot, and optimize big data workflows for performance and cost efficiency.

Must-Have Skills

  • Strong hands-on experience with Hive and SQL for querying and data transformation.
  • Proficiency in Python for data manipulation and automation.
  • Expertise in Apache Spark (batch and streaming).
  • Experience working with Kafka for streaming data pipelines.

Good-to-Have Skills

  • Experience with workflow orchestration tools (Airflow, Oozie, etc.).
  • Knowledge of cloud-based big data platforms (AWS EMR, GCP Dataproc, Azure HDInsight).
  • Familiarity with CI/CD pipelines and version control (Git).

  • Big Data Engineer

    1 day ago

    Pune, Maharashtra, India Nice Software Solutions Pvt. Ltd. Full time

    Big Data Engineer (PySpark)Location: Pune/Nagpur (WFO)Experience: 8 - 12 YearsEmployment Type: Full-timeJob OverviewWe are looking for an experienced Big Data Engineer with strong expertise in PySpark and Big Data ecosystems. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high...

  • Big Data Engineer

    1 week ago

    Pune, Maharashtra, India Nice Software Solutions Pvt. Ltd. Full time

    Big Data Engineer (PySpark) Location: Pune/Nagpur (WFO) Experience: 8 - 12 Years Employment Type: Full-time Job Overview We are looking for an experienced Big Data Engineer with strong expertise in PySpark and Big Data ecosystems. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high...

  • Big Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India beBeeBigDataEngineer Full time

    Job Title: Big Data EngineerWe are seeking a highly skilled and experienced Big Data Engineer to join our team. The successful candidate will be responsible for designing, building, and maintaining large-scale data processing systems using Hadoop.Responsibilities:• Design, develop, and deploy big data solutions using Hadoop, Spark, and Scala.•...

  • Big Data Engineer

    1 week ago

    Chennai, Tamil Nadu, India Insight Global Full time

    Position: Big Data EngineerDuration: 12+ monthsLocation: India - Chennai or GurgaonRate: 4-17 LPARequired Skills & ExperienceAt least 6+ Years of experience with PySpark and MapRProgramming background with PythonExperience with SQL (Relational Databases) doing data modeling, Oracle experience is a plus.Experience with PowerBI and/or TableauJob...

  • Big Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Insight Global Full time

    Position: Big Data EngineerDuration: 12+ monthsLocation: India - Chennai or GurgaonRate: 4-17 LPARequired Skills & ExperienceAt least 6+ Years of experience with PySpark and MapRProgramming background with PythonExperience with SQL (Relational Databases) doing data modeling, Oracle experience is a plus.Experience with PowerBI and/or TableauJob...

  • Big Data Engineer

    5 days ago

    Pune, Maharashtra, India Nice Software Solutions Pvt. Ltd. Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    Big Data Engineer (PySpark)Location:Pune/Nagpur (WFO)Experience:8 - 12 YearsEmployment Type:Full-timeJob OverviewWe are looking for an experiencedBig Data Engineerwith strong expertise in PySpark and Big Data ecosystems. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines while ensuring high performance...

  • Big Data Engineer

    4 days ago

    Pune, Maharashtra, India Merito Full time

    Job Description :We are looking for a Big Data Engineer who will work on building, and managing Big Data Pipelines for us to deal with the huge structured data sets that we use as an input to accurately generate analytics at scale for our valued Customers. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining,...

  • Big Data Engineer

    3 weeks ago

    Pune, Maharashtra, India LION AND ELEPHANTS CONSULTANCY PRIVATE LIMITED Full time

    Job Description : We are looking for skilled Big Data Engineers using Java Spark with 5-10 years of experience in Big Data / legacy platforms, who can join immediately. Desired candidates should have design, development and optimization of real-time & batch data pipelines experience in a Big Data environment at an enterprise scale application. You will work...

  • Big Data Engineering Leader

    1 week ago

    Chennai, Tamil Nadu, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    We are seeking a highly skilled professional to lead our data engineering initiatives. As a Big Data Engineer, you will be responsible for designing and developing large-scale data processing systems.Key Responsibilities:Design and implement scalable data pipelines using PySpark and HiveDevelop and maintain big data applications using Spark, HBase, and DQ...

  • Big Data Engineer Position

    5 days ago

    Chennai, Bengaluru / Bangalore, India beBeebigData Full time ₹ 1,04,000 - ₹ 1,30,878

    Big Data Engineer OpportunityWe are seeking a skilled Big Data Engineer to join our team. 4+ years of experience in Big Data technologies.This is a permanent position with competitive benefits.Our company offers a dynamic work environment and opportunities for growth and development.