Big Data Specialist

8 hours ago

bangalore, India Brillio Full time

Role Overview

We are seeking a highly skilled Big Data Engineer to join our team. The ideal candidate will have strong experience in building, maintaining, and optimizing large-scale data pipelines and distributed data processing systems. This role involves working closely with cross-functional teams to ensure the reliability, scalability, and performance of data solutions.


Key Responsibilities

  • Design, develop, and maintain scalable data pipelines and ETL processes.
  • Work with large datasets using Hadoop ecosystem tools (Hive, Spark).
  • Build and optimize real-time and batch data processing solutions using Kafka and Spark Streaming.
  • Write efficient, high-performance SQL queries to extract, transform, and load data.
  • Develop reusable data frameworks and utilities in Python.
  • Collaborate with data scientists, analysts, and product teams to deliver reliable data solutions.
  • Monitor, troubleshoot, and optimize big data workflows for performance and cost efficiency.


Must-Have Skills

  • Strong hands-on experience with Hive and SQL for querying and data transformation.
  • Proficiency in Python for data manipulation and automation.
  • Expertise in Apache Spark (batch and streaming).
  • Experience working with Kafka for streaming data pipelines.


Good-to-Have Skills

  • Experience with workflow orchestration tools (Airflow etc.)
  • Knowledge of cloud-based big data platforms (AWS EMR, GCP Dataproc, Azure HDInsight).
  • Familiarity with CI/CD pipelines and version control (Git).

  • Big Data Specialist

    1 day ago

    Bangalore, India Brillio Full time

    Role Overview We are seeking a highly skilled Big Data Engineer to join our team. The ideal candidate will have strong experience in building, maintaining, and optimizing large-scale data pipelines and distributed data processing systems. This role involves working closely with cross-functional teams to ensure the reliability, scalability, and performance...

  • Big Data Analytics Specialist

    4 days ago

    Bengaluru / Bangalore, Hyderabad / Secunderabad, Telangana, India beBeeData Full time ₹ 1,50,000 - ₹ 28,00,000

    Job Title: Big Data Analytics Specialist\Hadoop Spark and Hive are at the core of our big data strategy. As a Big Data Analytics Specialist, you will be responsible for designing, developing, and maintaining large-scale data processing systems using Hadoop and Spark technologies.\Key Responsibilities:\\Design and develop ETL processes using Hadoop and Spark...

  • Big Data Data Processing Specialist

    2 weeks ago

    Hyderabad / Secunderabad, Telangana, Chennai, Bengaluru / Bangalore, India beBeeDataProcessing Full time ₹ 1,04,000 - ₹ 1,30,878

    Our ideal candidate is a highly skilled Data Processing Specialist with expertise in designing and implementing efficient data processing solutions using PySpark. The selected individual will be responsible for developing robust data pipelines to handle large-scale datasets, ensuring high-quality data integration and transformation.This role requires strong...

  • Data Engineer

    1 day ago

    Bangalore, India AMISEQ Full time

    Job Role:- Data Engineer Location: Bangalore Experience: 3- 6 years Required skills Kafka, Flink, Java Big Data experience- 3-4 years' 3 years of hands-on experience supporting Java-based applications Experience working with Apache Flink, Apache Spark , or other distributed data processing frameworks Understanding of production concerns like data...

  • Big Data Developer

    10 hours ago

    bangalore, India Tata Consultancy Services Full time

    Position - Big Data DeveloperExperience: 4 to 10 YearsNotice Period- 0-60 DaysLocation-BengaluruRequired Skill Set_ Hadoop, PySpark, Spark SQL, HIVEJob DescriptionHands-on experience of PySpark, Spark, Hive, Hadoop Big Data Eco System Tools.Strong SQL experience.Should be able to develop, tweak queries and work on performance enhancement.Solid understanding...

  • Architect - Big data and Cloud

    8 hours ago

    bangalore, India Societe Generale Global Solution Centre Full time

    Role: Cloud and Big data ArchitectRole DescriptionWe are looking for skilled and passionate Azure Cloud experts with a strong focus on Big Data management and data quality skills to join our team at Société Générale Global Solution Centre in Bengaluru.This is a full-time hybrid role for Digital & Data Solutions (DDS), offering flexibility for remote...

  • Big Data Engineer

    5 days ago

    Chennai, Bengaluru / Bangalore, India beBeeBigDataEngineer Full time ₹ 15,00,000 - ₹ 28,00,000

    Job OverviewWe are seeking a highly skilled Big Data Engineer to join our team. As a Big Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data processing systems.Key Responsibilities:Design and implement big data solutions using Hadoop and Spark technologies.Work with cross-functional teams to understand business...

  • Advanced Big Data Specialist

    6 days ago

    Bengaluru / Bangalore, Hyderabad / Secunderabad, Telangana, India beBeeDataEngineer Full time ₹ 1,04,000 - ₹ 1,30,878

    Big Data EngineerAt our organization, we're looking for an experienced Big Data Engineer to join our team. As a key member of our data engineering team, you will be responsible for designing, building, and maintaining large-scale data processing systems.Required Skills:Programming: Proficiency in Python is mandatory, as well as experience with other...

  • Senior Big Data Engineer

    9 hours ago

    bangalore, India Veltris Full time

    Veltris is a Digital Product Engineering Services partner committed to driving technology-enabled transformation across enterprises, businesses, and industries. We specialize in delivering next-generation solutions for sectors including healthcare, technology, communications, manufacturing, and finance. With a focus on innovation and acceleration, Veltris...

  • Big Data Analyst

    2 weeks ago

    Bengaluru / Bangalore, Chennai, Pune, India beBeeData Full time ₹ 8,00,000 - ₹ 12,00,000

    **Job Description:**A day in the life of a data scientist involves working with large datasets to uncover patterns and insights. This role requires strong analytical skills, attention to detail, and the ability to communicate complex ideas effectively.As a key member of our team, you will be responsible for developing and implementing solutions to business...