Big Data Specialist

6 days ago


Bengaluru, Karnataka, India Brillio Full time

Role Overview

We are seeking a highly skilled Big Data Engineer to join our team. The ideal candidate will have strong experience in building, maintaining, and optimizing large-scale data pipelines and distributed data processing systems. This role involves working closely with cross-functional teams to ensure the reliability, scalability, and performance of data solutions.

Key Responsibilities

  • Design, develop, and maintain scalable data pipelines and ETL processes.
  • Work with large datasets using Hadoop ecosystem tools (Hive, Spark).
  • Build and optimize real-time and batch data processing solutions using Kafka and Spark Streaming.
  • Write efficient, high-performance SQL queries to extract, transform, and load data.
  • Develop reusable data frameworks and utilities in Python.
  • Collaborate with data scientists, analysts, and product teams to deliver reliable data solutions.
  • Monitor, troubleshoot, and optimize big data workflows for performance and cost efficiency.

Must-Have Skills

  • Strong hands-on experience with Hive and SQL for querying and data transformation.
  • Proficiency in Python for data manipulation and automation.
  • Expertise in Apache Spark (batch and streaming).
  • Experience working with Kafka for streaming data pipelines.

Good-to-Have Skills

  • Experience with workflow orchestration tools (Airflow etc.)
  • Knowledge of cloud-based big data platforms (AWS EMR, GCP Dataproc, Azure HDInsight).
  • Familiarity with CI/CD pipelines and version control (Git).

  • Big Data Specialist

    2 days ago


    Bengaluru, Karnataka, India beBeeData Full time ₹ 8,00,000 - ₹ 15,00,000

    Job Title: Big Data SpecialistExperience:2+ years of experience in big data development.The ideal candidate will have expertise in:Big Data DevelopmentAdvanced AnalyticsCloud ComputingProgramming Skills (Python, Java)As a big data specialist, you will be responsible for designing and developing large-scale data processing systems. Your focus will be on...

  • Big Data Specialist

    6 days ago


    Bengaluru, Karnataka, India beBeeData Full time ₹ 12,00,000 - ₹ 15,00,000

    Big Data SpecialistGreetings from our team!We are seeking an experienced Big Data Specialist to join our team. As a key member, you will play a critical role in driving business growth through data-driven insights.This is an exceptional opportunity for a motivated and talented individual who thrives in a fast-paced environment. Our ideal candidate will have...


  • Bengaluru, Karnataka, India beBeeInfrastructure Full time ₹ 12,00,000 - ₹ 15,00,000

    Job Title:Big Data Infrastructure SpecialistJob Overview:We are seeking a skilled Big Data Infrastructure Specialist to design and implement scalable data infrastructure solutions.Key Responsibilities:Deploy and manage Hadoop clusters on Apache and CDHConfigure and secure Hadoop environments using Yarn and HiveDesign and implement data pipelines using Spark,...

  • Big Data Specialist

    5 hours ago


    Bengaluru, Karnataka, India beBeeDataEngineering Full time ₹ 18,00,000 - ₹ 2,02,50,000

    Job Opportunity: Big Data SpecialistAbout the Role:The ideal candidate will have experience working with large-scale data systems, designing and implementing efficient data pipelines, and ensuring seamless integration of data sources.Responsibilities:Design, develop, and maintain scalable big data processing systems using Python and PySparkCollaborate with...


  • Bengaluru, Karnataka, India beBeeData Full time ₹ 15,00,000 - ₹ 20,00,000

    Big Data SpecialistWe are seeking a highly skilled big data specialist to design, develop, and maintain large-scale data systems. This individual will be responsible for architecting and implementing data pipelines using various distributed data processing frameworks.",

  • Big Data Specialist

    2 days ago


    Bengaluru, Karnataka, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

    Data Engineer PositionJob Summary:We are seeking a skilled Big Data Specialist to join our team of experts. The ideal candidate will be responsible for designing, developing, and maintaining scalable data platforms and architectures to support our analytics and business intelligence needs.About Us:We believe in harnessing the power of technology to drive...

  • Big Basket

    4 days ago


    Bengaluru, Karnataka, India big basket Full time

    What would you be doing/ Expected from this role ?- Collaborate with cross-functional teams including data scientists, engineers, and product managers to deliver AI-driven solutions.- Drive the architecture of machine learning pipelines from data collection and preprocessing to model training, evaluation, and deployment.- Implement scalable machine learning...

  • Big Basket

    4 days ago


    Bengaluru, Karnataka, India big basket Full time

    As a Software Engineer we are looking for developers who share our passion for building world class software. If you are excited about the prospect of changing the way India shops for groceries and being a pioneer then this a great home for you. Not to mention the thrill of building products that millions use daily and using cutting edge technology to make...

  • Big Data Specialist

    4 days ago


    Bengaluru, Karnataka, India beBeeBigData Full time ₹ 15,00,000 - ₹ 25,00,000

    Job Opportunity: Big Data SpecialistAbout the Role:We are seeking a skilled data professional to design and develop high-quality applications using Google BigQuery.Main Responsibilities:Independently perform tasks and become an SME in your area of expertise.Participate actively in team discussions and contribute solutions to work-related problems.Collaborate...


  • Bengaluru, Karnataka, India beBeeTechnical Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    Job Title:Support SpecialistJob Description:We are seeking a highly skilled Support Specialist to join our team. The successful candidate will have experience in big data environments and be proficient in Apache Spark, Druid, and Trino.Key Responsibilities:Monitor Spark jobs and supervise data ingestion pipelines in Druid.Provide Level 1 production support...