Big Data Engineer

1 week ago


Bengaluru, Karnataka, India Brillio Full time

Key skills - Hive, SQL, Python, Spark, Hadoop and kafka , apache airflow

Job Title Big Data Engineer

Role Overview

We are seeking a highly skilled Big Data Engineer to join our team. The ideal candidate will have strong experience in building, maintaining, and optimizing large-scale data pipelines and distributed data processing systems. This role involves working closely with cross-functional teams to ensure the reliability, scalability, and performance of data solutions.

Key Responsibilities

  • Design, develop, and maintain scalable data pipelines and ETL processes.
  • Work with large datasets using Hadoop ecosystem tools (Hive, Spark).
  • Build and optimize real-time and batch data processing solutions using Kafka and Spark Streaming.
  • Write efficient, high-performance SQL queries to extract, transform, and load data.
  • Develop reusable data frameworks and utilities in Python.
  • Collaborate with data scientists, analysts, and product teams to deliver reliable data solutions.
  • Monitor, troubleshoot, and optimize big data workflows for performance and cost efficiency.

Must-Have Skills

  • Strong hands-on experience with Hive and SQL for querying and data transformation.
  • Proficiency in Python for data manipulation and automation.
  • Expertise in Apache Spark (batch and streaming).
  • Experience working with Kafka for streaming data pipelines.

Good-to-Have Skills

  • Experience with workflow orchestration tools (Airflow etc.).
  • Knowledge of cloud-based big data platforms (AWS EMR)

Familiarity with CI/CD pipelines and version control (Git).



  • Bengaluru, Karnataka, India Data Engineer Academy Full time US$ 4,80,000 - US$ 12,00,000 per year

    We're Hiring: Data Engineering Subject Matter Expert (SME) – Snowflake,Cloud & DatabricksRemote | Part Time | Flexible Timings | Pay: $600 – $1000/MonthlyAbout the OpportunityWe are seeking a highly experienced SME with extensive expertise in Snowflake, Databricks, and AWS cloud platforms.What We're Looking ForMinimum 9-15 years of experience in cloud...

  • Big Data Engineer

    1 week ago


    Bengaluru, Karnataka, India Acme Services Full time ₹ 5,00,000 - ₹ 15,00,000 per year

    We are looking for Big Data Engineer who has experience with:- 1) Big Query 2) Pyspark 3) GCP or AWS or Azure Required Candidate profileCandidate should have more than 5 Years of overall experience in Data Engineer. We are not looking for Data Analyst or Snowflake Developer.

  • Big Data Engineer

    4 days ago


    Bengaluru, Karnataka, India Aqilea (formerly Soltia) Full time

    We are a consulting company with a bunch of technology-interested and happy people We love technology, we love design and we love quality. Our diversity makes us unique and creates an inclusive and welcoming workplace where each individual is highly valued. With us, each individual is her/himself and respects others for who they are and we believe that when...

  • Big Data Engineer

    2 days ago


    Bengaluru, Karnataka, India Aqilea Full time

    We are a consulting company with a bunch of technology-interested and happy peopleWe love technology, we love design and we love quality. Our diversity makes us unique and creates an inclusive and welcoming workplace where each individual is highly valued.With us, each individual is her/himself and respects others for who they are and we believe that when a...

  • Big Data Engineer

    2 days ago


    Bengaluru, Karnataka, India RiskInsight Consulting Full time

    ResponsibilitiesDesign, develop, and implement robust Big Data solutions using technologies such as Hadoop, Spark, and NoSQL databases. Build and maintain scalable data pipelines for effective data ingestion, transformation, and analysis. Collaborate with data scientists, analysts, and cross-functional teams to understand business requirements and translate...

  • Data Engineer

    2 weeks ago


    Bengaluru, Karnataka, India NTT DATA Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    As a Data Engineer, your main responsibilities will involve: The project is a migration of a SAS code to DBT code, so it's important that the candidate has experience in this type of projects. Analyze data models and derive logical conclusions. Processes modeling. Hands on development and monitoring of the Azure cloud Platform and various associated...

  • Big Data Engineer

    1 week ago


    Bengaluru, Karnataka, India Adobe Full time ₹ 20,00,000 - ₹ 40,00,000 per year

    Our CompanyChanging the world through digital experiences is what Adobe's all about. We give everyone—from emerging artists to global brands—everything they need to design and deliver exceptional digital experiences We're passionate about empowering people to create beautiful and powerful images, videos, and apps, and transform how companies interact...

  • Big Data Engineer

    2 days ago


    Bengaluru, Karnataka, India Talent Worx Full time

    Location – Pune/ChennaiExp - 6 to 9RequirementsTechnology: - Big Data (Hadoop, Hive, Spark, Scala, PLSQL/ No SQL Database)Knowledge/Experience (Must):· 6-9 years of experience in Hadoop/big data technologies.· Experience with Spark/Storm/Kafka or equivalent streaming/batch processing and eventbased messaging.· Relational and NoSQL database integration...

  • Big Data Engineer

    1 hour ago


    Bengaluru, Karnataka, India Saransh Software Solutions Full time

    Job Title: Data EngineerExperience: 8+ yearsLocation: BengaluruNotice Period: Immediate to 20 DaysEmployment Type: Full-timeAbout the RoleWe are looking for an experienced Data Engineer with strong expertise in Python, Spark, and PySpark to design, develop, and optimize large-scale data pipelines and distributed systems. The ideal candidate will possess a...


  • Bengaluru, Karnataka, India datavruti Full time

    Hiring for:A 20 years+ old US based IT services and product businessRole:Senior Big Data EngineerExperience:6 years+Location:Bengaluru / GurgaonType:Hybrid - 3 days WFOSalary:Up to 30LPA based on fitmentIMMEDIATE JOINERS ONLY- must be able to join in 1st week of Jan'26Overview:Our client is looking for a skilled seniorBig Data Engineerwithextremely strong...