Big Data Engineer Lead

6 days ago


Bengaluru, Karnataka, India Brillio Full time ₹ 20,00,000 - ₹ 25,00,000 per year

Key Skills - Hadoop, PySpark, Python, SQL, Apache Iceberg (niche) , Hive

Requirements

  • Solid knowledge of Hadoop ecosystem such as Hive, Iceberg, Spark SQL and Proficiency in Python, Unix ,SQL
  • Experience with Apache Kafka, Apache Flink, and other relevant streaming technologies.
  • Strong hands-on experience with Hive and SQL for querying and data transformation
  • Proficiency in Python for data manipulation and automation
  • Expertise in Apache Spark (batch and streaming)
  • Deep understanding of Hadoop ecosystem (HDFS, YARN, MapReduce)
  • Experience working with Kafka for streaming data pipelines
  • Experience with workflow orchestration tools (Airflow, Oozie, etc.)
  • Knowledge of cloud-based big data platforms (AWS EMR, GCP Dataproc, Azure HDInsight)
  • Fam
  • iliarity with CI/CD pipelines and version control (Git)

Responsibilities

  • Design, develop, and maintain scalable data pipelines and ETL processes.
  • Work with large datasets using Hadoop ecosystem tools (Hive, Spark).
  • Build and optimize real-time and batch data processing solutions using Kafka and Spark Streaming.
  • Write efficient, high-performance SQL queries to extract, transform, and load data.
  • Develop reusable data frameworks and utilities in Python.
  • Collaborate with data scientists, analysts, and product teams to deliver reliable data solutions.
  • Monitor, troubleshoot, and optimize big data workflows for performance and cost efficiency.


  • Bengaluru, Karnataka, India CODERS BRAIN TECHNOLOGY PRIVATE LIMITED Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    About the Role : We are looking for an experienced AWS Big Data Lead Engineer to design, build, and manage scalable data solutions on AWS. The ideal candidate will have extensive hands-on experience in Big Data technologies, cloud-native architecture, and leadership to guide engineering teams in delivering enterprise-grade data pipelines and analytics...

  • AWS Big Data Lead

    2 weeks ago


    Bengaluru, Karnataka, India CODERS BRAIN TECHNOLOGY PRIVATE LIMITED Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    About the RoleWe are seeking a highly skilled AWS Big Data Lead & Technical Architect with over 10 years of experience in designing, implementing, and leading enterprise-grade Big Data solutions. The ideal candidate will combine deep technical expertise in AWS and Big Data technologies with strong leadership and architectural skills to drive large-scale data...

  • Big Data Lead

    1 week ago


    Bengaluru, Karnataka, India Persistent Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    About PersistentWe are a trusted Digital Engineering and Enterprise Modernization partner, combining deep technical expertise and industry experience to help our clients anticipate what's next. Our offerings and proven solutions create a unique competitive advantage for our clients by giving them the power to see beyond and rise above. We work with many...

  • Big Data Engineer

    1 week ago


    Bengaluru, Karnataka, India Reveille Technologies,Inc Full time ₹ 16,00,000 - ₹ 24,00,000 per year

    We're Hiring: Big Data Engineer – Full-TimeLooking for a skilledBig Data Engineerto join our team inBangaloreRole: Big Data EngineerGrade: G4 |TSRLocation: Bangalore (Onsite/Hybrid – specify if needed)Budget: ₹13,39,200Experience: 4–6 yearsSkills:Azure Databricks (ADB)PySparkSQLSpark (Scala/Python)TerraformAirflow & KubernetesData Warehousing &...

  • Big Data Engineer

    1 week ago


    Bengaluru, Karnataka, India The IT Firm Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Role OverviewAs a Big Data Engineer, you will be responsible for building and maintaining scalable data pipelines and architectures that support data analytics, reporting, and real-time processing. You will collaborate with data scientists, business analysts, and software engineers to design robust solutions that drive data-driven decisions.Mandatory Skill...

  • Big Data Engineer

    1 week ago


    Bengaluru, Karnataka, India Fractal Analytics Private Limited Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    BigData Engineer It's fun to work in a company where people truly BELIEVE in what they are doing We're committed to bringing passion and customer focus to the business.About the Role:We are seeking highly skilled Big Data Engineers/Senior Engineers with a strong background in building and supporting big data solutions. The ideal candidate will have...

  • Big Data Engineer

    17 hours ago


    Bengaluru, Karnataka, India Suzva Software Technologies Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Hiring: Big Data Engineer (PySpark)Client: [Details will be shared upon shortlisting]Location: Bangalore (Onsite)Experience: 6-8 years (Relevant)Joining: Immediate to 15 days Key Skills: Strong hands-on experience in PySpark Expertise in Big Data technologies Solid understanding of SQL, Data Pipelines, and ETLInterested candidates can share their resume...

  • Big Data Engineer

    2 days ago


    Bengaluru, Karnataka, India Innova ESI Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Role: Big Data EngineerExperience: 7+ YearsLocation: BangaloreNotice: Immediate Joiners OnlyJob Description:Azure Databricks, Azure Data Factory, Azure Function apps, Apache Spark, Scala, Java, Apache Kafka, event stream and Big DataOptional Skills: Airflow, PythonRoles & ResponsibilitiesOverall, 12 years of experience in IT industry, including 5 years in...

  • Big Data Engineer

    1 week ago


    Bengaluru, Karnataka, India Nagarro Softwares Pvt. Ltd Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Associate Staff Engineer, Big Data Engineer (LookML & Looker Studio Pro) Company Description We're Nagarro. We are a Digital Product Engineering company that is scaling in a big way We build products, services, and experiences that inspire, excite, and delight. We work at scale across all devices and digital mediums, and our people exist everywhere in...


  • Bengaluru, Karnataka, India Apple Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Apple is where individual imaginations gather together, committing to the values that lead to great work. Every new product we build, service we create, or Apple Store experience we deliver is the result of us making each other's ideas stronger. That happens because every one of us shares a belief that we can make something wonderful and share it with the...