Big Data Engineering Lead

8 hours ago

Chennai, Tamil Nadu, India beBeeData Full time ₹ 1,50,00,000 - ₹ 2,50,00,000
Senior Big Data Engineer

We are seeking a highly skilled Senior Big Data Engineer to lead our data engineering team and drive the development of scalable, secure, and high-performance big data solutions.

  • Experience in leading distributed teams across multiple geographies and time zones is essential.
  • Strong technical expertise in Scala, Python, Apache Spark (batch & streaming), HDFS internals, and migration strategies is required.
  • Proven experience with Apache Iceberg or similar table formats like Delta Lake / Apache Hudi for schema evolution, ACID transactions, and time travel is necessary.
  • Familiarity with running Spark and/or Flink jobs on Kubernetes (e.g., Spark-on-K8s operator, Flink-on-K8s) is beneficial.
  • Experience with distributed blob storages like Ceph or AWS S3 and similar is required.
  • Ability to build ingestion, transformation, and enrichment pipelines for large-scale datasets is essential.
  • Familiarity with Infrastructure-as-Code (Terraform, Helm) for provisioning data infrastructure is necessary.
  • Excellent communication skills and ability to work with cross-functional teams is required.

This role offers a challenging and rewarding opportunity to work on complex big data projects and develop your skills in data engineering.

  • Big Data Engineering Lead

    2 days ago

    Chennai, Tamil Nadu, India Professional Full time ₹ 15,00,000 - ₹ 20,00,000 per year

    The Senior Big Data engineering lead will play a pivotal role in designing, implementing, and optimizing large-scale data processing and analytics solutions. This role requires a visionary leader who can drive innovation, define architecture strategy, and ensure the scalability and efficiency of our big data infrastructure.Key Responsibilities Lead the...

  • Big Data Engineering Leader

    5 days ago

    Chennai, Tamil Nadu, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 25,00,000

    We are seeking a highly skilled professional to lead our data engineering initiatives. As a Big Data Engineer, you will be responsible for designing and developing large-scale data processing systems.Key Responsibilities:Design and implement scalable data pipelines using PySpark and HiveDevelop and maintain big data applications using Spark, HBase, and DQ...

  • Big Data Engineer

    1 week ago

    Chennai, Tamil Nadu, India beBeeBigDataEngineer Full time

    Job Title: Big Data EngineerWe are seeking a highly skilled and experienced Big Data Engineer to join our team. The successful candidate will be responsible for designing, building, and maintaining large-scale data processing systems using Hadoop.Responsibilities:• Design, develop, and deploy big data solutions using Hadoop, Spark, and Scala.•...

  • Big Data Engineering Lead- C13

    3 weeks ago

    Chennai, Tamil Nadu, India Citigroup Full time

    The Senior Big Data engineering lead will play a pivotal role in designing implementing and optimizing large-scale data processing and analytics solutions This role requires a visionary leader who can drive innovation define architecture strategy and ensure the scalability and efficiency of our big data infrastructure Key Responsibilities Lead the...

  • Big Data Engineer

    7 days ago

    Chennai, Tamil Nadu, India Insight Global Full time

    Position: Big Data EngineerDuration: 12+ monthsLocation: India - Chennai or GurgaonRate: 4-17 LPARequired Skills & ExperienceAt least 6+ Years of experience with PySpark and MapRProgramming background with PythonExperience with SQL (Relational Databases) doing data modeling, Oracle experience is a plus.Experience with PowerBI and/or TableauJob...

  • Big Data Engineer

    1 week ago

    Chennai, Tamil Nadu, India Insight Global Full time

    Position: Big Data EngineerDuration: 12+ monthsLocation: India - Chennai or GurgaonRate: 4-17 LPARequired Skills & ExperienceAt least 6+ Years of experience with PySpark and MapRProgramming background with PythonExperience with SQL (Relational Databases) doing data modeling, Oracle experience is a plus.Experience with PowerBI and/or TableauJob...

  • Big Data Technical Lead

    14 hours ago

    Chennai, Tamil Nadu, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Role: Big Data EngineerA dynamic and challenging opportunity has arisen for an experienced Big Data Engineer to join our organization.This role will be responsible for designing, developing, and maintaining large-scale data processing systems using big data technologies such as Hadoop, Spark, and NoSQL databases.The ideal candidate will have a strong...

  • Big Data Engineer

    3 weeks ago

    Chennai, Tamil Nadu, India Amazon Full time

    Profit Intelligence team in Amazon Retail is seeking a seasoned and talented Senior Data Engineer to join the Historical Contribution Profit (HCP) team. HCP is a fast growing team with a mandate to build tools to automate profit-and-loss forecasting and planning for the Physical Consumer business. We are building the next generation data and Business...

  • Big Data Engineer Lead

    5 days ago

    Chennai, Tamil Nadu, India beBeeData Full time ₹ 20,00,000 - ₹ 25,00,000

    Job Title: Senior Data ArchitectThe role involves designing and implementing data architectures that meet the evolving needs of our clients. Designing and implementing data pipelines that can handle large volumes of data.Developing and maintaining a robust data architecture that meets the needs of various business stakeholders.Collaborating with...

  • Big Data Engineer

    4 weeks ago

    Chennai, Tamil Nadu, India Impetus Career Consultants Private Limited Full time

    We are hiring for one the IT big4 consulting.Location : Req : - AWS (Big Data services) S3, Glue, Athena, EMR.- Programming Python, Spark, SQL, Mulesoft,Talend, Dbt.- Data warehouse ETL, Redshift / Responsibilities : - Work with business stakeholders to understand their business needs.- Create data pipelines that extract, transform, and load (ETL) from...