Leading Senior Data Engineer

17 hours ago

Madurai, Tamil Nadu, India beBeeDataEngineering Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

Job Title: Senior Big Data Engineer

About the Role:

We are seeking a seasoned Senior Big Data Engineer to lead our data engineering teams and drive innovation in data-driven solutions.

Key Responsibilities:
  • Lead distributed teams across multiple geographies and time zones.
  • Collaborate with architects, product managers, and operations to deliver high-quality results.
  • Develop and maintain large-scale data processing systems using Scala and Python.
  • Design and implement Apache Spark (batch & streaming) pipelines for real-time data processing.
  • Migrate HDFS clusters to modern storage solutions.
  • Experience with Apache Iceberg (or similar table formats like Delta Lake / Apache Hudi) for schema evolution, ACID transactions, and time travel.
  • Implement and manage Spark and/or Flink jobs on Kubernetes.
  • Design and develop scalable ingestion, transformation, and enrichment pipelines for large-scale datasets.
  • Provision data infrastructure using Infrastructure-as-Code (Terraform, Helm).
  • Communicate technical vision to stakeholders and cross-functional teams.
Requirements:
  • 7-12 years of experience in big data engineering.
  • Proven track record of leading data engineering teams and driving innovation.
  • Expertise in Scala, Python, Apache Spark, HDFS, and Apache Iceberg.
  • Experience with distributed blob storages like Ceph or AWS S3.
  • Strong communication and collaboration skills.
Benefits:

We offer a dynamic work environment, opportunities for growth and development, and a competitive compensation package. If you are passionate about big data engineering and want to make a meaningful impact, we encourage you to apply.

About Us:

We are a leading provider of technology consulting, platform and product engineering, and advanced analytics services. Our team of experts helps enterprises solve their most pressing technical challenges and achieve positive business outcomes. With 7+ years of experience in enterprise AI, we have developed profound expertise and ongoing investment in data, analytics, cloud & DevOps, application modernization, and customer experience. We are headquartered in Silicon Valley with offices across the Americas, Europe, and India.

  • Leading Data Engineer

    7 days ago

    Madurai, Tamil Nadu, India beBeeDataEngineering Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    About the RoleWe are seeking a seasoned Data Engineering professional to join our team. As a Senior Data Engineer, you will play a key role in designing, developing, and maintaining large-scale data processing systems that drive business intelligence and analytics.Your primary focus will be on architecting and implementing scalable and robust data pipelines...

  • Lead Data Architect

    1 week ago

    Madurai, Tamil Nadu, India beBeeDataEngineer Full time ₹ 2,00,00,000 - ₹ 2,50,00,000

    Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer to lead our team of data engineers in designing and implementing scalable big data architectures.The ideal candidate will have expertise in cloud infrastructure, data pipelines, and streaming data processing.Lead a team of data engineers to design and implement big data...

  • Senior Data Engineering Project Lead

    12 hours ago

    Madurai, Tamil Nadu, India beBeeDataEngineering Full time ₹ 20,00,000 - ₹ 25,00,000

    Scrum Master – Data Engineering Job DescriptionWe are seeking an experienced Scrum Master with a strong background in Data Engineering to lead agile teams and ensure smooth project execution.Key Responsibilities:Lead agile teams and facilitate daily stand-ups, sprint planning, and retrospectives.Collaborate with cross-functional teams to identify and...

  • Lead Data Pipeline Tester

    1 day ago

    Madurai, Tamil Nadu, India beBeeTesting Full time ₹ 10,00,000 - ₹ 15,00,000

    Senior Test Lead JobWe are seeking an experienced Senior Test Engineer to drive the testing strategy and execution for a project.The ideal candidate will have expertise in testing data pipelines using Python, SQL, etc. along with experience in JIRA test management.Drive testing strategy and execution for data pipeline projectsTest data pipelines using...

  • Senior Data Engineering Specialist

    7 days ago

    Madurai, Tamil Nadu, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,00,00,000

    Job Title: Senior Data EngineerWe are seeking an experienced Senior Data Engineer with expertise in Python, Pyspark, and SQL to join our team. This role requires a deep understanding of big data concepts, hands-on coding experience, and the ability to implement streaming analytics and data processing.Key Requirements:5+ years of experience as a Senior Data...

  • Cloud Data Engineering Lead

    4 days ago

    Madurai, Tamil Nadu, India beBeeEngineering Full time ₹ 15,00,000 - ₹ 17,50,000

    Job Title: Cloud Data Engineering Lead\The role involves overseeing the design, development and configuration of cloud-based data systems to drive business growth.\This position requires a technical leader with expertise in cloud data warehousing, data modeling and ETL processes. The ideal candidate will have at least 5 years of experience in designing and...

  • Lead Data Architect

    11 hours ago

    Madurai, Tamil Nadu, India beBeeDataEngineer Full time ₹ 21,84,000 - ₹ 27,36,000

    Senior Data ScientistThe primary function of a Senior Data Engineer is to oversee the design, development and maintenance of large-scale data processing systems. A skilled professional in this field has responsibility for creating architectures that meet business requirements.

  • Senior Databricks Data Engineer

    6 days ago

    Madurai, Tamil Nadu, India CodeVyasa Full time

    Looking for Senior Databricks Data Engineer | Pan India to join a team of rockstar developers. The candidate should have a minimum of 10+ yrs. of experience.About CodeVyasa: CodeVyasa is a mid-sized product engineering company that works with top-tier product/solutions companies such as McKinsey, Walmart, RazorPay, Swiggy, and others. We are about 550+...

  • Senior Software Engineering Lead

    4 days ago

    Madurai, Tamil Nadu, India beBeeSoftware Full time ₹ 20,04,000 - ₹ 24,30,000

    Job Title: Senior Engineering LeadDrive engineering excellence across the organization by fostering a culture of best practices, automation, reliability, and performance.Key Responsibilities:Design and Develop Tools: Create efficient software development lifecycle (SDLC) tools and frameworks to improve team productivity.Collaborate with Teams: Work with...

  • Senior Data Engineer

    17 hours ago

    Madurai, Tamil Nadu, India beBeeDataEngineer Full time ₹ 1,50,00,000 - ₹ 2,50,00,000

    **Job Description:**Our organization is seeking a highly skilled AWS data engineer with expertise in Python development and cloud-based data solutions. The ideal candidate will have a solid background in data migration, API ecosystem development, and modern data warehousing practices.The successful candidate will be responsible for designing, building, and...