Lead Data Engineer

7 hours ago

Chennai, Tamil Nadu, India SquareShift Full time ₹ 12,00,000 - ₹ 36,00,000 per year

Lead Data Engineer - Job Description

Overview

We are seeking a highly skilled Lead Data Engineer to architect, build, and scale enterprise-grade data platforms and pipelines. The ideal candidate has deep expertise in modern data engineering, cloud-native services, real-time data ingestion & CDC frameworks. This role involves leading complex data initiatives, guiding engineering teams, and collaborating with cross-functional stakeholders to deliver robust, scalable, and secure data solutions.

Key Responsibilities

Data Architecture & Pipeline Development

  • Design and build scalable ETL/ELT pipelines using Python / PySpark.
  • Architect and implement batch and real-time data pipelines using Kafka, Kafka Connect, and Debezium.
  • Develop and optimize data ingestion workflows across multiple databases (MySQL, Postgres, MongoDB).
  • Build and manage Apache Airflow DAGs for end-to-end pipeline orchestration.

Cloud Data Engineering

  • Lead data platform development on GCP (BigQuery, Data Proc, Data Stream ).
  • Implement data ingestion, orchestration, and transformation at scale using Pub/Sub, Cloud Functions/Cloud Run, Dataflow, and GKE.
  • Own data warehouse design and optimization across Snowflake and BigQuery.

Requirements

Required Skills & Experience

Core Technical Skills

  • Strong proficiency in Python, SQL, PySpark.
  • Hands-on expertise with Kafka, Kafka Connect, Debezium, Airflow, Databricks.
  • Deep experience with BigQuery, Snowflake, MySQL, Postgres, MongoDB.
  • Solid understanding of vector data stores and search indexing.
  • Knowledge of GCP services like Big Query, Cloud Functions, Cloud Run, Data Flow, Data Proc, Data Stream, etc..

Good to have

  • Certifications:

  • GCP Professional Data Engineer

  • Elastic Certified Engineer

  • AI

  • Gemini Enterprise

  • Vertex AI Agent Builder
  • ADK

Non-Technical & Leadership Skills

  • Communication: Exceptional verbal and written communication skills, with the ability to articulate complex technical concepts to both technical and non-technical audiences.
  • Mentorship & Coaching: Proven experience in mentoring junior and mid-level engineers, fostering a culture of continuous learning and growth.
  • Problem-Solving: Strong analytical and debugging skills, with a proactive approach to identifying and resolving technical roadblocks.
  • Ownership & Accountability: Demonstrates a high level of responsibility for project outcomes, system reliability, and code quality.
  • Agile Proficiency: Deep understanding and practical experience with Agile methodologies (Scrum/Kanban).
  • Stakeholder Management: Ability to effectively manage expectations and build consensus across different teams.

Qualifications

  • Bachelor's or Master's degree in Computer Science, Engineering, or a related field (or equivalent practical experience).

Typically 7+ years of progressive experience in data engineering, with 2+ years in a technical leadership or lead engineer role.

  • Lead Data Engineer

    5 hours ago

    Chennai, Tamil Nadu, India Eucloid Data Solutions Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Eucloid is looking for a Lead Data Engineer to join our Data Platform team supporting various business applications. The ideal candidate will support the development of data infrastructure for our clients by participating in activities that may include starting from upstream and downstream technology selection to designing and building different components....

  • Data Engineer

    11 hours ago

    Chennai, Tamil Nadu, India CX Data Labs Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    HiRole - Azure Data engineerLocation - Chennai, GurugramWork from officeFull timeImmediate joiner Only (15days Notice period only)Candidates with Certification in Databricks is good to haveTechnical Skills:o Proficiency in SQL for complex queries, database design, and optimization.o Strong experience with PySpark for data transformation and processing.o...

  • Data Engineer

    5 hours ago

    Chennai, Tamil Nadu, India CX Data Labs Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    HiRole - Azure Data EngineerLocation - Chennai, Gurugram (5days Office)Full timeShift timing 2pm to 11pmKey Responsibilities:· Develop, maintain, and optimize scalable ETL/ELT pipelines using PySpark and Databricks.· Collaborate with cross-functional teams to design and implement data models and data integration solutions.· Create and maintain robust SQL...

  • Lead Data Engineer

    7 days ago

    Chennai, Tamil Nadu, India Alans Group Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    The Role:As a Lead Data Engineer, you will be responsible for designing, building, and maintaining scalable data pipelines and architectures. You will lead a team of data engineers and collaborate with data scientists and analysts to ensure data is accessible and usable across the organization.Design and implement robust data pipelines for data ingestion and...

  • Lead Data Engineer

    7 hours ago

    Chennai, Tamil Nadu, India SquareShift Technologies Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job InformationDate Opened11/24/2025Job TypeFull timeIndustryTechnologyWork Experience5+CityChennaiState/ProvinceTamil NaduCountryIndiaZip/Postal Code600001Job DescriptionLead Data Engineer - Job DescriptionOverviewWe are seeking a highly skilled Lead Data Engineer to architect, build, and scale enterprise-grade data platforms and pipelines. The ideal...

  • Lead Data Engineer

    6 hours ago

    Chennai, Tamil Nadu, India Augusta HiTech Software Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Industry: ITQualification: Any DegreeRequired Skills: Data Architect, AWS Redshift, Glue, Quicksight, SQL, PythonWorking Shift: 2PM to 11PM ISTCity: Coimbatore / Chennai / BangaloreCountry: IndiaName of the position: Lead Data EngineerLocation: Coimbatore/ Chennai/ BangaloreNo. of resources needed: 01Mode: Contract to HireYears of experience: 10+ YearsShift:...

  • Lead Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India BrandIdea Granular Analytics Platform Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Lead Data Engineer – Python EcosystemChennai, India | HybridWe're growing our technology team atBrandIdea Analytics— a pioneer in spatial and behavioral analytics that helps businesses see their markets in entirely new ways. If you're passionate about building data platforms from the ground up and leading with Python, this is your opportunity to shape...

  • Lead Data Engineer

    1 week ago

    Chennai, Tamil Nadu, India ResourceTree Global Services Pvt Ltd Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Company DescriptionResourceTree Global Services Pvt Ltd. (ResourceTree) is a Talent Builder focused on offering careers in various industries, including Information & Technology, Manufacturing, Services, Banking, Agri-tech, Edu-tech, and others. The company collaborates with industry Leaders to fulfill their talent needs effectively. Over the past 7 years,...

  • Lead Data Engineer

    1 week ago

    Chennai, Tamil Nadu, India Forbes Technologies Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Company DescriptionForbes Advisor is a new initiative for consumers under the Forbes Marketplace umbrella that provides journalist- and expert-written insights, news and reviews on all things personal finance.We are an experienced team of industry experts dedicated to helping readers make smart decisions and choose the right products with ease. Marketplace...

  • Lead Data Engineer

    5 days ago

    Chennai, Tamil Nadu, India Mobile Programming Full time ₹ 12,00,000 - ₹ 18,00,000 per year

    Job DescriptionWe are seeking a Lead Data Engineer with strong expertise in GCP, Bigquery Informatica,Python, and Kubernetes. The role involves leading data integration initiatives, designingscalable ETL pipelines, and ensuring seamless data orchestration across platforms.Key ResponsibilitiesDesign and implement scalable data pipelines using GCP-native tools...