Lead Data Engineer

4 days ago

Chennai, India SquareShift Technologies Full time

Lead Data Engineer - Job Description Overview We are seeking a highly skilled Lead Data Engineer to architect, build, and scale enterprise-grade data platforms and pipelines. The ideal candidate has deep expertise in modern data engineering, cloud-native services, real-time data ingestion & CDC frameworks. This role involves leading complex data initiatives, guiding engineering teams, and collaborating with cross-functional stakeholders to deliver robust, scalable, and secure data solutions. Key Responsibilities Data Architecture & Pipeline Development Design and build scalable ETL/ELT pipelines using Python / PySpark. Architect and implement batch and real-time data pipelines using Kafka, Kafka Connect, and Debezium. Develop and optimize data ingestion workflows across multiple databases (MySQL, Postgres, MongoDB). Build and manage Apache Airflow DAGs for end-to-end pipeline orchestration. Cloud Data Engineering Lead data platform development on GCP (BigQuery, Data Proc, Data Stream ). Implement data ingestion, orchestration, and transformation at scale using Pub/Sub, Cloud Functions/Cloud Run, Dataflow, and GKE. Own data warehouse design and optimization across Snowflake and BigQuery . Requirements Required Skills & Experience Core Technical Skills Strong proficiency in Python, SQL, PySpark . Hands-on expertise with Kafka, Kafka Connect, Debezium, Airflow, Databricks . Deep experience with BigQuery, Snowflake, MySQL, Postgres, MongoDB . Solid understanding of vector data stores and search indexing. Knowledge of GCP services like Big Query, Cloud Functions, Cloud Run, Data Flow, Data Proc, Data Stream , etc..  Good to have Certifications: GCP Professional Data Engineer Elastic Certified Engineer AI Gemini Enterprise Vertex AI Agent Builder ADK Non-Technical & Leadership Skills Communication: Exceptional verbal and written communication skills, with the ability to articulate complex technical concepts to both technical and non-technical audiences. Mentorship & Coaching: Proven experience in mentoring junior and mid-level engineers, fostering a culture of continuous learning and growth. Problem-Solving: Strong analytical and debugging skills, with a proactive approach to identifying and resolving technical roadblocks. Ownership & Accountability: Demonstrates a high level of responsibility for project outcomes, system reliability, and code quality. Agile Proficiency: Deep understanding and practical experience with Agile methodologies (Scrum/Kanban). Stakeholder Management: Ability to effectively manage expectations and build consensus across different teams. Qualifications Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field (or equivalent practical experience). Typically 7+ years of progressive experience in data engineering, with 2+ years in a technical leadership or lead engineer role.

  • Lead Data Engineer

    2 days ago

    Chennai, Tamil Nadu, India Eucloid Data Solutions Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Eucloid is looking for a Lead Data Engineer to join our Data Platform team supporting various business applications. The ideal candidate will support the development of data infrastructure for our clients by participating in activities that may include starting from upstream and downstream technology selection to designing and building different components....

  • Data Engineer

    2 weeks ago

    Chennai, India NTT DATA Full time

    Job Description Req ID:     NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer to join our team in Chennai, Tamil Nādu (IN-TN), India (IN). Job Duties: As a data...

  • Data Engineer

    2 days ago

    Chennai, Tamil Nadu, India CX Data Labs Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    HiRole - Azure Data EngineerLocation - Chennai, Gurugram (5days Office)Full timeShift timing 2pm to 11pmKey Responsibilities:· Develop, maintain, and optimize scalable ETL/ELT pipelines using PySpark and Databricks.· Collaborate with cross-functional teams to design and implement data models and data integration solutions.· Create and maintain robust SQL...

  • Data Engineer

    3 days ago

    Chennai, Tamil Nadu, India CX Data Labs Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    HiRole - Azure Data engineerLocation - Chennai, GurugramWork from officeFull timeImmediate joiner Only (15days Notice period only)Candidates with Certification in Databricks is good to haveTechnical Skills:o Proficiency in SQL for complex queries, database design, and optimization.o Strong experience with PySpark for data transformation and processing.o...

  • Engineer (L1) - Titus Data Classification

    2 weeks ago

    Chennai, India NTT DATA Full time

    Job Description Make an impact with NTT DATA Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive. Your day at NTT...

  • Lead Data Engineer

    1 week ago

    Chennai, India Alans Group Full time

    The Role :As a Lead Data Engineer, you will be responsible for designing, building, and maintaining scalable data pipelines and architectures. You will lead a team of data engineers and collaborate with data scientists and analysts to ensure data is accessible and usable across the organization. Design and implement robust data pipelines for data ingestion...

  • Lead Data Engineer

    4 days ago

    Chennai, India SquareShift Technologies Full time

    Job Description Lead Data Engineer - Job Description Overview We are seeking a highly skilled Lead Data Engineer to architect, build, and scale enterprise-grade data platforms and pipelines. The ideal candidate has deep expertise in modern data engineering, cloud-native services, real-time data ingestion & CDC frameworks. This role involves leading complex...

  • Lead Data Engineer

    2 days ago

    Chennai, Tamil Nadu, India SquareShift Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Lead Data Engineer - Job DescriptionOverviewWe are seeking a highly skilled Lead Data Engineer to architect, build, and scale enterprise-grade data platforms and pipelines. The ideal candidate has deep expertise in modern data engineering, cloud-native services, real-time data ingestion & CDC frameworks. This role involves leading complex data initiatives,...

  • Lead Data Engineer

    1 week ago

    Chennai, Tamil Nadu, India Alans Group Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    The Role:As a Lead Data Engineer, you will be responsible for designing, building, and maintaining scalable data pipelines and architectures. You will lead a team of data engineers and collaborate with data scientists and analysts to ensure data is accessible and usable across the organization.Design and implement robust data pipelines for data ingestion and...

  • Lead Data Engineer

    1 week ago

    chennai, India Alans Group Full time

    The Role:As a Lead Data Engineer, you will be responsible for designing, building, and maintaining scalable data pipelines and architectures. You will lead a team of data engineers and collaborate with data scientists and analysts to ensure data is accessible and usable across the organization.Design and implement robust data pipelines for data ingestion and...