Lead Data Engineer

Lead Data Engineer

Experience - 10+ Years

Location - Bengaluru

Hybrid Model

We are seeking a Lead Data Engineer to serve as the technical lead for the team responsible for managing and engineering customer data.

The ideal candidate will have a strong background in data engineering and software development, with hands-on experience building scalable systems that collect, process, and activate customer data. In this role, you will lead a team of skilled engineers, promote engineering best practices, and drive the adoption of scalable solutions using both SaaS platforms and custom-built capabilities.

Working in an agile (Scrum) environment, you will collaborate closely with product owners, marketing stakeholders, data architects, and cross-functional teams to plan and execute key initiatives. You’ll ensure the team has clear direction, adequate resources, and strong technical guidance to deliver high-quality, high-impact solutions. In addition, you’ll play a key role in shaping system architecture, enforcing software quality standards, and fostering a culture of innovation and technical excellence.

Primary Responsibilities:

• Lead agile data engineering team focused on customer data management and big data solutions that power marketing analytics, personalization, and engagement strategies across Catalyst Brands’ digital ecosystem. Ensure initiatives are delivered on time, with high data integrity, performance, and scalability.
• Collaborate with cross-functional partners including architecture, product management, DevOps, DataOps, marketing analytics, and business operations to define and execute a robust data product backlog and technical roadmap.
• Ensure scalability, performance, and maintainability of big data pipelines, data lakes, and customer data platforms (CDPs) to support advanced marketing use cases such as segmentation and real-time personalization.
• Serve as a technical lead on the design and delivery of complex data engineering projects involving customer data integration, identity resolution, and data synchronization across internal and third-party platforms.
• Oversee development and data engineering practices, providing guidance through design/code reviews, architectural decisions, and best practices for building secure and compliant customer data solutions.
• Mentor and guide in a dynamic, data-driven environment, promoting a culture of excellence, collaboration, and customer-centric data solutions.

Core Competencies & Accomplishments:

• 10+ years’ experience, with 3+ years in a technical leadership role.
• Lead the design and implementation of scalable data pipelines for ingesting, transforming, and activating customer data.
• Develop and orchestrate workflows using Apache Airflow and Spark.
• Leverage AWS services (e.g., S3, Glue, Athena, Redshift, EMR) to manage and optimize large-scale data infrastructure.
• Write robust, modular, and testable code in Python to support data engineering use cases.
• Experience developing or supporting AI-powered solutions is a strong plus.
• Expert-level skills in SQL for data modeling, transformation, and analysis.
• Hands-on experience with Snowflake, Redshift, and traditional data warehousing principles (star/snowflake schema, performance tuning, etc.)
• Familiarity with real-time data streaming (Kafka) and event-based architectures.
• Implement and manage CI/CD pipelines for data applications and infrastructure.
• Exposure to CDPs (Customer Data Platforms) or Marketing Tech Platforms is a plus
• Strong sense of urgency and ability to drive change.
• Experience in agile methodologies (Scrum/Kanban/SAFE) and project management discipline.
• Strong interpersonal and communication (written and verbal) skills.
• Bachelor’s degree in computer science or related field.
• Experience in Retail is a plus.