Head of Data Engineering

Job Title: Head of Data Engineering

Department: Data Engineering

Reporting To: Chief Technology Officer (CTO)

Role Overview

Spiro is seeking an exceptional Head of Data Engineering to lead, design, and scale our next-generation data infrastructure. You will oversee the development of robust, scalable data pipelines, architect our big data platforms, and ensure the integrity and quality of the data fueling our AI, analytics, and digital products.

As a hands-on leader, you will guide a team of talented engineers while collaborating closely with analytics, software development, and product teams to drive business outcomes through data.

This is a critical leadership role shaping the future of data across EV manufacturing, IoT telematics, financial systems, and geospatial platforms.

Key Responsibilities

Strategic Leadership

Define and execute the data engineering roadmap aligned with Spiro's digital and business strategy.

Build and scale a high-performing data engineering team across Pune and other hubs.

Champion a data-driven culture, emphasizing data quality, security, governance, and innovation.

Technical Delivery

Design, develop, and optimize large-scale, high-throughput data pipelines using PySpark, Spark, and Big Data technologies.

Architect scalable data solutions for structured, semi-structured, and unstructured data.

Implement best-in-class ETL, data transformation, and data integration workflows.

Build and maintain data lake and warehouse solutions, ensuring efficient storage, access, and compute.

Data Quality & Optimization

Define and enforce data quality standards, implementing validation frameworks across ingestion and transformation layers.

Optimize the performance, reliability, and cost-efficiency of all data pipelines and storage systems.

Cross-Functional Collaboration

Work closely with business stakeholders, software engineers, data scientists, and product teams to translate business requirements into robust data solutions.

Collaborate with cybersecurity and compliance teams to ensure data security, privacy, and regulatory adherence (GDPR, DPDP, etc.).

Mentorship & Team Development

Lead, mentor, and coach a team of data engineers, fostering continuous learning and technical excellence.

Establish best practices in coding standards, architecture reviews, testing, and DevOps for data engineering.

Required Skills and Experience

Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

8+ years of progressive experience in Data Engineering, including 2+ years in a leadership or managerial role.

Strong hands-on expertise in PySpark, Apache Spark, and Big Data ecosystems (Hadoop, Hive, HDFS).

Proficient in Python (preferred), Scala, or Java for data engineering workflows.

Experience with cloud platforms (AWS, Azure, or Google Cloud) and cloud-native data services.

Strong expertise in data modeling, ETL/ELT pipeline development, and real-time data processing architectures.

Familiarity with data orchestration tools such as Apache Airflow, Prefect, or equivalent.

Deep understanding of building scalable and efficient Data Lakes and Data Warehouses.

Strong focus on data quality frameworks and operational excellence.

Excellent stakeholder management, team leadership, and communication skills.

Preferred Qualifications:

Experience with containerization and orchestration tools like Docker and Kubernetes.

Knowledge of SQL optimization and relational database management.

Familiarity with event-driven architectures and streaming platforms (e.g., Kafka).

Exposure to AI/ML pipeline deployment and support.

Behavioral Competencies

Strategic thinker with hands-on execution ability.

Highly organized, detail-oriented, and quality-driven.

Passionate about leading innovation in data platforms and digital ecosystems.

Strong mentor with a collaborative leadership style.