Team Lead-Advanced Data Systems Delivery

Team Lead – Advanced Data Systems Delivery

Location: Pune

Employment Type: Full-Time

Department: Delivery

Position Overview :

The Team Lead for Advanced Data Systems Delivery will lead a team responsible for 

delivering high-performance, scalable data solutions, with a focus on designing and 

optimizing data architectures for mission-critical Tier 1 systems. This role requires deep

expertise in deploying real-time and batch processing systems, utilizing the ArcOne AI 

platform to architect scalable topologies, tune critical components, and ensure 

compliance with NFRs and SLAs, while maintaining a strong foundation in data 

architecture principles.

Key Responsibilities:

Project Leadership & Delivery: 

• Lead the end-to-end delivery of advanced data systems for mission-critical Tier 1 

applications, focusing on revenue management and other high-impact use cases. 

• Ensure solutions meet NFRs (e.g., scalability, performance, availability, security) 

and SLAs, delivering on time, within scope, and budget. 

• Utilize the ArcOne AI platform to design, scale, and tune system topologies to 

support complex real-time and batch workloads.

Data Architecture Leadership: 

• Architect and oversee the implementation of robust, scalable, and secure data 

architectures to support mission-critical applications. 

• Design data models, schemas, and storage solutions (e.g., data lakes, data 

warehouses, NoSQL databases) optimized for performance, scalability, and 

accessibility. 

• Implement and optimize ETL/ELT pipelines, ensuring efficient data ingestion, 

transformation, and integration across distributed systems. 

• Ensure data governance, integrity, and security standards are maintained across 

all data architectures.

Team Management: 

• Manage, mentor, and inspire a team of data engineers, architects, and 

performance engineers, fostering a culture of technical excellence and 

collaboration. 

• Provide technical guidance on data architecture, advanced data systems, and 

system optimization, ensuring alignment with industry best practices. 

• Conduct performance reviews, set team goals, and support career development 

for team members.

Technical Expertise: 

• Oversee the deployment of high-scale, mission-critical systems handling real-time 

(e.g., streaming, event-driven) and batch processing workloads. 

• Leverage the ArcOne AI platform to design and scale system topologies, tuning 

components (e.g., compute, storage, network) to meet performance and 

scalability requirements. 

• Optimize data processing frameworks (e.g., Apache Kafka, Spark, Hadoop, Flink) 

and databases (e.g., SQL, NoSQL) for low latency, high throughput, and reliability. 

• Lead investigations into performance regressions, scalability issues, or system 

failures, ensuring rapid resolution and root cause analysis.

Stakeholder Collaboration: 

• Act as the primary point of contact for stakeholders, translating business 

requirements into technical solutions with a focus on robust data architectures. 

• Collaborate with infrastructure, platform, AI, and product teams to align on 

system design, performance, and data architecture goals. 

• Communicate project progress, risks, and performance metrics to senior 

leadership and clients.

System Design & Optimization: 

• Design and implement data architectures that support scalability, fault tolerance, 

and high availability for Tier 1 applications. 

• Optimize real-time (e.g., Apache Kafka, Flink) and batch processing (e.g., Spark, 

Hadoop) workloads for high-scale environments. 

• Ensure data pipelines are optimized for performance, leveraging the ArcOne AI 

platform to automate and enhance topology scaling and component tuning. 

• Advocate for simplicity and rigor in system and data architecture design to 

address complex performance and scalability challenges.

Process & Standards: 

• Define and enforce best practices for advanced data systems delivery, data 

architecture, CI/CD pipelines, and performance testing strategies. 

• Develop and monitor SLAs/SLOs for latency, throughput, data availability, and 

system reliability, ensuring compliance with mission-critical standards. 

• Stay updated on industry trends in advanced data systems, data architecture, and 

AI technologies to drive innovation within the team.

Qualifications:

Education: Bachelor's or Master's degree in Computer Science, or a related field. 

Experience: 

• 7+ years of experience in delivering advanced data systems or similar solutions, 

with at least 3 years in a leadership or team lead role. 

• Proven track record of deploying high-scale, mission-critical Tier 1 systems with 

real-time and batch workloads. 

• Extensive experience in designing and implementing data architectures for large-scale, distributed systems, including data modeling, ETL/ELT pipelines, and 

storage solutions (e.g., data lakes, data warehouses). 

• Hands-on expertise with the ArcOne AI platform or similar AI-driven tools to 

design, scale, and tune system topologies. 

• Deep knowledge of performance tuning for data processing frameworks (e.g., 

Apache Kafka, Spark, Hadoop, Flink) and databases (e.g., SQL, NoSQL, data lakes). 

• Strong understanding of NFRs (e.g., scalability, availability, performance, security) 

and SLAs for Tier 1 systems. 

• Experience in revenue management or similar domains (e.g., pricing, forecasting, 

optimization) is a plus.

Technical Skills: 

• Proficiency in programming languages such as Python, Java, or Scala. 

• Expertise in data processing technologies (e.g., Apache Kafka, Spark, Hadoop, 

Flink) and cloud platforms (e.g., AWS, Azure, GCP). 

• Strong experience with data architecture tools and platforms, including data 

modeling (e.g., ERD, dimensional modeling), data lakes (e.g., Delta Lake, Iceberg), 

and data warehouses (e.g., Databricks, Dremio, DuckDB, Trino). 

• Proficiency with performance profiling tools (e.g., perf, eBPF) and observability 

systems (e.g., Prometheus, Grafana, OpenTelemetry). 

• Deep understanding of OS internals, networking, storage, and compute 

optimization (e.g., GPU utilization, memory management). 

• Experience with containerization (e.g., Docker, Kubernetes) and data 

orchestration tools (e.g., Airflow, Dagster). 

• Knowledge of AI/ML frameworks (e.g., TensorFlow, PyTorch) for optimizing data-driven solutions is a plus.

Soft Skills : 

• Exceptional leadership, communication, and stakeholder management skills. 

• Strong problem-solving abilities with a focus on simplicity, rigor, and 

collaboration. 

• Ability to navigate ambiguity and drive alignment across cross-functional teams.

Preferred: 

• Experience with the ArcOne AI platform for system design and optimization. 

• Familiarity with agile methodologies and tools like Jira or Trello. 

• Certification in cloud platforms (e.g., AWS Certified Solutions Architect), data 

engineering, or data architecture (e.g., TOGAF, DAMA-DMBOK).