ArcOne - Team Lead - Advanced Data Systems

Position Overview
The Team Lead for Advanced Data Systems Delivery will lead a team responsible for delivering high-performance, scalable data solutions, with a focus on designing and optimizing data architectures for mission-critical Tier 1 systems.

This role requires deep expertise in deploying real-time and batch processing systems, utilizing the ArcOne AI platform to architect scalable topologies, tune critical components, and ensure compliance with NFRs and SLAs, while maintaining a strong foundation in data architecture principles.

Key Responsibilities
Project Leadership & Delivery :

• Lead the end-to-end delivery of advanced data systems for mission-critical Tier 1 applications, focusing on revenue management and other high-impact use cases.
• Ensure solutions meet NFRs (e.g., scalability, performance, availability, security) and SLAs, delivering on time, within scope, and budget.
• Utilize the ArcOne AI platform to design, scale, and tune system topologies to support complex real-time and batch workloads.

Data Architecture Leadership

• Architect and oversee the implementation of robust, scalable, and secure data architectures to support mission-critical applications.
• Design data models, schemas, and storage solutions (e.g., data lakes, data warehouses, NoSQL databases) optimized for performance, scalability, and accessibility.
• Implement and optimize ETL/ELT pipelines, ensuring efficient data ingestion, transformation, and integration across distributed systems.
• Ensure data governance, integrity, and security standards are maintained across all data architectures.

Team Management

• Manage, mentor, and inspire a team of data engineers, architects, and performance engineers, fostering a culture of technical excellence and collaboration.
• Provide technical guidance on data architecture, advanced data systems, and system optimization, ensuring alignment with industry best practices.
• Conduct performance reviews, set team goals, and support career development for team members.

Technical Expertise

• Oversee the deployment of high-scale, mission-critical systems handling real-time (e.g., streaming, event-driven) and batch processing workloads.
• Leverage the ArcOne AI platform to design and scale system topologies, tuning components (e.g., compute, storage, network) to meet performance and scalability requirements.
• Optimize data processing frameworks (e.g., Apache Kafka, Spark, Hadoop, Flink) and databases (e.g., SQL, NoSQL) for low latency, high throughput, and reliability.
• Lead investigations into performance regressions, scalability issues, or system failures, ensuring rapid resolution and root cause analysis.

Stakeholder Collaboration

• Act as the primary point of contact for stakeholders, translating business requirements into technical solutions with a focus on robust data architectures.
• Collaborate with infrastructure, platform, AI, and product teams to align on system design, performance, and data architecture goals.
• Communicate project progress, risks, and performance metrics to senior leadership and clients.

System Design & Optimization

• Design and implement data architectures that support scalability, fault tolerance, and high availability for Tier 1 applications.
• Optimize real-time (e.g., Apache Kafka, Flink) and batch processing (e.g., Spark, Hadoop) workloads for high-scale environments.
• Ensure data pipelines are optimized for performance, leveraging the ArcOne AI platform to automate and enhance topology scaling and component tuning.
• Advocate for simplicity and rigor in system and data architecture design to address complex performance and scalability challenges.

Process & Standards

• Define and enforce best practices for advanced data systems delivery, data architecture, CI/CD pipelines, and performance testing strategies.
• Develop and monitor SLAs/SLOs for latency, throughput, data availability, and system reliability, ensuring compliance with mission-critical standards.
• Stay updated on industry trends in advanced data systems, data architecture, and AI technologies to drive innovation within the team.

Qualifications
Education : Bachelors or Masters degree in Computer Science, or a related field.

Experience

• 7+ years of experience in delivering advanced data systems or similar solutions, with at least 3 years in a leadership or team lead role.
• Proven track record of deploying high-scale, mission-critical Tier 1 systems with real-time and batch workloads.
• Extensive experience in designing and implementing data architectures for large-scale, distributed systems, including data modeling, ETL/ELT pipelines, and storage solutions (e.g., data lakes, data warehouses).
• Hands-on expertise with the ArcOne AI platform or similar AI-driven tools to design, scale, and tune system topologies.
• Deep knowledge of performance tuning for data processing frameworks (e.g., Apache Kafka, Spark, Hadoop, Flink) and databases (e.g., SQL, NoSQL, data lakes).
• Strong understanding of NFRs (e.g., scalability, availability, performance, security) and SLAs for Tier 1 systems.
• Experience in revenue management or similar domains (e.g., pricing, forecasting, optimization) is a plus.

Technical Skills

• Proficiency in programming languages such as Python, Java, or Scala.
• Expertise in data processing technologies (e.g., Apache Kafka, Spark, Hadoop, Flink) and cloud platforms (e.g., AWS, Azure, GCP).
• Strong experience with data architecture tools and platforms, including data modeling (e.g., ERD, dimensional modeling), data lakes (e.g., Delta Lake, Iceberg), and data warehouses (e.g., Databricks, Dremio, DuckDB, Trino).
• Proficiency with performance profiling tools (e.g., perf, eBPF) and observability systems (e.g., Prometheus, Grafana, OpenTelemetry).
• Deep understanding of OS internals, networking, storage, and compute optimization (e.g., GPU utilization, memory management).
• Experience with containerization (e.g., Docker, Kubernetes) and data orchestration tools (e.g., Airflow, Dagster).
• Knowledge of AI/ML frameworks (e.g., TensorFlow, PyTorch) for optimizing data-driven solutions is a plus.

Soft Skills

• Exceptional leadership, communication, and stakeholder management skills.
• Strong problem-solving abilities with a focus on simplicity, rigor, and collaboration.
• Ability to navigate ambiguity and drive alignment across cross-functional teams.

Preferred

• Experience with the ArcOne AI platform for system design and optimization.
• Familiarity with agile methodologies and tools like Jira or Trello.
• Certification in cloud platforms (e.g., AWS Certified Solutions Architect), data engineering, or data architecture (e.g., TOGAF, DAMA-DMBOK)

)