AI Infrastructure Architect

As a seasoned professional in AI infrastructure, you will play a pivotal role in leading the end-to-end design, implementation, and scaling of our artificial intelligence infrastructure. This includes partnering with researchers, product teams, and DevOps to transform prototypes into production services that meet strict service-level agreements (SLAs) for latency, reliability, and cost efficiency.

The ideal candidate will possess strong expertise in cloud platforms (AWS/GCP/Azure), Kubernetes, Docker, Terraform, Helm, Kubeflow, and MLflow. Additionally, experience with inference frameworks (Triton, TensorFlow Serving, BentoML, TorchServe) and familiarity with distributed training, workload schedulers, and GPU-cluster orchestration are highly desirable. Proficiency in Python, TypeScript, and infrastructure-as-code (Terraform, Helm, etc.) is essential.

About the Role

• Design and implement scalable ML pipelines (training, evaluation, deployment) for LLMs, CV, and multimodal models.
• Lead efforts in model serving, versioning, automated CI/CD, and real-time monitoring of AI workflows.
• Build and optimize GPU-backed serving infrastructure targeting p99 latency < 100 ms, 99.9% uptime, and > 80% GPU utilization.
• Drive initiatives on model governance, automated drift detection (≤10% false positives), and data-management best practices.
• Integrate vector databases (Qdrant, Pinecone) for low-latency semantic retrieval, and build agentic workflows using LangChain or similar frameworks.
• Architect RBAC-driven, isolated ML services to securely serve 100–500+ organizations.
• Design Prometheus/Grafana dashboards, ELK/Fluentd logging pipelines, and alerting for all ML workloads.
• Maintain CI/CD pipelines for Python (FastAPI) and TypeScript (NestJS) inference services.
• Define and track SLAs/SLOs, optimize cloud spend by ≥ 20% year-over-year, and ensure GPU clusters operate at > 80% utilization.
• Partner with AI researchers, product managers, and legal to align MLOps standards with compliance and roadmap goals.
• Mentor junior engineers, run quarterly brown-bags, own onboarding docs (upskill 5+ engineers/quarter), and publish ≥ 1 open-source contribution or talk annually.

Must-Haves:

• 9–14 years in software engineering, including ≥ 4 years in MLOps or ML infrastructure.
• Strong expertise in cloud platforms (AWS/GCP/Azure), Kubernetes, Docker, Terraform, Helm, Kubeflow, and MLflow.
• Experience with inference frameworks (Triton, TensorFlow Serving, BentoML, TorchServe).
• Familiarity with distributed training, workload schedulers, and GPU-cluster orchestration.
• Proficiency in Python, TypeScript, and infrastructure-as-code (Terraform, Helm, etc.).
• A proven track record building reliable, scalable ML systems in production.

Critical Skills:

• Vector DB integration (Qdrant, Pinecone).
• Agent orchestration (LangChain, LlamaIndex).
• Multi-tenant security and RBAC.
• Observability stacks (Prometheus/Grafana, ELK).
• CI/CD for FastAPI/NestJS services.

Nice-to-Haves:

• Master's/PhD in CS/AI and certifications such as AWS ML Specialty, Google Cloud Professional ML Engineer, or CNCF CKA/CKAD.
• Prior experience at AI-focused startups or enterprises scaling ML for 100–500 orgs.
• Understanding of low-latency streaming inference or agent-based LLM systems.
• Excellent written and verbal communication, and a proven ability to drive consensus across functions.