Senior Infrastructure Engineer

We are seeking an experienced Senior Backend Engineer to design, develop, and maintain the infrastructure powering our generative AI applications. You will work closely with AI engineers, platform teams, and product stakeholders to build scalable, reliable backend systems that support AI model deployment, inference, and integration. This role combines traditional backend engineering expertise with cutting-edge AI infrastructure challenges to deliver robust solutions at enterprise scale.

Key Responsibilities:

• Design and implement scalable backend services and APIs for generative AI applications using microservices architecture and cloud-native patterns.
• Build and maintain model serving infrastructure with load balancing, auto-scaling, caching, and failover capabilities for high-availability AI services.
• Deploy and orchestrate containerized AI workloads using Docker, Kubernetes, ECS, and OpenShift across development, staging, and production environments.
• Develop serverless AI functions using AWS Lambda, ECS Fargate, and other cloud services for scalable, cost-effective inference.
• Implement robust CI/CD pipelines for automated deployment of AI services, including model versioning and gradual rollout strategies.
• Create comprehensive monitoring, logging, and alerting systems for AI service performance, reliability, and cost optimization.
• Integrate with various LLM APIs (OpenAI, Anthropic, Google) and open-source models, implementing efficient batching and optimization techniques.
• Build data pipelines for training data preparation, model fine-tuning workflows, and real-time streaming capabilities.
• Ensure adherence to security best practices, including authentication, authorization, API rate limiting, and data encryption.
• Collaborate with AI researchers and product teams to translate AI capabilities into production-ready backend services.

Required Technical Skills:

• Strong experience with backend development using Python, with familiarity in Go, , or Java for building scalable web services and APIs.
• Hands-on experience with containerization using Docker and orchestration platforms including Kubernetes, OpenShift, and AWS ECS in production environments.
• Proficient with cloud infrastructure, particularly AWS services (Lambda, ECS, EKS, S3, RDS, ElastiCache) and serverless architectures.
• Experience with CI/CD pipelines using Jenkins, GitLab CI, GitHub Actions, or similar tools, including Infrastructure as Code with Terraform or CloudFormation.
• Strong knowledge of databases including PostgreSQL, MongoDB, Redis, and experience with vector databases for AI applications.
• Familiarity with message queues (RabbitMQ, Apache Kafka, AWS SQS/SNS) and event-driven architectures.
• Experience with monitoring and observability tools such as Prometheus, Grafana, DataDog, or equivalent platforms.
• Knowledge of AI/ML model serving frameworks like MLflow, Kubeflow, TensorFlow Serving, or Triton Inference Server.
• Understanding of API design principles, load balancing, caching strategies, and performance optimization techniques.
• Experience with microservices architecture, distributed systems, and handling high-traffic, low-latency applications.

Qualifications:

• Bachelor's degree in computer science, Engineering, or related technical field, or equivalent practical experience.
• 4–6 years of experience in backend engineering with focus on scalable, production systems.
• 2+ years of hands-on experience with containerization, Kubernetes, and cloud infrastructure in production environments.
• Demonstrated experience with AI/ML model deployment and serving in production systems.

-

Job Family Group:

Technology

-

Job Family:

Infrastructure

-

Time Type:

Full time

-

Most Relevant Skills

Please see the requirements listed above.

-

Other Relevant Skills

For complementary skills, please see above and/or contact the recruiter.

-

Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.

If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.

View Citi's EEO Policy Statement and the Know Your Rights poster.