AI Architect

The AI Architect designs, governs, and delivers production-grade AI capabilities, including LLM integration, retrieval-augmented generation (RAG), and agentic workflows, as part of secure, scalable, and maintainable healthcare products. The role is weighted toward AI/GenAI architecture and core system architecture: distributed systems, microservices, APIs, data, cloud infrastructure, security, and operations. The AI Architect translates product goals, customer commitments, and engineering constraints into architecture that works end to end, from user experience and service boundaries through data pipelines, model and retrieval layers, integrations, deployment, and runtime monitoring. This is not a prompt-engineering or agent-building role in isolation. Prompts, models, and agents are components of a larger system, and this role is accountable for everything around them: service decomposition, contracts, data flow, state management, latency and cost budgets, failure handling, security and PHI protection, observability, evaluation, and long-term maintainability. The ideal candidate is a hands-on architect who has shipped GenAI to production and can defend the full system design, not only the model layer. Key Outcomes
· Production GenAI and agentic AI capabilities delivered as reliable, secure, observable product features rather than isolated prototypes or demos.
· A reusable enterprise AI platform (model gateway, retrieval/RAG services, agent orchestration, evaluation, and guardrails) that product teams adopt consistently.
· End-to-end architectures in which AI components are properly integrated with microservices, APIs, event flows, data stores, identity, and cloud infrastructure, with clear service boundaries and contracts.
· AI systems that meet defined targets for quality, latency, availability, cost, and safety, with governance, PHI protection, and Responsible AI practices built in from the start.
· Stronger alignment among product, engineering, QA, DevOps, data, security, and leadership stakeholders around feasible and sustainable AI-enabled solutions. Roles and Responsibilities
· Lead the architecture of GenAI, RAG, and agentic AI solutions, including model selection, prompt and context design, retrieval strategy, tool/function calling, multi-agent orchestration, memory and state management, and human-in-the-loop controls.
· Design and evolve a shared enterprise AI platform: LLM gateway and model abstraction, prompt and configuration management, embedding and vector search services, agent orchestration (e.g., LangGraph, Semantic Kernel), guardrails, and evaluation services.
· Define RAG and knowledge architectures end to end: ingestion, chunking, embedding, indexing, hybrid search, re-ranking, access control on retrieved content, data freshness, and grounding/citation strategy.
· Establish AI evaluation and LLMOps practices, including offline and online evaluation, golden datasets, regression testing of prompts and models, hallucination and quality metrics, A/B testing, and controlled model/version rollout and rollback.
· Drive AI governance, safety, and security: prompt-injection and data-exfiltration defenses, PII/PHI redaction, content filtering, auditability, model risk assessment, and alignment with Responsible AI and healthcare regulatory requirements such as HIPAA.
· Make and document build-vs-buy and model trade-off decisions (managed services such as AWS Bedrock vs. open-weight or self-hosted models) based on quality, latency, cost, data residency, and vendor risk.
· Own end-to-end system design for AI-enabled features: service decomposition, microservice boundaries, API and event contracts, synchronous vs. asynchronous patterns, data ownership, and integration with existing product and legacy systems.
· Design for non-functional requirements across the full stack: scalability, availability, resiliency (timeouts, retries, circuit breakers, fallbacks, and graceful degradation when models fail), performance, latency and token-cost budgets, multi-tenancy, and disaster recovery.
· Define data architecture for AI workloads, including operational and analytical stores, vector databases, caching, streaming and event pipelines, data lineage, retention, and secure handling of clinical and patient data.
· Architect cloud-native deployment on AWS containers and Kubernetes/ECS, serverless, networking, identity and access, secrets management, infrastructure as code, and CI/CD for both application and AI artifacts.
· Define observability across AI and non-AI components together: distributed tracing across services and model calls, prompt/response logging with privacy controls, quality and drift monitoring, cost tracking, SLOs, and alerting.
· Define architecture standards, reference architectures, design patterns, API guidelines, and service templates that product teams can adopt consistently.
· Analyze current-state architecture, technical debt, and integration constraints to identify where AI can be introduced safely and where modernization must come first.
· Partner with product management to translate business priorities and customer commitments into technically feasible AI roadmaps, implementation strategies, and incremental releases with measurable outcomes.
· Build proofs of concept and reference implementations hands-on to validate feasibility, and define the path from proof of concept to production-hardened service.
· Lead architecture reviews and lightweight governance aligned to agile delivery, covering both AI-specific risks and core design quality.
· Mentor senior engineers and engineering managers in GenAI engineering, system design, distributed systems, service ownership, and accountable AI practices.
· Document architecture decisions (ADRs), system and sequence diagrams, service catalogs, AI model and prompt inventories, technical risks, and roadmaps in formats usable by engineers and product stakeholders.
· Use AI-assisted engineering tools such as GitHub Copilot, Claude, and ChatGPT Enterprise to accelerate codebase exploration, design comparison, and documentation while retaining accountability for decisions.
· Track the evolving AI landscape (models, agent frameworks, protocols such as MCP, and evaluation methods) and turn relevant advances into pragmatic architecture guidance. Education
· Bachelor’s degree in Computer Science, Computer Engineering, Information Technology, Software Engineering, or a related discipline is required; equivalent deep technical experience may be considered.
· Master’s degree in Computer Science, Artificial Intelligence/Machine Learning, Data Science, Software Engineering, or a related discipline is preferred.
· Certifications such as AWS Solutions Architect, AWS Machine Learning/GenAI, TOGAF, AI Engineer, or CKA/CKAD are preferred when supported by practical implementation experience. Experience
· Overall Experience of 14+ years in software development and architecture.
· 8+ years of architecture or technical design experience across distributed systems, microservices, APIs, data, and cloud platforms.
· 3+ years architecting and delivering AI/GenAI solutions, including at least one LLM, RAG, or agent-based system running in production with real users.
· Proven experience designing end-to-end architectures for complex, high-scale applications, from service decomposition and data modeling through deployment, observability, and operations.
· Hands-on experience integrating AI components into existing products and legacy systems, including reliability, latency, cost, and security trade-offs.
· Experience with AI evaluation, LLMOps/MLOps, and AI governance in a regulated industry; healthcare/EHR experience is preferred. Skills
· Expert knowledge of GenAI architecture: LLM integration, prompt and context engineering, RAG, embeddings and vector search, agentic and multi-agent patterns, tool/function calling, and frameworks such as LangGraph, Semantic Kernel, LangChain, or LlamaIndex.
· Expert knowledge of system design and distributed systems: microservices, domain-driven design, API design (REST, gRPC, GraphQL), event-driven architecture, messaging and streaming, consistency and state management, caching, and scalability patterns.
· Strong understanding of AI quality and safety: evaluation frameworks, hallucination mitigation, guardrails, prompt-injection defenses, AI observability, and Responsible AI principles.
· Strong understanding of full-stack product engineering, relational, NoSQL, and vector databases, CI/CD, containers and Kubernetes, infrastructure as code, security architecture (identity, encryption, zero trust), and performance engineering.
· Ability to reason about and quantify trade-offs across accuracy, latency, cost, reliability, and maintainability, and to explain them clearly.
· Ability to create architecture decision records, reference architectures, system and sequence diagrams, service catalogs, and reusable engineering patterns.
· Strong communication skills with the ability to align technical and non-technical stakeholders around architecture decisions and trade-offs. Abilities
· Thinks in systems: designs the full request path, data flow, and failure modes around AI components, not only the model or prompt.
· Influences across engineering, product, QA, DevOps, data, security, and leadership teams without relying only on formal authority.
· Balances innovation speed with production readiness, customer commitments, and regulatory obligations.
· Converts ambiguity into clear architecture direction, incremental delivery plans, and executable engineering guidance.
· Mentors teams toward full-cycle ownership, reusable patterns, and accountable AI engineering practices.
· Maintains practical governance that improves quality and speed rather than increasing unnecessary process overhead. Tools and Technologies
· GenAI and LLM platforms: AWS Bedrock, Anthropic Claude, OpenAI models, and open-weight models (e.g., Llama, Mistral); Model Context Protocol (MCP).
· Agent and orchestration frameworks: LangGraph, Semantic Kernel, LangChain, LlamaIndex, and AWS Bedrock Agents.
· Vector and knowledge stores: pgvector, OpenSearch/Elasticsearch, Pinecone, Weaviate; knowledge graphs and graph databases such as Neo4j or Neptune.
· AI evaluation and observability: LangSmith, Langfuse, Ragas, or similar; OpenTelemetry, CloudWatch, and APM tools.
· Java, Python, TypeScript, JavaScript, SQL, REST/gRPC APIs, microservices, Angular/React, Node.js, and legacy technologies relevant to existing products.
· AWS services including Lambda, ECS/EKS, API Gateway, S3, IAM, Cognito, Step Functions, SQS/SNS, EventBridge, CloudWatch, Aurora, DynamoDB, Glue, and SageMaker.
· Kafka/streaming, relational and NoSQL databases, PostgreSQL, MongoDB, DynamoDB, Redis, data lakes, and distributed computing tools where applicable.
· CI/CD and platform tools such as Docker, Kubernetes, Terraform, Jenkins, GitHub Actions, Git/GitHub, NuGet, NPM, and feature management tools.
· AI-assisted engineering tools: GitHub Copilot, ChatGPT Enterprise/Team, Claude, and approved AI assistants; architecture and documentation tools for diagrams, ADRs, and service catalogs. AI Usage and Accountability Requirements
· AI capabilities must be designed as part of the full system, with defined service boundaries, fallbacks, and ownership; prompts and agents are not shipped as standalone artifacts.
· Every production AI feature must have defined evaluation criteria, quality thresholds, monitoring, and a rollback path before release.
· PHI/PII handling, access control on retrieved data, audit logging, and human-in-the-loop controls must be designed in from the start, aligned with HIPAA and organizational AI policy.
· Architecture decisions must remain traceable to business goals, quality attributes, cost, implementation feasibility, and known trade-offs.
· AI-assisted recommendations, including those from coding and architecture assistants, must be validated against product context, engineering constraints, performance needs, maintainability, and operational risk.
· Reusable prompts, agent patterns, evaluation datasets, reference architectures, and design-review checklists should be maintained as organizational assets. Success Measures / Accountable Outcomes
· Number of GenAI and agentic capabilities moved from proof of concept to production, and time from proof of concept to production.
· AI quality and reliability against agreed targets: groundedness/accuracy, hallucination and escalation rates, latency, availability, and cost per transaction.
· Adoption of the shared AI platform and reference architectures across product teams, and reduction in duplicated or one-off AI implementations.
· Reduction in production incidents, integration defects, and technical debt attributable to architecture, including in AI-enabled services.
· No critical AI security or PHI incidents, with audit-ready AI governance documentation.
· Improvement in delivery readiness for strategic AI product initiatives and customer-committed capabilities. AI Competency Alignment Competency Target Proficiency GenAI and LLM Solution Architecture Expert Agentic AI and Multi-Agent System Design Advanced to Expert RAG and Knowledge Architecture Advanced to Expert End-to-End System Design and Distributed Systems Advanced to Expert Microservices, API, and Event-Driven Integration Architecture Advanced Cloud-Native and AI Platform Engineering Advanced AI Evaluation, LLMOps, and Observability Advanced Security, Privacy, and Responsible AI Engineering Advanced Performance, Scalability, and Cost Optimization Advanced Data Architecture for AI Workloads Proficient to Advanced AI Strategy and Product Engineering Collaboration Advanced Authority and Autonomy Leads AI and system architecture decisions for product areas and complex initiatives. Determines AI platform direction, architecture approaches, standards, service models, and model and vendor choices in partnership with engineering, product, security, and compliance leadership. Holds authority to require architecture and AI readiness review before AI capabilities reach production. Work is reviewed through alignment with business goals, technical outcomes, product objectives, and departmental strategy. Disclaimer: This Job Summary indicates the general nature and level of work expected of the incumbent(s). It is not designed to cover or contain a comprehensive listing of activities, duties or responsibilities required of the incumbent. Incumbent(s) may be asked to perform other duties as requested. Greenway Health, LLC is an Equal Opportunity Employer. We do not discriminate on the basis of race, religion, age, gender, national origin, sexual orientation, disability, or veteran status.