Machine Learning Engineer II

Company Overview

CommerceIQ's AI-powered digital commerce platform is revolutionizing the way brands sell online. Our unified ecommerce management solutions empower brands to make smarter, faster decisions through insights that optimize the digital shelf, increase retail media ROI and fuel incremental sales across the world's largest marketplaces. With a global network of more than 900 retailers, our end-to-end platform helps 2,200+ of the world's leading brands streamline marketing, supply chain, and sales operations to profitably grow market share in more than 50 countries. 10 out of the top 12 CPG brands work with us, including Coca-Cola, Nestle, Colgate-Palmolive, and Mondelez. We've raised over $200M from some of the top investors including SoftBank, Insight Partners, and Madrona. Learn more at .

We are seeking a highly skilled and experienced Senior Generative AI Engineer to join our innovative team, with a paramount focus on developing and rigorously evaluating sophisticated multi-agent AI systems. This role is crucial for designing, building, deploying, and ensuring the accuracy and reliability of cutting-edge generative AI solutions that leverage collaborative AI agents.

The ideal candidate will possess a deep understanding of generative models, combined with robust MLOps practices, strong back-end engineering skills in microservices architectures on cloud platforms like AWS or GCP, and an absolute mastery of Python, Langgraph, and Langchain. Proven experience with evaluation methodologies, including working with evaluation datasets and measuring the accuracy of multi-agent systems using tools like Langsmith or other open-source alternatives, is a must-have.

Key Responsibilities:

Generative AI Development & Multi-Agent Systems:

• Design, develop, and implement advanced generative AI models (LLMs) for various applications, from ideation to production.
• Architect, build, and deploy intelligent multi-agent AI systems, enabling collaborative behaviors and complex decision-making workflows.
• Utilize and extend frameworks like Langchain and Langgraph extensively for building sophisticated, multi-step AI applications, intelligent agents, and agentic workflows, with a strong focus on their evaluability.
• Fine-tune and adapt pre-trained generative models to specific business needs and datasets, often as components within agentic systems.
• Develop strategies for prompt engineering and RAG (Retrieval Augmented Generation) to enhance model performance and control, particularly in multi-agent contexts.
• Research and stay abreast of the latest advancements in generative AI, natural language processing, multi-agent systems, and autonomous AI.

Multi-Agent System Evaluation & Accuracy:

• Design, develop, and execute comprehensive evaluation strategies for multi-agent systems, defining key performance indicators (KPIs) and success metrics.
• Create, manage, and utilize high-quality evaluation datasets to rigorously test the accuracy, coherence, consistency, and robustness of multi-agent system outputs.
• Implement and leverage tools like Langsmith or other open-source solutions (e.g., TruLens, Ragas, custom frameworks) to trace agent interactions, analyze trajectories, and measure the accuracy and effectiveness of multi-agent system behavior.
• Perform root cause analysis for evaluation failures and drive iterative improvements to agent design and system performance.
• Develop methods for assessing inter-agent communication efficiency, task allocation accuracy, and collaborative problem-solving success.

MLOps & Deployment:

• Establish and implement robust MLOps pipelines for training, evaluating, deploying, monitoring, and managing generative AI models and multi-agent systems in production environments.
• Ensure model and agent system scalability, reliability, and performance in a production setting.
• Implement version control for models, data, and code.
• Monitor model drift, performance degradation, and data quality, implementing proactive solutions for both individual models and interconnected agents.

Back-end Engineering (Microservices on AWS/GCP):

• Develop and maintain highly scalable and resilient microservices to integrate generative AI models and orchestrate multi-agent systems into larger applications.
• Design and implement APIs for model inference and agent interaction and coordination.
• Deploy and manage microservices on cloud platforms such as AWS or GCP, utilizing services like EC2, S3, Lambda, EKS/ECS, Sagemaker, GCP Compute Engine, GCS, GKE, Vertex AI, etc., with a focus on supporting agentic architectures.
• Implement best practices for security, logging, monitoring, and error handling in microservices, especially concerning inter-agent communication and system resilience

Collaboration & Leadership:

• Collaborate closely with data scientists, machine learning engineers, product managers, and other stakeholders to translate business requirements into technical solutions, with a keen eye on opportunities for multi-agent automation and their measurable impact.
• Mentor junior engineers and contribute to the growth of the team's technical capabilities, particularly in agentic AI development and rigorous evaluation.
• Participate in code reviews, architectural discussions, and technical design sessions, championing best practices for multi-agent system design and testability.

Required Qualifications:

• Bachelor's or Master's degree in Computer Science, Artificial Intelligence, Machine Learning, or a related quantitative field.
• 4+ years of experience in software engineering with at least 2+ years focused on Machine Learning Engineering or Generative AI development.
• Demonstrable prior experience in multi-agent product development, including designing, implementing, and deploying systems with interacting AI agents.
• Mandatory experience in working with evaluation datasets, defining metrics, and assessing the accuracy and performance of multi-agent systems using tools like Langsmith or comparable open-source alternatives.
• Exceptional proficiency in Python and its ecosystem for machine learning (e.g., PyTorch, TensorFlow, Hugging Face Transformers).
• Deep expertise with Langgraph and Langchain for building complex LLM applications, intelligent agents, and orchestrating multi-agent workflows.
• Solid understanding and practical experience with various generative AI models (LLMs)
• Proven experience with MLOps principles and tools (e.g., MLflow, Kubeflow, Data Version Control (DVC), CI/CD for ML), with an emphasis on agent system lifecycle management and continuous evaluation.
• Extensive experience designing, developing, and deploying microservices architectures on either AWS or GCP.
• Proficiency with containerization technologies (Docker) and orchestration (Kubernetes).
• Strong understanding of API design and development (RESTful, gRPC).
• Excellent problem-solving skills, with a focus on building robust, scalable, and maintainable solutions.
• Strong communication and collaboration skills.

Preferred Qualifications:

• Experience with Apache Spark for large-scale data processing
• Experience with specific AWS services (e.g., Sagemaker, Lambda, EKS) or GCP services (e.g., Vertex AI, GKE, Cloud Functions) for deploying and managing agentic systems.
• Familiarity with other distributed computing frameworks.
• Contributions to open-source projects in the AI/ML space, especially those related to multi-agent systems or agent frameworks (e.g., AutoGen, CrewAI).
• Experience with real-time inference for generative models and real-time agent decision-making and evaluation.

We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, disability status or any other category prohibited by applicable law.