Data Engineer II

About ADM: We are one of the world’s largest nutrition companies and a global leader in human and animal nutrition. We unlock the power of nature to provide nourishing quality of life by transforming crops into ingredients and solutions for foods, beverages, supplements, livestock, aquaculture, and pets. About ADM India Hub: At ADM, we have long recognized the strength and potential of India’s talent pool, which is why we have maintained a presence in the country for more than 25 years. Building on this foundation, we have now established ADM India Hub, our first GCC in India. At ADM India Hub, we are hiring for IT and finance roles across diverse technology and business functions. We stand at the intersection of global expertise and local excellence, enabling us to drive innovation and support our larger purpose of unlocking the power of nature to enrich quality of life. Job Title: Data Engineer II Job Summary: We are seeking a highly skilled and motivated Data Engineer II - AI/ML to join our innovative team, with a focus on OpenAI technologies, Azure AI services, and cutting-edge LLM and RAG architectures. The ideal candidate will possess deep technical expertise in large language models, hands-on experience with Azure's AI ecosystem, and a strong background in Python-based machine learning, CI/CD pipelines, and version control workflows. This role is pivotal in designing, developing, and deploying intelligent AI solutions that drive value through automation, natural language processing (NLP), and advanced retrieval mechanisms. Key Responsibilities: Design and implement AI-powered applications using OpenAI APIs (GPT, Whisper, Codex). Build and deploy scalable AI/ML solutions using Azure OpenAI, Azure Storage, Azure Container Apps, Azure Functions, and Azure Cognitive Services. Architect and implement LLM-based pipelines, including fine-tuning, prompt engineering, and RAG (Retrieval-Augmented Generation) using vector databases (e.g., Azure AI Search, Pinecone, FAISS). Build intelligent document processing systems with embeddings, semantic search, and LLM chaining (LangChain, Semantic Kernel). Develop and maintain robust APIs and AI services using Python, FastAPI, or Flask. Implement CI/CD pipelines for model deployment and service delivery using Azure DevOps, GitHub Actions, or similar tools. Utilize Git and version control systems to manage source code, workflows, and model versioning. Collaborate with cross-functional teams to deliver AI solutions in a production environment. Monitor, evaluate, and optimize AI systems for performance, accuracy, and cost-efficiency. Required Skills & Qualifications: 4 years of relevant experience Strong proficiency with OpenAI models and APIs. Deep hands-on experience with the Azure AI ecosystem: Azure OpenAI, Azure Storage, Container Apps, App Services, Azure Cognitive Search, etc. Solid understanding and hands-on implementation of LLMs , including: bapPrompt tuning and prompt chaining RAG with embeddings and vector databases Fine-tuning LLMs on domain-specific data Proficient in Python and ML/NLP libraries: transformers, langchain, sentence-transformers, pandas, etc. Experience building and maintaining CI/CD pipelines using Azure DevOps or GitHub Actions. Comfortable with version control using Git, GitHub, or Azure Repos. Strong foundation in machine learning/ deep learning/ natural language processing. Experience with vector databases (e.g., Pinecone, Weaviate, Qdrant, Azure AI Search). Exposure to MLOps practices and production-level deployment. Excellent problem-solving and analytical skills. Outstanding communication and collaboration abilities. Nice to Have: Experience with multi-modal AI, Generative AI, or RLHF Familiarity with Databricks, MLflow, or Azure Synapse Understanding of Responsible AI, AI governance, and data privacy Knowledge of containerization using Docker and orchestration with Kubernetes