Senior Data Scientist

Description :Responsibilities :- Architect and develop scalable AI applications focused on indexing, retrieval systems, and distributed data processing.- Collaborate closely with framework engineering, data science, and full-stack teams to deliver an integrated developer experience for building next-generation context-aware applications (i.e., Retrieval-Augmented Generation (RAG)).- Design, build, and maintain scalable infrastructure for high-performance indexing, search engines, and vector databases (e.g., Pinecone, Weaviate, FAISS).- Implement and optimize large-scale ETL pipelines, ensuring efficient data ingestion, transformation, and indexing workflows.- Lead the development of end-to-end indexing pipelines, from data ingestion to API delivery, supporting millions of data points.- Deploy and manage containerized services (Docker, Kubernetes) on cloud platforms (AWS, Azure, GCP) via infrastructure-as-code (e.g., Terraform, Pulumi).- Collaborate on building and enhancing user-facing APIs that provide developers with advanced data retrieval capabilities.- Focus on creating high-performance systems that scale effortlessly, ensuring optimal performance in production environments with massive datasets.- Stay updated on the latest advancements in LLMs, indexing techniques, and cloud technologies to integrate them into cutting-edge applications.- Drive ML and AI best practices across the organization to ensure scalable, maintainable, and secure AI :Educational Background :- Bachelor's or Masters degree in Computer Science, Data Science, Artificial Intelligence, Machine Learning, or a related field.- PhD preferred.- Certifications in Cloud Computing (AWS, Azure, GCP) and ML technologies are a plus.Technical Skills :- Expertise in Python and related frameworks (Pydantic, FastAPI, Poetry, etc.) for building scalable AI/ML solutions.- Proven experience with indexing technologies : Building, managing, and optimizing vector databases (Pinecone, FAISS, Weaviate) and search engines (Elasticsearch, OpenSearch).- Machine Learning/AI Development : Hands-on experience with ML frameworks (e.g., PyTorch, TensorFlow) and fine-tuning LLMs for retrieval-based tasks.- Cloud Services & Infrastructure : Deep expertise in architecting and deploying scalable, containerized AI/ML services on cloud platforms using Docker, Kubernetes, and infrastructure-as-code tools like Terraform or Pulumi.- Data Engineering : Strong understanding of ETL pipelines, distributed data processing (e.g., Apache Spark, Dask), and data orchestration frameworks (e.g., Apache Airflow, Prefect).- APIs Development : Skilled in designing and building RESTful APIs with a focus on user-facing services and seamless integration for developers.- Full Stack Engineering : Knowledge of front-end/back-end interactions and how AI models interact with user interfaces.- DevOps & MLOps : Experience with CI/CD pipelines, version control (Git), model monitoring, and logging in production environments. Experience with LLMOps tools (Langsmith, MLflow) is a plus.- Data Storage : Experience with SQL and NoSQL databases, distributed storage systems, and cloud-native data storage solutions (S3, Google Cloud Storage).Soft Skills :- Ability to think strategically about product design and scale, balancing speed and long-term infrastructure needs.- Strong problem-solving skills, creativity, and ability to handle complex, ambiguous problems in fast-paced environments.- Excellent communication and collaboration skills, with the ability to work crossfunctionally with data scientists, ML engineers, and developers.- Passionate about LLMs, AI applications, and their potential for transforming developer tools and end-user experiences (ref:hirist.tech)