Senior Data Scientist

Description :

Responsibilities :

- Architect and develop scalable AI applications focused on indexing, retrieval systems, and distributed data processing.

- Collaborate closely with framework engineering, data science, and full-stack teams to deliver an integrated developer experience for building next-generation context-aware applications (i.e., Retrieval-Augmented Generation (RAG)).

- Design, build, and maintain scalable infrastructure for high-performance indexing, search engines, and vector databases (e.g., Pinecone, Weaviate, FAISS).

- Implement and optimize large-scale ETL pipelines, ensuring efficient data ingestion, transformation, and indexing workflows.

- Lead the development of end-to-end indexing pipelines, from data ingestion to API delivery, supporting millions of data points.

- Deploy and manage containerized services (Docker, Kubernetes) on cloud platforms (AWS, Azure, GCP) via infrastructure-as-code (e.g., Terraform, Pulumi).

- Collaborate on building and enhancing user-facing APIs that provide developers with advanced data retrieval capabilities.

- Focus on creating high-performance systems that scale effortlessly, ensuring optimal performance in production environments with massive datasets.

- Stay updated on the latest advancements in LLMs, indexing techniques, and cloud technologies to integrate them into cutting-edge applications.

- Drive ML and AI best practices across the organization to ensure scalable, maintainable, and secure AI infrastructure.

Qualifications :

Educational Background :

- Bachelor's or Masters degree in Computer Science, Data Science, Artificial Intelligence, Machine Learning, or a related field.

- PhD preferred.

- Certifications in Cloud Computing (AWS, Azure, GCP) and ML technologies are a plus.

Technical Skills :

- Expertise in Python and related frameworks (Pydantic, FastAPI, Poetry, etc.) for building scalable AI/ML solutions.

- Proven experience with indexing technologies : Building, managing, and optimizing vector databases (Pinecone, FAISS, Weaviate) and search engines (Elasticsearch, OpenSearch).

- Machine Learning/AI Development : Hands-on experience with ML frameworks (e.g., PyTorch, TensorFlow) and fine-tuning LLMs for retrieval-based tasks.

- Cloud Services & Infrastructure : Deep expertise in architecting and deploying scalable, containerized AI/ML services on cloud platforms using Docker, Kubernetes, and infrastructure-as-code tools like Terraform or Pulumi.

- Data Engineering : Strong understanding of ETL pipelines, distributed data processing (e.g., Apache Spark, Dask), and data orchestration frameworks (e.g., Apache Airflow, Prefect).

- APIs Development : Skilled in designing and building RESTful APIs with a focus on user-facing services and seamless integration for developers.

- Full Stack Engineering : Knowledge of front-end/back-end interactions and how AI models interact with user interfaces.

- DevOps & MLOps : Experience with CI/CD pipelines, version control (Git), model monitoring, and logging in production environments. Experience with LLMOps tools (Langsmith, MLflow) is a plus.

- Data Storage : Experience with SQL and NoSQL databases, distributed storage systems, and cloud-native data storage solutions (S3, Google Cloud Storage).

Soft Skills :

- Ability to think strategically about product design and scale, balancing speed and long-term infrastructure needs.

- Strong problem-solving skills, creativity, and ability to handle complex, ambiguous problems in fast-paced environments.

- Excellent communication and collaboration skills, with the ability to work crossfunctionally with data scientists, ML engineers, and developers.

- Passionate about LLMs, AI applications, and their potential for transforming developer tools and end-user experiences