Data Engineer for AI/ML

1 month ago

kanpur, India Dhira Full time

Role: Data Engineer for AI/ML

Experience Required: 6+ years


We’re looking for a true data craftsman who can dexterously navigate the worlds of raw data engineering and refined machine learning pipelines.

You’re not the average “ETL engineer” who relies on pre-packaged tools like Informatica or Talend. You are a solid programmer, fluent in Python, and an artist with Py Spark, weaving together data flows that don’t just function—they perform.


Responsibilities:

• Develop and maintain APIs using Flask and FastAPI frameworks to support AI/ML services, including data access, transformation, and integration with Large Language Models (LLMs).

• Design and implement ETL processes to manage data pipelines and build Data Warehouses (DW) for efficient data storage and retrieval.

• Work on integrating LLMs with API services, ensuring secure and efficient data flow and model interaction.

• Develop APIs abstracting and concealing LLM functionality, providing a seamless interface for applications to interact with AI/ML models.

• Collaborate with database systems to handle data manipulation, storage, and retrieval, supporting API-driven machine learning workflows.

• Optimize Python code for performance and security, ensuring robust and scalable API deployment.

• Participate in cross-functional team discussions to align technical solutions with business objectives.

• Stay abreast of advancements in AI, machine learning, and software development practices to suggest and implement improvements.

• Research and develop new algorithms to improve AI system performance.

• Collaborate with cross-functional teams to integrate AI models and technologies into scalable products.

• Engineer solutions that are functional and optimized for performance, scalability, and maintainability.

• Work on batch processing and real-time data streams using tools like Kafka or Flink.

• Utilize cloud platforms (AWS, GCP, Azure) to manage data lakes and warehouses, optimize data engineering workflows, and support the full lifecycle of machine learning pipelines.

Key Qualifications:

• Strong proficiency in Python with 6 years of experience in developing APIs.

• Expertise in Flask and FastAPI for API development.

• Solid understanding of ETL processes, Data Warehousing, and working with relational databases such as PostgreSQL or MySQL.

• Experience with integrating and managing Large Language Models (LLMs) and concealing their APIs behind custom-built services.

• Knowledge of data transformation and access techniques to effectively feed AI/ML models.

• Familiarity with machine learning development is advantageous but not essential.

• Strong problem-solving skills with a focus on optimizing API performance and ensuring security.

• Ability to work both independently and collaboratively within a team.

• Effective communication skills to explain technical concepts clearly and concisely.

• Experience with distributed data processing using PySpark.

• Familiarity with cloud data services and orchestrators such as Airflow to automate data pipelines and workflows.

• Hands-on experience with big data tools like Spark or Flink to manage large-scale data pipelines.

• Cloud computing: Familiarity with cloud platforms such as AWS, Azure, or GCP.

• Big data technologies: Experience with big data tools and technologies like Spark or similar.

• Natural language processing (NLP): Knowledge of NLP techniques and applications.

• Computer vision: Understanding of computer vision algorithms and applications.

Desired Traits:

• Strong problem-solving skills focusing on optimizing API performance and ensuring security.

• Ability to work both independently and collaboratively within a team.

• Effective communication skills to explain technical concepts clearly and concisely.

• A deep curiosity to explore new data engineering challenges and continuously evolve your skills.

• A commitment to building clean, modular, and maintainable code that delivers high-quality solutions.


Educational Background:

Bachelor's degree in computer science, Information Technology, or a related field.

  • Gen AI Architect

    2 months ago

    Kanpur, India IGT Solutions Full time

    Job Title: Architect - Gen AI, LLM, Big DataExperience: 18+ YearsLocation: Pune/ GurugramEmployment Type: Full-Time / PermanentJob Summary:We are looking for an experienced Architect with deep expertise in Generative AI (Gen AI), Large Language Models (LLM), and Big Data technologies. The ideal candidate will have 18+ years of experience in designing and...

  • Senior Python Developer

    4 months ago

    Kanpur, India Attentions AI Full time

    Position Title: Senior Python DeveloperLocation: Pune, IndiaDepartment: Software DevelopmentEmployment Type: Full-timeCompany Overview:Attentions is a cutting-edge generative AI company dedicated to developing innovative products across all modalities, including text, vision, audio, and more. Our mission is to push the boundaries of AI through ongoing...

  • Flex AI | Senior Android Engineer | uttar pradesh

    4 weeks ago

    Kanpur, India Flex AI Full time

    Job Title: Senior Android EngineerLocation:RemoteAbout Us:Flex AI is a leading platform dedicated to helping people achieve their fitness goals through personalized workout plans, nutrition guidance, and community support. Our mission is to make fitness accessible and enjoyable for everyone. Join our dynamic team and help us create innovative solutions that...

  • Flex AI | Software Engineer

    2 days ago

    Kanpur, India Flex AI Full time

    Job Title: Software Engineer (React Native)Location:RemoteAbout Us:Flex AI is a leading platform dedicated to helping people achieve their fitness goals through personalized workout plans, exercise and nutrition guidance, and community support. Our mission is to make fitness accessible and enjoyable for everyone. Join our dynamic team and help us create...

  • AI Agentic system design Engineer

    3 months ago

    Kanpur, India QpiAI Full time

    About QpiAI:QpiAI is a world leader in Quantum and AI technologies. QpiAI is building the most powerful Quantum computers in India.Job Description:In this job, you will be Ideatic/Architecting/Implementing AI agentic systems and software to solve real-world applications in manufacturing, chemistry, software automation, and Quantum computer design. You will...

  • ThinkJS | Ai Engineer

    3 weeks ago

    Kanpur, India ThinkJS Full time

    AI Engineer JD - Info DumpThis role requires combined expertise in software development, programming, data science and data engineering.Be ready to take on varied responsibilities, lead by example, and contribute independently to key initiatives.A full-stack generalist who has built pieces of something, but wants more ownership to design a system...

  • Machine Learning Engineer

    4 weeks ago

    Kanpur, India Innosential Full time

    Company Description Innosential is an AI-focused organization in the Greater Bengaluru Area, dedicated to providing cutting-edge expertise in AI & data science. Our offerings include AI Solutions, Innosential Learning, and Recognition Labs, aimed at empowering businesses to build innovative solutions that deliver exceptional value. Algosential AI and Langue...

  • Data Analyst

    2 months ago

    Kanpur, India Meril Full time

    Job Title: Data Analyst (Python DB/ Parallel Training)Experience:0-1 YearsLocation:Kanpur (New Technopark @ IIT Kanpur)Job Type:Full TimeJob Description:We are seeking a highly motivatedData Analystwithstrong Python programming skillsand a passion for working with data. The ideal candidate should have basic knowledge of machine learning and be familiar with...

  • Data analyst

    1 month ago

    Kanpur, India Meril Full time

    Job Title: Data Analyst (Python DB/ Parallel Training)Experience: 0-1 YearsLocation: Kanpur (New Kanpur)Job Type: Full TimeJob Description:We are seeking a highly motivated Data Analyst with strong Python programming skills and a passion for working with data. The ideal candidate should have basic knowledge of machine learning and be familiar with...

  • Data Analyst

    1 month ago

    Kanpur, India Meril Full time

    Job Title: Data Analyst (Python DB/ Parallel Training)Experience: 0-1 YearsLocation: Kanpur (New Technopark @ IIT Kanpur)Job Type: Full TimeJob Description:We are seeking a highly motivated Data Analyst with strong Python programming skills and a passion for working with data. The ideal candidate should have basic knowledge of machine learning and be...

  • Azure DevOps Engineer(MLOPS) Transformation Specialist

    4 weeks ago

    Kanpur, Uttar Pradesh, India Tech Mahindra Full time

    We are seeking an experienced Azure DevOps Engineer with a strong focus on YAML coding, to join our team in Hyderabad or Bengaluru. The ideal candidate will have experience working on Azure ML using Python and Azure Kubernetes.Job Description:Proficiency in Azure DevOps with a strong emphasis on automation and CI/CD pipelines.Experience with Azure Machine...

  • Data Scientist

    3 days ago

    Kanpur, India Zazz Full time

    Company Description Zazz.io is a global tech company specializing in crafting incredible digital and mobile experiences using the latest technological platforms. Our team of experts, including UI/UX specialists, programmers, business analysts, and more, work together to deliver innovative solutions powered by technologies like AI, ML, IoT, and Metaverse. We...

  • Data Analyst

    2 months ago

    Kanpur, India Meril Full time

    Job Title: Data Analyst (Python DB/ Parallel Training)Experience: 0-1 YearsLocation: Kanpur (New Technopark @ IIT Kanpur)Job Type: Full TimeJob Description:We are seeking a highly motivated Data Analyst with strong Python programming skills and a passion for working with data. The ideal candidate should have basic knowledge of machine learning and be...

  • Data Analyst

    2 months ago

    Kanpur, India Meril Full time

    Job Title: Data Analyst (Python DB/ Parallel Training)Experience: 0-1 YearsLocation: Kanpur (New Technopark @ IIT Kanpur)Job Type: Full TimeJob Description:We are seeking a highly motivated Data Analyst with strong Python programming skills and a passion for working with data. The ideal candidate should have basic knowledge of machine learning and be...

  • Data Scientist for LTIM Hyderabad Office

    1 week ago

    Kanpur, Uttar Pradesh, India LTIMindtree Full time

    About the RoleWe are seeking a highly skilled Data Scientist to join our team at LTIM Hyderabad office. As a key member of our team, you will be responsible for developing and implementing machine learning models to drive business growth.Key ResponsibilitiesDesign and develop machine learning models using Python, RAG, and Cloud (AWS/Azure)Analyze large...

  • Mobile Software Architect

    4 weeks ago

    Kanpur, Uttar Pradesh, India Flex AI Full time

    Company Overview:Flex AI is a pioneering platform dedicated to revolutionizing the fitness industry by providing personalized workout plans, nutrition guidance, and community support. Our mission is to make fitness accessible and enjoyable for everyone.About Us:We are a dynamic team of innovators committed to creating cutting-edge solutions that empower...

  • Senior ios engineer

    4 weeks ago

    Kanpur, India Flex AI Full time

    Job Title: Senior i OS EngineerLocation: RemoteAbout Us:Flex AI is a leading platform dedicated to helping people achieve their fitness goals through personalized workout plans, nutrition guidance, and community support. Our mission is to make fitness accessible and enjoyable for everyone. Join our dynamic team and help us create innovative solutions that...

  • Zazz | Data Scientist

    3 days ago

    kanpur, India Zazz Full time

    Company Description Zazz.io is a global tech company specializing in crafting incredible digital and mobile experiences using the latest technological platforms. Our team of experts, including UI/UX specialists, programmers, business analysts, and more, work together to deliver innovative solutions powered by technologies like AI, ML, IoT, and Metaverse. We...

  • Full stack engineer

    2 months ago

    Kanpur, India Meril Full time

    Job Title: Full-stack Developer (Webapp for AI/ML projects)Experience: 1.5 – 3 YearsLocation: Kanpur (New Kanpur)Job Type: Full-TimeJob Description:We are seeking a talented Full-stack Developer with experience in building and deploying web applications throughout the Software Development Life Cycle (SDLC) to production environments. The ideal...

  • Full Stack Engineer

    2 months ago

    Kanpur, India Meril Full time

    Job Title: Full-stack Developer (Webapp for AI/ML projects)Experience: 1.5 – 3 YearsLocation: Kanpur (New Technopark @ IIT Kanpur)Job Type: Full-TimeJob Description:We are seeking a talented Full-stack Developer with experience in building and deploying web applications throughout the Software Development Life Cycle (SDLC) to production environments. The...