Data Engineer

2 days ago

Remote, India Prospus Consulting Pvt. Ltd. Full time

Job Title: Data Engineer

Location: Remote

Employment Type: Full-time

About the Role We are looking for a highly skilled Data Engineer with strong expertise in data ingestion, transformation and indexing pipelines for AI agents. The ideal candidate will have experience designing scalable data pipelines, implementing chunking strategies, and ensuring vector DB quality and optimal retrieval for AI agents.

Key Responsibilities

  • ETL & Data Feeds: Manage, optimize, and monitor ETL workflows and data ingestion pipelines.
  • Chunking Strategies for RAG: Design and implement effective document chunking methodologies to improve retrieval quality.
  • Vector Database Management: Create and maintain metadata enrichment and indexing logic; monitor and fine-tune vector DB performance.
  • Pipeline Scalability: Build and maintain scalable data pipelines using tools like Pandas, Spark, and Airflow.
  • Logging & Analytics: Develop pipelines to track retrieval success/failure and feed insights back into model optimization.
  • Data Privacy & Compliance: Ensure all pipelines adhere to data privacy regulations and organizational compliance standards.
  • Troubleshooting: Identify and resolve data-related root causes impacting system responses.

Required Skills & Qualifications

  • Strong proficiency in Pandas, Spark, Airflow, or similar data engineering tools.
  • Proven expertise in Vector Databases (e.g., Pinecone, Weaviate, Milvus, FAISS).
  • Hands-on experience with chunking strategies for RAG pipelines.
  • Solid understanding of scalable ETL/data pipelines.
  • Experience with metadata management and indexing strategies.
  • Strong debugging, optimization, and performance tuning skills.
  • Knowledge of data privacy laws (GDPR, HIPAA, etc.) and compliance practices.

Preferred Qualifications

  • Experience with ETL and ML data pipelines.
  • Familiarity with chunking and metadata strategies for RAG.
  • Proficiency in Python and SQL for data transformation and querying.

Job Type: Full-time

Pay: ₹501, ₹1,244,400.38 per year

Benefits:

  • Paid sick time
  • Paid time off
  • Provident Fund
  • Work from home

Work Location: Remote

  • Operation Analyst

    1 week ago

    Remote, India Data Engineer Academy LLP Full time ₹ 6,00,000 - ₹ 12,00,000 per year

    Job Title: Operations AnalystLocation: Remote | Experience: 0-1 YearsType: Full-TimeNotice-Immediate JoinerAbout Us: Data Engineering Academy is an edtech startup led by FAANG data engineers. We equip aspiring data professionals with the tools, training, and support to succeed in the tech industry.About the Role:We're looking for a proactive Operations...

  • Azure AI Data Engineer

    1 hour ago

    Remote, India Data PlatformExperts Full time ₹ 70,000 - ₹ 15,00,000 per year

    Job Description:We are looking for a skilled and motivated AI Azure Data Engineer with strong expertise in Microsoft Azure data services. The ideal candidate must have hands-on experience with Databricks or Microsoft Fabric and hold a valid certification in either. You will be responsible for designing, developing, and deploying scalable data pipelines,...

  • Senior Data Engineer

    1 week ago

    Remote, India AABM Cloud Data Solutions Full time ₹ 20,00,000 per year

    Senior Data Engineer (Remote, India)10+ Years Data Engineer Experience RequiredSQL, Python, Snowflake - Mandatory hands-on experienceExperience with any ETL Tools (SSIS Preferred), Cloud Environments, and DB2/DBT is a plus.Job Type: Full-timePay: From ₹2,000,000.00 per yearApplication Question(s):How many years of Python Experience do you have?How many...

  • ▷ High Salary! Data

    1 week ago

    Remote, India NTT Data Full time

    Job Description NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data & AI Engineer Lead to join our team in Remote, Karntaka (IN-KA), India (IN). Job Duties: Role Overview The Data...

  • Data engineer

    2 days ago

    Remote, India INNONEXIS TECHNOLOGY PRIVATE LIMITED Full time ₹ 4,80,000 - ₹ 6,00,000 per year

    We're Hiring: Freelance Data EngineerWe are looking for a talented Freelance Data Engineer to help us design, build, and optimize scalable data pipelines and platforms. Key Responsibilities:Design, develop, and maintain ETL pipelines and workflowsWork with structured and unstructured data across multiple sourcesImplement and optimize data solutions on AWS...

  • Data Engineer

    6 days ago

    Remote, India Mig Staffing Full time ₹ 12,00,000 - ₹ 30,00,000 per year

    We are seeking a dynamic and highly skilled Data Engineer who has extensive experience building enterprise scale data platforms and lead these foundational efforts. This role demands someone who not only possesses a profound understanding of the data engineering landscape but is also at the forefront of their game. The ideal candidate will contribute...

  • Data Engineer

    2 days ago

    Remote, India NS Global Corporation Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job Description : We are looking for an experienced GCP Data Engineer with a minimum of 5 years of professional experience in data engineering, cloud-based data solutions, and large-scale distributed systems. This role is fully remote and requires a hands-on professional who can design, build, and optimize data pipelines and solutions on Google Cloud...

  • Data Engineer

    2 days ago

    Remote, India Codash Solutions Full time ₹ 10,80,000 - ₹ 12,00,000 per year

    We are seeking a highly skilled and motivated Data Engineer to join our growing team. The ideal candidate will have a strong background in building and maintaining scalable data pipelines, with hands-on experience in ETL processes, Python programming, Apache Airflow, Apache Spark, and Neo4j graph database technology. You will play a crucial role in...

  • Data Engineer

    1 week ago

    Remote, India Deltacubes Technologies Full time ₹ 13,20,000 - ₹ 14,40,000 per year

    JD(1) for Data Engineer with strong Data Vault 2.0 - Chennai basedImmediate Joiners only· Experience: 6 -8yearsWFO/WFH/ Hybrid· Strong expertise in data transformation and pipeline development (SQL, Python, DBT)· Strong expertise and experience in Data Vault 2.0 modeling framework· IDE-savvy, working with dbt CLI locally and github copilot· Build dbt...

  • Data Engineer

    7 days ago

    Remote, India Panapps Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Job TitleData EngineerJob LocationRemote/Work From AnywhereJob TypeFull TimeNo. of Positions2Date PostedSeptember 8, 2025Preferred Experience6+Job DescriptionWe are seeking a skilled and experienced Data Engineer to join our team. The ideal candidate will have a strong background in data engineering, ETL processes, and data integration, with hands-on...