Senior Big Data Engineer

4 days ago

Remote, India UMENIT SOLUTIONS LLP Full time ₹ 2,40,000 - ₹ 6,00,000 per year

Job Title: Senior Big Data EngineerLocation: Remote / Hybrid (Preferred timezone overlap with [your region])

Type: Full-time / ContractAbout the ProjectWe're building an intelligent data pipeline that brings SAP AO (Analysis Office) data into a central database, which will serve as the foundation for an AI-powered chatbot. This chatbot will help users interact with complex enterprise data through natural language queries.We already have a data science team in-house. What we need now is a Big Data Engineer who can own the data extraction, transformation, and storage process from SAP AO to a scalable, queryable data layer.Key Responsibilities

  • Design, build, and manage data pipelines to extract, transform, and load (ETL/ELT) data from SAP AO (and related SAP systems such as BW or HANA) into a database or data warehouse.
  • Ensure data quality, consistency, and completeness during the migration process.
  • Set up a data lake or data warehouse (e.g., BigQuery, Snowflake, Redshift, or Azure Synapse) optimized for large-scale analytical queries.
  • Work closely with data scientists to prepare data for downstream AI and analytics use cases, including chatbot and natural language query systems.
  • Implement data modeling, partitioning, and indexing strategies for performance optimization.
  • Automate and monitor ETL pipelines using tools such as Apache Airflow, AWS Glue, Databricks, or similar orchestration frameworks.
  • Collaborate with AI/ML engineers to enable embedding and semantic search pipelines (RAG architecture).
  • Ensure compliance with data governance, access control, and security best practices.

Required Skills & Experience

  • 5+ years of experience as a Big Data Engineer / Data Engineer.
  • Strong experience in ETL/ELT pipeline design and implementation.
  • Hands-on experience with SAP data integration — particularly SAP AO, SAP BW, or SAP HANA data extraction.
  • Proficiency in Python / PySpark / SQL.
  • Experience with big data frameworks such as Spark, Hadoop, or Databricks.
  • Deep understanding of data warehousing concepts (fact/dimension modeling, partitioning, schema design).
  • Experience with cloud data ecosystems (AWS, GCP, or Azure).
  • Familiarity with data APIs, connectors, or SDKs for SAP.
  • Strong problem-solving skills and the ability to work collaboratively with AI and software teams.

Nice-to-Have

  • Experience integrating data pipelines with AI/LLM-based systems.
  • Knowledge of vector databases (Pinecone, Weaviate, FAISS) or RAG pipelines.
  • Exposure to chatbot architecture and natural language query systems.
  • Experience setting up CI/CD for data pipelines.

What You'll Work On

  • Building a robust bridge between SAP AO data and our internal database.
  • Creating a scalable data foundation for AI-driven insights and chatbot experiences.
  • Collaborating with data scientists and AI engineers to bring structured enterprise data to life through conversational interfaces.

Job Type: Contract

Pay: ₹ ₹50.00 per hour

Expected hours: 40 per week

Experience:

  • Big Data Engineer / Data Engineer.: 5 years (Required)
  • ETL/ELT pipeline design and implementation.: 5 years (Required)
  • SAP data integration : 3 years (Required)
  • Python / PySpark / SQL.: 5 years (Required)
  • CI/CD for data pipelines.: 3 years (Required)
  • data pipelines with AI/LLM-based systems.: 3 years (Required)

  • Big Data Trainer

    1 week ago

    Remote, India LCM Full time

    **Hiring: Part-Time Hadoop & Apache Spark Trainer (Big Data - Data Engineering)** **Join LCM - The Live Course Marketplace** Are you a Big Data professional with hands-on expertise in Hadoop, Apache Spark, and data engineering tools? LCM is looking for engaging trainers to deliver live, hands-on sessions on industry-relevant Big Data technologies. **What...

  • Architect - Big Data

    3 days ago

    Remote, India Databricks Full time

    CSQ125R88 As an **Architect (Big Data) **in our Professional Services team you will work with clients on short to medium term customer engagements on their **Big Data challenges using the Databricks platform**. You will provide data engineering, data science, and cloud technology projects which require integrating with client systems, training, and other...

  • Senior Data Engineer

    2 days ago

    Remote, India AABM Cloud Data Solutions Full time ₹ 20,00,000 per year

    Senior Data Engineer (Remote, India)10+ Years Data Engineer Experience RequiredSQL, Python, Snowflake - Mandatory hands-on experienceExperience with any ETL Tools (SSIS Preferred), Cloud Environments, and DB2/DBT is a plus.Job Type: Full-timePay: From ₹2,000,000.00 per yearApplication Question(s):How many years of Python Experience do you have?How many...

  • Big Data Architect

    7 days ago

    Remote, India Databricks Full time

    CSQ424R34 As a** Big Data Architect **in our Professional Services team you will work with clients on short to medium term customer engagements on their **Big Data challenges using the Databricks platform**. You will provide data engineering, data science, and cloud technology projects which require integrating with client systems, training, and other...

  • Senior Data Engineer

    6 days ago

    Remote, India Hunarstreet Technologies Pvt Ltd Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Description : Position : Senior Data EngineerOverview : We are seeking an experienced Senior Data Engineer to design, build, and optimize scalable data pipelines and infrastructure to support cross-functional teams and next-generation data initiatives. The ideal candidate is a hands-on data expert with strong technical proficiency in Big Data...

  • Big Data

    1 day ago

    Remote, India KEG HR Services Full time

    **Greetings From KEG HR Services!** Job Role: Big Data & Kubernetes Administrator Experience: 3 Years to 5 Years **Job Description**: - Install, configure, and maintain Cloudera Big Data clusters across multiple environments, ensuring optimal performance and resource utilization. - Ensure high availability and manage the performance of data services like...

  • Big Data Architect

    1 day ago

    Remote, India Databricks Full time

    As an **Architect** in our Professional Services team you will work with clients on short to medium term customer engagements on their big data challenges using the Databricks platform. You will provide data engineering, data science, and cloud technology projects which require integrating with client systems, training, and other technical tasks to help...

  • Senior Data Engineer

    6 days ago

    Remote, India IMRIEL TECHNOLOGY SOLUTIONS PRIVATE LIMITED Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Description : If you are a smart & passionate team player - then this Senior Data Engineer [Snowflake] opportunity is for you We at IMRIEL(An Allata Company) are looking for a Senior Data Engineer to implement methods to improve data reliability and quality. You will combine raw information from different sources to create consistent and...

  • Senior Data Engineer

    1 week ago

    Remote, India Mobile Programming Full time ₹ 50,000 - ₹ 18,42,514 per year

    We're Hiring: Data EngineerLocation: RemoteExperience: 5 YearsNotice Period: Immediate Joiners PreferredRequired Skills & Qualifications:Bachelor's or Master's degree in Computer Science, Data Engineering, or related field.Strong proficiency in SQL and NoSQL databases (GraphDB, MySQL, PostgreSQL, MongoDB, Cassandra).Expertise with big data tools — Apache...

  • Senior Data Engineer

    6 days ago

    Remote, India Hyqoo Full time ₹ 5,00,000 - ₹ 25,00,000 per year

    Remote (LATAM)5-8 yrs Exp.Contract6 MonthsStarts - Sep 7, 2025JOBJob DescriptionJob Responsibilities:Job Title: Senior Data Engineer Location: Latin America preferred (Mexico, Brazil, Caribbean); must work CST hoursInterview Process: 2 rounds Introduction: The Senior Data Engineer plays a critical role in designing, coding, testing, and analyzing software...