Senior Data Engineer

3 weeks ago

india, IN Sovereign IT Solutions Pvt Ltd Full time

We are seeking a Senior Data Engineer with 8+ years of real-world experience to act as a Subject Matter Expert (SME) and train students specifically for Data Engineering, Big Data, and AI/ML Data Platform interviews.


The ideal candidate will not only be hands-on with the latest data + AI/ML technologies but also excel at coaching, mentoring, and conducting mock interviews to prepare students for success in top-tier companies.


This role is perfect for someone who can bridge deep technical expertise with interview preparation and training.


Key Responsibilities


Act as SME & trainer, guiding students to crack Data Engineering, AI/ML, and Cloud Data Platform interviews.


Conduct mock interviews, Q&A sessions, and technical deep-dives.


Train students on real-world interview scenarios:


End-to-end ETL/ELT pipelines


Data modelling & warehousing


Data for AI/ML use cases (feature pipelines, vector databases, embeddings)


Streaming & batch processing at scale


Data governance, lineage, and security-first architectures


Core Data Engineering + AI/ML Pillars to Cover


1. Data Warehousing & Modelling: Star, Snowflake, and Data Vault modelling, SCD Types 1–6, OLTP vs OLAP vs Lakehouse, Schema evolution & data versioning


2. ETL/ELT & Orchestration


Orchestration: Apache Airflow, Prefect, Dagster, Azure Data Factory

Transformation: dbt, Spark SQL, Pandas, PySpark

Batch & streaming workflows (Kafka, Flink, Spark Structured Streaming, Kinesis)


3. Big Data & Distributed Processing: Spark (PySpark, Scala, Delta Lake), Hive, Presto/Trino, Iceberg, Hudi, Partitioning, bucketing, caching & shuffle optimisation, Lakehouse & Data Mesh architectures


4. Cloud Data Platforms


Snowflake, Databricks, BigQuery, Redshift, Synapse, Multi-cloud (AWS, Azure, GCP) + hybrid/on-prem migration


5. Data Storage & Ingestion: Data Lakes (S3, ADLS, GCS), Semi/unstructured data (Parquet, ORC, Avro, JSON, XML, multimedia), Real-time ingestion (Kafka, Pulsar, Debezium, CDC pipelines)


6. Observability & Monitoring


Pipeline observability (Prometheus, Grafana, ELK, CloudWatch, Datadog)

Data quality & reliability (Great Expectations, Soda, Deequ)

End-to-end lineage & metadata (Apache Atlas, DataHub, Purview, Collibra)


7. Security & Governance


IAM, RBAC, ABAC, fine-grained access controls

Data masking, tokenization, PII handling

GDPR, HIPAA, CCPA compliance

Secrets management (Vault, Key Vault, KMS)


8. AI/ML Enablement


Feature Engineering Pipelines: scalable pipelines for ML models

Feature Stores: Feast, Tecton, Databricks Feature Store

MLOps Practices: MLflow, SageMaker Pipelines, Vertex AI, Azure ML

Vector Databases & RAG: Pinecone, Weaviate, Milvus, Chroma for LLM apps

Model Serving: TensorFlow Serving, TorchServe, Kubernetes-based serving

AI Workflows: data prep for NLP, embeddings, recommendation systems

LLM Integration: prompt engineering, embeddings pipeline, data optimisation for GenAI workloads


Required Skills & Experience


8+ years in Data Engineering/Big Data/Cloud roles, with hands-on AI/ML data enablement.


Proven expertise with ETL/ELT, Spark, Data Lakes, Data Warehouses, and distributed systems.


Strong programming skills in Python, SQL, PySpark (Scala/R/ optional).


Expertise in cloud-native data + AI/ML platforms (AWS SageMaker, Azure ML, GCP Vertex AI, Databricks).


Hands-on with MLOps, Feature Stores, Vector Databases, and ML model integration.


Deep understanding of pipeline optimisation, governance, and cost efficiency.


Excellent communication & teaching skills — ability to simplify complex data & AI concepts.


Prior mentoring, training, or interview-prep experience strongly preferred.

  • Data Engineer

    3 weeks ago

    india, IN Data-Hat AI Full time

    Department: Data Engineering & AI Solutions Reports To: Lead Data Solutions Architect Travel: International travel required (up to 30–40%) Position Summary: We are hiring a senior-level Data Engineer to lead the design, development, and optimization of high-performance data infrastructure that underpins mission-critical AI systems. With 12+ years of...

  • Senior Data Engineer

    3 weeks ago

    india, IN CareerXperts Consulting Full time

    We are looking for a highly skilled Senior Data Engineer to design, build, and optimize scalable data pipelines and infrastructure. The ideal candidate will have strong expertise in modern data engineering tools and frameworks, along with the ability to collaborate closely with data scientists, analysts, and product teams to deliver reliable and efficient...

  • Senior Data Engineer

    3 weeks ago

    india, IN The Mastermind Group Full time

    Senior Data Engineer Location: Remote (India) – Occasional travel to Riyadh, KSAContract Type: 12-Month ContractWork Week: Sunday–Thursday preferred (9 AM – 5 PM IST)Start Date: As soon as possibleAbout The Mastermind Group (TMG)TMG is a global leader in IT Spend Optimisation, helping enterprises take control of their tech investments across software,...

  • Senior Data Engineer

    3 weeks ago

    india, IN HealthyMe AI Full time

    About the RoleWe are seeking a Senior Data Engineer with deep expertise in building data pipelines and integrating healthcare data. You will report to the Head of AI and Engineering and be responsible for designing, building, and operating the backbone of our AI-powered applications. This is a critical role where you will create scalable pipelines that...

  • Senior Data Engineer

    3 weeks ago

    india, IN CES Full time

    We are looking for an enthusiastic and highly skilled Senior Data Engineer to join our growing team and play a key role in shaping complex data-centric solutions that power smarter decisions for our clients and internal teams.As part of our data engineering team, you’ll build and maintain scalable data systems and pipelines managing acquisition, storage,...

  • Senior Data Analyst

    3 weeks ago

    india, IN Milestone Technologies, Inc. Full time

    We are seeking a Senior Data Analyst / Engineer with strong experience in manufacturing control systems data, data authentication, and readiness engineering. This role will ensure data ingested from manufacturing systems into Client's data lake and downstream applications (e.g., PI) is fit-for-purpose, reliable, and trustworthy. The successful candidate will...

  • Data Engineer

    2 weeks ago

    india, IN Idyllic Services Full time

    Experience: 5- 10 YearsNotice Period: 0-30 DaysMandatory Skills: Code Optimization , Shell Scripting, Data Engineering & ScalaPreferred Skills: Big Data tools like Spark, Hadoop & HiveJob Description:The Senior Data Engineer will be responsible for designing, developing, and maintaining scalable data pipelines and building out new API integrations to support...

  • Senior Data Engineer

    2 weeks ago

    india, IN NOQ Full time

    Senior Data & Reporting Developer (Remote | India | Contract Role)Company: NOQ GroupLocation: RemoteContract Duration: 6 months (possibility of extension based on performance)Start Date: Immediate⸻About NOQ GroupNOQ Group is a technology-driven organization dedicated to delivering innovative data solutions for businesses in the food and beverage industry...

  • Data Engineer

    2 weeks ago

    india, IN Insight Global Full time

    Job DescriptionInsight Global is looking for a skilled senior level Data Engineer in Hyderabad or remote India for a large AAA gaming company on a contract for 3 months with possibility of extension or conversion. You will be assisting the Data and Analytics team on one of the company's largest game titles set to launch this fall. Your day-to-day will be...

  • Senior Data Engineer

    2 weeks ago

    Bangalore Urban, Karnataka, India, IN Sonata Software Full time

    The Senior Software Engineer supports functions which require automation/systems development, including initial development and ongoing support of the suite of applications that would help manage loan, pricing, enterprise data pipeline, or other functions depending on the department. As the Application Developer, you must be proficient in Python, Data...