Data Engineer
1 week ago
We are seeking a Data Engineer with strong Apache NiFi expertise to design and implement pipelines that move and transform data from Cloudera (HDFS/Hive/Impala) into Apache Iceberg tables, with downstream integration into Snowflake and Databricks. The ideal candidate will have hands-on experience with modern data lakehouse architectures and will play a critical role in enabling scalable, governed, and high-performance data platforms. Key Responsibilities Data Ingestion & Pipeline Development=Design, configure, and maintain NiFi data flows to extract, transform, and load data from Cloudera into Iceberg tables.Implement streaming and batch ingestion pipelines with NiFi processors and custom scripting where needed.Optimize NiFi workflows for scalability, reliability, and monitoring. Data Lakehouse EnablementBuild and manage Apache Iceberg-based datasets for structured, semi-structured, and unstructured data.Ensure schema evolution, partitioning, and metadata management in Iceberg.Develop integration flows from Iceberg to Snowflake and Databricks for analytics, ML, and reporting use cases. Integration & OrchestrationWork with Snowflake to ingest curated data from Iceberg for enterprise reporting and commercial insights.Collaborate with Databricks teams to enable advanced analytics and machine learning use cases.Integrate NiFi pipelines with orchestration tools (Airflow, Oozie, or AWS/Azure/GCP schedulers). Performance, Security & GovernanceTune NiFi flows and Snowflake/Databricks ingestion for performance and cost optimization.Implement role-based security and ensure compliance (HIPAA, GDPR, SOX if applicable).Work with governance teams to enable lineage, metadata tracking, and auditability. QualificationsBachelor’s degree in Computer Science, Information Systems, or related field.5+ years of data engineering experience, with at least 2+ years working with Apache NiFi.Strong experience with Cloudera ecosystem (HDFS, Hive, Impala, Spark).Hands-on expertise with Apache Iceberg (schema evolution, time travel, partitioning, compaction).Working knowledge of Snowflake and Databricks integration patterns.Proficiency in SQL and one programming language (Python, Java, or Scala).Understanding of data lakehouse architectures and ETL/ELT best practices.
-
Data engineer
3 weeks ago
India Data-Hat AI Full timeDepartment: Data Engineering & AI Solutions Reports To: Lead Data Solutions Architect Travel: International travel required (up to 30–40%) Position Summary: We are hiring a senior-level Data Engineer to lead the design, development, and optimization of high-performance data infrastructure that underpins mission-critical AI systems. With 12+...
-
Data Engineer
3 weeks ago
India Data-Hat AI Full timeDepartment: Data Engineering & AI Solutions Reports To: Lead Data Solutions Architect Travel: International travel required (up to 30–40%) Position Summary: We are hiring a senior-level Data Engineer to lead the design, development, and optimization of high-performance data infrastructure that underpins mission-critical AI systems. With 12+...
-
Data Engineer
3 weeks ago
India Data-Hat AI Full timeDepartment: Data Engineering & AI Solutions Reports To: Lead Data Solutions Architect Travel: International travel required (up to 30–40%) Position Summary: We are hiring a senior-level Data Engineer to lead the design, development, and optimization of high-performance data infrastructure that underpins mission-critical AI systems. With 12+ years of...
-
Data Engineer
2 weeks ago
india, IN Data-Hat AI Full timeDepartment: Data Engineering & AI Solutions Reports To: Lead Data Solutions Architect Travel: International travel required (up to 30–40%) Position Summary: We are hiring a senior-level Data Engineer to lead the design, development, and optimization of high-performance data infrastructure that underpins mission-critical AI systems. With 12+ years of...
-
Data & AI Engineer Lead
1 week ago
India NTT DATA Full time ₹ 12,00,000 - ₹ 36,00,000 per yearReq ID: 344686NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data & AI Engineer Lead to join our team in Remote, Karnātaka (IN-KA), India (IN). "Job Duties: Role Overview The...
-
Data Engineer
2 weeks ago
Bengaluru, India NTT Data Full timeJob Description NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer to join our team in Bangalore, Karntaka (IN-KA), India (IN). Design and implement tailored data...
-
Data Engineer
7 days ago
India CX Data Labs Full timeJob Description Hi Role - Azure Data Engineer Location - Chennai, Gurugram (5days Office) Full time Shift timing 2pm to 11pm Key Responsibilities: Develop, maintain, and optimize scalable ETL/ELT pipelines using PySpark and Databricks. Collaborate with cross-functional teams to design and implement data models and data integration solutions. Create and...
-
Senior/Lead Data Engineer
3 weeks ago
India Eucloid Data Solutions Full timeAbout The Role: Eucloid is looking for a senior/ lead Data Engineer to join our Data Platform team supporting various business applications. The ideal candidate will support development of data infrastructure on Databricks for our clients by participating in activities which may includestarting from up- stream and down-stream technology selection to...
-
Senior/lead data engineer
3 weeks ago
India Eucloid Data Solutions Full timeAbout The Role: Eucloid is looking for a senior/ lead Data Engineer to join our Data Platform team supporting various business applications. The ideal candidate will support development of data infrastructure on Databricks for our clients by participating in activities which may includestarting from up- stream and down-stream technology selection to...
-
Senior/Lead Data Engineer
3 weeks ago
India Eucloid Data Solutions Full timeAbout The Role: Eucloid is looking for a senior/ lead Data Engineer to join our Data Platform team supporting various business applications. The ideal candidate will support development of data infrastructure on Databricks for our clients by participating in activities which may includestarting from up- stream and down-stream technology selection to...
