Data Engineer

4 weeks ago

Bengaluru India Mastek Full time

Job Description Data Engineer NiFi / Cloudera / Iceberg / Snowflake / Databricks Overview We are seeking a Data Engineer with strong Apache NiFi expertise to design and implement pipelines that move and transform data from Cloudera (HDFS/Hive/Impala) into Apache Iceberg tables, with downstream integration into Snowflake and Databricks. The ideal candidate will have hands-on experience with modern data lakehouse architectures and will play a critical role in enabling scalable, governed, and high-performance data platforms. Key Responsibilities: - Data Ingestion & Pipeline Development - Design, configure, and maintain NiFi data flows to extract, transform, and load data from Cloudera into Iceberg tables. - Implement streaming and batch ingestion pipelines with NiFi processors and custom scripting where needed. - Optimize NiFi workflows for scalability, reliability, and monitoring. - Data Lakehouse Enablement - Build and manage Apache Iceberg-based datasets for structured, semi-structured, and unstructured data. - Ensure schema evolution, partitioning, and metadata management in Iceberg. - Develop integration flows from Iceberg to Snowflake and Databricks for analytics, ML, and reporting use cases. - Integration & Orchestration - Work with Snowflake to ingest curated data from Iceberg for enterprise reporting and commercial insights. - Collaborate with Databricks teams to enable advanced analytics and machine learning use cases. - Integrate NiFi pipelines with orchestration tools (Airflow, Oozie, or AWS/Azure/GCP schedulers). - Performance, Security & Governance - Tune NiFi flows and Snowflake/Databricks ingestion for performance and cost optimization. - Implement role-based security and ensure compliance (HIPAA, GDPR, SOX if applicable). - Work with governance teams to enable lineage, metadata tracking, and auditability. Qualifications: - Bachelor's degree in Computer Science, Information Systems, or related field. - 5+ years of data engineering experience, with at least 2+ years working with Apache NiFi. - Strong experience with Cloudera ecosystem (HDFS, Hive, Impala, Spark). - Hands-on expertise with Apache Iceberg (schema evolution, time travel, partitioning, compaction). - Working knowledge of Snowflake and Databricks integration patterns. - Proficiency in SQL and one programming language (Python, Java, or Scala). - Understanding of data lakehouse architectures and ETL/ELT best practices.

  • Data Engineer

    5 days ago

    Bengaluru, India NTT Data Full time

    Job Description NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer to join our team in Bangalore, Karntaka (IN-KA), India (IN). Job Duties: Snowflake - working experience...

  • Data Engineer

    4 days ago

    India Crayon Data Full time ₹ 5,00,000 - ₹ 15,00,000 per year

    Job DescriptionData Engineer – ChennaiBuild the foundation of data intelligence — design, develop, and optimize data pipelines that power AI-driven insights.Location: Chennai, IndiaExperience: 2–3 years in data engineering or data-driven product developmentRole OverviewAs a Data Engineer, you will be responsible for building and maintaining scalable...

  • Data Engineer

    2 weeks ago

    India NTT DATA Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Req ID: 343254NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer to join our team in Remote, Karnātaka (IN-KA), India (IN). "Key Responsibilities: Design and implement...

  • Data Entry Operator

    4 days ago

    Bengaluru, Karnataka, India Design Engineer Full time ₹ 2,64,000 per year

    We are seeking a dedicated and detail-oriented Data Entry Operator (DEO) to support R&D projects. The role involves accurate data entry, documentation, and maintenance of research records in secure systems. The DEO will assist scientists, engineers, and administrative staff by ensuring timely and error-free handling of project information.Key...

  • Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India NTT DATA Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Design and implement tailored data solutions to meet customer needs and use cases, spanning from streaming to data lakes, analytics, and beyond within a dynamically evolving technical stack. Provide thought leadership by recommending the most appropriate technologies and solutions for a given use case, covering the entire spectrum from the application layer...

  • ▷ (3 Days Left) Data Engineer

    4 weeks ago

    Bengaluru, India NTT Data Full time

    Job Description NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer to join our team in Bangalore, Karntaka (IN-KA), India (IN). - Hands-on experience in building and...

  • Data Engineer

    2 weeks ago

    Mumbai, India NTT Data Full time

    Job Description NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer to join our team in Mumbai, Mahrshtra (IN-MH), India (IN). Job Duties: Key Responsibilities: . Design...

  • Data & AI Engineer Lead

    4 days ago

    India NTT DATA Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Req ID: 344686NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data & AI Engineer Lead to join our team in Remote, Karnātaka (IN-KA), India (IN). "Job Duties: Role Overview The...

  • Lead Data Engineer

    3 days ago

    India Data Science Solutions Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    MUST HAVE• Minimum of 4+ years of Azure DE experience • In-depth technical knowledge of tools like Azure Data Factory, Databricks, Azure Synapse, SQL DB, ADLS etc.• Experience in collaborating with business stakeholders to identify and meet data requirements• Experience in using Azure services and tools to ingest, egress, and transform data from...

  • Data Engineer- ADF

    10 hours ago

    India Optimum Data Analytics Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Role: Data EngineerExperience: 6–8 yearsShift Timing: 7:00 PM to 3:00 AMResponsible for design, development, and implementation of solutions to exchange data using the required ETL tool, Azure Data Factory, Synapse and DataBricks.Experience in data cleansing and data migration using Python.At least 3+ years of hands-on experience in Azure ADF.Coding in...