15h Left Data Engineer

4 days ago

Bengaluru, India Mastek Full time

Data Engineer – NiFi / Cloudera / Iceberg / Snowflake / Databricks Overview We are seeking a Data Engineer with strong Apache NiFi expertise to design and implement pipelines that move and transform data from Cloudera (HDFS/Hive/Impala) into Apache Iceberg tables, with downstream integration into Snowflake and Databricks. The ideal candidate will have hands-on experience with modern data lakehouse architectures and will play a critical role in enabling scalable, governed, and high-performance data platforms. Key Responsibilities: - Data Ingestion & Pipeline Development - Design, configure, and maintain NiFi data flows to extract, transform, and load data from Cloudera into Iceberg tables. - Implement streaming and batch ingestion pipelines with NiFi processors and custom scripting where needed. - Optimize NiFi workflows for scalability, reliability, and monitoring. - Data Lakehouse Enablement - Build and manage Apache Iceberg-based datasets for structured, semi-structured, and unstructured data. - Ensure schema evolution, partitioning, and metadata management in Iceberg. - Develop integration flows from Iceberg to Snowflake and Databricks for analytics, ML, and reporting use cases. - Integration & Orchestration - Work with Snowflake to ingest curated data from Iceberg for enterprise reporting and commercial insights. - Collaborate with Databricks teams to enable advanced analytics and machine learning use cases. - Integrate NiFi pipelines with orchestration tools (Airflow, Oozie, or AWS/Azure/GCP schedulers). - Performance, Security & Governance - Tune NiFi flows and Snowflake/Databricks ingestion for performance and cost optimization. - Implement role-based security and ensure compliance (HIPAA, GDPR, SOX if applicable). - Work with governance teams to enable lineage, metadata tracking, and auditability. Qualifications: - Bachelor's degree in Computer Science, Information Systems, or related field. - 5+ years of data engineering experience, with at least 2+ years working with Apache NiFi. - Strong experience with Cloudera ecosystem (HDFS, Hive, Impala, Spark). - Hands-on expertise with Apache Iceberg (schema evolution, time travel, partitioning, compaction). - Working knowledge of Snowflake and Databricks integration patterns. - Proficiency in SQL and one programming language (Python, Java, or Scala). - Understanding of data lakehouse architectures and ETL/ELT best practices.

  • ▷ [15h Left] Cloud Engineer

    4 days ago

    Bengaluru, India Tata Consultancy Services Full time

    Greetings from TCS!!!TCS has been a great pioneer in feeding the fire of Techies like you. We are a global leader in the technology arena and there's nothing that can stop us from growing together.Your role is of key importance, as it lays down the foundation for the entire project.Make sure you have a valid EP number before the interview. To create an EP...

  • (15h Left) Data Engineer

    1 week ago

    Bengaluru, India People Prime Worldwide Full time

    Job Title: Senior Data Engineer Skills: Data Engineering, AgenticAI, Python, SQL, Api’s, Azure cloud services, SQL and NoSQL ,Docker, Kubernetes. Location: Pan India Job Type: Full-Time Experience Level: 4-11 Years About the Role: We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. The ideal candidate will have a...

  • Data Engineer

    4 days ago

    Bengaluru, India Mastek Full time

    Data Engineer – NiFi / Cloudera / Iceberg / Snowflake / DatabricksOverviewWe are seeking a Data Engineer with strong Apache NiFi expertise to design and implement pipelines that move and transform data from Cloudera (HDFS/Hive/Impala) into Apache Iceberg tables, with downstream integration into Snowflake and Databricks. The ideal candidate will have...

  • Data Engineer

    1 week ago

    Bengaluru, India Alp Consulting Ltd. Full time

    Dear Candidate Please find the below JD for your reference: - Hands-on experience with AWS or similar cloud platform building data engineering solutions for analytics and science. (3+ years) - Must have experience building complex data pipelines – batch and/or real time event-based processing (3+ years) - Strong experience in designing, building, and...

  • ▷ 15h Left! Informatica Developer

    4 weeks ago

    Bengaluru, India NTT Data Full time

    Job Description NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Informatica Developer to join our team in Bangalore, Karntaka (IN-KA), India (IN). Informatica Developer Role Overview...

  • ▷ (3 Days Left) Data Engineer

    3 weeks ago

    Bengaluru, India NTT Data Full time

    Job Description NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer to join our team in Bangalore, Karntaka (IN-KA), India (IN). - Hands-on experience in building and...

  • Senior Data Engineer

    2 days ago

    Bengaluru, India Avathon Full time

    Who We Are & Why Join Us Avathon is revolutionizing industrial AI with a powerful platform that enables businesses to harness the full potential of their operational data. Our technology seamlessly integrates and contextualizes siloed datasets, providing a 360-degree operational view that enhances decision-making and efficiency. With advanced capabilities...

  • Senior Data Engineer

    4 weeks ago

    Bengaluru, India JRD Systems Full time

    We are seeking a Senior Data Engineer with deep expertise in AWS Cloud Development . This role is critical to both the development of new data products and the modernization of existing platforms. The ideal candidate is a seasoned data engineer with hands-on experience designing, building, and optimizing large-scale data pipelines and architectures in both...

  • ▷ [15h Left] Snowflake Data Engineer

    2 weeks ago

    Bengaluru, India Teamware Solutions Full time

    Job Description Job Summary: The Snowflake Data Engineer is responsible for building and managing data pipelines and data warehousing solutions using the Snowflake platform. This role involves working with large datasets, ensuring data quality, and enabling scalable data integration and analytics across the organization. Key Responsibilities: - Design,...

  • (15h Left) AWS Data Engineer

    2 weeks ago

    Bengaluru, India Tata Consultancy Services Limited Full time

    Job Description Key Skills Snowflake, SQL, AWS, DBT Must have Skills 1. 3+ years of data engineering experience, including practical experience using Snowflake for Data Engineer tasks. 2. A working knowledge of Restful APIs, SQL, semi-structured datasets, and cloud native concepts. 3. Experience in ELT toolsFivetran, Qlik replicate, Matillion,Experience with...