Data Engineer

3 days ago

Anantapur, India Keasis Full time

We are seeking a Data Engineer with strong Apache NiFi expertise to design and implement pipelines that move and transform data from Cloudera (HDFS/Hive/Impala) into Apache Iceberg tables, with downstream integration into Snowflake and Databricks. The ideal candidate will have hands-on experience with modern data lakehouse architectures and will play a critical role in enabling scalable, governed, and high-performance data platforms. Key Responsibilities Data Ingestion & Pipeline Development=Design, configure, and maintain NiFi data flows to extract, transform, and load data from Cloudera into Iceberg tables.Implement streaming and batch ingestion pipelines with NiFi processors and custom scripting where needed.Optimize NiFi workflows for scalability, reliability, and monitoring. Data Lakehouse EnablementBuild and manage Apache Iceberg-based datasets for structured, semi-structured, and unstructured data.Ensure schema evolution, partitioning, and metadata management in Iceberg.Develop integration flows from Iceberg to Snowflake and Databricks for analytics, ML, and reporting use cases. Integration & OrchestrationWork with Snowflake to ingest curated data from Iceberg for enterprise reporting and commercial insights.Collaborate with Databricks teams to enable advanced analytics and machine learning use cases.Integrate NiFi pipelines with orchestration tools (Airflow, Oozie, or AWS/Azure/GCP schedulers). Performance, Security & GovernanceTune NiFi flows and Snowflake/Databricks ingestion for performance and cost optimization.Implement role-based security and ensure compliance (HIPAA, GDPR, SOX if applicable).Work with governance teams to enable lineage, metadata tracking, and auditability. QualificationsBachelor’s degree in Computer Science, Information Systems, or related field.5+ years of data engineering experience, with at least 2+ years working with Apache NiFi.Strong experience with Cloudera ecosystem (HDFS, Hive, Impala, Spark).Hands-on expertise with Apache Iceberg (schema evolution, time travel, partitioning, compaction).Working knowledge of Snowflake and Databricks integration patterns.Proficiency in SQL and one programming language (Python, Java, or Scala).Understanding of data lakehouse architectures and ETL/ELT best practices.

  • Data Engineer

    3 weeks ago

    Anantapur, India SolarSquare Full time

    Job Description: Data EngineerAs a Data Engineer, you will own the end-to-end lifecycle of our data infrastructure. You willdesign and implement robust, scalable data pipelines and architect modern data solutionsusing a best-in-class technology stack. Your work will transform raw, messy data into clean,reliable, and actionable data products that power...

  • Backend and data pipeline engineer

    3 weeks ago

    Anantapur, India JRD Systems Full time

    Job Role: Backend and Data Pipeline Engineer - PythonLocation: RemoteJob Type : Fulltime** Only Immediate Joiners **Job Summary:The Team:We’re investing in technology to develop new products that help our customers drive their growth and transformation agenda. These include new data integration, advanced analytics, and modern applications that address new...

  • Senior Data Scientist

    3 days ago

    Anantapur, India Recro Full time

    Role : Senior Data Scientist (AI & Data)Location: Remote (Work from Home)We’re hiring a Senior Data Scientist to build the next generation of intelligentdecision systems that power pricing, supply optimization, ranking, and personalizationin our global B2B hotel marketplace.This is a high-impact role at the intersection of machine learning, optimization,...

  • Social Media Data Engineer

    3 days ago

    Anantapur, India Deeter Investments LLP Full time

    NOTE: IF YOU HAVE NO EXPERIENCE AQUIRING BULK SOCIAL MEDIA DATA DO NOT APPLY-Please explain your relevant experience clearly at the TOP of your CV. If you do not you will not be considered About Deeter InvestmentsDeeter Investments is a founder-led proprietary trading firm built around real-time, data-driven decision-making. We prize curiosity,...

  • sap abap

    3 days ago

    Anantapur, Andhra Pradesh, India NTT DATA Full time ₹ 5,00,000 - ₹ 15,00,000 per year

    Company DescriptionNTT DATA, a part of NTT Group, is headquartered in Tokyo and provides IT and business services globally. We help clients transform through consulting, industry solutions, business process services, digital and IT modernization, and managed services. Our commitment to our clients' long-term success allows us to enable not only them but...

  • Data Architect

    3 weeks ago

    Anantapur, India ThoughtFocus Full time

    ONLY IMMEDIATE JOINERS (0-10 days)Data Architect / Data ModelerPosition Overview Experienced Data Architect / Data Modeler with strong expertise in Financial Services, and preferably exposure to Private Equity. The role involves designing enterprise-level data models, defining data architecture standards, and ensuring the scalability, performance, and...

  • Enterprise Data Architect

    2 weeks ago

    Anantapur, India Patch the Skill Full time

    Company DescriptionPatch the Skill is a provider of corporate training services, specializing in technical and soft skills training such as Voice & Accent, Cloud, AI, Machine Learning, IoT, Data Science, and more. With a focus on enhancing communication skills and team building, we offer a wide range of training programs including AWS, Azure, Java...

  • Mern, data observability and ai tech lead

    10 hours ago

    Anantapur, India Seosaph-infotech Full time

    Company DescriptionSeosaph-infotech is a growing company in the field of customized software development, offering a range of tech solutions to businesses in different verticals. Within just two years, Seosaph-infotech has delivered exceptional solutions to industries such as Finance, Healthcare, and E-commerce. Our vision is to help our clients become...

  • Data Modernization Lead– Databricks Expert

    3 weeks ago

    Anantapur, India Saradysol Tech Ventures Private Limited Full time

    About the job Job description Job Title: Senior Data Modernization Lead– Databricks Expert (Contract)  Location: Remote Experience Level: 15+ Years Role Overview We are looking for an experienced Data Modernization Lead to spearhead our team’s transition to modern data engineering practices . This role will focus on migrating data assets and...

  • Data Analyst- Specialist

    2 days ago

    Anantapur, India Marsh McLennan Full time

    Marsh McLennan Global Services India Private Limited (MMGS) is the world’s leading professional services firm in the areas of risk, strategy and people. The Company’s more than 85,000 colleagues advise clients in over 130 countries. With annual revenue of $23 billion, Marsh McLennan helps clients navigate an increasingly dynamic and complex environment...