Hive/NiFi Engineer

4 days ago


india Taggd Full time

Responsibilities :

- Design and develop data pipelines using Apache Nifi for ingesting, transforming, and loading data into our Big Data platform.

- Utilize Apache Hive for efficient data warehousing and querying large datasets.

- Leverage Apache Spark for large-scale data processing and analytics.

- Collaborate directly with clients to understand their business needs and translate those needs into effective Big Data solutions.

- Participate in discussions around data models and architecture for the Spectrum Mobile Data Hub platform.

- Fine-tune Spark-submit scripts for optimal performance and efficiency.

- Continuously optimize Hive SQL queries to ensure fast and reliable data retrieval.

- Create and manage Hive and Kudu tables for efficient data storage and retrieval.

- Utilize XML data sources for effective data loading into Big Data platforms.

Requirements :

- Bachelor's degree in Computer Science, Information Technology, or a related field (preferred).

- 3-4 years of experience working with Big Data technologies like Hive, Nifi, and Spark.

- Strong understanding of data warehousing concepts and data modeling principles.

- Proficiency in SQL and experience with HiveQL for querying large datasets.

- Experience with data ingestion and transformation tools like Apache Nifi.

- Experience with distributed processing frameworks like Apache Spark is a plus.

- Excellent communication and collaboration skills with the ability to interact effectively with clients.

- Strong analytical and problem-solving skills

(ref:hirist.tech)
  • Hive/NiFi Engineer

    4 days ago


    Anywhere in India/Multiple Locations Taggd Full time

    Responsibilities :- Design and develop data pipelines using Apache Nifi for ingesting, transforming, and loading data into our Big Data platform.- Utilize Apache Hive for efficient data warehousing and querying large datasets.- Leverage Apache Spark for large-scale data processing and analytics.- Collaborate directly with clients to understand their business...

  • Hive/NiFi Engineer

    6 days ago


    Anywhere in India/Multiple Locations, IN Taggd Full time

    Responsibilities :- Design and develop data pipelines using Apache Nifi for ingesting, transforming, and loading data into our Big Data platform.- Utilize Apache Hive for efficient data warehousing and querying large datasets.- Leverage Apache Spark for large-scale data processing and analytics.- Collaborate directly with clients to understand their business...


  • india Gama Advisory Services LLP Full time

    Overview :We are seeking a proficient Big Data/ETL Test Engineer with 4-6 years of experience in testing Big Data platforms and ETL processes, specifically using Hadoop and Hive technologies. As a Test Engineer, you will play a critical role in ensuring the quality and reliability of our Big Data solutions by designing and executing comprehensive test...


  • India Moback Technologies India Pvt Ltd Full time

    Job Description : We are seeking a talented Integration Engineer to join our team. As an Integration Engineer at Moback, you will play a crucial role in designing, developing, and maintaining robust integration solutions for our clients. You will work closely with cross-functional teams to ensure seamless integration of various systems and applications.Key...


  • india Digihelic Solutions Pvt. Ltd. Full time

    Key Responsibilities :- Designing, developing, and implementing solutions using MarkLogic, XQuery, Optic query, and Semantic Technologies to address complex data requirements.- Hands-on experience with the datahub framework for managing data ingestion, curation, and mastering processes.- Utilizing Apache NiFi for data integration, streaming, and processing...


  • India Digihelic Solutions Pvt. Ltd. Full time

    Key Responsibilities : Designing, developing, and implementing solutions using MarkLogic, XQuery, Optic query, and Semantic Technologies to address complex data requirements. Handson experience with the datahub framework for managing data ingestion, curation, and mastering processes. Utilizing Apache NiFi for data integration, streaming, and processing...


  • india INTELLIGENCE INDIA.COM LIMITED Full time

    Job Description :Role : Senior Data Engineer (Flink + Spark + Kafka)- Minimum 7 years of experience- B.Sc. Of Computer Science or similar- Proficiency in programming languages Java, Python or R- Experience with design and development of ELT workflows and data pipelines for high-volume data systems.- Experience in Data Engineering and implementation of...


  • Anywhere in India/Multiple Locations Moback Technologies India Pvt Ltd Full time

    Job Description : We are seeking a talented Integration Engineer to join our team. As an Integration Engineer at Moback, you will play a crucial role in designing, developing, and maintaining robust integration solutions for our clients. You will work closely with cross-functional teams to ensure seamless integration of various systems and applications.Key...


  • india Encora Inc. Full time

    4+ years of developing software using object-oriented or functional language experience4+ years of SQL2-3 + years working with open source Big Data technology stacks (Apache Nifi, Spark, Kafka, HBase, Hadoop/HDFS, Hive, Drill, Pig, etc.) or commercial open source Big Data technology stacks (Hortonworks, Cloudera, etc.)2+ years of distributed version control...


  • India Encora Inc. Full time

    4+ years of developing software using object-oriented or functional language experience4+ years of SQL2-3 + years working with open source Big Data technology stacks (Apache Nifi, Spark, Kafka, HBase, Hadoop/HDFS, Hive, Drill, Pig, etc.) or commercial open source Big Data technology stacks (Hortonworks, Cloudera, etc.)2+ years of distributed version control...

  • Data Engineer

    6 days ago


    india Recro Full time

    Job Title: Data Engineer Location: Remote Experience: 4-6 years Responsibilities: Develop, maintain, and monitor data pipelines. Assemble large, complex data sets that meet functional/non- functional business requirements. Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources...

  • Data Engineer

    6 days ago


    India Recro Full time

    Job Title: Data EngineerLocation: RemoteExperience: 4-6 yearsResponsibilities:Develop, maintain, and monitor data pipelines.Assemble large, complex data sets that meet functional/non- functional business requirements.Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using Apache,...

  • PySpark Data Engineer

    4 weeks ago


    india Edstem Technologies Full time

    We are seeking a talented and experienced PySpark Data Engineer to join our dynamic team. The ideal candidate will have a strong background in data engineering, a passion for big data technologies, and the ability to work in a fast-paced, collaborative environment.   Key Responsibilities:Data Pipeline Development: Design, develop, and maintain scalable...

  • Data Engineer

    2 weeks ago


    India Mobile Programming LLC Full time

    Job Description : Position : Data EngineerExperience : 6+ yearsWork Mode : Work from OfficeLocation : BangaloreExperience in Nifi or Tibco is mandatory.Mandatory Skills : ETL, DevOps platform, Nifi or TibcoPlease note : This position is focused on development rather than migration. We are seeking an experienced Data Engineer to join our team. As a Data...

  • Valiance Solutions

    4 weeks ago


    Anywhere in India/Multiple Locations Valiance Analytics Pvt. Ltd. Full time

    About Us : Valiance is a global AI & Data analytics firm helping clients build cutting-edge technology solutions for digital transformation. We work with some of the marquee brands across India, US and APAC to build transformative solutions for Credit Risk, Fraud, Predictive Maintenance, Quality Inspection, Data lake, IOT analytics etc. Our team comprises...

  • GCP Data Engineer

    4 weeks ago


    Pune/Hyderabad/Anywhere in India/Multiple Locations Huquo Full time

    Job Description : Must-Have :- 5+ Years of Experience in Data Engineering and building and maintaining large-scale data pipelines.- Experience with designing and implementing a large-scale Data-Lake on Cloud Infrastructure - Strong technical expertise in Python and SQL- Extremely well-versed in Google Compute Platform including BigQuery, Cloud Storage,...

  • GCP Data Engineer

    1 month ago


    Pune/Hyderabad/Anywhere in India/Multiple Locations, IN Huquo Full time

    Job Description :Must-Have :- 5+ Years of Experience in Data Engineering and building and maintaining large-scale data pipelines.- Experience with designing and implementing a large-scale Data-Lake on Cloud Infrastructure - Strong technical expertise in Python and SQL- Extremely well-versed in Google Compute Platform including BigQuery, Cloud Storage, Cloud...

  • Senior Associate

    2 weeks ago


    India Publicis Sapient Full time

    Job Description :Role : Senior Associate Data Engineering Level 2In the role of Senior Associate L2 in Data Engineering, you'll be instrumental in translating client requirements into technical designs and implementing components for data engineering solutions. Your impact spans across data ingestion, integration, transformation, storage, computation...

  • Senior Data Engineer

    2 weeks ago


    India Freelancer HR Full time

    Job Description Responsibilities : - Understand the factories , manufacturing process , data availability and avenues for improvement- Brainstorm , together with engineering, manufacturing and quality problems that can be solved using the acquired data in the data lake platform.- Define what data is required to create a solution and work with connectivity...

  • Senior Data Engineer

    3 weeks ago


    india UNICO CONNECT PRIVATE LIMITED Full time

    Job Description :Key Responsibilities :- Understand the factories , manufacturing process , data availability and avenues for improvement- Brainstorm , together with engineering, manufacturing and quality problems that can be solved using the acquired data in the data lake platform.- Define what data is required to create a solution and work with...