NIFI Support Engineer

1 week ago

Chennai, Tamil Nadu, India Digital Minds Full time US$ 90,000 - US$ 1,20,000 per year

Roles & Responsibilities:

  • Implemented solutions utilizing Advanced AWS Components: EMR, EC2, etc integrated with Big Data/Hadoop Distribution Frameworks: Zookeeper, Yarn, Spark, Scala, NiFi etc.
  • Designed and Implemented Spark Jobs to be deployed and run on existing Active clusters.
  • Configured Postgres Database on EC2 instances and made sure application that was created is up and running, Trouble Shooted issues to meet the desired application state.
  • Experience in creating and configuring secure VPC, Subnets, and Security Groups through private and public networks.
  • Created alarms, alerts, notifications for Spark Jobs to email and slack group message job status and log in CloudWatch.
  • NiFi data Pipeline to process large set of data and configured Lookups for Data Validation and Integrity.
  • generation large set of test data with data integrity using java which used in Development and QA Phase.
  • Spark Scala, improving the performance and optimized of the existing applications running on EMR cluster.
  • Spark Job to Convert CSV data to Custom HL7/FHIR objects using FHIR API's.
  • Deployed SNS, SQS, Lambda function, IAM Roles, Custom Policies, EMR with Spark and Hadoop setup and bootstrap scripts to setup additional software's needed to perform the job in QA and Production Environment using Terraform Scripts.
  • Spark Job to perform Change Data Capture (CDC) on Postgres Tables and updated target tables using JDBC properties.
  • Kafka Publisher integrated in spark job to capture errors from Spark Application and push into Postgres table.
  • extensively on building Nifi data pipelines in docker container environment in development phase.
  • Devops team to Clusterize NIFI Pipeline on EC2 nodes integrated with Spark, Kafka, Postgres running on other instances using SSL handshakes in QA and Production Environments.

  • Big Data Specialist

    1 week ago

    Chennai, Tamil Nadu, India beBeeDataEngineer Full time ₹ 15,00,000 - ₹ 20,00,000

    Job Title:A highly skilled Data Engineer with expertise in designing, developing, and maintaining big data applications using Apache Spark with Scala.Key ResponsibilitiesDesign and develop efficient data processing workflows using Apache Oozie to optimize data pipelines for performance, scalability, and reliability.Implement and monitor data flows using...

  • High-Performance Data Architect

    2 weeks ago

    Chennai, Tamil Nadu, India beBeeDataEngineer Full time US$ 18,00,000 - US$ 22,00,000

    Big Data Engineer Job DescriptionWe are seeking a highly skilled data engineer to join our team. The ideal candidate will have strong expertise in Scala, Spark, Python, NiFi, and Apache Kafka.Key Responsibilities:Design, develop, and optimize large-scale data pipelines and distributed data processing systems.Work extensively with Scala, Spark (PySpark), and...

  • Data Engineer

    2 weeks ago

    Chennai, Tamil Nadu, India Seven Consultancy Full time

    JOB DETAILS 1.Experience in Big Data technologies and working exposure in at least one cloud platform on related data services (AWS / Azure / GCP)2.Hands-on experience with the Hadoop stack – HDFS, sqoop, kafka, Pulsar, NiFi, Spark, Spark Streaming, Flink, Storm, hive, oozie, airflow and other components required in building end to end data...

  • Senior Process Engineer – Medical Devices

    2 weeks ago

    Chennai, Tamil Nadu, India Tek Support Full time

    Job Title: Senior Process Engineer – Medical DevicesLocation: Chennai, INDEmployment type: Part-timeJob Summary:We are seeking a highly experienced Process Engineer Trainer with over 10 years of expertise in the medical device industry. The ideal candidate will be responsible for developing, optimizing, and sustaining robust manufacturing processes that...

  • Senior Validation Engineer – Male and Female

    1 week ago

    Chennai, Tamil Nadu, India Tek Support Full time

    Job Title: Senior Validation Engineer Trainer – CSV, CQV & CleaningLocation: RemotePosition Type: Part-timeJob Summary:We are seeking an experienced Senior Validation Engineer Trainer with a decade of proven expertise in Cleaning Validation, Computer System Validation (CSV), and CQV (Commissioning, Qualification, and Validation) within the pharmaceutical...

  • Scalable Data Engineer

    5 days ago

    Chennai, Tamil Nadu, India beBeeData Full time ₹ 18,00,000 - ₹ 24,00,000

    Job Title: Scalable Data EngineerWe are seeking a highly skilled Big Data Engineer with deep expertise in designing and implementing scalable data engineering solutions. The ideal candidate will bring strong hands-on experience in architecting and optimizing Big Data architectures using AWS services.Key Responsibilities:Design, develop, and optimize Big Data...

  • Desktop Support Engineer

    4 days ago

    Chennai, Tamil Nadu, India TRDFIN Support Services Pvt Ltd Full time

    Key Responsibilities: • Operating System Support: Installation and configuration of operating systems across devices. • Hardware Support: Assembling, upgrading, and troubleshooting of computer hardware. • Software Support: Installation, configuration, backup, and troubleshooting of standard applications including MS Outlook. • Network Support:...

  • Senior Process Engineer – Medical Devices

    1 week ago

    Chennai, Tamil Nadu, India Tek Support Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Job Title: Senior Process Engineer – Medical DevicesLocation:Chennai, INDEmployment type:Part-timeJob Summary:We are seeking a highly experiencedProcess Engineer Trainerwith over 10 years of expertise in the medical device industry. The ideal candidate will beresponsible for developing, optimizing, and sustaining robust manufacturing processes that ensure...

  • Senior Process Engineer – Medical Devices

    2 weeks ago

    Chennai, Tamil Nadu, India Tek Support Full time

    Job Title: Senior Process Engineer – Medical Devices Location: Chennai, IND Employment type: Part-time Job Summary: We are seeking a highly experienced Process Engineer Trainer with over 10 years of expertise in the medical device industry. The ideal candidate will be responsible for developing, optimizing, and sustaining robust manufacturing...

  • Senior Validation Engineer – Male and Female

    1 week ago

    Chennai, Tamil Nadu, India Tek Support Full time ₹ 1,04,000 - ₹ 1,30,878 per year

    Job Title:Senior Validation Engineer – CSV, CQV & CleaningLocation:RemotePosition Type:Part-timeJob Summary:We are seeking an experiencedSenior Validation Engineerwith a decade of proven expertise inCleaning Validation,Computer System Validation (CSV), andCQV (Commissioning, Qualification, and Validation)within thepharmaceutical and medical device...