Hadoop Administrator

5 days ago

Dindigul, India Smartavya Analytica Private Limited Full time

We are seeking an experienced Hadoop Administrator to manage and support our Hadoop ecosystem with 5+ Years of Experience. The ideal candidate will have strong expertise in Hadoop cluster administration, excellent troubleshooting skills, and a proven track record of maintaining and optimizing Hadoop environments.


Location: Mumbai


Key Responsibilities:

  • Install, configure, and manage Hadoop clusters, including HDFS, YARN, Hive, HBase, and other ecosystem components.
  • Monitor and manage Hadoop cluster performance, capacity, and security.
  • Perform routine maintenance tasks such as upgrades, patching, and backups.
  • Implement and maintain data ingestion processes using tools like Sqoop, Flume, and Kafka.
  • Ensure high availability and disaster recovery of Hadoop clusters.
  • Collaborate with development teams to understand requirements and provide appropriate Hadoop solutions.
  • Troubleshoot and resolve issues related to the Hadoop ecosystem.

Maintain documentation of Hadoop environment configurations, processes, and procedures.


Requirement:

  • Experience in Installing, configuring and tuning Hadoop distributions.
  • Hands on experience in Cloudera.
  • Understanding of Hadoop design principals and factors that affect distributed system performance, including hardware and network considerations
  • Provide Infrastructure Recommendations, Capacity Planning, work load management.
  • Develop utilities to monitor cluster better Ganglia, Nagios etc.
  • Manage large clusters with huge volumes of data
  • Perform Cluster maintenance tasks
  • Create and removal of nodes, cluster monitoring and troubleshooting
  • Manage and review Hadoop log files
  • Install and implement security for Hadoop clusters
  • Install Hadoop Updates, patches and version upgrades. Automate the same through scripts
  • Point of Contact for Vendor escalation. Work with Hortonworks in resolving issues
  • Should have Conceptual/working knowledge of basic data management concepts like ETL, Ref/Master data, Data quality, RDBMS
  • Working knowledge of any scripting language like Shell, Python, Perl
  • Should have experience in Orchestration & Deployment tools
  • Cloudera and CDP Certifications mandatory.