Hadoop Administrator

**Key Responsibilities**:

- Hadoop Ecosystem Management: Administer, configure, and optimize Big Data platforms, including Hortonworks, Cloudera, and MapR, in both secured and unsecured environments.
- Security & Compliance: Implement and manage security technologies such as Kerberos, Knox, Ranger, KMS, SSL certificates, and encryption zones.
- Cluster Administration: Perform capacity planning for HDFS file systems and YARN resources; ensure high availability for core Hadoop services.
- Performance Optimization: Fine-tune Java heap settings and allied parameters to maintain optimal performance for Hadoop services.
- Troubleshooting: Analyze logs to identify issues, collaborate with product vendors for root cause analysis, and propose optimization solutions.
- Product Upgrades & Maintenance: Conduct product upgrades, cluster expansions, and ongoing maintenance of the Big Data environment.
- Data Ingestion & Management: Manage data pipeline tools like Sqoop, Distcp, and file ingestion workflows; recommend appropriate file formats and compression techniques.
- Cloud Integration: Work with AWS and GCP services relevant to Big Data, ensuring seamless integration and administration.
- Disaster Recovery: Develop and implement disaster recovery solutions using platform-native tools or custom scripts.
- Scripting & Automation: Create and maintain shell scripts, Python, or Perl scripts for automation and administration tasks.
- Version Control & DevOps: Use tools like Git, GitHub, Jenkins, Bamboo, and Ansible for deployment and testing within Agile/DevOps environments.
- Database Administration: Administer open-source RDBMS like MySQL, PostgreSQL, and MariaDB, particularly for services like Ambari and Ranger.
- Service Operations: Align with ITIL concepts to manage service operations activities efficiently in a critical infrastructure environment.

**Required Skills and Qualifications**:

- Technical Expertise:

- Strong experience in Hadoop ecosystem administration with distributions like Hortonworks, Cloudera, and MapR.
- Deep knowledge of Hive, HBase, Kafka, Spark, and related Hadoop ecosystem tools.
- Experience with Hadoop security and compliance technologies (Kerberos, Knox, Ranger, etc.).
- Proficiency in Linux system administration and shell scripting (Python/Perl is a plus).
- Familiarity with disaster recovery, capacity planning, and high-availability setup.
- Working knowledge of RDBMS (MySQL, PostgreSQL) and integration with Hadoop components.
- Understanding of Agile/DevOps practices and experience with CI/CD tools like Ansible and Jenkins.
- Knowledge of cloud platforms (AWS/GCP) and their Big Data services.
- Soft Skills:

- Strong troubleshooting and problem-solving abilities.
- Effective communication and interpersonal skills.
- Ability to engage with stakeholders to drive effective decisions.
- Excellent time management and team collaboration skills.
- Certifications (preferred):

- Certifications in Hadoop, Cloudera, or related Big Data technologies.
- AWS or GCP certifications relevant to Big Data services.
- ITIL Foundation certification.

**Experience**:

- 5+ years of experience in Hadoop administration or related Big Data platform roles.
- Hands-on experience in Agile/DevOps-based projects.

**Work Environment**:

- Ability to work in an infrastructure-critical environment.
- Comfortable collaborating across cultures and distributed teams.

Pay: ₹600,000.00 - ₹1,100,000.00 per year

**Benefits**:

- Provident Fund

Schedule:

- Day shift

Supplemental Pay:

- Performance bonus

**Experience**:

- total work: 6 years (preferred)
- Hadoop: 5 years (preferred)

Work Location: In person