Aws Hadoop Administrator

10 hours ago

Uttar Pradesh, India Valiance Analytics Full time

We are seeking a skilled AWS Hadoop Administrator to join our team and manage our Hadoop infrastructure on the AWS platform. In this role, you will be responsible for deploying, configuring, monitoring, and maintaining our Hadoop clusters to ensure optimal performance, scalability, and reliability. As an integral member of our IT operations team, you will collaborate closely with data engineers, developers, and other stakeholders to support our big data initiatives and ensure smooth operation of our data processing workflows.skills required: Hands-on experience on installation of Hadoop ecosystem on AWS such as EMR,S3,EC2,IAM policy, MWAA/Airflow,Hadoop,yarn,spark, Mlflow. Experience on Linux enviroment and bash scriptingResponsibilities: 1. Deploy and configure Hadoop clusters on the AWS cloud platform including MWAA,MLflow and Apache-Nifi, ensuring proper integration with related AWS services such as EC2, S3, EMR, and IAM.
2. Monitor Hadoop cluster performance, health, and resource utilization using monitoring tools and AWS CloudWatch, and proactively address any issues to maintain optimal performance.
3. Manage Hadoop cluster capacity, scaling resources up or down as needed to accommodate changing workload demands.
4. Implement and enforce security best practices for Hadoop clusters on AWS, including access control, encryption, and compliance with regulatory requirements.
5. Perform regular backups and disaster recovery procedures to ensure data integrity and minimize downtime.
6. Collaborate with data engineers and developers to optimize Hadoop jobs and data processing workflows for performance and efficiency.
7. Troubleshoot and resolve issues related to Hadoop cluster operation, performance, and data processing.
8. Stay current with advancements in Hadoop technology and AWS services, and evaluate and recommend new tools and techniques to enhance our big data infrastructure.
9. Document system configurations, procedures, and best practices, and provide training and support to other team members as needed.
10. Participate in on-call rotation and respond to incidents outside of regular business hours as necessary.Qualifications: 1. Bachelor's degree in computer science, engineering, or a related field, or equivalent work experience.
2. Proven experience as a Hadoop administrator, preferably in an AWS environment.
3. Strong understanding of Hadoop architecture, components, and ecosystem technologies such as HDFS, YARN, MapReduce, Hive, Spark, and Kafka.
4. Experience deploying and managing Hadoop clusters on AWS using services such as EMR, EC2, S3, and IAM.
5. Solid understanding of cloud computing principles and best practices, particularly in AWS.
6. Proficiency in Linux operating systems and shell scripting.
7. Strong analytical and problem-solving skills, with the ability to troubleshoot complex issues and drive them to resolution.
8. Excellent communication and collaboration skills, with the ability to work effectively in a team environment.
9. AWS certification(s) such as AWS Certified Solutions Architect or AWS Certified SysOps Administrator is a plus.
10. Experience with containerization technologies such as Docker and orchestration tools like Kubernetes is desirable.

  • Big Data/Hadoop Administrator

    3 days ago

    madhya pradesh, India ClearTrail Technologies Full time

    At ClearTrail, work is more than ‘just a job’. Our calling is to develop solutions that empower those dedicated to keeping their people, places and communities safe. For over 23 years, law enforcement & federal agencies across the globe have trusted ClearTrail as their committed partner in safeguarding nations & enriching lives. We are envisioning the...

  • Hadoop Architect

    2 weeks ago

    Andhra Pradesh, India Virtusa Full time

    P1-C3-TSTS Incident manager (Hadoop & Netezza) Key Responsibilities: Manage and lead a team of Hadoop support engineers to provide 24/7 production support. Monitor, troubleshoot, and resolve Hadoop cluster issues related to performance, availability, and scalability. Ensure high availability of Hadoop clusters through proactive capacity planning, tuning,...

  • Hadoop and Big Data Administrator

    6 days ago

    madhya pradesh, India ClearTrail Technologies Full time

    What We Do At ClearTrail, work is more than ‘just a job’. Our calling is to develop solutions that empower those dedicated to keep their people, places and communities safe. For over 24 years, law enforcement & federal agencies across the globe have trusted ClearTrail as their committed partner in safeguarding nations & enriching lives. We are...

  • Hadoop and Big Data Administrator

    1 week ago

    madhya pradesh, India ClearTrail Technologies Full time

    What We DoAt ClearTrail, work is more than 'just a job'. Our calling is to develop solutions that empower those dedicated to keep their people, places and communities safe.For over 24 years, law enforcement & federal agencies across the globe have trusted ClearTrail as their committed partner in safeguarding nations & enriching lives. We are envisioning the...

  • Senior Database Administrator

    5 days ago

    uttar pradesh, India Times Internet Full time

    Job Title: MySql Database Admin EXPERIENCE: 6-12 Years Work Location: Sector 16, Noida Notice Period: Immediate/Serving only About Times Internet: Times Internet is the largest digital products company and the digital venture of Times of India, India's largest media and entertainment group. TIL websites are among the fastest growing Web / Mobile based...

  • Senior Database Administrator

    6 days ago

    uttar pradesh, India Times Internet Full time

    Job Title: MySql Database AdminEXPERIENCE: 6-12 YearsWork Location: Sector 16, NoidaNotice Period: Immediate/Serving onlyAbout Times Internet:Times Internet is the largest digital products company and the digital venture of Times of India, India's largest media and entertainment group. TIL websites are among the fastest growing Web / Mobile based networks...

  • AWS Tester

    2 weeks ago

    Andhra Pradesh, India Virtusa Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Role : AWS TesterWe are looking for a highly skilled Automation QA Engineer with hands-on experience in AWS and Big Data ecosystems to join our Quality Engineering team. The ideal candidate will be responsible for developing automated test frameworks and validating large-scale data processing pipelines in cloud-based environments. This role requires strong...

  • Big Data Systems Administrator

    1 week ago

    madhya pradesh, India beBeeBigDataAdministrator Full time

    About the RoleWe are seeking a seasoned data professional to lead our big data initiatives. As a Hadoop Administrator, you will be responsible for designing, deploying and maintaining large-scale data processing systems.This is an exciting opportunity to work with cutting-edge technologies such as Hortonworks, Cloudera, Apache Hadoop/Spark ecosystem. You...

  • Big Data Engineer

    1 week ago

    uttar pradesh, India Umanist Staffing Full time

    Job Summary:We are hiring an experienced Big Data Engineer to design, build, and maintain high-performance data processing systems. The role requires strong expertise in Hadoop ecosystem, Spark, and cloud-based data pipelines with a focus on scalability, reliability, and performance.Key Responsibilities:Design and develop end-to-end big data pipelines for...

  • AWS Infra Devops Consultant

    6 days ago

    uttar pradesh, India Sopra Steria Full time

    Position: AWS Infra Devops ConsultantLocation: Noida / ChennaiExperience: 9 to 12 yearsEducation: B.E./ B.Tech./ MCAPrimary Skills: Terraform, Kubernetes, AWS Certification: AWS Solution Arrchitect CertifiedJob DescriptionStrong exoerience in AWS Platform (Infrastructure, Services, Administration, Provisioning, Monitoring…)Managing Kubernetes...