BigData and Hadoop Ecosystems

6 days ago

Bengaluru Hyderabad Pune, India Tekskills Full time ₹ 15,00,000 - ₹ 25,00,000 per year
  • Job Description:

Role descriptions / Expectations from the Role

Experience in other messaging systems such as IBM MQ or Rabbit MQ.

Experience in the Atlassian stack tools Jira, Confluence and Bitbucket (GIT).

Experience in Jenkins automation development.

Experience in the scripting languages shell and python.

Experience in the operating systems Linux (RedHat and/or SuSE) and Windows Server.

Experience in search engines such as Elasticsearch.

Job description: must have

• Cluster Management:

Install, configure, and maintain Confluent and AWS MSK Kafka clusters.

Manage large-scale, multi-node Kafka cluster environments (on-prem and cloud).

Troubleshoot and resolve issues related to Kafka brokers, topics, and data flow.

Optimize Kafka configurations for performance, scalability, and fault tolerance.

Plan and execute Kafka version upgrades and apply patches.

Develop and maintain disaster recovery plans.

Create and maintain documentation.

  • Security:

    Implement and manage security protocols.
  • Performance and Monitoring:

    Monitor Kafka clusters to ensure optimal performance and availability.

    Collect and maintain performance metrics.

    Fine-tune Kafka configurations based on performance data.
  • Collaboration:

    Provide guidance and support to developers using Kafka.

    • Automation:

    Automate Kafka Cluster installations and topics/acls/connectors deployments using tools like Ansible, terraform, gitops.

    Develop and maintain automation for Kafka infrastructure.

Skills and Experience:

  • Kafka Administration

    Strong understanding of Kafka architecture, concepts, and best practices.

    Proven experience in administering Confluent platform Brokers, Zookeepers, Schema registry, Restproxy, Ksql, Connectors and Control Center, Topics, ACLs etc.,

    Experience in administering and supporting AWS MSK Clusters.

    Experience with Kafka AdminClient for managing topics programmatically.
  • Technical Skills:

    Experience with monitoring and alerting tools like Prometheus, Grafana, Alertmanager, Dynatrace etc.,

    Experience with Linux/Unix operating systems.

    Knowledge of software containerization and Kubernetes.

    Experience with scripting languages (Python, Shell).

    Automate installations and deployments using tools like Ansible, gitops, terraform etc.,

    Experience with GitLab pipeline.

    Knowledge of Service-Now, JIRA software.
  • Good to have:

    Strong troubleshooting skills.

    Excellent communication and collaboration skills.

    Ability to work as part of a team and independently.

    Experience in providing enhanced support to application teams.

  • Bigdata and Hadoop Engineer

    2 weeks ago

    Bengaluru, Chennai, Hyderabad, India Artech Full time ₹ 8,00,000 - ₹ 12,00,000 per year

    Job Title: Developer Experience: 4-6 Years Work Location: Chennai, TN || Bangalore, KA || Hyderabad, TSSkill Required: Digital : Bigdata and Hadoop Ecosystems Digital : PySparkJob Description:Need to work as a developer in Bigdata, Hadoop or Data Warehousing Tools and Cloud Computing ? Work on Hadoop, Hive SQL?s, Spark, Bigdata Eco System Tools.? Experience...

  • Bigdata Hadoop developer

    1 week ago

    Hyderabad, Telangana, India Tata Consultancy Services Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    TCS Hiring Role: Bigdata Hadoop developerExp: 8+ YearsLocation: Hyd/KolJD8+ Years of experience4+ year hands-on experience in Hadoop ecosystems – Python, HDFS, Hive, Sqoop, Nifi, Airflow etcExperienced in Working in DWH environmentStrong in SQL, Shell scripting, Database Experience.Requirements gathering, analysis, documentationAbility to learn on the job...

  • Bigdata Hadoop Platform

    4 weeks ago

    Hyderabad, India Tata Consultancy Services Full time

    Greetings from TCS!!!TCS is hiring for Bigdata Hadoop Platform -Cloudera .Job DescriptionRole: Bigdata Hadoop Platform -ClouderaDesired Experience Range** 4 - 10Location -HyderabadRequired Technical Skill SetBig Data, Hadoop Administration, Cloudera CDH Administration, Cloudera Manager Admin Console, Sqoop, Hive, Impala, YARN, HDFS, CDSW, Kerberos, Sentry,...

  • Bigdata Hadoop Platform

    4 weeks ago

    Hyderabad, India Tata Consultancy Services Full time

    Greetings from TCS!!! TCS is hiring for Bigdata Hadoop Platform -Cloudera . Job Description Role: Bigdata Hadoop Platform -Cloudera Desired Experience Range** 4 - 10 Location -Hyderabad Required Technical Skill Set Big Data, Hadoop Administration, Cloudera CDH Administration, Cloudera Manager Admin Console, Sqoop, Hive, Impala, YARN, HDFS, CDSW, Kerberos,...

  • Bigdata & Hadoop

    4 weeks ago

    Hyderabad, India Tata Consultancy Services Full time

    Role : Bigdata & Hadoop Location : Hyderabad Required Technical Skill Set** Hadoop, Python, PySpark, HIVE Must-Have** · Hands-on experience of Hadoop, Python, PySpark, Hive, Big Data Eco System Tools. · Should be able to develop, tweak queries and work on performance enhancement. · Solid understanding of object-oriented programming and HDFS concepts ·...

  • Bigdata & Hadoop

    4 weeks ago

    Hyderabad, India Tata Consultancy Services Full time

    Role : Bigdata & Hadoop Location : Hyderabad Required Technical Skill Set** Hadoop, Python, PySpark, HIVE Must-Have** · Hands-on experience of Hadoop, Python, PySpark, Hive, Big Data Eco System Tools. · Should be able to develop, tweak queries and work on performance enhancement. · Solid understanding of object-oriented programming and HDFS concepts ·...

  • Bigdata & Hadoop

    4 weeks ago

    Hyderabad, India Tata Consultancy Services Full time

    Role : Bigdata & HadoopLocation : HyderabadRequired Technical Skill Set** Hadoop, Python, PySpark, HIVEMust-Have** · Hands-on experience of Hadoop, Python, PySpark, Hive, Big Data Eco System Tools. · Should be able to develop, tweak queries and work on performance enhancement. · Solid understanding of object-oriented programming and HDFS concepts · The...

  • Bigdata & Hadoop

    4 weeks ago

    Hyderabad, India Tata Consultancy Services Full time

    Role : Bigdata & HadoopLocation : HyderabadRequired Technical Skill Set** Hadoop, Python, PySpark, HIVEMust-Have** · Hands-on experience of Hadoop, Python, PySpark, Hive, Big Data Eco System Tools. · Should be able to develop, tweak queries and work on performance enhancement. · Solid understanding of object-oriented programming and HDFS concepts · The...

  • Bigdata Hadoop Platform

    4 weeks ago

    Hyderabad, India Tata Consultancy Services Full time

    Greetings from TCS!!!TCS is hiring for Bigdata Hadoop Platform -Cloudera .Job DescriptionRole: Bigdata Hadoop Platform -ClouderaDesired Experience Range** 4 - 10Location -HyderabadRequired Technical Skill SetBig Data, Hadoop Administration, Cloudera CDH Administration, Cloudera Manager Admin Console, Sqoop, Hive, Impala, YARN, HDFS, CDSW, Kerberos, Sentry,...

  • Bigdata Hadoop Platform

    4 weeks ago

    Hyderabad, India Tata Consultancy Services Full time

    Greetings from TCS!!! TCS is hiring for Bigdata Hadoop Platform -Cloudera . Job Description Role: Bigdata Hadoop Platform -Cloudera Desired Experience Range** 4 - 10 Location -Hyderabad Required Technical Skill Set Big Data, Hadoop Administration, Cloudera CDH Administration, Cloudera Manager Admin Console, Sqoop, Hive, Impala, YARN, HDFS, CDSW,...