Big Data Administrator

7 days ago

Bengaluru, Karnataka, India UPWARDIQ SOFTWARE SOLUTIONS PRIVATE LIMITED Full time ₹ 15,00,000 - ₹ 28,00,000 per year

Job Summary:

We are seeking a Big Data Administrator with strong expertise in Linux systems, AWS infrastructure, and Big Data technologies. This role is ideal for someone experienced in managing large-scale Hadoop ecosystems in production, with a deep understanding of observability, performance tuning, and automation using tools like Terraform or Ansible.

Key Responsibilities:

  • Manage and maintain large-scale Big Data clusters (Cloudera, Hortonworks, or AWS EMR)
  • Develop and support infrastructure as code using Terraform or Ansible
  • Administer Hadoop ecosystem components HDFS, YARN, Hive (Tez, LLAP), Presto, Spark
  • Implement and monitor observability tools like Prometheus, InfluxDB, Dynatrace, Grafana, Splunk
  • Optimize SQL performance on Hive/Spark and understand query plans
  • Automate cluster operations using Python (PySpark) or Shell scripting
  • Support Data Analysts & Scientists with tools like JupyterHub, R-Studio, H2O, SAS
  • Handle data in various formats ORC, Parquet, Avro
  • Integrate with and support Kubernetes-based environments (if applicable)
  • Collaborate across teams for deployments, monitoring, and troubleshooting

Must-Have Skills:

  • 5+ years in Linux system administration and AWS cloud infrastructure
  • Experience with Cloudera, Hortonworks, or EMR in production
  • Strong in Terraform Ansible for automation
  • Solid hands-on with HDFS, YARN, Hive, Spark, Presto
  • Proficient in Python and Shell scripting
  • Familiar with observability tools: Grafana, Prometheus, InfluxDB, Splunk, Dynatrace
  • Familiarity with Active Directory, Windows VDI platforms (Citrix, AWS Workspaces)

Nice-to-Have Skills:

  • Experience with Airflow, Oozie
  • Familiar with Pandas, Numpy, Scipy, PyTorch
  • Prior use of Jenkins, Chef, Packer
  • Comfortable reading code in Java, Scala, Python, R

Qualifications:

  • Bachelors or Masters degree in Computer Science, Information Technology, or a related field
  • Strong communication, collaboration, and troubleshooting skills
  • Ability to thrive in remote or hybrid work environments

  • Big Data Administrator

    1 week ago

    Bengaluru, Karnataka, India beBeeDataAdministrator Full time US$ 15,00,000 - US$ 24,00,000

    Big Data Administration OpportunityWe are seeking a highly skilled professional to manage our big data platform, overseeing day-to-day operations and ensuring the smooth functioning of its components.The ideal candidate will have extensive experience in managing large-scale data platforms, including Hadoop Ecosystem technologies such as HDFS, MapReduce,...

  • Big Data Administrator

    2 weeks ago

    Bengaluru, Karnataka, India beBeeHadoop Full time ₹ 8,00,000 - ₹ 12,00,000

    Hadoop Administration JobBig Data Administrators play a vital role in managing and optimizing our extensive Hadoop platforms, spanning thousands of nodes per cluster. This position involves performing comprehensive Big Data Administration and Engineering activities across multiple open-source platforms.Job Responsibilities:Conduct effective root cause...

  • Senior Data Administrator

    7 days ago

    Bengaluru, Karnataka, India beBeeBigDataAdministrator Full time ₹ 20,00,000 - ₹ 25,00,000

    About This Role: We are seeking a seasoned expert in data administration to oversee the management of our big data platform. As a key member of our team, you will be responsible for ensuring the smooth operation of our Hadoop ecosystem, including HDFS, MapReduce, Hive, HBase, and Spark.Job Responsibilities: • Administer and monitor Big Data Platform...

  • Big Data Engineer

    3 weeks ago

    Bengaluru, Karnataka, India Coders Brain Full time

    Job DescriptionRoles and Responsibilities:- Develop scalable infrastructure and platform to collect, analyze and process large amounts of structured and unstructured data with real-time data interpretation.- Work closely across an array of various teams and organizations in the company (including partners, customers)Basic Qualifications:- Bachelor's degree...

  • Big Basket

    2 weeks ago

    Bengaluru, Karnataka, India big basket Full time

    What would you be doing/ Expected from this role ?- Collaborate with cross-functional teams including data scientists, engineers, and product managers to deliver AI-driven solutions.- Drive the architecture of machine learning pipelines from data collection and preprocessing to model training, evaluation, and deployment.- Implement scalable machine learning...

  • Big Basket

    2 weeks ago

    Bengaluru, Karnataka, India big basket Full time

    As a Software Engineer we are looking for developers who share our passion for building world class software. If you are excited about the prospect of changing the way India shops for groceries and being a pioneer then this a great home for you. Not to mention the thrill of building products that millions use daily and using cutting edge technology to make...

  • Senior Data Engineer for Hadoop and Big Data Platforms

    1 week ago

    Bengaluru, Karnataka, India beBeeBigDataAdministrator Full time ₹ 36,00,000 - ₹ 48,00,000

    Job OverviewWe are seeking an experienced Big Data Administrator to manage our big data platform, encompassing Hadoop, Hive, HBase, and BigSQL. The successful candidate will oversee daily operations, troubleshoot issues, maintain security, and work on projects related to data acquisition, processing, and utilization through the big data platform.Key...

  • Big Data Specialist

    2 weeks ago

    Bengaluru, Karnataka, India beBeeData Full time ₹ 8,00,000 - ₹ 15,00,000

    Job Title: Big Data SpecialistExperience:2+ years of experience in big data development.The ideal candidate will have expertise in:Big Data DevelopmentAdvanced AnalyticsCloud ComputingProgramming Skills (Python, Java)As a big data specialist, you will be responsible for designing and developing large-scale data processing systems. Your focus will be on...

  • Big Data Developer

    1 week ago

    Bengaluru, Karnataka, India Tech Consulting Full time

    Position Overview:We are seeking a skilled Big Data Developer to join our growing delivery team, with a dual focus on hands-on project support and mentoring junior engineers. This role is ideal for a developer who not only thrives in a technical, fast-paced environment but is also passionate about coaching and developing the next generation of talent.You...

  • Big Data Developer

    2 weeks ago

    Bengaluru, Karnataka, India LION AND ELEPHANTS Full time ₹ 9,00,000 - ₹ 12,00,000 per year

    Job Description:We are looking for skilled Big Data Engineers using Java Spark with 5-10 years of experience in Big Data / legacy platforms, who can join immediately. We have openings in Bengaluru, Chennai, Hyderabad, Pune, Delhi.Desired candidates should have design, development and optimization of real-time & batch data pipelines experience in a Big Data...