Big Data Administrator

4 days ago

Bengaluru, Karnataka, India UPWARDIQ SOFTWARE SOLUTIONS PRIVATE LIMITED Full time ₹ 20,00,000 - ₹ 25,00,000 per year

Job Summary:

We are seeking a Big Data Administrator with strong expertise in Linux systems, AWS infrastructure, and Big Data technologies. This role is ideal for someone experienced in managing large-scale Hadoop ecosystems in production, with a deep understanding of observability, performance tuning, and automation using tools like Terraform or Ansible.

Key Responsibilities:

  • Manage and maintain large-scale Big Data clusters (Cloudera, Hortonworks, or AWS EMR)
  • Develop and support infrastructure as code using Terraform or Ansible
  • Administer Hadoop ecosystem components HDFS, YARN, Hive (Tez, LLAP), Presto, Spark
  • Implement and monitor observability tools like Prometheus, InfluxDB, Dynatrace, Grafana, Splunk
  • Optimize SQL performance on Hive/Spark and understand query plans
  • Automate cluster operations using Python (PySpark) or Shell scripting
  • Support Data Analysts & Scientists with tools like JupyterHub, R-Studio, H2O, SAS
  • Handle data in various formats ORC, Parquet, Avro
  • Integrate with and support Kubernetes-based environments (if applicable)
  • Collaborate across teams for deployments, monitoring, and troubleshooting

Must-Have Skills:

  • 5+ years in Linux system administration and AWS cloud infrastructure
  • Experience with Cloudera, Hortonworks, or EMR in production
  • Strong in Terraform Ansible for automation
  • Solid hands-on with HDFS, YARN, Hive, Spark, Presto
  • Proficient in Python and Shell scripting
  • Familiar with observability tools: Grafana, Prometheus, InfluxDB, Splunk, Dynatrace
  • Familiarity with Active Directory, Windows VDI platforms (Citrix, AWS Workspaces)

Nice-to-Have Skills:

  • Experience with Airflow, Oozie
  • Familiar with Pandas, Numpy, Scipy, PyTorch
  • Prior use of Jenkins, Chef, Packer
  • Comfortable reading code in Java, Scala, Python, R

Qualifications:

  • Bachelors or Masters degree in Computer Science, Information Technology, or a related field
  • Strong communication, collaboration, and troubleshooting skills
  • Ability to thrive in remote or hybrid work environments

  • Big Data Developer

    2 weeks ago

    Bengaluru, Karnataka, India LION AND ELEPHANTS Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Description:We are looking for skilled Big Data Engineers using Java Spark with 5-10 years of experience in Big Data / legacy platforms, who can join immediately. We have openings in Bengaluru, Chennai, Hyderabad, Pune, Delhi.Desired candidates should have design, development and optimization of real-time & batch data pipelines experience in a Big Data...

  • Big Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Reveille Technologies,Inc Full time ₹ 16,00,000 - ₹ 24,00,000 per year

    We're Hiring: Big Data Engineer – Full-TimeLooking for a skilledBig Data Engineerto join our team inBangaloreRole: Big Data EngineerGrade: G4 |TSRLocation: Bangalore (Onsite/Hybrid – specify if needed)Budget: ₹13,39,200Experience: 4–6 yearsSkills:Azure Databricks (ADB)PySparkSQLSpark (Scala/Python)TerraformAirflow & KubernetesData Warehousing &...

  • Big Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India The IT Firm Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Role OverviewAs a Big Data Engineer, you will be responsible for building and maintaining scalable data pipelines and architectures that support data analytics, reporting, and real-time processing. You will collaborate with data scientists, business analysts, and software engineers to design robust solutions that drive data-driven decisions.Mandatory Skill...

  • Big Data Engineer

    6 days ago

    Bengaluru, Karnataka, India Brillio Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Key skills - Hive, SQL, Python, Spark, Hadoop and kafka , apache airflowJob Title Big Data EngineerRole OverviewWe are seeking a highly skilled Big Data Engineer to join our team. The ideal candidate will have strong experience in building, maintaining, and optimizing large-scale data pipelines and distributed data processing systems. This role involves...

  • Big Data Engineer

    14 hours ago

    Bengaluru, Karnataka, India Deqode Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Profile: Big Data Engineer (System Design)Experience:5+ yearsLocation:BangaloreWork Mode:HybridAbout The RoleWe're looking for an experiencedBig Data Engineerwith system design expertise to architect and build scalable data pipelines and optimize big data solutions.Key ResponsibilitiesDesign, develop, and maintain data pipelines and ETL processes using...

  • Big Data Engineer

    1 week ago

    Bengaluru, Karnataka, India Acme Services Full time ₹ 5,00,000 - ₹ 15,00,000 per year

    We are looking for Big Data Engineer who has experience with:- 1) Big Query 2) Pyspark 3) GCP or AWS or Azure Required Candidate profileCandidate should have more than 5 Years of overall experience in Data Engineer. We are not looking for Data Analyst or Snowflake Developer.

  • Big Data Engineer

    4 days ago

    Bengaluru, Karnataka, India Deqode Full time ₹ 5,00,000 - ₹ 25,00,000 per year

    Big Data Engineer Tech Stack: Python, Hive & Apache Spark Java & Advanced SQL Hadoop & Big Data Ecosystem Apache Airflow & ETL Pipelines Docker & Containerization Data Modelling & Warehousing Experience- 5 years Share your updated resume now.

  • Big Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Fractal Analytics Private Limited Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    BigData Engineer It's fun to work in a company where people truly BELIEVE in what they are doing We're committed to bringing passion and customer focus to the business.About the Role:We are seeking highly skilled Big Data Engineers/Senior Engineers with a strong background in building and supporting big data solutions. The ideal candidate will have...

  • Big Data Developer

    4 days ago

    Bengaluru, Karnataka, India Sourcebae Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    Job Alert: Senior Big Data Engineer (5-8 Yrs) | Spark, Python, Scala Are you a seasonedBig Data Engineerready to build and optimize large-scale, distributed data processing applications? We are seeking a senior engineer with5-8 yearsof experience in the core Big Data ecosystem to deliver high-performance solutions.This is a great opportunity to join a...

  • Big Data Engineer

    4 days ago

    Bengaluru, Karnataka, India Innova ESI Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Role: Big Data EngineerExperience: 7+ YearsLocation: BangaloreNotice: Immediate Joiners OnlyJob Description:Azure Databricks, Azure Data Factory, Azure Function apps, Apache Spark, Scala, Java, Apache Kafka, event stream and Big DataOptional Skills: Airflow, PythonRoles & ResponsibilitiesOverall, 12 years of experience in IT industry, including 5 years in...