Big Data Administrator

5 days ago

Bengaluru, Karnataka, India UPWARDIQ SOFTWARE SOLUTIONS PRIVATE LIMITED Full time ₹ 20,00,000 - ₹ 25,00,000 per year

Job Summary:

We are seeking a Big Data Administrator with strong expertise in Linux systems, AWS infrastructure, and Big Data technologies. This role is ideal for someone experienced in managing large-scale Hadoop ecosystems in production, with a deep understanding of observability, performance tuning, and automation using tools like Terraform or Ansible.

Key Responsibilities:

  • Manage and maintain large-scale Big Data clusters (Cloudera, Hortonworks, or AWS EMR)
  • Develop and support infrastructure as code using Terraform or Ansible
  • Administer Hadoop ecosystem components HDFS, YARN, Hive (Tez, LLAP), Presto, Spark
  • Implement and monitor observability tools like Prometheus, InfluxDB, Dynatrace, Grafana, Splunk
  • Optimize SQL performance on Hive/Spark and understand query plans
  • Automate cluster operations using Python (PySpark) or Shell scripting
  • Support Data Analysts & Scientists with tools like JupyterHub, R-Studio, H2O, SAS
  • Handle data in various formats ORC, Parquet, Avro
  • Integrate with and support Kubernetes-based environments (if applicable)
  • Collaborate across teams for deployments, monitoring, and troubleshooting

Must-Have Skills:

  • 5+ years in Linux system administration and AWS cloud infrastructure
  • Experience with Cloudera, Hortonworks, or EMR in production
  • Strong in Terraform Ansible for automation
  • Solid hands-on with HDFS, YARN, Hive, Spark, Presto
  • Proficient in Python and Shell scripting
  • Familiar with observability tools: Grafana, Prometheus, InfluxDB, Splunk, Dynatrace
  • Familiarity with Active Directory, Windows VDI platforms (Citrix, AWS Workspaces)

Nice-to-Have Skills:

  • Experience with Airflow, Oozie
  • Familiar with Pandas, Numpy, Scipy, PyTorch
  • Prior use of Jenkins, Chef, Packer
  • Comfortable reading code in Java, Scala, Python, R

Qualifications:

  • Bachelors or Masters degree in Computer Science, Information Technology, or a related field
  • Strong communication, collaboration, and troubleshooting skills
  • Ability to thrive in remote or hybrid work environments

  • Big Data Developer

    2 weeks ago

    Bengaluru, Karnataka, India LION AND ELEPHANTS Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    Job Description:We are looking for skilled Big Data Engineers using Java Spark with 5-10 years of experience in Big Data / legacy platforms, who can join immediately. We have openings in Bengaluru, Chennai, Hyderabad, Pune, Delhi.Desired candidates should have design, development and optimization of real-time & batch data pipelines experience in a Big Data...

  • Big Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Reveille Technologies,Inc Full time ₹ 16,00,000 - ₹ 24,00,000 per year

    We're Hiring: Big Data Engineer – Full-TimeLooking for a skilledBig Data Engineerto join our team inBangaloreRole: Big Data EngineerGrade: G4 |TSRLocation: Bangalore (Onsite/Hybrid – specify if needed)Budget: ₹13,39,200Experience: 4–6 yearsSkills:Azure Databricks (ADB)PySparkSQLSpark (Scala/Python)TerraformAirflow & KubernetesData Warehousing &...

  • Big Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India The IT Firm Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Role OverviewAs a Big Data Engineer, you will be responsible for building and maintaining scalable data pipelines and architectures that support data analytics, reporting, and real-time processing. You will collaborate with data scientists, business analysts, and software engineers to design robust solutions that drive data-driven decisions.Mandatory Skill...

  • Big Data Engineer

    17 hours ago

    Bengaluru, Karnataka, India Deqode Full time ₹ 12,00,000 - ₹ 24,00,000 per year

    Profile: Big Data Engineer (System Design)Experience:5+ yearsLocation:BangaloreWork Mode:HybridAbout The RoleWe're looking for an experiencedBig Data Engineerwith system design expertise to architect and build scalable data pipelines and optimize big data solutions.Key ResponsibilitiesDesign, develop, and maintain data pipelines and ETL processes using...

  • Big Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Fractal Analytics Private Limited Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    BigData Engineer It's fun to work in a company where people truly BELIEVE in what they are doing We're committed to bringing passion and customer focus to the business.About the Role:We are seeking highly skilled Big Data Engineers/Senior Engineers with a strong background in building and supporting big data solutions. The ideal candidate will have...

  • Big Data Engineer

    5 days ago

    Bengaluru, Karnataka, India Innova ESI Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Role: Big Data EngineerExperience: 7+ YearsLocation: BangaloreNotice: Immediate Joiners OnlyJob Description:Azure Databricks, Azure Data Factory, Azure Function apps, Apache Spark, Scala, Java, Apache Kafka, event stream and Big DataOptional Skills: Airflow, PythonRoles & ResponsibilitiesOverall, 12 years of experience in IT industry, including 5 years in...

  • Big Data Engineer

    2 weeks ago

    Bengaluru, Karnataka, India Tata Consultancy Services Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Greetings from TCS TCS is hiring for Big Data (PySpark & Scala)Location: - Pune/Mumbai/Chennai/BangaloreDesired Experience Range: 4 - 6 YearsMust-HavePySpark HiveGood-to-Have Spark HBase DQ tool Agile Scrum experience Exposure in data ingestion from disparate sources onto Big Data platformThanksAnshika

  • Big Data Developer

    2 weeks ago

    Bengaluru, Karnataka, India ITC Infotech Full time ₹ 5,00,000 - ₹ 15,00,000 per year

    Required Skills:Demonstrated ability in designing, building, refactoring and releasing software written in Java, Python.Strong experience with Big Data technologies including Hadoop, Spark, Flink, Hive, HDFS.Ability to handle data integration, bug fixing, performance optimization, and feature enhancements for distributed data pipelines.Having experience with...

  • Big Data Engineer

    3 days ago

    Bengaluru, Karnataka, India Suzva Software Technologies Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Hiring: Big Data Engineer (PySpark)Client: [Details will be shared upon shortlisting]Location: Bangalore (Onsite)Experience: 6-8 years (Relevant)Joining: Immediate to 15 days Key Skills: Strong hands-on experience in PySpark Expertise in Big Data technologies Solid understanding of SQL, Data Pipelines, and ETLInterested candidates can share their resume...

  • Big Data Developer

    2 weeks ago

    Bengaluru, Karnataka, India Grid Dynamics Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    7 to 15 Years of experience in Big Data Developer (Mandatory Splunk to ClickHouse and Grafana)Essential FunctionsKey Responsibilities7 to 15 Years of experience in Big Data DeveloperDesign and develop scalable data pipelines to migrate user knowledge objects from Splunk to ClickHouse and Grafana.Implement data ingestion, transformation, and validation...