Big Data Operations Engineer

2 days ago

Hyderabad, Telangana, India S M Software Solutions Inc Full time ₹ 15,00,000 - ₹ 25,00,000 per year

Job Description – Big Data Operations Engineer
Location :
Onsite, Hyderabad Softility

Experience
: 5+ years

Role Overview
We are looking for a skilled
Big Data Operations Engineer
with hands-on experience in supporting and maintaining large-scale data platforms. The role involves ensuring platform availability, managing incidents, monitoring performance through dashboards, and coordinating with development teams for enhancements. While the primary focus is on operations and support, the candidate should also have exposure to Big Data development concepts and tools.

Key Responsibilities

  • Monitor and maintain Hadoop clusters, HDFS storage, and YARN resource management.
  • Troubleshoot and resolve production issues, ensuring minimal downtime and quick recovery.
  • Support Informatica workflows in production, including error handling and performance tuning.
  • Manage Kafka topics, producers/consumers, and cluster health monitoring.
  • Work with Anthos for hybrid cloud platform management and operational tasks.
  • Support observability and log pipelines using Cribl.
  • Build and maintain dashboards for monitoring and reporting using tools like New Relic.
  • Perform incident management, root cause analysis, and preventive action planning.
  • Collaborate with development teams to roll out new jobs, updates, and fixes.
  • Maintain operational documentation, runbooks, and knowledge repositories.

Required Skills & Experience

  • 5-6 years of experience in Big Data operations/support.
  • Strong knowledge of the Hadoop ecosystem, including HDFS, YARN, Hive, and Oozie.
  • Experience supporting ETL workflows (preferably Informatica).
  • Familiarity with Kafka operations and troubleshooting.
  • Exposure to Anthos and Cribl for platform and pipeline monitoring.
  • Experience with dashboard/monitoring tools (Grafana, Kibana, or equivalent).
  • Hands-on with Linux/Unix administration, shell scripting, and automation.
  • Solid experience in incident, change, and problem management.
  • Good communication and coordination skills for working with cross-functional teams.

Nice-to-Have

  • Exposure to Big Data development (Hive, Spark, Python, or SQL).
  • Knowledge of cloud platforms (GCP/AWS/Azure).
  • Familiarity with Cloudera Manager or other cluster monitoring tools.

  • Big Data Engineer

    1 week ago

    Hyderabad, Telangana, India Sky Systems, Inc. (SkySys) Full time

    Role:Big Data EngineerPosition Type:Full-Time Contract (40hrs/week)Contract Duration:Long TermWork Schedule: 8 hours/day (Mon-Fri)Location: Hybrid - Hyderabad, India (3x days onsite/week)We are looking for a skilledBig Data Engineer (5+ years' experience)to join Experian's dynamic data engineering team. You will design and optimize scalable data pipelines,...

  • Big Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India IDESLABS PRIVATE LIMITED Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    We are looking for a skilled Big Data Engineer with 6 to 22 years of experience. The ideal candidate will have expertise in Pyspark, Azure Data Bricks, and workflows. This position is available as a contract role across Pan India.Roles and ResponsibilityDesign and develop scalable big data systems using Pyspark and Azure Data Bricks.Implement and manage...

  • Big Data Engineer

    1 week ago

    Hyderabad, Telangana, India Harp Technologies Services Full time ₹ 6,00,000 - ₹ 18,00,000 per year

    Job Title: Bigdata + PysparkLocation: HyderabadExperience Level: 5-8 YearsNotice Period: Immediate to 30 DaysInterview Mode: L1 Virtual / L2 F2F (Mandatory)Work Mode: HybridJob SummaryWe are seeking a Specialist with 5 to 8 years of experience in the Big Data Hadoop Ecosystem to design and implement scalable data solutions.Detailed Job DescriptionWork...

  • Big Data Engineer

    4 days ago

    Hyderabad, Telangana, India Skilviu Soft Solutions Full time ₹ 8,00,000 - ₹ 24,00,000 per year

    Role: Big Data EngineerLocation: Hyderabad, India (Hybrid).Experience: 5+ YearsDuration: 1 year+Notice Period: Immediate -15 days.Key Responsibilities:Design, develop, and maintain scalable data processing pipelines using Spark/PySpark and Hadoop on AWS.Implement and optimize data workflows on AWS EMR, EC2, and ECS.Develop and integrate RESTful APIs and AWS...

  • Big Data Engineer

    7 days ago

    Hyderabad, Telangana, India Hucon Solutions Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    KEY RESPONSIBILITIES:5+ years of experience working in a software product development organization building modern and scalable big data applications• Expertise with functional programming, preferably Scala• Fluency in Hadoop technologies and big data processing technologies, e.g. Spark, YARN, HDFS, Oozie, Hive• Experience implementing RESTful web...

  • Big Data Engineer

    2 weeks ago

    Hyderabad, Telangana, India Hucon Solutions Full time ₹ 12,00,000 - ₹ 36,00,000 per year

    5+ years of experience working in a software product development organization building modern and scalable big data applicationsExpertise with functional programming, preferably ScalaFluency in Hadoop technologies and big data processing technologies, e.g. Spark, YARN, HDFS, Oozie, HiveExperience implementing RESTful web services in Java, Scala, Python, or...

  • Big Data Engineer

    7 days ago

    Hyderabad, Telangana, India Techno Facts Solutions Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Experience in developing and delivering scalable big data pipelines using Apache Spark and Databricks on AWS.Position Requirements :Must Have : Build and maintain scalable data pipelines using Databricks and Apache Spark. Develop and optimize ETL/ELT processes for structured and unstructured data. Knowledge of Lakehouse architecture for efficient data...

  • Big data engineer/Developer

    2 weeks ago

    Hyderabad, Telangana, India IDESLABS PRIVATE LIMITED Full time ₹ 15,00,000 - ₹ 25,00,000 per year

    Big data engineer/DeveloperSpark-ScalaHQL, HiveControl-mJinkinsGitTechnical analysis and up to some extent business analysis (knowledge about banking products, credit cards and its transactions)Notice Period:Immediate.Employment Type: Contract

  • Staff Engineer, Big Data Engineer

    1 week ago

    Hyderabad, Telangana, India Nagarro Full time ₹ 20,00,000 - ₹ 25,00,000 per year

    We're Nagarro.We are a Digital Product Engineering company that is scaling in a big way We build products, services, and experiences that inspire, excite, and delight. We work at scale across all devices and digital mediums, and our people exist everywhere in the world experts across 38 countries, to be exact). Our work culture is dynamic and...

  • ETL or Big data tester

    2 weeks ago

    Hyderabad, Telangana, India Altura Innovative Technologes Full time ₹ 5,00,000 - ₹ 18,00,000 per year

    We're Hiring – ETL or Big data Testing (Hyderabad, India)Altura Innovative Technologies Pvt Ltd is looking for a QA Testing to join our client teamExperience: 6 to 12 yearsAnalyze business requirements and technical specifications to design test scenarios for ETL and big data processes.Develop, maintain, and execute test cases, test scripts, and SQL...