Big Data Operations Engineer
1 day ago
Job Description – Big Data Operations Engineer
Location :
Onsite, Hyderabad Softility
Experience
: 5+ years
Role Overview
We are looking for a skilled
Big Data Operations Engineer
with hands-on experience in supporting and maintaining large-scale data platforms. The role involves ensuring platform availability, managing incidents, monitoring performance through dashboards, and coordinating with development teams for enhancements. While the primary focus is on operations and support, the candidate should also have exposure to Big Data development concepts and tools.
Key Responsibilities
- Monitor and maintain Hadoop clusters, HDFS storage, and YARN resource management.
- Troubleshoot and resolve production issues, ensuring minimal downtime and quick recovery.
- Support Informatica workflows in production, including error handling and performance tuning.
- Manage Kafka topics, producers/consumers, and cluster health monitoring.
- Work with Anthos for hybrid cloud platform management and operational tasks.
- Support observability and log pipelines using Cribl.
- Build and maintain dashboards for monitoring and reporting using tools like New Relic.
- Perform incident management, root cause analysis, and preventive action planning.
- Collaborate with development teams to roll out new jobs, updates, and fixes.
- Maintain operational documentation, runbooks, and knowledge repositories.
Required Skills & Experience
- 5-6 years of experience in Big Data operations/support.
- Strong knowledge of the Hadoop ecosystem, including HDFS, YARN, Hive, and Oozie.
- Experience supporting ETL workflows (preferably Informatica).
- Familiarity with Kafka operations and troubleshooting.
- Exposure to Anthos and Cribl for platform and pipeline monitoring.
- Experience with dashboard/monitoring tools (Grafana, Kibana, or equivalent).
- Hands-on with Linux/Unix administration, shell scripting, and automation.
- Solid experience in incident, change, and problem management.
- Good communication and coordination skills for working with cross-functional teams.
Nice-to-Have
- Exposure to Big Data development (Hive, Spark, Python, or SQL).
- Knowledge of cloud platforms (GCP/AWS/Azure).
- Familiarity with Cloudera Manager or other cluster monitoring tools.
-
Big Data Engineer
1 day ago
Hyderabad, Telangana, India Sheryl strategic solutions Pvt. LTD . Full time ₹ 12,00,000 - ₹ 36,00,000 per yearDescription : Position : Big Data Engineer Location : Hyderabad, India (Hybrid, 2-3 times per week on any weekdays) Duration : 1 year Job Summary : Experian FSD is seeking a skilled and motivated Big Data Engineer to join our dynamic data engineering team. This hybrid role based in Hyderabad offers the opportunity to work on cutting-edge data...
-
Big Data engineer
2 weeks ago
Hyderabad, Telangana, India Cozzera Full time ₹ 15,00,000 - ₹ 25,00,000 per yearJob Title:Data Engineer Location:Hyderabad | Type:Full-time Description:We're hiring aBig Data Engineerto design and implement scalable data pipelines and streaming applications. You'll work with modern big data technologies and cloud platforms to power real-time data solutions. Key Skills:Strong software engineering with Java, Scala, or PythonExpertise in...
-
Big Data Engineer
2 weeks ago
Hyderabad, Telangana, India IDESLABS PRIVATE LIMITED Full time ₹ 12,00,000 - ₹ 36,00,000 per yearWe are looking for a skilled Big Data Engineer with 6 to 22 years of experience. The ideal candidate will have expertise in Pyspark, Azure Data Bricks, and workflows. This position is available as a contract role across Pan India.Roles and ResponsibilityDesign and develop scalable big data systems using Pyspark and Azure Data Bricks.Implement and manage...
-
Big Data Engineer
2 weeks ago
Hyderabad, Telangana, India Allegis Group Full time ₹ 5,00,000 - ₹ 15,00,000 per yearRole & responsibilitiesConsult on or participate in moderately complex initiatives and deliverables within Software Engineering and contribute to large-scale planning related to Software Engineering deliverables.Review and analyze moderately complex Software Engineering challenges that require an in-depth evaluation of variable factors.Contribute to the...
-
Big Data Engineer
2 weeks ago
Hyderabad, Telangana, India Vaakruthi Full time ₹ 15,00,000 - ₹ 25,00,000 per yearRole & responsibilitiesJob Description : Big Data EngineerAnalyze, Code , Test and Implement Big Data Applications (API & Batch) Manage and mitigate incidentsRequired Qualifiactions:years of experience in analysis, coding, implementation and testing of Big Data Applications (API & Batch) on AWS Cloud using java/scala/kotlin, Cassandra,Python, Hadoop, HDFS,...
-
Big Data Engineer
5 days ago
Hyderabad, Telangana, India Skilviu Soft Solutions Full time ₹ 8,00,000 - ₹ 24,00,000 per yearRole: Big Data EngineerLocation: Hyderabad, India (Hybrid).Experience: 5+ YearsDuration: 1 year+Notice Period: Immediate -15 days.Key Responsibilities:Design, develop, and maintain scalable data processing pipelines using Spark/PySpark and Hadoop on AWS.Implement and optimize data workflows on AWS EMR, EC2, and ECS.Develop and integrate RESTful APIs and AWS...
-
Big Data Developer
4 weeks ago
Hyderabad, Telangana, India Alp Consulting Ltd. Full timeJob Title: Big Data DeveloperWork Location: Hyderabad, Telangana (TG), India Experience Range: 6–8 Years Skill Area: Digital – Big Data and Hadoop EcosystemsJob Description:We are looking for an experienced and results-driven Big Data Developer to join our dynamic team in Hyderabad. The ideal candidate will have a strong background in designing,...
-
Big Data DevOps Lead
2 weeks ago
Hyderabad, Telangana, India Zeta Global Full time ₹ 1,04,000 - ₹ 1,30,878 per yearSummary: As a Senior or Lead Big Data DevOps Engineer, you will be working with a team responsible for setting up, scaling, and maintaining Big Data infrastructure and tools in private and public cloud environments.Main Responsibilities: Driving improvement of the efficiency of Big Data infrastructure.Coordinating cross-team infrastructure and Big Data...
-
Big Data Developer
2 weeks ago
Hyderabad, Telangana, India Alp Consulting Ltd. Full time ₹ 15,00,000 - ₹ 20,00,000 per yearJob Title:Big Data DeveloperWork Location:Hyderabad, Telangana (TG), IndiaExperience Range:6–8 YearsSkill Area:Digital – Big Data and Hadoop EcosystemsJob Description:We are looking for an experienced and results-drivenBig Data Developerto join our dynamic team in Hyderabad. The ideal candidate will have a strong background in designing, developing, and...
-
Senior Data Engineer
2 weeks ago
Hyderabad, Telangana, India Enable Data Full time ₹ 15,00,000 - ₹ 20,00,000 per yearExperience Required: 8+YearsMode of work: RemoteSkills Required: Azure DataBricks, Eventhub, Kafka, Architecture,Azure Data Factory, Pyspark, Python, SQL, SparkNotice Period : Immediate Joiners/ Permanent/Contract role (Can join within September 15th 2025)Design, develop, and maintain scalable and robust data solutions in the cloud using Apache Spark and...