Principal Observability Engineer

2 months ago


Hyderabad, India Experian Full time

Job Description

The Principal Observability Engineer is a critical role in our organization, dedicated to ensuring the robustness, performance, and scalability of our infrastructure and applications through superior monitoring and observability practices. This role will work closely with Infrastructure as Code (IAC) tooling like Terraform and will have a strong understanding of open telemetry standards. Familiarity with current monitoring and logging tools like Dynatrace and Splunk is essential. This role is ideal for a proactive go-getter who is eager to drive new technology adoption within the organization.

Summary of Primary Responsibilities

• Lead the design and implementation of observability solutions that provide deep insights into application performance, system health, and user experience.

• Establish and advocate for observability best practices across engineering teams.

• Work closely with the infrastructure teams to automate and optimize infrastructure provisioning and scaling using IAC tools like Terraform.

• Ensure infrastructure code is tested, reliable, and efficient.

• Champion the adoption of open telemetry standards to collect, process, and export telemetry data.

• Utilize and integrate monitoring tools like Dynatrace and Splunk to provide thorough insights and analytics.

• Drive the evaluation and adoption of new tools and technologies to keep the organization at the forefront of observability and monitoring practices.

• Collaborate with various engineering teams to ensure smooth adoption and transition to new technologies.

• Analyze existing monitoring and observability practices, identifying areas for improvement or optimization.

• Foster a culture of continuous learning and improvement within the observability team and across the organization.

• Provide leadership, guidance, and mentoring to the observability team.

• Foster a collaborative and inclusive environment that encourages innovation and growth.


Qualifications

What your background looks like

10+ years of observability engineering experience Information Technology degree and/or technology certifications preferred or substantial equivalent experience. Design and implement an observability system for a new microservices-based application. Migrate an existing monitoring system to Prometheus and Grafana Develop a new alerting system to detect and respond to performance issues. Work with the development team to instrument their code for better observability. Train and mentor other engineers on observability best practices Strong customer and communication skills to interact with team members, customers, vendors and leadership team. Advanced Shell scripting and IaC automation skills with Ansible and Terraform Deep understanding of open telemetry standards. Experience with monitoring and logging tools like Dynatrace and Splunk. Proactive, go-getter attitude with a passion for new technology adoption. Excellent communication and collaboration skills. Ability to lead and mentor a team of engineers. Working Knowledge of Python and any databases (SQL/NoSQL). SMEs in enterprise monitoring like APM, Custom attribute Implementation, synthetic monitoring, browser monitoring, and Log monitoring Knowledge of requirement gathering and rollout monitoring and observability solutions. Partner with the business and development teams to identify requirements, define monitoring solutions, and implement the same. Experience in Application Performance Monitoring (APM) and Infrastructure Monitoring for Different Hybrid Business Applications and Infrastructure Providing health and performance reports, developing AIOps rules, creating alerts, creating custom dashboards Experience in creating workloads and user onboarding.

  • Hyderabad, India Splunk Inc Full time

    Join us as we pursue our disruptive new vision to make machine data accessible, usable, and valuable to everyone. We are a company filled with people who are passionate about our product and seek to deliver the best experience for our customers. At Splunk, we’re committed to our work, customers, having fun, and most importantly to each other’s success.As...


  • Hyderabad, India Splunk Inc Full time

    Join us as we pursue our innovative vision to make machine data accessible, usable, and valuable to everyone. We are a company filled with people who are passionate about our product and seek to deliver the best experience for our customers. At Splunk, we’re committed to our work, our customers, having fun, and most meaningfully, to each other’s success....


  • hyderabad, India Splunk Inc Full time

    Join us as we pursue our innovative vision to make machine data accessible, usable, and valuable to everyone. We are a company filled with people who are passionate about our product and seek to deliver the best experience for our customers. At Splunk, we’re committed to our work, our customers, having fun, and most meaningfully, to each other’s success....


  • Hyderabad, India Splunk Inc Full time

    Join us as we pursue our innovative vision to make machine data accessible, usable, and valuable to everyone. We are a company filled with people who are passionate about our product and seek to deliver the best experience for our customers. At Splunk, we’re committed to our work, our customers, having fun, and most meaningfully, to each other’s success....


  • hyderabad, India Splunk Inc Full time

    Join us as we pursue our disruptive new vision to make machine data accessible, usable, and valuable to everyone. We are a company filled with people who are passionate about our product and seek to deliver the best experience for our customers. At Splunk, we’re committed to our work, customers, having fun, and most importantly to each other’s success.As...


  • Hyderabad, India Splunk Inc Full time

    Join us as we pursue our disruptive new vision to make machine data accessible, usable, and valuable to everyone. We are a company filled with people who are passionate about our product and seek to deliver the best experience for our customers. At Splunk, we’re committed to our work, customers, having fun, and most importantly to each other’s success.As...


  • Hyderabad, India Splunk Inc Full time

    Join us as we pursue our exciting new vision to make machine data accessible, usable, and valuable to everyone. We are a company filled with people passionate about our product and seeking to deliver the best experience for our customers. At Splunk, we’re committed to our work, customers, having fun and most importantly to each other’s success. Learn...


  • hyderabad, India Splunk Inc Full time

    Join us as we pursue our exciting new vision to make machine data accessible, usable, and valuable to everyone. We are a company filled with people passionate about our product and seeking to deliver the best experience for our customers. At Splunk, we’re committed to our work, customers, having fun and most importantly to each other’s success. Learn...


  • Hyderabad, India Splunk Inc Full time

    Join us as we pursue our exciting new vision to make machine data accessible, usable, and valuable to everyone. We are a company filled with people passionate about our product and seeking to deliver the best experience for our customers. At Splunk, we’re committed to our work, customers, having fun and most importantly to each other’s success. Learn...


  • Hyderabad, India Splunk Inc Full time

    Join us as we pursue our exciting new vision to make machine data accessible, usable, and valuable to everyone. We are a company filled with people passionate about our product and seeking to deliver the best experience for our customers. At Splunk, we’re committed to our work, customers, having fun and most importantly to each other’s success. Learn...


  • Hyderabad, India Splunk Inc Full time

    Join us as we pursue our ground-breaking vision to make machine data accessible, usable, and valuable to everyone. We are a company filled with people who are passionate about our product and seek to deliver the best experience for our customers. At Splunk, we are committed to our work, customers, having fun, and most significantly to each other’s...


  • hyderabad, India Splunk Inc Full time

    Join us as we pursue our exciting new vision to make machine data accessible, usable, and valuable to everyone. We are a company filled with people passionate about our product and seeking to deliver the best experience for our customers. At Splunk, we’re committed to our work, customers, having fun and most importantly to each other’s success. Learn...


  • Hyderabad, India Splunk Inc Full time

    Join us as we pursue our exciting new vision to make machine data accessible, usable, and valuable to everyone. We are a company filled with people passionate about our product and seeking to deliver the best experience for our customers. At Splunk, we’re committed to our work, customers, having fun and most importantly to each other’s success. Learn...


  • hyderabad, India Splunk Inc Full time

    Join us as we pursue our ground-breaking vision to make machine data accessible, usable, and valuable to everyone. We are a company filled with people who are passionate about our product and seek to deliver the best experience for our customers. At Splunk, we are committed to our work, customers, having fun, and most significantly to each other’s...


  • Hyderabad, India Splunk Inc Full time

    Join us as we pursue our ground-breaking vision to make machine data accessible, usable, and valuable to everyone. We are a company filled with people who are passionate about our product and seek to deliver the best experience for our customers. At Splunk, we are committed to our work, customers, having fun, and most significantly to each other’s...


  • hyderabad, India Splunk Inc Full time

    At Splunk, our mission is to forge a safer, more resilient digital world. The world's leading enterprises use our unified security and observability platform to keep their digital systems secure and reliable. While customers love our technology, our people make Splunk stand out as a fantastic career destination and why we've won many awards as the best...


  • Hyderabad, India Splunk Inc Full time

    At Splunk, our mission is to forge a safer, more resilient digital world. The world's leading enterprises use our unified security and observability platform to keep their digital systems secure and reliable. While customers love our technology, our people make Splunk stand out as a fantastic career destination and why we've won many awards as the best...


  • Hyderabad, India Dell International Services India Pvt Ltd (7451) Full time

    IT Principal Software EngineerThe Software Engineering team delivers next-generation software application enhancements and new products for a changing world. Working at the cutting edge, we design and develop software for platforms, peripherals, applications and diagnostics — all with the most advanced technologies, tools, software engineering...


  • Hyderabad, India Genpact Full time

    Genpact (NYSE: G) is a global professional services and solutions firm delivering outcomes that shape the future. Our 125,000+ people across 30+ countries are driven by our innate curiosity, entrepreneurial agility, and desire to create lasting value for clients. Powered by our purpose – the relentless pursuit of a world that works better for people – we...


  • hyderabad, India Splunk Inc Full time

    Join us as we pursue our exciting new vision to make machine data accessible, usable and valuable to everyone. We are a company filled with people who are passionate about solving problems using data and seek to deliver the best experience for customers. At Splunk, we're committed to our work, our customers, having fun, and most importantly to each other's...