Principal SRE

About the Job:

The IT AI Application Platform team is seeking a Principal Senior Site Reliability Engineer (SRE) to design, develop, scale, and operate our AI Application Platform based on Red Hat technologies, including OpenShift AI (RHOAI) and Red Hat Enterprise Linux AI (RHEL AI). As a Principal SRE you will contribute to running core AI services at scale by enabling customer self-service, making our monitoring system more sustainable, and eliminating toil through automation.

On the IT AI Application Platform team, you will have the opportunity to lead and influence the complex challenges of scale which are unique to Red Hat IT managed AI platform services, while using your skills in coding, operations, and large-scale distributed system design. We develop, deploy, and maintain Red Hat's next-generation AI application deployment environment for custom applications and services across a range of hybrid cloud infrastructures.  We are a global team operating on-premise and in the public cloud, using the latest technologies from Red Hat and beyond.

What will you do?

• Working with live systems and coding automation

• Design, build, and manage our large scale infrastructure and platform services, including public cloud, private cloud, and datacenter-based

• Automate cloud infrastructure through use of technologies (e.g. auto scaling, load balancing, etc.), scripting (bash, python and golang), monitoring and alerting solutions (e.g. Splunk, Splunk IM, Prometheus, Grafana, Catchpoint etc)

• Design, develop, and become expert in AI capabilities leveraging emerging industry standards

• Breakdown complex engineering efforts into consumable chunks while working with teams to understand deliverables

• Design and development of software like Kubernetes operators, webhooks, cli-tools

• Implement and maintain intelligent infrastructure and application monitoring designed to enable application engineering teams

• Ensure the production environment is operating in accordance with established procedures and best practices

• Lead escalation support for high severity and critical platform-impacting events

• Provide feedback around bugs and feature improvements to the various Red Hat Product Engineering teams

• Design software tests and lead peer reviews to increase the quality of our codebase

• Help and develop peers' capabilities through knowledge sharing, mentoring, and collaboration

• Participate in a regular on-call schedule, supporting the operation needs of our tenants

• Drive sustainable incident response and lead blameless postmortems

• Work within a small agile team to develop and improve SRE methodologies, support your peers, plan and self-improve

What will you bring?

• 5+ years of experience of using cloud providers and technologies (Google, Azure, Amazon, OpenStack etc)

• 4+ years of experience administering a kubernetes based production environment

• 5+ years of experience with enterprise systems monitoring

• 5+ years of experience with enterprise configuration management software like Ansible by Red Hat, Puppet, or Chef

• 5+ years of experience programming with at least one object-oriented language; Golang, Java, or Python are preferred

• 5+ years of experience delivering a hosted service

• Demonstrated ability to quickly and accurately troubleshoot system issues, assess risks, and support teams through resolution

• Solid understanding of standard TCP/IP networking and common protocols like DNS and HTTP

• Demonstrated comfort with leading collaboration, open communication and reaching across functional and organizational boundaries

• Passion for understanding and anticipating users' needs and delivering outstanding user experiences

• Independent problem-solving and self-direction; motivated to develop these skills in others

• Desires to solve problems through collaboration as part of a global team

• Experience leading teams working with Agile development methodologies

• Bachelor's degree in Computer Science or a related technical field involving software or systems engineering is required

• Hands-on experience that demonstrates your ability and interest in Site Reliability Engineer

• Experience programming in at least one of these languages: Python, Golang, Java, C, C++ or another object-oriented language

•  Experience working with public clouds such as AWS, GCP, or Azure

• Collaboratively troubleshoot and solve problems in a team setting

• Experience troubleshooting an as-a-service offering (SaaS, PaaS, etc.) 

• Experience working with complex distributed systems

• Demonstrated ability to debug, optimize code and automate routine tasks

• Basic understanding of Unix/Linux operating systems

#LI-SH4

About Red Hat

Red Hat is the world's leading provider of enterprise open source software solutions, using a community-powered approach to deliver high-performing Linux, cloud, container, and Kubernetes technologies. Spread across 40+ countries, our associates work flexibly across work environments, from in-office, to office-flex, to fully remote, depending on the requirements of their role. Red Hatters are encouraged to bring their best ideas, no matter their title or tenure. We're a leader in open source because of our open and inclusive environment. We hire creative, passionate people ready to contribute their ideas, help solve complex problems, and make an impact.

Inclusion at Red Hat
Red Hat's culture is built on the open source principles of transparency, collaboration, and inclusion, where the best ideas can come from anywhere and anyone. When this is realized, it empowers people from different backgrounds, perspectives, and experiences to come together to share ideas, challenge the status quo, and drive innovation. Our aspiration is that everyone experiences this culture with equal opportunity and access, and that all voices are not only heard but also celebrated. We hope you will join our celebration, and we welcome and encourage applicants from all the beautiful dimensions that compose our global village.

Equal Opportunity Policy (EEO)
Red Hat is proud to be an equal opportunity workplace and an affirmative action employer. We review applications for employment without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, ancestry, citizenship, age, veteran status, genetic information, physical or mental disability, medical condition, marital status, or any other basis prohibited by law.

Red Hat does not seek or accept unsolicited resumes or CVs from recruitment agencies. We are not responsible for, and will not pay, any fees, commissions, or any other payment related to unsolicited resumes or CVs except as required in a written contract between Red Hat and the recruitment agency or party requesting payment of a fee.
Red Hat supports individuals with disabilities and provides reasonable accommodations to job applicants. If you need assistance completing our online job application, email application- General inquiries, such as those regarding the status of a job application, will not receive a reply.

---