Staff Site Reliability Engineer

Saviynt's AI-powered identity platform manages and governs human and non-human access to all of an organization's applications, data, and business processes. Customers trust Saviynt to safeguard their digital assets, drive operational efficiency, and reduce compliance costs. Built for the AI age, Saviynt is today helping organizations safely accelerate their deployment and usage of AI. Saviynt is recognized as the leader in identity security, with solutions that protect and empower the world's leading brands, Fortune 500 companies and government institutions. For more information, please visit  Our Monitoring and Alerting team within the SaaS Operations team combines Operations Excellence with the Development Experience to deliver services at high scale, high availability with resilience by using automation and Infrastructure Code. We build reliability into our ecosystem by applying best practices in Resiliency Engineering, Automation, Observability & Chaos Testing.   The team comes from diverse technical backgrounds, and the responsibilities provide the opportunity for a variety of challenges. Ideal candidates will have a background in either software engineering or systems engineering with a desire to learn the other or previous experience with building and managing Monitoring and Alerting systems. We are looking for a Systems Thinking, Principal Engineer who has helped teams scale through production insights, operational automation, building observability program, developer guidance, real-time metrics, automation, automation, automation WHAT YOU WILL BE DOINGImplement monitoring and alerting systems to guarantee high availability and performance, with a dedicated focus on SLA and availability metrics.Collaborate with engineering and operations teams to identify critical components and systems requiring enhanced availability measures.Design and implement strategies, tooling, and processes to enhance system uptime and reliability.Continuously evaluate and recommend improvements to platform infrastructure and processes, enhancing efficiency and reliability.Align the platform with customer needs and business goals by working closely with cross-functional teams.Run the production environment by monitoring availability and taking a holistic view of system health.Build software and systems to monitor platform infrastructure and applications.Monitor and Improve reliability, quality, and time-to-market of our suite of software solutions.Measure and optimize system performance, with an eye toward pushing our capabilities forward, getting ahead of customer needs, and innovating for continual improvement.Provide primary operational support and engineering for multiple large-scale distributed software applications.Gather and analyze metrics from operating systems as well as applications to assist in performance tuning and fault finding. WHAT YOU BRINGBachelor's degree or higher in a technology related field (e.g. Engineering, Computer Science, etc.) required, Master's degree a plus6+ years professional experience Monitoring and Alerting roles on major cloud platforms (AWS, Azure), preferably someone with project leadership roles.4+ experience in Cloud development (AWS, Azure) and observability skills; Experience with building and operating highly resilient platforms in AWS cloud environments.3+ years of experience in software development with Python, NodeJS, or Java with a focus on SDLC and automationHands-on experience with container orchestration, preferably with KubernetesHands-on experience with building observability, monitoring and alerting on large scale distributed systems.Leadership/design of application and/or infrastructure migration projects from on-prem to cloudCloud architecture design and implementation to solve key business needs and meet team goals.Familiarity with current AWS solutions; Azure experience also considered.Containerized workloads (Prefer Helm; Related: AKS & EKS, other K8s distributions, Docker, JFrog)Logging and monitoring tools (Prefer: Prometheus, Grafana, Dataddon, AWS Cloudwatch; Related, , Azure Monitor, Log Analytics, Fluentd)Network Security (e.g. AWZ Policy, Azure Policy, VPN, Active Directory/RBAC, ACLs, NSG rules, private endpoints)Proven experience in implementing advanced observability practices and techniques at scale.Hands on experience with one or more observability tools (Prometheus, Grafana,ELK/OpenSearch, OpenTelemetry, Datadog, etc.)Experienced in Instrumentation with systems skills on building and operating,monitoring, logging, alerting services of distributed systems at scale.Demonstrated ability to utilize modern monitoring tools (DataDog, Prometheus, etc)Experienced in Instrumentation with systems skills on building and operating,monitoring, logging, alerting services of distributed systems at scale.Ability to build monitoring ecosystem with high fidelity alerting.Ability to automate resolution of alerts.Ability to automate with various scripting languages (Python, Golang, Shell scripting,etc.)Knowledge of managing systems using infrastructure as code tools (IAM, ARM,Terraform, Chef)Solid understanding of Cloud Computing and DevOps concepts.Hands-on Kubernetes skills and knowledge.Proven experience in maintaining scalability and resiliency of complex environment.Ability to triage, execute root cause analysis, and be decisive under pressureExperience managing and interpreting large datasets using query languages and visualization toolsProficient communication skills with an ability to reach both technical and non-technical audienceAbility to learn new software, method and practices and bringing them to our developersAbility to work with a variety of individuals and groups, both in person and virtually, in aconstructive and collaborative manner and build and maintain effective relationships