Principal Software Engineer, Distributed Cloud

At F5, we strive to bring a better digital world to life. Our teams empower organizations across the globe to create, secure, and run applications that enhance how we experience our evolving digital world. We are passionate about cybersecurity, from protecting consumers from fraud to enabling companies to focus on innovation.

Everything we do centers around people. That means we obsess over how to make the lives of our customers, and their customers, better. And it means we prioritize a diverse F5 community where each individual can thrive.

Principal Engineer in QA, Distributed Cloud Data Platform

We are looking for a Principal QA Engineer with expertise in data systems, observability, and automation to join our Distributed Cloud Data Platform team to lead the quality strategy for next-generation data infrastructure that serves mission-critical workloads across our SaaS platform.

This is a hands-on, technical role where you will design test frameworks, define data validation strategies, lead performance benchmarking, and mentor team on reliability, automation, and test best practices for large-scale distributed data systems throughout the development lifecycle.

Key Responsibilities
Quality Strategy & Automation Framework

• Design QA strategies for data streaming, storage, and observability services.
• Partner with Engineering, Product, SRE, and Platform teams to embed quality and reliability throughout the SDLC.
• Build automation framework for data validation, regression, and integration testing.
• Extend automation to handle real-time data streams, schema evolution, workflows, and data consistency checks.

Data Systems & Observability Testing

• Design and execute tests for streaming platforms (e.g. Kafka), ETL pipelines, and data storage systems (ClickHouse, ElasticSearch, Iceberg, S3).
• Develop tools to validate ingestion, transformation, and query accuracy.
• Automate validation of logs, metrics, and traces for correctness & completeness.
• Validate telemetry and SaaS usage pipelines (e.g. Prometheus, OpenTelemetry).
• Simulate failure and recovery scenarios for distributed systems.
• Ensure system instrumentation for high coverage automated observability testing.

Cloud Infrastructure Quality & Performance

• Validate deployments across multi-cloud and K8s native data clusters.
• Implement chaos and resilience testing for data system components.
• Collaborate with SRE/Ops to ensure test environments are production-parity.
• Establish performance and load testing frameworks for streaming (e.g. Kafka topics/partitions), ingestion of APIs, and warehouse/Data lake (e.g. ClickHouse queries).
• Build synthetic data generators and benchmarking tools for large-scale test datasets.
• Analyze bottlenecks and help optimize system throughput and latency.
• Perform performance, scalability, and reliability testing to ensure our data platform can handle global-scale workloads.

QA Best Practice & Mentorship

• Integrate test frameworks into CI/CD pipelines, validate complex, distributed systems across multi-cloud environments.
• Identify, document, and track defects through resolution.
• Create and maintain test plans, test cases, and documentation.
• Participate in design and code reviews to ensure quality is built into every stage of development.
• Mentor junior QA engineers and promote best practices in test automation and quality assurance.
• Investigate production issues and contribute to root cause analysis and remediation strategies.

Required Skills & Experience

• 10+ years of experience in Quality Assurance, with at least 7 years focused on automation, with Computer Science or equivalent practical experience.
• Strong background testing data-intensive or observability systems (e.g. Kafka, Flink, Spark, ClickHouse, ElasticSearch, Prometheus, OpenTelemetry).
• Proficiency coding/scripting skills in Python, Go, or Java for automation and tooling.
• Experience with automation frameworks (e.g. Selenium or similar).
• Expertise in performance testing tools (e.g., Locust, Gatling, k6, JMeter) and benchmarking distributed systems.
• Expertise in streaming data validation, schema, and event-driven architectures.
• Exposure to warehouse/data lake performance tuning and query optimization.
• Familiarity with compliance validation in data pipelines (e.g. PII masking).
• Familiar with cloud-native architectures (K8s, Terraform, Helm, CI/CD pipelines).
• Experience testing in cloud, distributed systems, microservices, and APIs.
• Familiarity with CI/CD pipelines, version control (Git), and DevOps practices.
• Excellent analytical, debugging, and communication skills.
• Experience leading QA strategy in SaaS, observability, or analytics platforms.

The Job Description is intended to be a general representation of the responsibilities and requirements of the job. However, the description may not be all-inclusive, and responsibilities and requirements are subject to change.

Please note that F5 only contacts candidates through F5 email address (ending with ) or auto email notification from Workday (ending with or
)
.
Equal Employment Opportunity
It is the policy of F5 to provide equal employment opportunities to all employees and employment applicants without regard to unlawful considerations of race, religion, color, national origin, sex, sexual orientation, gender identity or expression, age, sensory, physical, or mental disability, marital status, veteran or military status, genetic information, or any other classification protected by applicable local, state, or federal laws. This policy applies to all aspects of employment, including, but not limited to, hiring, job assignment, compensation, promotion, benefits, training, discipline, and termination. F5 offers a variety of reasonable accommodations for candidates. Requesting an accommodation is completely voluntary. F5 will assess the need for accommodations in the application process separately from those that may be needed to perform the job. Request by contacting