Senior Data Platform Architect

Welcome to a challenging opportunity where you can grow as an expert in data engineering.

We are looking for a highly skilled Lead Big Data Engineer to lead the development of our Enterprise TruRisk Platform.

This platform helps customers measure, communicate, and eliminate cyber risks. As part of our team, you will work with engineers and architects to design, develop, and support a highly scalable, distributed SaaS-based Security Risk Prioritization product.

This is a fantastic opportunity to be part of a team building Qualys' next-generation platform using Big Data & Micro-Services technology.

The platform processes over billions of transactions per day, leveraging open-source technologies and working on challenging business-impacting initiatives.

• Be the thought leader in data platform and pipeline along with risk evaluation.
• Provide technical leadership to the engineering organization on data platform design, roll out, and evolution.
• Liaison to product teams, professional services, and sales engineers on solution and trade-off reviews, representing engineering in such conversations.
• Drive technology explorations and roadmaps.
• Serve as a technical lead on demanding cross-functional departments.
• Ensure the quality of architecture and design of systems.
• Functionally decompose complex problems into simple solutions.
• Fully understand system interdependencies and limitations.
• Possess expert knowledge in performance, scalability, enterprise system architecture, and engineering best practices.
• Leverage knowledge of internal and industry prior art in design decisions.
• Effectively research and benchmark cloud technology against competing systems.
• Able to document requirements for developers.
• Assist developers with proper requirements and directions.
• Assist in career development by mentoring individuals and the community on advanced technical issues.
• Exert technical influence over multiple teams, increasing productivity and effectiveness.

• Bachelor's degree in computer science or equivalent.
• 8+ years of total experience.
• 4+ years of relevant experience in designing and architecting Big Data solutions using Spark.
• 3+ years of experience working with engineering resources for innovation.
• 4+ years of experience understanding Big Data events flow pipeline.
• 3+ years of experience in performance testing for large infrastructure.
• 3+ years of in-depth experience in various search solutions (solr/elastic).
• 3+ years of experience in Kafka.
• In-depth experience in Data Lakes and related ecosystems.
• In-depth experience in message queues.
• In-depth experience in giving requirements for scalable architecture in Big Data and Micro-Services environments.
• In-depth understanding of caching components or services.
• Knowledge in Presto technology.
• Knowledge in Airflow.
• Hands-on experience in scripting and automation.
• In-depth understanding of RDBMS/NoSQL, Oracle, Cassandra, Kafka, Redis, Hadoop, lambda architecture, kappa, kappa++ architectures with Flink data streaming and rule engines.
• Experience in working with ML models engineering and deployment.
• Design and implement secure Big Data clusters meeting many compliance and regulatory requirements.
• Experience leading delivery of large-scale systems focused on managing the infrastructure layer of the technology stack.
• Strong experience in performance benchmarking testing for Big Data technologies.
• Strong troubleshooting skills.
• Experience leading development life cycle process and best practices.
• Experience in Big Data services administration would be an added value.
• Experience with Agile Management (SCRUM, RUP, XP), OO Modeling, working on internet, UNIX, Middleware, and database-related projects.
• Experience mentoring/training the engineering community on complex technical issues.
• Project management experience.

Becoming part of our team means that you will have opportunities to grow professionally and personally, we offer a dynamic and supportive work environment that encourages creativity, innovation, and collaboration.