DevOps Architect II

Job Description

About the role....

This SRE professional will have the opportunity to work for an AI-based Unicorn which is recognized as one of the fastest-growing companies on the Inc. 5000 list. This role will provide you opportunity to deploy, maintain and support the o9 Digital Brain Platform across the world on AWS, AZURE, GCP &; Samsung Cloud utilizing state of the art CI/CD tools. This role will empower you to continuously challenge the status quo and implement the great ideas you may have to create value for o9 clients.

Role Overview:

This role involves the management, installation, and configuration of Hadoop/Spark components (Hive, HDFS, Ambari, NiFi), as well as designing the architecture to integrate with the o9 platform. You will play a key role in the design, maintenance, and productionization of Hadoop/Spark/Delta Lake clusters, addressing daily production issues, and implementing containerization and automation to facilitate easier maintenance for the DevOps team.

Key Responsibilities:

- Collaborate with customers and technical consultants to design and recommend big data architecture solutions tailored to customer requirements.
- Analyze complex distributed production environments and provide actionable recommendations for performance optimization.
- Develop and present detailed architectural documentation for customers technical teams.
- Partner with o9 s Development, DevOps, and project teams to ensure the successful execution of projects.
- Design and implement distributed systems and configurations for Spark, Delta Lake, Hadoop, and related technologies, particularly in cloud-based deployments.
- Produce and maintain comprehensive technical documentation and user manuals for customers.
- Stay current with the latest advancements in the distributed big data ecosystem and cloud technologies.

Qualifications and Skills:

- Experience:
- 8+ years of experience in DevOps, with a focus on architecting large-scale storage, data processing, and globally distributed solutions.
- Proven expertise in designing and deploying large-scale, data-intensive solutions using Spark, Delta Lake, and Airflow schedulers.
- Strong experience designing and optimizing data queries within Spark/Delta Lake environments, using tools such as Apache Hive, HDFS, Apache Spark, and Hadoop.
- Experience with the installation, administration, and tuning of cloud-agnostic and distributed data systems.
- Extensive experience implementing solutions across cloud platforms (Azure, AWS, GCP).
- Hands-on experience implementing big data use cases and a solid understanding of standard design patterns for Spark/distributed systems, including Airflow, NiFi, Kafka, and others.
- Significant experience working with network-based APIs, ideally REST/JSON or XML/SOAP.
- Skills:
- Strong understanding of enterprise security solutions such as LDAP, Kerberos, and SSL implementation.
- Expertise in networking, troubleshooting, and optimizing data transfer across systems.
- Proficient in Python and Java development, including debugging and profiling.
- Solid background in database administration and design; experience with data modeling is a plus.
- Excellent verbal and written communication skills.
- Familiarity with R and machine learning algorithms is an advantage.

What we ll do for you

- Competitive Salary
- Flexibility: Work in Hybrid mode and flexible working hours
- Get social: When we work from home, we play from home with fun after-work activities like Friday Socials. If you re in the office, feel free to join these events in person.
- Flat organization: With a very strong entrepreneurial culture (and no corporate politics).
- Support network: Work with a team you can learn from and every day.
- Diversity: We pride ourselves on our international working environment.

---