AI Data Solutions Manager

About Skyhigh Security:

Skyhigh Security is a dynamic, fast-paced, cloud company that is a leader in the security industry. Our mission is to protect the world’s data, and because of this, we live and breathe security. We value learning at our core, underpinned by openness and transparency.

Since 2011, organizations have trusted us to provide them with a complete, market-leading security platform built on a modern cloud stack. Our industry-leading suite of products radically simplifies data security through easy-to-use, cloud-based, Zero Trust solutions that are managed in a single dashboard, powered by hundreds of employees across the world. With offices in Santa Clara, Aylesbury, Paderborn, Bengaluru, Sydney, Tokyo and more, our employees are the heart and soul of our company.

Role Overview:

We are seeking an experienced and strategic Data Engineering Manager to lead our team. The ideal candidate will have a strong background in big data architecture, cloud-native services, and team leadership. You will be responsible for building and optimizing our data platforms, ensuring data security and privacy, and integrating emerging technologies like AI/LLMs to enhance our analytics capabilities. This role requires a blend of technical expertise, leadership skills, and a forward-thinking mindset to drive our data initiatives.

Responsibilities:

• Lead, mentor, and grow a team of talented data engineers. Provide technical guidance and conduct code reviews to ensure best practices.
• Architect and develop scalable and robust data pipelines using a blend of big data frameworks (e.G., Spark, Kafka, Flink) and cloud-native services (AWS) to support security analytics use cases.
• Drive CI/CD best practices, infrastructure automation, and performance tuning across distributed environments.
• Evaluate and pilot the use of AI/LLM technologies in data pipelines for tasks like anomaly detection, metadata enrichment, and automation.
• Ensure data security and privacy compliance across all data platforms and processes.
• Evaluate and integrate LLM-based automation and AI-enhanced observability into engineering workflows.
• Collaborate with data scientists, product managers, and business leaders to understand data needs and deliver solutions that drive business value.
• Oversee the design and implementation of various databases, including NoSQL databases like HBase and Cassandra.

What We’re Looking For (Minimum Qualifications)

• 10 to 15 years of experience in big data architecture and engineering, including deep proficiency with the AWS cloud platform, with at least 3-5 years in a leadership or management role.
• Expertise in distributed systems and frameworks such as Apache Spark, Scala, Kafka, and Flink, with experience building production-grade data pipelines.
• Strong programming skills in Java for building scalable data applications.
• Hands-on experience with ETL tools and orchestration systems.
• Solid understanding of data modeling across both relational (PostgreSQL, MySQL) and NoSQL (HBase) databases, as well as performance tuning.
• Demonstrated experience with AWS services including Lambda functions, AWS Step Functions, and CloudFormation (CF).
• Strong interpersonal and communication skills to effectively lead a team and collaborate with a diverse group of stakeholders.
• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

What Will Make You Stand Out (Preferred Qualifications)

• Experience integrating AI/ML or LLM frameworks (e.G., LangChain, LlamaIndex) into data workflows.
• Experience implementing CI/CD pipelines with Kubernetes, Docker, and Terraform.
• Knowledge of modern data warehousing (e.G., BigQuery, Snowflake) and data governance principles (GDPR, HIPAA).