Data & Ai/ml Architect

**Company Overview**:
Milestone Technologies is a global IT managed services firm that partners with organizations to scale their technology, infrastructure and services to drive specific business outcomes such as digital transformation, innovation, and operational agility. Milestone is focused on building an employee-first, performance-based culture and for over 25 years, we have a demonstrated history of supporting category-defining enterprise clients that are growing ahead of the market. The company specializes in providing solutions across Application Services and Consulting, Digital Product Engineering, Digital Workplace Services, Private Cloud Services, AI/Automation, and ServiceNow. Milestone culture is built to provide a collaborative, inclusive environment that supports employees and empowers them to reach their full potential.

**Job Overview**:
Job Summary:
Key Responsibilities:

- Lead the **architecture, design, and implementation** of scalable and secure **Lakehouse solutions using Databricks** and **Delta Lake**.
- Define and implement **data modeling best practices**, including **medallion architecture** (bronze/silver/gold layers).
- Champion **data quality and governance** frameworks leveraging **Databricks Unity Catalog** for metadata, lineage, access control, and auditing.
- Architect real-time and batch **data ingestion pipelines** using **Apache Spark Structured Streaming**, **Auto Loader**, and **Delta Live Tables (DLT)**.
- Develop **reusable templates, workflows, and libraries** for data ingestion, transformation, and consumption across various domains.
- Collaborate with enterprise data governance and security teams to ensure compliance with regulatory and organizational data standards.
- Promote **self-service analytics and data democratization** by enabling business users through **Databricks SQL** and **Power BI/Tableau integrations**.
- Partner with Data Scientists and ML Engineers to enable ML workflows using **MLflow**, **Feature Store**, and **Databricks Model Serving**.
- Provide architectural leadership for enterprise data platforms, including **performance optimization**, **cost governance**, and **CI/CD automation** in Databricks.
- Define and drive the adoption of **DevOps/MLOps best practices** on Databricks using **Databricks Repos**, **Git**, **Jobs**, and **Terraform**.
- Mentor and lead engineering teams on modern data platform practices, **Spark performance tuning**, and efficient **Delta Lake optimizations (Z-ordering, OPTIMIZE, VACUUM, etc.)**.

Technical Skills:

- 10+ years in **Data Warehousing, Data Architecture, and Enterprise ETL design**.
- 5+ years hands-on experience with **Databricks on Azure/AWS/GCP**, including advanced **Apache Spark** and **Delta Lake**.
- Strong command of **SQL, PySpark, and Spark SQL** for large-scale data transformation.
- Proficiency with **Databricks Unity Catalog**, **Delta Live Tables**, **Autoloader**, **DBFS**, **Jobs**, and **Workflows**.
- Hands-on experience with **Databricks SQL** and integration with **BI tools** (Power BI, Tableau, etc.).
- Experience implementing **CI/CD on Databricks**, using tools like **Git**, **Azure DevOps**, **Terraform**, and **Databricks Repos**.
- Proficient with **streaming architecture** using **Spark Structured Streaming**, **Kafka**, or **Event Hubs/Kinesis**.
- Understanding of **ML lifecycle management** with **MLflow**, and experience in deploying **MLOps** solutions on Databricks.
- Familiarity with **cloud object stores** (e.g., AWS S3, Azure Data Lake Gen2) and **data lake architectures**.
- Exposure to **data cataloging and metadata management** using **Unity Catalog** or third-party tools.
- Knowledge of **orchestration tools** like **Airflow**, **Databricks Workflows**, or **Azure Data Factory**.
- Experience with **Docker/Kubernetes** for containerization (optional, for cross-platform knowledge).

Preferred Certifications (a plus):

- Databricks Certified Data Engineer Associate/Professional
- Databricks Certified Lakehouse Architect
- Microsoft Certified: Azure Data Engineer / Azure Solutions Architect
- AWS Certified Data Analytics - Specialty
- Google Professional Data Engineer

**Compensation**:
Estimated Pay Range:
Exact compensation and offers of employment are dependent on circumstances of each case and will be determined based on job-related knowledge, skills, experience, licenses or certifications, and location.

**Our Commitment to Diversity & Inclusion**:

- At Milestone we strive to create a workplace that reflects the communities we serve and work with, where we all feel empowered to bring our full, authentic selves to work. We know creating a diverse and inclusive culture that champions equity and belonging is not only the right thing to do for our employees but is also critical to our continued success._
- Milestone Technologies provides equal employment opportunity for all applicants and employees. All qualified applicants will receive consideration for employ