Tech Architect

Job Description While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth. If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi Job Title: Architect - Data (Azure Databricks) Experience: 10+ Years Location: Bangalore - Hybrid About the Role: We are seeking a highly experienced Data Architect with deep expertise in Azure Databricks and the broader Azure data ecosystem to lead the design and implementation of scalable, secure, and high-performance data architectures. This role is critical to building enterprise-grade data platforms that power analytics, AI/ML, and data governance initiatives. As a Data Architect, you will define data platform architecture standards, guide engineering teams, and ensure best practices are applied across data lake, pipeline orchestration, access control, and distributed processing layers. Key Responsibilities: - Architect and design end-to-end data platform solutions using Azure Databricks as the primary compute engine. - Define data lake strategies and structures leveraging Azure Data Lake Storage Gen2 for raw, curated, and consumer zones. - Design and optimize ETL/ELT pipelines using Azure Data Factory (ADF) and Azure Databricks notebooks/jobs. - Lead platform-level security design by integrating with Azure Active Directory (AAD) and applying role-based access control (RBAC), Unity Catalog (if applicable), and data masking policies. - Define standards for Delta Lake implementation, partitioning, schema evolution, and cost optimization in Spark jobs. - Guide teams on the use of CI/CD, infrastructure-as-code, and automated deployment patterns for data workloads. - Ensure data solutions meet governance, quality, and compliance requirements across environments. - Collaborate with business, data governance, and engineering teams to align architecture with enterprise data strategies. - Provide architecture oversight for performance tuning, scalability, disaster recovery, and observability setup. Must-Have Skills: - 10+ years of experience in data engineering or architecture, with at least: - 3-5 years of hands-on experience with Azure Databricks and Apache Spark. - Strong exposure to Azure Data Lake Storage Gen2. - Proficiency in Azure Data Factory for pipeline orchestration. - Good understanding of Azure Active Directory, AAD groups, service principals, and access control models. - Expertise in designing multi-zone lakehouse architectures and distributed processing solutions. - Strong in Python (PySpark), SQL, and Spark job performance tuning. - Experience with Delta Lake, schema evolution, and versioning. - Proven ability to lead and mentor engineering teams on architecture and technical decisions. Nice-to-Have Skills: - Knowledge of Unity Catalog for data governance and metadata management. - Experience with Azure Synapse Analytics, Event Hubs, or Purview. - Familiarity with data mesh, domain-driven data ownership, or MDM integration. - Understanding of data quality frameworks and observability tooling. Preferred Certifications: - Databricks Certified Data Engineer - Professional - Azure Data Engineer Associate