Azure Data Lead

What you'll do:

• Lead the design, development, and maintenance of scalable data pipelines and lakehouse architectures using Azure Databricks (PySpark, SQL).
• Architect and implement Medallion Architecture (Bronze, Silver, Gold layers) to drive optimal data transformation and organization.
• Oversee data governance, access control, and lineage using Databricks Unity Catalog; champion best practices for metadata management.
• Manage and mentor a team of data engineers; provide technical guidance and career development opportunities.
• Direct the management of data storage solutions using Azure Data Lake Storage (ADLS), ensuring robust access policies.
• Establish and maintain Delta Lake tables for reliable, scalable ACID transactions and incremental processing across projects.
• Optimize Spark jobs and Databricks clusters for performance, scalability, and cost efficiency at enterprise scale.
• Lead ingestion and transformation strategies from diverse sources (APIs, files, databases, cloud platforms).
• Partner with stakeholders and cross-functional teams to deliver well-governed, reusable data models and analytics solutions.
• Drive implementation of data quality frameworks, error handling mechanisms, and robust monitoring practices.
• Enforce data security, governance, and compliance requirements across all Azure and cloud data assets.
• Utilize Azure Synapse Analytics, Key Vault, and Azure Monitor to build resilient, secure, and observable data platforms.
• Own CI/CD and deployment best practices using Git and Azure DevOps; establish automation standards and code review processes.
• Stay ahead of industry trends in big data, ETL, and lakehouse technologies; proactively recommend improvements.
• Collaborate with data science and AI teams to support machine learning (ML), generative AI, and large language model (LLM) initiatives using enterprise data.
• Guide data consultation efforts for enterprise digital transformation, advising stakeholders on data architecture modernization and strategic priorities.
• Develop and articulate an "AI-ready" data roadmap, enabling the business to leverage advanced analytics, ML, and LLM solutions.
• Ensure data pipelines and platforms are optimized for ML and AI workloads, including integration and preparation of datasets for model training and inference.

What we seek in you:

• Bachelor's or Master's in Computer Science, Engineering, or related field.
• 8+ years of experience in data engineering, with 2+ years in a technical lead or supervisory role.
• Advanced expertise in Databricks, PySpark, SQL, Azure cloud data engineering, and related tools.
• Proven track record architecting and leading end-to-end pipeline solutions leveraging Medallion architecture, Unity Catalog, and Delta Lake.
• Experience mentoring and scaling data engineering teams; excellent leadership and communication skills.
• Deep understanding of big data principles, data lakehouse architectures, and cloud ETL orchestration.
• Hands-on with Azure Data Lake Storage (ADLS), Data Factory (ADF), Synapse Analytics, Key Vault, Sql Server and Azure Monitor.
• Skilled in CI/CD pipelines, automation with Git and Azure DevOps, and code quality enforcement.
• Strong analytical mindset with a strategic approach to problem-solving and stakeholder management.
• Certifications in Azure/Data Engineering (Databricks, Azure) are highly desired.
• Proven ability to consult on data strategy and architecture for large-scale digital transformation initiatives.
• Strong knowledge of data preparation, feature engineering, and optimization for ML and AI workloads.
• Experience collaborating cross-functionally with data science, analytics, and business teams on ML/AI use cases.

Life at Next:

At our core, we're driven by the mission of tailoring growth for our customers by enabling them to transform their aspirations into tangible outcomes. We're dedicated to empowering them to shape their futures and achieve ambitious goals. To fulfil this commitment, we foster a culture defined by agility, innovation, and an unwavering commitment to progress. Our organizational framework is both streamlined and vibrant, characterized by a hands-on leadership style that prioritizes results and fosters growth.

Perks of working with us:

• Clear objectives to ensure alignment with our mission, fostering your meaningful contribution.
• Abundant opportunities for engagement with customers, product managers, and leadership.
• You'll be guided by progressive paths while receiving insightful guidance from managers through ongoing feedforward sessions.
• Cultivate and leverage robust connections within diverse communities of interest. Choose your mentor to navigate your current endeavors and steer your future trajectory.
• Embrace continuous learning and upskilling opportunities through Nexversity.
• Enjoy the flexibility to explore various functions, develop new skills, and adapt to emerging technologies. Embrace a hybrid work model promoting work-life balance. Access comprehensive family health insurance coverage, prioritizing the well-being of your loved ones.
• Embark on accelerated career paths to actualize your professional aspirations.

Who we are?

We enable high growth enterprises build hyper personalized solutions to transform their vision into reality. With a keen eye for detail, we apply creativity, embrace new technology and harness the power of data and AI to co-create solutions tailored made to meet unique needs for our customers.

Join our passionate team and tailor your growth with us