Cloud-Native Data Architect

We are seeking a highly skilled and experienced Senior Data Engineer to join our team.

The ideal candidate will have a strong background in cloud-native data engineering, primarily on Microsoft Azure, and familiarity with AWS and GCP. They will be responsible for designing, developing, and optimizing data ingestion, transformation, and governance frameworks, enabling real-time and batch data analytics across our data platform.

The successful candidate will possess deep expertise in building scalable data pipelines, implementing enterprise-grade data governance, security, and AI-powered engineering automation. They will work closely with cross-functional teams to drive the adoption of data cataloguing, profiling, and classification tools, and collaborate with data stewards and compliance teams to ensure governance alignment.

This role requires a strong understanding of data architecture patterns, including Lakehouse, Medallion, and Delta architecture. The candidate should also have hands-on experience with Azure Data Factory, Azure Data Lake Gen2, Databricks, Synapse, and Purview.

In addition to technical skills, the successful candidate will possess excellent problem-solving, communication, and stakeholder collaboration skills. They will be proactive in learning and adopting emerging technologies in data & AI, and have a consultative approach to working with teams.

Key Responsibilities:

• Design and implement scalable, robust, and secure data pipelines using Azure Data Factory, Databricks, Synapse Analytics, Azure Data Lake Gen2, Event Hub, and Azure Functions.
• Develop and maintain ETL/ELT processes for structured and unstructured data.
• Implement Change Data Capture (CDC), streaming pipelines, and batch ingestion.
• Work with AWS Glue, S3, Redshift, or GCP BigQuery, Dataflow as needed.
• Optimize performance and cost across data workloads and cloud environments.
• Establish and maintain data quality frameworks, observability tools, and monitoring dashboards.
• Define and enforce data rules, validations, anomaly detection, and reconciliation.
• Implement data lineage and metadata tracking using Azure Purview.
• Drive adoption of data cataloguing, profiling, and classification tools.
• Collaborate with data stewards and compliance teams to ensure governance alignment.
• Implement role-based access control (RBAC), data masking, encryption, and tokenization.
• Responsible for technical design, coding, unit testing, test case documentation, and walkthroughs for all assigned Azure-related projects to support business and operational needs.
• Critically evaluate information gathered from multiple sources, reconcile conflicts, decompose high-level information into details, abstract up from low-level information to a general understanding, and distinguish user requests from the underlying true needs.
• Collaborate with developers and subject matter experts to establish the technical vision and analyze trade-offs between usability and performance needs.
• Mentor junior data engineers and contribute to technical best practices.

Qualification & Experience:

• 4+ years in data engineering with at least 2+ years on Azure cloud platform.
• Hands-on with Azure Data Factory, Azure Data Lake Gen2, Databricks, Synapse, and Purview.
• Proficiency in SQL, PySpark, Python, and data orchestration tools.
• Strong understanding of data architecture patterns – Lakehouse, Medallion, delta architecture.
• Familiarity with Snowflake, BigQuery, Redshift, or AWS Glue.
• Experience with data versioning and GitOps for data.
• Working knowledge of data observability, lineage, cataloguing, data quality.
• Exposure to privacy-enhancing techniques, access control, security auditing.
• Exposure to machine learning use cases in data engineering pipelines, data quality, anomaly detection, and schema change detection.
• Exposure to GenAI or agentic AI to Automate data cataloguing and metadata enrichment etc.
• Experience with NLP or LLMs in metadata extraction or data classification is a plus.

Personal Attributes/Traits:

• Consultative
• Socially confident
• Achievement oriented
• Decisive and action oriented
• Creative
• Eager to learn
• Resilient

Competencies:

• Business Foresight
• Influencing Others
• Fostering Partnerships With Customers
• Managing Transformation
• Driving Excellence
• Leading Teams
• Working Across Boundaries