Azure Data Engineer

Job Requirements :

Mandatory Skills :

- Bachelors or higher degree in Computer Science or a related discipline; or equivalent (minimum 7+ years work experience).

- At least 6+ years of consulting or client service delivery experience on Azure Microsoft data engineering.

- At least 4+ years of experience in developing data ingestion, data processing and analytical pipelines for big data, relational databases such as SQL server and data warehouse solutions such as Synapse/Azure Databricks, Microsoft Fabric

- Hands-on experience implementing data ingestion, ETL and data processing using Azure services: Fabric, onelake, ADLS, Azure Data Factory, Azure Functions, services in Microsoft Fabric etc.

- Minimum of 5+ years of hands-on experience in Azure and Big Data technologies such as Fabric, databricks, Python, SQL, ADLS/Blob, pyspark/SparkSQL.

- Minimum of 3+ years of RDBMS experience

- Experience in using Big Data File Formats and compression techniques.

- Experience working with Developer tools such as Azure DevOps, Visual Studio Team Server, Git, etc.

Preferred Skills :

Technical Leadership & Demo Delivery :

- Provide technical leadership to the data engineering team, guiding the design and implementation of data solutions.

- Deliver compelling and clear demonstrations of data engineering solutions to stakeholders and clients, showcasing functionality and business value.

- Communicate fluently in English with clients, translating complex technical concepts into business-friendly language during presentations, meetings, and consultations.

ETL Development & Deployment on Azure Cloud :

- Design, develop, and deploy robust ETL (Extract, Transform, Load) pipelines using Azure Data Factory (ADF), Azure Synapse Analytics, Azure Notebooks, Azure Functions, and other Azure services.

- Ensure scalable, efficient, and secure data integration workflows that meet business requirements.

- Preferably to have following skills Azure doc intelligence, custom app, blob storage

- Design and develop data quality frameworks to validate, cleanse, and monitor data integrity.

- Perform advanced data transformations, including Slowly Changing Dimensions (SCD Type 1 and Type 2), using Fabric Notebooks or Databricks.

- Preferably to have following skills Azure doc intelligence, custom app, blob storage

Microsoft Certifications :

- Hold relevant role-based Microsoft certifications, such as:

- DP-203 : Data Engineering on Microsoft Azure

- AI-900 : Microsoft Azure AI Fundamentals.

- Additional certifications in related areas (e.g., PL-300 for Power BI) are a plus.

- Azure Security & Access Management:

- Strong knowledge of Azure Role-Based Access Control (RBAC) and Identity and Access Management (IAM).

- Implement and manage access controls, ensuring data security and compliance with organizational and regulatory standards on Azure Cloud

Additional Responsibilities & Skills :

- Team Collaboration : Mentor junior engineers, fostering a culture of continuous learning and knowledge sharing within the team.

- Project Management : Oversee data engineering projects, ensuring timely delivery within scope and budget, while coordinating with cross-functional teams.

- Data Governance : Implement data governance practices, including data lineage, cataloging, and compliance with standards like GDPR or CCPA.

- Performance Optimization : Optimize ETL pipelines and data workflows for performance, cost-efficiency, and scalability on Azure platforms.

- Cross-Platform Knowledge : Familiarity with integrating Azure services with other cloud platforms (e.g., AWS, GCP) or hybrid environments is an added advantage.

Soft Skills & Client Engagement :

- Exceptional problem-solving skills with a proactive approach to addressing technical challenges.

- Strong interpersonal skills to build trusted relationships with clients and stakeholders.

- Ability to manage multiple priorities in a fast-paced environment, ensuring high-quality deliverables.