Azure Data Engineer/Developer

Job Summary

As a Microsoft Fabric Data Engineer/Developer, you will play a vital role in designing, developing, and implementing robust and scalable data solutions within the Microsoft Fabric ecosystem. You will collaborate closely with data architects, business stakeholders, and cross-functional teams to transform raw data into actionable insights, driving informed decision-making across the organization. If you are passionate about data engineering, possess a strong technical background, and excel in collaborative environments, we invite you to join our growing data team.

Career Level - IC2

Responsibilities

1. Microsoft Fabric Development:
2. Design, develop, and deploy end-to-end data solutions using various components of Microsoft Fabric, including Lakehouse, Data Warehouse, Data Factory, and Data Engineering.
3. Implement and optimize data pipelines for ingestion, transformation, and curation of data from diverse sources (e.g., Azure Data Lake Storage Gen2, on-premises databases, APIs, third-party systems).
4. Develop and optimize data models within Microsoft Fabric, ensuring adherence to best practices for performance, scalability, and data quality.

5. Utilize Power BI for data visualization and reporting, ensuring seamless integration with Fabric data assets.

6. Azure Data Services Integration:

7. Demonstrate strong hands-on experience with core Microsoft Azure data services, including Azure Data Factory (for ETL/ELT orchestration), Azure Databricks (for advanced analytics and processing), and Azure Data Lake Storage Gen2.

8. Integrate Microsoft Fabric solutions with existing Azure data services and other enterprise systems.

9. Data Architecture & Governance:

10. Contribute to the design and implementation of robust, scalable, and secure data architectures within the Microsoft Fabric platform.
11. Implement data quality, validation, and reconciliation processes to ensure data integrity and accuracy.

12. Apply data governance best practices, including security, access controls (e.g., role-based access control), and compliance within Fabric and Azure Purview.

13. Documentation & Knowledge Sharing:

14. Maintain comprehensive documentation for data architectures, pipelines, data models, and processes.

15. Stay updated with the latest advancements in Microsoft Fabric, Azure data services, and data engineering best practices.

16. Qualifications & Skills

17. Mandatory:
18. Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
19. 4-7 years of professional experience as a Data Engineer, Data Developer, or in a similar role.
20. Hands-on experience with Microsoft Fabric, including its core components (Lakehouse, Data Warehouse, Data Factory, Data Engineering).
21. Strong expertise in Microsoft Azure data services:
22. Azure Data Factory (ADF)

23. Azure Data Lake Storage Gen2

24. Proven experience in designing, developing, and maintaining scalable data pipelines.

25. Solid understanding of data warehousing concepts, dimensional modeling, and data lakehouse architectures.
26. Proficiency in SQL for data manipulation and querying.
27. Experience with version control systems (e.g., Git, Azure Repos).
28. Strong analytical and problem-solving skills with meticulous attention to detail.

29. Excellent communication skills (written and verbal) and the ability to collaborate effectively with cross-functional teams.

30. Good-to-Have:

31. Certification in Microsoft Azure or Microsoft Fabric.
32. Experience with cloud-based data platforms, such as Amazon Web Services (AWS) or Google Cloud Platform (GCP).
33. Knowledge of data governance frameworks and best practices.
34. Additional Notes

35. Ideal to have some background knowledge around Finance / Investment Banking / Fixed Income / OCIO Business

36. Self-Assessment Questions

37. To help you determine if this role is a good fit, please consider the following questions:
38. 1)Can you describe your experience with Microsoft Fabric and its core components, highlighting specific projects or accomplishments?
39. 2)How do you ensure data quality, validation, and reconciliation in your data pipelines, and can you provide an example from a previous project?
40. 3)Can you explain your approach to data governance, including security, access controls, and compliance, and how you've applied this in a previous role?
41. 4)How do you stay up-to-date with the latest advancements in Microsoft Fabric, Azure data services, and data engineering best practices?

5)Can you provide an example of a complex data problem you've solved in the past, highlighting your analytical and problem-solving skills.