Senior Data Engineer

Guidance (Text Only)Job Posting Content Content for the Responsibilities and Qualifications section of the job posting should be leveraged from the Role Library found on HRWeb: HRweb - Role Library. For guidance on how to create an impactful job description, reference the Job Posting Aid. For roles in the EU or UK: Years' experience should not be included. Accessibility Guidance - Use only the "Paste as Text" button in the text editor to transfer content, especially if using Microsoft Word, PPT, Outlook or OneNote - Change the Font Family to Calibri​ - Do not use special characters ​ - Use only one sub-heading for each of the 3 sections – e.g. - Responsibilities -> Key responsibilities​ - Qualifications -> Skills required​ - Only format content within the text editor (e.g. create bullets and bold styling using the associated buttons in the editor)​ - Remove all blank characters at the end of a sentence by pressing Enter to end each line Follow these suggestions for what to avoid in your job advertisement: - Do not mention unannounced products, technologies or project codenames. - Do not discuss future product milestones or roadmaps. - Do not mention executive names. - Do not use a team name unless well known to external audiences. - Do not discuss internal company goals or business strategies. - Do not discredit current Microsoft products in market or use language with a negative connotation when describing Microsoft products or services. - Do not use stats that may be newsworthy such as downloads, units, market share, sales, revenue, or other business metrics without first consulting your Public Relations team. - Do not mention specific competitors. - Absolute security, safety, and privacy do not exist. Therefore, do not make any statements that express or imply absolute security, safety, and privacy. - Do not reference internal URLs, job levels, or terminology. Responsibilities (Text Only)​​• You will design, develop, and maintain data pipelines, including solutions for data collection, cleansing, transformation, and usage, ensuring accurate data ingestion and readiness for downstream analysis, visualization, and AI model training. • You will build frameworks to validate data quality and completeness, detect anomalies, enhance data pipeline resiliency, and support unit and integration testing. • You will design and implement end-to-end software life cycles, encompassing design, development, CI/CD, service reliability, recoverability, and participation in agile development practices, including on-call rotation. • You will write code to implement performance monitoring protocols across data pipelines, building visualizations and aggregations to monitor pipeline health. You'll also implement solutions and self-healing processes that minimize points of failure across multiple product features. • You will anticipate data governance needs, designing data modeling and handling procedures to ensure compliance with all applicable laws and policies. • You will implement and enforce security and access control measures to protect sensitive resources and data. • You will perform database administration tasks, including maintenance, and performance monitoring. • You will mentor junior engineers, lead technical discussions, and drive best practices in data engineering. ​ Embody our culture and values Qualifications (Text Only)Required/Minimum Qualifications ​​• Bachelor's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 6 years' experience in data engineering, software development, or data modeling work. • OR Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 5 years' experience in data engineering, software development, or data modeling work. • OR equivalent experience. • 4 years of experience in software or data engineering, with proven proficiency in C#, Java, or equivalent. • 4 years in one scripting language for data retrieval and manipulation (e.g., SQL or KQL). • 4 years of experience in big data scripting languages like Python, SCOPE, SQL or equivalent. • 4 years of experience with ETL and data cloud computing technologies, including Azure Data Lake, Azure Data Factory, Azure Synapse, Azure Logic Apps, Azure Functions, Azure Data Explorer, and Power BI or equivalent platforms Other Requirements Ability to meet Microsoft, customer and/or government security screening requirements arerequired for this role. These requirements include, but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: - This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter. Preferred/Additional Qualifications • 2 years of demonstrated experience implementing data governance practices, including data access, security and privacy controls and monitoring to comply with regulatory standards. • Experience with big data technologies such as: Hadoop, Hive, Spark. • Experience with distributed database engines like Microsoft Synapse. Equal Opportunity Employer (EOP) Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, color, family or medical care leave, gender identity or expression, genetic information, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran status, race, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or reasonable accommodation due to a disability during the application or the recruiting process, please send a request via the Accommodation request form (Accessibility | Microsoft Careers). Benefits/perks listed below may vary depending on the nature of your employment with Microsoft and the country where you work. #azdat #azuredata ​​#fabricdata #dataintegration #azure #synapse #databases #bigdata​