Senior Data Engineer

Microsoft is a company where passionate innovators come to collaborate envision what can be and take their careers further This is a world of more possibilities more innovation more openness and the sky is the limit thinking in a cloud-enabled world Microsoft s Azure Data engineering team is leading the transformation of analytics in the world of data with products like databases data integration big data analytics messaging real-time analytics and business intelligence The products our portfolio include Microsoft Fabric Azure SQL DB Azure Cosmos DB Azure PostgreSQL Azure Data Factory Azure Synapse Analytics Azure Service Bus Azure Event Grid and Power BI Our mission is to build the data platform for the age of AI powering a new class of data-first applications and driving a data culture Within Azure Data the Microsoft Fabric platform team builds and maintains the operating system and provides customers a unified data stack to run an entire data estate The platform provides a unified experience unified governance enables a unified business model and a unified architecture At the Fabric Telemetry Data Platform team we are building a next-generation data platform to revolutionize how data engineers and partner teams work Our platform is designed to be highly scalable resilient and automated supporting high-performance querying and self-service capabilities for partner teams Our goal is to enhance the productivity of data engineers streamline metric development and enable robust data monitoring As a Senior Data Engineer on our team you will play a key role in shaping the future of our next-generation data platform Using the latest Azure technologies you will design and build scalable data pipelines develop tools for metadata management and create job monitoring and notification services You will also optimize data processing and storage implement automated anomaly detection and streamline daily operations through automation Additionally you will mentor junior engineers lead technical discussions and drive best practices in data engineering We are looking for Senior Data Engineers who want to build the Future of Big Data Data Insights and become Full Stack Data Professionals We do not just value differences or different perspectives We seek them out and invite them in so we can tap into the collective power of everyone in the company As a result our customers are better served Responsibilities o You will design develop and maintain data pipelines including solutions for data collection cleansing transformation and usage ensuring accurate data ingestion and readiness for downstream analysis visualization and AI model training You will build frameworks to validate data quality and completeness detect anomalies enhance data pipeline resiliency and support unit and integration testing You will design and implement end-to-end software life cycles encompassing design development CI CD service reliability recoverability and participation in agile development practices including on-call rotation You will write code to implement performance monitoring protocols across data pipelines building visualizations and aggregations to monitor pipeline health You ll also implement solutions and self-healing processes that minimize points of failure across multiple product features You will anticipate data governance needs designing data modeling and handling procedures to ensure compliance with all applicable laws and policies You will implement and enforce security and access control measures to protect sensitive resources and data You will perform database administration tasks including maintenance and performance monitoring You will mentor junior engineers lead technical discussions and drive best practices in data engineering Embody our and Qualifications Required Minimum Qualifications o Bachelor s Degree in Computer Science Math Software Engineering Computer Engineering or related field AND 6 years experience in data engineering software development or data modeling work OR Master s Degree in Computer Science Math Software Engineering Computer Engineering or related field AND 5 years experience in data engineering software development or data modeling work OR equivalent experience 4 years of experience in software or data engineering with proven proficiency in C Java or equivalent 4 years in one scripting language for data retrieval and manipulation e g SQL or KQL 4 years of experience in big data scripting languages like Python SCOPE SQL or equivalent 4 years of experience with ETL and data cloud computing technologies including Azure Data Lake Azure Data Factory Azure Synapse Azure Logic Apps Azure Functions Azure Data Explorer and Power BI or equivalent platforms Other Requirements Ability to meet Microsoft customer and or government security screening requirements are required for this role These requirements include but are not limited to the following specialized security screenings Microsoft Cloud Background Check This position will be required to pass the Microsoft Cloud background check upon hire transfer and every two years thereafter Preferred Additional Qualifications 2 years of demonstrated experience implementing data governance practices including data access security and privacy controls and monitoring to comply with regulatory standards Experience with big data technologies such as Hadoop Hive Spark Experience with distributed database engines like Microsoft Synapse Equal Opportunity Employer EOP Microsoft is an equal opportunity employer All qualified applicants will receive consideration for employment without regard to age ancestry color family or medical care leave gender identity or expression genetic information marital status medical condition national origin physical or mental disability political affiliation protected veteran status race religion sex including pregnancy sexual orientation or any other characteristic protected by applicable laws regulations and ordinances If you need assistance and or reasonable accommodation due to a disability during the application or the recruiting process please send a request via the Accommodation request form Accessibility Microsoft Careers azdat azuredata fabricdata dataintegration azure synapse databases bigdata