Data Engineer

Position: Data Engineer (Sr. Analyst)JOB SUMMARYThe Data Engineer will have responsibility to use various methods to transform raw data into usefuldata systems and should have strong analytical skills and the ability to combine data from differentsources. The person must have familiarity with several programming languages and knowledge oflearning machine methods.ESSENTIAL JOB FUNCTIONSThe primary responsibilities of this position include: Development of high-quality code (efficient and maintainable) in Python, PySpark, SQL toingest, process and transform large datasets Design and implement scalable data pipelines using ETL and ELT processes in Azure DataFactory, Databricks and Snowflake Work closely with cross-functional teams to understand data requirements and deliversolutions that meet business needs Design, develop and optimize data models in Databricks and Snowflake environments Ensure data quality and integrity by establishing and enforcing data modelling standardsand best practices Support and optimize data tools and associated cloud environments for consumption bydownstream systems, data analysts and data scientists Ensure code, configuration and other technology artifacts are delivered within agreed timeschedules and any potential delays are escalated in advance Collaborate across developers as part of a SCRUM team ensuring collective teamproductivity Implement CI/CD pipelines and DevOps practices using Azure DevOps Ensure that 100% of code is well documented and maintained in source code repositoryTECHNICAL SKILLS AND EXPERIENCE 5-7 years professional experience as a Data engineer/Data Modeler Strong experience working with Microsoft Azure Data platform services like: Azure Data LakeGen2, Azure Data Factory, Azure Databricks Advanced in building ETL and data pipelines on Databricks using data engineeringlanguages like Python, PySpark. Hands-on experience with Snowflake Proficiency in SQL Proven experience with all aspects of the ETL processes (Data Sourcing, Transformations,Data Quality, etc.…) Familiarity with Azure DevOps and CI/CD pipelines Advanced knowledge of data warehousing concepts and best practices Experience with relational and dimensional database modelling (Relational, Kimball, DataVault, Star Schemas)NON-TECHNICAL SKILLS AND EXPERIENCE Creative thinking, problem solving and decision making. Proactive and self-starter. Has high degree of initiative, dependability and ability to work with little supervision. Responds to and handles unfamiliar situations with little supervisory guidance. Ability to solve problems that are complex, varied and only mildly related to those seenbefore. Simply determining what the problem is requires major individual effort and/orconsultation with others within the department. A high degree of analytical ability and inductive thinking may be required to solve highlyintricate, technically complex problems. Must be able to develop new and nonstandardapproaches. Ability to complete work that requires developing imaginative and complex methods,procedures, products or systems. Strong understanding of the Business Intelligence lifecycle, from requirements gathering todevelopment and acceptance. Has excellent written and verbal communication skills, interpersonal and collaborative skills,and the ability to communicate technical concepts to both technical and nontechnicalaudiences. Requires regular external contacts to carry out organization programs and to explainspecialized matters. Position requires contacts with customers or the public involving personsof some importance on matters requiring explanation and discussion and where the effort toinfluence becomes important. Strong communication and interpersonal skills to work within a team environment. Excellent organizational and troubleshooting skills with attention to detail. Strong presentation and documentation skills. Must have high proficiency in spoken and written English.EDUCATION Bachelor's degree required in Mathematics, Statistics, Computer Sciences, or related field.