Reply - Data Engineer - Python/Spark

Description :- Reply is looking for a Data Engineer with experience building and contributing to the design of database systems, both normalized transactional systems and dimensional reporting systems.- Strong experience with SQL Server as a database engine, as well as Microsoft and Databricks technology experience implementing Big Data with Advanced Analytics solutions.- Successful candidates will have experience and skill in providing solutions for storing, retrieving, transforming and aggregating data to support line of business applications as well as reporting systems.- The Data Engineer will work with customers to deliver solutions utilizing strong business, technical and data modelling skills.- This position will represent Valorem's approach to data visualization and information delivery solutions and as such must demonstrate proficiency with Power BI and Azure Data Services.Key Responsibilities :- Design and implement general architecture for complex data systems- Translate business requirements into functional and technical specifications- Design and implement lakehouse architecture- Develop and manage cloud-based data architecture and reporting solutions- Apply data modelling principles for relational and dimensional data structures- Design Data Warehouses following established principles (e.g., Kimball, Inmon)- Create and manage source-to-target mappings for ETL/ELT processes- Mentor junior engineers and contribute to architectural decisions and code reviewsMinimum Qualifications :- Bachelors degree in computer science, Computer Engineering, MIS, or related field- 5+years of experience with Microsoft SQL Server and strong proficiency in T- SQL, SQL performance tuning (Indexing, Structure, Query Optimization)- 5+years of experience in Microsoft data platform development and implementation- 5+years of experience with PowerBI or other competitive technologies- 3+years of experience in consulting, with a focus on analytics and data solutions- 2+ years of experience with Databricks, including Unity Catalog, Databricks SQL, Workflows, and Delta Sharing- Proficiency in Python and Apache Spark- Develop and manage Databricks notebooks for data transformation, exploration, and model deployment- Expertise in Microsoft Azure services, including Azure SQL, Azure Data Factory (ADF), Azure Data Warehouse (Synapse Analytics), Azure Data Lake, and Stream AnalyticsPreferred Qualifications :- Experience with Microsoft Fabric- Familiarity with CI/CD pipelines and infrastructure-as-code tools like Terraform or Azure Resource Manager (ARM)- Knowledge of taxonomies, metadata management, and masterdata management- Familiarity with data stewardship, ownership, and data quality managementExpertise in Big Data technologies and tools : - BigData Platforms : HDFS, MapReduce, Pig, Hive - General DBMS experience with Oracle, DB2, MySQL, etc - No SQL databases such as HBase, Cassandra, DataStax, MongoDB, CouchDB, etc- Experience with non-Microsoft reporting and BI tools, such as Qlik, Cognos, MicroStrategy, Tableau, etcAbout Reply :- Reply specializes in the design and implementation of solutions based on new communication channels and digital media.- Reply is a network of highly specialized companies supporting global industrial groups operating in the telecom and media, industry and services, banking, insurance and public administration sectors in the definition and development of business models enabled for the new paradigms of AI, cloud computing, digital media and the Internet of Things.- Reply services include Consulting, System Integration and Digital Services (ref:hirist.tech)