Data Engineering Team Lead

Summary data42 is Novartis ground-breaking initiative that harnesses the power of R D data in one of the largest and most diverse datasets in the pharmaceutical industry to reimagine medicine data42 applies machine learning artificial intelligence and sophisticated analytics to generate new insights that increase our understanding of disease and medicines improve R D decision-making and ultimately reimagine drug discovery and development And to take this a step further we are expanding data42 to create a first-of-its-kind diverse ecosystem A key aspect of the program is to centralize streamline Real world data collected across Novartis to enable secondary research RWD pipeline team with focus to streamline end-to-end operations developing new pipelines and building products to help RWE teams with data driven insights across the RDC continuum The position will work closely with RWD pipeline lead and data engineering team Major accountabilities Responsible for the detailed design development and delivery of pipeline solutions to address business needs Define solution architecture and develop detailed process designs for day-to-day operation of the preclinical pipelines Ensure solution designs align with the overall data42 architecture Facilitate peer reviews and secure sign-off from the RWD pipeline lead and business stakeholders Coordinate and oversee the data engineering teams day-to-day activities review and merge pull requests manage the product delivery roadmap and track progress on user stories in the JIRA board Review pipeline releases and migration activities conduct code reviews serve as technical SME for the team Lead and actively contribute to documentation efforts and audit readiness activities Proactively manage release notes and communicate pipeline updates to stakeholders Initiate and review architecture changes in the pipeline ensuring regular optimization redesign and continuous improvement Work on application development in support of business needs e g dashboards LLM AIP based applications Support end-users with their technical issues Key performance indicators Achieve high level of quality and timeliness of delivering preclinical pipeline deliverables as assessed by the RWD pipeline lead Ensure that technical documentation and pipeline management are aligned with standards and effectively maintained Collaboration with other data42 product teams and product technical leads Ability and effectiveness in training mentoring and coordinating internal and external analysts assigned to the same project as assessed by the functional operational manager Job Dimensions The role involves close collaboration with the RWD pipeline lead to define the roadmap and delivery plans Work with the team to ensure timely execution of deliverables maintain technical documentation and manage release communications to end users Minimum Requirements Education Bachelors Master s degree in Computer Science Applied Mathematics Engineering or any other technology related field equivalent of the same in working experience may also be accepted Work Experience 8 years IT experience with 6 years in Data Engineering on Big Data platforms Ability to work and lead cross-functional team in a matrix organization Led and mentored technical teams for 2 years Must have experience managing pipeline development activities with strong project management skills Strong communication skills with the ability to effectively collaborate with cross functional teams and stakeholders Proficient in working with Git workflow for project execution a strong understanding of DevOps CI CD framework is essential Had actively participated in agile work practices and coordinated with team members to ensure smooth project execution Expertise in Python PySpark and Spark Hands-on experience with JIRA and Confluence for technical documentation Responsible for technical documentation audit readiness and release management Strong Analytical thinking and problem-solving skills Expertise on Palantir Foundry Platform using Code Repository Code Workbook Data Connection etc components to develop data pipelines Strong knowledge of AI ML concepts with hands-on experience Good knowledge of application development including LLM models using AIP Knowledge of Real-World Data common data models will be desirable Experience with Snowflake DataBricks will be desirable Knowledge of High Performance Computing environment is desirable Skills Back-End Development Code Analysis OOD Object-Oriented Design Data Wrangling Software Documentation Software Data Engineering Software Data Testing Analytical thinking RWD Palantir Foundry Unit Testing HPC Languages Fluent English Oral and Written Skills Desired Back-End Development Code Analysis OOD Object-Oriented Design REST Representational State Transfer Software Design Software Documentation Software Engineering Software Testing Palantir Foundry Unit Testing RWD APIs Analytical thinking Skills Desired Algorithms Computer Programming Computer Science Computer Vision Data Science People Management R D Research And Development Waterfall Model