Job summary

Expected Shortlisting Date:

17/01/2025

Planned Interview Date:

27/01/2025

Due to recent and continued expansion an exciting opportunity has arisen at NPIC (National Pathology Imaging Cooperative) based at LTHT for a Data Engineer. The candidate will work closely with the NPIC Senior Data Engineer to ensure that data architecture is effectively implemented, maintained and developed - liaising closely with clinical, technical and system development staff and system suppliers. The role is crucial for undertaking world leading research, enabling data and AI scientists to create new insights from digital pathology data. NPIC works collaboratively with Hospital Trusts, Universities and commercial companies ranging from spinouts and SMEs to large multinationals to expand to use and impact of digital pathology and artificial intelligence and as well as the development of quality assurance tools for digital pathology. Our research team plays a vital role realising our ambitions by helping to develop project opportunities and support projects through their lifecycle.

Main duties of the job

We are seeking to appoint an experienced individual who is driven, ambitious and passionate about improving patient care through the use of cutting-edge technology. The post-holder will be responsible for building, testing, maintaining data architecture and undertaking new developments in relation to NPICs research activity, building cohort creation pipelines for digital pathology and associated healthcare data. The Data Engineer will be responsible for design, implementation and delivery of platform enhancements, upstream data linkages, and data acquisition and wrangling. The role will work in a data rich environment together with data analysts and scientists. Data engineers will work with NPICs clinical and research platforms, performing integrations and implementing workflows and models for key use cases. Previous experience with SQL and Python is essential for this role and experience with NHS information governance procedures will be seen as advantageous.

About us

Leeds Teaching Hospitals NHS Trust has been at the forefront of Digital Pathology research and innovation for over 15 years. A £50m initiative from Office for Life Sciences and the Industrial Strategy Challenge Fund has led to the scale-up of the programme. NPIC builds on that advanced starting position, uniting a consortium of academic, clinical and industry partners with a common vision - to create a world-leading centre for digital pathology and artificial intelligence (AI) on which a UK industrial sector can grow and thrive. NPIC is multidisciplinary and highly motivated team that is working to deliver these challenging ambitions, as a part of this team you can expect a supportive environment and management that is receptive to the needs of individuals and allows for continued development through training and opportunities to be involved diverse aspects of a multimillion-pound program.

Leeds Teaching Hospitals is committed to our process of redeploying 'at risk' members of our existing workforce to new roles. As such, all our job adverts are subject to this policy and we reserve the right to close, delay or remove adverts while this process is completed. If you do experience a delay in the shortlisting stage of the recruitment cycle, please bear with us while this process is completed, and contact the named contact if you have any questions.

Job description

Job responsibilities

JOB DETAILS

Job Title: NPIC Data Engineer

Reports to: Digital Pathology Systems Lead

Band: 7

Department: Research and Innovation

Location : SJH

AfC Job No: 4059

JOB PURPOSE/SUMMARY

The National Pathology Imaging Co-operative is a £37m programme funded by the UK Industry Strategy Challenge Fund to accelerate the development of opportunities in Digital Pathology and Artificial Intelligence. We provide a welcoming, professional approach to the facility, a world class centre for training and knowledge exchange in digital pathology and artificial intelligence.

We are seeking to appoint an experienced individual who is driven, ambitious and passionate about improving patient care through the use of cutting-edge technology. The post-holder will be responsible for building, testing, maintaining data architecture and undertaking new developments while maintaining existing data models and reporting relating to the Deployment of the Digital Pathology national footprint.

The role is crucial in laying the foundation, enabling data scientists and data analysts to create new insights from data. The Data Engineer will be responsible for design, implementation and delivery of platform enhancements, upstream data lineage, and data acquisition and wrangling. The role will work in a data rich environment together with data analysts and data scientists. Data engineers will configure platform, performing initial integrations and implement workflows and models for key use cases.

The role will be responsible to ensure compounding value is created through shared data, common ontologies, and cross-organisation projects.

Previous experience in SQL data warehouse and ETL is essential for this role and experience in Power BI Data Models including use of DAX, Python and R will be seen as advantageous. You will be given ample growth opportunity to learn and gain experience in Power BI and data modelling.

The post holder will liaise with senior managers, clinicians and system users across a national footprint with, directorates and partner organisations. The post holder will work with partners and the local Informatics Departments to assist in the development of any necessary interfaces. In addition, the post holder will be responsible for the provision of an effective, efficient and robust information capture and analysis service to meet national, regional and local business requirements and essentially improve recording and reporting for the patients and services in the Trust.

The post holder will work closely with the Digital Pathology Systems Lead to ensure that the architecture is effectively implemented, maintained and developed liaising closely with clinical, technical and system development staff and system suppliers.

The post holder will manage a set of projects to professional standards.

The post holder will line manage a small team of specialists as well providing matrix-management of project staff with management of project budgets, this will vary by project.

PRINCIPAL DUTIES & AREAS OF RESPONSIBILITY

As a Data Engineer you will:

inspire best practice for data products and services within the team

build data engineering capability by providing technical leadership

work with other senior team members to identify, plan, develop and deliver data services

The day to day role involves working with NPIC Teams and the corporate data assets. The role will involve driving forward improvements in data services and supporting the organisation with on-going projects from a data perspective.

The responsibilities of the role will include:

Design, develop, construct, test and maintain architectures and processing workflows

Build robust, efficient and reliable data pipelines

Optimisation of information management approaches through an understanding of evolving business needs and technology capabilities.

Provide technical support and expert advice on best practices in Power BI, performance analysis of Power BI reports with recommendations for improvement to the other teams in healthcare analytics

Integrating external or new datasets into existing data pipelines

Develop solutions for data acquisition

Design, develop and maintain data acquisition API for central data Lake

Ensure architecture supports business requirements

Develop dataset processes for data modelling, mining, and production

Drive the collection of new data and refinement of existing data sources

Recommend ways to improve data reliability, efficiency, and quality

Applying feature transformations for machine learning models on new data.

Continuously monitoring and testing the system to ensure optimized performance

Use SQL queries, programming languages like DAX, M, Python & R and tools like DAX studio and tabular model in building data models and other data frames.

Identify ways to improve data reliability, efficiency and quality

Conduct research for advancements and incorporation of new functionalities to meet the reporting requirements of the organisation

Develop tools and techniques to manage large datasets with disparate sources for consolidated reporting.

Build and deploy sophisticated analytics programs, machine learning and statistical methods in data models and data frames for predictive and prescriptive modelling including finding hidden patterns using data and undertake tasks to automate data processing and data management

Ensure new solutions are produced in accordance with local policies and guidelines for all stages of development including design, build, testing and deployment

Develop and maintain a detailed working knowledge of the technical content of source systems to inform the design and on-going development of reporting

Undertake a rolling maintenance programme for published data models and reports built by the NPIC Team to provide quality assurance and ensure products are fit for purpose and meet the needs of users

Develop and maintain an in-depth knowledge of NHS data flows, datasets, information availability, scope, reliability and data quality issues. This includes a thorough understanding of the NHS data dictionary and knowledge of guidance for specific returns.

Develop and maintain effective working relationships with colleagues across the Trust to enable the continuous monitoring, development and improvement of reporting solutions

Represent the NPIC Deployment Team at internal and external meetings

As a subject matter expert for technical solutions and data management; act as a reference point and provide technical guidance and advice as required

Build validation checks and processes into the creation of data models, data frames, reports and dashboards to ensure data integrity at all stages of the development and implementation of reporting

Contribute to the development of the NPIC Power BI reporting, NPIC data warehouse

Make decisions where appropriate to escalate risks and issues that may impact on the completeness and accuracy of reporting

Ensure all handling of patient related data is in accordance with information governance, Caldecott and data protection policies.

Ensure all handling of patient related data is in accordance with Trust policies and procedures.

Deal with highly complex data management issues and identify and use knowledge of systems and data flows to resolve areas of dispute.

Interpret local and national guidance and develop, produce and recommend changes to policies and procedures as required to ensure compliance

Continually review the work undertaken previously and implement changes and improvements as dictated by customer feedback, changes to guidance and local changes such as ward reconfigurations

Other duties as commensurate with the grade and role as agreed with line manager.

Person Specification

Experience

Essential

Experience of working with remote and multi-disciplinary teams and driving forward complex work programmes. Expert user of Microsoft office products including advanced functions of MS Excel Experience of working in an NHS information environment in an information management role Significant experience of designing, developing and publishing reports and dashboards across a broad range of areas using different reporting tools such as SSRS. Have good project management skills, including the ability to juggle competing priorities and tight deadlines for self and team, and work autonomously to drive own work programme and that of team members.

Qualifications

Essential

Educated to degree level or equivalent level of knowledge gained through relevant work experience

Desirable

Educated to masters level. A relevant qualification in information management

Other Criteria

Essential

SQL Fluent in SQL-based systems like MySQL, PostgreSQL Microsoft SQL Server Significant experience of extracting and processing data from multiple source system using SQL. ETL solutions Must have ETL skills to build processes to move data between systems eg Microsoft ADF API design and implementation Understanding of best practice when setting up APIs and the coding ability to develop and deliver the design. Big Data Tools Strong understanding and Knowledge of big data tools Coding ability Proficiency in Python

Skills & behaviours

Essential

Can work autonomously to achieve agreed objectives without reference to a manager. Excellent problem-solving skills. Capable of working with clinical & non-clinical staff at all levels. Strong written & verbal communication skills.