Health Education England

Machine Learning for Improved Insight (Discovery & Alpha)

Incomplete applications

8
Incomplete applications
8 SME, 0 large

Completed applications

18
Completed applications
13 SME, 5 large
Important dates
Opportunity attribute name Opportunity attribute value
Published Friday 13 December 2019
Deadline for asking questions Friday 20 December 2019 at 11:59pm GMT
Closing date for applications Friday 27 December 2019 at 11:59pm GMT

Overview

Overview
Opportunity attribute name Opportunity attribute value
Summary of the work HEE is seeking a partner with expertise in AI to assist the organisatioin in understanding how machine learning practices can be harnessed to better address challenges in analysis and prediction across the trainee doctor management and workforce planning problem space.
Latest start date Saturday 1 February 2020
Expected contract length To be delivered by April 2020
Location London
Organisation the work is for Health Education England
Budget range Total budget for Discovery & Alpha is £50k (inclusive of VAT and any incurred infrastructure costs).

Note - HEE currently use Azure platform.

About the work

About the work
Opportunity attribute name Opportunity attribute value
Why the work is being done This discovery and alpha project seeks to help HEE understand how machine learning (ML) practices can be harnessed within the organisation to better address the significant and ongoing challenges faced with regards to analysis and prediction across the medical trainee management and workforce planning problem space.

This work comes in response to the urgent workforce challenges outlined in the Long Term Plan and its ambition to address these by ‘making better use of data and digital technology’ as well as recommendations within the Topol Review to utilise AI to better understand the uniqueness of each individual with the aim to deliver education, training, and workforce planning on a far more rational, efficient and tailored basis.

Both Discovery and Alpha phases to be complete by April 2020.
Problem to be solved A number of potential use cases have been identified below. These will be finessed during Discovery to a suitable subset for development within Alpha.

• classifying staff groups carrying high retention risk
• identifying determinants behind staff attrition, knowing more about why individuals intend to leave
• predicting ARCP outcomes and identifying trainees in difficulty early
• developing models to improve rotation scheduling of trainees
• improving current predictive analytics by identifying relationships across multiple trainee, workforce, education and learning datasets when combined with social, economical, and environmental data

Objectives
Working in partnership with HEE to
• identify the different types of ML user
• establish use cases for Alpha and success measures of implementation
• evaluate the viability, cost-effectiveness, and ethics of utilising ML
• identify appropriate development platform
• identify constraints of future ML projects inc. compliance, governance, bias
• develop alpha process for utilising ML within HEE
• prototype and evaluate ML models against use cases
• develop staff and set foundations for establishing an ML capability within HEE
• identify what is required for HEE to build and support ML products that better meet user needs
• identify whether to move to beta or repeat discovery/alpha
Who the users are and what they need to do As a workforce planning lead
I need to identify segments of the workforce who are at a higher risk of leaving the NHS
So that I can put suitable plans in place to address shortfalls

As a workforce planning lead
I need to understand the factors which contribute most to indivdiuals leaving their role/organisation/NHS
So that I can develop tailored rentention strategies

As an educational supervisor
I need to be able to identify medical trainees who may be experiencing difficulty in their training
So that I can take early action and provide support

As a BI analyst
I need to be able to run ML models against datasets
So that I can use information in the reports and dashboards I build

As a BI Developer
I need a platform on which to build, train, and deploy ML models
So that I can prototype and evalute predictive modelling on new and existing use cases

Note - user identifcation is an objective of the Discovery phase. Above are examples only.
Early market engagement
Any work that’s already been done Prelimary work has already been conducted regarding identification and suitabaility of uses cases. It is expected that this will help expedite Discovery phase.
Existing team Business Intelligence and Systems Development Team,
HEE London

BI team are skilled in SQL Server and Tableau.
Sys Dev team support variety of platforms/tech utilising Sharepoint, C#, JavaScript, Python, etc.

Team are keen to move into ML. Some training has taken place (mainly using Azure ML and a little Python/R) but no ML models deployed.
Current phase Discovery

Work setup

Work setup
Opportunity attribute name Opportunity attribute value
Address where the work will take place Health Education England,
Stewart House
32 Russell Square
London
WC1B 5DN
Working arrangements We require individuals on-site at appropriate times to work with our users and particularly in partnership with our analysts and developers. This is so that skills and knowledge can be shared and that HEE area able to support and develop anything built in Alpha.
Security clearance

Additional information

Additional information
Opportunity attribute name Opportunity attribute value
Additional terms and conditions

Skills and experience

Buyers will use the essential and nice-to-have skills and experience to help them evaluate suppliers’ technical competence.

Skills and experience
Opportunity attribute name Opportunity attribute value
Essential skills and experience
  • have experience designing and deploying ML services within the NHS
  • have experience working with NHS-specific datasets
  • be able to work in partnership with HEE so that knowledge and skills can be shared to support any alpha services that are developed
Nice-to-have skills and experience

How suppliers will be evaluated

All suppliers will be asked to provide a written proposal.

How suppliers will be evaluated
Opportunity attribute name Opportunity attribute value
How many suppliers to evaluate 5
Proposal criteria
  • Technical solution
  • Approach and methodology
  • Estimated timeframes for the work
  • How the approach or solution meets our goals
Cultural fit criteria
  • Work as a team with our organisation and other suppliers
  • Take responsibility for their work
  • Share knowledge and experience with other team members
  • Be transparent and collaborative when making decisions
Payment approach Fixed price
Additional assessment methods Work history
Evaluation weighting

Technical competence

45%

Cultural fit

10%

Price

45%

Questions asked by suppliers

Questions asked by suppliers
Supplier question Buyer answer
1. Are you only interested in organisations with has experience designing and deploying ML services within the NHS? Preference will be given to such organisations as it is expected that good domain knowledge in this area is required in order to assure delivery of alpha within a relatively short timeline.

If supplier can demonstrate NHS domain knowledge through other means, they too will be considered.
2. What specific data/analysis and the application of ML within the health sector preclude organisations with expertise and experience in other fields/sectors? Whilst experience in other fields/sectors will be considered within applications, some knowledge of and experience working with NHS-specific datasets, collections, systems, and analyses across any of workforce, trainee, learner, activity, finance, performance functions is essential.