Awarded to ICS.AI LTD

Start date: Tuesday 12 February 2019
Value: £41,371
Company size: SME
Health Education England

Machine Learning for Improved Insight (Discovery & Alpha)

8 Incomplete applications

8 SME, 0 large

18 Completed applications

13 SME, 5 large

Important dates

Friday 13 December 2019
Deadline for asking questions
Friday 20 December 2019 at 11:59pm GMT
Closing date for applications
Friday 27 December 2019 at 11:59pm GMT


Summary of the work
HEE is seeking a partner with expertise in AI to assist the organisatioin in understanding how machine learning practices can be harnessed to better address challenges in analysis and prediction across the trainee doctor management and workforce planning problem space.
Latest start date
Saturday 1 February 2020
Expected contract length
To be delivered by April 2020
Organisation the work is for
Health Education England
Budget range
Total budget for Discovery & Alpha is £50k (inclusive of VAT and any incurred infrastructure costs).

Note - HEE currently use Azure platform.

About the work

Why the work is being done
This discovery and alpha project seeks to help HEE understand how machine learning (ML) practices can be harnessed within the organisation to better address the significant and ongoing challenges faced with regards to analysis and prediction across the medical trainee management and workforce planning problem space.

This work comes in response to the urgent workforce challenges outlined in the Long Term Plan and its ambition to address these by ‘making better use of data and digital technology’ as well as recommendations within the Topol Review to utilise AI to better understand the uniqueness of each individual with the aim to deliver education, training, and workforce planning on a far more rational, efficient and tailored basis.

Both Discovery and Alpha phases to be complete by April 2020.
Problem to be solved
A number of potential use cases have been identified below. These will be finessed during Discovery to a suitable subset for development within Alpha.

• classifying staff groups carrying high retention risk
• identifying determinants behind staff attrition, knowing more about why individuals intend to leave
• predicting ARCP outcomes and identifying trainees in difficulty early
• developing models to improve rotation scheduling of trainees
• improving current predictive analytics by identifying relationships across multiple trainee, workforce, education and learning datasets when combined with social, economical, and environmental data

Working in partnership with HEE to
• identify the different types of ML user
• establish use cases for Alpha and success measures of implementation
• evaluate the viability, cost-effectiveness, and ethics of utilising ML
• identify appropriate development platform
• identify constraints of future ML projects inc. compliance, governance, bias
• develop alpha process for utilising ML within HEE
• prototype and evaluate ML models against use cases
• develop staff and set foundations for establishing an ML capability within HEE
• identify what is required for HEE to build and support ML products that better meet user needs
• identify whether to move to beta or repeat discovery/alpha
Who the users are and what they need to do
As a workforce planning lead
I need to identify segments of the workforce who are at a higher risk of leaving the NHS
So that I can put suitable plans in place to address shortfalls

As a workforce planning lead
I need to understand the factors which contribute most to indivdiuals leaving their role/organisation/NHS
So that I can develop tailored rentention strategies

As an educational supervisor
I need to be able to identify medical trainees who may be experiencing difficulty in their training
So that I can take early action and provide support

As a BI analyst
I need to be able to run ML models against datasets
So that I can use information in the reports and dashboards I build

As a BI Developer
I need a platform on which to build, train, and deploy ML models
So that I can prototype and evalute predictive modelling on new and existing use cases

Note - user identifcation is an objective of the Discovery phase. Above are examples only.
Early market engagement
Any work that’s already been done
Prelimary work has already been conducted regarding identification and suitabaility of uses cases. It is expected that this will help expedite Discovery phase.
Existing team
Business Intelligence and Systems Development Team,
HEE London

BI team are skilled in SQL Server and Tableau.
Sys Dev team support variety of platforms/tech utilising Sharepoint, C#, JavaScript, Python, etc.

Team are keen to move into ML. Some training has taken place (mainly using Azure ML and a little Python/R) but no ML models deployed.
Current phase

Work setup

Address where the work will take place
Health Education England,
Stewart House
32 Russell Square
Working arrangements
We require individuals on-site at appropriate times to work with our users and particularly in partnership with our analysts and developers. This is so that skills and knowledge can be shared and that HEE area able to support and develop anything built in Alpha.
Security clearance

Additional information

Additional terms and conditions

Skills and experience

Buyers will use the essential and nice-to-have skills and experience to help them evaluate suppliers’ technical competence.

Essential skills and experience
  • have experience designing and deploying ML services within the NHS
  • have experience working with NHS-specific datasets
  • be able to work in partnership with HEE so that knowledge and skills can be shared to support any alpha services that are developed
Nice-to-have skills and experience

How suppliers will be evaluated

All suppliers will be asked to provide a written proposal.

How many suppliers to evaluate
Proposal criteria
  • Technical solution
  • Approach and methodology
  • Estimated timeframes for the work
  • How the approach or solution meets our goals
Cultural fit criteria
  • Work as a team with our organisation and other suppliers
  • Take responsibility for their work
  • Share knowledge and experience with other team members
  • Be transparent and collaborative when making decisions
Payment approach
Fixed price
Additional assessment methods
Work history
Evaluation weighting

Technical competence


Cultural fit




Questions asked by suppliers

1. Are you only interested in organisations with has experience designing and deploying ML services within the NHS?
Preference will be given to such organisations as it is expected that good domain knowledge in this area is required in order to assure delivery of alpha within a relatively short timeline.

If supplier can demonstrate NHS domain knowledge through other means, they too will be considered.
2. What specific data/analysis and the application of ML within the health sector preclude organisations with expertise and experience in other fields/sectors?
Whilst experience in other fields/sectors will be considered within applications, some knowledge of and experience working with NHS-specific datasets, collections, systems, and analyses across any of workforce, trainee, learner, activity, finance, performance functions is essential.