Machine Learning for Improved Insight (Discovery & Alpha)
8 Incomplete applications
8 SME, 0 large
18 Completed applications
13 SME, 5 large
- Friday 13 December 2019
- Deadline for asking questions
- Friday 20 December 2019 at 11:59pm GMT
- Closing date for applications
- Friday 27 December 2019 at 11:59pm GMT
- Summary of the work
- HEE is seeking a partner with expertise in AI to assist the organisatioin in understanding how machine learning practices can be harnessed to better address challenges in analysis and prediction across the trainee doctor management and workforce planning problem space.
- Latest start date
- Saturday 1 February 2020
- Expected contract length
- To be delivered by April 2020
- Organisation the work is for
- Health Education England
- Budget range
Total budget for Discovery & Alpha is £50k (inclusive of VAT and any incurred infrastructure costs).
Note - HEE currently use Azure platform.
About the work
- Why the work is being done
This discovery and alpha project seeks to help HEE understand how machine learning (ML) practices can be harnessed within the organisation to better address the significant and ongoing challenges faced with regards to analysis and prediction across the medical trainee management and workforce planning problem space.
This work comes in response to the urgent workforce challenges outlined in the Long Term Plan and its ambition to address these by ‘making better use of data and digital technology’ as well as recommendations within the Topol Review to utilise AI to better understand the uniqueness of each individual with the aim to deliver education, training, and workforce planning on a far more rational, efficient and tailored basis.
Both Discovery and Alpha phases to be complete by April 2020.
- Problem to be solved
A number of potential use cases have been identified below. These will be finessed during Discovery to a suitable subset for development within Alpha.
• classifying staff groups carrying high retention risk
• identifying determinants behind staff attrition, knowing more about why individuals intend to leave
• predicting ARCP outcomes and identifying trainees in difficulty early
• developing models to improve rotation scheduling of trainees
• improving current predictive analytics by identifying relationships across multiple trainee, workforce, education and learning datasets when combined with social, economical, and environmental data
Working in partnership with HEE to
• identify the different types of ML user
• establish use cases for Alpha and success measures of implementation
• evaluate the viability, cost-effectiveness, and ethics of utilising ML
• identify appropriate development platform
• identify constraints of future ML projects inc. compliance, governance, bias
• develop alpha process for utilising ML within HEE
• prototype and evaluate ML models against use cases
• develop staff and set foundations for establishing an ML capability within HEE
• identify what is required for HEE to build and support ML products that better meet user needs
• identify whether to move to beta or repeat discovery/alpha
- Who the users are and what they need to do
As a workforce planning lead
I need to identify segments of the workforce who are at a higher risk of leaving the NHS
So that I can put suitable plans in place to address shortfalls
As a workforce planning lead
I need to understand the factors which contribute most to indivdiuals leaving their role/organisation/NHS
So that I can develop tailored rentention strategies
As an educational supervisor
I need to be able to identify medical trainees who may be experiencing difficulty in their training
So that I can take early action and provide support
As a BI analyst
I need to be able to run ML models against datasets
So that I can use information in the reports and dashboards I build
As a BI Developer
I need a platform on which to build, train, and deploy ML models
So that I can prototype and evalute predictive modelling on new and existing use cases
Note - user identifcation is an objective of the Discovery phase. Above are examples only.
- Early market engagement
- Any work that’s already been done
- Prelimary work has already been conducted regarding identification and suitabaility of uses cases. It is expected that this will help expedite Discovery phase.
- Existing team
Business Intelligence and Systems Development Team,
BI team are skilled in SQL Server and Tableau.
Team are keen to move into ML. Some training has taken place (mainly using Azure ML and a little Python/R) but no ML models deployed.
- Current phase
- Address where the work will take place
Health Education England,
32 Russell Square
- Working arrangements
- We require individuals on-site at appropriate times to work with our users and particularly in partnership with our analysts and developers. This is so that skills and knowledge can be shared and that HEE area able to support and develop anything built in Alpha.
- Security clearance
- Additional terms and conditions
Skills and experience
Buyers will use the essential and nice-to-have skills and experience to help them evaluate suppliers’ technical competence.
- Essential skills and experience
- have experience designing and deploying ML services within the NHS
- have experience working with NHS-specific datasets
- be able to work in partnership with HEE so that knowledge and skills can be shared to support any alpha services that are developed
- Nice-to-have skills and experience
How suppliers will be evaluated
All suppliers will be asked to provide a written proposal.
- How many suppliers to evaluate
- Proposal criteria
- Technical solution
- Approach and methodology
- Estimated timeframes for the work
- How the approach or solution meets our goals
- Cultural fit criteria
- Work as a team with our organisation and other suppliers
- Take responsibility for their work
- Share knowledge and experience with other team members
- Be transparent and collaborative when making decisions
- Payment approach
- Fixed price
- Additional assessment methods
- Work history
- Evaluation weighting
Questions asked by suppliers
- 1. Are you only interested in organisations with has experience designing and deploying ML services within the NHS?
Preference will be given to such organisations as it is expected that good domain knowledge in this area is required in order to assure delivery of alpha within a relatively short timeline.
If supplier can demonstrate NHS domain knowledge through other means, they too will be considered.
- 2. What specific data/analysis and the application of ML within the health sector preclude organisations with expertise and experience in other fields/sectors?
- Whilst experience in other fields/sectors will be considered within applications, some knowledge of and experience working with NHS-specific datasets, collections, systems, and analyses across any of workforce, trainee, learner, activity, finance, performance functions is essential.