This opportunity is closed for applications

The deadline was Friday 25 September 2020
NHSX

NHS Artificial Intelligence Lab - AI Imaging Discovery Exercise

16 Incomplete applications

8 SME, 8 large

20 Completed applications

11 SME, 9 large

Important dates

Published
Friday 11 September 2020
Deadline for asking questions
Friday 18 September 2020 at 11:59pm GMT
Closing date for applications
Friday 25 September 2020 at 11:59pm GMT

Overview

Summary of the work
To partner with the NHS AI Lab to undertake a multi-stream discovery exercise to identify health and care system lead requirements for the development of a multi-speciality AI imaging platform. Multi supplier
applications will be considered.
Latest start date
Monday 2 November 2020
Expected contract length
16 to 24 Weeks
Location
London
Organisation the work is for
NHSX
Budget range
Max £400,000

About the work

Why the work is being done
The COVID-19 Pandemic has helped to identify the potential of AI powered Medical Imaging solutions. As part of the NHS AI Lab’s wider work programme, is an ambition to create a pipeline of scalable AI technologies which include these AI powered medical imaging technologies. From this, the NHS AI Lab is looking to investigate and identify the requirements needed to procure a multi-speciality medical imaging platform. Which will allow NHS Trusts to plug into this platform and have a collection of images reviewed by trained and validated medical algorithms to achieve a diagnosis for that population group.

The NHS AI Lab team is therefore looking to source a partner to undertake accelerated Discovery Exercises to invite system led perspectives and identify the requirements for this imaging platform.
Problem to be solved
It is well recognised that developments with Medical AI Technologies are progressing at significant pace, with potential value added and system efficiencies growing constantly. However, patient data is currently collected and scarcely used in a manner that returns value to patients. The National Medical Imaging Platform will enable value return to patients by creating a direct pipeline for patient data to be used towards the development, validation, and adoption of AI healthcare technologies.

The NHS AI Lab team is therefore looking to source support to undertake a discovery exercise which will inform the development of the National Medical Imaging Platform. This discovery exercise with focus on the 5 five following workstreams:

1. System-level engagement and buy-in
2. End-user definition for data pipelines and Trusted Research Environment
3. End-user assessment for deployment
4. Activities required to begin building data pipelines and Trusted Research Environment
5. Development of a deployment platform
Who the users are and what they need to do
The NHS AI Lab programme is a joint Government and NHS England initiative to accelerate the safe adoption of robust and appropriate AI technologies for the benefit of the NHS and the Social Care sector. As part of the overarching NHS AI Lab programme is a workstream focused on realising the benefits of AI powered technologies through the development of a singular AI Medical Imaging Platform. However, the NHS AI Lab is ambitious to ensure this Medical Imaging Platform is led by the needs of the NHS. The NHS AI Lab, therefore needs to undertake a comprehensive, diverse and evidence informed discovery exercise in order to possess a clear set of requirements for the procurement and standing up of a AI Medical Imaging Platform.
Early market engagement
N/A
Any work that’s already been done
A technical discovery for an Imaging Platform has been completed over a 10 week period between 8 June and 7 August 2020. This focused on the infrastructure requirements to technically facilitate a National Medical Imaging Platform.

Over 20 interviews were completed with technology companies and healthcare institutions with medical imaging expertise. These interviews focused on validating the below 3 components that should make up the Imaging Platform, and their user and technical requirements:

Data pipelines

Trusted research environment (TRE)

Deployment platform

The user requirements focused on the technical solution that the users need from the Imaging Platform.
Existing team
A growing NHS AI Lab Team, composed of Civil Servants from the Department of Health & Social Care and NHS England & Improvement Employees. Within the NHS AI Lab team, this work will be led by one Civil Servant and one NHS England & Improvement employee. The discovery will be supported by the NHS AI Lab PMO.
Current phase
Discovery

Work setup

Address where the work will take place
Predominantly virtually, with the possibility of London based physical working, if required.
Working arrangements
The successful applicant will be expected to engage with and report to the directing organisation (NHSX) in a manner stipulated by the directing organisation. These expectations will be agreed during contact formalisation.
Security clearance
N/A

Additional information

Additional terms and conditions
N/A

Skills and experience

Buyers will use the essential and nice-to-have skills and experience to help them evaluate suppliers’ technical competence.

Essential skills and experience
  • Please describe the relevant expertise and experience you hold in running discovery exercises which relate to emerging technology and the NHS and Social Care.
  • Over the last three years, please provide a work history that sets out similar discovery exercises that have been undertaken and detail how this experience demonstrates your suitability.
  • Please set out the infrastructure you have available and outline the consent, data storage and disposal processes you will use to support the delivery of the discovery exercise
Nice-to-have skills and experience

How suppliers will be evaluated

All suppliers will be asked to provide a written proposal.

How many suppliers to evaluate
5
Proposal criteria
  • Experience with user focused design methodologies which demonstrate a robust approach to service design and development. 20%
  • Practical experience of working in the UK health and care imaging ecosystem or equivalent. 20%
  • Experience of delivering national services against the Government Digital Service Service Standard and/or Technology Code of Practice. 20%
  • Evidence of delivering technical discovery projects for data driven technologies. 20%
  • Strong stakeholder management skills with the ability to communicate design and technical concepts to a lay audience of senior stakeholders. 20%
Cultural fit criteria
  • Demonstrate how the supplier will work collaboratively with NHS and Public Sector organisations to support the realisation of public benefit. 20%
  • Demonstrate how they will work in an open and transparent manner with the directing authority. 20%
  • Operate in an agile and responsive manner to meet the developing needs of the directing authority with a can do attitude. 20%
  • Demonstrates how they will work proactively to share learning and experience with members of the NHS AI Lab team. 20%
  • Can demonstrate how they would build positive and reciprocal relationships with a diverse range of senior stakeholders . 20%
Payment approach
Time and materials
Additional assessment methods
  • Case study
  • Presentation
Evaluation weighting

Technical competence

60%

Cultural fit

20%

Price

20%

Questions asked by suppliers

1. Is the Imaging platform expected to include other data formats for a complete systems biology approach driven analysis to understand outcomes and disease progression? E.g. Genomics (.VCFs), Publications (.PDF,.DOC), Reports associated with Imaging (Structured and unstructured), Sequencing (.BAM) etc.
The imaging platform is expected to include other data formats. However, these data formats are expected to be defined as part of the discovery exercise.
2. Will evaluating and/or requirements gathering to support medical imaging algorithms be in-scope of the discovery exercise ?
Yes understanding how models are to be validated would inform part five: development of a deployment platform.
3. Have the data sources and formats in-scope for the Imaging platform been decided or will it be part of the discovery exercise? E.g. DICOM, NIFTI, etc.
Data sources and formats will be a part of the planned discovery exercise.
4. Are you anticipating that we store and process data on our infrastructure, your infrastructure, or an infrastructure we specifically set up for this project and you?
Currently, there is no plan to build anything during the discovery exercise so there will not be a requirement to store or process data within your infrastructure. This will inform the options paper you will produce.
5. Will the discovery exercise include design of the Imaging platform? E.g. solution architecture
Yes, the selected supplier is expected to set out the design requirements which will be set out in the Invitation to Tender for the building of the imaging platform
6. Will the discovery exercise include vendor assessment for various parts of the platform?
No, the discovery will inform the options paper you produce - which may recommend vendor assessments for parts of the platform.
7. Will the discovery exercise include building and delivering a POC?
No, the selected supplier is only expected to undertake a user research led discovery exercise and draft a requirements paper to inform the procurement of a partner who will build the imaging platform.
8. Will any other organisation aside from NHSX be part of the delivery team for the discovery phase ?
No, we intend for NHSX to be the only other organisation which will be involved as part of the delivery team for the discovery exercise. However, NHSX maintains the right to introduce other parties depending on progression with the discovery exercise. However, the selected supplier would be informed in advance if this was the intention.
9. How many organisations/institutions will we need to interview/talk to as part of the discovery exercise
You will need to map the existing landscape and shortlist the organisations and people who will provide the most value to the discovery. We would expect this to be around a dozen organisations.
10. Do you envisage the provider to use your existing technology, infrastructure and data to inform the discovery exercise
We expect the chosen supplier to understand and capitalise on relevant learnings, including from the National Covid-19 Chest Imaging Database built in response to the pandemic, to inform the delivery of the discovery.
11. Are you looking for a supplier with pre-existing infrastructure and storage set up as a Trusted Research Environment?
No, the chosen supplier is only expected to undertake a discovery exercise to identify and inform the development of requirements for the procurement and building of an AI Imaging Platform.
12. Is the discovery exercise limited to the technical viability of AI products and pipeline or for the whole operating model for the NHS AI Lab?
Whilst the discovery exercise has been broken into workstreams, this exercise will inform the operating model for the AI Imaging Platform.
13. Given the work completed, as part of this requirement is the supplier being asked to define the requirements?
No, the supplier will be provided with predefined requirements which need to be delivered as part of this discovery exercise. We are, however, open to discussions on additional requirements or amendments to the set ones that the chosen supplier would like to put forward.
14. As part of this requirement is the supplier being asked to build or prototype a solution?
No, the selected supplier will only be asked to identify and define the requirements for procuring, building and establishing the imaging platform.
15. Can you clarify / elaborate on question three regarding "outline the consent"?
As part of this discovery exercise, the chosen supplier will be expected to run user research exercises to build insight and understanding during the discovery. To ensure the selection of an appropriate supplier, we expect all interested applicants to demonstrate their robust procedures in line with current data protection practices.
16. Can the party that is awarded the Discovery project still join possible follow-up Solution procurement projects?
The procurement exercises for the Discovery and Building of the Imaging Platform will be kept separate. However, the supplier selected to undertake the discovery will be able to submit a separate application for the competition to build the imaging platform.
17. Is there an existing Architecture in place today? If so, can you share any diagram on that?
A high level architecture document has been created internally. However, this documentation is seen as commercially sensitive and would only be shared under the signing of contracts, non disclosure agreements and conflict of interest forms.
18. “Please set out the infrastructure you have available and outline the consent, data storage and disposal processes you will use to support the delivery of the discovery exercise” – what is the reason behind setting this as an essential skill/experience? Can you please provide some information on your expectations from suppliers regarding this point when conducting the discovery?
As part of this discovery exercise, the chosen supplier will be expected to run user research exercises to build insight and understanding during the discovery. To ensure the selection of an appropriate supplier, we expect all interested applicants to demonstrate their robust procedures in line with current data protection practices.
19. Under proposal criteria you state “Practical experience of working in the UK health and care imaging ecosystem or equivalent. 20%” – can you please clarify what you mean by equivalent? If a supplier does not have experience of working in the UK health and care imaging ecosystem are they excluded from this first stage based on the essential skills you’ve highlighted?
The NHS AI Lab will not discriminate against applicants who have undertaken similar discovery exercises within other international health and care systems. However, the supplier is expected to articulate how their experience will help them to deliver this exercise in the UK and how they meet the outcomes of the digital outcomes and specialists framework.
20. Can you please share the technical discovery findings report (8 June and 7 August 2020) at this stage with suppliers?
A high level architecture document and outputs from a technical discovery exercise has been created internally. However, this documentation is seen as commercially sensitive and would only be shared under the signing of contracts, non disclosure agreements and conflict of interest forms.
21. You mentioned that a previous technical discovery was done (8 June and 7 August 2020). Was this conducted internally or by a third-party supplier? If supplier, please can you clarify who delivered this discovery?
The previous technical discovery exercise was undertaken by a contracted third party organisation. This third party was Faculty Science.
22. Are you working with any existing suppliers in relation to this tender opportunity? If you are, who are they?
The NHS AI Lab Team has been working with an existing third party partner to help with the design of this tender opportunity. This third party was Faculty Science.
23. Who from your team will be scoring applications for this opportunity and what positions do they hold at NHSX?
The evaluation team is still being finalised but will comprise of NHS AI Lab PMO, AI Policy and AI Imaging Programme Leads. Senior Leadership Officials which will include organisational directors may be invited to join the presentation day panel based on their availability.
24. On question 3: 'Please set out the infrastructure you have available and outline the consent, data storage and disposal processes you will use to support the delivery of the discovery exercise' – is it correct to assume that such infrastructure relates to document storage / security, as we don't anticipate any patient data will be collected as part of the discovery exercise?
To ensure the selection of an appropriate supplier, we expect all interested applicants to demonstrate their robust procedures in line with current data protection practices.
25. At what point is the 'written proposal' required, as there doesn't seem to be the option to submit this along with the three responses at this stage?
All shortlisted applicants will be invited to respond to a challenge which will be set five working days in advance of the planned presentation day
26. Can you host a call with suppliers about this opportunity where we can have a live Q&A and the discussion notes are shared on this tender ad? This has proven helpful on other DM opportunities.
Unfortunately, due to the timescales needed to establish and run this discovery exercise, the NHS AI Lab Team at NHSX does not have time to hold a Q&A session.
27. Please can you provide details of what the next steps are following this initial application i.e. deadlines for the written proposal/if there will be an interview and when?
Following a moderated shortlisting exercise, five applicants will be selected to progress to a more detailed secondary application stage. The five shortlisted applicants will be request to submit a written proposal as well as carry out a presentation. Details will be shared with the five shortlisted applicants at the appropriate time
28. In the essential skills and experience section, we are asked to provide a work history that sets out similar discovery exercises that have been undertaken. Is this the work history of an individual resource?
No, suppliers are expected to set out a work history of similar discovery exercises, rather than the work history of an individual resource, although this can be included within your response if deemed appropriate. Suppliers are encouraged to present previous examples similar to the discovery exercise described for this opportunity.
29. Can you please share any documentation or background information on the National Medical Imaging Platform?
Documentation and background information are still in the early stages of development and are not yet ready to be shared with potential suppliers. However, the NHS AI Lab commits to sharing background information to support the delivery of the discovery exercise once contracts are agreed and signed with Non Disclosure Agreements and Conflict of Interest Forms in place.
30. Can you please advise on the number of resources and typical skillsets that you will need to deliver this work?
The NHS AI Lab expects the supplier to propose this in their submission. The NHS AI Lab also expects suppliers to make sure the minimum skills for a discovery, as outlined in the Government Digital Service Manual, are met.
31. Can you please share the outputs from the technical discovery work?
Due to the rapidly emerging opportunities relating to AI driven medical imaging technologies, the outputs of the technical discovery exercise are still in the early stages of development and are not yet ready to be shared with potential suppliers. The NHS AI Lab will be willing to share the outputs of the technical discovery exercise once contracts, non disclosure agreements and conflict of interest forms have been signed.
32. Can you please share the name of the incumbent who completed the technical discovery work and are they invited to participate in this opportunity?
The previous technical discovery exercise was undertaken by a contracted third party organisation who was Faculty Science. Since this tender was launched on the Crown Commercial Service's Digital Outcomes and Specialists Framework as an open competition, Faculty Science will not be prohibited from submitting an application via the DOS Framework.
33. For Question 2, we assume that NHSX is looking for suppliers to detail one example from within the past 3 years – is this correct?
We would expect suppliers to detail as many examples as they feel appropriate for their application
34. Are suppliers permitted to use multiple examples for each question?
We would expect suppliers to detail as many examples as they feel appropriate for their application
35. You have three 'skills and experience' questions. This gives first round applicants only 3 x 100 words to apply. Are you happy that this will give you sufficient information to make a decision on who to take forward given the size and complexity of the challenge?
Yes, the NHS AI Lab Team sees three one hundred answers as sufficient at this stage. Shortlisted applicants will have the opportunity to provide more complex insight and supporting information during planned virtual supplier presentations.