Health Data Research UK

Metadata Onboarding and Process Improvement

Incomplete applications

14
Incomplete applications
11 SME, 3 large

Completed applications

10
Completed applications
6 SME, 4 large
Important dates
Opportunity attribute name Opportunity attribute value
Published Tuesday 3 September 2019
Deadline for asking questions Tuesday 10 September 2019 at 11:59pm GMT
Closing date for applications Tuesday 17 September 2019 at 11:59pm GMT

Overview

Overview
Opportunity attribute name Opportunity attribute value
Summary of the work HDR UK requires a supplier to project manage, support and improve the process for rapidly preparing and onboarding metadata held in multiple formats across data controllers who are members of the UK Health Data Research Alliance. The project supports the development of a national portal supporting data discovery and access.
Latest start date Monday 14 October 2019
Expected contract length 6 months
Location No specific location, eg they can work remotely
Organisation the work is for Health Data Research UK
Budget range No commitment to minimum spend over the term of the contract.

The maximum spend is £250,000 inclusive of VAT.

About the work

About the work
Opportunity attribute name Opportunity attribute value
Why the work is being done This work forms part of the Digital Innovation Hub programme (https://www.hdruk.ac.uk/digital-innovation-hubs/), a national initiative which aims to increase the fair and ethical use of health data for research and innovation. Currently, discovery of and access to health data that is available for research is challenging. Many data controllers across the UK make their data available, but there is no common route for discovering these datasets. HDR UK are developing the Health Data Research Innovation Gateway (https://www.hdruk.ac.uk/digital-innovation-hubs/gateway/), a national portal which will act as a common route for discovering and accessing data.
Problem to be solved HDR UK have recently identified a supplier of a metadata catalogue to support the functionality of the Gateway. This catalogue needs populating with metadata from the datasets held by the data controllers. There are hundreds of datasets which could potentially be discovered across the collaborating data controllers, members of the UK Health Data Research Alliance. A team is required to co-ordinate and manage the process and work with the Alliance members and metadata catalogue team to extract the metadata and load it into the catalogue. The team will develop a standard process for doing this in a rapid, scalable manner.
Who the users are and what they need to do There are two users of the metadata catalogue and associated innovation gateway:
1. As a researcher or innovator, from the NHS, academia or industry, I need to discover, understand, access and potentially link the health data that is relevant to my specific research question or innovation (such as understanding causes of disease, earlier diagnosis, developing new treatments, evaluating the impact of specific interventions or deciding on the most effective approaches for managing the NHS).
2. As a Data custodian, I want a streamlined process for making the metadata available and maintaining its currency.
Early market engagement HDR UK has undertaken a six-month Design and Dialogue process as part of the Digital Innovation Hub programme. Through this process, we have engaged with over 2,700 people and over 350 organisations. This has allowed a detailed insight into the needs of users. For example, over 70% of users from industry identified the inability to quickly access data as being a major barrier to the use of data for research and innovation.

HDR UK have convened the UK Health Data Research Alliance (https://www.hdruk.ac.uk/digital-innovation-hubs/uk-health-data-research-alliance/), a group of data controllers committed to increasing the responsible and secure use of health data for research and innovation. There are currently 17 members of the Alliance spanning national data custodians, NHS trusts and custodians of population and disease cohort data. All Alliance members have signed a letter of intent, committing to the aims of the programme and the principles for participation.
Any work that’s already been done HDR UK have appointed a supplier of the metadata catalogue, which will specify the means and format for the metadata to be loaded into the tool. This builds on development work undertaken as part of NIHR’s Health Data Finder http://www.hdf.nihr.ac.uk/
We have a provisional list of within scope datasets and are in the process of confirming Data Officers for all Alliance members who are expected to be the first point of contact for this work

From September to January a development team is designing the Portal Gateway -MVP) front end, but this is unlikely to impact the dataset onboarding work.
Existing team Suppliers will be required to work alongside the HDR UK Infrastructure team:
- David Seymour, Partnership Director
- Gerry Reilly, Chief Technology Officer
- Ben Gordon, Digital Innovation Hub Programme Director

In addition, the successful suppliers will be required to work with or alongside the team providing the metadata catalogue element of the Gateway
Current phase Alpha

Work setup

Work setup
Opportunity attribute name Opportunity attribute value
Address where the work will take place Remotely, but will need to work closely with:

1. Health Data Research UK team at:
Health Data Research UK
Gibbs Building
215 Euston Road
London
NW1 2BE

2. Metadata catalogue team

3. Data custodians (various UK locations)
Working arrangements Working remotely and on location with Metadata catalogue team and Alliance members as required. Key members of the team will be required to attend project steering and co-ordination meetings.
Security clearance No expected specific security clearance is required, but ISO27001 certification is highly desirable.

Additional information

Additional information
Opportunity attribute name Opportunity attribute value
Additional terms and conditions Suppliers should subscribe to the Health Data Research UK Principles for Participation, which includes a commitment to open source, open standards and open data, and FAIR data principles.

Skills and experience

Buyers will use the essential and nice-to-have skills and experience to help them evaluate suppliers’ technical competence.

Skills and experience
Opportunity attribute name Opportunity attribute value
Essential skills and experience
  • Recent experience working with public sector clients on data related projects.
  • Demonstrable understanding of the relevant organisations in UK healthcare landscape
  • Experience influencing a range of stakeholders including with data controllers
  • Demonstrable deep current understanding of the challenges of data engineering and curation, including working with different data modalities and metadata
  • Experience in solving complex problems in a creative ways
  • Experience working in a fast-moving, changing environment with proven ability to support a flexible way of working
  • Ability to adapt to short notice changes to plans
  • Rigorous focus on delivery, over and above client requirements
  • Delivery of repeatable high-quality outputs
  • Understanding of research and innovation use cases of healthcare data
  • Relevant Information Governance knowledge, skills and experience
Nice-to-have skills and experience

How suppliers will be evaluated

How suppliers will be evaluated
Opportunity attribute name Opportunity attribute value
How many suppliers to evaluate 6
Proposal criteria
  • Approach and methodology
  • Team structure
Cultural fit criteria
  • Work as a team with our organisation and other suppliers
  • Be transparent and collaborative when making decisions
  • Can work with clients with a range of technical expertise
Payment approach Capped time and materials
Assessment methods
  • Written proposal
  • Case study
  • Work history
  • Presentation
Evaluation weighting

Technical competence

70%

Cultural fit

10%

Price

20%

Questions asked by suppliers

Questions asked by suppliers
Supplier question Buyer answer
1. Could you please share the Discovery Report and recommendations? The Discovery phase was undertaken through the Design and Dialogue phase of the wider Digital Innovation Hubs programme. The key findings from this phase were developed into the documents available on the HDR UK website for the components of the programme: https://www.hdruk.ac.uk/infrastructure/
2. Who delivered the Discovery phase? The Design and Dialogue phase was delivered by HDR UK, in association with the Office of Life Sciences, Medicines Discovery Catapult, ABPI, NIHR and others.
3. Does the work primarily include Project Management work? A team is required to co-ordinate and manage the process and work with the UK Health Data Research Alliance members and metadata catalogue team to extract the metadata and load it into the catalogue. The team will develop a standard process for doing this in a rapid, scalable manner. Much of this will require project management and delivery, rather than specific technical solutions.
4. What is the metadata catalog product? The metadata catalogue toolkit is a proprietary solution and will be supplied by NHS Digital with the University of Oxford.
5. Does the metadata catalog product support the connection, transformation and ingestion of the various forms of data sources? or does this project require the development of "data connectors" to perform this function? The catalogue provides a REST API and Java client libraries for import and export. Import tools, which generate metadata from software artefacts, exist for relational databases (SQLServer, MySQL, Oracle, PostgreSQL, Access), UML (via XMI or StarUML JSON), XML schema, XML, JSON, and Excel. The project team for this piece of work will not be required to develop data connectors.
6. Is there a requirement for ongoing updates of metadata and if yes, does the metadata catalog support for this scenario or is there a requirement to develop "data connectors" for incremental updates? The metadata will need to be updated, rather than a static upload, so part of the process for each dataset will require a means of ensuring that the metadata is kept up to date.
7. Where is the metadata catalog hosted – public/private cloud? The metadata catalogue is hosted in a public cloud.
8. If there is a need to develop tools and automation, where will these be hosted/executed? HDR UK will support the hosting of these tools, with the expectation that these would be made available on the HDR UK GitHub.
9. If tools need to be developed, are there any restrictions on the technologies/languages used? Or is this open to the supplier to choose provided they integrate with data sources and catalog? There are not specific restrictions, provided they work with the data controllers and the catalogues, however as detailed above, many of the tool for ingestion are already available from the metadata catalogue.
10. Are there any metrics on the volume of data that needs to be ingested in this phase? The expectation for the number of datasets to have their metadata loaded onto the catalogue is several hundred. The list of datasets to be loaded should be co-developed with HDR UK during the early phase of the project.
11. Would you be able to provide a more specific understanding of the number of datasets that are expected to be processed for loading into the catalogue? (We understand that this is still in development and may change) The expectation for the number of datasets to have their metadata loaded onto the catalogue is several hundred. The list of datasets to be loaded should be co-developed with HDR UK during the early phase of the project.
12. Who is the catalogue supplier for the metadata catalogue please and what tool are they going to be using? NHS Digital with the University of Oxford. The metadata catalogue toolkit is a proprietary solution.
13. We are developing an innovative approach which would meet the budget and quality of outcome but wanted to check if we would be able to undertake some support work offshore. The supplier is welcome to use a team that is a mix of UK and off shores resources provided that the requirement to be available for face to face meetings in London and with Health Data Alliance members nationwide can be met. Much of the work will involve interaction with data controllers in the UK, so there is a limit to the activity that could be completed offshore.
The deadline for asking questions about this opportunity was Tuesday 10 September 2019.