Monitor on behalf of NHS Improvement

Patient-level costing system pilot across Nottinghamshire Integrated Care System (ICS)

Incomplete applications

13
Incomplete applications
13 SME, 0 large

Completed applications

19
Completed applications
12 SME, 7 large
Important dates
Opportunity attribute name Opportunity attribute value
Published Thursday 17 May 2018
Deadline for asking questions Thursday 24 May 2018 at 11:59pm GMT
Closing date for applications Thursday 31 May 2018 at 11:59pm GMT

Overview

Overview
Opportunity attribute name Opportunity attribute value
Summary of the work Alpha phase (including rapid discovery) to develop a clinical analytical information system to support the delivery of patient care across an Integrated Care System. Work to be completed in Nottinghamshire and London. Contract delivery to commence by 11 July 2018. SUPPLIERS' BRIEFING MEETING SCHEDULED: 16:30 on 22 May, London, SE1.
Latest start date Wednesday 11 July 2018
Expected contract length 6 months.
Location London
Organisation the work is for Monitor on behalf of NHS Improvement
Budget range Up to £450,000 including all expenses and exclusive of VAT.

About the work

About the work
Opportunity attribute name Opportunity attribute value
Why the work is being done Nottinghamshire STP, like other healthcare systems, comprises healthcare professionals within separate Providers and Commissioner bodies, resulting in a silo approach where patient care can be impacted by difficulties in seeing beyond each organisation. Funding has been secured for a six month project to test with Nottinghamshire, as an accelerator site for the national Integrated Care System programme, ways of overcoming these barriers to support a population view of patient care. This project aims to provide clinicians with a view of the patients total care provision through health and social care providers, thus establishing a new way of transforming healthcare.
Problem to be solved GPs, hospital clinicians, community healthcare professionals and social workers do not have a single view of patient care across the Sustainability and Transformation Partnership (STP), to support best possible care of patients and allow care pathways to be improved.

There is no current transparency of the cost of interventions and resources used across the care pathway. There is no mechanism to allow clinicians to access further clinical details from specialist data sets held outside the projects remit.
Who the users are and what they need to do As a clinician/ manager within Nottinghamshire ICS:
I need an accessible, flexible and easy to use source of information about patient care, regardless of care setting so I can identify opportunities to improve health outcomes.

As NHS Improvement:
We need to create a prototype solution for an ICS to link together data about care (including activity and cost) from different settings within an ICS , so we can support other ICS to manage the health needs of their population, including identifying opportunities to improve the delivery of care.
Early market engagement N/A
Any work that’s already been done This is a new project. We are seeking a short discovery phase (up to 6 weeks), followed by an up to 18 week iterative Alpha phase (subject to a decision gateway at the end of discovery).
Existing team The Supplier will be working with members of the project team (in Nottinghamshire) including clinicians, and others responsible for service management and members of the Cost Collection and Analysis team (in NHS Improvement) including Project Lead and Product Manager.
Current phase Alpha

Work setup

Work setup
Opportunity attribute name Opportunity attribute value
Address where the work will take place The supplier will be expected to work closely with the NHS Improvement delivery team (located at Wellington House 133-155 Waterloo Road London SE1 8UG, and for some parts of the work will need to be physically located on site e.g. for data and infrastructure access);

Additionally the supplier will be expected to work with the Nottinghamshire ICS delivery team (located at various locations in and around Nottingham, e.g. Nottingham University Hospital Hucknall Rd, Nottingham NG5 1PB. Working in Nottinghamshire is anticipated to make up to 50% of the engagement.
Working arrangements Members of the supplier team shall be on-site (London and Nottinghamshire) as required to support an agile delivery. The development team will host agile ceremonies (stand-ups, show-&-tell, review, retrospective, sprint planning and stakeholder workshops) which will be mandatory for the supplier staff/team.

The successful Supplier will have access to patient-level information so prior experience handling sensitive data and demonstrable internal processes and training around Information Governance (IG) is required.

Suppliers may be required to sign specific NHS Improvement data sharing agreement T&Cs to protect data disclosure (NDAs, etc.).
Security clearance N/A

Additional information

Additional information
Opportunity attribute name Opportunity attribute value
Additional terms and conditions All IPR shall vest in NHS Improvement. No license or permission to use the artefacts delivered by the Supplier is granted except insofar as to deliver the requirements of NHSI.

Skills and experience

Buyers will use the essential and nice-to-have skills and experience to help them evaluate suppliers’ technical competence.

Skills and experience
Opportunity attribute name Opportunity attribute value
Essential skills and experience
  • Demonstrable expertise and experience designing scalable analytical solutions (e.g. discovery work to facilitate requirement elicitation, process design). (20 points)
  • Demonstrable expertise and experience in data capture from multiple systems. (20 points)
  • Proven track record of project delivery working with NHS data. (20 points)
  • Demonstrable expertise and experience at an organisational level in data linkage between new data systems and existing data systems (including cloud integration). (20 Points)
  • Demonstrable expertise and experience building on and maximising existing architecture and integrating it with new solutions (leveraging Microsoft SQL server/APS for analysis). (20 points)
  • A minimum of five years of stakeholder engagement experience, including managing challenging stakeholders and those with clinical and/or non-technical backgrounds. (20 points)
  • A minimum of three years’ experience in application of agile approaches to delivering complex systems, in line with GDS Service Standards. (10 points)
  • Demonstrable expertise and experience using data visualisation packages (e.g. Tableau, Power BI). (10 points)
  • Experience of handling sensitive data, demonstrated by information handling policies and processes. Recognised accreditation desirable. (10 points)
Nice-to-have skills and experience
  • A comprehensive understanding of the NHS landscape.
  • A substantial understanding of NHS costing data.
  • Demonstrable experience using data blending packages (e.g. Alteryx).

How suppliers will be evaluated

How suppliers will be evaluated
Opportunity attribute name Opportunity attribute value
How many suppliers to evaluate 5
Proposal criteria
  • Demonstration of in-depth understanding of our requirement. (20 points)
  • Approach and methodology to requirements delivery and project management (including project plan and resource allocation) (30 points)
  • A technical solution enabling scalable and future-proof target architecture development to meet high-level functional & non-functional requirements, without significant system refactoring. (30 points)
  • A team that can deliver, including team structure, roles and CVs of anticipated resources you will deploy and a rationale regarding the skills and seniority mix. (30 points)
  • In-depth understanding of NHS data across multiple stakeholders and systems. (20 Points)
  • A robust understanding of Information Governance and its application in healthcare settings. (20 points)
  • Approach and methodology to stakeholder mapping and engagement to address a wide variety of stakeholders, including clinicians. (20 points)
  • Identification of risks and dependencies and proposed mitigations to manage them. (10 points)
  • Relevance and quality of interview presentation. (20 points)
  • Demonstration of a relevant case study as part of the interview and how it aligns to our requirements. (20 points)
  • Answers to questions during the interview/presentation stage. (10 points)
  • Price will be assessed on the basis of a like-for-like comparison of a costed resource profile which shortlisted suppliers will return as part of their proposal.
Cultural fit criteria
  • Agile and flexible to changing/uncertain requirements. (30 points)
  • Transparent and collaborative when making decisions. (20 points)
  • Share knowledge, experience and skills with other team members. (10 points)
Payment approach Capped time and materials
Assessment methods
  • Written proposal
  • Case study
  • Reference
  • Presentation
Evaluation weighting

Technical competence

70%

Cultural fit

10%

Price

20%

Questions asked by suppliers

Questions asked by suppliers
Supplier question Buyer answer
1. How do I register for the supplier briefing meeting? You will be able to see the Suppliers' briefing meeting details, including full venue information and instructions on how to register, once you have logged in to the portal.

Dial in facilities are available to suppliers unable to attend in person.
2. When is the Suppliers' briefing meeting? SUPPLIERS' BRIEFING MEETING SCHEDULED: 16:30 on 22 May at Wellington House, 133-155 Waterloo Road, London, SE1 8UG.

Please log in for details on how to register - Suppliers are welcome to attend the briefing in person (only limited places are available). Dial in facilities are also available for those unable to attend in person.
3. Hi, we wish to attend tomorrow and according to the above Q&A have to so this via the portal. When we login (via the gov.uk portal there is no info and the only option is to respond to the questions). Can you please direct us with a URL link to the correct portal where we can register? We would greatly appreciate this information soon as we wish to attend by tomorrow. To see registration information regarding the Suppliers' Briefing Meeting on 22 May at 16:30, click this link and then log in.

https://www.digitalmarketplace.service.gov.uk/user/login?next=%2Fsuppliers%2Fopportunities%2F6892%2Fquestion-and-answer-session

The email address by which to register will then be presented.

To help us, please include in the subject heading: T-RES-0518-300 - Registration for Suppliers' Briefing Meeting.

Please include the name and job title of your delegate and whether you are requesting to join by phone or in person. If joining by phone the dial in details will be provided.
4. Please can you share a link for the portal to sign-up to the event on the 22nd May? References to the "Portal" means the digital marketplace where the opportunity is advertised.

Please see the answer above for a link to registration information for the Supplier Briefing Meeting at 16:30 on 22 May.

If the link is not working, there is a hyper link on the Opportunity page, towards the bottom under the heading "Question and answer session"
5. Please could we have the dial in facilities as we are unable to attend by person? Dial in facilities can be found by logging in using the links in the two questions above.

Please note even if dialling in it is necessary to register by emailing the address provided (only visible once logged in).

To help us, please include in the subject heading: T-RES-0518-300 - Registration for Suppliers' Briefing Meeting.
6. Would you consider using open source software? And would you consider open sourcing any IP developed during the project? Yes, we would consider using open source software, but we would want to understand as part of any detailed proposal what this would entail. We could consider open sourcing any IP following conclusion of the project but at this stage we wouldn’t want to commit.
7. What volume and frequency of data are you expecting? And are you expecting data conforming to national standards or will there be other local proprietary formats? This will be scoped as part of the discovery phase but as far as is possible National Standards should be applied. This will depend on what data is available, how quickly and what is achievable in the time available.
8. Have you already been working with a Supplier for some or all of the requirement? If so, who? No.
9. Are there any existing secondary use data sharing agreements across the STP in other workstreams? The Notts system has previously used data sharing agreements and intend to use the experience from this to assist in the project.
10. Does Nottinghamshire ICS and by extension this project incorporate Social Care providers? Social care and local authorities are a shadow member of the ICS. Whilst we will examine what data we hold in Notts system as part of the discovery phase, inclusion of social care providers may not be possible at this stage.
11. Who makes up Nottinghamshire ICS (Notts.)? Please see this link:

https://www.england.nhs.uk/systemchange/integrated-care-systems/nottinghamshire-ics/

However we recognise we need to be pragmatic with the scope of the pilot and not all providers or their data will be included.
12. Are you anticipating a patient level costing system or a data linkage system for some cost settings? Some providers have patient level costing (PLICS), for example Nottingham University Hospitals NHS Trust (NUS), other providers won’t have patient level costing. It may be necessary to develop a less granular cost in some instances to ensure that we can cost across the patient pathway.
13. Is there an appetite to feed cost data through Notts existing systems? The assumption is that costing data will be imported in to NHS Improvement’s data warehouse and deployed back out to users through a Tableau based system.
14. Are commissioners & commissioner data involved in the pilot? Commissioners are an interested stakeholder group however it is providers and provider data that is the priority.
15. Does the project scope include supporting providers with costing methodology and capture? We need to remain pragmatic - we will assist providers with this where possible and proportionate but largely we will work with the current state of costing methodology. In some instances where there is no patient level data we will have to agree a costing methodology.
16. You reference data specifications, have these been subject to consultation with Notts? As a starter, we have shared the data specifications with Notts and they are currently reviewing these. This is very much a partnership approach based on what is available and achievable and we are not dictating the format.
17. Are you seeking costs based on HRG tariffs or average spend across a groups of patients or the costs to the provider for performing the work? We are seeking to capture costs recorded on the providers’ general ledger.
18. Are their common costing systems across Primary Care in Notts? The system in place across those we are anticipated to be working with is GPRCC.
19. Your use of DOS2 Outcomes means you intend to own the IP in the system developed, however there are proprietary systems that can provide the service, would you consider these. Solutions must be potentially scalable by NHS Digital (in an affordable way) and complement existing PLICS systems. We must own the IPR in any solution to further our work across the sector.
20. You state that using the Outputs described “…variation in the care patients receive can be identified as well as opportunities to improve patient pathways and provide more effective and better value care. This will support service redesign to support efficiency and sustainable services.” Are you looking to achieve this from this pilot? It is an effect of the pilot and will support Notts clinicians to make decisions based on the data brought together by the pilot. It’s not intended that the pilot itself will reconfigure a given patient pathway.
21. Do you anticipate a data sharing agreement at the commencement of the project? Not at the commencement but this will be a key deliverable. Data sharing and impact discussions with Information Governance colleagues locally have begun.
22. Regarding the diagram at slide 10, is this a CTP cost and not a cost to the system? Regarding the diagram at slide 10, is this a CTP cost and not a cost to the system?
23. Regarding the diagram at slide 10, is this a CTP cost and not a cost to the system? That’s correct. We are looking to capture at episode level across a variety of pathways and spells. This will result in PLICS data being married up with higher level data from GP providers.
24. Are you anticipating Clinical user stories as part of the Discovery phase? We have some understanding from previous local work however a Discovery phase will need to engage clinicians and establish user stories. A clinical leadership group is being established to provide this.
25. Is the budget for Discovery and Alpha Yes
26. Are your users identified? The clinical leadership community are the key end users, including identified users from; each Acute provider; each Mental Health and Community provider, a GP and secondary care doctor.
27. You state you recognise that short term may need different approaches than for delivering at scale. Where do you see the balance? This will be addressed as part of the discovery phase.
28. How far in to a usable prototype or service do you want to progress? Ideally we want to achieve a working prototype that informs the stated pilot objectives and leaves a MVP in situ for Notts clinicians to continue to use, post project closure.
29. What is the budget split between Discovery and Alpha? We anticipate a rapid Discovery (4-6 weeks) following by up to 18 weeks of Alpha development.
30. Please clarify the project timetable. Discovery: 11 July - 31 August.
Data collection: September – November
Reporting Solution: November – December
User Guidance: November – December
Technical & operational guidance: November – December
Scale up and technical handover report: November – December
System Deployment: December.

The above dates are a guide only and phases may overlap to a greater or lessor degree. We require the successful Supplier to work flexibly and to the demands of the project.
31. Can you share details of the data environment with the Suppliers? Microsoft APS, Microsoft Azure, Tableau, OKTA (for external user authentication).
32. Are you intending to use the NHS Digital pseudonym tool? NHS Improvement cannot receive patient identifiable data and we are discussing the approach to data pseudonymisation with IG colleagues internally and locally.
33. Please confirm the Buyer is the data controller and the Supplier will be a data processor? This will be clarified as we progress, in general terms, the organisation who collects the data is the data controller and when this is transferred to our data environment either Monitor will become the data controller or both Notts and Monitor become joint data controllers. At this pilot stage, we will need to be pragmatic and collaborative, so a joint arrangement may be most practical,
34. Will you consider the skills and experience of our proposed supply chain to be relevant for submission at shortlisting stage? Yes but your supply chain / delivery team must be established in principle and not be entirely speculative.
35. Will you provide Tableau licences for the Supplier? Yes
36. Has any discovery work been done? Yes. We are in the process of setting up meetings with clinicians who have been engaged in this process and are keen to get underway.
37. Will wider central stakeholders (e.g. DHSC & NHSE) be involved? They are interested parties and we will interface with them accordingly. The Supplier is not expected to engage with these groups directly, but we can identify if this is required at Discovery Phase.
38. At the intended meetings with Clinicians should we consider integration of quality and performance measure? We can, but it may be beyond the scope of this relatively short project but discovery will confirm this.
39. What time series are you anticipating this project to consider data from? We anticipate this to be in year data only but discovery will confirm this.
40. Given the scope of the project, would you consider system functionality which may not be available for the pilot but was designed in to function at scale? We would consider this, but not if it put the pilot outcomes and working prototype for Notts at risk.
41. How will you be expecting rates to be presented at proposal stage? We anticipate rates to be shown as framework rate, discounted framework rate, discounted framework rate plus expenses for London & discounted framework rate plus expenses for Notts. Roles outside the framework should be displayed the same way, replacing “framework” for “standard”.
42. Can you increase the word limit for the Essential Skills and Experience? We don’t have the facility to do this as it is built in to the DOS2 portal.

Common issues where Suppliers are rejected at shortlist stage:

 Not identifying or building responses around the key words stated in the required essential skills and experience.
 Leaving the evaluator to infer that the essential skill/experience is addressed, rather than making express statements.
 Providing examples of previous work without commenting on the delivery performance or stating the outcome.
43. Do you anticipate chunks of work (e.g. a week or more) to be undertaken at one location or will this be more fluid. At this stage we don’t know but we require the successful provider to be flexible ad work to the demands of the project.
44. Do you anticipate chunks of work (e.g. a week or more) to be undertaken at one location or will this be more fluid. At this stage we don’t know but we require the successful provider to be flexible ad work to the demands of the project.
45. What is Monitor’s infrastructure for reuse? Microsoft APS, Microsoft Azure, OKTA (for external user authentication) BI stack is Tableau (v10.5). We will support the infrastructure.
46. Are you anticipating for the data on-shore delivery? Yes
47. Will you score nice to haves? “Nice to have” responses will be scored by the evaluation panel and can have a bearing on shortlist selection so Suppliers’ should complete this section also.
48. What are the biggest risks associated with this project? At this moment in time, poor clinical engagement and data governance challenges, however both are subject to early and ongoing discussions.
49. You ask for "experience in data capture from multiple systems". Please confirm you mean extracting data rather than data mapping and analysis. Yes, extracting data is another way of explaining this requirement.
50. Your nice-to-have skills mentions data blending packages (e.g. Alteryx). Would you consider a different technology & approach for extracting data from existing IT systems for reporting in Tableau? We would consider similar packages to Alteryx (nice-to have skills). Tableau skills are essential.
51. If the Discovery phase is successful and the decision gateway at the end of Discovery approves work on the Alpha phase, will this be (1) awarded to the successful applicant or (2) opened for tender with other suppliers? This procurement process is for Alpha, with a rapid discovery phase forming the first SOW. There will not be a further tender for Alpha and this procurement and the budget listed covers rapid discovery followed by Alpha.
52. Are you able to share the names of the systems and/or end user computing tools you wish to extract data from? This will be determined through the discovery phase – it could be different for each provider. The majority of providers will have a costing system in place (usually underpinned by SQL) and we will require an extract from that system – the current expectation is that will be XML/CSV.
53. What technology holds the information from the systems you wish to extract data from, e.g. SQL database, DB2, Oracle DB, Excel spreadsheets, Hadoop, Elasticpath This will be determined through the discovery phase – it could be different for each provider. The majority of providers will have a costing system in place (usually underpinned by SQL) and we will require an extract from that system – the current expectation is that will be XML/CSV.
54. What experience do your operational and technical staff have with Agile delivery? The Technology & Data team within NHS Improvement develop products and services using agile methodologies.
55. Are you currently using Scaled Agile Framework? No
56. We were unable to attend the supplier meeting on 22 May. Are minutes available summarising the questions and answers shared in this session? Please see above.
57. Your enquiry focused on technical delivery, data security and stakeholder management. Which do you perceive as the biggest risk for this project? At this moment in time, clinical engagement and data governance challenges, however both are subject to early and ongoing discussions
58. We have been asked to consider a response to an Essential Skill/Experience and whether this would be acceptable or not. This is for the evaluation panel to consider following closure of the portal opportunity and will not be answered here.
59. Are all the organisations involved on N3 / HSCN? If not is there any existing infrastructure for data transfer? Confirming the in-scope Notts ICS provider data network arrangements (Health and Social Care Network, its precursor N3 network or alternatives) will form part of the discovery phase. It should not be assumed that all providers use the same data network