NHS Business Services Authority

Predictive Analytics Partner for NHSBSA Insight

Incomplete applications

14
Incomplete applications
7 SME, 7 large

Completed application

1
Completed application
1 SME, 0 large
Important dates
Opportunity attribute name Opportunity attribute value
Published Thursday 4 July 2019
Deadline for asking questions Thursday 11 July 2019 at 11:59pm GMT
Closing date for applications Thursday 18 July 2019 at 11:59pm GMT

Overview

Overview
Opportunity attribute name Opportunity attribute value
Summary of the work Delivery partner to apply cutting edge fraud/predictive analytical models to data. Initial SOW to develop plan during the 1st month, further potential SOWs to productionise/embed outputs into reporting environment/operational systems (by March 2020). Focus on transferring skills to NHSBSA Team, and requires expertise in Oracle exadata / exalytics technology.
Latest start date Monday 9 September 2019
Expected contract length 2 years. Note first SOW completed within 1 month start date.
Location North East England
Organisation the work is for NHS Business Services Authority
Budget range We will publish a price template for completion by shortlisted suppliers.

Note this contract will include an initial SOW re developing a plan for creating reusable fraud predictive analytical models to identify trends/patterns/anomalies in pharmacy payments/prescribing activities. Any further work will be dependent on other SOWs being agreed.

The contract will include a maximum spend of £750,000 excluding VAT over the 2 year period, but the NHSBSA makes no commitment to any particular level of spend (actual spend agreed in SOWs) nor does it grant exclusivity to the selected supplier in respect of the services described in this opportunity.

About the work

About the work
Opportunity attribute name Opportunity attribute value
Why the work is being done Critical to NHSBSA's strategic ambitions/NHS Long Term Plan re reducing fraud/error/waste using predictive analytics to support decisions/planning.

Commence asap September 2019. Initial stage- produce plan to create reusable fraud predictive analytical models to identify trends/patterns/anomalies in pharmacy payments/prescribing activities, presented to the BSA within 1 month. Additional potential statements of works (at NHSBSA's discretion) inc: development of models, gather/adapt to feedback from stakeholders, implementation/productionisation of models, development of further plans and predictive models re use cases in prescriptions/workforce data, with knowledge transfer/training throughout to enable NHSBSA refresh/maintenance of models, all to conclude by 31st March 2020.
Problem to be solved The requirement is to:

- make better use of data analytics and increase our capability to reduce fraud/error/waste, by creating reusable fraud and predictive analytical models to identify trends/ patterns/anomalies in pharmacy payments/prescribing activities;

- Identify and create reusable predictive analytical models on pertinent use cases in prescriptions and workforce data;

- develop a portfolio of successful examples/case studies of our fraud and predictive analytics work;

- subsequently maintain/refresh the models within the NHSBSA's internal team.
Who the users are and what they need to do Data Scientist-I need to improve my predictive analytics skills to support NHSBSA's strategic ambitions.

Provider Assurance team-I need to be able to use the outputs from predictive analytic models to target risk areas and individuals for investigation.

Workforce Manager-I need insight from the data to assist with workforce planning.

NHS commissioner-I need reliable forecasts of activity and cost.

Fraud Team-I need to use predictive analytics so that I can proactively identify, monitor and combat fraud.

Insight Team-I need to deliver predictive analytics that will improve decision making, identify potential fraud and help create algorithms that can be embedded into systems/processes.
Early market engagement See comments in section titled "Any work that's already been done".
Any work that’s already been done Strategic reviews have recently been completed by external consultancy services to:
•Assess the current state of data analytics and business intelligence within the Insight Directorate
•Review the current prescription business intelligence service, delivered through ePACT2, within the Insight Directorate.

These reviews have helped to inform our approach to making better use of data and analytics, underpinning our strategic ambitions and supporting the NHS Long Term Plan.

This has led to the decision to pursue predictive and fraud analytics.

Note: we are in the process of gaining access to Workforce data, all relevant prescription/pharmacy data already within NHSBSA Data Warehouse.
Existing team Delivery partner managed within Data Analytics Lab, but working across other areas, including BI/Data Warehouse/Provider Assurance/NHSBSA Fraud Team. BI/Data Warehouse Team teams will provide insight re how our data is structured/assist reshaping/expanding data as required.

Work underway in Business Intelligence/Data Warehouse team to improve Data Warehouse design/delivery capability. We may use third party to assist: if so, you may need to work with them as appropriate.

We are constantly reviewing Insight Directorate's underlying technology (inc. various oracle cloud services) - if changes are implemented up to end March 2020, you may need to work with the implementation team/Oracle as appropriate.
Current phase Not started

Work setup

Work setup
Opportunity attribute name Opportunity attribute value
Address where the work will take place NHSBSA Head Office:
Stella House
Goldcrest Way
Newburn Riverside
Newcastle upon Tyne
NE15 8NY
Working arrangements Work as one team with the BSA delivery team, on site 5 days per week unless otherwise agreed.
Participate in organisational activities (e.g. communities of practice), enabling shared learning, and embed knowledge transfer activities to Data Analytics Lab staff into all stages of activity.
Ensure models built in manner that maximises ease of use/maintenance by BSA team.
Flex resources as required to the needs of the project, and work constructively alongside other Insight projects/suppliers.
Comply with NHSBSA policies/procedures.
No expense claims to/from Newcastle base.
Use the existing Oracle applications/Infrastructure/RStudio. Any solutions should not require the purchase of additional hardware/software.
Security clearance •Hold Security Check (SC) clearance vetting due to the sensitivity of the data they will handle
•Comply to the NHSBSA Information Security Policy set including utilisation of NHSBSA supplied hardware/software in adherence with the Acceptable Use Policy.

Additional information

Additional information
Opportunity attribute name Opportunity attribute value
Additional terms and conditions For the avoidance of doubt, all intellectual property rights in any data or deliverable shall be retained by, or transferred to, the NHSBSA, as appropriate.

Suppliers who are shortlisted and then wish to submit proposals must register on the NHSBSA’s e-tendering solution which can be found at https://nhsbsa.bravosolution.co.uk/web/login.shtml

Expenses: to and from the Base Location in Newcastle are not eligible to claim. Any expenses outside of the Base Location must be agreed in advance with the NHSBSA’s Delivery Manager and shall be in accordance with the NHSBSA’s policy for Travel and Subsistence.

Cultural Fit assessed by Presentation - Dates TBC.

Skills and experience

Buyers will use the essential and nice-to-have skills and experience to help them evaluate suppliers’ technical competence.

Skills and experience
Opportunity attribute name Opportunity attribute value
Essential skills and experience
  • Demonstrate experience of successfully developing reusable predictive analytical models to detect and predict fraud/error(10points)
  • Demonstrate experience of successfully developing predictive models in relation to workforce data(10points)
  • Demonstrate experience of successfully productionising/embedding predictive analytical models into reports and operational systems(10points)
  • Demonstrate strong expertise and experience of using Oracle exadata/exalytics applications and infrastructure(10points)
  • Demonstrate experience of successfully upskilling/transferring knowledge to the internal customer's team in relation to predictive analytics(10points)
  • Demonstrate experience of successfully handing over the models to the customer, to allow the customer to maintain and refresh these models(10points)
  • Demonstrate experience of successfully working collaboratively with a client as part of a joint team(10points)
  • Demonstrate experience of successfully gathering continuous feedback from stakeholders on work and enhancing work as a result(10points)
  • Demonstrate experience of successfully embedding processes/ new ways of working into a client’s team, which lead to improvements for that client’s team(10points)
  • Demonstrate experience of following data security and information governance processes and procedures including in respect of special categories of personal data(10points)
  • Demonstrate how your team keeps abreast of new developments and techniques in relation to predictive analytics(10points)
  • Demonstrate experience of successfully flexing resources up and down at short notice within a given project to meet needs(10points)
  • Must be able to start work by 9 September at latest, co-locating team with NHSBSA in Newcastle base(10points)
Nice-to-have skills and experience
  • Demonstrate experience of successfully developing predictive analytic models in the banking or related sectors(10points)
  • Demonstrate experience of successfully developing predictive analytic models in the public sector(10points)
  • Demonstrate experience of successfully working collaboratively with other suppliers of a client to ensure successful completion of a project(10points)
  • Demonstrate experience of working successfully in several different industries/sectors(10points)
  • Demonstrate experience of working successfully using ‘agile’ methodology with an internal client team who have limited experience of agile working(10points)

How suppliers will be evaluated

How suppliers will be evaluated
Opportunity attribute name Opportunity attribute value
How many suppliers to evaluate 4
Proposal criteria
  • Broad activities to be completed as part of the first SOW as described in the requirements (re. developing a plan), including milestones/deliverables/dates(10points)
  • Identification of assumptions, dependencies and risks in respect of above mentioned plan(10points)
  • Description of proposed approach to understanding the relevant pharmacy, prescription and workforce data(10points)
  • How you will engage with stakeholders as part of above mentioned plan(10points)
  • Description of how you will develop predictive analytics models including proposed quality assurance and appropriate acceptance testing by the internal team(10points)
  • Description of proposed methodology for gathering stakeholder/user feedback on analytical models and adapting models as appropriate(10points)
  • Description of proposed methodology for productionising/embedding predictive analytic models into reports and operational systems(10points)
  • Explanation re how supplier will ensure that the predictive analytics models developed are capable of being maintained and refreshed by the NHSBSA internal team in future with minimal resources(10points)
  • Description of proposed methodology for transferring knowledge/up-skill BSA internal team(10points)
  • Description of how you will produce a portfolio of case studies/successful examples of work undertaken(10points)
  • Proposed Team Structure including CVs(10points)
  • Ability to scale resources to the demands of this project (including potentially delivering subsequent SOWs) including how you recruit/allocate staff if we require you to ramp up resources(10points)
Cultural fit criteria
  • Collaboration:-How will you ensure a BSA/Supplier one team ethos for delivery, including how you work as a team, manage collaboration across the team/with other suppliers, share know-how/best practice?(10points)
  • Adventurous:-How will you encourage innovation within the delivery team and how far will you take us beyond where we are now?(10points)
  • Reliable:-How will you ensure a consistently high quality of work across the delivery team?(10points)
  • Energetic:-How will you motivate and maintain high level of engagement with the delivery team?(10points)
  • How will you ensure a no blame culture and encourage people to learn from their mistakes?(10points)
  • How do you ensure that you work collaboratively with client teams which include some members with limited knowledge of predictive analytics and limited practical experience of agile working?(10points)
Payment approach Capped time and materials
Assessment methods
  • Written proposal
  • Presentation
Evaluation weighting

Technical competence

65%

Cultural fit

15%

Price

20%

Questions asked by suppliers

Questions asked by suppliers
Supplier question Buyer answer
1. Will there be a presentation to shortlisted suppliers from NHSBSA Insight Team during the procurement exercise? Yes.

There will be a presentation on Monday 29th July from the Insight Team to shortlisted suppliers, which will either be on-site and face to face, or via web-ex.

The presentation will be to re-iterate our requirements and answer any questions that shortlisted suppliers have.

Further details will be provided to those suppliers that are shortlisted.

Please ensure this date is held in diaries and we will confirm the method of engagement following the shortlisting process.
2. What is the date for the Cultural Fit presentations following the shortlist? Cultural Fit presentations will be held with shortlisted suppliers on Thursday 8th August 2019.

All suppliers that submit an application should hold this date in the diaries of the team who would be delivering the presentation if shortlisted (to include members of the delivery team).
3. One of the requirement states that there is a requirement to "subsequently maintain/refresh the models within the NHSBSA's internal team".

Can you please clarify who will be maintaining/refreshing the models?
This point in the requirement relates to NHSBSA internal staff maintaining and refreshing the models in future, not the partner.

Therefore the requirement relates to models being built by the partner in a manner that maximises ease of use/maintenance for NHSBSA, and ensure appropriate handover of the models to NHSBSA and up-skilling of internal staff where necessary.
4. Amendment to the date for Cultural Fit Presentations. Cultural Fit presentations will be held with shortlisted suppliers on Thursday 1st August 2019 in London.

All suppliers that submit an application should hold this date in the diaries of the team who would be delivering the presentation if shortlisted (to include members of the delivery team).
5. The link to the e-tendering website Bravo is not working? The link we have originally stated is incorrect.

Shortlisted suppliers will be requested to register at the following website on our e-tendering portal.

https://nhsbsa.bravosolution.co.uk/nhsbsa/login.html
6. Do you either require or prefer that public sector examples are cited rather than private sector examples? Or, if we have a relevant private sector example, will it be given equal consideration to a public sector example? As long as the example demonstrates suitable skills and experience then it can be public sector or private sector.
7. Will the scores from the evidencing round be taken through to final evaluation? Or will they only be used for the purposes of shortlisting suppliers? The scores from the evidencing round will be taken through to final evaluation.
8. In order to compare suppliers like for like, how will price be evaluated? Price will be evaluated based on 10% for the initial Statement of Works and 10% on a rate card which needs to be priced up using all SFIA levels 1-7.

Further details and pricing schedules will be provided to the shortlisted suppliers.
9. Will you be able to organise access to stakeholders throughout the project? Yes
10. As per DOS guidelines 'You should only provide one example for each essential or nice-to-have requirement', are you only requesting one example per skills and experience question? As per the buyers guide we will not mark down a supplier if they only provide one example.

https://www.gov.uk/guidance/how-to-shortlist-digital-outcomes-and-specialists-suppliers

Equally we will not score a supplier down on the basis of providing more than 1 example as long as the example/s demonstrates suitable skills and experience.
11. For the first evaluation round (evidence), what would you expect to see from an answer for it to be deemed 'exceeding' and score 3 marks? NHSBSA must adopt the scoring matrix of 0-3 in line with the DOS buyers guide which includes the terminology 0 - Not met or no evidence,
1 - Partially met, 2 - Met and 3 – Exceeded.

The basis that we will be evaluating is that scores will be awarded depending on the level of confidence that the evidence presented demonstrates that the criteria has been satisfied.

Suppliers will not receive additional points for demonstrating skills and experiences other than those that have been identified by NHSBSA in each of the published skills and experience criteria.
12. What roles are you expecting suppliers to provide? In addition, are you expecting all roles to have the ability to be full time? It is for the contractor to decide on the roles to fulfil the requirements. It may include but is not restricted to, roles such as Data Scientists, Predictive Modellers/Data Analysts, Big Data Engineers, Software Engineers, ETL Developers, Agile Manager, Counter Fraud Specialist.
13. Has any other supplier been engaged with this project so far? No. Prior to this project, a strategic review assessed the current state of data analytics and business intelligence within the Insight Directorate.
14. Has work has been fully defined, bounded and prioritised for the first SOW? The first piece of work is to develop a plan to:
Create reusable fraud and predictive analytical models to identify trends, patterns and anomalies in pharmacy payments and prescribing activities. Train our team in how to refresh the models.

Written plan and presentation may include:
• Data understanding
• Use cases
• How the model(s) will be developed
• Ethical considerations
• How the model(s) will be productionised/implemented for use by Provider Assurance
• Scalability, transferability, support and manageability considerations
• How the model success will be measured
• Expected value of fraud / savings
• Proposed knowledge transfer activities
15. Considering the requirement for “Demonstrate strong expertise and experience of using Oracle exadata/exalytics applications and infrastructure”, would you find it acceptable to respond in terms of a technology agnostic capability with experience working with a range of technologies and the ability to onboard new technologies but without specific experience with Oracle exadata/exalytics?

Considering the brief to “apply cutting edge fraud/predictive analytical models to data”, we believe the technology agnostic ability to develop mathematical models by applying a data science led thinking underpinned by an Agile philosophy may be equally worthy of consideration to specific technology experience.
Data is held in an Oracle database on an Exadata machine. The partner needs to be able to extract the data, develop analytical models and embed models back into existing reports/create new reports within our current Oracle Exadata infrastructure.
We are not seeking to invest in new technologies as part of this project.
16. Are there known restrictions on the choice of tools including open-source libraries for use in predictive modelling? The existing team use R open source software in addition to using Oracle exadata/exalytics applications and infrastructure.

Any solutions should not require the purchase of additional hardware or software.

The existing NHSBSA team must be able to maintain and refresh any models created.
17. Are there any known restrictions on the use of distributed networks of annotators if annotation or labelling of data is required? This will not be permitted as the data will not be leaving the BSA environment.
18. Will data provided be from real production sources without restriction, and if not what restrictions or obfuscation, or anonymisation would be necessary? The data is stored in our Data warehouse in an Oracle database on an Exadata machine. The data must be used within the BSA environment.

The supplier must comply with NHSBSA Information Governance and the NHSBSA Information Security Policy set including utilisation of NHSBSA supplied hardware and software in adherence with the Acceptable Use Policy. https://www.nhsbsa.nhs.uk/our-policies/policies-and-procedures
19. Are there other known technology restrictions or constraints? There are other uses of our database and Data Warehouse, so these impacts need to be considered. R is currently deployed locally, and so restricted by available RAM, but this is being looked into.
20. Do you know or can you give an indication of the duration of each SOW after the initial month? This will depend on the plan developed by the partner to some extent. We are aiming for all work to be completed within 7 months of commencement.
21. Whilst you have asked for ideas on team structure at the next stage, it would be good to understand the roles that you think will be required to complement the existing structure. It may include but is not restricted to, roles such as Data Scientists, Predictive Modellers/Data Analysts, Big Data Engineers, Software Engineers, ETL Developers, Agile Manager, Counter Fraud Specialist.
22. Under ES&E (Q6)...Is the experience expected specifically around handing over of the predictive models to the customer or solutions and maintenance in general ? Creating reusable models and upskilling/transferring knowledge to the internal customer's team to allow them to maintain and refresh these models.
23. ES&E (Q3): Can you please provide a more detailed definition of what is meant by "embedding predictive analytical models into reports and operational systems". A specific example would be helpful as the question is open to wide interpretation. We intend to use the analytical models to create fraud and error detection within operational systems and to improve business intelligence outputs. E.G.create a methodology for assessing when an individual claim for payment looks wrong based on historic data for the claimant or on other claimant activity which would then be used to improve rules within operating systems so that such payments were passed for further checks.

We want to use predictive analytics to enhance the information that we make available to the wider NHS through our business intelligence tools. E.G.alerting BI users to a forecast increased in medication spend.
24. ES&E (Q12): "Demonstrate experience of successfully flexing resources up and down at short notice within a given project to meet needs" – please define your SLA definition of "short-notice". Within a week.
25. Is co-location mandatory for this project or will remote working be considered? We require the partner to work as one team with the BSA delivery team, on-site for stakeholder meetings, knowledge transfer and up-skilling of internal staff. Remote working will be considered if you can demonstrate that this will achieve the outcomes and the security of our data will be upheld.
26. The specification states that no software will be purchased. Is this a mandatory requirement or is the NHSBSA open to a software-driven approach? This is a mandatory requirement.
27. Could contact details be made available for alternative approaches that do not meet the specific requirements of this opportunity? We are only interested in solutions and partners that can meet the requirements as detailed in this DOS procurement exercise.

We are not willing to share contact details to discuss alternative approaches at this time while we are in a formal competitive procurement exercise.
28. You say that you would like to use existing applications, including RStudio. Would you be open to the use of alternative open-source predictive analytics tools/languages, such as Jupyter Notebooks and Python, if these were suited to the task? The supplier must comply with the Information Security Policy set including utilisation of NHSBSA supplied hardware and software in adherence with the Acceptable Use Policy. https://www.nhsbsa.nhs.uk/our-policies/policies-and-procedures. Any solutions should not require the purchase of additional hardware or software.

The existing NHSBSA team must be able to maintain and refresh any models created. If alternative open-source software is to be used then this should be covered as part of any upskilling activity.
29. Would you be willing to consider working arrangements based on fewer than five days per week on-site at the Newcastle office? For example, a supplier's team could work on-site for two-three days per week to build capability in the NHSBSA team, and work remotely for the remaining days – with daily stand-ups and other meetings conducted via video conferencing. We require the partner to work as one team with the BSA delivery team, on-site for stakeholder meetings, knowledge transfer and up-skilling of internal staff. Remote working will be considered if you can demonstrate that this will achieve the outcomes and the security of our data will be upheld.
30. Will NHS BSA accept data masking as an option if a suitable SC person is not available? SC is a requirement of this project. We could sponsor SC if required, although the cost would need to be met by the tendering organisation.
31. Are you happy to sponsor security clearance where required? We would ask that staff have DBS (https://www.gov.uk/request-copy-criminal-record) before allowing them to commence and progress SC. If SC is rejected for any reason, the contract would need to be terminated. We could sponsor SC if required, although the cost would need to be met by the tendering organisation.
The deadline for asking questions about this opportunity was Thursday 11 July 2019.