NHS Digital

NHS Digital Website Evolution

Incomplete applications

21
Incomplete applications
10 SME, 11 large

Completed applications

8
Completed applications
7 SME, 1 large
Important dates
Opportunity attribute name Opportunity attribute value
Published Thursday 21 February 2019
Deadline for asking questions Thursday 28 February 2019 at 11:59pm GMT
Closing date for applications Thursday 7 March 2019 at 11:59pm GMT

Overview

Overview
Opportunity attribute name Opportunity attribute value
Summary of the work Improving digital experiences across the NHS Digital web estate using full stack agile teams to iteratively build upon the existing CMS platform and connected web applications, extranets and portals.
Latest start date Monday 1 April 2019
Expected contract length 2 years
Location London
Organisation the work is for NHS Digital
Budget range £1m

About the work

About the work
Opportunity attribute name Opportunity attribute value
Why the work is being done The Government asked the NHS to set out, by the end of November, a Long- Term Plan for the NHS which delivers on the existing Five Year Forward View commitments and sets out a five-year costed plan to deliver clear improvements and financial stability for the NHS. We will enrich the NHS Digital website estate to become an increasingly valuable portal for users across the system.
Problem to be solved • Focus on platforms, micro-services and data-services shared openly where viable
• Support for increasingly large and complex data sets, flowing faster
• Support for, and the encouragement of, interoperability
• Provision of increasingly sophisticated analytics capabilities
• Being the ‘go-to’ partner for researchers across Academia and Life Sciences
• Being a Centre of Excellence and a strong operational partner
• Providing expertise and clear guidance in Information Governance
• Coherent cross-system architectural guidance
• Leading the migration to public cloud
• Optimising the system’s relationship with technology suppliers
• Driving and fostering innovation, from concept to deployment and use
Who the users are and what they need to do There are multiple user needs to meet. First wave of this work the following high-level user needs are:
As a technical developer producing systems and services for the NHS
require access to resources, documentation, APIs, sandpits and microservices
So that I can produce the best technology, products and services that will enhance patient experiences across the NHS
As a supplier providing things for/to the NHS
Need to understand how best to access the knowledge, services, projects and programmes that NHS Digital provide
So that I can provide the best things to the NHS to enhance patient experiences across the NHS
Early market engagement None
Any work that’s already been done
Existing team A full stack DevOps team internal to NHS Digital consisting of 13WTEs who are multiskilled in Product management, UX, Design, Solutions Architecture and content delivery
Current phase Live

Work setup

Work setup
Opportunity attribute name Opportunity attribute value
Address where the work will take place Leeds (multi-site) tbc
London – Skipton House
Working arrangements Suppliers are requested to co-locate on either an NHS Digital site or supplier’s location. 4/5 days per week (but this is flexible). Agile ceremonies are undertaken and supplier staff are expected to join these whilst partaking in the work.
Security clearance N/A

Additional information

Additional information
Opportunity attribute name Opportunity attribute value
Additional terms and conditions NHS Digital have additional terms which I will add

Skills and experience

Buyers will use the essential and nice-to-have skills and experience to help them evaluate suppliers’ technical competence.

Skills and experience
Opportunity attribute name Opportunity attribute value
Essential skills and experience
  • Technology stack. It is essential that the supplier has skills and experience in the following: • Java/JavaScript/Bloomreach Hippo content management system/GitHub/Travis CI/JUnit or equivalent
  • Cucumber or equivalent/Selenium WebDriver or equivalent/Gatling or equivalent/RunDeck/CloudFormation/AWS S3
  • Jira /Confluence/Slack or Microsoft Teams/Ansible/ FreeMarker/CSS
  • Agile development. Supplier must be able to use relevant technologies in context of an Agile development programme working with and in blended teams, using Scrum or Kanban or of both.
  • Government Digital Service (GDS). The supplier must understand and be committed to helping the project meet the GDS Digital Service Standard.
  • Security and confidentiality. The supplier must understand and be committed to the confidentiality, data protection, and data and system security standards in use within government and the NHS.
  • Continuous integration / continuous deployment. The supplier must demonstrate they have experience in using modern CI / CD methods as part of an integrated DevOps process, utilising the Cloud.
  • Risk management. Supplier must demonstrate they have risk management processes that provide information suitable for NHSD to incorporate/build on for production of project and programme management products including risk registers.
  • Financial management. Supplier must demonstrate evidence of financial management processes to provide information suitable for NHSD to monitor/control project costs, incorporate, build on to produce project/programme management products/ financial reports.
Nice-to-have skills and experience Experience of working with brXM (previously Hippo CMS) platform

How suppliers will be evaluated

How suppliers will be evaluated
Opportunity attribute name Opportunity attribute value
How many suppliers to evaluate 3
Proposal criteria
  • Technical solution. Solution must be realistic, utilise core technologies listed (or their equivalents) and not introduce risk, technical debt, lead to divergence from agreed NHS Digital standards and technical roadmaps.
  • Approach and methodology. These must be compatible with NHS Digital’s commitment to Agile, user-centred and data-driven design as expressed in the GDS Digital Service Standard.
  • How approach or solution meets user needs. The proposal must show a realistic approach to developing solutions using an existing backlog, capturing, prioritising new requirements and stories as work progresses.
  • How the approach or solution meets your organisation’s policy or goal. Proposal should be drawn up in the context of relevant NHS and NHS Digital strategies published here (add link).
  • It will be evaluated by estimating how well the proposed work and overall supplier approach will contribute to the strategies, both now and in the future.
  • Estimated timeframes for the work. The proposal will be evaluated to determine if the work and the approach can realistically produce a minimum viable product within the time available.
  • Risks and dependencies. The proposal will be evaluated to determine how the supplier has identified risks and dependencies and offered approaches to manage them.
  • Team structure. Proposed team structure must be capable of supporting the full range of activities needed to fulfil the project’s objectives
  • Team Structure must maximise the potential of the existing NHS Digital staff and integrating relevant individuals into a blended team.
  • Value for money. The proposal must represent good value for money when compared with similar ones both within and outside NHS Digital and across different sectors
Cultural fit criteria
  • Work as a team with NHSD and other suppliers. We will seek evidence for an inclusive approach and a willingness to co-operate/collaborate, with practical mechanisms suggested to achieve this.
  • Be transparent/collaborative when making decision. Cover the decision-making process, set out the role of the key team members and the mechanisms by which the processes will be adopted (Agile ceremonies/reports/etc.)
  • How approach or solution meets user needs. The proposal must show a realistic approach to developing solutions using an existing backlog, capturing, prioritising new requirements and stories as work progresses.
  • Take responsibility for their work. We would expect the supplier’s proposal to show evidence of process for setting quality standards for their own work
  • Provide examples of how they would devise tests and information on how they would implement checks and reviews of their output.
  • Share knowledge and experience with other team members. Proposal should set out how information and knowledge is captured, shared and interpreted, in terms of day-to-day activity and in longer-term reporting/record-keeping.
  • Share knowledge and experience with other team members. Proposal should set out how information and knowledge is captured, shared and interpreted, in terms of day-to-day activity and in longer-term reporting/record-keeping.
  • Helping to review the alignment of deliverables with goals and to challenge ones that are potentially unnecessary or not achieving the project or sprint objectives.
  • Be comfortable standing up for their discipline. The supplier should be able to give examples of how they have justified and proven the value of their approach in the past.
  • Work with clients with low technical expertise. Supplier should be able to explain how they bridge gaps between technical expertise of their team and the business/specialisms of the NHSD team.
  • Enables both parties learn from each other, and how the full implications of any specialised technical discussions can be made clear to key team members including the product owner.
Payment approach Time and materials
Assessment methods
  • Written proposal
  • Case study
  • Work history
  • Reference
  • Presentation
Evaluation weighting

Technical competence

55%

Cultural fit

20%

Price

25%

Questions asked by suppliers

Questions asked by suppliers
Supplier question Buyer answer
1. There is a clear indication that Bloomreach Hippo CMS (and now brXM) is important to this project. Please can you confirm why this is the case and whether you are open to consideration of any other CMS? Thank you. brXM (formerly Hippo CMS) is the strategic choice of CMS for the NHS Digital website. No other CMS will be considered.
2. You have stated the solution must utilise the core technologies listed (or their equivalents), and there are also references to micro-services in the description. As an equivalent technology to the core technologies listed, would you consider micro-services built in Rails? Ruby isn’t ruled out – although there is an inhouse preference for Python or Node
3. You mention it is essential the supplier has skills and experience in the listed tech stacks "or equivalent" – does that mean you will consider expertise in alternative tech stacks? Expertise in other tech stacks will be but you must show how this is equivalent to what we have defined as the requirement.
4. Please can you hold a Q&A briefing for this to enable us to understand the requirements fully? Ideally this would be 26th/27th Feb. We can’t hold a call with individual tenderers we would have to speak to all.  The dates they propose are not achievable. Consideration will be given to hosting a future Q&A/call. Early notification will be given.
5. It could be any from the full stack suite of DevOps roles. Typically it will be to fill a resource gap or capability gap. Typical roles are listed

https://www.gov.uk/service-manual/the-team/what-each-role-does-in-service-team#roles-your-team-must-have
It could be any from the full stack suite of DevOps roles. Typically it will be to fill a resource gap or capability gap. Typical roles are listed https://www.gov.uk/service-manual/the-team/what-each-role-does-in-service-team#roles-your-team-must-have
6. Are NHS Digital able to recruit and provide access to users for the purpose of user research? Yes we can in most instances. Where we need access to specific users we use an agency to source them.
7. Is there a preferred technology stack within the NHS Digital estate? The website runs on the Bloomreach DXP. We code with Java and some js. Web apps often in Python or Node.
8. Will NHS Digital be able to organise access to stakeholders throughout the project? Yes both internal and external
9. Can you host a call with suppliers about this opportunity where we can have a live Q&A and the discussion notes are shared on this tender ad? This has proven helpful on other DM opportunities. We can’t hold a call with individual tenderers we would have to speak to all.  They dates they propose are not achievable. Consideration will be given to hosting a future call. Early notification will be given.
10. Will there be a full-time Product Owner provided by NHS Digital to this project? Yes
11. What roles are NHS Digital expecting suppliers to provide? In addition, are you expecting all roles to have the ability to be full time? It could be any from the full stack suite of  DevOps roles. Typically it will be to fill a resource gap or capability gap. Typical roles are listed https://www.gov.uk/service-manual/the-team/what-each-role-does-in-service-team#roles-your-team-must-have
12. Can you describe the current team setup, including numbers of each role, and how you see additional teams fitting in? We have a full stack DevOps team (up to 32 people) using Agile working methodologies. Roles flex in and out as needed within a squad often depending on the phase of the project / delivery. We may need multiple roles from a supplier depending on our commitments. these roles would be expected to integrate with the existing team. This often required Co location. Leeds. London. Exeter. Or on supplier site in some instances.
13. What are the main epics in you backlog? The detail of these will not be shared at this stage. They are in constant evolution and are being delivered by internal teams. Themes can be picked up on the 10 year plan regressed by the DHSC recently and our organisational strategy plans available on our website.
14. Have you defined an MVP? If so, what is it? The website is live and will be further developed. Digital.nhs.uk
15. For the purposes of calculating possible expenses, which location will be deemed the main location for this work? Leeds
16. Who from your team will be scoring applications for this opportunity and what positions do they hold? Service Head, Lead Product Owner, Delivery Manager.
17. Are you looking for an organisation with GDS assessment experience or just certain individuals proposed in the supplier team? Both preferably
18. You've listed brXM as both an essential requirement and a nice-to-have? Is that a copy error, or are you only interested in suppliers with experience in brXM (i.e. not transferable skills from other CMS experience)? brXM is a core strategic CMS product for NHSD. We will consider cross transferable skills.
19. Can you provide an indication of the specific security and confidentiality standards that you believe to be pertinent to this project? I won’t deliver into the detail but we consider the following key areas to be applicable
https://digital.nhs.uk/data-and-information/looking-after-information/data-security-and-information-governance/codes-of-practice-for-handling-information-in-health-and-care

https://digital.nhs.uk/data-and-information/looking-after-information
20. Micro-services and data-services are offered from NHS or will these be
offered by other providers whom NHS has partnered or will partner with in future? Will there be a need for integrating with any third party service providers or partners for NHS digital?
All of the points you raise are possible scenarios for NHS Digital. A supplier will be expected to react to these situations in collaboration with NHS Digital and our partners.
21. What is size of the current data. How much data is being added annually? That’s a difficult question to answer. NHS Digital is responsible for collecting all the data from across the NHS ecosystem in England. Much of this is published via the NHS Digital website. Large data sets include GPES and HES.
22. Will there be a need to integrate with third party partners? Will these systems comply with open standards or industry accepted interoperability process like Web-Services or API for integration? Multiple integrations are likely to be required. We will work to the standards set by the NHS and as policy from Government dictates.
23. What are current technologies adopted at NHS digital Where is it being currently hosted? In relation to what aspect of what NHS Digital does is this question asking?
24. Who are all the technology suppliers and how do they integrate currently with the NHS digital website. What is the plan and roadmap for the future NHS Digital integration? Main tech supplier for the NHS Digital website is Bloomreach. It’s presently Saas / PaaS
25. Is it mandatory to have worked on Hippo CMS. Will NHS consider the experience of working on other CMS too? Bloomreach is a strategic choice of CMS for NHS Digital. Cross transferable skills will be considered.
26. A user story is a great start to communicating the requirements of this project. Could you elaborate more on the outcome you are looking for? There are too many epics to list at the present moment. You can understand the direction of travel from the NHS 10 year plan as release by the SoS recently.
27. What are the NHS technical roadmaps? In this context we are referring to technical architecture roadmaps. These define the systems and infrastructure we are / plan to use.
28. What key dependencies have you already identified to achieve the expected outcome? There are many dependencies across multiple projects, workstreams and programmes within and connected to NHS Digital. These exist already and as we develop and evolve the strategy to meet the objectives of the 10 year plan we will share these and collaborate on these with the chosen supplier.
29. Have you already been working with an existing supplier on your Bloomreach platform in 2018-19? Will they be bidding to win this new funding for the next financial year? The requirement has been published on Digital Marketplace. We are unaware of the identity of suppliers who are currently responding or their intentions.
30. Are you looking for the supplier to be embedded within your existing team or the other way around. In other words, are you looking for us to lead the project? There is potential for both. But normally we adopt a shared delivery where you and we collocate.
31. Can you please detail what user research you have done to date? Significant amounts of user research have been conducted across many sections and user types spanning the breadth of our offers. We are committed to continuing user research as part of our GDS aligned development process. We will share our user research with the chosen supplier.
32. Your user needs mentioned two groups of users a) technical developer, and b) supplier. Are any other group of users? How are each group of user similar and or different? And how many sub-roles are within each user group? NHS Digital has many user types / groups -based on the wide breath of products, services, infrastructure and data that is our business. These are documented internally and form part of our development approach aligned to GDS service design principles.

Some core high level users groupings

Internal to NHS
• Commissioners
• Health & Social Care Doers
• Health & Social Care IT Implementors
• NHS Digital workforce
• Partners

External to NHS
• Partners
• Citizens / Patients
• Technical community
• Research & Life Sciences
• External Care Orgs
• 3rd sector
33. What is considered by your organisation as a minimum viable product and what time frame is available? MVPs already exist across our live websites. This work is about evolving these offers.
34. "In your “Problem to be solved” section, you have a mix of items that are organisational aspirations, platform specific etc.
Would you be looking to your partner to help with the strategic alignment of those problems?
OR
Would you be able to share your strategy and plan to solve the problems listed in your “Problem to be solved”?"
This is unlikely to be a requirement.
35. Do you have an existing supplier? There are multiple suppliers involved in the delivery of multiple NHS Digital websites. The NHS Digital website (digital.nhs.uk) supplier is Bloomreach.
36. Please confirm which NHS websites fall under this tender? Scope is presently

Digital.nhs.uk
Developer.nhs.uk

Other sites may be added latterly.
37. Is there any defined milestone or phase for this project?
Is there any detailed KPI expectations and Acceptance Criteria?
Hence there is no information about the volume of raw data without replication, could you please clarify how many portion of mentioned data is unstructured?
Does this solution require any certain level of DOS protection?
The procurement intention is to appoint a supplier to assist and augment the team within NHS Digital.

The roadmap and plan are still evolving. We currently have a roadmap of activity planned for digital.nhs.uk and we are developing one for developer.nhs.uk.

KPIs and acceptance criteria will be a part of backlog items defined using agile working practices. These will be based on high level goals, objectives / outcomes.

Solutions to our web estate will have cyber security requirements. For clarity our strategic CMS is already approved and security tested.
38. Will the scores from the evidencing round be taken through to final evaluation? Or will they only be used for the purposes of shortlisting suppliers? The scoring process will be as per the DOS guidance. NHS Digital intends scoring from the shortlisting stage to be taken through to the final assessment evaluation scoring.
39. For the first evaluation round (evidence), what would NHS Digital expect to see from an answer for it to be deemed 'exceeding' and score 3 marks? The evaluation will be conducted in accordance with the Digital Outcomes and Specialists buyers’ guide: https://www.gov.uk/guidance/how-to-evaluate-digital-outcomes-and-specialists-suppliers. Beyond the guidance that has already been provided, NHD Digital will not advise Bidders how to respond to specific requirements in order to gain maximum scores. There is an expectation that Bidders should competently provide responses that are relevant by describing how their organisation meets each of the requirements and, if applicable, articulate within their responses how their organisation may exceed the requirement, provided it is within the context/scope of the requirement and overall solution as a whole.
40. Have you chosen a front-end framework and if so, what is it? We use a jQuery custom implementation and also Freemarker
41. In order to compare suppliers like for like, how will price be evaluated? We will evaluate price in line with the evaluation criteria set out on the digital marketplace. Price is evaluated on how close each suppliers price is to the cheapest quote and is weighted by 25%
42. As per DOS guidelines 'You should only provide one example for each essential or nice-to-have requirement', are NHS Digital only requesting one example per skills and experience question? Yes
43. Please can you provide details of what the next steps are following this initial application i.e. deadlines for the written proposal/presentation? The closing date for applications is Thursday 7 March 2019 at 11:59pm GMT. Once all applications are received we will evaluate to produce a shortlist of suppliers to take forward to the next stage in the process
44. Are you working with any existing suppliers in relation to this tender opportunity? If you are, who are they? There are multiple suppliers involved in the delivery of multiple NHS Digital websites. The NHS Digital website (digital.nhs.uk) supplier is Bloomreach.
45. What are the additional terms and conditions that will be added? "Any alternative clauses can be found at
https://www.gov.uk/guidance/g-cloud-templates-and-legal-documents "
46. What roles are NHS Digital expecting to provide and will these all be full time? It could be any from the full stack suite of DevOps roles. Typically it will be to fill a resource gap or capability gap. The request could be for both full or part time roles. Typical roles are listed https://www.gov.uk/service-manual/the-team/what-each-role-does-in-service-team#roles-your-team-must-have
47. There is a missing link to the relevant NHS and NHS Digital strategies. What is this link? Our strategy can be found https://digital.nhs.uk/about-nhs-digital/our-work
https://digital.nhs.uk/about-nhs-digital/corporate-information-and-documents/our-strategy