HM Court and Tribunal Service (HMCTS) & Crown Prosecution Service (CPS)

HMCTS CPP Test Automation & Execution Services

Incomplete applications

14
Incomplete applications
12 SME, 2 large

Completed applications

12
Completed applications
8 SME, 4 large
Important dates
Opportunity attribute name Opportunity attribute value
Published Wednesday 30 August 2017
Deadline for asking questions Wednesday 6 September 2017 at 11:59pm GMT
Closing date for applications Wednesday 13 September 2017 at 11:59pm GMT

Overview

Overview
Opportunity attribute name Opportunity attribute value
Summary of the work Provide a flexible team of up to 10 testers to work on agile projects which deliver specific products/transformations via multi-disciplinary, skilled digital teams. You’ll be adept at ensuring the quality of the digital service by testing it manually and writing automated tests covering a range of conditions.
Latest start date Sunday 15 October 2017
Expected contract length 24 months
Location London
Organisation the work is for HM Court and Tribunal Service (HMCTS) & Crown Prosecution Service (CPS)
Budget range Target day rate is circa £700 per day.

About the work

About the work
Opportunity attribute name Opportunity attribute value
Why the work is being done CJSCPP is a change programme to enable the CJS and agencies be more efficient and deliver a better service. For the first time, criminal case information will be kept together in one place. The CJSCPP is part of a wider portfolio of change being delivered by HMCTS under the HMCTS Reform Programme, affecting services in all jurisdictions.
Problem to be solved Current CJS services are typically supported by paper and hard media based processes which are expensive, subject to frequent delay and often baffling to citizens without legal training. The CJSCPP is designed to develop efficient, proportionate and accessible digital services delivering access to justice via appropriate modern channels.
Who the users are and what they need to do 1) As a citizen, view and track my case online, be alerted to changes and for the case to proceed efficiently. 2) As a prosecutor, understand case evidence to ascertain likelihood of a successful prosecution. 3) As a defence practitioner, understand the evidence of a case to best advise my client. 4) As a HMCTS admin, manage the delivery of justice services so the process is delivered efficiently. 5) As a member of the judiciary, access all relevant information so that I can make a judgement 6) As a prosecuting agency, submit evidence to provide HMCTS with relevant information.
Early market engagement
Any work that’s already been done The CJSCP Programme has produced the technical foundations for a domain driven micro-service based back end architecture with service model architecture on the front end. These include frameworks and design patterns that new team members will be expected to worth with. Information on this work can be found here https://github.com/CJSCommonPlatform/microservice_framework and here https://github.com/CJSCommonPlatform/govuk_single_page_pdk. The business has identified approximately 35 service/service fragments to be produced and the change planning has taken place with approximately 7 of the services in various stages of completion.
Existing team The current projects all have teams in place. Each existing team comprises; a Delivery Manager, a Technical Lead, a Business Analyst, a User Researcher, a UX/HTML Designer, one or more front end and back end developers and a QA specialist. It is expected that the successful provider will augment some of these teams as well as providing Test Automation & Execution capabilities for new teams yet to be formed, either to extend the capacity on existing projects, or provide the capability for the other projects due to start during 2017.
Current phase Not applicable

Work setup

Work setup
Opportunity attribute name Opportunity attribute value
Address where the work will take place Primarily Southern House, Wellesley Grove Croydon CR9 1WW and occasionally Rose Court, 2 Southwark Bridge Road London SE1 9HS (the "Base locations”, although some visits to other locations may be required.)
Working arrangements The supplier’s team will be required to work onsite at the base location working a five (5) day week. The Service is being delivered against the GDS Service Manual (e.g. agile delivery aligned to scrum methodology). For any work performed at a location different to that of the Base location and London (outside of the M25), all reasonable travel and expenses costs shall be met in accordance with the rates set out in the MoJ travel and subsistence policy. All expenses will require prior approval from HMCTS before being reimbursed.
Security clearance All supplier resources must Baseline Personnel Security Check (BPSS) clearance which must be dated within three months of the start date. See https://www.gov.uk/government/publications/government-baseline-personnel-security-standard for further guidance. Some roles may require SC security clearance.

Additional information

Additional information
Opportunity attribute name Opportunity attribute value
Additional terms and conditions Any expenses shall be submitted in line with the Ministry of Justice standard Travel and Subsistence policy and agreed in advance of expenditure. Suppliers shall provide transparency to the Buyer on the rates paid to resources and any third parties in the supply chain.

Skills and experience

Buyers will use the essential and nice-to-have skills and experience to help them evaluate suppliers’ technical competence.

Skills and experience
Opportunity attribute name Opportunity attribute value
Essential skills and experience
  • Demonstrable experience of producing and maintaining automated checks using Protractor=3%.
  • Demonstrable experience of producing and maintaining automated checks using RestAssured or RestEasy=3%.
  • Demonstrable experience of producing and maintaining automated checks for OWASP using ZAProxy=3%.
  • Demonstrable experience of producing and maintaining automated performance checks using JMeter=3%.
  • Demonstrable experience of coding automated checks in Java, following typical design patterns and good practices=3%.
  • Demonstrable experience of coding automated checks in JavaScript to a high standard, following typical design patterns and good practices=3%.
  • Demonstrable experience of quantifying application and infrastructure quality through measures including; performance, vulnerabilities, defects and static analysis results=3%.
  • Demonstrable experience of providing expert exploratory testing across multiple scrum teams=3%.
  • Demonstrable experience of conducting Non-functional and Operational Acceptance testing=3%.
  • Demonstrable experience of completing all testing activities within Sprint boundaries=1%.
  • Demonstrable experience of collaborating with developers to push automated checks down the Automation Pyramid=2%.
  • Demonstrable experience of operating within a Scrum team=2%.
  • Demonstrable experience of providing advice and guidance to teams on what to test, where to test and how to test features=2%.
  • Demonstrable experience of implementing a Programme Test Approach within development teams=2%.
  • Demonstrable experience of working across a programme to contribute to Platform automated checks and activities=2%.
  • Demonstrable experience of working with Continuous Integration and infrastructure as code tools (Jenkins and Ansible on Azure)=1%.
  • Demonstrable experience of providing guidance to developers when issues occur utilising Event debugging tools, Dead Letter Queues, Monitoring, Alerting and aggregated logs in Kibana=1%.
Nice-to-have skills and experience
  • Demonstrable experience of CQRS and/or asynchronous systems=1%.
  • Demonstrable experience of Continuous Delivery in Azure=1%.

How suppliers will be evaluated

How suppliers will be evaluated
Opportunity attribute name Opportunity attribute value
How many suppliers to evaluate 3
Proposal criteria
  • Describe how will you ensure that HMCTS is supplied with high quality and appropriately skilled team specialists, whilst minimising HMCTS's effort in screening the proposed team members=3%.
  • Explain how you plan to retain key resources for the duration of the contract and how you can commit to meet HMCTS's continuous need for testers=2%.
  • Provide 6 named individuals and CV’s available to meet the initial requirements demonstrating experience of the essential skills and experience and working in agile multi-vendor teams=10%
  • Provide three referenceable client focussed case studies where your company have provided the desired service capability; please identify the proposed team member(s) that were part of the delivery team=2%
  • Describe your approach to adding, replacing and rotating new team members, within a 2 week notice period, to the existing team without reducing the team’s velocity=2%.
  • Propose your approach for enabling digital teams, delivering capability and ensuring that knowledge transfer is conducted during handover of project outputs to the HMCTS BAU maintenance team=3%.
  • Describe how you intend to measure and manage quality and speed of delivery teams and individuals to ensure continuous improvement=2%.
  • Give examples of KPIs and SLAs that you would be prepared to commit to for this contract, including ability to provide high calibre substitutes/replacements within 2 weeks=2%.
  • Explain the Value Proposition; value for money, how the proposal will optimise costs, and generate savings. Demonstrate how discounts will be linked to volume and duration=2%.
Cultural fit criteria
  • Explain the approach for working with multi-vendor teams across multiple projects at different locations=2%.
  • Explain how you will ensure the team collaborate at all levels of the project and programme delivery; between users, project team members, and management. Give examples for each team member=3%.
Payment approach Time and materials
Assessment methods
  • Written proposal
  • Case study
  • Reference
Evaluation weighting

Technical competence

70%

Cultural fit

5%

Price

25%

Questions asked by suppliers

Questions asked by suppliers
Supplier question Buyer answer
1. Will sponsorship be giving to qualified candidate if they don't have clearance? Yes. But suppliers should have there own security clearance processes in place to ensure that delays to starters and substitutions are minimised. Bidders will note that there are various evaluation criteria on speed of substitution and the effects on velocity of the bidders processes.
2. Please advise, how many testers you would expect to be available from the start of the engagement (Sunday 15th October?) - and the profile of resources you would expect to be available over the 2 year period? We expect an initial 6 to be available to start at the beginning of the engagement. Over the course of the period, we expect a maximum of around 15 people to be required.

It should have read Monday 16th October
3. Please can you advise, in terms of SFIA Grade, what levels of staff are you expecting to carry out this work? We don’t use SIFA Grades. The testers with all of the skills listed will typically be experienced/senior testers with strong development backgrounds.
4. Does the programme currently use a “traditional” test automation process or does it use BDD and/or TDD and therefore is there a need for Cucumber and Gherkin knowledge in the team? It is very possible to do BDD without Cucumber - Cucumber is just a tool its not the practice. We use BDD tools for expressing behaviours to be checked within our domain model. We use TDD everywhere. Cucumber/Gherkin is not a requirement.
5. Three of the skills and experience we would like clarified:

a. Demonstrable experience of coding automated checks in Java, following typical design patterns and good practices – does this include the use of Selenium using Java?
A). Possibly. Selenium is not listed as a skill needed at all for the programme,infact we don’t use it. We use Protractor for UI tests. We’re looking for a general competency in Java here. Experience of coding automated checks in Selenium is possibly beneficial, but use of other tools or practices could also possibly demonstrate capability here;unit testing or building automated checks against rest services could also possibly show this skill. We’re not looking for people who can throw together some Java and live with it, we need solid coders who understand and insightfully use design patterns and good practices.
6. B. Demonstrable experience of coding automated checks in JavaScript to a high standard, following typical design patterns and good practices. – Is this implying a specific tool or tool set e.g. Jasmine? B). No. The skills listed in the opportunity require language and tools separately. This section is asking for JavaScript skills.
7. c. Demonstrable experience of working across a programme to contribute to Platform automated checks and activities. – Is this implying the use of automated tests at all test levels including smoke tests? C). No. We’re looking for people who are capable of working on a specific project with a capability of keeping an eye on the bigger picture. CPP is developing a series of services that sit on a “Common Platform”. Testers must be mindful of platform level impacts of automation choices as well as impacts of other testing decisions on the platform as a whole.
8. Is there an overall design that can be reviewed?
And how many users will this overall system will need to handle? Number of concurrent users?
This information is not relevant to supply an effective bid for this work.
9. Could you clarify if the roles would fall inside of IR35? CPP currently assess this contract to be a Services contracts and outside of the scope of IR35. We are not in a position to confirm to bidders with certainty any future assessments of IR35 as those will be subject to review by MoJ and HMRC. Bidders on this competition should take into account, particularly in regard to pricing, that services may be deemed to be in scope of IR35 by other parties beyond our control
10. Are all the vendors based in the UK or will some be off shore? All based in the UK, all delivering on MOJ site in Croydon.
11. Is the programme using a purest form of scrum where the resulting code is operational and put into production after each sprint or a number of sprints are performed and result in a Minimum Viable Product (MVP) which then may undergo further testing prior to putting into production? Neither description is quite there. Testing happens in Sprint, including non-functional and operational testing. Every Sprint doesn’t lead to a release, but we are actively working on reducing the time between releases.
12. Given the programme is already underway what test documentation already exists? There is a programme Test Approach. Most of the documentation is held in the code base, expressed as automated checks.
13. Is there a management structure into which this new team needs to fit? If so may we know what it is? Testers will work with individual delivery teams to deliver pieces of the programme with an eye on the bigger picture and where they fit in. There is a wider Test Practice across the programme to which all the testers will belong, which is there to facilitate learning and innovation across the programme.
14. Will there be an opportunity to influence how the testing is done or is this already set out? The test approach and tooling is set out and is unlikely to change in the short to medium term. As the programme moves through implementing additional features, the test approach will obviously evolve with this. We don’t need the winning supplier to come and change the approach, but rather help us realise it.