HM Courts and Tribunal Service (HMCTS)

Common Platform Programme - Testers

Incomplete applications

0
Incomplete applications

Completed applications

27
Completed applications
17 SME, 10 large
Important dates
Opportunity attribute name Opportunity attribute value
Published Monday 15 August 2016
Deadline for asking questions Monday 22 August 2016 at 11:59pm GMT
Closing date for applications Monday 29 August 2016 at 11:59pm GMT

Overview

Overview
Opportunity attribute name Opportunity attribute value
Summary of the work Provide a flexible team of 3 to 5 testers to work on agile projects which deliver specific products/transformations via multi-disciplinary, skilled digital teams. You’ll be adept at ensuring the quality of the digital service by testing it manually and writing automated tests covering a range of conditions.
Latest start date 26th September 2016
Expected contract length 2 years. The first SOW will be for a 3 month duration with other SOWs developed over the term.
Location London
Organisation the work is for HM Courts and Tribunal Service (HMCTS)
Budget range Suppliers should note that proposals are assessed on DOS using the cheapest price. For this requirement we will be using this methodology but applying it against the average day rate of the roles proposed to compare bids on a like for like basis.

About the work

About the work
Opportunity attribute name Opportunity attribute value
Why the work is being done The CJS Common Platform Programme is a business transformation programme which will enable the criminal justice system as a whole, and the individual agencies working within it, to fundamentally change the way they work, to be more efficient and effective and deliver a better service for society and taxpayers. For the first time, criminal case information will be kept together in one place, instead of being re-entered and replicated on numerous different systems. Parties will be able to access and update the same data set based on their security profile and associated access rights.
Problem to be solved The CJS Common Platform Programme is looking to expand the support team to help HMCTS and CPS transition and run great digital services. Suppliers resources must have strong Tester skills and be ready to quickly get to work within a multi-disciplinary team of developers, designers, editors and analysts focussed on one or more products or projects providing services capable of running high volume, reliable and flexible services that are relentlessly user-focussed and continually improved through iterative development.
Who the users are and what they need to do The focus is on systems in Her Majesty’s Courts and Tribunals Service (HMCTS) and Crown Prosecution Service (CPS), in partnership with the legal defence community, Police, NOMS and other prosecuting authorities with respect to cases heard in the Magistrates’ and Crown Courts in England and Wales. benefits include: • that staff are working from and sharing the same set of accurate data; information being readily available and easily accessible; • reduction of workplace duplication and the need for paper based systems ; increased efficiency and productivity through joined up, flexible and innovative ways of working.
Early market engagement
Any work that’s already been done We have completed a series of discovery and alpha phases for the various strands of work within the service.
Existing team The supplier will be working with Civil Servants and multiple Suppliers across various strands of work for delivery of the Programme.
Current phase Beta

Work setup

Work setup
Opportunity attribute name Opportunity attribute value
Address where the work will take place Primarily Southern House, Wellesley Grove Croydon CR9 1WW and occasionally Rose Court, 2 Southwark Bridge Road London SE1 9HS.(the "Base locations”, although some visits to other locations may be required.)
Working arrangements The supplier’s team will be required to work onsite at the base location working a five (5) day week. The Service is being delivered against the GDS Service Manual (e.g. agile delivery aligned to scrum methodology). For any work performed at a location different to that of the Base location and London (outside of the M25), all reasonable travel and expenses costs shall be met in accordance with the rates set out in the MoJ travel and subsistence policy. All expenses will require prior approval from HMCTS before being reimbursed.
Security clearance Disclosure Scotland. See www.disclosurescotland.co.uk/basicdisclosureonline/index.htm

Additional information

Additional information
Opportunity attribute name Opportunity attribute value
Additional terms and conditions Any expenses shall be submitted in line with the Ministry of Justice standard Travel and Subsistence policy and agreed in advance of expenditure.

Skills and experience

Buyers will use the essential and nice-to-have skills and experience to help them evaluate suppliers’ technical competence.

Skills and experience
Opportunity attribute name Opportunity attribute value
Essential skills and experience
  • Coordinate what, where and how to test features within a delivery team
  • Define and implement appropriate automated functional checks/tests within programme approved toolkits
  • Lead exploratory testing in each story
  • Define and implement non-functional tests within programme approved toolkits and define and implement operational tests
  • Assist delivery team with story shaping and kick off and Provide quality measures and metrics for the delivery
  • Test against shared understanding of Done with team (and help understand what’s important)
  • Able to quickly understand the domain and the different users of the system and the external qualities they need and identify what to test to achieve those qualities
  • Knowledge of automation practices and where they help most
  • Understand the automation test triangle, how things go wrong and what right should feel like
  • Strong collaboration: 1 on 1 leading and guiding skills
  • Systems understanding - ability to reason about the system being built, what it is for and where its weak points could be
  • Ability to highlight debt when team slips
  • Ability to support a progressive methodology to quality improvement
  • Ability to code in Java or JavaScript
  • Ability to demonstrate what to test and where
  • Understanding of OWASP and implications for testing applications
  • Demonstrable ability to collaborate with developers
  • Must not have predominately been working as WebDriver framework developers and must not have been working outside of our tech stack for a year or more
  • Must be able to work at a tier no lower than a User Interface
  • Must be able to work with Web Services. REST is preferred. SOAP experience only is not a barrier to entry.
Nice-to-have skills and experience
  • Understanding of open source testing tools, like Protractor, JMeter & RestAssured and commercial tools like BurpSuite
  • Commercial experience of CQRS and/or asynchronous systems
  • Experience of performance, operational and security testing
  • Rapid Software Testing course qualification
  • Experience of Continuous Delivery

How suppliers will be evaluated

How suppliers will be evaluated
Opportunity attribute name Opportunity attribute value
How many suppliers to evaluate 5
Proposal criteria
  • Overall Technical Solution and Approach - High Level Delivery Plan, Approach and Method, Timescales, Risks and Dependencies and Quality (30%)
  • Proposed team structure (including CVs of key members) - Reference contract/ case studies examples including linked CV’s (20%)
  • • Value for Money - how the proposal will optimise costs and generate savings. (10%)
Cultural fit criteria
  • Proposed approach for working with co-located, multi-vendor HMCTS team - demonstration of transparency and collaboration, including ability to engage and work with users at all levels (4%)
  • Proposed approach for enabling digital team and capability. (3%)
  • Have a no-blame culture and encourage people to learn from their mistakes (3%)
Payment approach Time and materials
Assessment methods
  • Written proposal
  • Work history
  • Presentation
Evaluation weighting

Technical competence

60%

Cultural fit

10%

Price

30%

Questions asked by suppliers

Questions asked by suppliers
Supplier question Buyer answer
1. What is the maximum daily rate please? The Authority has not set a maximum day rate, however you should note that price makes up 30% of the evaluation score.
2. We would be very interested in submitting a response as a supplier for this requirement. I would like to attend the Q&A session on the 18th, can I book a place? Arrangements for the Q&A session on the 18th will be confirmed shortly. All interested suppliers are welcome.
3. It appears by the skill set mentioned that the requirement is for blended QA testers with deep, cross discipline test engineering expertise e.g. functional, security, etc. However, please can you confirm that this is the requirement? Yes. We're looking for highly technical (ideally ex developers) testers who can help our teams to understand how to test and where to test the features they implement. We want our teams to be able to prove in-Sprint that their software works, so understand non-functional and operational testing is important to us.
4. As you are seeking 3 / 5 testers and average pricing, is it your intention to select one supplier, or might you consider multiple suppliers to fill these roles? We are looking for one primary supplier who may partner with other suppliers to provide resources as necessary . The contract however will be with the Primary Supplier.
5. Please keep me informed about the proposed meeting on the 18th as we would like to attend. Thanks To confirm that the Q&A session will take on Thursday 18th August 2016 at Rose Court, 2 Southwark Bridge Road, London, SE1 9HS from 3pm to 4pm. Attendance is limited to two representatives per supplier. Please confirm if you wish to attend and your attendees.
6. Owing to prior engagements, will it be possible to dial-in to the Q & A session? Number: 0800-917-1956
Participant's Code: 2397 1218#
From mobile phones, call 0208-839-3652
7. If you are seeking 3 to 5 Testers, when the next phase of this procurement commences, e.g. proof of skills and experience with supporting CVs, will you be requiring 3, 5, or a greater number? A minimum of 3.
8. Is there a protocol for joining the Q & A session? Do we have to confirm in advance of attending?

We would like to reserve a place at tomorrow's Q&A event. Please let me know how we may confirm a place.
Yes confirm your attendees in advance via this portal.
9. How do we contact you to confirm our attendees for the Q&A? Is it through this question feature? Thanks Yes
10. Bidder Event - Q&A Session 18th August 2016
Q1 What are the security clearance requirements?
A1 Disclosure Scotland, issued within three months.
11. Bidder Event - Q&A Session 18th August 2016
Q2 Are CVs judged on the numbers of years of experience held in a particular field, i.e. would a junior tester be considered?
A2 No, we want testers who meet our stated requirements.
12. Bidder Event - Q&A Session 18th August 2016
Q3 What do you not want to see on CVs?
A3 We don’t want to see candidates primarily with the experience in building tests in the user interface, or building tools and not focusing on testing. If candidates can’t identify how to split up tests between automation pyramid layers we do not consider them to meet our requirements. People who come from a test driven background tend to be more successful.
13. Bidder Event - Q&A Session 18th August 2016
Q4 Who are CPPs stakeholders?
A4 There’s an introduction the Common Platform Programme available on the Inside HMCTS Gov.UK blog here: https://insidehmcts.blog.gov.uk/2016/06/30/introduction-to-common-platform-programme/
14. Bidder Event - Q&A Session 18th August 2016
Q5 How are the current team steam's setup, are they agile teams?
A5 There are many teams on within the programme; any appointed testers from this Outcome procurement will be embedded in different teams which contains a Delivery Manager, BPO, CS lead, BAs, User Researcher. There are approximately 1 tester to 5 developers.
15. Bidder Event - Q&A Session 18th August 2016
Q6 Is the Delivery Manager the scrum master?
A6 Yes
16. Bidder Event - Q&A Session 18th August 2016
Q7 What are the numbers and timescale you’re working to?
A7 3-5 CVs though that number could increase as programme is running until June 2018, we want to make sure we partner up with a supplier who can consistently provide us with quality resources.
17. Bidder Event - Q&A Session 18th August 2016
Q8 In terms of testing teams and the environments they’ll be using, what are they?
A8 SDLC is moving away from having separate test teams, recently having procured Azure, the new SDLC is pushing testing into teams. Testers will need to interact with different teams as code is shared by all teams on the platform.
18. Bidder Event - Q&A Session 18th August 2016
Q9 You’re expecting the Scrum team to be self-sufficient?
A9 Yes
19. Bidder Event - Q&A Session 18th August 2016
Q10 Will the 3-5 testers be on the same team?
A10 No. They’ll be embedded in different teams.
20. Bidder Event - Q&A Session 18th August 2016
Q11 Is a multi-skilled tester preferred?
A11 Yes. We’re looking for ‘skilled generalist’.
21. Bidder Event - Q&A Session 18th August 2016
Q12 Can you explain the process and procedure going forward from here?
A12 Part 1 The process is it goes through a number of stages,

Stage 1: Suppliers Yes/No responses to criteria published in requirements. Suppliers who don’t meet the ‘essential’ criteria are automatically excluded on the DOS portal.

Stage 2: Successful suppliers invited to submit written evidence against the published criteria. If more suppliers shortlisted than published maximum of 5 then written evidence will be evaluated for short listing.
22. Bidder Event - Q&A Session 18th August 2016
Q12 Can you explain the process and procedure going forward from here?
A12 Part 2

Stage 3: Successful suppliers submit a written proposal

Stage 4 : Supplier presentations including interview.Two parts to interview to testers: 1. generic user stories and how we’d test something with a particular test stack. 2. Look at some code.
23. Bidder Event - Q&A Session 18th August 2016
Q13 Are you expecting 3-5 people to do that at Stage 4?
A13 We’ve not yet decided what to do at Stage 4, we may implement a group element.
24. Bidder Event - Q&A Session 18th August 2016
Q14 Do we send 3 or 5 CVs?
A14 Send your best, up to five.
25. Bidder Event - Q&A Session 18th August 2016
Q15 Is evidence needed in line with CVs?
A15 Evidence sought will be relied upon to prove your candidates meet our requirements.
26. Bidder Event - Q&A Session 18th August 2016
Q16 Do you want suppliers to provide you with permanent staff they hold, or contractors?
A16 We want you to provide 3-5 testers that meet our requirements and we’re looking for a partner that can constantly supply high quality CVs.
27. Bidder Event - Q&A Session 18th August 2016
Q17 Test environment – would there be separate test environments with unrestricted access?
A17 Yes. We’re in a position to have copies of environments for specific needs such as OAT and NFT.
28. Bidder Event - Q&A Session 18th August 2016
Q18 Is there a Disaster Recovery environment?
A18 All of the environments are the same as production; any DR would be done in operational acceptance environment.
29. Bidder Event - Q&A Session 18th August 2016
Q19 Will the testers be assigned to a team, or act as a pool of testers?
A19 The testers will be moved into individual project teams, once a project finishes we will reallocate as needed if appropriate. There is a Programme Test Practice that works together to improve principles and procedures for the whole platform.
30. Bidder Event - Q&A Session 18th August 2016
Q20 What is the length of your statements of work?
A20 Three months initially, but then likely to increase to six months.
31. Bidder Event - Q&A Session 18th August 2016
Q21 What is expected on the 29th August?
A21 Initial two weeks for Yes/No responses to criteria published in requirements.
32. Bidder Event - Q&A Session 18th August 2016
Q22 What is the timeline for evidence?
A22 5 days to provide evidence.
33. Bidder Event - Q&A Session 18th August 2016
Q23 If we don’t answer if nice to have, do we get struck out?
A23 We’re asking for ticks against essential and nice to haves, if everyone meets essentials and some meet nice to have, then there will be a judgment in terms of who goes through. Nice to haves are important.
34. Bidder Event - Q&A Session 18th August 2016
Q24 We might have people who tick all the boxes situated outside of London, will they be considered?
A24 Only if they can commute to and work at Croydon 5 days a week.
35. Bidder Event - Q&A Session 18th August 2016
Q25 Evidence – so how would you like the evidence to be presented?
A25 DOS process to be used with stated maximum word limits. You want to demonstrate your organisation can sustain and support us throughout the relationship.
36. Would it be possible for some of the team to work from home one day a week? No
37. Please can you provide more information regarding the Rapid Software Testing course qualification that is a 'nice-to-have'? If your candidates have done it then that would be great.
38. Do you have any slides available from the Supplier session yesterday? No
39. I was turned away from the QA session. Will this harm my application? No attending the Q&A session was optional not mandatory.
40. Apologies for asking what may be a silly question, but can you please make clear what you are expecting to be provided by the 29th? Is it just the tick box response, or are you expecting anything in writing also, for example, cvs. Just the tick box response.