Department for Environment, Food and Rural Affairs (‘Defra’)

DEFRA UnITy Programme User Acceptance Testing (UAT) Support Service

8 Incomplete applications

5 SME, 3 large

25 Completed applications

12 SME, 13 large

Published: Friday 7 September 2018
Deadline for asking questions: Friday 14 September 2018 at 11:59pm GMT
Closing date for applications: Friday 21 September 2018 at 11:59pm GMT

Overview

Summary of the work: DEFRA is seeking a trusted partner to lead, shape and manage the successful delivery of UAT across the Defra Group and UnITy procurements.

Further Instructions on how to respond to this supplier opportunity are available if required. Please email if you would like to receive these instructions.
Latest start date: Monday 5 November 2018
Expected contract length: The Contract duration is 24 months
Location: London
Organisation the work is for: Department for Environment, Food and Rural Affairs (‘Defra’)
Budget range: Total approved framework value is £1.5M

About the work

Why the work is being done: The Defra Group makes significant use of ICT services in the delivery of its wider strategic objectives. The two largest ICT contracts supporting these services are due to expire in 2021. The Defra UnITy Programme (DUP) was commissioned with addressing how Defra might exploit this opportunity to generate savings and improve ICT services for its uses.
Problem to be solved: The UAT Support Service is intended to support the Authority and new suppliers of UnITy services to ensure continued access to, and availability of, the UnITy services during the period of implementation. The implementation phase will involve the cut over of services to new UnITy suppliers and service models. To mitigate the risk of disruption to the business, a UAT process is needed, which will build on the technical testing that will be performed by the new UnITy suppliers.
Who the users are and what they need to do: As a user of Defra ICT services I need to have continued access to Defra ICT services during the cut over to UnITy services so that day to day operations are not effected.
Early market engagement
Any work that’s already been done
Existing team: The initial and most significant users of the UAT Support Services will be the Hosting and Applications Support workstream and the End User Environment workstream. There is the expectation that the UAT Support Service could be further leveraged as required to support a broader range and scope of UnITy services.
Current phase: Not applicable

Work setup

Address where the work will take place: The majority of user workshops would occur in London. However, some users are located outside London, therefore travel to locations such as Reading, Newcastle, Bristol, Birmingham and York may be required.
Working arrangements: The requirement is for the Supplier to provide end to end management of the UAT process. The Authority holds limited testing collateral; the ‘stand-up scripts’ to be used during migration do not offer the level of functional coverage needed to fully mitigate the business risk. The execution of the UAT of each relevant application/service will be conducted by nominated user representatives, as they are the business experts and best placed to maximise value from test execution.
Security clearance: The Supplier’s Staff shall be Baseline Personnel Security Standard (BPSS) cleared (via Disclosure Scotland) and sign a confidentiality agreement prior to commencing delivery of the Services.

Additional information

Additional terms and conditions: The additional terms for this procurement:

a) An interpretation provision which reflects the fact that the Authority is contracting on behalf of other Defra bodies.
b) A term which has the effect that the parties acknowledge that the delivery of services will not use an agile methodology.
c) A term obliging the UAT Support Service supplier (“Supplier”) Staff to comply with requirements and policies of third parties when on their premises.
d) A standard change control clause instead of the current drafting.
e) A term obliging the Supplier to hold sufficient insurance to cover its liabilities under the call-off contract.

Skills and experience

Buyers will use the essential and nice-to-have skills and experience to help them evaluate suppliers’ technical competence.

Essential skills and experience: An experienced authoritative lead with demonstrable expertise

Ability to introduce and manage oversight governance, identifying potential critical business impacts, propose mitigations and appropriate escalation routes

Ability to act as a trusted advisor and communicate in written and verbal forms at executive and technical levels

Ability to deliver a service that commences on 30th September 2018 and delivers the Services in accordance with the dates outlined in the Buyer’s Requirements and/or the UnITy Implementation Plans

Ability to deliver the UAT phase with specific reference to the business risk profile similar to that described in this document

Significant experience and high-level skills in production of UAT artefacts

Previous and demonstrable record and relevant experience in managing end to end UAT programs at a scale similar to that described here.

Ability to respond flexibly to the development and changes in the implementation approach and timescales of the new UnITy services while maintaining continuity and consistency of service and quality.

All resources provided will need to be available to attend meetings in London; willing to sign a confidentiality undertaking; and be capable of BPSS security clearance (basic Disclosure Scotland).
Nice-to-have skills and experience: Ability to share supplier-site facilities/services eg test lab or office, within close geographic proximity to Buyer locations

Previous experience with existing Defra estate

Ability to provide accessible tooling platform

How suppliers will be evaluated

How many suppliers to evaluate

Proposal criteria

Comprehensive approach to the overall delivery of the contract, fully committing to the Buyer’s Requirements
How well the proposed UAT approach and methodology meets the Buyer’s Requirements
How well the proposal fits with the required timeframes for the Buyer’s Requirements
How complete are the identified risks and dependencies and suitability of the approaches to mitigate/manage them
Sufficient, appropriate resources allocated and structured in a workable manner
Value for money
Referee confirmation of satisfactory performance on previous contracts

Cultural fit criteria

Work as a team with our organisation and other suppliers
Take responsibility for work undertaken

Payment approach

Capped time and materials

Assessment methods

Written proposal

Evaluation weighting

Technical competence

65%

Cultural fit

Price

30%

Questions asked by suppliers

1. The opportunity states the latest start date to be 5 November 2018 but you also make reference to a service needing to start on the 30 September 2018. Can you please clarify?: The correct expected service commencement date is 5th November 2018
2. Can you please clarify start date, 5th November mentioned along with 30th September under Skills and Experience?: The correct expected service commencement date is 5th November 2018
3. Will our team be able to work remotely or be required on DEFRA Sites?: Work is to be conducted on-site for all expected interactions with users. Some back office elements elements may be suitable to be completed off-site, however, there is an expectation that UAT managers will be on-site.
4. Can you please supply the email mentioned to apply further instructions?: Suppliers should submit a contact email via a question requesting the further Instructions on how to respond to this supplier opportunity.
5. Can you please provide the e-mail address by which we have to request the additional information which you say is available: Suppliers should submit a contact email via a question requesting the further Instructions on how to respond to this supplier opportunity.
6. In response to your statement below please can you share more information to [email address removed] "Further Instructions on how to respond to this supplier opportunity are available if required. Please email if you would like to receive these instructions": Further information will be sent as requested
7. Would like to request for additional information on the following email id: [email address removed]: Further information will be sent as requested
8. Please could I request further information on how to apply for this work – email address is [email address removed]: Further information will be sent as requested
9. Please could I request further information on how to apply for this work – email address is [email address removed]: Further information will be sent as requested
10. Can you please send further instructions as mentioned to. [email address removed]: Further information will be sent as requested
11. Good afternoon, Can the above mentioned further information on how to apply for this project please be sent to [email address removed]: Further information will be sent as requested
12. Please provide further details to [email address removed]: Further information will be sent as requested
13. As requested can you please supply further information on this opportunity to : [email address removed]: Further information will be sent as requested
14. Please could you provide further information on this procurement to [email address removed]: Further information will be sent as requested
15. Please can you send further informaiton to [email addess removed] thank you.: Further information will be sent as requested
16. Please could you provide further instructions on how to respond to this supplier opportunity to the email address linked to this account?: Suppliers should submit a contact email via a question requesting the further Instructions on how to respond to this supplier opportunity.
17. Please can you send further information to [email address removed]: Further information will be sent as requested
18. Please could you send further information to the following email address: [email address removed]: Further information will be sent as requested
19. Please could you send further information to the following email address: [email address removed]: Further information will be sent as requested
20. Please could you send the further information to [email address removed]: Further information will be sent as requested
21. Can you please send further instructions on how to respond to this supplier opportunity to [email address removed]: Further information will be sent as requested
22. Please can you provide further details via my e-mail address: [email address removed]: Further information will be sent as requested
23. Please could I request further information on how to apply for this work – email address is [email address removed]: Further information will be sent as requested
24. Please can you provide further details via my e-mail address below [email address removed]: Further information will be sent as requested
25. Quality and Outcomes from Testing is often considered to be more honest and objective when conducted by independent resources. Are incumbents of Defra, RPA, EA, etc permitted to bid for this opportunity?: There are no restrictions
26. Can you please provide the further informtaion available on this opportunity to [email address removed]: Further information will be sent as requested
27. Please can you supply me with additional information as stated in the summary. Please send a copy to [email address removed]: Further information will be sent as requested
28. Please can you send further information to – [email address removed]: Further information will be sent as requested
29. In the RFP, the payment approach has been identified as “Capped time and materials”. Given the nature of the requirement, please can you confirm whether the service will be capped time and materials, or whether Defra has a capped budget for a time and materials service and will manage the work within this cap.: Each defined task/activity will be conducted on a capped time and materials basis.
30. Hello, Would like to request for additional information on the following email id [email address removed]: Further information will be sent as requested
31. Please may we also have the additional information referred to in the notice? Thank you: Please submit a contact email via a question requesting the further Instructions on how to respond to this supplier opportunity.
32. If successful at this stage – for the written submission, how would you like the 'team of resources' to be presented to you? Typically, we would present CV's or role profiles for each role type. Will that be the case here?: Instructions will be provided to shortlisted suppliers when appropriate
33. If successful at this stage – for the written submission, can we expect evidence and pricing template to be provided?: Instructions will be provided to shortlisted suppliers when appropriate
34. Can you please send further instructions on how to respond to this supplier opportunity to [email address removed]: Further information will be sent as requested
35. I just wanted to confirm that you are looking for these 100 word responses at this stage as your instructions could be interpreted as only wanting a YES/NO statement.: Suppliers are required to provide a 100-word response for each of the Essential and Nice to Have requirements in addition to a YES/NO response
36. In the instructions it implies that all is required at this stage for you to shortlist to the next stage is simply a yes/no answer against each of the essential and nice to have skills and there is no need for evidence unless too many suppliers pass the shortlist. Am I right in my conclusion?: Suppliers are required to provide a 100-word response for each of the Essential and Nice to Have requirements in addition to a YES/NO response
37. With regard to price, what format are you expecting to see (resource numbers and day rates, breakdown of non-resource costs, a model that shows how flex in demand would vary costs, etc.)?: Instructions will be provided to shortlisted suppliers when appropriate
38. How many Key Resource CVs need to be provided as part of the bidding process?: Instructions will be provided to shortlisted suppliers when appropriate
39. Who are the other UnITy suppliers? Will the UAT management supplier be expected to work directly with these suppliers?: The UAT Support Service supplier will be required to work with other UnITy suppliers as per the UnITy implementation plans. The planning for the implementation of the new UnITy suppliers is underway and relevant documents will be made available to the Supplier throughout the project lifecycle.
40. Is the UnITy programme delivering a SIAM / Service Tower type model?: The UnITy programme combined the scope of 2 SI’s and disaggregated them in to discrete Services creating a disaggregated, multi-supplier ICT capability, consisting of flexible and innovative partners aligned to Defra’s goals and principles.
41. Are there contractual commitments on the incumbent suppliers regarding supplying system information and knowledge transfer to the UAT management supplier for the UnITy programme?: The UnITy Programme will work with the UAT Support Service Supplier to determine the information required.
42. Will there be contractual commitments on the new UnITy suppliers requiring them to supply system information and knowledge transfer to the UAT management supplier for the UnITy programme?: The UnITy Programme will work with the UAT Support Service Supplier to determine the information required.
43. Will there be contractual commitments on the new UnITy suppliers requiring them to share test assets, such as scripts, with and make visible the test sets run and the evidence from test executions to the UnITy UAT management supplier?: The UnITy Programme will work with the UAT Support Service Supplier to determine the information required.
44. Are there contractual commitments on the incumbent suppliers requiring them to provide support to test activities, including provision of test environment, provision of test data, creation of user accounts and diagnosis of anomalies?: The UnITy Programme will work with the UAT Support Service Supplier to determine the information required.
45. Will there be contractual commitments on the new UnITy suppliers requiring them to provide support to test activities, including provision of test environment, provision of test data, creation of user accounts and diagnosis of anomalies?: The UnITy Programme will work with the UAT Support Service Supplier to determine the information required.
46. If successful, can we expect to have the UnITy suppliers test approach prior to the written submission stage?: Yes, you will receive details of the UnITy test approach prior to written submission stage
47. Would the buyer be open to alternative suggestions for tooling?: No
48. Will the buyer take responsibility for the appropriate amount of licenses needed to successfully deliver UAT? Are you able to share how many licenses might be available?: No, it should be a fully managed service
49. Could you please provide additional information regarding the scale of the change? e.g. • As-Is architecture vs. To-Be architecture • Approximate number of users affected • Number of departments affected: Tradtional 3-tier applications and further will remain the same.
Around 22000+ Users
Approximately 233 Applications
50. Could you please describe the nature of the user communities impacted by the programme i.e. key functions and roles.: Authority users, external Internet facing application and govt organisations
51. What tooling does DEFRA currently have in place to support testing, including UAT?: HP Quality Centre
52. What provision has DEFRA made for UAT test environments? Are there any constraints that would limit the scope, or types, of UAT tests that could be conducted?: Pre-production environment exist that support most applications being migrated.
53. Could DEFRA please advise as to the size of the user communities impacted i.e. number of staff, roles and functions in each location.: 150 Sites supporting 22000 Users
54. 1 How many DEFRA agencies are included as part of this programme, and does the programme scope include IT services to be provided to other agencies? For example RPA, Natural England, Environment Agency, etc.: Defra agencies included are: Rural Payments Agency(RPA), Marine Management Organisation (MMO), Core Defra, Environment Agency (EA), Natural England (NE) and Animal & Plant Health Agency (APHA)
55. How many applications are hosted as part of the new UnITy services?: There are 223 Business Applications of varying complexity that can be considered as part of this scope.
56. Are there any planned application changes/ enhancements as part of this change or is it just an upgrade and cut over to new infrastructure services?: The migration will involve a number of Platform version and OS upgrades. Application upgrades are not part of the Hosting suppliers requirement.
57. Are any services other than UAT Management required? Designing of User Acceptance tests and execution of tests?: UAT Management, Acceptance and Execution are all expected as part of the service
58. What tools are currently in place? Will the supplier be expected / able to use those tools, or are you expecting supplier to implement their own tools in order to deliver services?: Any supplier tools required to deliver the service are expected to be included as part of the service offer.
59. Will any supplier tooling costs (e.g. licences) be chargeable back to the buyer, or is this expected to be covered in the supplier’s charge for the service?: Any supplier tools required to deliver the service are expected to be included as part of the service offer.
60. What test environments are available? Will suppliers be able to influence environments provisioning (e.g. determine requirements for a suitable performance or operational test)?: There are a number of environments that the migration impacts and all will need to be migrated. The UAT is expected to support the migration of authority Business Application in all environments. The UAT suppliers proposal is expected to provide a view of its dependencies on any environments it believes is required to provide its service should any be required. The requester should define why they require a test environment for UAT services
61. Are these UAT services currently being provided by another supplier (or is this a new requirement)? If so, how many resources are currently used?: No existing supplier, this is a new service to support the migration of services within Defra.
62. How much demand are you expecting for these services over the next 24 months?: Over the next 24 months from Feb 19 around 223 known applications of different apps and complexities will all need to be UAT tested in line with the Hosting suppliers programme deliverables in order to reach their FMO milestone.
63. Will referee confirmation on key personnel be considered or just the supplier organisation?: Personnel are core to the service and the authority reserve the right to review when considered appropriate to distinguish appropriate selection of supplier
64. Can bidding suppliers use their own location (is there a preference for suppliers to offer that)?: The supplier can use the most appropriate location, as long as the supplier is able to run a ‘like for like’ test. This includes 3rd party connection for example MPLS, PSN and VPN connectivity.
65. How many people do you expect will be needed, at what grades?: This is a services so would expect the supplier to resource in accordance with the migration plans via some form of price x quantity model.
66. Will the winning bidder have access to Defra people to take part in UAT activities (including preparation)?: The supplier will have access to appropriate personal that would be critical in them delivering their service. This will need to be highlighted in a project plan so that resources/times can be approved and forecasted.
67. Could you please clarify what is meant by 'UAT management' as opposed to 'doing' UAT activities? I.e. what tasks do you expect the bidder to cover, and what tasks do you expect Defra and other suppliers to cover?: UAT Management, Acceptance and Execution are all expected as part of the service
68. The Brief mentions (a) "applications and services are due to be disaggregated to the new supply chain " + " migrate these services to the new UnITy suppliers" and (b) " implementation of the new UnITy services". Statement (a) could mean existing applications, source code etc., will be transferred to the new suppliers to run / maintain. Statement (b) could mean new applications will be implemented and data / workload / users will transition to the new applications. Is the plan to do (a) or (b) or a combination of (a) and (b)? If the latter, what proportion of each?: a)
69. If there are to be new implementations, rather than transfer of ownership, then could the replacement implementations provide additional capability / functions and/or support new business scenarios or are they intended to be identical, in this respect, to the ones they are superseding?: No, the supplier is supporting the migration of existing business and application services to new Unity suppliers services.
70. The Brief to Suppliers classifies applications as being "Gold", "Silver", "Bronze" or "Hosting Only". Is it correct to assume that "Gold" is the most business critical category?: That would be correct assumption.
71. The Brief to Suppliers classifies applications as being "Gold", "Silver", "Bronze" or "Hosting Only". What does the category "Hosting Only" denote and what is the difference between an application that is "Hosting Only" and one that has one of the three other categories?: For a Hosting only service, the hosting supplier is responsible for supporting up to and included the OS/ only. Any application installs/support is completed by a 3rd party.
Gold and Silver:
The Hosting supplier supports the OS/Hardware and application but with different RPO RTO for Silver and Gold
72. In section 4.5 of the Brief to Suppliers the results of an assessment of what is most likely to fail are given. What is the reasoning that has resulted as these items being the highest risk items? Is it because these are aspects that the UnITy supplier(s) test teams will be unable to test, or test effectively, or is it because these are areas where there are more likely to be defects or is it something else?: Assume these are the most likely areas where defects will be found
73. Where business functions cross application boundaries and these applications are not migrated at the same time, will it be necessary to coordinate the connection and alignment of test environments owned / operated by the incumbents with test environments owned / operated by UnITy suppliers, where end to end testing is required, and if not how is envisaged the environment coordination will work?: It is the responsibility of all suppliers to work with each other in delivery a solution to the authority.
74. If coordination of cross supplier environment connection and alignment is required then is that a responsibility of the UAT management supplier or is it dealt with elsewhere?: It should be highlighted and brought to the other suppliers so that the work is co-ordinated as a joint cross-supplier function.
75. Has a functionally representative end to end test environment, whether centralised or federated (supplier environments), been budgeted for? If so what assumptions have been made around the nature of this environment solution?: It is for the bidder to decide whether a centralised or federated test environment will be most appropriate. The authority has made no assumptions.
76. Is the UAT management supplier expected to (a) facilitate the participation of business resources into UAT or (b) coordinate their participation or (c) champion, negotiate and enforce their participation?: Introduction to application owners will be provided and the expectation is that supplier manages the relationship for all aspects of UAT
77. Who will provide knowledge transfer to the UAT management supplier personnel on existing business processes and system operation?: Introduction to application owners will be provided and the expectation is that supplier manages the relationship for all aspects of UAT
78. Who will provide knowledge transfer to the UAT management supplier personnel on existing business processes and system operation?: H&AS team can facilitate
79. Please ignore Question 78: Please ignore Answer to Question 78
80. Please ignore Question 77: Please ignore Answer to Question 77
81. Who will provide knowledge transfer to the UAT management supplier personnel on existing business processes and system operation?: H&AS team can facilitate
82. Will UAT management supplier personnel be provided with access to existing test / training environments to allow them to familiarise with the operation of existing systems and business processes?: Yes, access will be granted to the appropriate services/applications to the UAT testing provider.
83. How long before the planned start of UAT testing will all of the specification documents for an "Application/Service Implementation Bundle" be available?: Would expect first migration wave bundles to be available by DEC 18
84. How long before the planned start of UAT testing for an "Application/Service Implementation Bundle" will the application(s) be available to the UAT management supplier personnel in a test or training environment so that the supplier's personnel can familiarise with the application?: This would need further discussion as not all applications have DEV or TEST environments
85. How long before the planned start of UAT testing for an "Application/Service Implementation Bundle" will user documentation and training material be available so that (a) UAT management supplier personnel can assess it and (b) business personnel can be trained prior to UAT?: Some scripts are available and will be provided to supplier.
86. It is assumed quality assurance of the UnITY suppliers technical testing will be done by the Authority rather than the UAT supplier, is this a correct assumption?: It is expected that the UnITy suppliers will completed their own comprehensive tests and will include the Authority where appropriate. We would expect the UAT supplier to highlight any gaps in the other UnITy suppliers tests.
87. It is assumed that non-functional, and other, technical testing, will be the responsibility of the UnITy supplier and does not form part of UAT, is this a valid assumption?: Yes this is a valid assumption
88. Once an application / service has been subject to UAT and has gone into service then where does the testing of any subsequent change sit? If there are changes then, is progression testing (a) something done by the UnITY supplier, (b) something that involves the business but that is outside of this UAT management contract or (c) something that falls within this UAT management contract?: Applications may be migrated in a multitude of ways and as such thee may be extended UAT cycles for certain application. When an application is migrated in total then no additional change would be expected within this service provision. Where any additional requirements are made then these would be additional.
89. Once an application / service has been subject to UAT and has gone into service then where does the regression testing of any subsequent change sit? If there are changes then, is regression testing (a) something done by the UnITY supplier, (b) something that involves the business but that is outside of this UAT management contract or (c) something that falls within this UAT management contract?: It is expected that that once a service/application has gone into service, then any subsequent changes (minor/major) to the app/server and a cross party decision/agreement will need to be made as to whether testing is required , what type of testing and who will need to complete the testing.
90. Is the UAT management contract expected to deal with any post deployment testing by the business following deployment / cutover rehearsals in test environments?: No
91. Is the UAT management contract expected to deal with any post deployment testing by the business in production environments?: No
92. "The Brief to Suppliers mentions ""Application/Service Implementation Bundle"": (a) Ballpark, how many separate bundles will the programme execute? (b) Ballpark, how many separate bundles could be in-flight, in some stage of their lifecycle, at the same time? (c) Ballpark, how many separate bundles could be performing UAT at the same time?": a)     Should assume 50
b)    Should assume 10
c)     Should assume 10
93. Has the Authority estimated the number of UAT workshops that will be required, and if so what was the estimated number?: No, the Supplier should came back to the Authority with estimated number of days they require for workshops.
94. Will the business representatives nominated to perform the testing have prior experience of performing business testing of systems, and therefore understand the additional disciplines that are required when formally testing a system, or will they need training and mentoring on these things?: Assume that there will be a combination of the three. They may require training/mentoring or have adequate experience.
95. How many "stand-up" tests are there across the 200 applications in scope?: The Question was unclear and therefore the Authority is unable to answer it at this time.
96. In the outline plan with the Brief to Suppliers there is a line "Staff Transfer". What staff transfers are expected to occur as part of an application migration?: Hosting and EUE suppliers will try to TUPE appropriate staff for the services being transferred but there are no end user transfers.
97. How and from where is the HP Test Management tool able to be accessed?: From devices directly attached to the Defra network provided by the Authority
98. What version is the HPQC tool?: Current Version
99. You have asked for an accessible tooling platform to be provided, how do you envisage this working with the current test management tool?: This is to record information relating to testing and will be provided as a windows vmware server on the Defra network

Can we store analytics cookies on your device?

Awarded to Atos IT Services UK Ltd