This opportunity is closed for applications

The deadline was Tuesday 23 June 2020
Money and Pensions Service

digital, accessibility, disability testing, statements, device testing, training, audit

1 Incomplete application

1 SME, 0 large

11 Completed applications

9 SME, 2 large

Important dates

Published
Tuesday 9 June 2020
Deadline for asking questions
Tuesday 16 June 2020 at 11:59pm GMT
Closing date for applications
Tuesday 23 June 2020 at 11:59pm GMT

Overview

Summary of the work
Full accessibility audits of 3 live websites and 1 website currently under construction all within the Public Sector/ Government bodies. Accessibility statements for 4 websites, Accessibility testing for online tools and calculators. Reports and advisory fixes for accessibility issues.
Location
London
Research dates
August 2020 to July 2021
Organisation the work is for
Money and Pensions Service
Budget range
• User testing
• Audit/certification (including tech audit)
• Desktop, Mobile/tablet device testing
• Apps – Native and Hybrid
• Produce detailed accessibility reports
• Provide accessibility statements

Full scope of all the requirement. Budget range £40,000 to £60,000.

About the work

Early market engagement
Not Applicaple

About the research

Description of your participants
The Supplier will need to test for a wide demographic of people with varied disabilities, including blindness, vision impairment or physical disabilities that affect the way they use a computer
Assisted digital and accessibility requirements
screen reader, magnifier, mouth stick or head wand
Research plan
MaPS will need to work with the Supplier to define the research plan.
Research location
MaPS will require that testing is carried out on the Supplier's (agency's) site. Hence the Supplier is to ensure they have all necessary facilities (Hardware, Software and people ) to carry out the testing
Access restrictions at location
Number of research rounds
4 times on each of the 4 websites
Number of participants per round
the maximum participants per day will be 11 participants in total made up of:
1 Consultant,
5 to 10 researchers per day.

Maps will require that there is at least 1 consultant every day, hence the Supplier is to ensure that they have a spare participant (Consultant) in case of any cancellations
How often research will happen
There are 4 sites in total to be tested, ongoing testing for the 3 live sites is to be carried out every fortnight and once a month form the new (4th)site.

Supplier will need to work alongside MaPS in further developing the schedule for testing.
Evening or weekend research

Additional information

Additional terms and conditions

Skills and experience

Buyers will use the essential and nice-to-have skills and experience to help them evaluate suppliers’ technical competence.

Essential skills and experience
  • Previous experience of working for a regulated environment
  • Able to demostrate that they have a range of accessibility tools and participants for testing
  • Ability to work remotely
  • Flexible in availability - Participants available at short notice
Nice-to-have skills and experience
  • Previous experience of testing people with physical disabilities
  • Previous experience of testing people with mental disabilities

How suppliers will be evaluated

All suppliers will be asked to provide a written proposal.

How many suppliers to evaluate
5
Proposal criteria
  • Approach to recruiting specialist participants
  • Estimated timeframes for the work
  • Risks Indentified and mitigating actions
Additional assessment methods
Reference
Evaluation weighting

Technical competence

60%

Availability

20%

Price

20%

Questions asked by suppliers

1. Do you require expert accessibility audits, testing with users with disabilities, or both? NB we would always recommend undertaking an audit, fixing the issues identified, and only then undertake testing people with disabilities
We are looking for guidance from the appointed agency on this. I agree that plan makes sense and it will only be the newly branded website that will require full testing with people with disabilities I think as we only need to make sure that the existing websites have accurate accessibility statements. They will be live until early 2021, once everything has been migrated from them into the newly branded website.
2. Do you require expert accessibility audits, testing with users with disabilities, or both? NB we would always recommend undertaking an audit, fixing the issues identified, and only then undertake testing people with disabilities
We are looking for guidance from the appointed agency on this. I agree that plan makes sense and it will only be the newly branded website that will require full testing with people with disabilities I think as we only need to make sure that the existing websites have accurate accessibility statements. They will be live until early 2021, once everything has been migrated from them into the newly branded website.
3. Do you require expert accessibility audits, testing with users with disabilities, or both? NB we would always recommend undertaking an audit, fixing the issues identified, and only then undertake testing people with disabilities
We are looking for guidance from the appointed agency on this. I agree that plan makes sense and it will only be the newly branded website that will require full testing with people with disabilities I think as we only need to make sure that the existing websites have accurate accessibility statements. They will be live until early 2021, once everything has been migrated from them into the newly branded website.
4. Please can you clarify what you mean by consultant and researchers under “number of participants per round“? Are you referring to research participants and accessibility experts?
By consultant we mean a representative/contact at the agency We would expect the consultant to produce and provide accessibility report and assist with guidance on advisory fixes and improvements. The Consultant will be the accessibility experts while the research participants would be the individuals involved in the testing itself.
5. Research dates are stated to be August 2020 to July 2021 – 12 months. Under ‘number of research rounds’ you state four times on each of the four websites – i.e. 16 rounds of testing. However, under ‘How often the research will happen‘ you state that ongoing testing for the three live sites is to be carried out every fortnight and once a month for the new site. That suggests that in the August 2020 to July 2021 period there will be in total 84 rounds of testing. Please could you clarify how much testing is actually envisaged?
We would be seeking guidance from the appointed agency as to how the testing would actually work and how many rounds there would be.
We expect a full test of the three existing websites in order to produce accurate accessibility statements.
We will expect more in-depth tests of the tools/calculators in the estate (around 40).
We will expect ongoing testing of the newly branded website as we build and fix errors based on the testing and results.
6. Research dates are stated to be August 2020 to July 2021 – 12 months. Under ‘number of research rounds’ you state four times on each of the four websites – i.e. 16 rounds of testing. However, under ‘How often the research will happen‘ you state that ongoing testing for the three live sites is to be carried out every fortnight and once a month for the new site. That suggests that in the August 2020 to July 2021 period there will be in total 84 rounds of testing. Please could you clarify how much testing is actually envisaged?
We would be seeking guidance from the appointed agency as to how the testing would actually work and how many rounds there would be.
We expect a full test of the three existing websites in order to produce accurate accessibility statements.
We will expect more in-depth tests of the tools/calculators in the estate (around 40).
We will expect ongoing testing of the newly branded website as we build and fix errors based on the testing and results.
7. Research dates are stated to be August 2020 to July 2021 – 12 months. Under ‘number of research rounds’ you state four times on each of the four websites – i.e. 16 rounds of testing. However, under ‘How often the research will happen‘ you state that ongoing testing for the three live sites is to be carried out every fortnight and once a month for the new site. That suggests that in the August 2020 to July 2021 period there will be in total 84 rounds of testing. Please could you clarify how much testing is actually envisaged?
We would be seeking guidance from the appointed agency as to how the testing would actually work and how many rounds there would be.
We expect a full test of the three existing websites in order to produce accurate accessibility statements.
We will expect more in-depth tests of the tools/calculators in the estate (around 40).
We will expect ongoing testing of the newly branded website as we build and fix errors based on the testing and results.
8. Please confirm your interpretation of a 'regulated environment'
A regulated environment is basically any controlled environment. Rules state which conditions must be met by a company to produce valid results or goods of a guaranteed level of quality.
9. You mention participants being flexible in availability – please clarify your expectations around 'short notice' availability
We would expect to build a good working relationship with regular ongoing communication and a fast response time from the appointed agency/consultant(s)
10. Is the stated budget range inclusive or exclusive of VAT?
The stated budget is inclusive of VAT
11. What do you define as ‘mental disabilities’?
Cognitive, learning, neurological disabilities involve neurological disorders, as well as behavioural and mental health disorders. They impact how well people process and comprehend information.

Attention deficit hyperactivity disorder.
Autism spectrum disorder.
Mental health disabilities - Including anxiety, delirium, schizophrenia, mood disorders. Relating to web accessibility may cause difficulty in focusing, processing, understanding information. The medications used frequently have side effects including blurred vision, hand tremors, and memory impairments.
Perceptual disabilities - difficulty processing sensory information like auditory, tactile, visual. Including dyslexia, writing or dysgraphia, mathematic concepts or dyscalculia.
Seizure disorders - epilepsy and migraines, in reaction to visual/audio stimulation.
12. Question on “number of participants per round“: please can you clarify what you mean by consultant – the wording suggests this means a user research participant – is that right (Because you require a spare participant – consultant in case of any cancellations)? And who are the 5 to 10 researchers per day?
By consultant we mean a representative/contact at the agency We would expect the consultant to produce and provide accessibility report and assist with guidance on advisory fixes and improvements. The Consultant will be the accessibility experts while the research participants would be the individuals involved in the testing itself

MaPS will require that there is a representative available as defined above and that they are able to manage the testers to avoid any delays.

This was an estimated amount of testers. We expect the agency to advise us as to how their testing strategy would work.
13. Please clarify your definition of mental disabilities in the context of this requirement?
Cognitive, learning, neurological disabilities involve neurological disorders, as well as behavioural and mental health disorders. They impact how well people process and comprehend information.

Attention deficit hyperactivity disorder.
Autism spectrum disorder.
Mental health disabilities - Including anxiety, delirium, schizophrenia, mood disorders. Relating to web accessibility may cause difficulty in focusing, processing, understanding information. The medications used frequently have side effects including blurred vision, hand tremors, and memory impairments.
Perceptual disabilities - difficulty processing sensory information like auditory, tactile, visual. Including dyslexia, writing or dysgraphia, mathematic concepts or dyscalculia.
Seizure disorders - epilepsy and migraines, in reaction to visual/audio stimulation.
14. If any of the rounds of testing are to be conducted during the COVID lockdown, is remote user testing acceptable rather than lab-based testing? The remote sessions would be live streamed and recorded, but of course local observation would not be possible.
Yes, remote user testing is acceptable.
15. Can you provide URLs for the 3 live websites? If not, can you indicate the number of pages that would need to be tested on each?
https://www.moneyadviceservice.org.uk/en
https://www.pensionsadvisoryservice.org.uk/
https://www.pensionwise.gov.uk/en

And we are rebuilding and rebranding this website https://moneyandpensionsservice.org.uk/
16. The "Summary of the work" section says there are 4 websites to test, so why does the "Budget range" section refer to native and hybrid apps?
By apps we are referring to our online tools/calculators.
17. Can you define your requirements for "Desktop, Mobile/tablet device testing" more fully? Can these variants be covered during the user testing or do you expect this testing to be done separately (which the budget is not sufficient for).
We do our own user testing with different devices but we would expect the accessiblity testing to include different devices because different devices provide different way in which to access content. i.e. keyboard on laptop and touch screen on tablet/mobile.
18. The opportunity is currently listed under the user research participant lot and appears to be more aligned with the digital outcomes lot. Please can you advise if this is listed under the correct lot and only for the provision of user research participants?
We considered all options prior to using this lot and the research participant lot was selected as the most suitable framework on this occassion.

We conduct separate user testing. This piece of work involves accessiblity testing and the individuals necessary to carry out that work. The Consultant will be the accessibility experts while the research participants would be the individuals involved in the testing itself.
19. Can you confirm that the user testing participants (except the Consultants) should be members of the public, not "professional users" employed by the testing organisation?
We require the tests to be carried out by users with varied learning and physical disabilities.
20. If you want a fixed price proposal, can you be more specific about the number of participants per day? The cost of recruitment and incentives would be significantly higher for 10 participants than for 5.
We would be seeking guidance from the appointed agency as to how the testing would actually work and how many rounds there would be.
We expect a full test of the three existing websites in order to produce accurate accessibility statements.
We will expect more in-depth tests of the tools/calculators in the estate (around 40).
We will expect ongoing testing of the newly branded website as we build and fix errors based on the testing and results.
21. The references to "Public Sector/ Government bodies" and "Audit/certification (including tech audit)" imply that you require a WCAG audit. When will this be conducted relative to the user testing? The budget is not sufficient to conduct 4 rounds of WCAG audits and user testing on 4 websites, and the user testing should not start until the websites are WCAG conformant.
Our user testing is a separate piece of work. For this we are only referring to accessiblity testing. We would be seeking guidance from the appointed agency as to how the testing would actually work and how many rounds there would be.
We expect a full test of the three existing websites in order to produce accurate accessibility statements.
We will expect more in-depth tests of the tools/calculators in the estate (around 40).
We will expect ongoing testing of the newly branded website as we build and fix errors based on the testing and results.