Awarded to Web Usability

Start date: Thursday 2 November 2017
Value: £20,000
Company size: SME
Department for Transport

National Travel Survey (NTS) Production automation and user query service – discovery phase.

3 Incomplete applications

2 SME, 1 large

10 Completed applications

8 SME, 2 large

Important dates

Published
Thursday 31 August 2017
Deadline for asking questions
Thursday 7 September 2017 at 11:59pm GMT
Closing date for applications
Thursday 14 September 2017 at 11:59pm GMT

Overview

Summary of the work
• Undertake discovery of users and user needs in line with GDS Service Design Manual
• Answer the research questions (problem to solve)
• Provide recommendations for approach for delivery of Alpha and Beta
• Presentation and written report clearly outlining findings and recommendations with knowledge transfer to NTS team.
Latest start date
Monday 2 October 2017
Expected contract length
4 weeks
Location
London
Organisation the work is for
Department for Transport
Budget range
Budget of £15,000 - £20,000

About the work

Why the work is being done
The NTS is a large complex dataset that is analysed by DfT staff and results published in the form of tables, and raw data. This labour intensive activity is believed be inefficient.
A high volume of queries from stakeholders are received often looking for a subset of data. Providing responses prevents work on more complex analysis that would unlock value for DfT.
There is a good understanding of the questions asked but not the motives behind them and/or what the data is being used for. This understanding needs to be developed to ensure the product is fit for purpose.
Problem to be solved
The discovery phase focuses on three questions:

1. How is it possible to make the process more efficient and reduce the time taken for the team to analyse the dataset and produce outputs for publication?
2. What mechanism for internal/external stakeholders can be provided to directly query the dataset and export data items (supports the Open Data aspiration)?
3. How can technology innovation augment the human capabilities to carry out work that will unlock the perceived value of the data to the department?
In addition understanding how the concepts could be applied to other datasets should be considered.
Who the users are and what they need to do
The purpose of this discovery is to understand both the users and the user needs in line with the GDS service manual: https://www.gov.uk/service-manual/user-research

Historically the users have been:
o The NTS Team
o DfT Internal stakeholders
o Non-DfT (participants to be identified from existing contacts to the NTS team):
 External GDs and Agencies
 Academics
 Researchers
 Students
 Transport Consultants
 Research companies
 Local Authorities
 Charities
 Members of the public
 Transport Companies (e.g. bus companies)
 Significant Departmental Stakeholders such as Sustrans, Living Streets
Early market engagement
A previous piece of work that had been commissioned by the department to explore how the challenges of compiling and analysing the dataset could be addressed highlighted the complexity of the dataset but didn’t lead to any suitable options of satisfactory quality.
A segment of the NTS data was used for a recent DfT Hackathon and one of the participants used this to create a prototype solution. This was the winning idea at the Hackathon and with further discussion has shown that it might be possible to address the challenges the team faces.
Any work that’s already been done
The prototype ‘Query’ created at the DfT Hacks hackathon: http://clients.onesixfour.digital/query/index.html
Existing team
The appointed agency will be expected to work closely with members of the NTS team as well as the DfT Data Science community. The NTS team consists of 4 statisticians and the data science team consists of 4-5 data scientists / developers. The Product Owner and Service Manager will be appointed by the NTS team.
Current phase
Discovery

Work setup

Address where the work will take place
The majority of the work will be conducted at the suppliers own site (or other chosen location). Exact days to be worked at the department offices will be agreed once the supplier is appointed.

Address: Department for Transport, Great Minster House, Horseferry Road, SW1P 4DR
Working arrangements
Discovery should be completed in line with the GDS Service Manual: https://www.gov.uk/service-manual.

Indicated milestones are:
Week 1: Research
Week 2-3: Analysis (user personas and user needs defined)
Week 4: Results and recommendations

Suppliers to provide preferred approach. The exact milestones will be agreed with the supplier when appointed.

Weekly status updates (email/telephone) detailing achievements of key milestones and issues that may impact timescales and cost should be provided.

It is expected that there will be a short presentation or report at the end of each milestone. The research questions should be fully answered in a final presentation
Security clearance
The Department will require current baseline security clearances in order to work on our premises. The details of these will be worked out with the successful supplier with proof of status being supplied by shortlisted suppliers at written proposal stage (note these will not form part of any word count).

Additional information

Additional terms and conditions
N/A

Skills and experience

Buyers will use the essential and nice-to-have skills and experience to help them evaluate suppliers’ technical competence.

Essential skills and experience
  • • Proven track record of delivering discovery phase of agile projects as speed with diverse stakeholder groups.
  • • Evidence of the ability to work with complex hierarchical data and understanding user needs associated with this.
  • • Experience of delivering in line with the GDS Service Design Manual
  • • Evidence of being able to carry out user research with similar stakeholder groups to those identified.
  • • Knowledge of advanced data analysis, automation and query technologies and their application to similar datasets
  • • Demonstrable experience of using innovative techniques to speed up identification of user persons and user needs
  • • Credibility with similar stakeholder organisations
  • • Experience of working on similar data related projects that have successfully delivered outcomes articulated
Nice-to-have skills and experience
  • • Experience of data science and data visualisation
  • • Evidence of having undertaken discovery work that has led to significant transformation of data analysis activities

How suppliers will be evaluated

How many suppliers to evaluate
5
Proposal criteria
  • • Plan & Approach (10%)
  • • Application and outcome of methodologies (10%)
  • • Identification of risks and dependencies and offered approaches to manage them (10%)
  • • Incorporating new technologies/innovative thinking to development (10%)
  • • Capability to deliver (10%)
Cultural fit criteria
  • • Experience of working as part of an agile team with organisation similar to ours or who have similar audiences (2.5%)
  • • Experience of sharing approach, knowledge and skills with internal team members and listening to their feedback (2.5%)
  • • Experience of transparent and collaborative decision making (2.5%)
  • • Experience of taking responsibility for their work and resolving problems effectively (2.5%)
Payment approach
Capped time and materials
Assessment methods
  • Written proposal
  • Case study
  • Presentation
Evaluation weighting

Technical competence

50%

Cultural fit

10%

Price

40%

Questions asked by suppliers

1. Question 1 . In essence is the successful agency going to be doing the research element of this project if so would it be wise to accept that one person from our agency could potentially do all of the work stated?
Response

The successful bidders will be carrying out the research elements of the project, as defined in the specification. Bidders are free to quote for as few or as many people as they feel are required, as long as they demonstrate they can undertake the work to the required specification and timescale shown.
2. One of your requirements in the summary of work are to "Provide recommendations for approach for delivery of Alpha and Beta" Will the successful bidder be excluded from bidding for Alpha/Beta? Do you intend to outsource the Alpha/Beta work? Do you have any timescales in mind for beginning Alpha/Beta?
Successful bidders for this Discovery phase will not be excluded from any subsequent Alpha or Beta projects. Whether the Department proceeds with an Alpha and Beta will depend on the outcome of the Discovery phase and so timescales have not been confirmed yet. However, if the Department proceeds to Alpha and Beta , then the projects would be outsourced through a further procurement exercise.
3. When dealing with external users, we would recommend longer than 4 weeks for this research as the recruitment and booking for users usually takes 1-2 weeks. Would you be open to this?
We would expect the work to be completed in the 4 week period. However, details of the final timetable will be discussed with the winning supplier on award of the contract.
4. Do you have easy access to your users? Due to you tight 4 week timescales, it'll be hard to book and arrange face-to-face research activities with many of these users. We have worked with many of your listed users groups before and many of them have very busy working arrangements.
We have extensive lists of users that we can make available to the winning supplier. Suppliers are expected to provide innovative ways of researching large numbers of users.
5. Your budget seems small for a full user research discovery. As many of your users are external to the organisation, I would recommend paying travel and incentives to face-to-face activities. Would this need to be covered with this budget or can this be paid from another budget?
Any travel costs and incentives should be included in the supplier’s estimate of their overall costs in the bid. We will expect suppliers to provide innovative and flexible ways of researching large numbers of users in a cost-efficient way that may in include face-to-face calls or other methods.
6. 'Working arrangements' states that week 1 should be 'research'. However, in order to undertake the research it will be necessary to agree the number and types of users to be involved in the research, to recruit suitable respondents, to schedule these respondents, and then to undertake this research. One week to achieve all of this seems unrealistic. Are you open to alternative timescales?
We would expect the work to be completed in the 4 week period. However, details of the final timetable will be discussed with the winning supplier on award of the contract.
7. Do you have an expectation of how many respondents will be involved in this research? We note you say that some non-DfT participants can be identified from existing contacts to the NTS team – will this cover all respondent types and will they require incentives? If not, will you require the agency to recruit required respondents independently? Are respondent incentives to be included in the overall budget for this project?
There is no expectation of the number of respondents involved in the research, but we would expect a number that allows reasonable conclusions to be drawn to answer the 3 questions. We have extensive lists of users that will be available to the winning supplier. These include both internal/external users. NTS team themselves are a customer for 1 aspect of the research.
Any travel costs/incentives should be included by the supplier in their estimate of overall costs for the contract. We will expect suppliers to provide innovative ways of researching large numbers of users in a cost-efficient way.
8. When do you anticipate the presentations to take place?
The presentations would be after the second evaluation. They are currently not mandatory and will be undertaken if and when required.
9. Will you be asking all 5 shortlisted suppliers to present, or just the higher scorers? Please could you clarify this process?
The presentations would be after the second evaluation. They are currently not mandatory and will be undertaken if and when required.
10. Why the work is being done' states 'There is a good understanding of the questions asked but not the motives behind them and/or what the data is being used for. This understanding needs to be developed...'. 'Who the users are...' states 'The purpose of this discovery is to understand both the users and the user needs...'. However 'problem to be solved' talks about solutions required without the context of user needs. Please could you clarify the discovery phase objectives and desired outcomes.
The desired outcome is to answer the three research questions listed in the specification within the context of the user need. For example:

• What might be changed about the IT infrastructure used by the NTS team to answer research question 1.
• Researching whether a query tool is required by users for question 2, what functionality it might need and what mechanisms might be used to support that.
• For question 3, how technologies can support users in unlocking the value of the data.
11. You talk about the prototype solution from the Hackathon. However at this point users have not been engaged nor user needs established. Please could you explain how you have come to the conclusion that the results from the Hackathon that may address any challenges? As part of this discovery do you want the supplier to review the prototype against the established user needs or will you do that once the discovery is complete
An interactive prototype was developed at our hackathon that showcased the potential for this kind of tool. Whilst we have performed some preliminary investigation of the software and technology required to make this a reality, we have not yet developed or tendered for a technical solution. The prototype is the IP of the team who attended the hackathon event, and as such we are not in a position to share it. We will not expect the supplier to the review this prototype against user needs.
12. It’s mentioned that a hackathon solution was developed and may be a possible solution. Only the data query has been provided. Can you provide details of what the solution was that used this query and deemed a possible solution please?
An interactive prototype was developed at our recent hackathon event that showcased the potential for this kind of tool. Whilst we have performed some preliminary investigation of the software and technology required to make this a reality, we have not yet developed or tendered for a technical solution. The prototype is the IP of the team who attended the hackathon event, and as such we are not in a position to share it.