DfT (on behalf of Passenger Demand Forecasting Council (PDFC)).

Discovery Phase – new Passenger Demand Forecasting System

15 Incomplete applications

12 SME, 3 large

16 Completed applications

9 SME, 7 large

Published: Tuesday 29 January 2019
Deadline for asking questions: Tuesday 5 February 2019 at 11:59pm GMT
Closing date for applications: Tuesday 12 February 2019 at 11:59pm GMT

Overview

Summary of the work: The Discovery will explore the user needs for the creation of a new model. The Discovery will identify the users, explore their needs and identify potential solutions to meet those needs to take forward into the Alpha stage.
Latest start date: Monday 18 March 2019
Expected contract length: Project Delivery within 6-8 weeks
Location: London
Organisation the work is for: DfT (on behalf of Passenger Demand Forecasting Council (PDFC)).
Budget range: Budget range: £75,000 - £150,000

About the work

Why the work is being done: The Scope of this project is for the Department for Transport (DfT) on behalf of Passenger Demand Forecasting Council to define its requirements for a passenger demand forecasting system. DfT is seeking to procure a system that will be: open and modular; with DfT on behalf of PDFC having clear and sole ownership of all IPR; compliant with state of the art IT infrastructure and computer language; be easily installed and user friendly.
Problem to be solved: • Need for a system to provide the rail industry demand forecasting requirements;
• Need for additional requirements with more innovative ways of achieving same or better results (language, IT architecture);
• System flexibility and ease of use (less rigidity, user interface, multiple users);
• Reduce limitations of the existing system (runtime, accessibility, network setup);
• No reliance on proprietary code (CMS) and proprietary database software (Oracle), leading to significant management cost savings.
Who the users are and what they need to do: • Train operators use existing demand forecasting models to decide what best timetable for their network would be, and to decide types of trains should be used for each service
• Most users predict demand and passenger revenue, especially after change of timetable, rail service operated, or rolling stock capacity
• Government uses these models to decide whether projects to build more rail infrastructure, or buy more trains would be value for money
• Bidders for rail franchise competitions use demand forecasting models to show the effects their proposed timetables may have, and DfT uses models to assess those bids
Early market engagement: Both Rail Delivery Group (RDG) and DfT have commissioned independent model and code reviews of the leading current demand forecasting model, known as MOIRA2.2. The language and functionality was independently reviewed. Additionally, work has been pulled together on user lessons learnt, particularly for use in franchising.
These studies are:
• Source code review by Andrew Tomlinson ITS Leeds - 2017
• Source code review for DfT - 2017
• MOIRA2.2 review by Leigh Fisher - 2017
• Lessons learned by Systra on CrossCountry franchise - 2018
Any work that’s already been done: Model users have developed a good understanding of how existing models work, and the use cases where an improvement of its functionality is crucial.
An overall architectural review of MOIRA2.2 and its dependencies has been mapped out by DfT.
The model is written in Delphi and needs to be upgraded to a language that make development and future iterations easier to manage.
Existing team: • RDG Strategic Research and DfT Rail Analysis – business leads
• RDG and DfT Procurement
• RDG and DfT IT and Digital Architects
• PDFC MOIRA2 working group – which cover stakeholder representatives from across industry
Current phase: Not started

Work setup

Address where the work will take place: The supplier will be required to split their time between the various users of demand forecasting models as well as allowing time for them to work at their own offices. It is anticipated that primary locations will be Rail Delivery Group offices EC1A 4HD and Department for Transport (Great Minster House SW1P 4DR and Windsor House SW1H OTL).
Working arrangements: The supplier would be expected to engage closely with the existing team, as well as external stakeholders on a frequent basis to fully meet the requirements. This is a full-time, 5 days a week arrangement for 6-8 weeks. There are no expenses allowances associated with this work.
Security clearance: As the team will need to access a central Government building, current basic security clearance (BPSS) for all team members is a mandatory requirement before contract award.

Additional information

Additional terms and conditions: Following Discovery, there will be a separate procurement exercise to progress the chosen Discovery option, looking to implement new system/s through Alpha and into Beta (subject to successful delivery of Alpha). There is potential that the successful Discovery supplier will remain on-board for a period to assist in providing client-side support and subject matter expertise through the transition into Alpha. This is subject to satisfactory completion of Discovery, a successful evaluation of the skill-set required and business needs.

Skills and experience

Buyers will use the essential and nice-to-have skills and experience to help them evaluate suppliers’ technical competence.

Essential skills and experience: Experience in delivering evidence-based research to inform a user-centred design, focus on user needs (including accessibility), end-to-end user journeys, motivations and goals.

Demonstrable evidence of incorporating latest technologies to enhance product specifications, resulting in increased functionality and improved user experience.

Experience of mathematical modelling and developing algorithmic models and their interfaces, and experience in transforming old models using new languages, technologies and approaches.

Experience in designing secure and scalable solutions in cloud-based environments with appropriate data management and security access protocols

Demonstrable evidence of excellent planning and communication, including robust risk management process and governance.

Recent and demonstrable experience writing architectural diagrams and presenting detailed architectural dependencies, successfully delivering technical architecture design meeting clients’ business, digital drivers and objectives, and to inform future project stages
Nice-to-have skills and experience: Demonstrable understanding and ability to deliver digital services/products to the Government Digital Service standards, working with users and stakeholders at all levels in large and complex organisations.

Demonstrable evidence of using clear development methodologies, outlining how this would work in conjunction with a client’s internal development team.

Experience at leading on a discovery phase for cross-sector across multiple partners, on a data-driven product including data-sharing protocol.

Demonstrable understanding of the policy and approach the government and the data community are taking with data and open data and of the benefits this provides.

Proven ability to support a flexible way of working, taking responsibility to make late changes to plans and re-formulating to maintain project progress against timeline.

How suppliers will be evaluated

How many suppliers to evaluate

Proposal criteria

Evidence that the proposed approach and methodology for discovery will successfully ascertain user needs and meet specified objectives for the discovery. (10%)
Proposed plan of activities with clear deliverables and dates demonstrating the ability to complete the delivery of the discovery project within the required challenging timescales. (10%)
Evidence of technical ability, skills and expertise to understand the complexity of the model and transform the solution into a secure, sustainable solution. (10%)
Identification of key options, assumptions, dependencies, risk and issues, with robust approaches to addressing them. (5%)
Evidence of how the proposal represents value for money for the buyer. (5%)
Demonstration of appropriate approach to working with in-house teams with limited technical and Agile expertise. (5%)
Proven knowledge and understanding of data security and access management in developing data solution, preferably in a cloud-based environment. (5%)
Proposed supplier team structure, with evidence to demonstrate a strong team and how their skills and experience will deliver the required outcome. (10%)

Cultural fit criteria

Experience of transparent and collaborative decision making and problem solving capability (2.5%)
Evidence where status quo has been challenged resulting in an improved final outcome. (2.5%)
Proposed workable and flexible approach to co-location of the team within the buyer’s premises. (2.5%)
Evidence demonstrating an ability to transfer knowledge & skills (2.5%)

Payment approach

Capped time and materials

Assessment methods

Written proposal
Case study
Presentation

Evaluation weighting

Technical competence

60%

Cultural fit

10%

Price

30%

Questions asked by suppliers

1. Are mentioned studies available? • Source code review by Andrew Tomlinson ITS Leeds – 2017 • Source code review for DfT – 2017 • MOIRA2.2 review by Leigh Fisher – 2017 • Lessons learned by Systra on CrossCountry franchise – 2018: “The documentation requested will be made available to the shortlisted suppliers, together with an introductory note which will describe the current modelling framework and provide an overview of the main user needs with associated documentation. The information is specific to the current Moira system and does not provide additional value at the first stage evaluation, where the submissions provided will demonstrate evidence against the essential and nice to have skills publicised”.
2. They asked: Can you issue, or direct us to the results of the early market engagement, particularly the outputs of: • Source code review by Andrew Tomlinson ITS Leeds – 2017 • Source code review for DfT – 2017 • MOIRA2.2 review by Leigh Fisher – 2017 • Lessons learned by Systra on CrossCountry franchise – 2018: “The documentation requested will be made available to the shortlisted suppliers, together with an introductory note which will describe the current modelling framework and provide an overview of the main user needs with associated documentation. The information is specific to the current Moira system and does not provide additional value at the first stage evaluation, where the submissions provided will demonstrate evidence against the essential and nice to have skills publicised”.
3. Clarification of selection timelines. What is the estimated date for publication of the shortlisted organisations? What is the estimated final date for the shortlisted organisations to submit the written proposal, case studies and presentation? What is the estimated date for publication of supplier selected? In addition please confirm that the shortlisted organisations are required to make their presentations in person at the customer's location.: Indicative procurement timelines are:
Publish Advert – 29.01.2019
Clarification Starts – 29.01.2019
Clarification Finishes – 23:59 on 05.02.2019
Deadline for 1st stage submissions – 23:59 on 12.02.2019
1st stage Evaluation - 13.02.2019 & 14.02.2019
1st stage Moderation – 15.02.2019 – 15/02/19, 10:00-13:00
Go back out to shortlisted suppliers for their written proposals – 18.02.2019
Clarification Starts – 18.02.2019
Clarification Finishes – 26.02.2019
Deadline for 2nd stage submissions – Midday on 01.03.2019
2nd stage Evaluation – 04.03.2019 & 05.03.2019
2nd stage Moderation – 06.03.2019
Supplier presentations @ customer venue TBC – 07.03.2019 & 08.03.2019
4. The brief states that DfT is seeking to "procure a system". Does this mean you are looking for an off the shelf system/model or is a custom-build solution desired/acceptable?: As per the DOS issued, RDG and DfT are looking for a new solution that, whilst continuing to deliver compliant to the demand forecasting requirements, will meet user needs better than the current model does. This new solution is expected to work in more innovative ways, besides being more flexible and easy to use. This evaluation could look at both bespoke and off the shelf technologies or applications.
5. In order to answer yes for the first nice to have "Demonstrable understanding and ability to deliver digital services/products to the Government Digital Service standards, working with users and stakeholders at all levels in large and complex organisations." must we evidence passing stages of the digital services standard or is it sufficient to show comparable experience? https://www.gov.uk/service-manual/service-standard: “Government Digital Service standards offers best practice in a mobile-first, user-led digital design using agile methodology including co-design with the business and a micro/mini service approach to architecture. As this criteria asks for “Demonstrable understanding and ability to deliver digital services/products to the Government Digital Service standards”, a comparable standard is sufficient but the supplier would need to demonstrate in the answer why this standard is an equivalent and it would be preferable to reference the key GDS principles and how you have applied similar principles.”
6. For avoidance of doubt does CMS refer to Content Management System, in the phrase "no reliance on proprietary code" (CMS).: CMS is the brand name of the proprietary software then adopted by the current model developer.
7. For avoidance of doubt, does the statement "no reliance on proprietary" code also include no reliance on Open Source?: The evaluation should not exclude open source solutions which offer sufficient flexibility and breadth of support to ensure they do not constrain future development or use of this service. We are looking for a solution where intellectual property and code will be owned and controlled by the partners/DfT.
8. Whilst we recognise this is a Discovery, can you please provide any indication as to what the scale of the model and the time frames that are needed for forecasting? i.e. A granular single line model will be different from one that forecasts the whole system and the same with the time domain: The aim is to produce a new national (GB) passenger demand and revenue forecasting system that is a functional replacement of the MOIRA suite of models. Discovery will enable the project team understanding which solutions are at hand.
9. Whilst we recognise this is a Discovery, is scenario forecasting likely to be in scope? i.e. headline event scenarios that need forecasting i.e. a flood, a music festival?: The aim is to produce a functional replacement of the MOIRA suite of models. Discovery will explore user needs and support the decision on whether any additional functionality shall be added during the subsequent model development. The new model must anyway be scalable and easy to further develop.
10. Is there total transparency of the existing model across all routes between all franchise operators?: No. Franchise operators don't have transparency of each other's data.
11. How is new data ingested into the current model? i.e. are there live data feeds, or has this been populated on a more manual basis over time?: The current model is manually updated.
12. Is it likely that the new system will need to maintain assurance of output with the old system i.e. because franchise contracts reference specific outputs from the existing model?: No
13. Does a test harness or test data set already exist in the as-is MOIRA 2.2 world?: No
14. Does the functionality of MOIRA 2.2 need to be upgraded beyond additional functionality that is crucial? i.e. If this principally a refresh of the programming language required? Alternatively, can you please provide an indicative view of what extra functionality users might need?: The aim is to produce a functional replacement of the MOIRA suite of models. Discovery will explore user needs and support the decision on whether any additional functionality shall be added during the subsequent model development. The new model must anyway be scalable and easy to further develop.
15. Whilst "use cases where an improvement of its functionality is crucial" are understood, does scope of the requirement also encompass exploring user need that might lead to unexpected or novel improvements to the model?: The aim is to produce a functional replacement of the MOIRA suite of models. Discovery will explore user needs and support the decision on whether any additional functionality shall be added during the subsequent model development. The new model must anyway be scalable and easy to further develop.
16. Are the modelling domain and features included in the model well defined and documented?: There is in the DOS a list of the model documentation currently available.
17. Does DfT have either a clear view or indicative view of the roles it requires a supplier to provide during Discovery?: RDG and DfT have a view on what roles we would expect a prospective bidder to provide based on the description of what we want the discovery phase to cover. It would be inappropriate for us to specify what specific roles they should be as each bidder may have a different approach but we would expect roles to cover project management (agile), user research and technical architecture as well as providing subject matter expertise in the area of modelling and data management and analysis.
18. What roles will DfT provide during Discovery?: RDG and DfT will make a number of staff available to the discovery team including project managers, rail modellers, rail analysts, end user groups, technical architects and business partners.
19. Does Capped Time and Materials refer to the DOS definition, where suppliers must complete all work at its own cost?: A capped time and materials approach is the same as a time and materials approach but there’s a limit on how much we have to pay for the work. If a supplier reaches the limit before the work is finished, the supplier has to complete the work at their own cost. If the supplier finishes the work before they said they would, we only pay them for the time they took to do the work.
20. Is it possible for suppliers to propose time and materials within a maximum total budget?: A capped time and materials approach is the same as a time and materials approach but there’s a limit on how much we have to pay for the work. If a supplier reaches the limit before the work is finished, the supplier has to complete the work at their own cost. If the supplier finishes the work before they said they would, we only pay them for the time they took to do the work.
21. Does capped Time and Materials, mean time and materials within a maximum total?: A capped time and materials approach is the same as a time and materials approach but there’s a limit on how much we have to pay for the work. If a supplier reaches the limit before the work is finished, the supplier has to complete the work at their own cost. If the supplier finishes the work before they said they would, we only pay them for the time they took to do the work.
22. Will scoring from the evidence stage be taken through to final evaluation scoring? As opposed to being used solely for the purpose of shortlisting?: There are 2 distinct stages with a DOS procurement. 1st stage scores are used to shortlist interested parties with the subsequent written proposal/presentation stage essentially a standalone step. No scores are carried over.
23. What criteria will DfT be using to judge whether an evidence answer 'exceeded' requirements and therefore scores full marks (3)?: DfT will assess the robustness of the evidence provided against the Essential and Nice to have skills outlined in our published advert, with scores allocated accordingly.

Can we store analytics cookies on your device?

Awarded to BJSS Limited