Highways England

ITPD0312 IAMIS Structures Data Migration Assurance PRO.1058

Incomplete applications

11
Incomplete applications
7 SME, 4 large

Completed applications

10
Completed applications
5 SME, 5 large
Important dates
Opportunity attribute name Opportunity attribute value
Published Friday 5 July 2019
Deadline for asking questions Friday 12 July 2019 at 11:59pm GMT
Closing date for applications Friday 19 July 2019 at 11:59pm GMT

Overview

Overview
Opportunity attribute name Opportunity attribute value
Summary of the work IAMIS civil structures asset data migration assurance
Latest start date Monday 5 August 2019
Expected contract length 6 months - expected to complete in January 2020.
Location West Midlands
Organisation the work is for Highways England
Budget range £200,000 (two hundred thousand pounds)

About the work

About the work
Opportunity attribute name Opportunity attribute value
Why the work is being done Highways England is implementing a new asset management system for its civil structures assets to replace the current system which is at end of life. The new system will hold structures asset data that is needed to comply with licence condition 5.9. This work is required to assure that integrity of the structures asset data is maintained during the migration and up until go-live of the new system.
The work is required to be completed to allow the new system to go live on 28th October 2019.
Problem to be solved We are seeking independent assurance and documentation of the integrity of the structures data during migration and up until go-live of new system.
Artefacts generated as part of this project must demonstrate, document, and communicate the methodology and outcome of the assurance of the data integrity before, during, and after migration. We expect this will require the use of data profiling tools, data models, and 'checksum' calculations or similar. Time must be spent explaining the assurance methodology and outcomes to relevant Highways England staff, and providing documentation and knowledge share that will inform replication of the approach in the future.
Who the users are and what they need to do Users (includes HE staff, engineers, HE service providers, DBFOs) need to be able to access structures asset data in the new system with confidence there has been no loss of data integrity during the migration.

The data migration team need to be able to communicate method, progress, and ultimately outcome of this work so the Highways England exec can understand and trust the methodology applied to the project.

HE Head of Data Architecture must be able to review and update the logical data model(s) generated as part of this project using common data modelling toolsets (not including Microsoft Office products)."
Early market engagement Nothing to add
Any work that’s already been done Nothing to add
Existing team The supplier will be working directly with the Highways England project delivery team. The work may involve engagement with Leidos (who are extracting the data from source) and Bentley (who will transform and load the data in the target environment). Any engagement with these parties will be managed by Highways England.
Current phase Not applicable

Work setup

Work setup
Opportunity attribute name Opportunity attribute value
Address where the work will take place Highways England, 5 St Phillips Place, Birmingham, B3 2PW
Working arrangements During the initial 2-3 weeks of the engagement we require the supplier to co-locate with the Highways England team in Birmingham in order to develop a detailed plan for the work. During this time the supplier must agree a methodology, project plan, milestones, and engagement strategy for the project, as well as acceptance criteria for the final deliverables. After this, the supplier may work remotely but will be required to attend weekly governance meetings to review progress. This can be virtual attendance but physical presence will be required for any significant risks or issues.
Security clearance BPSS clearance required

Additional information

Additional information
Opportunity attribute name Opportunity attribute value
Additional terms and conditions

Skills and experience

Buyers will use the essential and nice-to-have skills and experience to help them evaluate suppliers’ technical competence.

Skills and experience
Opportunity attribute name Opportunity attribute value
Essential skills and experience
  • Experience in data migration assurance within a large public body
  • Experience of assuring enterprise scale, complex datasets of asset management data
  • Data architecture, data science skills aligned with DDaT framework
  • Accountability for data assurance at exec level within a large public body
  • Data modelling - conceptual, logical, and physical data models, data flow diagrams.
  • Experience working with civil structures data.
  • Awareness and understanding of the relevant PAS/ISO standards for asset management and data quality.
Nice-to-have skills and experience
  • Experience with multiple asset management software standards
  • Enterprise architecture and enterprise data models
  • Data migration (ETL)
  • Data profiling
  • Agile project management and kanban

How suppliers will be evaluated

How suppliers will be evaluated
Opportunity attribute name Opportunity attribute value
How many suppliers to evaluate 4
Proposal criteria
  • The proposed technical solution
  • The proposed approach, methodology, project management method, and technical artefacts
  • Past experience in delivering similar projects
  • Adherence with required timescales (initial assurance phase by 27th October 2019, final phase by 31st January 2019
Cultural fit criteria
  • Evidence of working collaboratively in an agile manner within a multi supplier project organisation.
  • Being transparent
  • Supporting a 'one team' environment
  • Take accountability for delivering agreed outcomes
Payment approach Fixed price
Assessment methods
  • Written proposal
  • Case study
  • Work history
Evaluation weighting

Technical competence

60%

Cultural fit

20%

Price

20%

Questions asked by suppliers

Questions asked by suppliers
Supplier question Buyer answer
1. When will the Q&A session be and what are the dial in details: Monday 15/07/2019 2PM

Join Skype Meeting: https://meet.lync.com/highwaysengland.co.uk/xmozadm/QV70NHYC

Join by phone

Toll number: +442034436279, access code: 60690004 (Dial-in Number) English (United Kingdom)
2. Has a timeline, with defined milestones dates, for the work to be done in the wider programme in the period August 2019 through to January 2020 been baselined? If so, please supply the timeline / milestones so that the structure of the programme of work can be understood. The baseline project plan contains the following milestones:

• Anticipated start date for Assurance work . Target early August but no later than 6th September 2019. N.B. Highways England is seeking to award the contract for this assurance as soon as possible.
• Completion of the initial data migration 4th September 2019,
• First delta loaded by 27th September 2019.
• Second delta loaded by 14th October 2019.
• Third delta loaded by 21st October 2019.
• All data exceptions addressed by 24th October 2019.
• Structures Cutover - 21st-28th October 2019 with go-live 28th October for all Structures users
3. Is there an incumbent currently responsible for Data Migration Assurance? HE has a data assurance lead in situ who is working with the suppliers but this is an independent assurance layer above that. So answer is no.
4. Is there a need for Security Clearance, BPSS etc? Yes, BPSS
5. Is the new asset management system a bespoke software development or a commercial off the shelf package. If a package, what package is being used? COTS, Bentley Asset Wise, Structures module
6. What database system will hold the asset information in the new asset management system? Please identify the database product being used. See above, with underlying Oracle database
7. In what time window was / will functional system testing of the new asset management system be performed? This has completed
8. In what time window was / will business acceptance testing, by HE, of the new asset management system be performed? This has completed
9. What database system holds the asset information in the existing asset management system? Please identify the database product being used. Oracle
10. How many systems consume receive asset management data from the existing system and will expect to receive data from the new system when it goes live? Are receiving systems expected to make any changes to enable them to receive data from the new system or is the data to be presented in an identical way? There are 8 external interfaces that are being modified to point to the new Structures system. The specification for the data exchange across these interfaces has not been changed from that of the current system.
11. Does the existing asset management system just provide for the maintenance of asset data or does it have a set of analysis / reporting functionality built into it? If so, does the new system provide the same set of analysis / reporting functionality? Both have reporting capabilities. The new system has a reporting data hub that allows Power BI Users to analyse and report on data.
12. Can you confirm that the Assurance Supplier's team will be able to have access to a read-only copy of the source asset management database and access to database tools allowing them to query this database? We will provide access to a read only copy of the source and target data. Querying tools provided: Source - PL/SQL, Target - Power BI
13. Can you confirm that the Assurance Supplier's team will be able to have access to a copy of the new asset management database, after execution of the migration process, and access to database tools allowing them to query this database? We will provide access to a read only copy of the source and target data. Querying tools provided: Source - PL/SQL, Target - Power BI
14. Are the migration suppliers contracted to and resourced (people and tools) for performing trial migrations on a regular basis to support assurance work? Would it be reasonable to expect weekly migrations to be performed to enable the assurance process, if not then what frequency would be acceptable? "The trial migrations have been completed and we are now in the process of migrating data in to the new production environment. This is an initial load of an Oracle data dump followed by a series of deltas on the source data. This process will continue until go-live at the end of October 2019. The actual frequency of deltas has yet to be decided.

N.B. We are not asking the assurance provider to assure the data migration process. We are only interested in them performing the assurance of the migrated data."
15. Does HE expect the Assurance Supplier to (a) assure the process in test environments, (b) perform assurance and reconciliation of the final production data transfer or (c) perform both of these activities? The scope is b) only. We will provide copies of the source and target environments for the supplier to verify that all data has been migrated.
16. How many different categories of asset are managed in the system and how many different assets models does the existing system provide to support these different asset categories? "The current and new systems will only hold structures assets.
There are 8 Structural Assets in the systems: Bridge And Large Culvert, Mast, Mast Scheme, Retaining Wall, Road Tunnel, Service Crossing and Other Structures, Sign/Signal Gantry, Small Span Structure plus approximately 300 component type. Each Structural Asset Model is different and an individual Structural Asset may have multiple Models over time.
17. Typically, how many attributes are held on an asset or on particular types of assets? The number of attributes for a full Asset/Component Hierarchy is in the region of several hundred
18. Does the existing system retain asset history or just the current state of the assets? Full history of all assets is retained.
19. If it is held in the current system then, what historical data is to be migrated to the new system and what historical data is to be left in the existing system but not migrated to the new system? All asset related historical data is to be migrated. (System specific tables in the current system will be left behind but these do not hold asset data.)
20. How many years has the existing asset management system been in service? Circa 20 years.
21. Is the strategy to establish the data in the new system and then to replicate updates made to the data in the existing system into the new system up until the point where the new system goes live or is the new system to be loaded at go-live? If the former then what process is to be used to keep the data in sync? It's the former. The service provider who supports the current system will periodically extract deltas that will be transformed and loaded into the target database by the new supplier, to an agreed spec.
22. A go live of 28th of October is indicated but the end date for the work is indicated as January 2020. Please can you clarify the aspects of the work that it is envisaged will take place after a go-live at the end of October? There is some audit data to be migrated post go live and some small areas of functionality that have no impact on the data migration. The period of 2 months is for the new supplier to evidence that the system meets the required service levels before it is finally signed off by HE.
23. Is there any period of parallel running of the two systems planned? Is there any plan to migrate the live management of different asset types at different times? No parallel running. The current system will remain fully live during data migration. Deltas will be extracted to keep the new system in synch up to go live. Users will be switched across to the new system on 28th October.
24. Will a dedicated test environment with a copy of the new system be available for the use of the assurance team? If not, what test environments will be available? Yes
25. Which organisation / who will be responsible for knowledge transfer to the assurance supplier covering the nature of the assets managed and the uses made of asset data? HE will provide the following documentation: Source/Target Data Dictionaries and Data Mapping documents etc. and will support knowledge acquisition for the assurance provider. We may use the providers of the current and new systems to support this.
26. Which organisation / who will be responsible for knowledge transfer to the assurance supplier the details of logical and physical data model used in the existing system? HE will provide the following documentation: Source/Target Data Dictionaries and Data Mapping documents etc. and will support knowledge acquisition for the assurance provider. We may use the providers of the current and new systems to support this.
27. Which organisation / who will be responsible for knowledge transfer to the assurance supplier the details of the extract / transfer and load processes that are to be used to migrate data from the existing system to the new system? HE will provide the following documentation: Source/Target Data Dictionaries and Data Mapping documents etc. and will support knowledge acquisition for the assurance provider. We may use the providers of the current and new systems to support this.
28. The Payment Approach is defined as being Fixed Price. What is HE's assumption about the impact on price of delays on the ability of the supplier to perform work on planned dates due to slippages in the delivery of artefacts, tools or activities that are not supplied / performed by the supplier and where such slippages increase the costs and/or reduce the efficiency of the supplier? Is it assumed the cost will remain unchanged under the influence of such factors? Any such impact / delay to the timeline will be handled via the contract change control process. We will work in close collaboration with the supplier to achieve the target go-live date. This will be agreed and documented in the call off contract document
29. There are 10 working days from the closure date to the latest start date. How does HE propose that contract terms can be finalised in a 10 day period? Is HE proposing that the supplier start work before a contract is in-place? No work will start without a contract in place. Please refer to our response to the first question for outline timescales. After 2 weeks of the tender process, we will shortlist to 4 suppliers and ask for tenders within 1 week. Our decision to award will be made 1 week after receipt of tenders.
30. Please, can the buyer provide an estimate of the total number of asset classes and assets being migrated? And / or data volume in megabytes? There are 8 Asset Classes and approximately 40,000 Assets comprising of approximately 1.5m Components. As stated above the source database is approximately 2 Terabytes
31. Please, can the buyer confirm the data formats the data will be made available in (.csv,.shp,.json, or similar)? Source and Target databases are Oracle.
Any data exports from these can be in any form required (primarily CSV and XML)"
32. Do you have a preferred template with word limit for the written proposal, case study and work history? No template, word limit of 300 words each
33. WRT "We are not asking the assurance provider to assure the data migration process. We are only interested in them performing the assurance of the migrated data.". Does "assurance of the migrated data" mean (a) reconciling the (sub-)elements back to the source system data or (b) does it incorporate aspects of determining if the representation of the asset has the same "meaning" and "behaviour" as it did in the existing system? The answer would imply (a) not (b) as (b) would seem to be a characteristic of the migration process design itself and not of the execution of the process It's a)
34. A 2TByte and 1.5million components indicates around 1.4MByte per component. On the surface this seems high. Does the database contain particular types of data assets that make up a large proportion of the data? For example does it contain high-resolution images / video or other bulk data streams? A large proportion of the size of the source database constitutes 1.3m documents (of varying types) held in BLOBS. The target database holds these in its own file Management System.
35. Question from Clarification call on 15/07: – Volume of data question was asked on Friday but no response yet. David Mole (DM) and Mike Groves (MG) answered as per the response that will be published. There is a difference between the documents and the assets. We will not be looking for byte level assurance on documents. These documents form the largest part of the data in SMIS. They are BLOBS but contain documents, videos, high definition photographs. Robin Gatward (RG) confirmed this was the case. Checks need to be made on the metadata rather than every byte.
DM stated that asset information will need to be assured at that lower level.
36. Question from Clarification Call 15/07 – Access for deploying tools, what sort of environment and access will you give us? DM no direct access to the source as it is live. Target is part production and part live. You will have a copy or access to a copy. MG Source SMIS system you can run queries against copies. Target system there is a data hub (Target reporting Environment) that has Power BI and we could give you access through it. Semantic data model could be queried. Data extracts would mean some security questions would need to be met. No GDPR issues.
37. question from clarification call 15/07: will we have to get HE access arranged? Yes
38. question from clarification call 15/07: Physical or Remote Access? MG that has to be our intent. DM we will sort remote access. You do not need to come to the building.
39. Question from clarification call 15/07 - RAS access? Yes