Home Office

Themis Data Transformation, Migration and Management Information

Incomplete applications

12
Incomplete applications
7 SME, 5 large

Completed applications

26
Completed applications
10 SME, 16 large
Important dates
Opportunity attribute name Opportunity attribute value
Published Friday 10 May 2019
Deadline for asking questions Friday 17 May 2019 at 11:59pm GMT
Closing date for applications Friday 24 May 2019 at 11:59pm GMT

Overview

Overview
Opportunity attribute name Opportunity attribute value
Summary of the work The Home Office requires a supplier/consortium to build the components for Data Transformation and Enrichment, Data Migration and Management Information for the Themis solution, and deliver the associated migration, reports and transformation services into the live environment.
Latest start date Friday 30 August 2019
Expected contract length 2 years, with the potential for an additional period (up to 6 months)
Location No specific location, eg they can work remotely
Organisation the work is for Home Office
Budget range £1m - £3m

About the work

About the work
Opportunity attribute name Opportunity attribute value
Why the work is being done The Home Office is seeking to replace an intelligence capability that is vital to the UK Law Enforcement Community. The legacy solution used for the management, storage and analysis of intelligence is a large monolithic application. The replacement capability will be developed within a modular framework based on a Service Oriented Architecture (SOA) that provides greater speed of change and reduced operating cost.
Problem to be solved The Authority is seeking either an individual supplier or consortium that can use evolutionary development techniques to deliver subsystems and provide capability to support data profiling, migration and transformation using system APIs. The activity also includes development of marts and initial reports for business MI to enable a transition to self service operation. Migration is an essential part of the ongoing development process and needs to be performed incrementally and repeatably on a combination of synthetic and live data across different environments.
Who the users are and what they need to do The primary users of the THEMIS system overall will be from Law Enforcement and will include operational users (such as intelligence officers and operations managers) and business users (e.g. system administrator or inspectors).

Some indicative examples of user needs for the Management Information component might include:
- As an Inspector, I need to view a report showing the number of legal authorisations of a particular type that have been granted within a time window.
- As an Operations Manager, I need to view the workload of my staff so that I can understand how to adjust the allocation of work.
Early market engagement
Any work that’s already been done The Design and Architecture team have performed a significant amount of user engagement across all stakeholder organisation to define the user requirements. These have been captured as a business capability map, key capability concepts, user stories (capturing functional requirements), user journeys, scenarios, and user personas.

These requirements have been used to develop an initial Themis architecture model that articulates the key elements of the Themis system as well as some key architecture principles informed by a proof of concept.
Existing team The Authority is supported by a Design & Architecture function which includes:
- A product owner;
- A business analysis team;
- An architecture team.

The remainder of the team will be procured to build the Themis solution as follows:
- Lot 1: Framework and DevOps pipeline (supplier chosen)
- Lot 2: Operations and Intelligence Management (procurement in process)
- Lot 3: Process and Oversight (procurement in Q3 2019)
- Lot 4: Analysis and Nominals (procurement in Q3 2019)
- Lot 5: Data Transformation, Migration and MI (this lot)
- Lot 6: UI Framework (procurement in process)
Current phase Discovery

Work setup

Work setup
Opportunity attribute name Opportunity attribute value
Address where the work will take place It is anticipated that development activities will be conducted from the suppliers own location.

Some non-development members of the suppliers team will be required to work from locations in proximity to the Home Office (2 Marsham Street) in London on a regular basis for coordination activities. Work may be required at other locations nationally from time to time.
Working arrangements Suppliers are required to participate in collaborative planning and delivery involving the wider Themis supply chain for each programme increment using a tailored SAFe approach. This will require some members of the team to work in proximity to the Home Office (2 Marsham Street) in London on a regular basis, and some development
activities will be conducted from the suppliers own location. For data migration it will be necessary to undertake some work at customer sites across the UK due to the protective marking of the material.
Security clearance SC level clearance is required as a minimum for staff working under this contract or any subcontracts.

The supplier must have a location and technology to support development up to and including Official Sensitive.

Additional information

Additional information
Opportunity attribute name Opportunity attribute value
Additional terms and conditions 1. Unless agreed, Home Office will retain ownership of the Intellectual Property Rights (IPR) created during the contract.
2. Travel and Expenses will not be paid within the M25.
3. Key Supplier staff included in proposals will be expected to work on the contract.
4. Home Office reserves the right to agree fixed price Statements of Work when it is deemed prudent; the Supplier shall not unreasonably withhold its agreement to such requests made by the Home Office.
5. Sub-contracting is permitted.

Skills and experience

Buyers will use the essential and nice-to-have skills and experience to help them evaluate suppliers’ technical competence.

Skills and experience
Opportunity attribute name Opportunity attribute value
Essential skills and experience
  • Demonstrate experience in data transformation and enrichment.
  • Demonstrate experience in performing automated and repeatable data migration.
  • Demonstrate experience in delivering MI marts.
  • Demonstrate experience of incremental delivery within a SOA/microservice-based architecture.
  • Demonstrate understanding of entity, intelligence and authorisations data within the Law Enforcement or Intelligence domain.
  • Demonstrate experience in the complete evolutionary software development lifecycle including testing and deployment.
  • Demonstrate experience of working in an Official Sensitive environment.
  • Have a suitable location and technology to support Software Development at Official Sensitive.
Nice-to-have skills and experience
  • Demonstrate experience of delivering within a scaled version of agile e.g. SAFe.
  • Provide evidence of successful collaborative delivery of an integrated solution where there were dependencies on other supplier teams.
  • Demonstrate experience in software development using Open Source and relevant COTS technologies.
  • Demonstrate expertise in complex IHMs (Information Handling Mod
  • Provide evidence of recent successful delivery of technology into the Law Enforcement or Intelligence domains.
  • Demonstrate experience of performing data migration by consuming APIs.

How suppliers will be evaluated

How suppliers will be evaluated
Opportunity attribute name Opportunity attribute value
How many suppliers to evaluate 5
Proposal criteria
  • Approach and Methodology
  • Value for Money
  • Risk
  • Team Structure and Composition
Cultural fit criteria
  • Work as a Team
  • No-Blame Culture
  • Knowledge Sharing
  • Collaboration
Payment approach Time and materials
Assessment methods
  • Written proposal
  • Case study
  • Work history
  • Presentation
Evaluation weighting

Technical competence

57%

Cultural fit

13%

Price

30%

Questions asked by suppliers

Questions asked by suppliers
Supplier question Buyer answer
1. Are you able to provide any information about the current technology being used to host THEMIS and any indication what you are planning to use for the future? We will be using a hosting environment based upon kubernetes and docker which will be deployed onto stakeholder provided infrastructure. Deployment will be achieved via the CI/CD pipeline provided by Lot 1.
2. Is there an expectation for the supplier to provide ongoing support/DEVOPS? No, this is being delivered as part of Lot 1.
3. What is the mechanism for document, source code and data transfer between the client and suppliers? Suppliers are assumed to have an Official Sensitive environment. It will be possible to connect to the CI/CD pipeline to pull and push code and images. It is envisaged that the Lot 1 supplied knowledge management environment will be used to store documents and wiki style information.
4. Could you please provide any additional detail on the downstream evaluation process? In particular the likely requirements for a proposal (beyond the outline criteria provided) and time scales for submission. Once evaluation of Stage 1 has completed the successful suppliers will be given more detail of what to include in their proposal, and additional information about the project. The deadline for proposal submission will be three weeks after the Request For Proposal material is distributed.
5. Are you open to deploying your application on PaaS platforms such as OpenShift or PCF? Our target deployment state will be Kubernetes/Docker/Istio.
6. What is the volume of data? (daily, historical and current) Our current understanding of the data held on the legacy system can be provided to suppliers who make it through to stage 2, although more detailed analysis is still required.
7. How many phases are planned for the migration? The detailed approach to the migration will need to be defined and agreed between the D&A Function, the supplier of this Lot, and relevant stakeholders.
8. Does the existing database adhere to any standards? What standards are to be followed/ adhered for the migrated data? The existing database is a bespoke Oracle database. The migration target will be populated via APIs so the target persistence schema is not relevant. The D&A Function will outline a handover format derived from other stakeholders within the domain.
9. Are you planning to or would be open to using container based platforms such as Docker and kubernetes? Yes, we are planning on Kubernetes/Docker/Istio.
10. Please provide an overview of your current development process. Can the supplier propose their own Development accelerators and frameworks? Development has not yet started, although planning with the Lot 1 supplier to design the CI/CD pipeline within a SAFe deliver model is ongoing. We are currently working on establishing a definitive set of processes and ConOp with the Lot 1 supplier.
11. What ETL tool is currently used? We need this information to ascertain our migration strategy for in As Is if data is being fetched from multiple sources. There is no current migration capability. Tooling choices will need to be worked through with the chosen supplier. We have a preference for open source, low cost and open interfaces.
12. Do you currently have DevOps capability? If yes, please provide the details else can supplier recommend the DevOps approach. Our DevOps capability will be provided by our Lot 1 supplier.
13. We are assuming that there are upstream systems from where the legacy intelligence system gets data feeds (or pulls data from these up steam systems). Is there any requirement to build / rework/ wrap integrations/interfaces with these upstream systems, to have continuity of the data flow? We will require you to build extraction components to obtain information in a repeatable manner from the source Oracle database and possibly other stores. Further upstream integration to solutions which consume information from THEMIS will be addressed in other Lots and subsystems, based upon the patterns and general approach outlined in this Lot.
14. How are integrations currently built and managed? Please see response to Q13.
15. We understand that you are going on a journey to SOA and Micro services – Is there a preferred technological stack such as Java, Spring boot, Angular etc..? Not at this time; we are evaluating the detail of this with other Lot providers. We do not believe that this should be unduly relevant to this Lot as the migration targets will be accessed via their API. Our requirement is that this Lot is developed using relevant technologies, possibly including our process engine.
16. Is it possible to share the To Be architecture, or a high level list of specs, so that an aligned solution can be recommended? This will be provided to the suppliers who make it through to stage 2 of the competition.
17. Is there any requirement for supplier to do process migration as well from the existing legacy application? Process state is currently expressed as business data in the source system. It may prove necessary to instantiate processes in our process engine in the target system
18. Does the application involve semi or unstructured data? The data is mainly well structured and transactional, with some CLOB storage of XML documents. As commonly occurs in legacy solutions, there are a number of large text based comments fields with variable data quality.
19. Do you have requirement for big data based data management and analytics solution? No. The scope of THEMIS is not to build a data fusion platform and existing data volumes do not require this approach. Scalability is important to us, however.
20. Please suggest, if Any specific tools or technology has been already identified for the target system, for reporting, creating data mart etc. For e.g. PowerBI, Qlikview No specific tools or technology choices have been made yet. We expect to work through these choices with the chosen supplier. We have a preference for open source, low cost and open interfaces.
21. Has the Home office identified or is interested in any specific migration, data profiling or data transformation tool? If yes, please suggest. Please refer to Q20.
22. Can supplier propose their own framework and accelerator for migration, data profiling and transformation? In principle, yes, but we have views on modularity and extensibility, plus any use of proprietary software within THEMIS would need to be assessed. We expect to work through the detailed approach with the chosen supplier.
23. Please clarify, if this work scope is only for Lot5 – Data Transformation, Migration and MI, or we need to consider requirements for other lots as well. Only Lot 5. The other Lots are being run as separate procurements through DOS, although all suppliers will need to collaborate to deliver an integrated solution.
24. We understand that the Home Office is planning to migrate a Legacy Intelligence solution. Is there an identified target platform and Dbase you want to migrate to? There will be several target databases. Migration will be via an API so the target database is not a concern.
25. Which database is currently being used? The source database is Oracle.
26. Please can the authority confirm whether a supplier can be awarded more than one Lot? i.e. if a supplier is awarded one of the lots will they be precluded from winning any of the others? The chosen supplier will be free to bid for future Lots.
27. Can you clarify what is meant by “suitable location and technology” to support development at Official Sensitive, with particular emphasis on the “technology” aspect? Please refer to the Government Security Classifications guidance for the specific requirements for working at OS: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/715778/May-2018_Government-Security-Classifications-2.pdf
28. When referring to an Official Sensitive “environment”, are you referring to data residing on the infrastructure/product/service rather than the physical office environment? Both the technical and physical development environments will need to be accredited for working at Official Sensitive.
29. Can you please share the security aspects letter for Themis suppliers, or give more information about the specific criteria/requirements regarding having "a suitable location and technology" to support software development at Official Sensitive. Please refer to the Government Security Classifications guidance for the specific requirements for working at OS: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/715778/May-2018_Government-Security-Classifications-2.pdf