Defence Science and Technology Laboratory

Prototype for Automated Network Defence Actions (PANDA)

Incomplete applications

15
Incomplete applications
12 SME, 3 large

Completed applications

7
Completed applications
5 SME, 2 large
Important dates
Opportunity attribute name Opportunity attribute value
Published Wednesday 9 October 2019
Deadline for asking questions Wednesday 16 October 2019 at 11:59pm GMT
Closing date for applications Wednesday 23 October 2019 at 11:59pm GMT

Overview

Overview
Opportunity attribute name Opportunity attribute value
Summary of the work This SOR is for software development resources to support continued development of PANDA. The desired outcome is to mature PANDA into a Technology Readiness Level (TRL) 4 capability to support internal research projects and international collaboration.
Latest start date Saturday 30 November 2019
Expected contract length 12 Months
Location No specific location, eg they can work remotely
Organisation the work is for Defence Science and Technology Laboratory
Budget range £500-600k for the initial requirement (without extensions), spread over FY19/20 and FY20/21.

About the work

About the work
Opportunity attribute name Opportunity attribute value
Why the work is being done The UK MOD along with the commercial sector is heavily reliant on strategic and enterprise systems in both fixed and deployable contexts. Such systems can contain hundreds or thousands of endpoints across multiple locations, potentially in different countries and with varying levels of communication bandwidth and stability. These systems of systems are becoming highly dynamic and complex, which requires highly skilled operators to respond and recover from an ever increasing variety (in terms of sophistication and methods of operation) of threats.
Problem to be solved Currently response and recovery processes that are predominately mandraulic, with the ever increasing complexity placing a larger cognitive burden on a limited resource. Moreover, in increasingly federated/coalition mission environments, boundaries of system ownership and management authority are more porous. In these environments, response and recovery actions must be coordinated across operating authorities. Existing security orchestration products are ill suited to these situations.
Who the users are and what they need to do To address these challenges, Dstl is currently conducting research to raise the technological readiness level of automated network defence capabilities. To support this research, Dstl has developed the Prototype for Automated Network Defence Actions (PANDA). PANDA, which is written in Python 3, provides a proof of concept implementation that is capable of responding to incidents on networks automatically. It has also been designed to allow new components and concepts to be integrated quickly. This SOR is for software development resources to support continued development of PANDA.
Early market engagement
Any work that’s already been done During FY 14/15, Dstl ran an industry call through the Defence and Security Accelerator (DASA) to research and develop techniques to support the planning and orchestration of automated responses to protect networks facing complex multi vector attacks. Follow-on work in 2015/16 specifically examined automated Course of Action (CoA) analysis. In FY18/19, Dstl selected one of these concepts to form the basis of PANDA. Through two phases of development activity, PANDA has been matured from a high level conceptual design into a proof of concept software implementation.
Existing team The supplier will be working with a dedicated technical partner from Dstl for day to day interaction. Alongside this will be a small team from the project, who will contribute new concepts and components to PANDA. This team will act as hosts, when or should the need to work on the software on our site arises. This team operates out of their own laboratory space which is equipped to support software integration and development.
Current phase Alpha

Work setup

Work setup
Opportunity attribute name Opportunity attribute value
Address where the work will take place We would expect the majority of the work to be conducted at the suppliers own address, but there maybe times when both formal meetings and development work will take place at our Salisbury Site.
Working arrangements To ensure that new capabilities can be readily integrated into PANDA as the concept develops and evolves, the project will utilise an iterative and incremental approach to delivery. The supplier must adopt an Agile system engineering approach (e.g. Scrum) and work closely with the Authority throughout the development process to realise the project’s aims. It is expected that this work will require significant dialogue between the Authority and the supplier throughout the contract period; the Authority will make staff available to support this.
Security clearance Able to handle OFFICIAL material only.

Additional information

Additional information
Opportunity attribute name Opportunity attribute value
Additional terms and conditions DEFCON 76 (EDN 12/06); DEFCON 501 (EDN 05/17) – (NOTE ONLY TO BE USED WHEN INTERPRETING THE DEFCONS); DEFCON 531 (EDN 11/14); DEFCON 608 (EDN 10/14) ;DEFCON 611 (EDN 02/16); DEFCON 649 (EDN 12/16); DEFCON 659A (EDN 02/17); DEFCON 703 (EDN 08/13).FULL TEXT VERSIONS OF THE DEFCONS CAN BE FOUND VIA KNOWLEDGE IN DEFENCE (KID) https://www.gov.uk/guidance/knowledge-in-defence-kid

Skills and experience

Buyers will use the essential and nice-to-have skills and experience to help them evaluate suppliers’ technical competence.

Skills and experience
Opportunity attribute name Opportunity attribute value
Essential skills and experience
  • Demonstrate with evidence the ability to deliver without authority funded capability enhancements (for example, procurement of additional ICT to support development activities).
  • Demonstrate, with evidence the ability to conduct agile software development in-house, without support from subcontractors.
  • Demonstrate with evidence the ability to continue and evolve software development from a complex inherited codebase.
  • Demonstrate experience creating software using Python 3 to run on Debian-based Linux operating systems (>= Ubuntu 18.04 LTS) without reliance on closed-source, proprietary, software dependencies.
  • Demonstrate expertise and experience of developing software that utilises machine learning techniques to realise complex data analytics.
  • Demonstrate experience developing cyber security or system management software (preferably for high threat and government organisations).
  • Demonstrate, with evidence, experience of third party technology integration.
  • Demonstrate, with evidence, the ability to produce appropriate levels of documentation, and conduct verification and validation of software.
  • Demonstrate experience using the Git Source Code Management (SCM) system and broader collaborative development tool suites.
  • Provide examples of how you have provided customers with access to developmental and release versions of software source code, and engaged them in the development process.
Nice-to-have skills and experience

How suppliers will be evaluated

All suppliers will be asked to provide a written proposal.

How suppliers will be evaluated
Opportunity attribute name Opportunity attribute value
How many suppliers to evaluate 5
Proposal criteria
  • Demonstrate experience of developing robust, high quality software solutions in demanding timeframes, within the quoted budget.
  • Demonstrate expertise and experience of developing software that utilises machine learning techniques to realise complex data analytics.
  • Demonstrate experience developing cyber security or system management software (preferably for high threat and government organisations).
  • Demonstrate with evidence the ability to continue and evolve software development from a complex inherited codebase.
  • Provide a breakdown of the team structure (please include CVs of the team members doing the work).
  • Provide estimated timeframes for the work (include a Gannt chart with approximate sprint timeframes).
  • Detail and explain identified risks and dependencies. Provide details of offered approaches to manage these risks (include a Risk Register).
  • Provide a detailed breakdown of costs and include information against the following: Hours and rates (including hours allocated to each team member); Cost of materials; and Travel and Subsistence.
Cultural fit criteria
  • Demonstrate with evidence ability to follow industry best practice throughout the whole software development lifecycle from requirements gathering to documentation, testing, verification and validation (including examples of tool chains used).
  • Demonstrate consistent cultural commitment to agile software development practices, including embracing and responding to an evolving customer requirement
  • Provide a summary of the suppliers work ethos and working environment.
  • Demonstrate ability to successfully deliver within the UK Defence landscape.
  • Show evidence of an internal culture of knowledge and experience sharing.
  • Show evidence of being transparent and collaborative both internally and with the customer when making decisions.
Payment approach Capped time and materials
Additional assessment methods
Evaluation weighting

Technical competence

60%

Cultural fit

20%

Price

20%

Questions asked by suppliers

Questions asked by suppliers
Supplier question Buyer answer
1. Is this solution hosted on public Cloud or on-prem? “PANDA is a research prototype rather than a solution. The iterative development of it will be conducted by the supplier utilising an appropriately secured infrastructure, which may be public cloud based. Dstl will host PANDA using an on-premises research network. The PANDA code base includes a set of scripts to automate the containerized (using Docker) deployment of PANDA on Dstl's research network.”
2. How many stuff the supplier need to provide? It is down to the supplier to propose a solution to the outcome this would including the number of staff they requires to deliver the outcome.
3. Is it iside IR35 or outside? As per the CCS guidance Services to be delivered by suppliers under any lots of DOS 4 should be outside the scope of IR35.
4. Can you explain a bit more "Demonstrate with evidence the ability to deliver without authority funded capability enhancements (for example, procurement of additional ICT to support development activities)." please?
Are you gonna give us an environment where we can do our developments or you expect us to have some sort of hardware capacity for the project?
Dstl will not be providing an environment for suppliers to use for development. Dstl operates an internal test bed that will be accessible, as required, for demonstration and knowledge transfer purposes. However, for development and testing purposes, we expect the supplier to use their own development environment. Funding for PANDA should not be used to purchase additional ICT (including hardware and software) for the supplier’s development environment.
5. To meet your proposal criteria we need (quite) detailed requirements. However, you only provided very generic ones. Can you please advise how can we obtain concrete requirements, please? Shortlisted suppliers, will be provided all the information they’ll need to participate in the assessment stage. Initial down selection will be made on the essential skills.
6. Can you clarify the latest start date, please? Is there any flexibility around 30th Nov or is it a blocker if we’re not able to start on this date? Our intent is for suppliers to start work on the 30th November – unfortunately, we cannot offer much flexibility in start date.
7. 1 Is there an incumbent for this piece of work?
2 With tight timescales will there be a handover period between any incumbent and any new supplier?
1. Yes. Dstl have been working with Deep3 Software Ltd. since September 2019 to develop PANDA. This contract will end on 27th November 2019.
2. There will not be a handover process with the incumbent supplier. Dstl will provide the successful supplier with documentation for PANDA (which includes design documentation, developer guidance, as well as automatically generated documentation), as well as the source code and a copy of the working backlog. We appreciate that the successful supplier will require time to become familiar with PANDA and that this will be adversely affected by the lack of a handover period.
8. 3 As this has been running in some sort of guise since FY 14/15, What collateral if any will be provided? Dstl will provide the successful supplier with documentation for PANDA (which includes design documentation, developer guidance, as well as automatically generated documentation), as well as the source code and a copy of the working backlog.
9. 4 We will need stand-up dev and test platforms to replicate the working platform, are there any restrictions in using data, datasets, libraries etc? Dstl does not anticipate any restrictions in deploying PANDA for test and development purposes. PANDA does not make use of any background intellectual property and all open source software libraries used in PANDA have been selected for their permissive licensing. The source code does not include any sensitive data or datasets. PANDA includes a test harness that can be used to generate unclassified input data for test and evaluation purposes. Dstl has provided the incumbent supplier with anonymised examples of specific types where this has been necessary for testing functionality. These examples were classified UK OFFICIAL
10. 4 We will need stand-up dev and test platforms to replicate the working platform, are there any restrictions in using data, datasets, libraries etc? PANDA does integrate with one commercial product, Splunk Phantom, to orchestrate network devices. However, Splunk offers a Community Edition of Phantom, which is sufficient for testing PANDA.
11. 5 Will Deep3 Software Ltd, winners of DOS Outcome 6916 – Prototype for Advanced / Automated Network Defence Actions (PANDA), be eligible to bid for this procurement?
6 Please can the authority publish guidance on how to score 3 in relation to the Essential skills and experience?
7 Will employees from any potential suppliers / bidders be involved in the evaluation panel?
8 Please can you provide details of any Early Market Engagement which took place for this procurement?
5. Yes. Deep3 Software Ltd. will be eligible to bid for this outcome.
6. Suppliers can obtain a score of 3 by providing relevant evidence and experience as per the CCS guidelines. Please see <link> for further details.
7. No – there will not be any potential suppliers on the evaluation panel
8. There has not been any early market engagement for this procurement.
12. What is the role and responsibilities of the "dedicated technical partner"? The technical partner is Dstl’s internal technical lead for PANDA and will be the supplier’s primary / day to day point of contact at Dstl. They will act as customer for PANDA and work with the supplier to identify and manage backlog items to achieve the desired outcomes. Alongside this will be a small team from the project, who will acts as hosts, when or should the need to work on the software on our site arises. The Dstl technical partner will also be supported by an internal Dstl project manager.
13. Is there a frontend/ui element in the scope of this work? And if so what is it's technology stack? PANDA includes a web frontend that is built on Angular 7 and the CoreUI Angular Admin Template. This is run in a Docker container. The UI has been developed primarily for test purposes and as such does not have many production features (such as authentication).
14. Is there incumbent and is there going to be hand-over arrangements in place? Yes. Dstl have been working with the incumbent supplier to develop PANDA. There will not be a handover process with the incumbent supplier. Dstl will provide the successful supplier with documentation for PANDA (which includes design documentation, developer guidance, as well as automatically generated documentation), as well as the source code and a copy of the working backlog. We appreciate that the successful supplier will require time to become familiar with PANDA and that this will be adversely affected by the lack of a handover period.
15. What is the maturity of current work backlog? The current project backlog includes approximately 95 user stories that should be addressed by this outcome / contract period (PANDA release v0.3). Approximately 20% of these user stories have been elaborated and assigned estimates of effort. However, a significant proportion have not been elaborated – is expected that the supplier will work with Dstl to refine these items. It is also likely that further user stories will be added prior to contract award to reflect additional areas Dstl wishes to focus on.
16. Does the engagement include building and or testing of machine learning models? Or in other words – have you built ML techniques or is this something that will have to be designed and developed as part of the engagement. If the above is yes is there data available for training of ML models and if so is it hosted by DSTL and what are estimated volumes? What is the classification of the datasets? PANDA determines viable courses of action using a supervised learning approach and it is therefore important that the supplier is comfortable working with this type of technology. The intent during the initial phases of the contract is to modify this component of PANDA allowing multiple recommendation approaches to work in parallel. We may subsequently ask the supplier to implement or integrate new recommendation techniques developed by Dstl or other suppliers. Specific requirements will be elaborated as part of sprint planning activity. If training data is required, Dstl will work with the supplier to determine how this data should be generated.
17. While there is a requirement to handle official documents only could we leverage non-UK nationals as part of development team? Any non–UK or Dual National will be subject to DSTL security checks and we reserve the right to reject any Supplier staff that do not pass these checks.