Defence Science and Technology Laboratory (Dstl).

CHATBOT digital assistant for tactical military users

Incomplete applications

14
Incomplete applications
11 SME, 3 large

Completed applications

24
Completed applications
18 SME, 6 large
Important dates
Opportunity attribute name Opportunity attribute value
Published Wednesday 2 January 2019
Deadline for asking questions Wednesday 9 January 2019 at 11:59pm GMT
Closing date for applications Wednesday 16 January 2019 at 11:59pm GMT

Overview

Overview
Opportunity attribute name Opportunity attribute value
Summary of the work Can CHATBOT digital assistants and Natural Language Processing technologies help tactical military users access relevant information and intelligence via their tactical radio (with no internet access anywhere in the system)? Dstl is looking for innovators to build a demonstrator and evaluate its performance and military utility.
Latest start date Monday 11 March 2019
Expected contract length Up to a maximum of 2 years. Full details to be provided to shortlisted suppliers in the ITT.
Location No specific location, eg they can work remotely
Organisation the work is for Defence Science and Technology Laboratory (Dstl).
Budget range The total available budget for tasks one and two is £700,000. The maximum available budget for each task is up to £350,000. A supplier must not exceed the maximum budget available per task(maximum £350k).
Task 1 : up to £350k
Task 2: up to £350k

About the work

About the work
Opportunity attribute name Opportunity attribute value
Why the work is being done To show how access to information and intelligence may be improved for military users who are operating via tactical radios (emulated here) though the use of a digital assistant (CHATBOT). The CHATBOT is to be located on the opposite end of the radio link to the military user at a location with good connectivity to military information/intelligence resources and services (but still no Internet access for security reasons).
Problem to be solved Tactical users do not have internet connectivity and are restricted to low data-rate error-prone communications (tactical radios). They often need rapid access to information and intelligence in a succinct form that is relevant to their current situation to aid their survival and mission success. Task 1 will develop XMPP-based chat services accessing information in documents via a remote CHATBOT interfacing with (existing) federated search capabilities and (to be provided) Natural Language Processing (NLP) capabilities. Task 2 shall address the need to access remote intelligence data via a CHATBOT with suitable machine reasoning capabilities.
Who the users are and what they need to do (Task 1) As a tactical user, via a radio link and using an existing Chat-application, I need to be able to search documents available at multiple remote locations, without knowing where they are, and be able to find information and intelligence that is relevant to my current situation without downloading whole documents, so that my mission success likelihood is improved.
(Task 2) I need to be able to access intelligence products in the Single Intelligence Environment Demonstrator, and either download a compact product, or by dialogue retrieve facts/intelligence assessments about pertinent Objects of Interest (OOI).
Early market engagement Not Applicable
Any work that’s already been done Prior research has produced a Federated Search capability, available openly at https://github.com/dstl/Open-Federated-Search (supplier may provide an alternative). The Single Intelligence Environment Demonstrator exists on a Dstl private cloud and must be used. Its capabilities and APIs are fully described in an Open Architecture document, which will be made available to those who are invited to submit a full proposal. It includes a Knowledge Base, which stores intelligence about Objects of Interest (OOI). GraphQL is used to access the Knowledge Base, returning a JSON-LD graph (an array of JSON objects, each providing some intelligence about an OOI).
Existing team The supplier team will work with User representatives (comprising Dstl, its Military Advisors and selected stakeholders) to ensure that the capabilities developed by the supplier are relevant to the military need. These User representatives shall also be invited to take part in a user-evaluation of the solution. No other contractors will be involved. Dstl will appoint a "Technical Partner" who will provide liaison with User representatives, provide technical advice to the supplier and to the Dstl Project Manager regarding technical choices proposed by the supplier, and technically review outputs for acceptance purposes.
Current phase Alpha

Work setup

Work setup
Opportunity attribute name Opportunity attribute value
Address where the work will take place Development anywhere in UK. Demonstration at either of the DSTL sites at Porton Down or Portsdown West.
Working arrangements Software development should be done offsite with unclassified data, including on public clouds. The solution should be transferred to Dstl's private Official-Sensitive Development Cloud (D-CLOUD) for system testing, evaluation and demonstration. This transfer should be via Docker containers. Dstl's D-CLOUD provides Windows 7 and Centos 7 VMs. Task 2 should start with a military requirements activity to agree what intelligence questions the tactical user may require to be answered, and what intelligence products and intelligence data can be generated by the supplier in the SIEC Knowledge Base to satisfy such needs. Fortnightly virtual Agile review meetings with Dstl.
Security clearance HMG Baseline Personnel Security Standard clearance is required for members of the team who need to access the Dstl Official-Sensitive cloud environment (which is where the demonstration and evaluation is required to take place towards the end of the project).

Additional information

Additional information
Opportunity attribute name Opportunity attribute value
Additional terms and conditions These include: - DEFCON 76 (EDN 12/06), DEFCON 501 (EDN 05/17) – (Note only to be used when interpreting the DEFCONS), DEFCON 531 (EDN 11/14), DEFCON 608 (EDN 10/14), DEFCON 611 (EDN 02/16), DEFCON 649 (EDN 12/16), DEFCON 658 CYBER, DEFCON 659A (EDN 02/17), DEFCON 705 (EDN 11/02). Further additional T's&C's may be included in the ITT issued to shortlisted suppliers.

Skills and experience

Buyers will use the essential and nice-to-have skills and experience to help them evaluate suppliers’ technical competence.

Skills and experience
Opportunity attribute name Opportunity attribute value
Essential skills and experience
  • Experience of working with Chatbot frameworks .The supplier has provided evidence of working with one or more CHATBOT framework which one(s) were used and why. ( Weighting 35%)
  • Experience of doing Natural Language Processing, Machine Reasoning/Machine Learning. The supplier has provided evidence including details of systems/services/methods used. (Weighting 35%)
  • Experience of using XMPP. The supplier has experience of using XMPP and has provided evidence to support this. (Weighting 5%)
  • Experience of software development, including working under a defined Agile development methodology and in a defined DevOps environment. The supplier has provided detailed evidence to support this. (Weighting 15%)
Nice-to-have skills and experience
  • Experience of using communications link emulators. The supplier has provided evidence of their experience. (Weighting 5%)
  • Experience of software development and delivery using Docker containers. The supplier has provided evidence of their experience. (Weighting 5%)

How suppliers will be evaluated

How suppliers will be evaluated
Opportunity attribute name Opportunity attribute value
How many suppliers to evaluate 6
Proposal criteria
  • Understanding the requirements. Full details to be provided to shortlisted suppliers in the ITT
  • The merits/limitations of the proposal. Full details to be provided to shortlisted suppliers in the ITT.
  • Merits/limitations of proposed chat client/server applications. Full details to be provided to shortlisted suppliers in the ITT.
  • Utility of proposed Natural Language Processing/Machine Reasoning capabilities to deliver Tasks 1&2. Full details to be provided to shortlisted suppliers in the ITT.
  • Suitability of the proposed radio channel emulator and proposed channel characteristics to be emulated. Full details to be provided to shortlisted suppliers in the ITT.
  • Suitability of the proposed industry supplier collaborative team. Full details to be provided to shortlisted suppliers in the ITT.
  • Acceptability of plans for system test, evaluation, and demonstration. Full details to be provided to shortlisted suppliers in the ITT.
  • Suitability/cost of proposed arrangements for future development by the Authority. Full details to be provided to shortlisted suppliers in the ITT.
  • Presentation. Full details to be provided to shortlisted suppliers in the ITT.
Cultural fit criteria
  • Evidence of working collaboratively on research tasks in an Agile manner to maximise the useful capabilities of solutions whilst keeping costs to client(s) down. The Supplier has provided evidence.
  • Supplier held proprietary capabilities utilised for the delivery of this task, should be offered to the Authority for research purposes (Under terms) both during and post-contract. The supplier has agreed.
  • Provide evidence of your track record of rapid development of innovative software solutions, for research and demonstration purposes. The Supplier has provided evidence.
Payment approach Capped time and materials
Assessment methods
  • Written proposal
  • Case study
  • Work history
  • Presentation
Evaluation weighting

Technical competence

50%

Cultural fit

20%

Price

30%

Questions asked by suppliers

Questions asked by suppliers
Supplier question Buyer answer
1. Is there a timeline for delivery of both Task 1 and task 2.
Also are these tasks sequential or is there a requirement to deliver on both concurrently.
The duration of each of task is expected to be one year each. These tasks could be delivered sequentially or partially overlapping. Task 1 should be started first.
2. As this is an Alpha phase, can you clarify who completed the discovery phase and when this was completed? The discovery phase was delivered internally and completed in Q4 of 2018.
3. Do you have any preferences on the natural language frameworks that may be used, or would this selection be made by the chosen supplier? No preference on Natural Language Frameworks. We would expect suppliers to identify what they intend to use and why, including any expected limitations/assumptions.
4. Is there an incumbent supplier that has been tasked with previous phases of work? There is not an incumbent Supplier. Previous research work in our demonstration environment has been delivered by a variety of suppliers who are no longer on contract with us.
5. What are the use cases beyond the problem described in project overview? In particular, we are interested in what sort of information the user may be seeking and whether the user has any other digital devices that may be able to receive retrieved data other than just voice radio? Part 1 of response
Examples of information that may be requested from documents (Task 1) include:
- A description of the kind of document that it is and provide meta-data extracted from it.
- List people/places/dates in the document.
- return sentences in the document that contain or are 'similar or related to' search terms (which can include people/places from above). The meaning of 'similar or related to' would need to be further defined in any proposal offering this capability.
- As previous bullet but expanded to the whole paragraph containing the matching search terms.
6. What are the use cases beyond the problem described in project overview? In particular, we are interested in what sort of information the user may be seeking and whether the user has any other digital devices that may be able to receive retrieved data other than just voice radio? Part 2 of the response
- If a document comprises a set of headings with text under them, provide a list of headings and the capability to return the text under a specified heading.
- Question-Answer capabilities allowing the user to ask questions whose answers are either explicitly stated in the document text, or are derived from what is explicitly stated by some inference process.

For Task 2 we would expect the supplier to work with us to develop a representative analysis workflow and products to support the chat interface.
7. The proposal criteria refer to an “industry supplier collaborative team” whilst also referring to a single supplier; would you be able to clarify your preferred approach? The Supplier could be a single supplier or a group of suppliers who chose to work together. We have no preference either way
8. Are there any existing solutions or technologies been used in this area? If so, can you provide an overview? Other than the ‘back end’ technologies mentioned in the call (the Federated Search and the Single Intelligence Environment demonstrator).
9. Are there any limitations on licensing or licensing costs? Is there a preference for open source components over COTS software? If there are any licencing costs they need to be included as part of the total budget for this work. Licencing costs should cover a period of at least 3 years beyond the completion of this work.

We would welcome the use of Open Source Software, especially in those aspects where good Open Source solutions already exist. If a supplier proposes COTS software we would expect them to be able to justify why.
10. Has the research programme previously investigated CHATBOT frameworks and have any research reports been published that would be relevant to this task?
a. Related to this, are there any CHATBOT frameworks that have been ruled out?
b. And any that are preferred based on their characteristics in military environments and use cases?
We are aware of a number of different potential CHATBOT frameworks, but we have not done any previous work on these frameworks. As such we have not ruled any out yet, nor have we selected any preferred frameworks.

Given the context of the work, suppliers should state which framework they intend to use and why.

Note it is essential that the CHATBOT built under this contract, can run on our IT infrastructure without the use of an internet connection.
11. How representative is the unclassified data that is to be used for development? In terms of the overall task requirement, the data sources will be representative in that they will be a repository of documents with associated meta data, the documents themselves containing information that will be relevant to the queries.
12. Do you have any preferences on the radio simulator that may be used, or would this selection be the responsibility of the chosen supplier? A suitable emulator could be CORE/EMANE which is available free from the US Naval Research Lab at -https://www.nrl.navy.mil/itd/ncs/products/emane, but other solutions can be considered by the supplier.