UK Research and Innovation: Science & Technology Facilities Council

: UK SBS DDaT19253 UKRI-STFC Data Movement Framework for Scientific Analysis Service

3 Incomplete applications

1 SME, 2 large

7 Completed applications

6 SME, 1 large

Important dates

Tuesday 12 November 2019
Deadline for asking questions
Tuesday 19 November 2019 at 11:59pm GMT
Closing date for applications
Tuesday 26 November 2019 at 11:59pm GMT


Summary of the work
For many National Facilities user communities, the computing infrastructure remain very difficult to use. The Ada Lovelace Centre is developing a set of tools to make the computing infrastructure easier to use.
Latest start date
Monday 6 January 2020
Expected contract length
3 months with up to 3 months optional extension
South East England
Organisation the work is for
UK Research and Innovation: Science & Technology Facilities Council
Budget range
The estimated value of this opportunity for the initial 3 month period is a maximum of £100,000.00 excluding VAT.

Should the optional extension be utilised, the total value shall not exceed £200,000.00 excluding VAT.

About the work

Why the work is being done
The Ada Lovelace Centre (ALC), is being established as an integrated, cross-disciplinary data intensive science centre, for better exploitation of research carried out at our large scale National Facilities including the Diamond Light Source (DLS), the ISIS Neutron and Muon Facility, the Central Laser Facility (CLF) and the Culham Centre for Fusion Energy (CCFE).
The Centre has the potential to transform research at the Facilities through a multidisciplinary approach to data processing, computer simulation and data analytics. The impact will be felt across the many science disciplines and communities these facilities support, including industry and academia.
Problem to be solved
Researchers processing their experiment data are required to transfer their data to an appropriate compute resource. The Data Movement Service and Data Transfer Service aim to automate the transfers of data in order to simplify the user experience.

We have two prototype software components that need further development to turn them production ready. They need to be tested and possibly modified to verify their functionality meets expected requirements with respect to performance and scalability. Further work is required to implement JSON Web Tokens (JWT) for cross component authentication as well as the creation of new caching strategies.
Who the users are and what they need to do
As a National Facilities user and academic researcher I need to access my experimental data on appropriate compute resources so that I can process and analyse my experimental data to make more effective world-class scientific research discoveries for the benefits of society.
Early market engagement
Any work that’s already been done
We have two existing software components: Data Movement Service (DMS) and Data Transfer Service (DTS).

The DTS is responsible for handling the transfer of data between two points. It comprises of two components: a control service running Python Tornado, a worker service that communicates with the control service and uses websockets to transfer the data.

The DMS is responsible for the coordination of what data needs to be transferred and where to. It comprises of two components: a FUSE application that captures filesystem operations, a cache manager that decides if a transfer request needs to be passed to the DTS.
Existing team
The Dynamic Infrastructure Group from the Scientific Computing Department is in place at the Rutherford Appleton Laboratory. You will be working with this team – and will have occasional interactions with the National Facilities teams.
STFC will also be recruiting internal staff who are expected to arrive subject to successful recruitment (recruitment is currently in process). The supplier’s team will be expected to work closely with the recruited internal staff, and to perform a complete and thorough handover to them.
Current phase

Work setup

Address where the work will take place
Rutherford Appleton Laboratory, Chilton, Oxfordshire OX11 0QX.
Working arrangements
Monday-Friday 9am-5pm and occasional remote working subject to agreement.
Security clearance
Baseline Personnel Security Standard (BPSS) and/or Disclosure Scotland. UKRI will sponsor this, if required.

Additional information

Additional terms and conditions
T&S as per UKRI policy.

Skills and experience

Buyers will use the essential and nice-to-have skills and experience to help them evaluate suppliers’ technical competence.

Essential skills and experience
  • Demonstrate...... (See “Problem to be solved”)
  • your experience in deploying production services on Linux systems (6%)
  • your expertise in developing web APIs using Python (6%)
  • your expertise in performance and scalability testing of Python applications (6%)
  • your expertise in data caching strategies and large volume data movement (6%)
Nice-to-have skills and experience
• Demonstrate your expertise in using JSON Web Tokens (JWT) (6%)

How suppliers will be evaluated

All suppliers will be asked to provide a written proposal.

How many suppliers to evaluate
Proposal criteria
  • Technical solution (6%)
  • Approach and methodology (6%)
  • Estimated time frames for the work (6%)
  • How they’ve identified risks and dependencies and offered approaches to manage them (6%)
  • Demonstrate how you will solve technical problems throughout this project (6%)
Cultural fit criteria
  • Demonstrate how you will work with the other software developers and system providers throughout this project (5%)
  • Demonstrate how you will share knowledge and expertise with the software developers and service managers (5%)
Payment approach
Capped time and materials
Additional assessment methods
Evaluation weighting

Technical competence


Cultural fit




Questions asked by suppliers

1. Latest start date
The commencement of the services is scheduled for Monday 6th January 2020 in accordance with the published requirements. However, this is subject to change should there be any delays in the procurement process
2. Procurement timeframe
The anticipated procurement timeframes are as follows:
6th December 2019 – Notification of shortlisting outcome
13th December 2019 – Written proposal deadline
17th December 2019 – Presentations
30th December 2019 – Notification of award outcome
3. Technical Competence Criteria
The technical competence section included within the above requirements has been allocated 30% of the overall technical competence weighting for the shortlisting stage.
4. Technical Competence Criteria (in addition to above)
The remaining 30% for technical competence will be allocated to the assessment stage. Further information on this will be shared with shortlisted bidders.
5. Alpha
All outputs and relevant documents from the previous alpha phase will be shared with shortlisted bidders.
6. "Can you please confirm what kind of response is required for your first listed 'essential skill', ""Demonstrate....... (See 'Problem to be solved')"", which appears to be a non-question and to refer to the required essential skills which come after it.
If you confirm that this does not require a full answer, will you nevertheless score each applicant the same?"
No response is required for this question, it's a heading for the below criteria. No score will be given to suppliers for this question.
7. Have you considered not using Python for the DTS? Other technologies exist which are better suited to this task and would offer lower upfront and maintenance costs.
The DTS has already been started in Python. We have also written everything else in Python where we can, this helps with maintainability. As we are already using Python it would be too much work to start again using another technology.