Met Office

Met Office Space Weather Programme Delivery Partner 2020-22

Incomplete applications

13
Incomplete applications
9 SME, 4 large

Completed applications

22
Completed applications
14 SME, 8 large
Important dates
Opportunity attribute name Opportunity attribute value
Published Monday 16 March 2020
Deadline for asking questions Monday 23 March 2020 at 11:59pm GMT
Closing date for applications Monday 30 March 2020 at 11:59pm GMT

Overview

Overview
Opportunity attribute name Opportunity attribute value
Summary of the work This work will undertake migrating the main components of the Met Office’s own operational Space Weather (SW) service from its own internal IT infrastructure to AWS. This project will also enhance the existing Met Office space weather forecast system and service delivery to customers.
Latest start date Monday 8 June 2020
Expected contract length 2 years
Location South West England
Organisation the work is for Met Office
Budget range £1.5M - £2M over the contract period

About the work

About the work
Opportunity attribute name Opportunity attribute value
Why the work is being done Space weather is a medium-high risk on the UK Government’s National Risk Register and the Met Office Space Weather Operations Centre (MOSWOC) is a key component of the UK’s approach to mitigate the space weather risk. The IT capability at the heart of this contract is the mechanism by which MOSWOC forecasters access, visualise and manipulate observational data and models, as well as the delivering products and services to a wide range of expert and non-expert users.

This contract will provide continuity, whilst developing and supporting (DevOps) of the operational Space Weather service. The delivery partner shall undertake all evolving space weather BaU activities, alongside lifecycle management including migration of the current operational service to AWS. The purpose of this is to reduce operational costs for the service and reduce cycle time for changes. The space weather development project will continue to adhere to standard assurance processes with regular architecture reviews etc.

Regular communication will be maintained with key projects to ensure consistent approaches and integration, where possible.
Problem to be solved The Met Office owns the UK space weather risk on behalf of BEIS.  Space weather is a risk high on the National Risk Register and describes changing environmental conditions in near-Earth space. Magnetic fields, radiation, particles and matter, which have been ejected from the Sun, can interact with the Earth's upper atmosphere and surrounding magnetic field to produce a variety of effects. In our increasingly technologically dependent society, the impact of space weather events can disrupt power grids, radio communications and satellite operations including GPS. 

As part of the National Risk Assessment mitigation strategy, BEIS fund the Met Office to provide a 24/7 operational service, through the Space Weather Operations Centre (MOSWOC) to ensure government and critical national infrastructure providers receive the vital forecast information needed to help mitigate the effects of space weather.  

This work will undertake migrating the main components of the Met Office’s own operational Space Weather (SW) service from its own internal IT infrastructure to AWS. This project will also enhance the existing Met Office space weather forecast system and service delivery to customers.
Who the users are and what they need to do Met Office Space Weather forecasters use the system to visualise and manipulate forecast model data, create alerts and warnings and deliver these to a huge range of customers from the Government, to businesses, the general public, armed forces, researchers and other organisations.
Early market engagement
Any work that’s already been done We have a 24/7 365 service, which has been operational since 2014. We are working towards migrating the service to AWS, operationalising new models and making our data available to a wider range of customers. This has been done in conjunction with the incumbent partner.
Existing team For the last six years IT development has been delivered in conjunction with a software development and testing partner with requirements set by Met Office Technical Lead, Project Manager, Product Owner and Solutions Architect.  Members of the Met Office Science and forecaster teams also assist in the translation of requirements and refining of deliverables.
Current phase Live

Work setup

Work setup
Opportunity attribute name Opportunity attribute value
Address where the work will take place During on-boarding the partner’s team will be expected to work onsite at the Met Office’s HQ based in Exeter before switching to work predominantly at their own corporate offices, with regular 2 weekly (subject to change) meetings taking place in Exeter to fit with the scrum sprint process. 

Quarterly formal service/contract review meetings will also take place (either on-site or via teleconference).

Met Office HQ
Fitzroy Road
Exeter
UK
EX1 3PB
Working arrangements Regular face-to-face meetings in Exeter encompassing demonstrations and reviews of new functionality and reflection on progress and planning for the next fortnight.

There will be daily stand-ups. The partner’s senior software developer will meet with the Met Office project team to review the backlog and priority of work. We expect our partner to attend these face-to-face or via collaborative communication methods, as appropriate. Our partner will work with the Met Office release management team, agreeing regular release dates. They will also work with other Met Office teams to deliver some work packages where additional support is required.
Security clearance The partner’s development team must be capable of successfully progressing through the security clearance process. Met Office can sponsor clearances if required. It is desirable to have access to readily cleared developers and possibly with higher classifications, as and when required.

Additional information

Additional information
Opportunity attribute name Opportunity attribute value
Additional terms and conditions All expenses must be pre-agreed between the parties and must comply with the Met Office Travel and Subsistence (T&S) Policy.

We will use a credit reference agency (CRA) to assess the supplier's economic and financial standing and reserve right not to award a Call-Off Contract to a supplier.

All staff who access our assets must agree to our non-disclosure/confidentiality agreements and other Met Office compliance standards.

Skills and experience

Buyers will use the essential and nice-to-have skills and experience to help them evaluate suppliers’ technical competence.

Skills and experience
Opportunity attribute name Opportunity attribute value
Essential skills and experience
  • Tech/Team Lead Developer: (Reference level: SFIA 5)
  • Developers: (Mix of Reference level: SFIA 3 and 4)
  • Recent proven ability of delivering solutions with cross-functional teams in an Agile way, including Scrum and Kanban
  • Demonstrate recent experience of engineering secure full stack web solutions using technologies: Java, Python, Javascript/Typescript, HTML5, CSS3/SASS
  • Demonstrate recent experience of engineering solutions with AWS managed services (S3, DynamoDB, Lambda, IAM, API Gateway, SQS, SNS)
  • Demonstrate recent experience of building complete deployment and CI automation using technologies such as Jenkins, AWS CodeBuild, AWS CodePipeline, CloudFormation, CDK
  • Demonstrate recent experience of supporting applications using Open Source, IDL, Java, Dojo, MongoDB, Mongo Atlas, Apache Camel, ActiveMQ, Bitbucket, Jira, Maven, Jenkins
  • Experience of creation and maintenance of Documentation; training of users
  • Experience of working with and transforming large and frequently updating data sets
  • Demonstrate experience of secure coding compliant with Cyber Essentials Plus
  • Proven ability to define and create testing strategies and automate the testing of code
  • Experience of providing 4th line support
  • Recent experience migrating platforms to AWS
  • Demonstrate experience of being able to adapt and align to other organisational ways of working to enable successful project delivery
  • Evidence of working with academic/research personnel
  • Demonstrate ability to explain complex technical information to non-technical stakeholders
Nice-to-have skills and experience
  • Knowledge in the space weather domain
  • Ability to bring in team members with DV security clearance as and when required
  • Experience of developing on a shared on-premises platform
  • Experience of cross-organisation collaborative working
  • Ability to call on User Experience skills (Reference level: SFIA 3 or 4)
  • Ability to call on Business Analyst skills (Reference level: SFIA 3 or 4)

How suppliers will be evaluated

All suppliers will be asked to provide a written proposal.

How suppliers will be evaluated
Opportunity attribute name Opportunity attribute value
How many suppliers to evaluate 3
Proposal criteria
  • Technical solution
  • Approach and methodology
  • How the approach or solution meets user needs
  • How they've identified risks and dependencies and offered approaches to manage them
  • Team structure
  • Value for money
  • Ability to meet security requirements included in Cyber Security Questionnaire
Cultural fit criteria
  • Demonstrate alignment with Met Office collaborative working practices
  • Have a defined knowledge transfer process and be prepared to share knowledge with internal and external personnel
  • Possess demonstrable experience of working in an Agile collaborative environment to deliver similar project(s) to a customer
  • Demonstrate four key Met Office behaviours: Drive, Integrity, United, Vision coupled with Clear and Inspiring communication
  • Demonstrate a commitment to equality, diversity and inclusion
  • Demonstrate a commitment to sustainable working practices
Payment approach Time and materials
Additional assessment methods
  • Case study
  • Work history
  • Reference
  • Presentation
Evaluation weighting

Technical competence

55%

Cultural fit

20%

Price

25%

Questions asked by suppliers

Questions asked by suppliers
Supplier question Buyer answer
1. What is the end state for this project, i.e. data lake with analytics systems consuming the data in the data lake? Space Weather is a program with multiple work streams. One is the operational Space Weather observation and forecasting system, which will be migrated from on-premises to cloud during the period of the upcoming contract. There are additional workstreams to expose APIs to partner meteorological organisations, and to facilitate research-to-operations transition for partner scientific organisations. There is no requirement at this point for analytics in an OLAP sense, or for data to be in a format that facilitates OLAP analytics.
2. Is there an OpX budget for run? Yes there is an OpX budget external to the declared contract budget.
3. Will we using normal AWS or govAWS? We use normal AWS.
4. Are we migrating data from on-prem to the cloud? Space Weather data is perishable and is archived off after 30 days. Therefore, there is no need to migrate any data from on-prem to the cloud. However, there is the possibility that we will in future transition our archived data to be AWS-resident.
5. Are there any data requirements, i.e. metadata tagging, data lineage, data cataloguing? There is currently no hard requirement for data cataloguing, but a data dictionary is maintained by the team. There are no formal requirements for data lineage tracking or metadata tagging. However, there is some minimal processing to add some metadata during data ingestion to indicate (for example) which satellite instrument was designated as the primary source at the time of the observation.
6. Do they know what format the data needs to be in for the analytics tooling? Please refer to the response to question (1). With that in mind, Space Weather APIs, by default, serve data as JSON.
7. What is the expected engineer/development split? Is the dev planned to be frontloaded or is it expected to have a steady level of support throughout the project? The Space Weather program has a long lifetime, and so the development process is on-going and interleaved with operations support. A fixed percentage of level-of-effort is dedicated to bug fixing and technical debt reduction. This percentage can be adjusted in response to urgent work (e.g., the sudden disappearance of a data source).
8. We have a query regarding one of the abbreviation IDL which was mentioned in the question with regards to demonstration of recent experience on technologies. Can you please confirm whether IDL stands for
Interface Description Language or Interactive Data language or something else?
nteractive Data Language. Please refer to https://en.wikipedia.org/wiki/IDL_(programming_language) for further detail.
9. What does IoC and FoC look like? What is the desired endstate? Please refer to the response to question (7). IOC was achieved several years ago. Rather than a defined FOC, we anticipate the continued rollout of new capabilities as the application suite evolves.
10. Does the end user have a planned technology path or is it up to the supplier to offer the solution – i.e. are you aiming for a serverless solution? If this refers to the AWS migration part of the effort, then yes, there is a high-level plan and some intended technology direction. We expect the migration to include wholesale replacement of some components with lambdas, and containerisation of others with subsequent refactoring where there is value in doing so. We have an initial estimate of the technology approach to be taken with each component.
11. Are there defined milestones already laid out? At the start of this contract firm milestones are yet to be finalised. The contractor will be expected to provide input to guide the setting of realistic contract milestones.
12. Please can you provide further detail about the 4th line support you require? The development team will also act as the 4th line support team. They are responsible for resolving incidents to the Space Weather service that could not be resolved by the centralised 24/7 support teams. This will usually require much more detailed system knowledge and is likely to result in a code or configuration change. 4th line support operates within normal business hours. Incidents that are passed to the development team are prioritised by the Product Owner.
13. Please can you provide further detail regarding what you mean by on premises platform? Is this the on premise instance of AWS? The current Space Weather application runs on-premises on existing Met Office servers and network infrastructure in two redundant Met Office data halls. This is not an on-premises instance of AWS.
14. Has an initial product backlog been crafted? We have an existing backlog.
15. How many teams are you expecting to be working on this project? We are expecting 1 team.
16. Are you expecting the main components of the Space Weather service to be replaced or simply migrated? That will be determined on a case by case basis but it is likely to be a mixture of the two.
17. Does the existing Space Weather system have design, development and support documentation available? Yes.
18. What is the technical stack for the existing Space Weather system? Please refer to the Skills and Experience section for an indication of the languages and frameworks in use. To summarise: the current Space Weather system is a multi-tiered application based on MongoDB, Java (including Apache Camel) and JavaScript (including Dojo). Some components run in Fortran, IDL or Python on the High Performance Computer (HPC), and other components run in Python and IDL on on-premises Met Office servers. There is a small amount of code running on AWS lambdas and EC2 instances.
19. Is the contract deemed to be outside IR35? IR35 is not applicable. This is a buyer-supplier contractual relationship governed by DOS4 T&Cs.