Met Office

SurfaceNet Collate Platform

Incomplete applications

Incomplete applications
5 SME, 2 large

Completed applications

Completed applications
3 SME, 5 large
Important dates
Opportunity attribute name Opportunity attribute value
Published Monday 26 February 2018
Deadline for asking questions Monday 5 March 2018 at 11:59pm GMT
Closing date for applications Monday 12 March 2018 at 11:59pm GMT


Opportunity attribute name Opportunity attribute value
Summary of the work Using AWS the project will deliver a range of functionality including:

ETL data flow engine with flexible rules driven quality control processes

Data migration / conversion of existing data

Metadata querying and production e.g. calibration and quality flags

Delivering visualised data to consumers in near real time

Early life support
Latest start date Wednesday 30 May 2018
Expected contract length 2 years
Location South West England
Organisation the work is for Met Office
Budget range In the region of £1,000,000.00

About the work

About the work
Opportunity attribute name Opportunity attribute value
Why the work is being done The current on premise system for the collection & processing of surface sensor data is reaching its end of life. The Met Office are looking to leverage Amazon cloud platform technology to provide a flexible, scalable and resilient data processing platform to meet customer requirements around timeliness and quality of data.

Observations data is at the heart of everything that the Met Office undertakes including climate research and the initiation and validation of forecasting models. Raw observations contribute to the national archive, scientific research and international exchange of data and improved access to this data is a key project driver.
Problem to be solved The project needs to be able to collect and disseminate observations data to allow existing systems to be retired while also meeting rules around quality and timeliness.

Key Outcomes:

Automated tools to improve UX and deliver efficiencies.

Ingest raw data from a range of sources.

Transform data through calibration and a flexible data QC rules engine.

Load data for consumption in agreed API formats

Handle comprehensive Metadata.

Meet NFRs such as security, timeliness, reliability, and WMO/CAA etc. standards.

Ensure Continuity of existing services and data.

Use AWS tool sets where possible
Who the users are and what they need to do Direct consumers of the data will include:

Network managers.


Quality control team.



Commercial consumers.

Each grouping will have some bespoke requirements and may have slightly different perspectives on the same core requirement or element this will drive a significant degree of user flexibility to be incorporated into the system design.
Early market engagement
Any work that’s already been done A detailed requirements capture process has informed a high level system architecture. The concept has also undergone peer review from industry experts. Rules around processing metrological data are well defined with definition from the WMO and others.

A Proof of Concept has been undertaken with AWS IOT team to demonstrate core functionality.

The project sits as part of a wider program of work including a new data gateway and is underpinned by the subject matter experts within the observations department.

Initial hardware testing has been undertaken on integration of the field infrastructure with the AWS platform.
Existing team The Met Office team includes:

Project Manager.

Product Owner.

Solution Architect.

Tech Lead.

Business Analyst.

Hardware Test Engineer.

Met Office developer staff will also be assigned to ensure transfer of skills during development to ensure future operating capability.

Timely and appropriately skilled resource to complement and support the supplier team as and when required will be made available in order to remove impediments or provide domain knowledge.
Current phase Beta

Work setup

Work setup
Opportunity attribute name Opportunity attribute value
Address where the work will take place Met Office, Exeter, EX1 3PB. The majority of work is expected to take place on site to ensure robust knowledge transfer.
Working arrangements The supplier will deliver through Agile methods and will be able to work closely with Met Office teams. It should be noted that the project is required to meet a series of stringent regulatory requirements. The supplier will be expected to use the Met Office’s own systems such as JIRA (Confluence, Stories on Board) etc.
Security clearance Work on the Met Office premise or its IT environment will require SC.

Additional information

Additional information
Opportunity attribute name Opportunity attribute value
Additional terms and conditions It is the Met Offices intention to hold the full IPR for all systems components and at the completion of the project all source code, business rules and documentation should be provided to the Met Office.

Skills and experience

Buyers will use the essential and nice-to-have skills and experience to help them evaluate suppliers’ technical competence.

Skills and experience
Opportunity attribute name Opportunity attribute value
Essential skills and experience
  • Experience/evidence of working on AWS utilising PaaS based tools and technologies to implement modular solutions to provide ETL functionality.
  • Experience of working alongside a customer team to deliver an integrated platform (ensuring knowledge transfer).
  • Ability to use Agile methodologies while complying with mandated international (meteorological) standards (provided).
  • Experience of dealing with data with prescriptive data processing and timeliness rules.
  • Ability to surge resources to support agreed project plan.
  • Experience of working with and designing scalable APIs for a wide variety of end users.
  • Utilise the most up to date IoT toolsets and understanding of IoT Security Foundation best practice.
  • Digital design skills with a focus on user experience.
  • Experience using Continuous Integration development approaches.
Nice-to-have skills and experience
  • Experience of delivering projects within the environmental monitoring sector ideally from a dispersed network of remote monitoring solutions.
  • A proven partnership with Amazon delivering AWS cloud hosted solutions, for large organisations.
  • Proven experience of working with government sector organisations to deliver projects with complex regulatory requirements including international standards and governing bodies.

How suppliers will be evaluated

How suppliers will be evaluated
Opportunity attribute name Opportunity attribute value
How many suppliers to evaluate 5
Proposal criteria
  • Track record in similar applications
  • Experience with working with customer teams and integrating with existing platforms
  • Approach and methodology
  • How data latency can be managed effectively
  • Key risks and dependencies of your proposed strategy and mitigations you would put in place to manage them
  • Proposed Team structure and capabilities on a phase by phase basis
  • Organisational capacity to provide additional or flexible capability aligned to the scheduled deliverable
  • How knowledge will be imparted to ensure the MO can manage operation of the system into the future including examples of this from previous projects
  • Value for money
Cultural fit criteria
  • Alignment to the GDS digital principles with a ‘user first’ focus Embedded agile alignment
  • A culture of openness and collaboration, a willingness and understanding of the principles involved in being a ‘strategic partner’ rather than a supplier
  • A flexible approach to communication, especially in the promotion of agile principles
  • A willingness to support co-location and face to face working
  • A cohesive approach to working with internal capabilities
  • Evidence of thought leadership in the creative and strategic space
  • Focus on open knowledge transfer to ensure future Met Office internal support
Payment approach Capped time and materials
Assessment methods
  • Written proposal
  • Case study
  • Reference
  • Presentation
Evaluation weighting

Technical competence


Cultural fit




Questions asked by suppliers

Questions asked by suppliers
Supplier question Buyer answer
1. Reference: "The majority of work is expected to take place on site to ensure robust knowledge transfer". Can suppliers propose a hybrid approach in which the majority of the work takes place at the premises of the supplier and supplier's staff meet frequently with the Met Office's project team? No.
The majority of the work is to take place on the MO premises in Exeter to allow pair programing and robust knowledge transfer.

We expect the supplier to be embedded with our staff.
Kind Regards
2. Apart from AWS, what other technology stack does the project wish to use e.g. Java, Python? The core technologies we use are:

Java (in AWS Lambdas)
Spring Boot for micro services
AngularJS for web applications

Python is also used when appropriate.