The DataShed

The Data Refinery (Data Integration Platform)

The DataRefinery is a data integration platform which, at its heart, is a single view of a citizen/customer. Created by pulling together data sources, matching records and enhancing data using behaviour, demographics and interactions to build a single source of truth - providing instant insight and empowering informed business decisions.


  • Our industry leading algorithms enhance and de-duplicate your data
  • Standardise business definitions through intelligently plotting your data
  • Combine data from different sources into a single citizen view
  • Our dashboards allow a real-time view of processed citizen data
  • Enhancements enrich your data to provide further insight
  • Features to help with GDPR-compliance and overall data quality
  • Programmatic access via web based API


  • Simple, intuitive interface allows users to access insight quickly
  • Creates golden records, the combined best data for a citizen
  • Access data analysis without the need for a Data Scientist
  • Understand your long-term citizen value and behaviour patterns
  • Segment citizen/customer base for more effective targeting
  • Use characteristics of citizen profiles to target activity
  • Summarise numerous disparate citizen services into a single view
  • Search for and view individual citizens and their interactions
  • Valuable insight without additional investment in IT
  • Understand the citizens you serve as individuals


£3000 to £40000 per instance per month

Service documents


G-Cloud 11

Service ID

9 7 3 3 0 3 7 8 9 8 5 9 4 7 4


The DataShed

Ed Thewlis


Service scope

Service scope
Software add-on or extension No
Cloud deployment model
  • Public cloud
  • Private cloud
Service constraints The Data Refinery can be deployed in a public cloud, allowing the greatest possible flexibility to meet the buyer’s requirements. The product will be associated with a set of SLA’s and NFR’s for availability.

As a SaaS, The Data Refinery can be provisioned across a wide range of hardware and software configurations including all modern internet browsers.
System requirements The product is not associated with any specific system requirements

User support

User support
Email or online ticketing support Email or online ticketing
Support response times Service Desk operates between 08:00 & 17:30 during on weekdays (excluding bank holidays in England). On Call support is provided 24/7.
User can manage status and priority of support tickets Yes
Online ticketing support accessibility WCAG 2.1 A
Phone support Yes
Phone support availability 9 to 5 (UK time), Monday to Friday
Web chat support Yes, at an extra cost
Web chat support availability 9 to 5 (UK time), Monday to Friday
Web chat support accessibility standard WCAG 2.1 A
Web chat accessibility testing None
Onsite support Yes, at extra cost
Support levels Support for The Data Refinery is provided depending on the buyer’s requirements. The bundled support tier provides ticket resolution based on issues categorised by priority 1-4. This tier is available during business hours (09:00 – 17:00 Monday – Friday).

A dedicated account manager is available to the buyer for clarification and escalation of the support levels.
Support available to third parties Yes

Onboarding and offboarding

Onboarding and offboarding
Getting started The Data Refinery is supplied with a set of user documentation and training material which can either be downloaded or accessed through the website. In addition, other types of training can be supplied in order to meet the buyer’s requirements and at additional cost. This includes online interactive training courses or onsite workshops.
Service documentation No
End-of-contract data extraction The Data Refinery allows users to import and export data using the built-in tools. For customers on the paid subscription, and at any time through the life of the contract users can export the original data sets previously uploaded to the site, or in addition, export the enhanced version of data including the golden records. These exports are provided in the CSV format.
End-of-contract process Throughout the life of The Data Refinery subscription, users can access and use the tool, upload, deduplicate and enhance data sets and access enterprise level insights. After the end of the contract, users will be unable to access this insight or any of the data tools. However, they will be able to export the existing data and the previously created golden records prior to instance being deleted.

Using the service

Using the service
Web browser interface Yes
Supported browsers
  • Microsoft Edge
  • Firefox
  • Chrome
  • Safari 9+
Application to install No
Designed for use on mobile devices Yes
Differences between the mobile and desktop service Software is designed to be responsive and adapt to device screen size. A single service is consumed by both mobile and desktop devices, in some instances some processes or tasks are better suited to a desktop device.
Service interface No
What users can and can't do using the API Users can utilise the API to post and retrieve customer/citizen data into The Data Refinery.

In addition, the core matching API can be deployed as a REST API without the front end user interface to deliver a matching service between multiple systems.
API documentation Yes
API documentation formats Open API (also known as Swagger)
API sandbox or test environment Yes
Customisation available No


Independence of resources Where the availability of the product is prioritised by the buyer, The Data Refinery is provided within a single cloud instance. The service is monitored by our support team and where required can be scaled to meet user's demand.


Service usage metrics Yes
Metrics types The Data Refinery users can access service metrics via the internal audit feature within the product. This includes a list of user actions by type i.e. user login times, import and export of data files and create, read, update and delete of records.
Reporting types Regular reports


Supplier type Not a reseller

Staff security

Staff security
Staff security clearance Staff screening not performed
Government security clearance Up to Security Clearance (SC)

Asset protection

Asset protection
Knowledge of data storage and processing locations Yes
Data storage and processing locations European Economic Area (EEA)
User control over data storage and processing locations Yes
Datacentre security standards Managed by a third party
Penetration testing frequency At least once a year
Penetration testing approach ‘IT Health Check’ performed by a Tigerscheme qualified provider or a CREST-approved service provider
Protecting data at rest Other
Other data at rest protection approach Encryption of RDS databases by third-party host (AWS) using AES 256.
Data sanitisation process Yes
Data sanitisation type
  • Explicit overwriting of storage before reallocation
  • Deleted data can’t be directly accessed
Equipment disposal approach A third-party destruction service

Data importing and exporting

Data importing and exporting
Data export approach The Data Refinery allows users to import and export data using the built-in tools. This includes manual bulk upload from within the refinery and via an external API.
Data export formats
  • CSV
  • Other
Other data export formats
  • JSON
  • XML
Data import formats
  • CSV
  • Other
Other data import formats
  • JSON
  • XML

Data-in-transit protection

Data-in-transit protection
Data protection between buyer and supplier networks
  • TLS (version 1.2 or above)
  • Legacy SSL and TLS (under version 1.2)
Data protection within supplier network
  • TLS (version 1.2 or above)
  • IPsec or TLS VPN gateway
  • Legacy SSL and TLS (under version 1.2)

Availability and resilience

Availability and resilience
Guaranteed availability The Data Refinery is provided with an SLA of 99.5% availability.
Approach to resilience The Data Refinery has been deployed across multiple regions using the public cloud providers recommended approach for high availability within a deployment.

Additional resiliency information is available on request.
Outage reporting Unexpected service outages are reported on the Data Shed status page, which is independent hosted from The Data Refinery. For planned down time including maintenance windows and feature releases, users are contacted by email ahead of time.

In addition, status information is also reported by The Data Refinery external API.

Identity and authentication

Identity and authentication
User authentication needed Yes
User authentication Username or password
Access restrictions in management interfaces and support channels Access to the application is controlled by an individual user account directory based on RBAC (Role based access control). Access to management tiers for support and administration purposes is controlled through a combination of RBAC, Multi Factor Authentication (MFA) and a centralised user based directory.
Access restriction testing frequency At least every 6 months
Management access authentication 2-factor authentication

Audit information for users

Audit information for users
Access to user activity audit information Users have access to real-time audit information
How long user audit data is stored for User-defined
Access to supplier activity audit information Users contact the support team to get audit information
How long supplier audit data is stored for At least 12 months
How long system logs are stored for At least 12 months

Standards and certifications

Standards and certifications
ISO/IEC 27001 certification No
ISO 28000:2007 certification No
CSA STAR certification No
PCI certification No
Other security certifications No

Security governance

Security governance
Named board-level person responsible for service security Yes
Security governance certified No
Security governance approach The Data Shed takes security very seriously. Our Security Policy sets out the processes and policies we have employed to ensure The Data Refinery meets the buyer’s expectations regarding security. This includes our approach to network and server security, software and web application security, data retention, disaster recovery and organisational security. All engineers working on the product have been trained on and regularly review our policies.
Information security policies and processes All Data Shed employee's undergo regular security briefings and training. Employees use encrypted storage, encrypted tunnels (VPN and SSH), and encrypted communications for sensitive internal communications and operations where appropriate. We also maintain detailed application-level and system-level logs following our retention policy.

Operational security

Operational security
Configuration and change management standard Supplier-defined controls
Configuration and change management approach The Data Shed's Change Management Policy describes how changes are managed through the software development lifecycle and deployed into the live environment. This includes managing our source code and the branching strategy we follow.

All changes follow a defined path to live, and are subjected to various testing cycles prior to release. Changes are raised and documented by the team and reviewed and approved prior to deployment. Additional artefacts are created including release notes, runbooks and rollback plans.
Vulnerability management type Supplier-defined controls
Vulnerability management approach The Data Shed operate a vulnerability management policy which includes controls specifically designed to reduce the number of defects which allow our source code to be exploited.

In addition, we regularly run testing cycles including penetration testing by third parties to identify any potential threats. We also include a security research and disclosure policy which encourages the responsible disclosure of any vulnerabilities directly to us by other parties.

When threats are discovered, we assign team members to address them as a matter of priority.
Protective monitoring type Supplier-defined controls
Protective monitoring approach The Data Refinery is hosted utilising public cloud infrastructure, this includes a Web Application Firewall architecture which actively monitors inbound traffic to identify and alert the service desk for patterns of unusual traffic and volume. We also review audit access logs for each user session and component. Where incidents are identified they are raised in our service desk ticket management system and actively progressed against on a priority SLA's for time to respond and time to fix.
Incident management type Supplier-defined controls
Incident management approach The Data Refinery is supported by a dedicated team working on The Data Shed service desk. This team have defined incident management processes in place based on the ITIL v3 standard. We utilise an incident management system to record, respond and resolve all incidents and problems raised against The Data Refinery. These processes include both standard and non-standard requests along with SLA’s and targets for time to respond and time to fix. Users can raise new incidents and requests to the service desk via the support portal. Incident reports are generated and reviewed by our support managers regularly.

Secure development

Secure development
Approach to secure software development best practice Supplier-defined process

Public sector networks

Public sector networks
Connection to public sector networks No


Price £3000 to £40000 per instance per month
Discount for educational organisations Yes
Free trial available Yes
Description of free trial The free version of The Data Refinery allows an organisation to create an instance of the software as a service with the following limitations:
1 user only, maximum of 10,000 unique records, no extract functionality, no API, no development tools.
Link to free trial

Service documents

Return to top ↑