The Data Refinery (Data Integration Platform)
The DataRefinery is a data integration platform which, at its heart, is a single view of a citizen/customer. Created by pulling together data sources, matching records and enhancing data using behaviour, demographics and interactions to build a single source of truth - providing instant insight and empowering informed business decisions.
- Our industry leading algorithms enhance and de-duplicate your data
- Standardise business definitions through intelligently plotting your data
- Combine data from different sources into a single citizen view
- Our dashboards allow a real-time view of processed citizen data
- Enhancements enrich your data to provide further insight
- Features to help with GDPR-compliance and overall data quality
- Programmatic access via web based API
- Simple, intuitive interface allows users to access insight quickly
- Creates golden records, the combined best data for a citizen
- Access data analysis without the need for a Data Scientist
- Understand your long-term citizen value and behaviour patterns
- Segment citizen/customer base for more effective targeting
- Use characteristics of citizen profiles to target activity
- Summarise numerous disparate citizen services into a single view
- Search for and view individual citizens and their interactions
- Valuable insight without additional investment in IT
- Understand the citizens you serve as individuals
|Software add-on or extension||No|
|Cloud deployment model||
The Data Refinery can be deployed in a public cloud, allowing the greatest possible flexibility to meet the buyer’s requirements. The product will be associated with a set of SLA’s and NFR’s for availability.
As a SaaS, The Data Refinery can be provisioned across a wide range of hardware and software configurations including all modern internet browsers.
|System requirements||The product is not associated with any specific system requirements|
|Email or online ticketing support||Email or online ticketing|
|Support response times||Service Desk operates between 08:00 & 17:30 during on weekdays (excluding bank holidays in England). On Call support is provided 24/7.|
|User can manage status and priority of support tickets||Yes|
|Online ticketing support accessibility||WCAG 2.1 A|
|Phone support availability||9 to 5 (UK time), Monday to Friday|
|Web chat support||Yes, at an extra cost|
|Web chat support availability||9 to 5 (UK time), Monday to Friday|
|Web chat support accessibility standard||WCAG 2.1 A|
|Web chat accessibility testing||None|
|Onsite support||Yes, at extra cost|
Support for The Data Refinery is provided depending on the buyer’s requirements. The bundled support tier provides ticket resolution based on issues categorised by priority 1-4. This tier is available during business hours (09:00 – 17:00 Monday – Friday).
A dedicated account manager is available to the buyer for clarification and escalation of the support levels.
|Support available to third parties||Yes|
Onboarding and offboarding
|Getting started||The Data Refinery is supplied with a set of user documentation and training material which can either be downloaded or accessed through the website. In addition, other types of training can be supplied in order to meet the buyer’s requirements and at additional cost. This includes online interactive training courses or onsite workshops.|
|End-of-contract data extraction||The Data Refinery allows users to import and export data using the built-in tools. For customers on the paid subscription, and at any time through the life of the contract users can export the original data sets previously uploaded to the site, or in addition, export the enhanced version of data including the golden records. These exports are provided in the CSV format.|
|End-of-contract process||Throughout the life of The Data Refinery subscription, users can access and use the tool, upload, deduplicate and enhance data sets and access enterprise level insights. After the end of the contract, users will be unable to access this insight or any of the data tools. However, they will be able to export the existing data and the previously created golden records prior to instance being deleted.|
Using the service
|Web browser interface||Yes|
|Application to install||No|
|Designed for use on mobile devices||Yes|
|Differences between the mobile and desktop service||Software is designed to be responsive and adapt to device screen size. A single service is consumed by both mobile and desktop devices, in some instances some processes or tasks are better suited to a desktop device.|
|What users can and can't do using the API||
Users can utilise the API to post and retrieve customer/citizen data into The Data Refinery.
In addition, the core matching API can be deployed as a REST API without the front end user interface to deliver a matching service between multiple systems.
|API documentation formats||Open API (also known as Swagger)|
|API sandbox or test environment||Yes|
|Independence of resources||Where the availability of the product is prioritised by the buyer, The Data Refinery is provided within a single cloud instance. The service is monitored by our support team and where required can be scaled to meet user's demand.|
|Service usage metrics||Yes|
|Metrics types||The Data Refinery users can access service metrics via the internal audit feature within the product. This includes a list of user actions by type i.e. user login times, import and export of data files and create, read, update and delete of records.|
|Reporting types||Regular reports|
|Supplier type||Not a reseller|
|Staff security clearance||Staff screening not performed|
|Government security clearance||Up to Security Clearance (SC)|
|Knowledge of data storage and processing locations||Yes|
|Data storage and processing locations||European Economic Area (EEA)|
|User control over data storage and processing locations||Yes|
|Datacentre security standards||Managed by a third party|
|Penetration testing frequency||At least once a year|
|Penetration testing approach||‘IT Health Check’ performed by a Tigerscheme qualified provider or a CREST-approved service provider|
|Protecting data at rest||Other|
|Other data at rest protection approach||Encryption of RDS databases by third-party host (AWS) using AES 256.|
|Data sanitisation process||Yes|
|Data sanitisation type||
|Equipment disposal approach||A third-party destruction service|
Data importing and exporting
|Data export approach||The Data Refinery allows users to import and export data using the built-in tools. This includes manual bulk upload from within the refinery and via an external API.|
|Data export formats||
|Other data export formats||
|Data import formats||
|Other data import formats||
|Data protection between buyer and supplier networks||
|Data protection within supplier network||
Availability and resilience
|Guaranteed availability||The Data Refinery is provided with an SLA of 99.5% availability.|
|Approach to resilience||
The Data Refinery has been deployed across multiple regions using the public cloud providers recommended approach for high availability within a deployment.
Additional resiliency information is available on request.
Unexpected service outages are reported on the Data Shed status page, which is independent hosted from The Data Refinery. For planned down time including maintenance windows and feature releases, users are contacted by email ahead of time.
In addition, status information is also reported by The Data Refinery external API.
Identity and authentication
|User authentication needed||Yes|
|User authentication||Username or password|
|Access restrictions in management interfaces and support channels||Access to the application is controlled by an individual user account directory based on RBAC (Role based access control). Access to management tiers for support and administration purposes is controlled through a combination of RBAC, Multi Factor Authentication (MFA) and a centralised user based directory.|
|Access restriction testing frequency||At least every 6 months|
|Management access authentication||2-factor authentication|
Audit information for users
|Access to user activity audit information||Users have access to real-time audit information|
|How long user audit data is stored for||User-defined|
|Access to supplier activity audit information||Users contact the support team to get audit information|
|How long supplier audit data is stored for||At least 12 months|
|How long system logs are stored for||At least 12 months|
Standards and certifications
|ISO/IEC 27001 certification||No|
|ISO 28000:2007 certification||No|
|CSA STAR certification||No|
|Other security certifications||No|
|Named board-level person responsible for service security||Yes|
|Security governance certified||No|
|Security governance approach||The Data Shed takes security very seriously. Our Security Policy sets out the processes and policies we have employed to ensure The Data Refinery meets the buyer’s expectations regarding security. This includes our approach to network and server security, software and web application security, data retention, disaster recovery and organisational security. All engineers working on the product have been trained on and regularly review our policies.|
|Information security policies and processes||All Data Shed employee's undergo regular security briefings and training. Employees use encrypted storage, encrypted tunnels (VPN and SSH), and encrypted communications for sensitive internal communications and operations where appropriate. We also maintain detailed application-level and system-level logs following our retention policy.|
|Configuration and change management standard||Supplier-defined controls|
|Configuration and change management approach||
The Data Shed's Change Management Policy describes how changes are managed through the software development lifecycle and deployed into the live environment. This includes managing our source code and the branching strategy we follow.
All changes follow a defined path to live, and are subjected to various testing cycles prior to release. Changes are raised and documented by the team and reviewed and approved prior to deployment. Additional artefacts are created including release notes, runbooks and rollback plans.
|Vulnerability management type||Supplier-defined controls|
|Vulnerability management approach||
The Data Shed operate a vulnerability management policy which includes controls specifically designed to reduce the number of defects which allow our source code to be exploited.
In addition, we regularly run testing cycles including penetration testing by third parties to identify any potential threats. We also include a security research and disclosure policy which encourages the responsible disclosure of any vulnerabilities directly to us by other parties.
When threats are discovered, we assign team members to address them as a matter of priority.
|Protective monitoring type||Supplier-defined controls|
|Protective monitoring approach||The Data Refinery is hosted utilising public cloud infrastructure, this includes a Web Application Firewall architecture which actively monitors inbound traffic to identify and alert the service desk for patterns of unusual traffic and volume. We also review audit access logs for each user session and component. Where incidents are identified they are raised in our service desk ticket management system and actively progressed against on a priority SLA's for time to respond and time to fix.|
|Incident management type||Supplier-defined controls|
|Incident management approach||The Data Refinery is supported by a dedicated team working on The Data Shed service desk. This team have defined incident management processes in place based on the ITIL v3 standard. We utilise an incident management system to record, respond and resolve all incidents and problems raised against The Data Refinery. These processes include both standard and non-standard requests along with SLA’s and targets for time to respond and time to fix. Users can raise new incidents and requests to the service desk via the support portal. Incident reports are generated and reviewed by our support managers regularly.|
|Approach to secure software development best practice||Supplier-defined process|
Public sector networks
|Connection to public sector networks||No|
|Price||£3000 to £40000 per instance per month|
|Discount for educational organisations||Yes|
|Free trial available||Yes|
|Description of free trial||
The free version of The Data Refinery allows an organisation to create an instance of the software as a service with the following limitations:
1 user only, maximum of 10,000 unique records, no extract functionality, no API, no development tools.
|Link to free trial||https://www.thedatashed.co.uk/|