HMRC

Data Platform Services - Data Acquisition, Preparation and Management

Incomplete applications

28
Incomplete applications
15 SME, 13 large

Completed applications

17
Completed applications
5 SME, 12 large
Important dates
Opportunity attribute name Opportunity attribute value
Published Monday 13 January 2020
Deadline for asking questions Monday 20 January 2020 at 11:59pm GMT
Closing date for applications Monday 27 January 2020 at 11:59pm GMT

Overview

Overview
Opportunity attribute name Opportunity attribute value
Summary of the work Impacting
Demand Management
Delivery Management
Business Requirements
Solution Design 
ETL Code Delivery
Virtualisation layer code Delivery/maintenance
Technical Metadata layer maintenance
Incident/problem management
Batch operation -support/recovery
Event Management –Provide support when required
HPI Management -Provide support when required
Change Management –Evaluate change within SLA’s
Code Change to support evergreening
Capacity/performance management
Latest start date Wednesday 1 April 2020
Expected contract length 2 Years
Location London
Organisation the work is for HMRC
Budget range £9.5m

About the work

About the work
Opportunity attribute name Opportunity attribute value
Why the work is being done Data Platform Services (DPS) provides enterprise-wide platform engineering capabilities that enable the acquisition, administration (management) and application (exploitation and consumption) of enterprise data.
DPS provides the core platform capabilities and building blocks needed by:
• CDIO Chief Data Office to implement enterprise data management (governance, models & standards), implement / manage data sharing and provide advanced analytics exploitation services.
• CDIO Customer Groups to assemble, build and run line of business data services.
• Data consumers (standard and advanced) within the wider HMRC Business to access and exploit data, within a supported self-serve framework.
DPS will also provide the support offerings that enable HMRC to become a data-driven and insight-led organisation, drive data innovation and drive adoption of emerging technologies like AI and ML.
Problem to be solved HMRC has a large and complex legacy Data Analytics estate which needs simplification, and consolidation and transformation to meet the challenges the department now faces.
We design and build services to provide environments on which to provision, prepare and transform data and provide services to exploit that data across the organisation, and other government departments.
Who the users are and what they need to do All HMRC customer groups (Customer Compliance, Customer Services, Customer Strategy and Tax Design, Corporate, VOA) need platform services they can consume for data on which they can build their specific line of business exploitation services.

HMRC Chief Data Office needs platform services which enable Data Management, Governance and Data sharing.

Other Government Departments rely on the Platform for receiving data for use within HMRC, and sharing HMRC data for their use.
Early market engagement Supplier Day held in Dec 19 for Data Platform Services covering 5 Separate DOS requirements including this one.
Suppliers may request a copy of the presentation slides by emailing ben.hopkinson@hmrc.gov.uk, email requests MUST include 'Supplier Day Slides' or 'Data Platform Services Clarifications' in the email title
Any work that’s already been done DPS has been working since 2018 on a new strategy for provision of Data and Analytics services based around a hybrid hosting model with a Logical Data Warehouse at its core. There is a significant amount of work remaining to continue delivery of that evolving platform roadmap.
Existing team HMRC and other suppliers
Typically teams comprise of the following roles:
Technical Project Manager
Scrum Master
Solution Architect
Product Owner
ETL/ELT Developers
Data Modellers
ETL Administrators
Data Fabric Engineer
Data Fabric Administrator
Metadata Administrator
Data Virtualisation Administrator
Data Analysts
Business Analysts
QA
Service Manager
Current phase Not applicable

Work setup

Work setup
Opportunity attribute name Opportunity attribute value
Address where the work will take place 3 Primary Locations:

Trinity Bridge House
2 Dearmans Place
Salford, Manchester
UK
M3 5BS

Alexander House
21 Victoria Avenue
Southend on Sea
UK
SS99 1AA

14 Westfield Avenue ,
Stratford
London
UK
E20 1HZ
Working arrangements The supplier will be required to co-locate with the existing platform teams, mixed teams comprising both internal staff and other contractors, 5 days per week.

Travel will be required to other HMRC Delivery Centres. No additional expenses chargeable to HMRC for travel or accommodation in agreed base office locations per SOW.

We expect the successful supplier to provide upskilling to permanent staff to increase internal capability.
Security clearance Must have BPSS before employment commences and resources come on site
Must be SC eligible. SC process can start after employment commences.
Some staff must be DV eligible.
Clearance costs to be met by supplier

Additional information

Additional information
Opportunity attribute name Opportunity attribute value
Additional terms and conditions HMRC Mandatory Terms to be added

Skills and experience

Buyers will use the essential and nice-to-have skills and experience to help them evaluate suppliers’ technical competence.

Skills and experience
Opportunity attribute name Opportunity attribute value
Essential skills and experience
  • Experience working with and leveraging the benefits of AWS and Microsoft Azure Cloud hosted services
  • Experience working with range of ETL and ELT tools to design, create, deploy and operate enterprise services focussing on re-use, consistency and performance. Must include Pentaho Data Integrator and Talend.
  • Experience defining and implementing Data Quality rules at enterprise scale using SAS Data Flux / Data Mmanagement Studio.
  • Experience Data Modelling and creation of physical models conformant with Enterprise Data Model and organisation standards using SAP Powerdesigner.
  • Capability for Java and Javascript development.
  • Creating performant and resilient ETL/ELT jobs for a large enterprise, including use of the apache Spark engine with Scala.
  • Supporting 24x7 Data Acquisition and Data Preparation services.
  • Experience Managing and Administering relational, graph, in memory and NOSQL database including Oracle, MYSQL, Redshift, NEO4j and Hbase.
  • Test driven development with Business Intelligence services including use of Continuous Integration environments
  • Automated and Performance testing of ETL / ELT services
  • Experience using and administering Metadata and Information Management solutions.
  • Experience using and administering Data Virtualisation solutions including Denodo.
  • Experience Managing and Administering ETL services including Pentaho and Talend.
  • Experience Managing and Administering Data Quality services with Data Management Studio.
  • Designing and building solutions fully conformant with WCAG 2.0.
  • Experience of customer facing issue and incident resolution.
  • Producing and maintaining technical documentation and a knowledge base.
  • Providing accurate and timely information to maintain a CMDB.
  • Working safely within our UK based secure engineering environments at SC level.
  • Experience with Batch, Message and Streaming data Acquisition and associated acquisition architectures such as lamda.
Nice-to-have skills and experience
  • Experience with Red Hat Linux 6 and later
  • Experience working with Databases containing 100’s TB’s of data and object / Hadoop storage containing PB’s of data.
  • Experience working with public sector organisations

How suppliers will be evaluated

All suppliers will be asked to provide a written proposal.

How suppliers will be evaluated
Opportunity attribute name Opportunity attribute value
How many suppliers to evaluate 4
Proposal criteria
  • Shortlisted suppliers will be asked to provide Case-Study, Written Proposal and Presentation. These will be scored using a selection of Essential Criteria outlined above to be provided with Proposal Scenario.
  • Below outlines the expected ways of working to be evidenced:
  • Demonstrate how you as a supplier have been fully accountable for delivery of a project or outcome and successfully delivered to specification, time and with value for money
  • Demonstrate how you as a supplier have been fully accountable for operating/running a service to agreed service levels and standards
  • Demonstrate as a supplier how you have taken remedial action where outcomes have not been to agreed standards
  • Agility - delivering iteratively and incrementally within a context of evolving requirements following SCRUM and Kanban approaches
  • Making knowledge capture and sharing part of day to day activities
  • Ensuring successful handover at any required stage of a project or a service
  • Delivering business value – making sure business requirements are understood, validated via positive challenge and delivered against while understanding the value being generated (i.e. business intent)
  • Innovative thinking – think outside the set of existing organisational constraints to arrive the most appropriate and efficient route to deliver desired outcomes
  • MVPs – effective definition and delivery of true MVPs that shorten time to delivering business value
  • Reusability by design – ensuring solutions are formulated from reusable capabilities as well as ensuring new capabilities are reusable
  • Simplicity by design – a clear focus on producing and positively challenging solutions to be as simple as possible
  • Design for minimising build and run costs
  • Responsive mobilisation – ability to resource work and start delivering in 4-5 weeks
  • Enough capacity to ensure supplier can cope with DPS demand, in the specified locations
  • Enough depth and breadth of capability across the supplier resource pool, in the specified locations
  • Supplier teams are expected to use HMRC's collaboration software like Jira, Confluence and Teams
  • Experience working with SAP Business Objects Data Services hosted on Solaris. Experience working with Informatica Powercentre
  • Experience Managing and Administering Azure SQL warehouse, Azure Data Lake & DB2 database
Cultural fit criteria
  • At DPS we want to grow our in-house skills: How would you support this? What skills, time and expertise would you make available and where?
  • Evidence how you would engage and communicate with DPS: Senior Leadership Team and our Extended Leadership Team who will be running the projects
  • Evidence how you would engage and communicate with DPS: Other supplier colleagues who may be part of the same project/programme of works
  • Evidence how you would engage and communicate with DPS: Our Business Ops team who run support functions including Resourcing & financials including billing
  • Active involvement in promoting and communicating the work and successes of DPS
  • Evidence how you would deliver in a multi-supplier environment: Ensuring a productive work approach with all involved parties, employing a “badge at the door” mind-set
  • Evidence how you would deliver in a multi-supplier environment: Ensuring HMRC’s interest is reflected in the approach to work and outputs delivered, demonstrating openness and transparency in engagement
  • Demonstrate how you as a supplier have successfully adhered to a client’s code of practice or adopted their organisational behaviours to ensure cultural fit; especially HMRC's
  • Demonstrate how you will ensure inclusivity and collaboration
Payment approach Fixed price
Additional assessment methods
  • Case study
  • Presentation
Evaluation weighting

Technical competence

50%

Cultural fit

20%

Price

30%

Questions asked by suppliers

Questions asked by suppliers
Supplier question Buyer answer
1. You have listed three office locations and expressed the necessity to co-locate our personnel with yours at these locations.
Is it a requirement to co-locate at all three offices simultaneously, or is it sufficient to embed a team at one location (London, for eg) and collaborate remotely with your teams at the other locations?
If the latter, do you envisage that the time of our personnel will be split evenly between the locations or will you require a permanent presence at each one?
Flexibility to resource in all 3 locations is needed based on organisational demand and proximity to other teams/business.
2. Could you please confirm if any of the existing team (HMRC or other suppliers) could be in scope for TUPE? Could you please confirm if any of the existing team (HMRC or other suppliers) could be in scope for TUPE?
3. Under additional information, additional terms and conditions, the notice states "HMRC Mandatory Terms to be added". Can you confirm what these terms are please? In relation to tax compliance, invoicing through HMRC systems, data protection
4. One of the essential skills and experience criteria is “Working safely within our UK based secure engineering environments at SC level”.
Can you clarify this please – does it mean that suppliers must have a track record of working in HMRCs secure environment, or only that they must be able to do so?
Suppliers should understand what it means for them and how to operate teams / resources within a secure environment. Optionally experience of doing so in the past would obviously demonstrate how this has been achieved previously.
5. Could the authority please:
1. share the draft copy of the 3 additional T&C topics (tax compliance, invoicing through HMRC systems, data protection)
2. confirm that these are the only additional T&Cs beyond the standard DOS4 terms.
Unfortunately attachments can't be added. For a draft copy of the additional terms please contact arron.egan@hmrc.gov.uk
6. With regards to the scoring of this opportunity. At Stage 2 suppliers will be asked to provide a Case-Study, Written Proposal and Presentation. Bullet point 1 of the Proposal criteria states that “These will be scored using a selection of the Essential Criteria outline above to be provided with the Proposal Scenario”. Can you please confirm that the Essential Criteria being assessed at Stage 2, together with any other criteria that affects scoring, will be specifically and clearly stated to those suppliers who are shortlisted at Stage 1? I can confirm Essential Criteria will be assessed at Stage 2. As the Proposal Scenario may not require all Essential Criteria to be evidenced in this opportunity, the specific criteria being used and scoring methods will be outlined to shortlisted suppliers.

The Cultural Fit of suppliers will be scored separately. All of the criteria outlined for Cultural Fit will be considered at Stage 2.
7. Is there any significance to the mention of the following technologies in the proposal criteria, given that these skills are not mentioned in the Skills and Experience section ?
• Experience working with SAP Business Objects Data Services hosted on Solaris.
• Experience working with Informatica Powercentre
• Experience Managing and Administering Azure SQL warehouse, Azure Data Lake & DB2 database"
Skills in these areas are expected to be necessary in the duration of this contract. Whilst not assessed at Stage 1 they may be an element of the Proposal Scenario at Stage 2
8. Does the Data Acquisition POD also include Data Exports e.g. European exchange of information? Yes
9. How much are we building the required solution to the HMRC design? HMRC specifies the technology and has the design approval gate.
10. How much do we need to define the architecture? Within the approved HMRC technology landscape supplier would be expected to undertake solution design.
11. How mature is the HMRC meta data system? HMRC is undertaking a separate commercial procurement of its metadata software, due for completion within the next 40 days.
12. Is the requirement migration from or expansion of legacy technology components? Primarily migration from legacy technology although some work may necessitate changes to existing legacy technology.
13. What are the primary technologies in the “to be” solution? Terraform
Jenkins
Ansible
PowerDesigner
RHEL
ELK
Java
Pentaho
Talend
Denodo
Cloudera
MYSQL
Oracle
Redshift
SQL Data Warehouse
KAFKA
SAS
RStudio
RConnect
Sagemaker
MS Cognitive
14. Is HMRC planning to use Azure to host all environments, including Production? AWS and Azure for services in Public Cloud
15. Does HMRC security policies permit PII data to be hosted in the cloud (such as Azure)? Yes given specified controls
16. If security policies do not permit PII data to be hosted in the Cloud then how will be code and/or data to be promoted across the environments? Not Applicable
17. Could you please confirm if any of the existing team (HMRC or other suppliers) could be in scope for TUPE? N/A