Home Office

DAC053 - Data Engineering Support

Incomplete applications

6
Incomplete applications
3 SME, 3 large

Completed applications

7
Completed applications
2 SME, 5 large
Important dates
Opportunity attribute name Opportunity attribute value
Published Tuesday 21 November 2017
Deadline for asking questions Tuesday 28 November 2017 at 11:59pm GMT
Closing date for applications Tuesday 5 December 2017 at 11:59pm GMT

Overview

Overview
Opportunity attribute name Opportunity attribute value
Summary of the work Capability to;
• Develop/automate routines for timely and accurate processing of data sets (databases, real time feeds, flat files and email)
• Transpose the data using ETL tooling into MPP database
• Form a business oriented POLE model for data matching
• Fix issues and bugs
• Re-platform and/or upgrade
Latest start date Monday 29 January 2018
Expected contract length 2 years
Location London
Organisation the work is for Home Office
Budget range

About the work

About the work
Opportunity attribute name Opportunity attribute value
Why the work is being done The Data Analytics Competency Centre has the capability to obtain, load and analyse data sets from Home Office, other Government departments and Commercial partners to provide aggregated, more accurate and timely data to users.

Current overall data volume is 21 Tb, from around 20 sources and is expected to double in size over the coming 12 months with data deltas and new sources.
Problem to be solved The current capability has reached a level of maturity, and project now needs to build the capability to :-
• Improve current loads and capability from existing data
• Build new capabilities for new data sets
• Migrate other existing HO services into the DACC
• Migrate the DACC capability from Beta to LIVE
• Review the technologies in use with a view to future proofing the capability and moving away from commercial products to more Open Source vendors.
• Knowledge transfer plan.
Who the users are and what they need to do Users are principally Data Analysts and Scientists within the Home Office.

Customers range across Home Office departments and output from the DACC are supplied to other parts of Government.
Early market engagement
Any work that’s already been done Work completed so far includes :-
• Choice of software component which are now under review for replacement
• Development of the current capability to ingest and process data sets to business requirements.
• Orchestration, Prioritisation and automation of data ingest.
• Development of a core set of skills, knowledge and capability within the Home Office and its contracting partners.
Existing team Existing team is composed of 2 Home Office employees and a flexible group supplied by a contracting partner.
Current phase Alpha

Work setup

Work setup
Opportunity attribute name Opportunity attribute value
Address where the work will take place Supplier will be based at the DACC Lab, Lunar House, Wellesley Road, Croydon.

Occasional travel to regional sites may be required, expenses incurred from travel outside of the M25/Greater London will be subject to Home Office Travel and Subsistence Policy.
Working arrangements The supplier is expected to stand up a team made of up of the specialist skills required to deliver the outcomes. We expect the supplier to be flexible to our needs and make working with a mixed team across multiple sites effective.

In general we would expect resources to be on site 5 days a week, although some out of hours support may be required from time to time.
Security clearance All staff must hold or be prepared to undergo SC Security Clearance to operate on this project, (with willingness for some resources to undergo NPPV3 and DV clearance)

Additional information

Additional information
Opportunity attribute name Opportunity attribute value
Additional terms and conditions • The work will be progressed by a number of SoWs which may be fixed priced, Capped T&M or T&M as appropriate.
• Standard DOS framework and call-off terms and conditions.

Skills and experience

Buyers will use the essential and nice-to-have skills and experience to help them evaluate suppliers’ technical competence.

Skills and experience
Opportunity attribute name Opportunity attribute value
Essential skills and experience
  • Comprehensive experience of ETL data processing
  • Comprehensive experience of using MPP database(s)
  • Expertise in the use of Structured Query Language (SQL)
  • Expertise in Orchestration, Prioritisation and automation (some home built capability)
  • Expertise in the development, troubleshooting and maintenance of Python code
  • Comprehensive capability to deal with structured and unstructured data. In many formats including (but not exclusively) JSON, XML, SQL Server, Oracle, Access, MySQL
  • Proven ability to provide support within a fast paced operational environment
  • Demonstrable experience of improving efficiency and optimising processing cycles
  • Demonstrable experience in loading and managing large volumes of data
  • Demonstrable experience of enabling data governance criteria for Data access, deletion/removal and general compliance.
Nice-to-have skills and experience
  • Capability to maintain and develop Visual Basic scripting
  • Capability to maintain and use ActivEon (Scheduler)
  • Expertise in the use of Postgres databases
  • Expertise in Pivotal GreenPlum MPP database
  • Experience in developing and maintaining ETL with SAS Data Integration
  • Experience of operating in a cloud hosted environment
  • Experience in loading streamed data sources and the related technologies.
  • Experience of operating in Public sector environment

How suppliers will be evaluated

How suppliers will be evaluated
Opportunity attribute name Opportunity attribute value
How many suppliers to evaluate 3
Proposal criteria
  • Approach and methodology
  • Technical skills and experience
  • Service model
  • Mobilisation plan
  • A plan for optimising resources
  • Identification of risks and dependencies
Cultural fit criteria
  • Ability to function effectively and collaborate in a multi-supplier environment
  • Working successfully in complex public sector organisations
  • Approach to proactive issue management, problem resolution and improving ways of working
Payment approach Time and materials
Assessment methods
  • Written proposal
  • Reference
  • Presentation
Evaluation weighting

Technical competence

65%

Cultural fit

5%

Price

30%

Questions asked by suppliers

Questions asked by suppliers
Supplier question Buyer answer
1. Is this role for one or two FTE onsite or a as and when needed remote support contract? I would expect this to be more than 1-2 FTE's permanently on site.
2. What is the number of users expected for the DACC? Currently just over 1000 users of DACC prepared information. Expected to rise to around 3000 during 2018.
3. How many people does the Home Office expect to require for support? We are looking for a team to provide a business as usual capability, further details will be provided at the second stage of this process, it is up to the bidder to say what they believe will be sufficient to support this capability.