Hackney Council

Enhancing Hackney’s Data Platform so that it is so good analysts want to use it

6 Incomplete applications

6 SME, 0 large

0 Completed applications

Important dates

Published
Thursday 25 November 2021
Deadline for asking questions
Thursday 2 December 2021 at 11:59pm GMT
Closing date for applications
Thursday 9 December 2021 at 11:59pm GMT

Overview

Off-payroll (IR35) determination
Contracted out service: the off-payroll rules do not apply
Summary of the work
Hackney is developing a cloud-based data platform to enable deeper insight, break down data silos, put security at the heart of our approach, and promote stability through reusable, scalable processes. We need to enhance the platform and increase adoption from analysts across the council.
Latest start date
Monday 10 January 2022
Expected contract length
6 months
Location
London
Organisation the work is for
Hackney Council
Budget range
We expect to spend up to £293,000 + VAT on this phase of work. We will collaboratively agree smaller phases through Statements of Work to ensure we continue to meet user needs.

About the work

Why the work is being done
Hackney manages personally identifiable and sensitive data about its residents. This rich range of data can generate valuable insights about our people, their needs and the impact of our services. In order to derive insight from our data, we are building a cloud data platform to:

- Put security at the heart of our approach: Data must be stored, transferred, and accessed securely at every stage so that residents are assured we are handling their data responsibly and we meet our legal obligations. We need joined up information governance processes so that users can easily request and grant access to data.

- Break down technology and service data silos: We need to enable officers across the council to understand what data is available; minimise and anonymise data so that it can be shared more easily; and enable analysts to use data in a range of analytical tools that meet their needs.

- Embed reusable, scalable processes: Dependable, reusable, automated processes will save time and allow analysts to focus on higher value tasks. Analysts also need to see, learn from, and reuse each other’s work to reduce duplication and generate meaningful insights.
Problem to be solved
We have been developing our data platform since March 2021 and put key processes in place to ingest, transform, and publish data. We now need increase adoption so that analysts can use the platform in their day-to-day work through a holistic approach that considers:

- Skills & Capabilities: the platform will introduce new tools and ways of working for analysts, so they need access to learning resources and support in order to use the platform effectively.

- Culture & Mindset: previously many services relied on ICT to get access to their own data. Our approach gives analysts more visibility and ownership of their data pipelines.

- Products & Services: we need to ensure that our playbook enables users to share and reuse each other’s work, and our catalogue enables users to find and understand data.

- Technology & Data: we need to ensure that we have the right tools and processes in place so that users can ingest, transform and analyse data.

- Systems & Processes: we need to develop our approach to onboarding users and supporting them to use the platform. We also need to join up the platform with our information governance processes.
Who the users are and what they need to do
At a very high level, the user needs for our data platform are:

- As a resident, I need my data to be managed securely and responsibly so that my personal information is secure and I can trust the council to deliver a fair and transparent service to meet my needs.

- As a service manager, I need timely insights about the effectiveness of my service so that I can take action and improve outcomes for residents.

- As a data analyst, I need to access, manipulate and analyse a range of data sources through the data platform so that I can generate meaningful insights more quickly and robustly.

- As an engineer, I need to create reusable tools and processes in the data platform which require minimal maintenance so that I can focus on developing and improving the platform.

Further detail about the profile of our users, more detailed user needs, and prioritisation is set out in this document (available on request): https://docs.google.com/presentation/d/1Bp9hUPKnMnNQDM8I2S-Jh2kdwF4xx2zvKyf-GLn61ZM/edit?usp=sharing
Early market engagement
Any work that’s already been done
The first phase of developing our data platform was done in partnership with MadeTech. The following documents provide a summary of the work to date:

- A summary slidedeck of our objectives, work to date, and current position (available on request): https://docs.google.com/presentation/d/1s1vJcUMZOYsSSuYc7B_c2O7AzJwps487dePGQjXh4Tw/edit?usp=sharing

- Our project weeknotes (available on request): https://docs.google.com/document/d/1ppU4xCmpShHYDDDPHjywzTmnJq-sfjgH4Iewi6dfQJw/edit?usp=sharing

- The Data Platform Playbook which provides documentation of our existing processes and architectural decisions: https://playbook.hackney.gov.uk/Data-Platform-Playbook/
Existing team
Hackney would expect to provide:
- A product manager
- 2-4 data analysts on a part time basis
- A junior developer
- A delivery manager
- A data engineer
- User research capability

As part of this work we are aiming to recruit 3 in-house developers. We also have an emerging community of practice of around 50 analysts across the council.

The successful agency will bring particular value through:
- Experienced developers with experience of the toolset
- Experienced data engineering
- Experienced data architect who can advise on the design of the platform
Current phase
Beta

Work setup

Address where the work will take place
Remotely, and in Hackney Council offices in Mare Street, E8 where face to face contact is required
Working arrangements
This work can be done remotely but will require fortnightly reporting (e.g. via show & tells) to involve key stakeholders and regular engagement with practitioners through video conferences. The current team have also found some face-to-face working beneficial. We will provide access to key systems to project team members where required.
Security clearance

Additional information

Additional terms and conditions

Skills and experience

Buyers will use the essential and nice-to-have skills and experience to help them evaluate suppliers’ technical competence.

Essential skills and experience
  • Demonstrate an understanding of some of the key services, datasets and analytical capacity within a local authority
  • Demonstrate a track record of user-centred design and Agile approaches to delivery
  • Demonstrate a track record of mentoring junior developers and teams to become self-sufficient
  • Experience of solution architecture within a broader organisational framework to achieve security, resilience and value for money
  • Experience of using AWS S3, Glue, and Redshift to deliver data pipelines
  • Experience of using Python and Spark to deliver data pipelines
  • Experience of using Terraform to deliver infrastructure as code
  • Experience of developing ‘in the open’ - using tools such as Github
  • Experience of content design to enable users to perform technical tasks
Nice-to-have skills and experience
  • Demonstrate an understanding of Hackney’s approach to technology and data as set out on https://playbook.hackney.gov.uk
  • Demonstrate how they provide opportunities for apprentices
  • Experience of business intelligence tools - particularly Qlik
  • Knowledge or experience of machine learning

How suppliers will be evaluated

All suppliers will be asked to provide a written proposal.

How many suppliers to evaluate
5
Proposal criteria
  • Understanding of the brief
  • Clarity of the approach
  • Experience from a similar project
  • Team structure, including skills, experiences and relevance of individuals
  • Identification of risks and plan to mitigate them
  • Value for money
Cultural fit criteria
  • Work as a team with our organisation and other suppliers
  • Be transparent and collaborative when making decisions
  • Share knowledge and experience with team members and the wider service
  • Hackney Council is a diverse organisation, reflecting the diversity of our borough. Describe how you would successfully work as a team with Hackney and other suppliers in a diverse environment.
Payment approach
Time and materials
Additional assessment methods
Presentation
Evaluation weighting

Technical competence

60%

Cultural fit

5%

Price

35%

Question and answer session

Log in to view question and answer session details

Questions asked by suppliers

1. If successfully shortlisted, what will be the deadline for submitting a written proposal?
We expect the deadline for written proposals to be 9am on Monday, 20 December.

Log in to ask a question