Department for Business, Energy and Industrial Strategy (BEIS)

BEIS Data Management Specialists

18 Incomplete applications

14 SME, 4 large

21 Completed applications

9 SME, 12 large

Important dates

Published
Wednesday 14 September 2022
Deadline for asking questions
Wednesday 21 September 2022 at 11:59pm GMT
Closing date for applications
Wednesday 28 September 2022 at 11:59pm GMT

Overview

Off-payroll (IR35) determination
Contracted out service: the off-payroll rules do not apply
Summary of the work
We are looking for a supplier to work alongside the Data Team in BEIS Digital to help deliver a wide range of digital and data outcomes. Primarily focused on data projects, the team is responsible for maintaining and developing the Azure Data Management System.
Latest start date
Tuesday 3 January 2023
Expected contract length
2 years
Location
London
Organisation the work is for
Department for Business, Energy and Industrial Strategy (BEIS)
Budget range
The budget for this service is up to £4m which breaks down into two parts:
1) Maintenance and support of the DMS platform i.e. to help with regular updates and changes needed to keep live services running with a budget of up to £1m.
2) The delivery of data projects has a variable demand. The amount of demand will depend upon policy teams across the BEIS group. Based on current demands, we expect this to amount up to £3m but, there is no commitment to spend any money on this if there is no demand from colleagues.

About the work

Why the work is being done
The Data Management System (DMS) that has been developed by BEIS Digital is now being extensively used to manage the end-to-end processing of operational data which is being used to inform policy decisions and monitor the impact those services are having on achieving key departmental objectives.
The demand for better quality data in the business has grown and we are now need to provide BEIS with robust data from a wider range of sources, much faster and far more efficiently.
The demand for data solutions is generally led by policy teams across BEIS which can be unpredictable and fast paced. The data team needs to respond to these demands quickly, standing up project teams to deliver solutions using existing components of the DMS or building new ones to deliver individual project outcomes.
This tender is to bring in a supplier that can support the data team in the delivery of a wide variety of data projects by supplying technical support and resources to meet demand over the next 2 years. The type of resource will vary depending on the project, the supplier will need to be capable of supplying high quality data specialists and other resources on a flexible basis.
Problem to be solved
Due to rising demand, the Data Team in BEIS Digital urgently need the support of an external supplier to help with the build of new data solutions on our Azure platform. This could involve supplying us with developer and data engineering resource as well as consultancy on the solutions. We are planning to build the capability of the team at the same time as delivering projects so, knowledge transfer and up skilling of the team is an essential part of the contract.
Over the last 18 months, the team has had to take on several data projects including the processing of data on Covid grants by the local authorities, the take up and utilisation of Green Homes Grants across the country, the transfer of financial data from our shared service provider; UKSBS to BEIS for analysis and pulling in data from various external providers in the private sector. We have created a Data Management system on Azure to deliver these projects and we are keen to continue to extend the capability of that system to deliver more projects in-house. The current system will need to be re-engineered over the next 18 months to scale up and simplify the architecture.
Who the users are and what they need to do
Grant Administrators need to provide BEIS with regular updates on how grants are being distributed to eligible customers. The requirement is for both qualitative and quantitative data on a regular cycle and to a required standard.
Policy Officials are responsible for the scheme, grant or other support measures, to monitor the take up and participation rates, monitor the impact those measures are having on the desired outcomes and provide accurate information to Ministers, auditors and other interested parties. The data is used to inform policy decisions in the future to deliver better outcomes.
Data Analysts need to access reliable, accurate data in as near real time as possible to be able to experiment with data in the current data management system using python, R and other analytical tools to provide policy teams with new insight into how effective their measures are at delivering BEIS objectives and informing the design of future support schemes.
Ministers and Senior Management need regular updates on how support measures are being delivered and their effectiveness. Reliable, good quality data is essential to run the department effectively and make the best use of resources available.
Early market engagement
Any work that’s already been done
The Data Management System has been in place for 18 months now and we have already delivered several successful projects for the department. As a result, demand for new services has increased. Every project goes through a Discovery phase to determine user needs and provide us with a clear set of requirements for the build of data solutions. We have a growing library of code and components of infrastructure that can be reused to deliver new projects and a team of permanent staff that are gaining skills and experience every day.
Existing team
The successful bidder will be working in the Data Team at BEIS Digital. The Data Team has three areas of responsibility, Data infrastructure (the DMS team), Business Insights and Data Policy.
There is a small team:
• The Senior Responsible Officer for the project
• A Lead Data Scientist
• 2 Data Engineers
Because the DMS is part of the wider BEIS IT infrastructure, there are additional support services involved in the delivery and management of the Data Management system. These are the Project Management Office, Departmental Security Unit, IT Platform Team and an external end user compute supplier.
Current phase
Not applicable

Work setup

Address where the work will take place
BEIS Office, 1 Victoria Street, London SW1H 0ET
Working arrangements
This project is currently based in 1 Victoria St, London.

However, since the Covid outbreak, BEIS has adopted a hybrid working model with staff coming into the office on a part time basis, usually 40-60% of the working week. We would therefore expect project teams to visit our office on a regular basis for stand ups, meetings, show and tells etc. Alternatively the project teams and supporting workers could visit your offices for these meetings. We use Microsoft Teams for virtual meetings and expect to conduct the majority of meetings on a hybrid basis i.e. both in person and virtually.
Security clearance
Data Engineers and Developers and others working on the DMS platform must have SC clearance.
All other staff must have a least basic security clearance.

Additional information

Additional terms and conditions
Subject to any pre-existing rights of third parties and of the Contractor, the Intellectual Property Rights (other than copyright) in all reports, documents and other materials which are generated or acquired by the Contractor (or any of its sub-contractors or agents) (“the Contractor Materials”) in the performance of the Services shall belong to and be vested automatically in the Department for Business, Energy and Industrial Strategy.

Skills and experience

Buyers will use the essential and nice-to-have skills and experience to help them evaluate suppliers’ technical competence.

Essential skills and experience
  • Have experience of working on the Microsoft Azure platform, be able to configure and deploy a solution on to that platform as infrastructure as code using a CI/CD process.
  • Have experience of building reproducible analytical pipelines i.e. transforming data using pythons, R scripts, data lakes and visualise the data using R Shiny, Power BI or via APIs
  • Have a working knowledge of Government digital standards, governance processes and ways of working
  • Have a detailed understanding of Digital, cyber security standards
  • Have a proven track record of delivering successful data projects
  • be able to develop knowledge sharing topic guides, scripts and materials for facilitating workshops deferring final decisions to the BRE team
  • Be able to supply digital and data specialists to work on projects including Data Engineers, Data Scientists, Data Architects, Full Stack Developers, Project and programme support staff
Nice-to-have skills and experience
  • be able to work alongside a diverse range of users and other groups such as SMEs, academics, industry regulatory experts, compliance officers and senior officials
  • Have a proven record of delivering successful data management projects in the public sector
  • be able to share knowledge with the existing team to facilitate mentoring of the existing team

How suppliers will be evaluated

All suppliers will be asked to provide a written proposal.

How many suppliers to evaluate
5
Proposal criteria
  • Understanding of the Microsoft Azure environment
  • Experience of building and deploying data pipelines and analytical platforms as IaaC using Terraform or similar products
  • Approach and methodology
  • How you plan to work with us to deliver projects and achieve outcomes
  • How you would identify risks and dependencies and offered approaches to manage them
  • Proposed team structure and staff experience
  • Approach for working with existing staff
  • How you will approach and manage knowledge transfer to permanent staff
Cultural fit criteria
  • Evidence of supporting health and wellbeing, including physical and mental health, in the workforce
  • Describe your social value objectives and how you intend to deliver them, including how BEIS will be able to measure them as part of the contract
  • Detail how you will ensure you work in a way that is that is open, honest and constructive
  • Detail how your team will respond to change throughout the project
  • Detail your plan for taking ownership of the requirement
Payment approach
Time and materials
Additional assessment methods
Presentation
Evaluation weighting

Technical competence

60%

Cultural fit

10%

Price

30%

Questions asked by suppliers

1. Is there a current incumbent for this opportunity?
The current work is being managed by a supplier as individual work packages. Further work will be placed through this new contract to the winning supplier.
2. Will the team need SC Clearance prior to starting, or would BEIS be willing to put them through it?
SC Clearance is needed prior to starting, assistance will be provided to complete the necessary documentation.
3. Can you confirm if this opportunity requires any QA/QAT services. We deliver these services to the Home Office and Scottish Government as their preferred supplier?
Work is allocated as required from the Department. QA/QAT services are not the main role of this work but may be required as part of the service.
4. Please can you confirm that BEIS will sponsor SC clearance?
SC Clearance is needed prior to starting, assistance will be provided to complete the necessary documentation.
5. Regarding the expectation that the ‘the supplier will need to be capable of supplying high quality data specialists and other resources on a flexible basis’, please can you confirm if you are expecting the supplier to ramp up and ramp down resources and also what the expected lead time would be?
The supplier will be expected to complete work packages. The supplier will allocate Staff for that work package. Work packages are expected to be agreed one month before the commencement date.
6. Please can you confirm if you are expecting suppliers to evidence their experience in all the technologies listed for transforming data (pythons, R scripts, data lakes) and visualising data (R Shiny, Power BI or via APIs)? While we have all of this experience, it will be challenging to cover it all in only 100 words.
We require you to provide evidence experience in all technologies. Please cover all technologies as broadly as possible within 100 words. Focus mainly on Dataries, Python and Azure products including Synapse.
7. Could you please provide indicative timelines for Stage 2 of the procurement including the dates you intend to notify suppliers if they have been shortlisted, the submission deadline for Stage 2, and the award date etc.?
Provisional timetable:
Shortlisted suppliers notified to submit written proposal 10/10/2022
Shortlisted suppliers submit written proposal 07/11/2022
Supplier presentations week commencing 31/11/2022
Anticipated contract award 03/01/2023
8. Please can you confirm if Stage 1 scores will be carried forward and used as part of the overall evaluation at Stage 2?
The scores for stage 1 will not be carried over for stage 2, scoring criteria for both stages will be separate.
9. Please can you clarify what is meant by “BRE team”?
This is a typo in the published document and should read "the BEIS Data Team".
10. Re: “...Have a detailed understanding of Digital, cyber security standards...” can you clarify if you mean this in terms of a client delivery project, or would it be more like ISO27001 implementation or audit?
The supplier will be expected to understand NCSC cyber security standards and the Digital standards as listed by the Cabinet Office.
11. Good afternoon
Could you please confirm who the ‘BRE’ team are?
Thanks
This was a typo in the published opportunity. The team the supplier will be reporting to is the BEIS Data Team.
12. Please can you confirm if the scope of this opportunity also includes the re-engineering work scheduled over the next 18 months to scale up and simplify the architecture?
The work is currently undefined but this will include reviewing and simplifying the architecture.
13. For Question 6 in the “Essential skills and experience” section, please can you confirm if suppliers can provide other examples where we they have ‘develop[ed] knowledge sharing topic guides, scripts and materials for facilitating workshops deferring final decisions’ to organisations similar to the BRE rather than to BRE itself and will not score a 1 or be disqualified?
Suppliers can confirm their relevant knowledge sharing techniques and experience. This is not limited to BEIS experience therefore a supplier will not be disqualified for providing examples of knowledge sharing topic guides within a different organisation.
14. Please can you confirm whether the platform was built and is currently managed by an in-house team or a third party supplier?
The current platform was built using a blend of internal and external resources. The Azure platform is hosted on the BEIS secure IT environment which is maintained by a 3rd party supplier.
15. What is the estimated size of the team required for a) Maintenance of the DMS b) Data projects?
2-3 people are required to compliment the permanent staff in the data team.
16. What is the lead time for provisioning a data specialist on demand?
The commissioning of a new project consists of a discovery phase where resources are estimated in advance. However, the amount of time we have to stand up a development team can vary depending on the urgency of the project. Once the project has started, work on 2 weekly sprint cycles where resources are identified and allocated at the beginning of the sprint.
17. What is the expected distribution of resources required among Data Infrastructure, Business Insights and Data Policy?
There is no expectation of how the resources will be distributed. We allocate resource depending on business needs.
18. What is the expected duration of deployment for the teams?
That will depend on the complexity of the project.
19. Please can you confirm how many supplier roles are expected to be required across the lifetime of the contract?
The number of roles will vary depending on the demand and type of project, however the following core roles are anticipated; Data Engineers, Data Architects, Full Stack Developers and Data Scientists.
20. Please can you confirm if there are any particular Social Value objectives which you’re looking to achieve?
We are not focused on any particular social value objective but we will be looking for evidence that the supplier is aware of and actively adopting social value objectives.
21. Is there an incumbent who is providing this service to BEIS currently? If yes, why is BEIS coming to market?
The current work is being managed by a supplier as individual work packages, that contract is coming to an end, so we are re-competing.
22. Is the BRE team the Better Regulation Executive? If yes, could you explain how they are connected to the Data Management Platform?
This was a typo in the published opportunity. The team the supplier will be reporting to is the BEIS Data Team.
23. Will BEIS sponsor SC clearance of individuals for this service?
SC Clearance is needed prior to starting, assistance will be provided to complete the necessary documentation.
24. For the following question ‘be able to develop knowledge sharing topic guides, scripts and materials for facilitating workshops deferring final decisions to the BEIS Data team’, please can you explain the deferring decisions part?
Deferring decision to the BEIS Data Team really just means that we want to have final sign off and visibility of this work. We will work collaboratively with the supplier to agree on the approach and deliverables. As the "customer" we reserve the right to make the final decision in the event of any disagreements.
25. Please can you confirm that suppliers will not be penalised for using multiple examples in their response to each question?
A supplier will not be penalised for using multiple examples in their responses.
26. Please can you clarify what is meant by the term ‘Digital’ in the essential criterion ‘Have a detailed understanding of Digital, cyber security standards’?
All Government digital services must comply with Digital Service Standards and National Cyber Security Standards. Details of both can be found on Gov.uk.
27. Who is the supplier referenced in the response to question 1 please?
The current supplier is Aiimi Limited.
28. How does BEIS anticipate a new supplier picking up maintenance and support of the DMS platform? Is a handover period anticipated?
There will be a handover period with the current supplier. There is also a permanent team of civil servants working in this area that will provide some continuity for the new supplier.
29. Can you confirm that the small team outlined (SRO + Lead Data Scientist + 2x Data Engineers) are the BEIS staff who would continue to support DMS following this new procurement?
Yes we can confirm this. We also intend to expand the team over time by recruiting more permanent members of the team.
30. If there are potential gaps in the data strategy and/or data architecture, are BEIS open to these elements being closed-out within the proposed data management partnership?
Yes, we are very open to new ideas and improvements to the current architecture and approach to data management.
31. What technology products do you have in place outside of Microsoft Azure?
We currently use AWS, R, R Shiny, Python, Databricks, Alteryx, Oracle and many other products depending on the requirements of the project we are delivering.
32. Are there real-time data requirements, e.g. streaming/web/IOT... and if so, how much?
There is an ambition to get nearer to real time reporting but, at this point in time, it is not feasible to deliver this. We do not use IOT devices but there is a project in mind that may involve this. We do use Google 360 for some projects to monitor as near real time interaction with our web based services on Gov.uk and other platforms.
33. What is the status of BEIS’s data strategy? Is this codified, and can it be made available to the shortlisted partners? Which elements are present and/or missing, if any?
We are currently drafting a new BEIS Data strategy that will align to the Government's National Data Strategy published in Dec 2020. You can find details of that in the following link https://www.gov.uk/government/publications/uk-national-data-strategy/national-data-strategy
34. Criteria 6 refers to deferring final decisions to the BEIS team. Please could you elaborate on what we would be deferring final decisions on?
All decision needs to be agreed and signed off by BEIS officials. We have a robust governance process for the deployment of new services which sits outside of the BEIS Data Team. Our Departmental Security Unit plays a key role in the decision to deploy new services and the government digital spend controls also control and limit what we can or cannot deliver. Government wide standards must be met across all digital services.
35. Would working on this contract preclude us from bidding for any others?
Working on this contract will not stop a supplier from bidding for other opportunities.
36. Can you please elaborate on Q6 “be able to develop knowledge sharing topic guides, scripts and materials for facilitating workshops deferring final decisions to the BEIS team”.
Suppliers can confirm their relevant knowledge sharing techniques and experience. Deferring decision to the BEIS Data Team really just means that we want to have final sign off and visibility of this work. We will work collaboratively with the supplier to agree on the approach and deliverables.
37. In CQ 6 – can you clarify what you mean by Dataries please?
This is a typo and should read "DataBricks" which is a software package we use.

The deadline for asking questions about this opportunity was Wednesday 21 September 2022.