Home Office Digital Data & Technology (DDaT) – including Borders Immigration Citizenship programmes.

‘DevOps as a Service’ – Evolution of Business Delivery and Support for Cloud Tooling EBSA Platform

18 Incomplete applications

15 SME, 3 large

20 Completed applications

10 SME, 10 large

Important dates

Published
Thursday 12 March 2020
Deadline for asking questions
Thursday 19 March 2020 at 11:59pm GMT
Closing date for applications
Thursday 26 March 2020 at 11:59pm GMT

Overview

Summary of the work
Delivering ‘evolution, business delivery, support cloud tooling platforms’ providing: strong architecture; engineering leadership capability; accountability for platform design; infrastructure integration of all services using tooling platforms; gaining approval of new tooling platforms; driving ongoing tooling/ process innovation; continuously exploring industry improvements; making operational efficiencies and reducing costs.
Latest start date
Monday 15 June 2020
Expected contract length
2 years
Location
London
Organisation the work is for
Home Office Digital Data & Technology (DDaT) – including Borders Immigration Citizenship programmes.
Budget range

About the work

Why the work is being done
The HO has several enterprise-scale Cloud Tooling Platforms - a collection of cloud services, commodity products and bespoke frameworks built on the public cloud, that enable HO DDaT portfolios to deliver high quality digital services to the HO and its customers.

Deliver the ‘evolution, business delivery, support of cloud tooling platforms’ providing; strong architecture, engineering leadership capability; accountable for platform design, infrastructure integration of services; gaining approval of new tooling platforms through HO Technical Design Authority; driving ongoing tooling, process innovation, continuous industry improvements making operational efficiencies and reduce costs.
Problem to be solved
To prevent an ever-increasing landscape of cloud service offerings, open source products as well as commercial tools and services on the market being adopted across the Home Office across multiple supply bases.
Continuing with this approach will create an even more vast and varied landscape which are technically not required, producing additional cost.
This cloud tooling platform needs a strong architecture and engineering capability to maximise efficiency and minimise costs.
Who the users are and what they need to do
Users across the HO Digital Data Technology (DDaT) programme including ITP (Immigration Transition Programme), HM Passports Office (HMPO) Digital Services at Border Programme (DSAB) and other Borders Immigration and Citizenship programmes and wider HO and government agencies need to secure UK borders, and detect criminal behaviour.
Early market engagement
Any work that’s already been done
Cloud Tooling Platforms are in development. Industry leading technologies have been selected to create a Cloud Tooling Platform that empowers project teams to build, deploy and operate products throughout the full product lifecycle using the provided platform tooling services; provides as much of a self-service experience to project teams as possible for common needs; and is constantly evolving to maximise efficiency and minimise costs.
Existing team
It should be noted that there is an incumbent supplier, who will have the same opportunity to bid as every other supplier on the Framework.

The Supplier will be undertaking these activities in a multi-disciplinary / multi-vendor environment. We anticipate that the contracted supplier will develop strong collaborate working relationships with the existing Home Office DDaT portfolio teams and with the other suppliers supporting Home Office DDaT with whom the supplier will work.
Current phase
Not applicable

Work setup

Address where the work will take place
The Supplier will be predominantly based at Home Office premises within Croydon, but may be required to attend alternative Home Office locations and alternate/ offsite working locations as necessary.
Working arrangements
Resources will work to standards and practices set by GDS and Home Office Teams, as part of existing delivery teams focused on specific outcomes. On-site delivery partner resources will conform to HO policies & procedures.
For on-site based requirements, resources are likely to be required on site Mon–Fri or a number of days to be confirmed. Approximate hours are 09:00–18:00.
The Home Office has a strategy of Smarter Worker, at the discretion of the Senior Leadership Team.
Expenses will conform to Home Office policies and will not be payable within the M25.
Home to office expenses will not be considered.
Security clearance
Provider personnel (and sub-contractors) must comply with HMG Baseline Personnel Security Standard (BPSS) as a minimum, and CTC clearance to work in HO buildings. There may also be a requirement for some postholders to undergo SC clearance, if not already held and, in limited circumstances, DV clearance.

Additional information

Additional terms and conditions
Travel and accommodation payable outside M25 when pre-approved and in line with HO T&S policy. Subsistence will not be paid
Exit Activities
Rate cap - daily rates over £1000 will not be accepted and will render a bid non-compliant
SFIA 7 level resource is excluded (exceptional circumstances will be considered)
SFIA 6 level resource is excluded for 'hands on' delivery roles

Skills and experience

Buyers will use the essential and nice-to-have skills and experience to help them evaluate suppliers’ technical competence.

Essential skills and experience
  • Provide: Details of recent (within the last 2 years) experience of designing, building and operating an enterprise scale Cloud Tooling Platform on AWS.
  • Details of recently implementing mature cost optimisation techniques on public cloud
  • Details of recently driving lean methodologies in the delivery of 'DevOps as a Service' that aspires to optimise efficiency and the removal of unnecessary overheads
  • Details of recent experience of delivering continuous improvement of security controls on a cloud tooling platform against security risk assessments
  • Details of recent experience of placing a delivery team with experience of setting up enterprise scale performance monitoring and alerting tools.
  • Details of recently supporting continuous delivery of projects deploying services onto a centrally managed cloud tooling platform, including: network boundary controls; allocation of cloud resources, object storage, VPCs, subnets, namespaces
  • Details of recently providing operational support of cloud tooling platform, including: incident management; change management; problem management; application and infrastructure management; security management; running operational procedures; collaborative tooling support
  • Details of recently providing resources with at least 3 years’ experience and practical knowledge of continuous delivery and improvement of cloud tooling services on AWS including:
  • infrastructure build pipelines; containerisation platforms; development tooling; environment build tooling; host management tooling; environment management tooling; security management tooling; collaborative tooling; test tooling
Nice-to-have skills and experience
  • Evidence of resources that have existing SC clearance that will support the speeding up of on-boarding teams
  • Demonstrate experience of handing over products to another team, including service transition

How suppliers will be evaluated

All suppliers will be asked to provide a written proposal.

How many suppliers to evaluate
5
Proposal criteria
  • T1(a) - How will you manage the work and maintain quality? - 5%
  • T1(b) - How will you quickly identify resources and on-board them on site and on the start date? - 5%
  • T1(c) - What is your approach to transition and handover from the incumbent? - 5%
  • T1(d) - How will you adopt the approach/solution, allowing for ongoing support and further development? - 5%
  • T1(e) - How will you enhance the methodology and offer opportunities for improvement? - 5%
  • T2 - Describe your organisation and team capability and knowledge of the technology required to deliver the service - 10%
  • T3(a) - Identify the key team roles and proposed team members - 3%
  • T3(b) - Describe your proposed team structure - 3%
  • T4 - Describe how you would add value to individual project teams on EBSA and the wider Home Office (do not discuss pricing) - 6%
  • T5 - Presentation (shortlisted bidders only) - 10%
  • P1 - Whole life cost (broken down by year) - 30%
  • P2(a) - Overall Average Day Rate (total cost divided by number of staff days required) - 10%
  • P2(b) - Rate card (not evaluated)
  • P2(c) - Breakdown of staff days by SFIA Level and daily rate (not evaluated)
  • C1 - Cultural fit - 5%
Cultural fit criteria
  • Work as a team with our organisation and other suppliers
  • Transparent and collaborative when making decisions
  • Have a no-blame culture and encourage people to learn from their mistakes
  • Application of Agile Principles
  • Ability to add value
Payment approach
Capped time and materials
Additional assessment methods
Evaluation weighting

Technical competence

55%

Cultural fit

5%

Price

40%

Questions asked by suppliers

1. Who is the incumbent supplier?
The incumbent is Capgemini.
2. The same opportunity was published first week of February and then subsequently cancelled. Please could you let us know what has changed between now and then?
Following a review of the likely whole life cost of the contract, it became apparent that further governance measures were required, so the initial advertisement had to be withdrawn. Also, some additional information has been made available this time.
3. What is the timeline for the process? Application feedback, Presentation, Award.
Details are shown in the advertisement.
4. Is there a budget for this work?
This is a call for competitive bids, which will be evaluated partly on the proposed prices. As such, it is for bidders to decide what prices to include in their proposal.
5. Who is the incumbent supplier?
The incumbent is Capgemini.
6. You mention ''It should be noted that there is an incumbent supplier, who will have the same opportunity to bid as every other supplier on the Framework.'' Who is the incumbent supplier?
The incumbent is Capgemini.
7. Is there a current incumbent?
The incumbent is Capgemini.
8. One essential skill appears to be split across two bullets – how should we address this: "- Details of recently providing resources with at least 3 years’ experience and practical knowledge of continuous delivery and improvement of cloud tooling services on AWS including: infrastructure build pipelines; containerisation platforms; development tooling; environment build tooling; host management tooling; environment management tooling; security management tooling; collaborative tooling; test tooling"
This happened because the question word limit was exceeded, so 2 fields had to be used. Similarly, if you choose to use 2 fields for the response to this question, please feel free to do so; it would help if this is made clear in the response.
9. Please confirm who the incumbent is and how long they have been in place. Also if possible the current team size.
The incumbent is Capgemini. The current team is 43-strong, and is expected to increase over the lifetime of the contract.
10. Can you please share the current Cloud Tooling Platforms? like architecture and any other related document?
We follow industry best practice in regards to architectural and cloud usage patterns. Including but not limited to many AWS Whitepapers.
11. Can you please share the peak count of resources in the team that developed current Cloud Tooling Platforms?
The peak count was 43.
12. Can you please share the list of tools and technologies used in current Cloud Tooling EBSA Platform?
List of tools and technologies – Nginx, squid, Fortigate, Stunnel, DNSmasq, OpenVPN, Zabbix, Dynatrace, Grafana, Prometheus, Thanos, OpenLdap, Consul, Vault, Packer, Terraform, Puppet, Ansible, ELK, Yum, Kubernetes(KOPS), Helm, Red Hat SSO, Jira, Confluence, Crowd, Bitbucket, Python, Sonarqube, and AWS including:  EC2, S3, VPC, NACLS, NAT Gateways, Athena, RDS, Glue, Lambda, SNS, SQS, Cost Explorer, Chime, Workmail, Spot, Autoscaling, ECR, Aurora, DMS, Workspaces, Cloudwatch, Cloudtrail, Config, Organizations, API Gateway, Privatelink, GuardDuty.
13. As per the clarification, the question word limit was exceeded, so 2 fields had to be used for 2 of the questions in essential sections. Could the Authority please clarify if we can use in total 200 words response for this combined question- 100 words in the first half and 100 words in the other half of the response.
100 in each is fine; please just indicate that this is what you have done.
14. Please can the Home Office clarify the evaluation weighting of the Proposal Criteria as presently this appears to total 102%?
Apologies for the arithmetical error. To save confusion and additional effort by bidders, the total written Technical score will be out of the published 47 (instead of 45); the bid score awarded will be multiplied by 0.96 to give a score out of 45.
15. Please can the Home Office confirm that ‘Presentation’ equates to 10% of the overall 55% Technical Competence weighting?
That is correct.
16. Please confirm if there is any expected change to either the submission deadline or the expected start date due to the coronavirus outbreak?
As it currently stands, no changes are expected; however, everything is subject to events.
17. Please can the Home Office clarify the approximate number of resources required in total and more specifically a breakdown of the resources that require the following security clearances: BPSS, CTC, SC and DV.
Current team size is 43, although expected to grow over the life of this procurement. All resources are expected to be SC cleared.
18. Regarding the requirement on operational support of cloud tooling platform; please confirm if this describes the configuration and operation of support tools (e.g. issue and bug tracking software), or building/running operational support processes using an existing toolset?
This includes both, support of hosted JIRA instance for issue and bug tracking purposes, and running platform support systems such as monitoring, secret management tools etc.
19. Are you happy for us to quote multiple client examples for one bid question? for example: Details of recently supporting continuous delivery of projects deploying services onto a centrally managed cloud tooling platform, including: network boundary controls; allocation of cloud resources, object storage, VPCs, subnets, namespaces.
Unless there are instructions to the contrary, multiple examples are acceptable. However, bidders will need to observe the word limit.
20. While you've mentioned in your response to an earlier clarification question that you're looking at competitive bids, it is important for bidders to be able to understand the size/scale that you are looking for. The earlier procurement stated £10-£12 million. Are you looking at a similar scale for this one too?
The team size, technologies and tasks have been detailed in the original scope and questions which will give an indication of scale. This is a call for competitive bids which will be evaluated partly on price.
21. ''Details of recent experience of placing a delivery team with experience of setting up enterprise-scale performance monitoring and alerting tools.'' What are the monitoring / alerting tools in use now?
Prometheus/Grafana/Thanos/Dynatrace/Zabbix/Ops Genie.
22. ''Collaborative tooling support'' – What does this mean? Are you looking for examples of collaboration tools and if so, which ones are in use currently?
This in regards to the maintenance and configuration of the Atlassian tool suite, Confluence/JIRA.
23. Will the Cloud Tooling Platform be deployed in one cloud, multiple clouds, or cloud plus on premise environment(s)?
In one cloud.
24. A successful DevOps adoption is dependent on people and process, is DevOps as a Service expected to provide process consultation and change management governance?
It is expected the resources will be advising on the platform CI processes, guidance to project teams on processes and will steer the Home Office change team on best practices.
25. Does EBSA have well defined Security and Vulnerability assessment practices for Security risk assessments?
EBSA has a number of security and vulnerability assessment practices however we are always looking for improvements.
26. Is EBSA security control tooling also expected to include threat protection and threat modelling?
Threat protection yes. Threat modelling no.
27. Please provide an overview of how the current DevOps processes work and how the DevOps organisation is structured from a functional perspective.
The platform team has three core teams, one focused on improvements and upgrades, one focused on integrations with external parties, and the third focused on BAU support for the platform. The wider programme of work has separate devops resource (not included in this contract) for building CI/CD integrations for specific teams.
28. Please clarify whether the requirement is purely staffing only, or whether you require suppliers to provide a run/manage/optimise service for the EBSA platform management operations.
The contract is for staff to provide the capabilities mentioned in advertisement.
29. Please state what proportion of the supplier staffing will be required to be on Home Office sites on an ongoing basis.
As per the advertisement, the supplier will be predominantly based at Home Office premises in Croydon.
30. How big is the current AWS estate? In particular how many VMs, how much DB and how much storage is currently in situ. Does the Home Office have an anticipated annual growth rate that can be applied to the current estate?
Approximately 2,000+ VM’s, 1,200+ managed databases, 35,000+ pods, 50,000+ docker containers.
31. Please could the Home Office provide metrics on the current/envisioned support requirement i.e. the number of tickets generated, backlog levels and any associated backlog trends (e.g. up, down or stable).
~7500 tickets resolved in last 12 months of varying size and complexity. The backlog is expected to fluctuate throughout the year based on the projects being run.
32. Does the Home Office have existing collaborative, security management and test tooling that should be considered as part of integration with vendor tooling solutions?
Yes, examples tools given in the response to a previous question.
33. Under essential skills and experience, 'allocation of cloud resources', do you mean creation / modification / teardown of AWS resources?
Creation and management of cloud VM’s and Kubernetes clusters based on use by multiple teams.
34. Please state whether the suppliers will be able to bring their own or other 3rd party tooling (subject to Home Office vetting) to provide any of the services required, or whether suppliers are restricted to an approved Customer tooling catalogue.
There is an existing set of tooling but we are always willing to review new tools based on a sufficient use case and security approval.
35. What proporion of the team are likely to need SC clearance of higher?
All resources are expected to be eligible and go through SC clearance, no higher.
36. Can you provide the current roles for the team supporting the Cloud Tooling EBSA Platform?
Devops/Platform/Network engineers, infrastructure/cloud architects, scrum masters, product owner, project manager.
37. Please could you talk a bit about what you mean by ''development tooling; environment build tooling; host management tooling; environment management tooling''. It is a lot to ask for in 100 words and we want to make sure our understanding is the same as yours.
Development tooling – tooling used by developers e.g. git repository. Environment build tooling – tools used to build environments e.g. infrastructure code. Host management tooling – tools to manage hosts/VM’s – e.g. monitoring tools. Environment management tooling – tools to manage environments – e.g. configuration management tools.