The National Archives

Developer - Scala (Play) and ETL

8 Incomplete applications

8 SME, 0 large

0 Completed applications

Important dates

Published
Friday 16 September 2022
Deadline for asking questions
Friday 23 September 2022 at 11:59pm GMT
Closing date for applications
Friday 30 September 2022 at 11:59pm GMT

Overview

Specialist role
Developer
Off-payroll (IR35) determination
Supply of resource: the off-payroll rules will apply to any workers engaged through a qualifying intermediary, such as their own limited company
Summary of the work
Develop iterations of a consolidated pan-archival catalogue to meet users’ needs, including services/APIs in Scala (Play) and ETL processes.
Latest start date
Monday 31 October 2022
Expected contract length
6 months contract with possible extension of 3 additional months
Location
London
Organisation the work is for
The National Archives
Maximum day rate
Up to £700 per day

About the work

Early market engagement
We ran Discovery and Alpha phases. You can read more about the project: https://www.nationalarchives.gov.uk/about/our-role/plans-policies-performance-and-projects/our-plans/our-digital-cataloguing-practices/project-omega/
The Discovery phase produced a proposal for a new Catalogue Data Model using RDF, a new identifier scheme, and an Extract, Transform and Load process to migrate catalogue data from a relational database to RDF. We have built the first version of an API and are just starting to build the initial screens having held workshops identifying the key ways that staff managing the catalogue work with the data and what they would like in future. The archivist needs to search, analyse, add to, correct, edit, and enhance record descriptions individually and in bulk so that the catalogue is properly maintained.
We now seek a specialist developer to speed up progress on the delivery of a Private Beta with the following work strands:

• Data extraction and transformation into RDF
• Cloud-based data services
• Editorial web application services user interface for users to view, edit and save metadata
• API services
Who the specialist will work with
The specialist will work with a Technical Architect and two RDF Developers. The core in-house team is the Service Owner, Product Manager/Data Analyst, Delivery Manager, a Senior Archivist, and the Head of Cataloguing, Taxonomy and Data. The specialists will also work with a wider group of users including archivists and editors across the organisation responsible for the management of the catalogue.
What the specialist will work on
We are developing a Pan-Archival Catalogue and management system, bringing together record descriptions from multiple catalogues into a single new system. You will join a Technical Architect and two other Scala and RDF developers to work on multiple iterations of the Beta development.

We will be continuing with the Extract, Transform and Load process and will be developing services in Scala, using Play Framework for the frontend, to enable users to search, select, add, edit, import, export and delete data via a UI. We will be developing further API services to enable integration between the Catalogue and other TNA systems.

Work setup

Address where the work will take place
The National Archives, Bessant Drive, Richmond, Surrey TW9 4DU
Working arrangements
Remote working is the main arrangement for this project, with monthly team work sessions in person (2-3 consecutive days, occasionally up to 5 days) conducted at The National Archives, Bessant Drive, Richmond, Surrey TW9 4DU.
Although this has been advertised as a resource, we have checked the role requirements (to the best of our knowledge) using the assessment tool found at https://www.gov.uk/guidance/check-employment-status-for-tax and, in our opinion, for the role(s) as advertised the intermediaries legislation does not apply to this engagement (outside IR35).
Security clearance
Baseline security clearance will be required.

Additional information

Additional terms and conditions

Skills and experience

Buyers will use the essential and nice-to-have skills and experience to help them evaluate suppliers’ technical competence.

Essential skills and experience
  • Have experience with using standards-based ontologies/vocabularies, such as W3C PROV data model and Dublin Core
  • Have experience of working with graph databases (for example AWS Neptune), SPARQL, and validating RDF data (for example using RDF SHACL).
  • Have experience, knowledge and understanding of building Extract, Transform, Load (ETL) processes
  • Have experience, knowledge of Scala and an understanding of working with mixed content in the context of large, semi-structured datasets
Nice-to-have skills and experience
  • Have experience developing a user interface/front end to support non-expert, editorial engagement with graph data
  • Experience of using Play Framework

How suppliers will be evaluated

All suppliers will be asked to provide a work history.

How many specialists to evaluate
4
Cultural fit criteria
  • Have collaborative and flexible working approach, e.g., working with in-house technical and other digital specialists, ready to attend monthly work sessions in person
  • Approach to supporting teams to adopt new technologies
  • Examples of delivering transition, knowledge transfer and handover of code
  • An appreciation for the importance of technical documentation as a means of ensuring ongoing maintainability of systems
  • Demonstrable commitment to working within a diverse working environment, with a team comprised of experts from a wide variety of backgrounds
Additional assessment methods
  • Reference
  • Interview
  • Scenario or test
Evaluation weighting

Technical competence

60%

Cultural fit

20%

Price

20%

Questions asked by suppliers

1. just to confirm, this opportunity is Outside IR35?
We refer you to the statement in the advertisement that, in our opinion, for the role the intermediaries legislation does not apply to this engagement (outside IR35).
The HMRC CEST tool was completed by the contract manager making certain assumptions at this stage in the process. Should any of those assumptions prove incorrect, we reserve the right to re-run the HMRC CEST tool.
HMRC remain the final arbiter as to whether IR35 applies or not.
2. Is there a current incumbent supplier?
Yes, There is a supplier currently in the role whose contract ends on 30 September 2022 and is at liberty to re-apply. We wish to assure all potential suppliers that all bids will be treated on their own merit, and the incumbent will receive no special credit or favour.
3. If this process is to renew the existing current incumbent, why this role publishes on DOS for other suppliers to apply. How this could be a fair assessment ?
Do you have preferred supplier?
If we give a reduced margin, will you transfer current incumbent to our books?
The 'incumbent' supplier's contract is coming to an end. The purpose of this exercise is not to 'renew' the existing incumbent; the current incumbent may or may not choose to apply. We will run this competition fairly and objectively, scoring all responses according to the framework methodology.
There is no preferred supplier.
We have no contractual ability to transfer the current incumbent.

The deadline for asking questions about this opportunity was Friday 23 September 2022.