Awarded to Mercator IT Solutions

Start date: Monday 18 January 2021
Value: £120,000
Company size: SME
The National Archives

Technical Architect/Developer to lead build of an alpha system for managing catalogue data using RDF

11 Incomplete applications

11 SME, 0 large

20 Completed applications

18 SME, 2 large

Important dates

Published
Monday 26 October 2020
Deadline for asking questions
Monday 2 November 2020 at 11:59pm GMT
Closing date for applications
Monday 9 November 2020 at 11:59pm GMT

Overview

Specialist role
Technical Architect
Summary of the work
After a successful discovery, we now need an expert Technical Architect and Lead Developer to help us implement an alpha of a new pan archival catalogue, developing a solution meet the users’ needs, using AWS Neptune and Elasticsearch.
Latest start date
Monday 4 January 2021
Expected contract length
5 to 8 months
Location
No specific location, for example they can work remotely
Organisation the work is for
The National Archives
Maximum day rate
Up to £1000 per day

About the work

Early market engagement
We have run a discovery phase leading to plans for an alpha using AWS Neptune and Elasticsearch. You can read more about the project: https://www.nationalarchives.gov.uk/about/our-role/plans-policies-performance-and-projects/our-plans/our-digital-cataloguing-practices/project-omega/

The discovery produced a proposal for a new Catalogue Data Model using RDF, a new identifier scheme, and transformation routines for the existing data to the new model. We have held workshops identifying the key ways that staff managing the catalogue work with the data and what they would like in future. The archivist needs to search, analyse, add to, correct, edit, enrich, and enhance record descriptions so that the catalogue is properly maintained. The archivist needs to work with catalogue entries individually or as large sets, making (or reversing) bulk changes, so they can work efficiently. The archivists need to understand the version history of the catalogue so they can be confident about where the information has originated.

We have investigated all the current databases that hold catalogue data and how they inter-relate. We have investigated a wide range of existing data standards and ontologies. We have documented all the findings in a detailed published report.
Who the specialist will work with
The specialist will work with an RDF Developer (another specialist being recruited at the same time). The core in-house team consists of a data analyst, two senior archivists and the Head of Cataloguing, Taxonomy and Data. The specialists will also work with a wider group of users, archivists across the organisation responsible for the management of the catalogue.
What the specialist will work on
We are developing a pan-archival catalogue, bringing together record descriptions from multiple catalogues into a single new system. We are looking for a technical architect and developer to lead the development work on an alpha catalogue management system.

This work will involve developing API functions to search, select, add, export, edit, import and delete catalogue data; developing search for use by expert users (using SPARQL in combination with Elasticsearch); developing an Extract, Transform, Load process to migrate The National Archives catalogue data from multiple relational database (SQL Server) and RDF databases to a cloud based native RDF database (AWS Neptune).

Work setup

Address where the work will take place
Whilst mainly working remotely, the Catalogue, Taxonomy and Data team is based at The National Archives, Bessant Drive, Richmond, Surrey TW9 4DU
Working arrangements
Given the situation with COVID-19, remote working will be the main arrangement for this project. Some work may be conducted at The National Archives, Bessant Drive, Richmond, Surrey TW9 4DU, but only when it is appropriate and safe to do so.
Security clearance
Baseline security clearance will be required.

Additional information

Additional terms and conditions

Skills and experience

Buyers will use the essential and nice-to-have skills and experience to help them evaluate suppliers’ technical competence.

Essential skills and experience
  • Ability to understand business needs and priorities and apply them to project planning
  • Excellent communication skills, translating technical relating to software engineering, delivery management and service management so that they are understood by all.
  • Strategic overview of standards and technology, with the capacity of turning business problems into technical design.
  • Have experience with using standards based ontologies/vocabularies, such as W3C PROV data model, Dublin Core and W3C ODRL
  • Have experience of working with RDF databases and SPARQL, for example AWS Neptune
  • Have experience, knowledge and understanding of Extract, Transform, Load (ETL) processes
  • Have experience, knowledge and understanding of create resilient and secure systems using IAM in a cloud context.
Nice-to-have skills and experience
  • Have experience developing a user interface/front end to support non-expert, editorial engagement with RDF
  • Understanding of informational architecture in an archival context

How suppliers will be evaluated

All suppliers will be asked to provide a work history.

How many specialists to evaluate
6
Cultural fit criteria
  • Have collaborative and flexible working approach, e.g. working with in-house technical and other digital specialists
  • Approach to supporting teams to adopt new technologies
  • Examples of delivering transition, knowledge transfer and handover of code.
  • An appreciation for the importance of technical documentation as a means of ensuring ongoing maintainability of systems.
  • Demonstrable commitment to a diverse working environment, with a team comprised of experts from a wide variety of backgrounds.
Additional assessment methods
  • Reference
  • Interview
Evaluation weighting

Technical competence

60%

Cultural fit

20%

Price

20%

Questions asked by suppliers

1. Please confirm if you have an incumbent
No, there is no incumbent.
2. Would it be an option to sponsor a T2ICT Visa for someone for this role?
No. This opportunity is open to UK, Commonwealth and European Economic Area (EEA) and certain non EEA nationals. Further information on whether you are able to apply is available at https://www.gov.uk/government/publications/nationality-rules
3. Given that most of the work will be done remotely, can resource work from offshore location too? or you are looking for only UK based candidates?
Given the situation with COVID-19, remote working will be the main arrangement, although some work may be conducted at The National Archives if appropriate and safe. The resource would need to work from the UK or European Economic Area (EEA) and if appropriate be prepared to travel (and pay for all their travel expenses).
4. Is this role inside/outside IR35?
We have checked the role requirements (to the best of our knowledge) using the assessment tool found at https://www.gov.uk/guidance/check-employment-status-for-tax and, in our opinion, for the role(s) as advertised the intermediaries legislation does not apply to this engagement.
5. Have you done a pricing analysis of projected cost for AWS neptune for data transmission and data storage over the next 2-3 years?
An analysis has been undertaken and the results will be shared with the appointed supplier. If this response does not satisfy you, and more detail is required in order for you to apply for this opportunity, please re-post with an explanation of your need for the information.
6. Is there an ETL solution or strategy in place for the AWS Neptune?
The project page at https://www.nationalarchives.gov.uk/about/our-role/plans-policies-performance-and-projects/our-plans/our-digital-cataloguing-practices/project-omega details the work done during the first stage of the project. Building on this initial work, including the ETL pipeline, would be included in the role.
7. Please could you confirm whether the focus of this role is Technical Architecture, or whether the specialist will be expected to be writing code as a developer?
The role would encompass both taking the lead in designing the technical architecture and implementing that design, including hands-on development of the systems, which would include writing coding as necessary.
8. What is the project number of concurrent users and average users?
(how many users will be using it at anyone time and how many as a whole)
Approximately 25 staff use it daily (not necessarily exactly at the same time), out of approximately 120 active users who use it less frequently.
9. How many days per week on site will the successful consultant be required for?
The consultant is expected to work mostly remotely with on-site visits only very sporadically, if a specific need arises and is safe to do so. We do not anticipate the need of more than 2 or 3 visits in total.
10. Would you consider responses that offered between 0.5 and 0.75 FTE?
We might be able to accommodate something between 0.5 and 0.75 (or explore a time extension) for the successful consultant, but this type of flexibility would have an impact on other team members and product delivery. We would consider this possibility in the context of the work history assessment and interview.
11. Would you consider responses that offer two experienced individuals with an aggregate effort level up to 1 FTE?
We might be able to consider sharing arrangements. We would consider this possibility in the context of the paper work histories submitted, before interviews take place.