Awarded to Equine Register Ltd

Start date: Monday 28 October 2019
Value: £183,500
Company size: SME
Livestock Information Limited (subsiduory of Agriculture & Horticulture Development Board)

LIP (Livestock Information Programme) Data Management Support

22 Incomplete applications

17 SME, 5 large

31 Completed applications

22 SME, 9 large

Important dates

Tuesday 13 August 2019
Deadline for asking questions
Tuesday 20 August 2019 at 11:59pm GMT
Closing date for applications
Tuesday 27 August 2019 at 11:59pm GMT


Summary of the work
Planning and deployment of consolidation of multiple single species system to a multi species database, incorporating ETL scripts, deduplication, data matching with Business Intelligence services for data exposure.
Latest start date
Monday 21 October 2019
Expected contract length
6-12 months
No specific location, eg they can work remotely
Organisation the work is for
Livestock Information Limited (subsiduory of Agriculture & Horticulture Development Board)
Budget range
Expectation is between 100-200 days

About the work

Why the work is being done
To support the UK's world-leading food and farming industry, Defra is aiming to replacing its livestock traceability systems by the end of 2019. As a pre requisite to this, we need to consolidate, de-duplicate and data match various differing data sources into a single multi-species system
Problem to be solved
Multiple single species databases need to be collapsed into a single multi-species system
Who the users are and what they need to do
To be able to manage multiple livestock species, from birth to death including movements.
Early market engagement
Any work that’s already been done
We have undertaken an initial assessment of the current databases in terms of the structure which will be shared with the winning provider
Existing team
The current in house team is very small. The supplier will be working with and reporting to the Enterprise Architect. It is expected that the supplier will work is a fast, agile manner. Due to the small team size, the supplier must be very flexible, innovative and have the ability to adapt quickly. The supplier will also work with Defra's system integration teams who will facilitate integration to legacy systems.
Current phase

Work setup

Address where the work will take place
Various Defra sites including; London, Workington, Reading and Crewe. AHDB headquarters at Stoneleigh Park, Warwick.
Working arrangements
The supplier is expected to engage on a daily basis with the Enterprise Architect and team via a tele conference to; discuss current and future work, uncover any blockers. Attendance on site for technical design meetings, as required, to facilitate system integration and ensure adherence to Defra technical standards . Expenses are limited to actual receipts.
Security clearance
BSC - evidence will be sought on request

Additional information

Additional terms and conditions
CCS Digital Market place terms used with two additions;
Please see 'Additional clauses' within the 'Brief to Supplier FV' document.

Skills and experience

Buyers will use the essential and nice-to-have skills and experience to help them evaluate suppliers’ technical competence.

Essential skills and experience
  • Database management skills associate to rapid design and implementation of a Microsoft Azure based solution
  • Database specialisms
  • Data sciences and business intelligence skills
  • Integration skills required for access to source data
  • Experience of GDS standards.
Nice-to-have skills and experience
  • Ability to work in a supplier eco-system
  • Ability to work remotely

How suppliers will be evaluated

How many suppliers to evaluate
Proposal criteria
  • Agile
  • Remote delivery
  • Technical competence
  • Livestock traceability
  • Integration
Cultural fit criteria
  • The supplier will co-operate in accordance with the following principles:
  • Innovative use of technology to implement cost efficiency
  • Working in partnership with other suppliers
  • Communicating complex issues using clear and concise language in both written and verbal ways to a variety of audiences and expertise levels.
  • Knowledge of the livestock industry
  • Knowledge of the EU animal identification
  • Knowledge and understanding of securely handling sensitive data
  • Understanding of both legal and data protection issues in using personal data
Payment approach
Capped time and materials
Assessment methods
Written proposal
Evaluation weighting

Technical competence


Cultural fit




Questions asked by suppliers

1. What is the budget for this procurement?
The budget is between £100k - £200K
2. Which database technologies are being used for these existing systems and where are these hosted? existing LIL hosting, supplier or public cloud AWS, Azure for example.
SQL, Oracle and PostgreSQL
3. How many data sources are in scope and approx data volumes: Total storage capacity (MB/TB) and/or the total number of physical records are stored in all systems that are in scope?
three data sources, each with approximately 500Gb consisting of 25 million records.
4. What format is the current data stored in? e.g. XML, Relational, JSON, Binary etc..
Predominately relational.
5. Can you please explain further how this work feeds into the replacement of the Livestock traceability systems?
The existing services are split over 3 different systems, ie Cattle, Sheep/Goats/Deer and Pigs. The new livestock service consolidates these 3 systems into a single multi species service.
6. Does the LIL (Livestock Information Limited) have any preferred technology platform for the new Livestock Database?
The target platform will be Azure.
7. Has LIL considered NoSQL technology when looking at new data integration solutions?
We are open to other platform considerations.
8. You mention Business Intelligence Services for data exposure. Do you have any preferred technology for this?
This will be investigated with the chosen supplier.
9. Apart from Business Intelligence does your data need to be shared with any other application or agencies?
Internal users only for this opportunity.
10. Does the database need to be accessed on mobile devices or via any public-facing web portal?
No, the prototype is for internal use only.
11. You mentioned Microsoft Azure in your requirements. Are you open to considering use of other public cloud vendors (e.g AWS)?
Our preferred platform is Azure
12. Hosting, please can you confirm whether you are looking for the bidder to provide the hosting element of the solution or will this be provided by an existing supplier or LIL directly?
Hosting is out of scope and being procured outside this procurement.
13. User Management, please detail any user authentication requirements that will be needed and confirm what is the expected growth rate of these users for the period of the contract?
This is an internal service only, users will be limited to a maximum of 10.
14. Can you please confirm who the people on your selection panel are and their individual roles and responsibilities.
The following people are on the evaluation panel and will be responsible for reviewing and scoring submissions.

Robert Stanley - LIP Data work stream lead, scoping and initiation (Specification development and evaluator)

Matt Catlin - AHDB Digital Director (evaluator)

Mark Matthews - LIP Technical & Digital delivery lead evaluator)

Danielle Woodall - Commercial Lead
15. Can you advise how suppliers obtain the 'Brief to Supplier FV' document mentioned in the additional terms section.
The 'Brief to Supplier FV' will be made available to the suppliers who are invited to the shortlist stage. This will be between 3 - 6 suppliers.
16. We would like to understand the 'additional clauses' within the 'Brief to Supplier FV' document. Please could you explain how to access either the document or these clauses?
1. Where any aspect of the Services is to be carried out at a premises belonging to a third party, the Supplier shall ensure that the Supplier Staff shall comply with all instructions, policies and directions of such third party.
2. The Supplier acknowledges that:
a. The Buyer is procuring services on behalf of itself (the "Service Recipients"); and
b. Subject to the Clause 33, Losses suffered by a Service Recipient that are caused by a Default of the Supplier shall be deemed to be Losses that have been suffered by, and which are recoverable by, the service recipient.
17. Where are the existing Oracle, SQL and Postgres SQL databases hosted please? Are these all solely on-premise?
These are a mix of onsite and offsite however the live databases will be ported to the Prototype system by in house resources prior to engagement with the chosen supplier.
18. You mention that an "initial assessment of the current databases in terms of the structure" has already been done. Was this work done by the DEFRA internal team?
This work has been completed by internal resources.
19. You mention that an "initial assessment of the current databases in terms of the structure" has already been done. Did this result in any kind of a data catalogue or data model?
This will be one of the elements the supplier will be asked to complete in conjunction with LIP resources
20. You mention that an "initial assessment of the current databases in terms of the structure" has already been done. Did this include metadata as well as data, and what is DEFRA's view of the quality of its metadata?
No, just the data
21. Has DEFRA already established a metadata ontology for the desired consolidated data and forward-looking processes?
This would form part of the collaboration between the chosen supplier, internal LIP resources and other suppliers.
22. Has DEFRA already conducted any design work regarding the eventual information outcomes and what particular data are required to provide those?
The data assets required to provide the traceability service have been defined as the project will replace an existing service.
23. Do you currently have an incumbent supplier in place for previous parts of this project. Or are all suppliers going to be new to this area and the LIP project
There are existing suppliers for the legacy services, and there will be a small number of new suppliers for the prototype and live service. Suppliers must be able to work within a supplier ecosystem on a fix first basis.
24. Is this project for a prototype or full production service? In the response to Q10, you mention that this is a prototype
This is for a prototype service to aid the extract, transformation and load to the live multi species service.
25. Have you analysed the content of the source systems as well as the structure? Do you have any indication of the quality / cleanliness of source data?
This exercise will form part of the procurement in conjunction with the LIP resources.
26. The source systems are managed by different organisations. Will the winning supplier have access to appropriate source system technical support?
Yes, exit provisions exist within 3rd party contracts
27. Is there an Azure Automation entitlement in place?
This is not currently in place.
28. Who delivered Discovery?
Internal resources supplemented by contract resource
29. Under 'Essential skills and experience' you list 'Data sciences and business intelligence skills'. Will the role of Data Scientists be solely in support of the deduplication of entities within and across the three source systems or do you anticipate other activities using these skills?
Data Scientists will mainly support the deduplication of entries but there is also the possibility for other activities following discovery, due to the source systems.
30. Business Intelligence typically shows a historic view of data and/or current snapshot but do you anticipate there being any predictive elements within the BI Services?
Predictive services are not currently in scope, predictive services and/or AI will form part of phase 2 of the project.
31. Will BI Services running on the current source system need to be migrated and if so, please can you provide more information about these services (technology, number of dashboards/views and users)?
The existing BI srvices will not need to be migrated, however we will use the business rules as the basis for the new BI provision.
32. Do you anticipate the need to evolve the capability to utilise streaming or messaging technologies or will it continue to use non-streaming data storage (i.e. relational database in this case)?
Not in scope
33. How will future data volumes and formats be affected by the potential departure of the UK from EU and is there a need to consider more flexible storage formats (e.g. Snowflake) as opposed to typical Azure relational databases?
We expect little impact as the movements tracked are within the UK.
34. Do you have preferential ETL tools and is your preference to use tools organic to Azure (e.g. Data Factory and Data Catalog)?
We have no current preference, best practice will be used to ensure we have the right solution.
35. There is a Cultural Fit Criteria which makes up 20% of the assessment. However, there is not a Technical Fit Criteria. Can you give specific requirements you will be marking?
Technical competence has been set at 55% as detailed in the spec requirements ''Database management skills associate to rapid design and implementation of a Microsoft Azure based solution, database specialisms, data sciences and business intelligence skills. Integration skills required for access to source data and experience of GDS standards.'' If you are successful at the first stage you will get the second stage brief which will list the full 6 questions that equate to the 55%
36. Are there any defined success criteria for the prototype?
Defined success criteria will be shared in the next stage of the procurement
37. Data Science is listed as an essential skill: are there any use cases defined around machine learning outcomes?
Additional information will be shared in the next stage of the procurement
38. Is the single multi-species system going to serve analytical reporting needs?
39. Is there a preference between IaaS and PaaS services?
Out of scope
40. Are there any defined governance policies that the delivery will need to comply to?
Additional information will be shared in the next stage of the procurement
41. Can you please clarify if resources to be used on the project can be a mix of on-shore and off-shore or is it your expectation that resources used are UK based only?
We are not adverse to a mixed model of on-shore and off-shore
42. Please confirm for the question on integration skills will the prototype and final solution be a data migration or integration and will there some kind of enterprise service bus be used for integrating data?
The data prototype will require integration to ensure the data is kept up to date during the investigation (mitigating risk to live services) and ultimately migrated to the target solution. Our internal assumption is that existing integrations can be utilised to provide overnight batch updates. Intergration of the target solution falls outside the scope of this procurement.
43. Is the Azure environment to be provided by the supplier?
The Azure environment has been procured internally and is out of scope.