Education and Skills Funding Agency

Data Science - Data Centric Testing Service

Incomplete applications

7
Incomplete applications
6 SME, 1 large

Completed applications

10
Completed applications
5 SME, 5 large
Important dates
Opportunity attribute name Opportunity attribute value
Published Friday 1 February 2019
Deadline for asking questions Friday 8 February 2019 at 11:59pm GMT
Closing date for applications Friday 15 February 2019 at 11:59pm GMT

Overview

Overview
Opportunity attribute name Opportunity attribute value
Summary of the work Data Centric Testing Services to work in fast-paced agile delivery teams in a Data Science environment. Assuring the end-to-end dataflows, for MSDT BI Stack development (DWH design/development/ETL/OLTP/OLAP). The individuals put forward for work must have the essential experience, knowledge and technical skills set out in this proposal.
Latest start date Monday 1 April 2019
Expected contract length 2 years, but may be shorter depending on progress to deliver the service.
Location West Midlands
Organisation the work is for Education and Skills Funding Agency
Budget range The budget could range up to a maximum of £4.0m consisting of multiple SoWs, but that scope could expand or contract. DfE does not commit to any minimum spend at this point. A pricing template will be given at the proposal stage to those suppliers we shortlist.

About the work

About the work
Opportunity attribute name Opportunity attribute value
Why the work is being done The ESFA Data Collections systems are used to collect financial, participation, attainment and building information from Academy Trusts, Local Authorities, FE and Apprenticeship Providers and Diocese to inform its funding allocations, payments, services and market oversight remit. The data is collected through bulk upload, online forms and document exchange platforms. As Data Collections systems are the source and starting point of Data Flow in Data Science, Assurance of Data Flows is required at the point of collection, data storage, data transformation & reporting to meet quality assurance requirements and high quality product delivery demanded by GDS assessments
Problem to be solved During 2018 and 2019, ESFA is upgrading its data collections software, consolidating its platforms and upgrading the software to improve the quality and timeliness of data and reduce the burden on its data suppliers.
The ESFA is also consolidating its data storage and data management systems and automating the data flows to reduce the reliance on manual processes to transform, upload and manipulate its data for its data scientists.
We need to assure that data is processed correctly and data is reported correctly, poor output will impact users referenced above and risks financial and reputational damage of ESFA and DfE.
Who the users are and what they need to do As a provider of Primary/Secondary/Tertiary education, I need to submit financial returns to the DfE so that our financial status can be established for approved funding and the DfE can provide accurate reports on our Funding Allocations/ Achievement Rates/ Risk Assessment / Qualitative Reports.

As a DfE civil servant, I need to support critical services and support the development and improvement of essential Data Collections systems so that our users above recieve the highest quality ESFA Systems.
Early market engagement
Any work that’s already been done
Existing team The supplier will be working with civil servants and multiple suppliers across various strands of work for delivery of the service. Typically within Agile delivery teams, working with Product Owners, Delivery Managers, Business Analysts, Developers, other Test Engineers and Test Analysts
Current phase Not applicable

Work setup

Work setup
Opportunity attribute name Opportunity attribute value
Address where the work will take place Education and Skills Funding Agency, Cheylesmore House, 5 Quinton Road, Coventry CV1 2WT, with occasional travel to other locations nationwide.
Working arrangements Onsite full-time 5 days a week co-located with the rest of the team at the ESFA offices in Coventry. Occasional travel to other ESFA (or partner organisation) offices may be required, subject to ESFA travel and subsistence policy. This programme uses agile methodology. Successful application of agile principles and techniques requires that the teams are co-located and able to work directly with colleagues representing digital delivery, operational and policy requirements.
Security clearance The successful supplier must be able to demonstrate that all proposed team members have been subject to Baseline Personnel Security Standards checks. However higher levels of Security Clearance maybe required dependant on work agreed in the Data Science Programme of work.

Additional information

Additional information
Opportunity attribute name Opportunity attribute value
Additional terms and conditions Expenses must be pre-agreed and comply with DfE Travel and Subsistence Policy. Any expenses shall be submitted in line with DfE standard T&C policy. Primary work location stated in SoW will not attract expenses. No parking is available on site.

Skills and experience

Buyers will use the essential and nice-to-have skills and experience to help them evaluate suppliers’ technical competence.

Skills and experience
Opportunity attribute name Opportunity attribute value
Essential skills and experience
  • At least 5 years demonstrable experience in Agile (preferably Scrum and/or Kanban) methodology to projects.
  • At least 5 years demonstrable experience in testing BI products including OLTP, ETL, SSIS, SSAS, OLAP, SSRS, Power BI.
  • Reviewing Control Flow, Data Flow, Transformations, Variables, SCDs, SCD Design, Error Handling, Transactions and Checkpoints.
  • Experience of reviewing Schema design; highlighting the advantages and disadvantages of Star vs Snowflake Schemas.
  • Use quality characteristics such as functionality, reliability, usability, efficiency, maintainability and portability helps define the context of value and level and type of testing required to meet definition of done.
  • Demonstrable experience of using a risk based approach to testing.
  • Proven experience of ETL testing
  • Proven experience of influencing design and development of a Data Warehouse
  • Proven experience of QA of MS SQL MDM
  • Proven experience of setting up Test Automation in Data Centric environments
  • Proven experience of identifying DB Performance bottlenecks
  • Proven experience of QA of DB / Package deployment configurations.
Nice-to-have skills and experience
  • Experience of working on Data Migrations/ Data Warehouses / ETL / MSDT
  • Strong understanding of the Government Digital Service Standard and Service Manual.
  • Able to contribute to areas beyond the usual boundaries of the discipline, for example helping the team understand the service’s legal and regulatory environment.
  • Knowledge of the Education and skills sector
  • Experience of open source
  • Passed certified Agile Tester course
  • ISEB foundation and practitioner qualified.
  • Having worked on a Azure platform
  • Experienced in using the following tools, Visual Studio, MSDT, MS BI Stack, SQL Server, Power BI, Tabular Model, R, R-Shiny and Python

How suppliers will be evaluated

How suppliers will be evaluated
Opportunity attribute name Opportunity attribute value
How many suppliers to evaluate 3
Proposal criteria
  • Demonstrate/provide evidence of experience and knowledge of agile testing and Data Centric skilled Quality Assurance testers (QA).
  • Demonstrate/provide evidence of experience and knowledge test automation in Data Products within CI pipeline.
  • Demonstrate/provide examples of working with delivery teams delivering on Data Warehouse implementations, from sourcing data, ETL, OLAP through to end point Dashboards and Reports in a Data Centric Testing role
  • Demonstrate/provide examples of how the supplier has worked with delivery teams delivering Data Products in a Data Centric Testing role
  • Demonstrate/provide evidence of how the supplier contributes to building an effective multidisciplinary team
  • Demonstrate/provide evidence of the supplier’s approach to incorporating user needs and government needs into the product backlog and prioritises work effectively and efficiently
  • Demonstrate/provide evidence of how the supplier actively promotes understanding and acceptance of agile, user-centred delivery
  • Demonstrate/provide evidence of how the supplier has worked in a multi-team environment with multiple product owners collaborating to deliver a broader service
Cultural fit criteria
  • Work as a team – Collaborate, establish good working relationships, generate team spirit
  • Show a commitment to ensuring a team understands the user need over simply delivering work
  • Innovate – Act on initiative, drive improvements, balance risks
  • Build capability – Encourage learning, share knowledge, address capability gaps, act on feedback
  • Demonstrate responsibility (for example, act like a leader, serve your team, provide solutions to problems, engage in external/internal environments, seek the best for the organisation beyond your team)
  • Focus on delivery – Focus on goals, set stretching targets, take responsibility for delivery
  • Approach to developing a "One Team" approach
Payment approach Capped time and materials
Assessment methods
  • Written proposal
  • Presentation
Evaluation weighting

Technical competence

50%

Cultural fit

20%

Price

30%

Questions asked by suppliers

Questions asked by suppliers
Supplier question Buyer answer
1. For the avoidance of doubt, can you please define what you mean by the acronym MSDT? Microsoft SQL Server Data Tools
2. Good morning- in which section are we able to insert our rate card, please. This will form part of the 2nd stage of the evaluation process.
3. Would you accept a mixed team of skills and experience who as a team provide the essential skills and experience you ask for, or does each individual supplied have to have those essential skills and experience. We would need all the essential criteria met where one or two representatives have been put forward to meet delivery outcomes. Where a team of 4 or more have been proposed we would accept lesser skills for 2 representatives.
4. 1) Does ESFA expects staff augmentation or complete managed testing
service?
2) Please provide the following details:
– Applications for data collection, storage and management – Tools for OLTP, OLAP, ETL, MDM and BPM (for business data corrections) – Number of existing functional test cases for applications under test
3) What is the frequency of input data (daily, weekly, monthly)?
4) Do we need to validate historical data and how many years of
historical data is there to validate?
5) Please share architectural diagram for the applications under test?
6) How many test environments would be provided for ETL testing?
We are looking for service provider to provide Managed Service for test assurance.
ESFA predominantly uses Microsoft Products and services, Technology for Applications includes, bespoke applications coded in C#, supported by MS Azure infrastructure. MS Dynamics CRM, Achieve Forms (COTS), OPA - Oracle Policy Automation (COTS) bespoke modules developed, MS SQL Server Data Tools.(SQL Server, SSIS, SSAS, SSRS, PowerBI, etc.)
Detailed requirements will be specified/agreed with provider in the Statement of Work following provider selection. We are looking for Service provider to evidence there skills and experience to meet our requirements and share there involvement/ expertise in Data Centric testing.
5. Is this requirement to provided skilled people to work under the direction of ESFA, as opposed to supplier taking responsibility for achieving project outcomes? The requirements will be specified in Statement of Work, the supplier will be responsible for the quality of work and delivery outcomes.
6. Is the work starting 1st April commencing with a very specific phase of the government delivery lifecycle (i.e Discovery, Alpha, Beta)? Alternatively is the requirements to provide under ESFA's general direction? Within the Data Science programme there are a number of projects being worked on at all different stages. Some of this work is currently in progress.
7. What role does ESFA expect will need to be deployed by a supplier on 1st April? The Service provider needs to advise on the team size following review of Statements of Work.
8. The requirement is "to work in fast-paced Agile teams" how many teams are relevant to 1st April start date? Please also describe at a high level the focus of each of these teams. 4 Agile teams are involved to develop new and enhance new data collections and data products/cubes/reports

The Data Science journey starts with
- Data Collections with Online Forms and File Collections
and moves through...
- Data Storage (Data Lake / Data Warehouse)
- Data Transformation ETL/ Modelling
- Data Visualisation - Reports/Dashboards/PowerBI/OLAP Cubes
- Statistical publications

With the range of projects ahead in Data Science for the next financial year, we will need support for future projects.
9. Can you please share a copy of the DfE Travel and Subsistence Policy, or explain the key elements? Expenses can only be claimed for meetings such as GDS assessments. Must be included in the RFQ and PO. Applied at the managers discretion.
10. What higher levels of Security Clearance (above BPSS) will be required to be deployed on 1st April? If so for which roles? None - but requirements of the Baseline Personnel Security Standard must be met prior to the start of work.
11. What level of security clearance above BPSS would be required to work in the Data Science programme of work? None, BPSS only.
12. To what degree is this work providing assurance of end-to-end data flows across existing systems, compared to providing assurance over new build? 60/40
13. Assurance of end-to end data flows is required. Which elements of the end-to end data flow already exist in some form i.e Data Collection This will be a mix depending on the team/project. In some projects the data collection will be new, but the downstream process will use existing systems. In other projects the collection method will be the same, but the subsequent data transformation/processing may be different. We would expect the testing resource to highlight the areas of the full end to end process that would require testing (based on risk/what has changes etc.)
14. ESFA are procuring multiple workstreams. Please explain the structure in place to provide delivery governance and technical governance between different workstreams? We don’t evisage that the two workstreams will overlap in this sense. Data testing tends to be separate from the digital/software testing as the products are developed in isolation. Each project/team will have it's own governance structure in place. The respective delivery manager will be responsible for reporting progress up the chain and ensuring co-ordination with any other related/dependent projects/teams.
15. What is the high-level delivery roadmap for this requirement? i.e Are there different workstreams, are there specific stages (Discovery, Alpha, Beta) and approximate schedule. The Data Science journey starts with
- Data Collections with Online Forms and File Collections
and moves through...
- Data Storage (Data Lake / Data Warehouse)
- Data Transformation ETL/ Modelling
- Data Visualisation - Reports/Dashboards/PowerBI/OLAP Cubes
- Statistical publications

There are different workstreams and stages

We need Service Providers to evidence the skills and experience described within.

The chosen service provider will be able to respond to SOW with the required detail.
16. Can we ask what criteria ESFA will be using to judge whether an evidence 'exceeded' requirements and therefore scores full marks (3)? Each question will be scored in line with standard DOS methodology outlined on the CCS website.
17. ESFA are procuring multiple workstreams in parallel, will ESFA reconsider staggering the procurement schedule? Change in the schedule is not planned, but we will inform you if this changes.
18. Will scoring from the evidence stage be taken through to final evaluation scoring? As opposed to being used solely for the purpose of shortlisting? Yes scoring will be taken through to the evaluation round.
19. What stages of the procurement timeline does ESFA expect between 15th February and 1st April? In particular are there planned dates for proposal submission and presentation Shortlisting will take place w/c 18th February, with evaluation panels scheduled for the first week of March.
20. Can you confirm the associated commercial timelines? Please see question 19