https://gdsdata.blog.gov.uk/apprenticeship-applications-service-assessment-2/

Apprenticeship Applications - Service Assessment

The exemplar’s vision is to create an easy to use digital service where apprenticeships can be advertised and applied for, with the transaction supported by clear information to inspire and advise users, so that they can self-serve, leading to minimal additional support being required.

Department / Agency:
BIS / SFA

Date of Assessment:
12/11/2014

Assessment stage:
Beta

Result of Assessment:
Pass

Lead Assessor:
R. Reynolds

Service Manager:
G. Tucker

Digital Leader:
T. Knighton


Assessment Report

The Apprenticeship Applications service is seeking permission to launch on service.gov.uk as a Beta service.

Outcome of service assessment

After consideration the assessment panel has concluded the service is mostly on track to meet the 26 points of the Digital by Default Service Standard at this stage of development, subject to some issues being addressed before the service can proceed to launch as a Beta service on a service.gov.uk domain.

Reasons

The service team are building the service based on user needs and iterating the service based on user research, which they have conducted with end users including assisted digital users. The team is working in an agile way, and are able to adapt and improve the processes as they go. The assessment panel are confident that the service team will continue to iterate and improve the service as they move into the next stage of development and that they have a plan in place to deploy frequently.

The service team has undertaken extensive research to find assisted digital users for their service through third parties, including charities, colleges and other government agencies. They have demonstrated a good understanding of barriers to using the service and have incorporated these into their personas, as well as plotting users on the digital inclusion scale. The proposed assisted digital user journey was clear and joined up with other government agencies. The team has iterated the digital service based on feedback from assisted digital users, which will support digital take-up. There are good plans for digital inclusion, to encourage users to complete the service independently in future. The team has plans to test their assisted digital support and estimated figures during the beta.

There are, however, some areas that need more work before the service can move to public beta, these are listed below.

Recommendations

The service team must address the points below before moving onto a service.gov.uk domain:

  • Ensure the cookie & privacy page accurately documents all cookies, including third party cookies, and that it is updated whenever the cookies set by the page change. The service currently sets some third party cookies (several from google.com when maps are displayed and one from stats.matraxis.net) which are not clearly documented on the cookies page.
  • Consider whether allowing third parties to set cookies (i.e. from stats.matraxis.net and google.com) is good for users. Third party cookies are intrusive, and can be used to track a user from one site to another. Use and inclusion of third party tools within your service e.g. mapping and analytics raise a number of security and privacy concerns. The service team must have a robust approach for evaluating these risks and for making subsequent decisions. The service manager should be involved in and understand these decisions.
  • Ensure that visitor IP collected by the analytics package is anonymised at the time of collection. (The assessment panel believes that currently, although visitor IP addresses are masked in service reports, SFA, Matraxis and Matraxis’ hosting company all have access to the full IP address of each visitor in the raw logs.) See https://www.gov.uk/service-manual/making-software/analytics-tools.html#privacy "anonymise IP addresses that your analytics provider collects" for the guidance and http://kb.webtrends.com/articles/How_To/Excluding-visitor-IP-addresses-from-data-collection-1365447689565 for details of how IP addresses can be excluded from collection using when WebTrends.
  • Ensure there are clear plans for publishing open source code in the next stage of development. The service team should also arrange for source code to be moved from https://github.com/Valtech-NAS to an organisation that doesn't include the name of the service supplier in order to avoid confusion over ownership of the source code.
  • The service team must agree an approach with the GOV.UK team regarding start and done pages for the service, and request any necessary content changes before public beta. (The assessment panel notes that the service team already has a planned meeting with the GOV.UK team regarding start pages and done pages to agree what happens to www.gov.uk/apply-apprenticeship and whether a done page is needed, etc.)

Additionally:

  • The assessment panel has some suggestions about some design considerations of the front end, which the design assessor will pass on.
  • Consider KPIs carefully, bearing in mind that the overall success of the service isn't defined by the proportion of users who complete their application. Consider whether overall successful completion, overall cost per apprenticeship filled, or a similar metric would be a useful additional key performance indicator for the service.
  • The service is in two parts (finding apprenticeships and applying for them) and the service team has already had to seek exemption from the normal https://www.gov.uk/service-manual/operations/operating-servicegovuk-subdomains guidance on allowing search engines to index the service. The service team will need to discuss the long term architecture of the service with the GOV.UK team during public beta, especially whether apprenticeship listings should eventually be information pages on www.gov.uk with only the registration and application part of the service being the service.gov.uk transaction.
  • In order to pass a live assessment the team must be able to demonstrate that they are sufficiently able to continue to iterate and improve the service without being dependent on one particular supplier.

Digital by Default Service Standard criteria

Criteria Passed Criteria Passed
1 Yes 2 Yes
3 Yes 4 Yes
5 Yes 6 Yes
7 Yes 8 Yes
9 Yes 10 Yes
11 Yes 12 Yes
13 Yes 14 Yes
15 Yes 16 Yes
17 Yes 18 Yes
19 Yes 20 Yes
21 Yes 22 Yes
23 Yes 24 Yes
25 Yes 26 Yes