Human Factors / usability evaluation of the internet based electronic-COSHH-Essentials system

Harpur Hill, Buxton
Derbyshire, SK17 9JN
T: +44 (0)1298 218000
F: +44 (0)1298 218590
W: www.hsl.gov.uk
Human Factors / usability evaluation of the
internet based electronic-COSHH-Essentials
system
HSL/2007/60
Project Leader: Malcolm Cope
Author(s): Malcolm Cope
Science Group: Human Factors
© Crown copyright (2007)
CONTENTS
1
Introduction
6
1.1
1.1.1
1.1.2
1.1.3
1.1.4
1.1.5
Background to this report .................................................................................. 6
The HSE COSHH-Essentials system..............................................................................6
Purpose of this report ......................................................................................................6
Scope of the report ..........................................................................................................6
Previous summary report ................................................................................................7
Some background on the paper based system.................................................................7
1.2
1.2.1
1.2.2
1.2.3
1.2.4
User centred design .......................................................................................... 7
ISO standards for developing software systems .............................................................7
Terms and definitions .....................................................................................................8
Potential benefits from evaluation & a user-centred design approach............................8
Background on the evaluation process............................................................................9
1.3
Caveats for using this document ..................................................................... 10
2
Methods
2.1
2.1.1
2.1.2
2.1.3
2.1.4
2.1.5
2.1.6
2.1.7
Choice of methods........................................................................................... 11
Appropriate methods.....................................................................................................11
Requirements / context of use analysis .........................................................................12
Evaluation against requirements ...................................................................................12
Cognitive walkthrough..................................................................................................12
Heuristic & expert evaluation .......................................................................................12
Content analysis of user feedback e-mails ....................................................................12
Site structure and page request mapping.......................................................................13
3
Findings
3.1
Structure of the findings section ...................................................................... 14
3.2
Findings from user feedback analysis ............................................................. 14
3.3
3.3.1
3.3.2
3.3.3
3.3.4
3.3.5
3.3.6
3.3.7
3.3.8
3.3.9
Findings from page use analysis ..................................................................... 17
‘Process & Tasks’ page.................................................................................................17
‘How many chemicals are you using’ page ..................................................................18
The ‘How harmful’ page (Risk phrase entry) ...............................................................20
The ‘Hazard Group’ page .............................................................................................21
Dustiness / Volatility page ............................................................................................22
The ‘How much are you using and how often’ page ....................................................23
Page request counts in the multiple tasks loop .............................................................24
Use of the Help pages ...................................................................................................25
Page requests on the industry specific or ‘Direct’ advice route....................................26
3.4
3.4.1
3.4.2
3.4.3
Findings from requirements analyses.............................................................. 27
Project summary ...........................................................................................................27
Context of use ...............................................................................................................28
Requirements analyses..................................................................................................30
3.5
3.5.1
3.5.2
Findings from cognitive walkthrough ............................................................... 32
The subjects ..................................................................................................................32
The findings ..................................................................................................................32
3.6
3.6.1
Findings from expert & heuristics Evaluation .................................................. 33
The evaluators...............................................................................................................33
11
14
1
3.6.2
The findings ..................................................................................................................33
3.7
3.7.1
3.7.2
3.7.3
3.7.4
3.7.5
3.7.6
3.7.7
3.7.8
3.7.9
Summary discussions on the findings ............................................................. 33
Cumulation of issues creating an overall load on the user............................................33
The ‘Process’ & ‘Task’ based approach (substance-based advice) ..............................35
Structuring of the “Direct advice” (industry-specific) area ..........................................39
Front-end of the system and entry to the ‘Direct’ and ‘Assessment’ advice routes......41
Provision of and access to essential learning & background ........................................42
The Help system ...........................................................................................................43
Fitting in with user circumstances ................................................................................44
Other Usability issues ...................................................................................................46
Maintenance & development issues..............................................................................49
3.8
Conclusions ..................................................................................................... 49
4
Conceptual solutions
4.1
4.1.1
Substance & task specific advice route ........................................................... 51
Structuring ....................................................................................................................51
4.2
Industry specific ‘Direct’ advice route .............................................................. 56
4.3
Concept for integrating the two advice routes ................................................. 56
4.4
Concept for revising the selection of tasks in the existing system
structure........................................................................................................... 59
4.5
Provision of help and learning ......................................................................... 61
5
APPENDICES
5.1
Appendix 1: Analysis of user feddback (from 19-12-2001 to 23-082005) ............................................................................................................... 62
5.2
Appendix 2: Page request statistics applied to site structure:
Substance & task specific advice .................................................................... 66
5.3
Appendix 3: site Page structure: Industry specific advice route ...................... 70
5.4
Appendix 4: User requirements analyses ........................................................ 73
5.5
5.5.1
5.5.2
Appendix 5: List of issues ................................................................................ 78
Appendix 5.1: Expert/heuristics evaluations ................................................................78
Appendix 5.2: Cognitive walkthrough sessions..........................................................130
5.6
Appendix 6: Usability heuristics list ............................................................... 155
5.7
Appendix 7: ISO 13407 / TR 18529 software development process ............. 157
5.8
Appendix 8: A plan for a User-Centred Design approach.............................. 158
5.9
Appendix 9: Visitor access counts (July 2005) .............................................. 159
5.10
Appendix 10: Page requests for Industry Advice sheets ............................... 161
6
REFERENCES
51
62
162
2
EXECUTIVE SUMMARY
Objectives
•
To perform an evaluation of the usability issues of the web-based COSHH-Essentials
system currently running at the http://www.coshh-essentials.org.uk/ site address.
•
To describe recommendations for resolving usability issues, and present possible
conceptual solutions for meeting usability needs.
Main Findings
1. The paper based COSHH-Essentials system apparently has some important usability
advantages over the web-based system, including:
a. Adoption of a risk assessment approach, which is based on substances, rather
than on tasks performed with substances. This approach would appear to better
meet with the expectations of users.
b. Provision of an overview of what COSHH Essentials can provide, so that user
expectations and understanding are correct at the outset.
c. Provision of flexibility for the user to return to an assessment exactly where
they left off, if they need to interrupt their use of COSHH Essentials.
2. A large number of issues with the system arose, which are listed in Section 3 of the
report and in Appendix 5.
3. The task-based approach to assessment can:
a. Confuse users who are thinking in terms of substances, not tasks;
b. Result in higher level control advice being given for lower hazard rated
substances where multiple substances are entered, and prevent advice about
skin contact being applied to just the substance that it applies to, when multiple
substances are entered.
c. Require the user to re-enter the same substance information for each different
task performed with the same substance;
d. Apparently cause problems if different quantity ranges are selected for different
substances in the same task, leading to a conflict in providing control advice.
4. Asking the user to (optionally) enter a ‘Process’ can potentially:
a. Confuse the user as to why it is optional;
b. Cause the user to think in too broad terms and try to deal with too many tasks
and substances at once.
3
5. The Help and learning systems:
a. Do not support the user well in understanding how to use the system to meet
their particular needs and scenarios of substance use;
b. Do not currently support predictable user requirements very well, especially in
terms of making the system understandable to the user by making it clear how
it will meet their various needs and goals (or not), and where to go if the system
does not meet their particular needs.
c. Do not cater for the different user types, with different learning objectives and
existing levels of knowledge about COSHH and COSHH Essentials;
d. Are not cascaded, to provide deeper levels of help and learning as required.
6. The system structure:
a. Does not support separate advice for multiple substances entered under one
task;
b. Does not support the assessment of a solid and a liquid under the same task;
c. Does not support multiple-sessions very well;
d. Does not fit with expectations of how the system will be structured;
e. Does not support the user well in understanding the system and creating a
useful mental model for how it will function and meet their needs and goals;
f. Has unnecessary steps in the ‘Direct advice’ route, which could be streamlined.
7. The interface:
a. Does not fit users’ needs for entering the duration and frequency of tasks, or
certain combinations of liquid and solid;
b. Has too much text presented to the user, which could be simplified or made
more context-sensitive;
c. Has some terms which present issues, such as ‘Direct advice’, ‘Assessment’,
‘Substance’, and ‘Chemical’;
d. Returns pages too slowly in places, making the ‘Direct advice’ route very slow
to use;
e. Is unresponsive in places, providing no indication to the user that anything is
happening after clicking the ‘Go’ button;
f. Does not make it clear when the user is within a repeating loop for entering
multiple substances or multiple task assessments;
g. Does not follow recommended layout and design characteristics in a number of
places.
4
Recommendations
The issues listed in section 3 Findings, and in Appendix 5, would require review and
prioritisation by COSHH Essentials experts and system developers before being addressed.
It needs to be made clear to the user exactly what goals the system will meet in terms of
providing information, performing risk assessment, providing control solutions and where eCOSHH-E fits into HSE’s 8 steps to comply with COSHH.
If this report is skim-read, it is recommended that that the reader takes note of section 1.3,
‘Caveats for using this document’.
5
1
INTRODUCTION
1.1
BACKGROUND TO THIS REPORT
1.1.1
The HSE COSHH-Essentials system
The Control of Substances Hazardous to Health Regulations (COSHH) requires organisations to
assess & control risks from the chemicals they deal with. The Health & Safety Executive have
produced a risk assessment tool and pack of information sheets to help organisations achieve
this, called COSHH Essentials. The pack includes a wide variety of information sheets that are
relevant to different industries and circumstances, and so the pack guides the user through an
assessment of their substances, leading to selection of control measures and advice sheets that
may be relevant.
In order to make the advice more widely available, the information sheets have been put onto a
publicly accessible web site called ‘electronic COSHH Essentials’ (e-COSHH-E). This webbased system also has a modified risk assessment tool for helping users to select the information
sheets relevant to them. The system acts as a ‘front-end’ to the pool of information sheets, to
make it easier for the individuals who are responsible for chemicals safety in a workplace to
access the information they need, and have a record of risk assessment.
1.1.2
Purpose of this report
This report describes a human factors input to the development of the user interface of the
e-COSHH-E system. More specifically, this report describes:
•
An evaluation of the usability issues of the existing system
•
Describes recommendations for resolving usability issues, and
•
Presents some concepts for meeting usability needs
1.1.3
Scope of the report
This report deals with:
•
All aspects of the user interaction with the e-COSHH-E assessment system, which leads
up to obtaining advice sheets
The report does not deal with:
•
Content of the Assessment Summary pages
•
Advice sheets content or design
6
1.1.4
Previous summary report
A discussion of the main findings of this work was produced in a preliminary summary report
(HSL report ERG/06/23).
1.1.5
Some background on the paper based system
The paper based COSHH Essentials system, which preceded the internet based system, has
several differences from the electronic system:
•
The paper based system gets the user to identify their substance first, and then the tasks
that are performed with the substance are identified later, as opposed to identifying
tasks and then the substances that are used in those tasks.
•
The paper system bases the assessment around supplied single substances or mixtures,
and does not attempt to deal with mixtures created by the user.
•
The paper system does not include the frequency of the task in the calculation of the
control approach, whereas the electronic system lowers the control approach by one
level if the task takes place less than 30mins per day.
•
The paper system allows the user to have an overview of the system very rapidly,
because they can handle and flick back and forth through the paper system very easily
and at will.
•
The paper system tends to provide important information about the system integrated
with the assessment steps, providing additional clues to the user about where they are
heading and what the system will provide, which is less the case with the electronic
system.
•
The graphical nature of the paper system provides additional information to the user,
which is not so easily provided by the electronic medium.
•
The steps that are necessary after completing an assessment (i.e. Step 5) are more
clearly part of the whole process in the paper system, whereas with the electronic
system the final steps are less integrated with the assessment process, being contained in
the advice summary sheet PDF document.
1.2
USER CENTRED DESIGN
1.2.1
ISO standards for developing software systems
ISO 13407 and ISO-TR (Technical Report) 18529 map out a strategy for developing systems
such as software applications. The strategy aims to ensure that the final system will be as useful
and useable as possible. This process is summarised in Figure 1 and described in more detail in
Appendix 7.
This report aims to understand some of the context of use issues with the electronic COSHH
Essentials system, provide specification of some requirements, evaluate the existing design and
provide some design recommendations.
7
Figure 1: ISO 18529 strategy for development
Ensure Human Centred Design (HCD) content in system strategy
Ð
Plan and manage the HCD process
Ð
Understand & specify the context of use
Ð
Specify the stakeholder and organisational requirements
Ð
Produce design solutions
Ð
Evaluate designs against requirements
Ð
Introduce and operate the system
1.2.2
Terms and definitions
Human Factors: Research into human psychological, social, physical, and biological
characteristics, and application of that information to the design, operation or use of products,
systems or environments for optimising human performance, health, safety, comfort and
pleasure. syn. ergonomics.
Usability: (From ISO 9241 series) The extent to which a product can be used by specified users
to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of
use. Or: (From ISO/IEC 9126-1) Usability: the capability of the software product to be
understood, learned, used and attractive to the user, when used under specified conditions.
Accessibility: A component part of ‘Usability’, the ‘accessibility’ of an interface is it’s ability to
meet the requirements of particular sub-sets of the user population who have particular needs,
for example because of impaired vision, movement or hearing, or because their first language is
different from the ‘first’ language of the system being used.
User-Centred Design (UCD): An approach that systematically elicits the needs of the
stakeholders involved in a design, and then builds a design based around those needs.
1.2.3
Potential benefits from evaluation & a user-centred design approach
Conducting human factors evaluations as part of a user centred design approach to the
development of the e-COSHH-E system, can provide the following benefits:
•
Keep conformity with relevant BSI/ISO standards. Relevant standards which a
quality software product might want to conform to are: ISO 9241 (Parts 10-17) which
has guidance on usability and interface components; ISO/IEC 9126 (Parts 2-4) which
describes the qualities of good systems and their evaluation; ISO 14915 which contains
advice on user interfaces, and BS 7830, ISO/IEC 9127, ISO/IEC 15910 and ISO/IEC
WD18019 which have advice relevant to user help systems and documentation. These
8
standards make extensive use of human factors principles with the aim of producing a
high quality product. ISO 18529 and ISO 13407 provide guidance on the correct way to
develop software applications so that they are designed around the needs of the
stakeholders, rather than being focussed on technical development.
•
Maximise acceptance and use of the system by users. The end users of a system can
reject it or make less use of it than was hoped if it does not meet their needs. Evaluation
helps to discover their needs and ensure the final system meets those needs. HSE has a
core objective to provide information, guidance and support to help create safer
working environments; the more the system is used, then the more HSE will be
supporting its core objectives.
•
Achieve the best cost-benefit ratio from the system. Software development is
expensive and inevitably involves some compromises. Evaluation can help ensure that
time and effort is put into the right places, so that the expense of development will
provide the best system possible within the budget.
•
Maintain a positive image for HSE. HSE knows that its image is important, and that
it needs to provide help and support as well as enforcement of Health & Safety. There
are potentially a great number of people who could benefit from the e-COSHH-E
system, and evaluation can help to maximise the positive acceptance of the tool.
•
Produce a system that will have a long life cycle. If a system is to be as useful as
possible it must remain relevant and up to date in terms of its content, usability and
image. Evaluation can help to identify these requirements and ensure that the final
system can be appropriately maintained for a long and useful life.
1.2.4
Background on the evaluation process
Evaluation can be carried out as part of a whole strategy for including human factors throughout
the development of software systems, as described by the ISO-TR 18529 and ISO 13407
standards. Evaluation is normally part of an iterative process of research, design, building and
testing which carries on until the system meets a set of requirements, and so is ready for release
and use.
Evaluation can be performed by seeing if the system meets a set of requirements that were
defined at an earlier stage in the project. This can be achieved by careful thought and
consultation with the people who will ultimately use, maintain and create the system. To do this
it is important to understand the ‘context of use’ of the system. This involves establishing such
things as the goals, capabilities and attitudes of the people who will use it, what computer
hardware will be used and under what circumstances the system will be used. In addition, many
general requirements for software are now so well established that they don’t need to be
formally defined at the beginning of the project, they just need to be checked for during the
evaluations, and any deficiencies diagnosed.
Because there can be so many system requirements to keep track of, it can be difficult to make
sure that the final system meets all the important requirements. It has been common for systems
to fail because the burden of technical development has pushed the requirements into the
background. For this reason, ISO standards for software systems development include formal
analysis, prioritisation and recording of requirements so that they can be tracked through to the
finished system.
9
Different types of evaluation are appropriate at different stages of development: earlier
evaluations work with prototypes and find broad and specific items of concern, whilst later
evaluations work with nearly-finished products and look at performance and user satisfaction in
greater detail. Evaluation can continue even after the system has gone ‘live’ so that any
problems or potential improvements can be picked up and considered for further developments.
1.3
CAVEATS FOR USING THIS DOCUMENT
The following points need to be borne in mind when reading this document, or applying any of
the findings or recommendations.
•
Evaluations inevitably appear negative because they are looking for issues, so this must
be kept in mind. All systems have their issues, and it is common for a certain number to
remain within the system because of the cost-benefit ratio of dealing with them.
•
An attempt has been made to rate the importance of each issue, however these should
remain open for review by HSE stakeholders.
•
Although solutions have been offered in places, these have not been through a testing
phase to check that they are the best solutions or suitable – they are open for review.
•
ISO recommended procedures describe an iterative ‘user-centred’ design process of
design-evaluation-redesign-evaluation etc. until a certain level of acceptable
performance has been reached. This document reports one evaluation-redesign cycle
and does not necessarily provide a final solution to all usability issues with the system.
•
The e-COSHH-E system is attempting to perform very technically & interactively
complex operations; it is generally recognised that it is very difficult for the technical
developers of a system to additionally deal with usability issues which is why usability
is now often given to a separate team to deal with. The focus of this report is usability
issues, not COSHH technical issues, but some technical issues are raised because they
have an impact on the user & their goals.
•
Regardless of any issues with the current e-COSHH-E system, the principle of internet
access to the advice sheets enables users to pay only for the cost of the internet
connection and the time to perform a risk assessment and download and print the sheets
they need. In contrast, with the paper based COSHH Essentials system, the user must
purchase the entire set of advice sheets at a cost of £30.00, many of which they may
never need.
•
The terminology in this report assumes a high degree of familiarity with the
e-COSHH-E system, the paper-based COSHH Essentials and the requirements of
COSHH Regulations.
10
2
2.1
CHOICE OF METHODS
2.1.1
Appropriate methods
METHODS
Appropriate methods were selected taking into account:
•
The current stage of development of the e-COSHH-E system, (that is, having been
running live for a number of years, but now having the opportunity for review);
•
ISO standards and Human Factors approaches to Human-Machine Interaction;
•
An appraisal of what Human Factors input might be appropriate at this stage, after some
consideration of what a plan for user centred design of e-COSHH-E might look like (as
presented in Appendix 8).
Appropriate evaluation methods were found to be:
•
Requirements / context of use analysis
•
Evaluation against requirements (testing against specific user requirements for the
e-COSHH-Essentials system)
•
Cognitive walkthrough (putting yourself in user’s shoes & their scenarios of use)
•
Heuristic evaluation (testing against established standards & norms)
•
Expert evaluation (testing against general Human Factors principles)
•
Content analysis of user feedback via e-mail
•
Mapping out of the web site structure with an analysis of web page access
ISO/IEC 14598 and BS/ISO/IEC 9126 provide details of structured software evaluation
methods. These were taken into consideration for the work described in this report, but it was
considered that a more customised and appropriate approach could be adopted which would
better meet the needs of HSE.
ISO/IEC 25062:2006 provides a ‘Common Industry Format’ (CIF) for usability test reports,
however this format was considered to be more appropriate for user trials than for the evaluation
methods employed in this work. The ISO CIF format also focuses on producing statistics rather
than detailed descriptions of specific issues, which was not considered to be of great benefit to
the current work.
11
2.1.2
Requirements / context of use analysis
The aims of requirements analyses are to systematically record the requirements that
stakeholders have of a planned system. This involves describing the characteristics of the
intended user groups, and the circumstances in which they will have to use the new system.
These circumstances include the computer equipment they will be using, the physical
environment in which they will be operating the computer, and the job and organisational
context in which they will be operating. Part of this ‘context of use’ will be the various goals
that different users will have when they use the new system.
The outputs from this process are usually a set of tables that describe all the users and the
context of use issues, and the resultant system requirements that the system will need to meet.
These tables can be found in Section 3.4.
2.1.3
Evaluation against requirements
The establishment of a set of system requirements allows the design and performance of
e-COSHH-E to be evaluated against something. The output is a set of descriptions of exactly
where the system does not meet certain requirements. This is usually accompanied by
suggestions for changes to meet the requirements.
2.1.4
Cognitive walkthrough
The aim of Cognitive Walkthrough is to have subjects use the e-COSHH-E system, putting
themselves in the shoes of the different types of users. Typical scenarios of use are simulated,
and the operator relates out loud what is going through their mind as they use the system to try
and meet their goals. This method is very useful for gaining an insight into what goes through a
user’s mind when trying to use the system, showing up areas where the system has issues, and
also potentially showing up new user requirements. The outputs from this method are a set of
notes on issues that can be compared against requirements.
2.1.5
Heuristic & expert evaluation
There are many established heuristics for the way software should look and behave (for
example, Nielsen, 1994b), and these were used as measures for the evaluation. A list of the
guiding principles and heuristics that were used is presented in Appendix 7. Heuristic
evaluation is accepted as a constructive and cost-effective method of evaluation. The heuristic
evaluation was combined with an expert evaluation method, where the system was evaluated
against established human factors principles.
2.1.6
Content analysis of user feedback e-mails
The user feedback and enquiries that has been collected on e-COSHH-E provides a useful
opportunity to back up the laboratory-based evaluations of the system with real user
experiences. The feedback was analysed for issues, with the main issues summarised and
linked up with the requirements analyses and other evaluation methods.
12
2.1.7
Site structure and page request mapping
The site data on page use provides some insight into how the system is being used. The aim
was to find where page access was different from what might be expected. The output from this
provides evidence towards how the system is really being used.
13
3
3.1
FINDINGS
STRUCTURE OF THE FINDINGS SECTION
To make the report easier to read, the findings section presents results, relevant discussion and
recommendations together, rather than having a separate section for discussion. Data is
generally placed in the Appendices. This section describes and discusses findings on:
•
Analysis of user e-mail feedback received via the e-COSHH-E web site
•
Analysis of the page access statistics
•
Analyses of user requirements / context of use
•
Cognitive walkthrough
•
Expert and heuristics evaluations
As may be expected, some similar issues were identified and are reported under more than one
evaluation method.
3.2
FINDINGS FROM USER FEEDBACK ANALYSIS
The e-COSHH-E system has supported the collection of feedback from users via e-mail since
2001. A database of approximately 140 feedback responses from 19-12-2001 to 23-08-2005
was analysed as part of the assessment of the system. The analysis summary of the user
feedback is available in Appendix 1.
Note that self-reported user feedback is often considered over-positive and ‘complainers’ may
actually report significant issues even though the numbers of complaints on the same issue are
small. It also needs to be realised that users are not expert at evaluating systems, and may not
know how to express or understand the problems they have had, especially with a system as
complex as e-COSHH-E. Because of this and due to the fact that the people who submitted
feedback were a self-selecting and relatively small sample of users, it would not be advisable to
apply statistical significance tests to the responses. It would also not be advisable to discount an
issue on the grounds that it had only been raised by one or two users, because:
•
Some issues may only be recognised by particularly insightful users, yet are important
to a significant number of other users;
•
The issue in question may result in user frustration and confusion, and be difficult for
the user to recognise or explain;
•
‘Clinical’ significance must also be considered as well as ‘statistical’ significance –
that is, the importance of the issue should be considered as well as the number of
people affected.
14
Main points from the user feedback (summarised in Appendix 1) were:
1. Task-led approach: There were several reports of people not being able to relate to the
task-led approach of e-COSHH-E. These users wanted an approach based on the
substances they are using rather than the task-led approach. One user felt the list of
tasks was too restrictive for their needs. Another user felt that the task-led approach
was too long winded when having to enter the same substance information several
times.
Recommendation: Design the task-led approach so that it supports user goals and
understanding better, or provide an optional substance based approach, or
fundamentally review the whole approach to substance assessment and selection of the
right advice sheets based on user requirements. Currently it is not clear to the users that
the limited list of tasks that is presented relate to a specific list of currently available
advice sheets. Without this explanation, the list of tasks would appear slightly arbitrary
to the user. Most relevant user requirements: The system should be rapidly
understandable to the various types of users who will access the site.
2. User expectations of advice sheets: Some users appeared to be disappointed with the
final advice sheets because they did not deal with specific substances, or dealt with
circumstances of use that they did not relate to readily. One user could not relate to the
advice because it focussed on the task rather than their substance (even though the
advice probably still applied).
Recommendation: The design of the site should ensure that users have the right
understanding of what the system provides and where it fits into their COSHH duties,
and also of the limitations of the advice offered.
3. Substance data: A number of users experienced problems with not having the
necessary boiling point / vapour pressure / volatility related information at hand, or
were unable to get hold of it. Some hoped to be able to get this information from eCOSHH-E, and would find this a useful additional service. It appeared that often it was
everyday branded substances that presented difficulties.
Recommendation: Possibly more support could be provided for sourcing information
from web sources, seeing as the user has an internet connection. Databases at, for
example,
Cornell
(http://msds.ehs.cornell.edu/msdssrch.asp)
and
Oxford
(http://physchem.ox.ac.uk/MSDS/#MSDS) universities have extensive databases, which
could be useful to the users. The site at http://www.msdssearch.com/DBLinksN.htm
lists a number of database sources. Everyday products such as bleaches can sometimes
be found on various supplier web sites.
4. The omitted risk phrases: Several users wanted to be able to enter risk phrases that
were not available on the site. The users viewed this as a problem. This may be further
compounded if they felt there was a significant health issue with the product.
Recommendation: There may be scope to manage the R-Phrases that are not included
in a different way, so that the user is reassured. This could include listing these phrases
in a separate area, providing an explanation when they are selected. The omitted phrase
could also be included in the advice summary that is generated, with an action advising
the user to manage the omitted risk issue as part of their COSHH activities. Ideally,
sources of help on the omitted risk phrase would be offered in the summary.
15
5. New and updated information: Although it was not mentioned frequently, the issue
of being aware of new and updated advice sheets is an issue. Currently users would
have to go through the entire process again to check up on this.
Recommendation: The home page should provide users with awareness that they can
obtain updates, and then provide a route to obtain new and changed advice relevant to
them. Subscription to e-mail alerts of specific updates might also help to achieve this
goal.
6. Meeting different user goals: From the feedback it appears that users approach the
system with various goals and ideas about how e-COSHH-E should be able to help
them. One user wanted a system which would tell them what action should be taken for
a specific substance, another wanted a package of advice for their particular industry.
Recommendation: The system could either meet the different needs, show how the
their needs can be met another way, or show the limits of the system.
7. Dealing with liquids and solids: There is evidence that some users might not be able
to decide whether they are using a liquid or a solid, if the substance is a paste, for
example.
Recommendation: The system should allow users to select a description of the
substance that is meaningful and accurate to them, for example ‘runny paste’ or ‘thick
paste’. The system may then internally re-interpret these entries as either liquid, solid
or ‘solid in liquid’ as necessary for correct risk assessment. This would be preferable to
providing explanations which help the user put their substance in a category that they do
not find meaningful.
8. Differences between industry and substance advice routes: One user found that the
advice provided through the industry specific route was not as thorough as that provided
through the substance-based route.
Recommendation: If the two types of advice complement each other rather than
replace each other, the possibility of combining the two into one advice package might
be worth considering.
9. Learning needs: A couple of users wanted help with understanding COSHH and what
they need to do to meet regulations.
Recommendation: e-COSHH-E will be attracting novice as well as expert COSHH
users because of the ‘Easy steps…’ message. It might therefore be useful if the site
supports clear learning routes for those that seek this, even if this is via an external site.
10. Flexibility in inputting data: There were several instances reported of users not being
able to enter data in the format that they prefer. For example, when entering the %
amount of substance in a mixture, the system does not support very small percentages.
There were also several incidences of users not being able to enter their usage
information properly in the ‘How much and how often’ page, for example when
performing a task that may occur 2 days in a fortnight.
Recommendation: Allow users to enter data in the format that they want to, even if the
system is not interested in such detail. The user will want a realistic record of their
substance use in the paperwork. The system should be designed to evaluate the input &
extract what it needs to calculate the values it needs.
16
11. Accommodating errors: One user reported system database errors when trying to add
an additional task to the assessment.
Recommendation: Because it takes users time to understand the system, and especially
the task-led approach; the system needs to tolerate the addition of extra tasks.
3.3
FINDINGS FROM PAGE USE ANALYSIS
The use of the e-COSHH-E system up to July 2005 was examined and the page access statistics
were applied to a structure diagram of the system. It has to be taken into account that:
•
The page statistics will include a certain amount of page access by non-users, such as
HSE and developers. Appendix 9 shows the top most frequent sources of visitors to the
site.
•
The type and frequency of user access could affect the significance of any conclusions
drawn from the page access counts, and this must be taken into account.
•
No attempt has been made to apply any kind of statistical significance to figures, as to
do this would be inappropriate given the type of data.
The structure diagram, which can be seen in Appendix 2 shows the page request counts, which
can be used to see where page usage dropped off. The structure diagram includes most of the
pages of the system, although some of the help screens only available via links during the
assessment routine have not been included. Possible issues detected from the page analysis are
described below.
3.3.1
‘Process & Tasks’ page
There is about a 4% increase in page request counts from the ‘Process & Tasks’ page
(UnitOperations.asp) to the ‘How many chemicals are you using’ page (StateMixDetails.asp).
This is because the StateMixDetails.asp page forms part of the looping system where users are
returned to, to complete the assessment of the next task. Figure 1 is an extract from Appendix
2, which shows the two pages described, the page counts and the return point of the ‘Task loop’.
Because the StateMixDetails.asp page is part of this repeated loop, the only way to measure the
dropout from the UnitOperations.asp page would be to examine database archives of the
assessments performed, and ascertain how many assessments involved multiple passes through
the task loop. Page requests which are due to multiple loops through the system could therefore
be calculated and removed from the total, to provide an estimate of dropout at the
UnitOperations.asp page.
17
Figure 1: Task loop return point
/StateMixDetails.asp
(‘’How Many chem…using?))
5,295
Option 1
(Number of chems…)
Option 2
(Chems in mixture…)
ChemicalCAS.asp
RPhrase.asp
HazardClassification.asp
DustinessVolatility.asp
UsageInfo.asp
Summary.asp
/UnitOperations.asp
(‘Process & Tasks’ page)
5,099
ControlApproach.asp
3,459
TASK
LOOP
Recommendation: If the assessment database exists in archive, the data is used to ascertain the
dropout from the ‘Process & Tasks’ page. Dropout from this page could then provide some
guidance as to the level of any problems with this page.
3.3.2
‘How many chemicals are you using’ page
The ‘How many chemicals are you using’ page (StateMixDetail.asp, Figure 3) has a slightly
higher count than the page before or after it (in the order of 3-4%). Figure 2 shows the page
counts, extracted from Appendix 2. The higher count than the previous ‘Unit Operations’ page
might be explained by the fact that this page is the page that the user is returned to after
completing the first task loop of the assessment.
18
Figure 2: Page count for StateMixDetails.asp page
help/ChemicalsOrProducts.htm
(Chemicals or products)
293
/StateMixDetails.asp
(‘’How Many chem…using?))
5,295
Option 1
(Number of
(Chemicals
in chems…)
mixture…)
Option 2
(Number
(ChemsofinChemicals…)
mixture…)
ChemicalCAS.asp
(…)
5,019
/UnitOperations.asp
(‘Process & Tasks’ page)
5,099
Figure 3: The StateMixDetails.asp page
The fact that the page count is higher than the next ‘Chemical or product name’ page could be
an indication that users are dropping out after the first task loop cycle, realising that they must
now re-enter the same chemical information for the next task. The higher page count could also
be a sign that some users are returning back to this page and then exiting the system, after
experiencing problems on the next page (ChemicalCAS.asp) where they must enter the
solid/liquid state of the substances used. This ties in with evidence from the user feedback and
the expert evaluations that this page can cause problems if the user needs to assess both a liquid
and a solid in the same task. The only way to get around this problem is to return to the
previous page where the number of substances is entered, and to lower the count to include only
the solids OR the liquids.
19
Further evidence that users are experiencing problems at this point comes from the fact that in
the order of 3-4% of users (almost the same amount as the page count difference) accessed the
help link on this page, possibly trying to solve their problem.
Recommendation: If the system is to remain in the current structure, the error message on the
ChemicalCAS.asp page resulting from selecting both a solid and liquid substance (Figure 4)
could inform the user that e-COSHH-E can only provide advice on liquids and solids separately.
Ideally, the user should be advised previously to separate their assessments into liquids and
solids when they define their ‘process’. If the system is to be redesigned, then this should be
resolved (it is partly a result of the task-based approach to assessment).
Figure 4: Error message after trying to enter both solid & liquid
3.3.3
The ‘How harmful’ page (Risk phrase entry)
The ‘How harmful’ page (RPhrase.asp) has a higher page request count than the preceding page
(ChemicalCAS.asp). This is probably because it forms part of the loop where one user is
dealing with several chemicals. Figure 5 below illustrates this loop, extracted from the page
structure diagram in Appendix 2.
There is a drop in page requests of around 2.5% from the ‘How harmful’ page to the ‘Hazard
Group’ page (HazardClassification.asp) which can only be explained by users dropping out at
the ‘How harmful’ page. Users may be experiencing some sort of problem at this stage, for
example not having the exact risk phrases available. They might also be confused about the
absence of certain risk phrases which e-COSHH-E doesn’t support. Because the absence of
these risk phrases are not dealt with as part of the interaction, and because users may feel the
absent phrases are important, they may be exiting the system at this point to enter the Help
system for advice, or losing confidence in the system and exiting altogether. This is supported
by some of the user feedback collected via the web site, and by the page request count of access
to the help pages (248, 3.9%, accessed the ‘RiskPhrases.htm’ page, and 304, 4.7% the
‘RPhraseList.asp’ page).
20
Figure 5: The page loop for entering more than one chemical
HazardClassification.asp
6,261
A
B
C
D
E
help/RPhraseList.asp
(here)
304
RPhrase.asp
6,426
Pg1.
skin
/RiskPhrases.htm
DustinessVolatility.asp
5,363
Volatile solid details
MIXTURE
LOOP
CHEM
LOOP
UsageInfo.asp
4,740
Volatile liquid details
Recommendation: All possible risk phrases should be included in the interface, and then those
that are not supported by e-COSHH-E should trigger help support for the user when selected.
3.3.4
The ‘Hazard Group’ page
After the ‘hazard rating page, there is a drop of over 14% into the DustinessVolatility.asp page.
Some possible reasons for this are:
1. Users have entered a category ‘E’ substance and have been advised that they might wish
to substitute their chemical before carrying on, resulting in the user exiting the system
(see Figure 6).
Figure 6: Message received from a hazard group ‘E’ substance
21
2. Users have entered the ‘expert’ override link because they consider themselves to meet
the description of an ‘expert’ i.e. they are someone “…with detailed information on the
chemical…” (see Figure 7).
Figure 7: The ‘expert override’ message
3. Users have received an ‘A’ classification, which describes their substance as a “Least
hazardous substance” and therefore may exit the system believing that they need not
continue.
4. Users entered the Help system and did not return to the current session.
Unfortunately, the page access data is not detailed enough to understand the reasons for the drop
in access in any detail, and therefore no definite conclusions can be made. However, to meet
the requirement for reducing cognitive load on the user and making the system as easy to use as
possible, it is recommended that the need to continue is made clearer. It is recommended that
‘Next’ buttons are used rather than ‘Go’ and that the interface clearly requires the user to
continue, without the need for the message to ‘..click Go…’ (see Figure 8).
Figure 8: Message to continue
3.3.5
Dustiness / Volatility page
There is a drop of nearly 12% from the Dustiness / Volatility page into the ‘How much are you
using and how often’ page (UsageInfo.asp).
This might be partly explained by the repeated entry of several substances in a mixture, because
the system loops back to the RPhrase.asp page after the DustinessVolatility.asp page. This can
be seen in Figure 5 above.
Another possible reason for the drop is that users did not have the required vapour pressure /
boiling point information to hand (from Safety Data Sheets) and therefore had to exit the
system.
Some users may be trying to enter a product with a boiling point that is below 20C and boils at
room temperature, or is, in other words, a gas. The system does not deal with gasses, so a value
of less than 20C generates an error message, as shown in Figure 9a. Some users might also be
trying to enter an operating temperature which is higher than the boiling point of the substance,
which also generates an error message (Figure 9b). The user feedback indicates that some users
found this to be an issue.
22
Figure 9: Error messages from entering out of range values in the
DustinessVolatility.asp page
a
b
The fact that access to the help advice on ‘Operating temperature’ was around 2% of the total
page requests on ‘DustinessVolatility.asp’, and access to other help items from this page was
lower still, suggests that the majority of the page drop into UsageInfo.asp was probably due to
the entry of mixtures in the ‘Mixture loop’ (Figure 5). If archives of the assessments are
available, it would be possible to find out how many mixtures were entered, which would help
to clarify what was really happening.
Recommendation: The system should either provide support for gasses that get used as liquids
in some circumstances, or prevent users of this type of substance from getting this far in the
assessment before finding that they cannot get advice for it. Users, who somehow process a
substance at above its boiling point, perhaps using it under pressure, need to understand why
their input value is unacceptable, and what they should do to assess their substance and
situation.
3.3.6
The ‘How much are you using and how often’ page
There is a large drop off of nearly 24% at the ‘How much are you using and how often’ page.
Some level of drop at this point would be expected because users with more than one substance
will be going through the chemical loop more than once before they move on to the
Summary.asp page.
However, another possible explanation for the drop in access might be because the system for
entering the frequency of doing the task (Figure 10) does not match with what the user wants to
enter. Evidence for this is provided from the user feedback and the expert evaluation. Note that
users believe the accuracy of this input to be very important, whereas in fact the risk assessment
system behind COSHH-E only needs to know if usage falls within broad ranges. Because the
user feels this is important info, but they cannot enter it as they want to, this can cause problems
for certain users.
23
Figure 10: Entry of frequency of substance use
Unfortunately, the data available is not sufficient to be able to determine how much of the
dropout is attributable to each of the explanations above. Access to the help link ‘How many
times a day’ was 3.8% of total requests to this page, which provides some indication of the
number of people who might have been experiencing problems with entering the frequency of
the task.
Recommendation: It is recommended that the system allow users to enter their actual pattern of
working, and that the system then extracts the frequency information that it needs. For
example, the user could have the option to enter frequency per day, week or month, or even
year. This is probably easier for the user to deal with than trying to understand an explanation
of why the system only deals with broad ranges.
3.3.7
Page request counts in the multiple tasks loop
The page request count for the beginning of the task loop (StateMixDetails.asp) has a count of
5,295, whereas the page request count for the final page in the loop where the advice is provided
(ControlApproach.asp) is 3,459. This represents a difference of 34%.
Some of the loss of page requests through the task loop have been discussed above. In addition
to those, taking the task loop as a whole, the difference in page requests might also be partly
explained by users completing one or two tasks, and then returning to the chemical entry pages
(StateMixDetails and ChemicalCAS.asp) and leaving the system when they find that they are
required to enter the same chemical data again for another task.
Analysis of the data entered by users in their assessments would be required to obtain a better
understanding of why the final page in the task loop has the request count that it does.
Recommendation: It is very unlikely that users find it acceptable to re-enter data on the same
substances a second time. This could be confirmed by discussing with potential or actual users,
but it could probably be assumed that it would be better to design this issue out of the system,
either by adopting a substance based approach or by allowing users to re-load data for the same
substance (and possibly omit going through the repeated interface pages).
24
3.3.8
Use of the Help pages
Requests for the ‘Which chemicals are covered’ page was about 7% of the total number of
visitors in July 05. This page had a slightly higher request rate compared with many of the
other help pages, which perhaps suggests that some users felt this information was important.
Table 1 shows the page requests for the Worked Examples in the help system pages. It can be
seen that the page requests gradually tail off toward the end of the example. This is possibly
because the users felt that they may as well enter the system for real and get on with their
assessment. It might also support the issue which was raised in the expert evaluation, that the
worked example does not add much learning opportunity to that which is gained from the
process of actually using the system for real.
Figure 11 Shows a screen capture of the help page “WorkedExample3.htm” and the associated
tutoring text at the top of the image. In this instance, the tutoring text appears to provide very
little help in understanding the important underlying operation of e-COSHH-E; this Worked
Example does not help the user to understand why it is not important to get the name of the
substance exactly right, and provides little or no learning in addition to using the ‘live’ system.
Table 1: ‘Worked Example’ page requests
Page
Request count
WorkedExample1.htm
1,383
WorkedExample2.htm
1,021
WorkedExample3.htm
939
WorkedExample4.htm
844
WorkedExample5.htm
764
WorkedExample6.htm
635
WorkedExample7.htm
582
WorkedExample8.htm
560
WorkedExample9.htm
567
25
Figure 11: The “WorkedExample3.htm” page
3.3.9
Page requests on the industry specific or ‘Direct’ advice route
Appendix 3 shows the hierarchical access routes to the various industry-specific advice sheets,
and Appendix 10 lists the number of requests for the individual advice sheets in July 05. From
a usability point of view, it would be interesting to see the requests right through the
hierarchical routing, for each of the industry areas. However, because the e-COSHH-E system
generates the routing pages for the user on request, and the page which is sent to the user always
has the same name (for example, ActivityOptions.asp) it is not possible to separate out the page
requests for individual industry areas. The individual PDF advice sheets, however, provide
some indication of the most requested industry areas.
26
3.4
FINDINGS FROM REQUIREMENTS ANALYSES
3.4.1
Project summary
This summary provides an overview of the project background, largely established from the
previous development of the system. It was helpful for the evaluators who needed to gain a
rapid overview of the system, and provides the first step for a user centred approach.
Questions
Assumptions
1.
What is the system or service?
Provision of advice on controlling chemical hazards in electronic
format
2.
What functions/services is it
intended to provide?
Risk assessment for COSHH; Appropriate selection & provision
of electronic versions of the COSHH Essentials advice sheets;
Provision of other HSE material relating to COSHH; Provision of
links to information sources on COSHH.
3.
What are the aims of the project? To provide help on controlling health risks from
chemicals/harmful substances & thereby reduce the incidence of
ill health (under the Fit3 Program). To reduce the burden of risk
assessment on organisations.
4.
Who is the system intended for? Mainly SMEs;
Larger organisations;
Providers of substances & control systems
5.
Who will use the system?
Those responsible for H&S & control of substances in
organisations;
Those who supply advice & control of substances to
organisations;
Those who supply substances to others.
6.
Why is the system needed?
Smaller organisations find it difficult to meet COSHH regs;
Risk assessment is a burden to organisations;
Performing suitable & sufficient risk assessment may be difficult
for some organisations, especially those without people who are
trained and dedicated to performing risk assessment;
Those inexperienced with COSHH may find it difficult to choose
the appropriate control measures for their workplace substances;
Paper advice sheet pack might be considered expensive by
smaller orgs;
Not all advice sheets of paper system are relevant, but have to be
paid for as a pack;
7.
Where will the system be used? Office or home computer workstation.
8.
How will the system be used?
User to be guided through a system which will perform a risk
assessment from provided information, and choose appropriate
advice for them to follow.
9.
How will the user obtain the
system?
Via internet connection.
10. How will the user learn to use
the system?
By step-by-step intuitive interface.
27
Questions
Assumptions
11. How will the system be
installed?
Pre-installed on an internet server.
12. How will the system be
maintained?
HSE to provide new & updated information & advice sheets;
Service provider to maintain user access, acceptable access speed
& access statistics;
System programmers to add in new features & advice
information as requested/supplied by HSE.
3.4.2
Context of use
3.4.2.1
Description of stakeholders
The stakeholders for the e-COSHH-Essentials system are described in Table 2.
Table 2: Stakeholder analysis
Stakeholder
Description
HSE
Major investors in the system. Possess the expert knowledge of the
safe use, storage & disposal of chemicals.
Organisations using chemicals
Cover a huge variety of industries. Vary in size from individuals to
SME to large conglomerates; Very large variation in types of
chemicals used, quantities used and circumstances of use; Very wide
range of budgets; wide range of computer types and qualities.
Organisational employees
Wide range of ages, ethnicities and abilities; Have a responsibility or
desire to get advice on chemicals; Employed; both genders; mainly
16-64; from a wide range of industries & areas of work; H&S
knowledge from nil to high; English may not be 1st language;
computer literacy may be nil to high; Different users/industries may
use different terminology and product names.
Chemicals manufacturers &
suppliers
Have detailed knowledge of their own chemicals.
System builders & maintainers
Have expertise in technology & usability; little or no knowledge of
chemicals safety.
3.4.2.2
Description of typical scenarios of use
From what is known implicitly about the user base, a top-level picture of the scenarios of use of
the e-COSHH-E system can be generated. This then forms one of the inputs to the analysis of
user requirements.
28
Typical scenarios of information need - The following scenarios of user information need can
be defined:
•
User has some or all of the relevant information that they need to control risks
associated with their substances under COSHH Regulations, but wants to confirm
whether there is additional useful information available, or wants to be reassured about
the assessment and controls they have performed themselves;
•
User is aware of their responsibilities under COSHH, knows that they need to assess the
risks associated with their substances and to implement appropriate controls, but does
not have the information they need to do this;
•
User does not know what their responsibilities are under COSHH Regulations, how to
perform risk assessment or implement controls (this may be the case with SME’s with
no full-time safety expert and no previous experience of complying with COSHH).
These scenarios are mapped out in Figure 12 below.
Figure 12: Typical scenarios of information need
Scenario
Description of user
1
2
3
User understands their responsibilities under COSHH
9
9
8
User has relevant information on their substances to be able to perform
risk assessment
9
9
8
User has relevant information on how to control risks associated with their
use of substances, and how to implement controls
9
8
8
Scenario of information need
User seeks reassurance that their assessments & controls are suitable &
sufficient, or to check for additional useful information that is available
User seeks information on suitable controls for their substances and
circumstances of use
User seeks to understand their responsibilities under COSHH, help in
performing risk assessment & choice of control measures
1
2
3
Typical scenarios for accessing the system - In addition to the above, it is useful to define
different scenarios for how the system might be accessed:
•
User wants one brief visit to get all the information needed
•
User wants to make repeated visits over time
•
First-time visit
•
Re-visit to update info / continue session
29
Typical scenarios of rejection - It is useful to define the situations where a user might reject
the system, i.e. give up using it, so that these situations can be guarded against.
•
User failed to get the information that was expected/wanted
•
System required too much time and effort to learn and use
•
System appears unfriendly, stressful or unpleasant to use
•
User loses confidence that the system is reliable, accurate and complete
•
The system failed i.e. ‘crashed’
Typical user levels of understanding of COSHH – Users will come to the e-COSHH-E
system with different levels of awareness of COSHH, and will have different learning needs.
The system will need to cater for the different learning needs of these different user types.
Broadly speaking, typical user types would be:
•
Little or no knowledge of COSHH regulations or methods
•
Aware of COSHH regulations or methods, but not expert
•
Fully conversant with COSHH and control methods - an ‘Expert’
Additionally, whether the user owns or has used the paper-based COSHH-Essentials system will
affect their knowledge levels and learning needs.
3.4.2.3
Description of user goals
Defining user goals is a fundamental step in a user centred design approach. These goals should
be added to, refined and referred back to during development, making sure that they underpin
design decisions. Typical user goals can be defined as:
•
To do what is required of them by regulations and law to control chemical hazards;
•
To protect their own health &/or of colleagues;
•
To find better ways of dealing with chemicals;
•
To get assurance about their current ways of dealing with chemicals
3.4.3
Requirements analyses
After understanding the context of use of the system, it is possible to start to define the user
requirements that will make the system perform in the specified context of use. Appendix 4
contains the user requirements analyses.
30
3.4.3.1
Top-level requirements
Table 3 lists the top-level requirements that were generated. These were the more obvious
requirements that can be defined without consultation with stakeholders.
Table 3: Top-level stakeholder requirements
Stakeholder
Top-level requirements
HSE
- To help reduce the burden of risk assessment on organisations
- Want the system to help meet core objectives of providing H&S
support in industry
- Want to help meet objectives Disease Reduction unit
- Value for money
- To disseminate COSHH-Essentials advice sheets to relevant users
across a broad range of industries and organisation types
- To look competent and helpful to the users
- To be consistent with HSE web ‘look & feel’ & to match any other
well designed HSE interfaces
- To be structured so that info can be added/updated at reasonable cost
- Users to be left with a positive impression of HSE
Organisations using chemicals
- Want a system which will help them protect their employees
- Help protect their organisation from enforcement action
- Be reliable and accurate
- Cost the organisation as little in resources as possible (time & on-line
access)
- Help to reduce the burden of risk assessment
Organisational employees
- Want the system to support them with minimum of effort
- Make life easier in dealing with chemicals in the workplace
- Be pleasant to use
- System to be quick and easy to navigate/use
- System to obey familiar norms for windows systems
- System to give them accurate, reliable and complete advice
- To know when new information/changes have occurred
- System to be reliable
- System to not adversely affect their computer
- System to be encouraging and welcoming to use
- To suffer no problems or errors, even if they ‘abuse’ system
- Language to be familiar to the user
- No need to learn the system before using effectively
Chemicals manufacturers &
suppliers
- Want a system which will support correct use, storage & disposal of
their products
System builders & maintainers
- Want a system which is viewed as successful
- Able to keep system up to date and improve without considerable redevelopment
31
3.4.3.2
Detailed requirements analyses
Detailed requirements analyses are presented in Appendix 4. The requirements are organised
into the following areas:
1. User group description – Identifies user groups and the depth to which their
requirements need to be examined.
2. User characteristics – Identifies the qualities of different user groups and their
requirements.
3. Technical environment – Identifies the technical environment in which the user groups
will operate, and the consequential requirements.
4. Psych-social & organisational environment – Identifies the attitudes and cultural and
organisational attributes of user groups, and the requirements that result from them.
5. User group goals – Identifies the goals of the user groups, taking into account all their
previously identified attributes above.
These requirements, along with those in section 3.4.3.1, were used when evaluating issues
within the system (listed in Appendix 5).
3.5
FINDINGS FROM COGNITIVE WALKTHROUGH
3.5.1
The subjects
4 evaluators, trained in Human Factors (3 of whom were completely new to the e-COSHH-E
system and one who had some awareness of the paper based COSHH-Essentials system), took
part in a cognitive walkthrough of the system. They were provided with a brief that stated the
user role that they were to play, and the goals that they were to try and achieve when using the
system.
3.5.2
The findings
Issues found during the cognitive walkthrough sessions are logged in Appendix 5.2. During the
sessions, issues arose as a result of specific events and user interactions with the system. The
issues are organised into ‘Event themes’ that are groups of events that are closely related. For
example, a series of events (and resulting issues) occurred when the users were entering the risk
phrases for their substances, and therefore these are grouped under one ‘Event theme’. The
event themes are:
1. User introducing themselves to the system
2. User at “Getting Started” page (EntryOptions.htm)
3. User at Process and tasks” page (UnitOperations.asp)
4. User entering chemical number and state
5. User entering risk phrases & obtaining hazard rating
32
6. User assessing Dustiness/Volatility and Usage
7. User at Summary page & collecting advice
8. User entering “Direct Advice” option
The issues that relate to these event themes cannot be summarised any further than they have
been in Appendix 5.2, and therefore the reader is referred to this appendix for details of the
results and resulting recommendations.
3.6
FINDINGS FROM EXPERT & HEURISTICS EVALUATION
3.6.1
The evaluators
4 human factors specialists gave their evaluation of the interface having been briefed about the
paper based system, and the aims of the electronic version. The issues were identified by the
four evaluators using heuristics, human factors principles and the user requirements.
3.6.2
The findings
The issues that were found were collated into subject areas, and are logged as presented in
Appendix 5. The subject areas are:
1. Functionality
2. Interaction & Graphical Interface
3. Structuring & navigation
4. Language
5. Error handling, error tolerance & flexibility
6. Help & Learning
The issues listed under these areas cannot be summarised any further than they have been in
Appendix 5.1, and therefore the reader is referred to this appendix for details of the results and
resulting recommendations.
3.7
SUMMARY DISCUSSIONS ON THE FINDINGS
3.7.1
Cumulation of issues creating an overall load on the user
Although many issues which were found during the evaluation were not critical to the user goal
of obtaining appropriate advice sheets, the cumulative effect of issues becomes an issue in itself
because users can experience mounting frustration resulting in rejection of the system. The
issues that can contribute to this effect include:
33
•
The ‘task’ based approach which does not match user expectations and takes effort to
understand;
•
Unexpected restrictions and inflexibilities in the system which do not match users’
goals, forcing the user to try and build a mental model of how the system functions in
order to find a ‘workaround’ to make it fit their needs;
•
Complex and text-intensive screens;
•
Help and background information which is not structured according to user needs and
goals, and in some cases not designed for on-screen viewing;
•
Some issues with language and terminology which mislead or add cognitive load to the
user;
•
Lack of distinction between links to documents, internal pages and external sites which
can adversely affect users with slow internet access;
•
Slow response times in some areas of the site, coupled with re-entering of the same data
several times, resulting in a lengthy process for some circumstances;
•
Difficulties and system errors if returning to edit or continue an ‘assessment’.
Whether or not a user reaches the final goal of performing a suitable risk assessment and
receiving suitable advice sheets will largely depend on:
1. The complexity of their particular scenario of chemical use;
2. Their particular knowledge and information requirements;
3. Their motivation to work through the system & understand it, which will be affected by
1 and 2 above.
Well motivated users or users with very simple assessment scenarios are likely to complete their
goals with a positive user experience, whereas users with multiple-task assessment scenarios or
average motivation, are not.
Recommendation: It is highly recommended that the cumulative sum of the issues above is
reduced by considering the recommendations that have been provided in sections 3, 4 and 5.5 of
this report, which deal with each of the issues listed. It is recommended that:
1. Issues are prioritised in terms of which are most obstructive to users achieving their
goals, that is, which are most critical to user goals;
2. Issues are prioritised in terms of which are likely to affect the most users;
3. Issues are prioritised in terms of how frequently they are expected to occur;
4. Issues are rated in terms of how easy/expensive they are to rectify.
These prioritisations could then be used to decide where resources should be targeted when
resolving issues.
34
3.7.2
The ‘Process’ & ‘Task’ based approach (substance-based advice)
The process and task based approach of e-COSHH-E raised several issues for users:
1. Difficulties resulting from the ‘task’ and ‘process’ terminology;
2. Expectations about the structure of the e-COSHH-E system;
3. Loss of confidence in e-COSHH-E risk assessment because of technical issues.
These issues are described in the next 3 sub-sections.
3.7.2.1
Difficulties resulting from the ‘task’ and ‘process’ terminology
The task-based approach is initially counter-intuitive for any users who have a substance in
mind and want to know what needs to be done to use it safely. When asked to think about the
‘Tasks’ that they do when carrying out a ‘Process’, users tend to be thinking about the tasks in
their job (especially having been asked to enter a ‘simple description of the job you are doing’
in the first step of assessment), however the ‘tasks’ referred to in the ‘Process & Tasks’ page
(Figure 13) relate to information sheets available on performing certain tasks with chemicals,
they do not necessarily relate to job tasks that the user has in mind.
Figure 13: List of tasks presented to the user
The user cannot understand at this stage why they are presented with a limited number of task
descriptions, most of which they need to get help on to understand. This causes confusion and
requires the user to reset their idea of what this list of tasks is all about. There seems to be
scope in helping the user to understand why there is a limited number of tasks listed, and that
they relate directly to the advice which is currently available. The paper system makes it clear
that ‘Tasks’ relate to the advice sheets available (Figure 14).
35
Figure 14: Extract from the paper COSHH-Essentials, describing ‘Tasks’
Currently the start of the system puts a fair degree of cognitive load on the user – there is scope
to reduce this load. There is a danger that users will not identify with the tasks listed and
choose the
option, thereby missing advice that might apply to them.
The user is asked to think about a name for their ‘Process’ (Figure 15). What the user chooses
as a ‘Process’ is open to interpretation – if they choose too broadly they can encounter problems
trying to cater for too many chemicals or tasks.
Figure 15: Use of the term ‘Process’ in the system
The term ‘Process’ has no boundaries for the user, even after looking at the ‘Worked example’.
This means that the user has to try to work out what the boundary is, in terms of deciding how
much and what parts of their ‘Process’ they should be dealing with at any one time. Potentially
the user can try to deal with too broad a process, which can make subsequent steps very difficult
to deal with. The system becomes easier to use if the user thinks in terms of a ‘Process’ with a
particular substance or mixture.
3.7.2.2
Structuring, expectation & mental models
Users expected the assessment to be led by the substance. Starting with tasks when they have
not entered the name of the substance generally conflicts with expectations, but can be
overcome by the user. The paper based COSHH-E adopts a substance based approach. Users
who have some familiarity with COSHH assessment and publications such as HSE’s “A step by
step guide to COSHH assessment” might find this approach even more difficult to become
familiar with because assessment generally starts by identifying substances and assessing their
potential for harm.
The task based approach can require the user to enter the same information about the same
chemicals several times over, repeated for each task that uses that substance. This was
36
unacceptable to all the evaluators, and was noted as a comment submitted via the website
feedback system. This would not be necessary with the paper based system because tasks are
listed per substance. However, after discussions with COSHH Essentials experts, it was
understood that the task-based system was adopted in order to allow the exposure from any
particular task involving several substances, to be assessed as a whole. This then allows the
Control Approach to be lowered for tasks which cause less than 30mins total exposure per day,
for several substances. This approach may not fit some scenarios, for example if the user has
selected a ‘Process’ which involves the task of ‘Transferring’ several substances, and which is
performed by several people in different locations. (When considering the task-based approach,
it might be helpful for the reader to refer to the schematics in Figure 32 ‘Existing structure of
the e-COSHH-E system’ and Figure 33, ‘Schematic of existing e-COSHH-E structure’).
Users cannot enter a liquid and a solid under the same task in an assessment. Users are unable
to create a mental model for why this should be so, because they do not realise that the system
cannot provide separate advice for each substance entered. This can cause confusion and a lot
of effort for the user in order to make the system fit their needs.
On the ‘How many chemicals are you using’ page, users are given the option to enter the
constituents of a mixture created before doing the ‘task’ (Figure 16). This option is provided for
all tasks, including ‘Mixing’ resulting in possible confusion for the user on how to deal with
mixtures. This situation is created by the task-led approach.
Figure 16: Option to enter constituents of a mixture
Recommendation: It highly recommended that the system support users in understanding the
structure of the system and how it might support their goals, because of the large potential for
frustration or failure to achieve goals without such support.
3.7.2.3
Confidence in e-COSHH-E risk assessment
The process and task based approach raised some issues which could potentially result in the
user losing confidence in the system because of apparent technical issues. Even though the
issues may not be significant from a COSHH or safety point of view, they are nevertheless of
significance to the user.
The task-based approach allows the user to list several chemicals, however the system cannot
then give separate advice for each chemical. Advice is provided for the highest control
approach required. This means a user can appear to get inappropriate advice for a substance,
which only requires a lower control approach. This potentially results in loss of confidence in
the system because the user could be recommended to over-invest in control measures for a
substance that does not require such extensive control. From a user point of view this could be
major issue if it occurs.
37
Not being able to provide advice for individual substances entered under each task has the
additional issue of making it impossible to enter a solid with a liquid in any one task. This
requires the user to rethink what they have defined as a process if they need to assess both a
liquid and solid, and then to start another separate assessment to cater for the other substance.
For the evaluation, chemicals experts were asked if combining liquids and solids is likely to be
an unreasonable occurrence, the advice received is that it is not. Feedback sent via the web site
confirmed that some users have found this a problem.
If a user enters a substance that has an ‘E’ hazard rating, then additional important warning
information is provided to the user at the point of hazard classification. This warning
information does not carry through to the downloadable Advice Summary PDF sheet, and so is
lost. If the user has entered several substances, and only one is classed ‘E’, then it becomes
impossible for the current system to tie the warning to the right substance at the end of the
assessment. This is because of the task-based approach that bundles several substances together
and uses the highest rated substance to set the control approach. However, it would seem
feasible to word the Advice Summary such that any warning advice clearly applies to the ‘E’
rated substance only. In this way, the user can continue to the end of the assessment, and have a
written record of the warning.
The ‘Summary of the Technical Basis for COSHH Essentials’ document describes how eCOSHH-E lowers the control approach required under some circumstances when the task being
performed takes less than 30mins per day. The document states that this is only applicable if
there is no further exposure. However, adopting the task based approach means that the user
can enter 30mins for several tasks with the same chemical. In theory this means that the control
approach could be lowered when the total exposure is over 30mins per day. In fact the system
does not appear to lower the approach method unless the task time drops below 15mins, which
provides some leeway. However, the discrepancy between the technical document and the
actual performance of the system could concern users who have noticed the discrepancy, and
could damage the image of the system in the eyes of those who look into the system in depth.
Theoretically there is still the possibility for several tasks of less than 15mins to add up to more
than 30mins per day, thus providing a control approach lower than it should be. This may not
be significant in safety terms because of the safety margin built into the 30min rule, but could
impact on the users impression of the system, which needs to be considered. Concerns about
this could be dealt with in the technical document.
If a user has entered more than one substance for a particular task, and then enters different
quantities for the substances (for example, Kg and tonnes) it is not clear how, or if, the system
can then provide different appropriate advice for each substance. From the experimental use
that the author gave the system, it appears that advice cannot be given for different quantities of
substances. This may lead the user to lose confidence in the system.
The paper-based system has a related issue in that different substance amounts might
conceivably be used for different tasks, but the system applies the quantity rating to all tasks.
The user can, however, get around this if they wish because they can easily apply two different
quantity ratings if they wish to, and select the right control approach and advice.
38
3.7.3
Structuring of the “Direct advice” (industry-specific) area
Appendix 3 shows the hierarchical structuring of the pages within the “Direct Advice” route to
obtaining industry-specific advice sheets, and Figure 18 shows the outline structure.
It can be seen that the structuring begins with the two categories ‘Production’ and ‘Service’
industries, as the screen extract in Figure 17 shows.
Figure 17: First structure level of the “Direct advice” route
It may be difficult for some users to decide which category is most appropriate for them,
especially if they think they could belong to either or both. It is likely to be easier for users to
directly choose their industry area from the second hierarchical level, even if the ‘Service’ and
‘Production’ categories remain. This would also remove an extra step in getting to the advice
sheets. If there is concern that the lists of industry areas could become too long to fit
comfortably on one interface screen, drop-down ‘combo’ selection boxes could be used, as
illustrated in Figure 19. This also allows users to explore the industry options that are
categorized under ‘Production’ and ‘Service’ before they make a choice.
39
Figure 18: Outline structure of the “Direct” advice route
Direct Advice
Industry selection
screen
Production Industry
Service Industry
Flour
Foundry
Rubber
Woodwork
Printing
Beauty
Car Parks
Cleaning
Florist
Funeral
Leisure
Maintenance
Pest
Pubs
Warehouse
MVR
Production Industry
Service Industry
Tasks and task selection
Tasks and task selection
Production Industry
Service Industry
COSHH sheets
Advisory sheets
Additional general
information sheets
COSHH sheets
Advisory sheets
Additional general
information sheets
Figure 19: ‘Combo’ boxes for industry selection
Production industries
Service industries
Beauty
Flour
40
3.7.4
Front-end of the system and entry to the ‘Direct’ and ‘Assessment’
advice routes
It has been previously recognised that entry to the two advice routes is not currently ideal, and
one of the specific aims of the evaluation was to deal with this issue. The following issues were
raised:
3.7.4.1
Terminology
The term ‘Direct advice’ was interpreted in a number of ways by the evaluators, none of them
correctly. The term ‘Sector/industry specific advice’ was considered to be more meaningful.
Different users have different ideas about what the term ‘Assessment’ means. Users with no
COSHH background can only view this term as ‘Assess my substances & tell me what I need to
do’. Users with more background on COSHH, however, will see the term as ‘COSHH
Assessment’ which involves (according to HSE published advice) certain steps. The term can
therefore establish the expectation that e-COSHH-E will complete the steps as suggested in
HSE advice, however the system adopts a different approach which could conflict with
expectations.
The ‘Assessment summary’ sheet which is provided at the end of an e-COSHH-E assessment
introduces the term ‘Risk assessment’ which could arguably be different from a complete
‘COSHH assessment’. Clarity in these terms could help users.
The home page implies that the site deals with ‘chemicals’, yet the advice in the site includes
other substances such as wood dusts which are not chemicals. This is an issue with the paper
based system also.
3.7.4.2
Structuring, expectation & mental models
e-COSHH-E gives the expectation that it will perform a suitable & sufficient COSHH
assessment for whoever uses it. What it actually provides is more complex than this, and is
explained better in the paper system: ‘COSHH essentials can help you control exposure and
meet your legal duties. It addresses some of the basic requirements of COSHH for some
hazardous substances…’ (authors italics). This description contains important clues for the user
about the system they are about to use. Similar important clues appear to be missing from the eCOSHH-E version.
All evaluators (and some users who provided feedback via the site) expressed disappointment
with the generic nature of the final advice. This was not necessarily because the advice was not
useful, but because the system created a different expectation – an expectation of specific
tailored advice.
41
3.7.4.3
Meeting user goals
Making predictions about user groups and their goals indicated that people will enter the site
with different goals and needs. Currently the front end of the site is not designed to match these
predicted goals.
For many users groups, the ‘Direct’ sector specific advice will be a good first step for them –
the front end could be designed to guide those people to this first. However, because the advice
here is limited to specific areas, the system must allow a rapid check of whether there is useful
advice here for the user. It is not acceptable to go through a long procedure with nothing of use
at the end. The current ‘Direct’ advice route contains unnecessary steps and is too slow to meet
this goal. The detailed report contains ideas for possible solutions to this issue.
Some groups will be accessing the site seeking advice on a particular substance – again, the
‘Direct’ advice section may provide useful information for these people, so a rapid check is
needed before guiding users into the ‘assessment’ route.
Different user groups will have very different levels of background knowledge on COSHH, and
this needs to be accounted for at the front-end of the system. This can be broken down into: 1)
Information on legal requirements of COSHH & steps to comply, and 2) Information on eCOSHH-E and what it provides in terms of COSHH. It would help meet users’ goals if access
to this background learning was separate from system ‘Help’ information.
The structure & content of the learning and help sections are dealt with in more detail in 3.3.8,
'Use of the Help pages’, 3.7.6 ‘The Help system’, 4.5 ‘Provision of help and learning’ and 5.5
‘Appendix 5: List of issues’.
3.7.5
Provision of and access to essential learning & background
3.7.5.1
Meeting user goals
A certain amount of background knowledge is essential in order to use e-COSHH-E, or at the
very least to make use of the advice it produces. The ‘context of use’ analysis which was
conducted as part of the evaluation of e-COSHH-E identified several user groups with different
learning requirements. It was found that there was room for improvement in meeting specific
learning goals.
If seeking basic information on the legal requirements of COSHH and steps required to comply,
users were unable to get this information easily without searching for and downloading a PDF
file which was mainly designed for reading on paper. When downloaded, the ‘COSHH – a brief
guide to the regulations’ (Rev 2 or 3) is very text intensive. People generally find it easier to
read through a lot of text when it is in hard copy, and dislike reading on-screen text. This means
that screen-based information and learning have different requirements from paper-based. The
provision of a COSHH ‘primer’ specifically designed for internet access would help users.
The ‘Getting started’ page provides a link to background and help information (Figure 20).
This link initially appears to fulfill a user goal, but when accessed, there is no help titled “What
the law says” or “How COSHH Essentials works”. The ‘Worked example’ shows how to use eCOSHH-E, but a broader (and graphical) overview of the steps in e-COSHH-E would be more
useful.
42
Figure 20: Link to background and help information
Obtaining background learning is a different goal from obtaining system help and therefore
should be separated; at least in the way it is accessed. Currently the e-COSHH-E system does
not deal with these different learning goals particularly well.
Although a worked example is given of how to go through the screens, there is no higher level
model provided of how COSHH-E fits in with the goals of the user. This is important because
all users appear to have a different expectation of how the system will work to the way it
actually does, and it takes some time for them to build a mental model of what is going on.
Ideally, the system would be so intuitive to use that no mental model would be required in the
user’s mind, but if a mental model is necessary, then the user should be supported in creating
that model and in understanding the system. Re-creating the screens is not hugely helpful
because the help advice that is provided is largely the same as that provided while actually at the
page.
3.7.5.2
Integration of learning into the assessment process
It is understood that one of the original aims of the e-COSHH-E system was to integrate the
process of learning about COSHH with obtaining the advice sheets, such that prior learning is
not required. It appears that initially the aim was to guide the user into obtaining the right
advice and then provide the steps to comply with COSHH as part of the final advice. This is a
very reasonable approach, and potentially could do away with the need for a COSHH ‘primer’,
but the actual implementation of this approach is not successful because of the design of the
system and the final ‘Summary Advice Sheet’.
Better support for both paths of learning ie. prior to or after collecting advice sheets, would be
ideal, and would be best achieved by using established methods for training development.
3.7.6
The Help system
A number of structuring, content and interaction issues make the help system less useful and
usable than it could be. User goals for seeking help could be more directly targeted which
would improve the user experience, for example:
•
A cascaded approach to the provision of help would enable users to obtain deeper levels
of help if they need it, and stop receiving help information as soon as they have obtained
the understanding they need. In other words, if help is provided in the smallest doses
possible, with links to deeper levels of help, then the user can receive only the level of
help information that they need.
•
Help which is accessed from the top or side menu buttons on the assessment pages, is
not context sensitive. That is to say, it does not bring up help on the current page, so the
user must search for it. Context sensitive help is therefore recommended.
43
Use of hyperlinks for retrieving pop-up help is generally initially unintuitive for users, but does
not cause huge issues. If a ‘?’ cursor could be implemented (
Figure 21) when the mouse is hovering over a help link (as used on some HSE sites) then this
could reduce unwanted clicks.
There are some issues with the wording and content of the help, which could be resolved by an
expert in the provision of on-line help.
Figure 21: 'Help' cursor
Help on COSHH (as opposed to help on the use of the system) is dealt with under ‘Provision of
learning and background’ above.
Recommendations: It is recommended that the suggestions listed in the bullets above are
considered if the help system is reviewed or replaced.
3.7.7
Fitting in with user circumstances
When considering the usability of a system such as e-COSHH-E, part of the ‘context of use’
includes the organisational environment in which the system is used. This raises the following
issues described below.
3.7.7.1
Fitting in with existing assessment methods
The requirements analysis process identified that different user groups are likely to have
different levels of experience with COSHH, risk assessment and applying controls. It is also
likely that some user groups have attempted to comply with COSHH and already have some
risk assessment and controls in place.
Recommendation: Given the above point, e-COSHH-E would ideally match the needs of all
these different user groups. This might be achieved by modularizing the Assessment Summary
Page, so that advanced users can discard further advice on complying with COSHH, but users
new to COSHH can still obtain the further advice to complete the 8-step process of compliance
with COSHH as established by HSE.
This becomes especially important if e-COSHH-E is to reduce the burden of risk assessment on
it’s users.
44
3.7.7.2
Combining on and off-screen COSHH tasks
Users may need to return to the system in a number of sessions for various reasons, so the
system needs to support multiple sessions well.
When re-entering the system, it is possible to apparently make changes to the ‘Process & Tasks’
entries, but these are not stored i.e. the next time you enter the system, changes to the process or
tasks have gone. Also, adding new tasks to an assessment created the error messages in Figure
22.
Figure 22: Error created by adding tasks to a previous assessment
If you are doing a large and complex assessment you are more likely to want to return to the
system after an exit (or time-out). Re-entering the system requires the user to run through all
tasks a second time that most users will find unacceptable.
If the web page expires, a warning is produced which leaves the user with no link to get back to
the system to resume or start again.
Recommendation: The system should support multiple sessions more reliably than it does, and
reduce the load on the user when re-entering the system, by allowing them to pick up nearer to
where they left off.
3.7.7.3
Organisational requirements for the supply of information
Currently, COSHH publications are provided via electronic means, including PDF files and
HSE Direct. Some organisations are likely to prefer or need hard copies of information, so
stronger support for obtaining this could be provided.
Some hyperlinks to publications do not state the file size, format or whether they are internal or
external links. It is considered good practice to do this because the user’s might want to
consider whether they want to initiate a download first, because of limitations with their
computer.
45
3.7.8
Other Usability issues
3.7.8.1
Complexity of screens
Usability heuristics state that screens should be as minimalist as possible, and present only
information and items that are important to the users current range of tasks and goals.
Many screens are heavily loaded with textual info which is not necessary to all users, or only
relevant under certain circumstances. This conflicts with the requirement for the system to be
quick and immediate to use and to minimise cognitive load on the user. Simplicity and
minimalism are key heuristics for quick-to-use interfaces.
An example is the page where risk phrases are entered (Figure 23). This information would be
easier to read as bullet points, but also, the ‘Important note’ in red is only relevant to some users
i.e. those who have phrases which do not appear in the list. This could be dealt with, for
example, by providing a help link, the title of which is “I have phrases which are not listed
here” then provide the information from that link.
Recommendation: It is recommended that the method of providing information is reviewed,
with consideration given to providing linking to information that is not required by most users.
Figure 23: Text on the risk-phrase entry page
An example of unnecessary text can be found on some of the ‘assessment’ pages (Figure 24).
This creates an extra cognitive load on the user which can be avoided. It is possible to make the
completion of entry boxes in the interface self explanatory, and to use standard ‘Next’ or
‘Continue’ buttons to make navigation self explanatory.
Figure 24: Unnecessary text & instructions
Recommendation: However, where more information is required by some users, providing it in
a cascading fashion, providing ever more levels of further information, would allow a user to
choose how much information to receive, and to stop receiving information when they are
satisfied.
46
3.7.8.2
Use of ‘click here’ navigation
Although the use of ‘click here’ navigation links (Figure 25) in internet pages is fairly common,
it is unhelpful and inefficient for several reasons and is best avoided by using self explanatory
links, buttons and other interface components. For example, the blue button shown in Figure 25
could simply be labeled “start” and be more clearly recognisable as a button.
Figure 25: ‘Click here’ navigation links
3.7.8.3
Fitting users’ mental models and expectations
As well as the issue with the task based approach, there were found to be other issues where
there was a conflict with the user’s mental model of the system.
Use of the ‘Process name’ box at the beginning of the ‘assessment’ route is optional. Having an
option makes the user wonder why it is only optional. If the user decides not to enter something
meaningful in the Process name box, this could make it harder for them to understand the
assessment later on if they return to a printed or on-line assessment. This could be especially
true if they are dealing with two different processes that require two different risk assessments,
but are using the same substances.
When entering the name of the chemicals, the system states that it is not important to get the
name right (Figure 26). This proved to be very much in conflict with the users’ mental models
and expectations, leaving the user feeling that the system was incompetent or not likely to
produce reliable results. The reason for this is that the user does not understand why the name
is not important.
Figure 26: Entry of accurate chemical name is not required
47
The industry specific advice route provides advice on substances other than chemicals & also
includes some gasses – this conflicts with the home page which implies only advice on
chemicals is provided (Figure 27).
Figure 27: Home page emphasises chemicals rather than substances
3.7.8.4
System constraints
Entry of the task time was found to be too restrictive for users if the task did not fit a daily
pattern (Figure 28). The help pop-up for this was considered unintelligible. Although
technically it may not matter, not being able to enter their actual frequency of use data in a way
that was sensible to them was frustrating for users and produced a loss of confidence in the
system.
Figure 28: Entry of task time & frequency
3.7.8.5
User access to required chemical data
Analysis of the page access counts showed a drop-off of users in the entry of the boiling point
information (Figure 29). This could be attributed to not having the relevant information.
Feedback via the site also indicated this was a problem.
48
Figure 29: Entry of boiling point data
3.7.9
Maintenance & development issues
HSE needs to be able to update and add the sheets easily – this is an important requirement
given the changing and expanding nature of COSHH-E. Feedback from developer stakeholders
on this issue is that the system cannot provide this important requirement at present.
3.8
CONCLUSIONS
1. The task-based approach can:
a. Confuse users who are thinking in terms of substances, not tasks;
b. Result in higher level control advice being given for lower hazard rated
substances where multiple substances are entered;
c. Require the user to re-enter the same substance information for each different
task performed with the same substance;
d. Apparently cause problems if different quantity ranges are selected for different
substances in the same task, leading to a conflict in providing control advice;
e. Prevent advice about skin contact being applied to just the substance that it
applies to, when multiple substances are entered.
f.
Asking the user to (optionally) enter a ‘Process’ can potentially:
g. Confuse the user as to why it is optional;
h. Cause the user to think in too broad terms and try to deal with too many tasks
and substances at once.
49
2. The Help and learning systems:
a. Do not support the user well in understanding the system and how to use it to
meet their particular needs and scenarios of substance use;
b. Do not cater for the different user types, with different learning objectives and
existing levels of knowledge about COSHH and COSHH Essentials;
c. Are not cascaded, to provide deeper levels of help and learning as required.
3. The system structure:
a. Does not support separate advice for multiple substances entered under one
task;
b. Does not support the assessment of a solid and a liquid under the same task;
c. Does not support multiple-sessions very well;
d. Does not fit with expectations of how the system will be structured;
e. Does not support the user well in understanding the system and creating a
useful mental model for how it will function and meet their needs and goals;
f.
Has unnecessary steps in the ‘Direct advice’ route, which could be streamlined.
4. The interface:
a. Does not fit users’ needs for entering the duration and frequency of tasks, or
certain combinations of liquid and solid;
b. Has too much text presented to the user, which could be simplified or made
more context-sensitive;
c. Has some terms which present issues, such as ‘Direct advice’, ‘Assessment’,
‘Substance’, and ‘Chemical’;
d. Returns pages too slowly in places, making the ‘Direct advice’ route very slow
to use;
e. Is unresponsive in places, providing no indication to the user that anything is
happening after clicking the ‘Go’ button;
f.
Does not make it clear when the user is within a repeating loop for entering
multiple substances or multiple task assessments;
g. Does not follow recommended layout and design characteristics in a number of
places.
50
4
CONCEPTUAL SOLUTIONS
Taking into account the findings of section 3 of this report, some conceptual solutions have
been generated. Note that:
•
These concepts are focussed on Human Factors rather than technical considerations
•
They have not been tested in any way with real users
•
They have not been thoroughly tested against the technical requirement of COSHH
Essentials
4.1
SUBSTANCE & TASK SPECIFIC ADVICE ROUTE
4.1.1
Structuring
Before considering how a concept for a revised structure might look, a Hierarchical Task
Analysis was performed of retrieving relevant advice sheets (by whatever means). HTA is a
commonly used Human Factors technique for recording and analysing tasks. The resulting
analysis can be seen in Figure 30.
Figure 30: Task Analysis of obtaining advice by chemical assessment
Retrieve relevant
advice sheets
Establish if
substance is
covered
Identify supplied
vs own mixed
substances
Identify Hazard
Group rating of
substances
Identify Control
Approach
Deal with Group
E substances
(special cases)
Establish
substance
state
Subs chem;
Lower temp;
Rduc Dst/Vol
Seek special
advice
Select sheets
according to
assessment
Identify tasks
performed
Identify
Dustiness /
Volatility
Identify usage
Identify how
much is used
Identify freq
of use
Establish
solids
dustiness
Identify category
of Risk Phrases
(A-E)
Identify highest
Hazard Group
substance(A-E)
Check:
Exact match;
Carc/Muta catR40
Select Control
Approach
Establish
liquids
volatility
Apply expert
override if
applicable
Identify
boiling point
/ Vap presr
Check for Haz
Group S
51
Identify
process temp
(Highest)
Calc volatility
(for tasks at
room temp)
Use graph to
calc volatility
(tasks >20c)
The results in the Findings section, and the task analysis in Figure 30 were used to develop a
conceptual structure, illustrated in Figure 31. The aims of this structure were to:
•
Create an assessment process that attempts to fit in with users model of how the system
might work;
•
Provide flexibility to deal with different users goals and scenarios of use;
•
Allow users to enter their scenarios of use in a format that matches their expectations &
goals.
This proposed structure needs to be examined carefully, and compared with the existing
structure which is presented in Figure 32 and Figure 33, in order to understand the proposed
changes.
It should be noted that the intention is that the final advice sheets would be presented to the user
in a portfolio of assessments, where the user can see which advice sheets relate to which
substances and tasks. The system would make it clear to the user where the same advice applies
to different substances.
52
Figure 31: A conceptual structure for substance based advice selection
53
Figure 32: Existing structure of the e-COSHH-E system
54
Figure 33: Schematic of existing e-COSHH-E structure
55
4.2
INDUSTRY SPECIFIC ‘DIRECT’ ADVICE ROUTE
Appendix 3 shows the existing access routes to the industry-specific advice sheets.
Since the completion of this work, HSE has provided alternative access to some of the industry
specific advice sheets on e-COSHH-E, via the public HSE web site. This has been achieved by
access through the menu systems shown in Figure 34.
Figure 34: Alternative access to advice sheets
4.3
CONCEPT FOR INTEGRATING THE TWO ADVICE ROUTES
The industry-specific and substance-specific advice sheets appear to compliment each other in
terms of providing a full range of advice which will meet a variety of user circumstances. As
such, the system should make it clear to the user what they can expect from each part of the
system, and what user needs each can fulfil.
Figure 35 shows a very top-level concept for a system structure to meet one set of identified
user groups. This concept sets up an optional learning path depending on the knowledge and
experience of the user. It should be noted that the intention here is not to restrict each user type
to a fixed route in the system, but to make the learning paths clearly accessible to them, so they
can choose them straight away or later-on if they feel they need it.
Another point to note is that this conceptual system would create a portfolio of assessments and
advice sheets for the user, so that they can see where the same advice sheet is applicable to
different assessments and substances that they have entered into the system.
The concept structure would present the user with the both ‘Industry specific advice’ and
‘Substance specific risk assessment’ tools which they can use to build the control advice
portfolio that meets their particular needs.
56
Figure 35: Concept structure for dealing with different user types
User type
Entry point
/ start goal
Path to completion
of goal
COSHH Info
(8 steps etc)
External sites
COSHH-E info
(RA* & Control Selection)
COSHH naive
Other advice info
e-COSHH-E info
(Establish expectations )
COSHH aware
(Inexperienced in risk assessment)
Industry advice
System help
(Experienced in risk assessment)
RA* & Substance advice
e-COSHH-E user
(Re-visiting)
Advice sheets
download
Assessment &
Advice Portfolio
(Specific + General Advice)
*RA = Risk Assessment
Figure 36: Skeleton interface structure
1. Learn about COSHH
2. Learn about COSHH-E
3. Ready to continue
Start
Check for
advice
updates
Continue
advice
search
1. Industry advice
2. New assessment
Continue assessment
News
Updates
New sheets
57
Figure 36 shows a ‘skeleton’ of the interface structure that might deliver a revised version of the
industry and substance advice routes to the user. This is designed to meet 3 main goals:
1. For new users to start using the system by being presented with a learning path for
essential knowledge and information
2. For existing users to quickly return to where they left off
3. For existing users to check for updates, changes and additions to the advice sheets and
assessments they already have.
Figure 37 shows a concept screen which meets these basic goals. The aim would be to
encourage users to follow the learning path that is right for them, but not restrict them to this,
before choosing which advice route to use. Figure 38 presents an alternative concept with
similar goals. These concepts are provided as discussion points rather than tested solutions.
Figure 37: Concept screen
COSHH Essentials
Easy steps to control chemicals
& hazardous substances
I’m new
to this
Tell me about COSHH?
Tell me about this system
My assessments
Old hats
Check for advice updates
58
Figure 38: 2nd concept screen
4.4
CONCEPT FOR REVISING THE SELECTION OF TASKS IN THE
EXISTING SYSTEM STRUCTURE
Figure 39 shows a concept for a new screen layout to make the existing task-based assessment
route more meaningful to a user. This concept:
•
Separates out tasks which are done with substances from those that are done to
substances;
•
Makes it clear to the user at the outset that there are only a limited number of control
advice sheets available;
•
Removes the concept of ‘Process’ selection, which reduces the risk of the user trying to
deal with too broad an area ie. it is predicted that this will make the user more likely to
associate a task with a single substance, thereby reducing issues later.
59
Figure 39: Concept for changing the existing selection of tasks
60
4.5
PROVISION OF HELP AND LEARNING
Although one of the fundamental aims of e-COSHH-E is to provide risk assessment for users
without the need to learn about COSHH regulations, it is possible that some users would prefer
to obtain some understanding of COSHH in order to understand the e-COSHH-E system and the
advice they will get. This type of information might be wanted before, after or during
completion of the risk assessment steps, and therefore the system should support this somehow.
Other areas of HSE provide ‘web-friendly’ versions of advice leaflets such as ‘Five steps to risk
assessment’ (INDG163.PDF) and ‘COSHH: A brief guide to the Regulations’ (INDG136.PDF)
but e-COSHH-E could provide help and learning paths for accessing this information. The
information could be re-edited into a cascading and quickly downloaded format, such that users
can adjust the level of information they receive to the level they require.
For example, the information in leaflet INDG136.PDF contains the HSE 8-step guide to
complying with COSHH, and this could form the basis for accessing linked information about
each step and other information within this leaflet. Figure 40 shows the 8-step guide; it can be
seen that this could form a convenient ‘menu’ for users to access further levels of information.
Figure 40: Cascaded help on COSHH
Assess the risks to health from hazardous substances
used in or created by your workplace activities.
About COSHH
Step 1
Assess the risks
Step 2
Decide what precautions are needed
Step 3
Prevent or adequately control
exposure
Step 4
Ensure that control measures are used
and maintained
Step 5
Monitor the exposure
Step 6
Carry out appropriate health
surveillance
Step 7
Prepare plans and procedures to deal
with accidents, incidents and
emergencies
Step 8
Ensure employees are properly
informed, trained and supervised
Links to more info….
You must:
• identify the hazardous substances present in your
workplace;
• consider the risks these substances present to
people’s health.
You must not carry out work which could expose your
employees to hazardous substances without first
considering the risks and the necessary precautions,
and what else you need to do to comply with COSHH.
You must prevent your employees being exposed to
hazardous substances. Where preventing exposure is
not reasonably practicable, then you must adequately
control it. The advice in COSHH Essentials, and in the
other guidance it refers to, will help you to make
correct assessments and to put the appropriate controls
into place.
Ensure that control measures are used and maintained
properly and that safety procedures are followed.
61
5
5.1
APPENDICES
APPENDIX 1: ANALYSIS OF USER FEDDBACK
(FROM 19-12-2001 TO 23-08-2005)
Key phrases
Issue
Freq of use
Sectors
No
Comment
1.user thought there was no
provision for entering chem usage at
freq= <1day
2.User stumbled because using
chem. only 1nce a month
3.Prob with only entering in days
3
Not an issue for proper
assessment, but is an
issue for the user.
Would it be fixed with
just an explan or is diff
entry type needed?
Scope of
advice sheets
User wanted control guidance sheet
for paints, silicone, mastic etc.
1
This user group needs
redirecting to advice
suitable for them
Temperature
check /
restrictions in
system
User mixing chems at above boiling
point (under pressure?) therefore
became confused by the interface
which does not allow chem to be
used above its boiling point –
‘COSSH-E does not deal with
gasses’ but under pressure, it is not
a gas.
1
System not meeting user
requirement (not
compatible with user’s
COSHH circumstance)
Data necessary
to complete
RA not
available from
MSDSs
1.User had common household
chem ‘Mr muscle’ but did not have
MSD Sheet – had trouble tracking
this down
2.User wanted info on many basic
cleaning products used in their org
3.Medical cleaning products
requested
4.User could not find necessary data
on boiling pnt/vap press on SDS
5.Could not find boiling point (flash
only)
6.Vapour pressr not on SDS
7.Domestic cleaning products – no
details on volatility = can’t proceed
8.Volatility n/a. Boil Pnt/Press n/a.
8
Help message on where
to get SDSheets? Info
about companies not
providing what they
should?
How do users get the
SDS and info they need?
Some users don’t know
what to do ie. they have
a simple cleaning
product with no info &
don’t know where to
start – excludes this user
group.
Users may need help
when stuck by not
having necessary data.
Scope of the
system
User wanted HSG
193 but could not find it
1
Point – some users come
to COSHH-E knowing
what they want and just
want quick links. User
did not realise the site
replaces the HSG193
pack
62
Cleaning
Care
Medical
Boat building
Manufacturing
Cleaning
Scope of the
system
1.User wanting to view whole list of
advice sheets
2.Wanting search & retrival of PDF
resources
Research/edu
Pharm
1
Although users of paper
system would have
access to all these, the
point of C-E is to guide
to correct ones. Would
access to the whole list
be acceptable?
Overall system
approach
User found the process very long
winded & gave up. Wanted product
selector
Building
1
User wanting something
different from what the
system provides. Can be
time consuming for
some.
Delivery
format of eCOSHH-E
User desired COSHH-E on CD
2
Most likely because of
speed/cost of being online. Maybe add to FAQ
– ‘Why is it not available
on CD?
Specific
industry
wanted
User requested section for specific
type of industry
User wanted industrial engineering
related processes
3
Might be addressed by
new industry specific
advice route?
Minimum user
system
requirements
User had problems with windows
not being full size – having to use
scroll bars
1
Make min requirements
clear – are they already
on site?
Revisiting site
User wants id of info revision date
so can determine which info is new
1
Could this be addressed
in the interface eg a
what’s new section?
Missing risk
phrase
R31
R33
R52/53
User had product which releases
gasses when in contact with acid,
but it is a liquid not a gas – felt that
his assessment was incomplete (No
risk phrase R31).
2.Wanting R31 &33 phrase
3.Wanting R51/R53
LAU
Manufact
3
May need to manage the
other RPrhases in a
different way.
R31 Releases gasses
R52/53 = environ
R 33 – cumulative risk
Need to reassure users
about missing phrases &
what they should do to
complete their
assessment
R-Phrases
1User requested info on flammables
2.Wanting R11 R8 phrases ie.
flammables physico-chemical risks
Care
Pharmaceutical
3
Need to make clear what
COSHH includes ie.
health not safety/environ
– maybe add small
explan to risk phrase
page?– inform of future
plans?
Loss of link
between
substance &
advice sheets
User noted there is no link between
the products being assessed and the
assessment ie. only id by task or
process, not by product name
Medical
1
User suggested id by
product as well as
process name/task –
possible solution = users
use the ‘Process name’
entry box which is
currently optional
63
Council
Industrial
Help system
User suggested easier access to the
technical basis of C-E
H&S Conslt
1
Safety data
sheets required
Inquiry about photo chems –
Photographic
1
Are MSDS available?
Issue – naïve users need
to know what steps to
take if SDS not available
or SDS don’t have right
info
Comparison
with paper
system; mental
model of the
system
User found it easier than using
paper based HSG pack
Consultanacy
1
Suggests that paper
system helps provide an
understanding / mental
model of the web
system?
Task based
approach
Options for use of chems limiting
1
Probably referring to
limited list of tasks at the
start. Could contact and
find out why limiting?
Chem. state –
solid / liquid
1.User has paste chems = not sure if
liquid or solid (entered as liquid!)
2.User feels state of chem. not
comprehensive enough
Boat
building(GRP)
1
Liquid/solid categories
may need to deal with
‘paste’? Incorrect
assessment possible in
this case?
R-Phrase
definitions
User cannot find def of risk phrases
Metal panel
manufacture
1
Think this has been
solved since?
Scope of the
industry
advice
User has found that the direct advice
route for nail extensions was not as
thorough as going through
individual scenario
LAU
1
Combined use of
industry / specific
substance advice
necessary to cover all
users?
User not
familiar with
COSHH
User wanted presentation on
COSHH
Training
1
Should the system
provide redirection to
basic learning on
COSHH before being
used?
User not
familiar with
COSHH
User wanting leaflet about H&S
‘rules’
beauty/nails
1
V. basic starter – the site
should have something
for these people? Links
to reliable info off-site?
Task led
approach
User wanting auto-complete on
repeated chem. names in different
tasks
Systems
engineering
1
Task led approach
resulting in repetition of
chemical entry
System error
User created error by returning to an
assessment and adding a new task to
a process
Systems
engineering
1
Database error generated
Small %ages
V small/large perc not possible – ie.
0.001 or 99.99
1
User needs support if
their particular
circumstances don’t fit
into the system. or.
system needs to fit their
data entry needs.
64
-ve values
Boiling point –ve values could not
be entered
Scope of the
system
User looking for advice to create
MSDS for own products mnfrd
abroad
Scope of the
system
Gap filler not accepted ie.
Pharm
65
2
Presume these might be
classed as gasses, even
though under pressure
they are liquid? Is this
OK or should –ves be
accepted? User needs to
be supported in this
scenario, pref before they
get so far into the system
1
Should the site be able to
redirect such people?
1
Because aerosol product?
Redirect to other advice
sources? Issue = system
only provides advice on
some parts of a users
COSHH circumstances
ie. not a one-stop shop.
Change user
expectations?
5.2
APPENDIX 2: PAGE REQUEST STATISTICS APPLIED TO SITE
STRUCTURE: SUBSTANCE & TASK SPECIFIC ADVICE
Page 1, left (Continued on Page 1, right)
66
Page 1, right (Continued from Page1, left)
67
Page 2, left (Continued on Page2, right)
Cont. from pg 1
Home.asp
Pg1.
‘Introduction 1.htm’
/EntryOptions.asp
(‘Getting Started’ page)
7136
1. What the law says…
2. Start a new assess…
3. Return to an assmnt…
4. Direct advice topics
Load data
Pg3.
Direct Advice Route
/UnitOperations.asp
(‘Process & Tasks’ page)
5,099
Pg1.
/Help/Tasks
/ProcessName
/GeneralAdvice
/Transferring
/Pelletising
/Surface coating
/Dust extraction
/Drying
/Screening
/Mixing
/Laminating
/Dipping
/Sieving
/Weighing
/Storing
Pg1. Home.asp
Pg1. Introduction1.htm
Pg1. Introduction6.htm
Operating temp error
Boiling point range error
68
Page 2, right (Continued from Page2, left)
help/ChemicalsOrProducts.htm
(Chemicals or products)
293
help/ProcessesAndTasks.htm
(Chemicals or products)
94
Pg1.
/ChemicalName.htm
(Chemical Name)
(Label)
Pg1. /Help
NB: For each task
assessed there is a
link to the relevant
help on that task eg.
Transferring,
Seiving etc
/StateMixDetails.asp
(‘’How Many chem…using?))
5,295
Option 2
(Chems in mixture…)
Option 1
(Number of chems…)
A
B
C
D
E
help/MaxConcentration.htm
(Maximum Concentration)
30
ChemicalCAS.asp
(…)
5,019
HazardClassification.asp
6,261
help/RPhraseList.asp
(here)
304
RPhrase.asp
6,426
Pg1.
skin
/RiskPhrases.htm
DustinessVolatility.asp
5,363
Volatile solid details
MIXTURE
LOOP
CHEM
LOOP
UsageInfo.asp
4,740
Volatile liquid details
Summary.asp
3,603
69
ControlApproach.asp
3,459
TASK
LOOP
5.3
APPENDIX 3: SITE PAGE STRUCTURE: INDUSTRY SPECIFIC ADVICE
ROUTE
The following charts show the site page structure of the industry-specific “Direct” advice route.
The advice sheets that are shown are the sheets that were presented to the user after following
the hierarchical links shown below. It was not practical to complete all of the links because of
the slow speed of the system. Incomplete areas have been blanked out.
Specific advice sheets
ActivityOption
General advice sheets
FL01-07 FD01-14 MR01-05 RB01-07 SR01-27 WD01-09 G1##
G2##
G4##
(Flour)
(Foundry) (Vehicle) (Rubber) (Srv&Ret) (Wood)
(CntrlAppr1) (CntrlAppr2) (CntrlAppr4)
Production
-Flour milling & craft bakery
Craft bakery
Bag opening and tipping, and dough mixing (craft bakery)
01
14
02
Hand moulding, dividing, and using a dough brake (craft bakery)
03
14
02
Immaterial lid dust extraction specification
05
Stand-alone dust collectors (craft bakery)
Weighing and handling flour improvers etc (craft bakery)
Flour milling
04
04
02
02
14
02
Flour bagging (smaller flour mill)
07
14
02
Improvers (enzyme additives) in flour milling
06
-Foundry work
Fettling and cleaning castings
Dust and fume when thermal cutting and gouging
10 14
Dust when fettling larger castings
08 14
Dust when fettling smaller castings
07 14
Dust when shotblasting smaller castings in a cabinet
Melting and casting
09
Foundry fume from melting
02 14
Foundry fume when ladle pouring
03
Fume and general ventilation
Other tasks
01
Dust and sludge when cleaning dust collectors
13 14
Fume from pattern assembly (investment casting)
Mist and vapour from spray coating a large casting (open workshop)
Work with sand
Dust and fume in knock-out, shakeout and attrition
Dust and vapour in smaller scale coremaking/shell moulding
Foundry dust - sand plant
-Rubber making
Rubber dust
Dust and fume from milling
Dust from bag opening and weighing
Dust from mixing
Rubber fume
14
02
04
14
03
04
Fume control and general ventilation
01
Fume from cooling racks for smaller articles
14
06
Fume from rubber presses (smaller articles)
14
05
Fume from trimming and finishing smaller articles
14
07
Bandsaws
14
01
02
Circular bench saws
14
02
02
Cross-cut saws
14
03
02
-Woodworking
Sawing
Shaping
Overhead and CNC routers
14
05
02
Vertical spindle moulders
14
04
02
Hand-held sanding machines
14
07
02
14
06
02
Sanding
Sanding machines
Other tasks
Furniture assembly
14
08
02
Stand-alone dust collectors (occasional use)
14
09
02
-Printing
External site
70
Service
-Beauty treatments
Cleaning
Manual cleaning and disinfecting surfaces
04
Ready-for-use insecticide sprays and dusts
07
Other tasks
Diluting chemical concentrates
02
Storing chemical products (small scale)
24
Treatments
Beauty salon - makeover, nails and depilatory services
13
Electrolysis, piercing and tattooing
12
Hairdressing
11
02
01
02
-Car parks and Bus depots
Cleaning and disinfection using low-pressure washers
Exhaust fumes from moving vehicles
15
Vehicle exhaust fumes (in warehouses and garages, etc)
14
Work involving motor vehicle fuel (diesel, petrol, LPG)
16
-Cleaning services
Cleaning and disinfection
Cleaning and disinfection using low-pressure washers
01
Diluting chemical concentrates
02
Manual cleaning and disinfecting surfaces
04
Ready-for-use insecticide sprays and dusts
07
Dry cleaning
Dry cleaning - spot cleaning
06
Dry cleaning using chlorinated solvent
05
Other tasks
Storing chemical products (small scale)
24
-Flori / Grocer
Cleaning
Diluting chemical concentrates
02
Manual cleaning and disinfecting surfaces
04
Shop activities
Ready-for-use insecticide sprays and dusts
07
Working with flowers, fruit and vegetables
26
Other tasks
Storing chemical products (small scale)
24
-Funeral services
Cleaning
Cleaning and disinfection using low-pressure washers
01
Manual cleaning and disinfecting surfaces
04
Ready-for-use insecticide sprays and dusts
07
Embalming
Diluting chemical concentrates
02
Embalming with formaldehyde solutions (formalin)
Other tasks
Storing chemical products (small scale)
-Leisure / sports venues
Controlling cooking fumes
Diluting chemical concentrates
Exhaust fumes from moving vehicles
Smokes and fogs in clubs, etc
Storing chemical products (small scale)
Water treatment - concentrates and treated water
71
Service
-Maintenance & repair services
Building maintenance
Cleaning and disinfection using low-pressure washers
Diluting chemical concentrates
Ready-for-use insecticide sprays and dusts
Water treatment - concentrates and treated water
Garden maintenance
Cleaning and disinfection using low-pressure washers
Diluting chemical concentrates
Fogging and misting using space sprayers
Working with flowers, fruit and vegetables
Other tasks
Storing chemical products (small scale)
Small repair shops
Degreasing with solvents (small scale work)
Dust from abrasive wheels
Soldering - fume from rosin-cored solder
Vapour (fumes) from solvent-based adhesives (small scale work)
Work with lubricants and waste oil
-Motor vehicle repair & maintenance
Chemicals in repair jobs
Degreasing with solvents (small scale work)
23
Dust from abrasive wheels
21
Isocyanate from small tasks - flatting, SMART repair, welding, etc
05
Work involving motor vehicle fuel (diesel, petrol, LPG)
16
Work with lubricants and waste oil
19
Cleaning jobs
Cleaning and disinfection using low-pressure washers
01
Diluting chemical concentrates
02
Valeting and cleaning cars with solvent-based products
17
Other tasks
Charging batteries for electrically powered vehicles
18
Storing chemical products (small scale)
24
Vehicle exhaust fumes (in warehouses and garages, etc)
14
Spraying and other paint jobs
Isocyanate from brush or roller application of two-pack products
04
Isocyanate from cleaning two-pack spray guns
03
Isocyanate from mixing two-pack paint, etc
01
Isocyanate from small tasks - flatting, SMART repair, welding, etc
05
Isocyanate from spraying two-pack products in a spray/bake booth
02
-Pest control
Other tasks
Cleaning and disinfection using low-pressure washers
Storing chemical products (small scale)
Water treatment - concentrates and treated water
Pest Control
Diluting chemical concentrates
Eradicating vermin (rats, etc)
Fogging and misting using space sprayers
Ready-for-use insecticide sprays and dusts
-Pubs / clubs & restaurents
Catering
Controlling cooking fumes
Working with flowers, fruit and vegetables
Cleaning
Cleaning and disinfection using low-pressure washers
Diluting chemical concentrates
Manual cleaning and disinfecting surfaces
Ready-for-use insecticide sprays and dusts
72
5.4
APPENDIX 4: USER REQUIREMENTS ANALYSES
.1
.2
.3
Direct users
Concerned with the usability of the interface & functionality of the system because Variable (described below).
they are the directly using the system
Roles will vary according to their use of the paper COSHH-E system
Variable (described below).
Will be starting from scratch, using the system instead of purchasing the paper
Deepest analysis required.
version.
COSHH-E (paper) user groups
non-owners
Dep
th
ana of
lysis
requ
ired
User | Stakeholder groups
(NB. Groups can overlap)
Des
cript
'n
3.1. User group description*
.4
owner-users
May be using the system to check their assessments done with the paper system; Less depth required because this group is probably
May use the system to check for updates / changes to the paper system.
getting the help they need already.
.5
owner-non-users
May be using the e-COSHH-E system in the hope that it will provide some benefit Realtively small group, important because of having
over the paper system eg. Easier to use.
rejected the paper system for some reason.
.6
.7
.8
COSHH Experience
No experience
Experienced
.9
.10
Organisation size
(S)ME <10 employees
.11
S(M)E 10-250 employees
.12
Large orgs
.13
Number of substances used
.14
Using small n. of substances
.15
Using medium n. of substances
.16
Using large n. of substances
.17
Industry sectors
.18
Using
.19 Indirect users
.20
.21
HSE
Fit3 policy stakeholders
.22
Fit3 program deliverers
.23
System builders
.24
.25
Service provider
Chemicals providers
Entering the system with no knowledge of COSHH
Entering the system with some or good knowledge of COSHH
Need to cater equally for both
In depth
In depth
Role changes according to organisation size
Variable (described below).
Person responsible for COSHH will probably have other roles in the organisation; Target user group with potentially the greatest
COSHH likely to be an additional/ancilliary part of job.
barriers/errors in performing COSHH- greatest depth of
analysis required.
Person responsible for COSHH likely to have a major role in general safety
management;
Role may be more dedicated to safety issues = more time.
Person responsible for COSHH very likely to have dedicated role in safety;
Likely to have greater resources to perform COSHH;
Likely to be more constrained by established organisational methods.
Target user group - greatest depth of analysis required.
Users with large number of substances have a different role to those using only a
few.
Using system briefly, possibly only once.
Using system for longer, possibly multiple visits.
Possibly using the system a lot, with re-visits.
In depth - must meet the variety of needs of these
groups.
In depth.
In depth.
In depth.
Less depth because e-COSHH-E is aimed at helping
the smaller orgs as priority.
Will be more concerned with the management of the system & the costs / benefits
that the system provides.
Development of strategy & targets for disease reduction in relation to COSHH.
Require resources to be applied to the most effective solutions to meet targets.
Development & delivery of the COSHH-E systems
Delivery of solutions which meet Fit3 targets
In depth required because the system must be able to
grow with changes & developments in COSHH-E paper
system
Required to build, maintain & modify the system.
In-depth required because of the potential maintenance
/ developments to COSHH-E.
73
.1
.2
.3
Direct users
Wide age range;
Wide range of computing skills
COSHH-E (paper) user groups
non-owners
.4
owner-users
.5
owner-non-users
.6
.7
.8
COSHH Experience
No experience
Experienced
Already have knowledge of COSHH
.9
.10
Organisation size
(S)ME <10 employees
Small budget for COSHH;
.11
S(M)E 10-250 employees
.12
Large orgs
.13
Rati
ng**
Link
s
Ana
lysis
User | Stakeholder groups
(NB. Groups can overlap)
Pote
nti
rqur al
mn
syst ts &
em
spec
s
3.2. User characteristics*
Make COSHH info easily accessible to this group
Don't want to be burdened with COSHH info; COSHH info to
be unobtrusive to this group
Requires low resourcing (time / technology / training);
Absolute minimum clicks
Works on min. spec computer & modem
Intuitive / instant usability
Number of substances used
.14
Using small n. of substances
.15
Using medium n. of substances
.16
Using large n. of substances
.17
Industry sectors
.18
Using
.19 Indirect users
.20
.21
HSE
Fit3 policy stakeholders
.22
Fit3 program deliverers
.23
System builders
.24
.25
Service provider
Chemicals providers
Focussed on reduction of ill health from Harmful substances Must suit very wide user base not lose users who try it
Seek evidence of performance ie. Reduction in ill health
System needs to provide evidence of performance against
Fit3 targets
1
1
Need system to record performance - details of essential &
desirable records from HSE
Changes / new content need o be easily /speedily
implemented
Receive payment for development & maintenance
More maintenance = potentially more payment
Do not want to have to abandon large parts of the system for System architecture needs to be flexible for future
future changes
developments
Want minimal loading on their server
2
Focussed on prgram delivery
Required to produce evidence of system performance
Creators of new sheets / content
74
1
.1
.2
.3
COSHH-E (paper) user groups
non-owners
owner-users
.5
owner-non-users
.6
.7
.8
COSHH Experience
No experience
Experienced
.9
.10
Organisation size
(S)ME <10 employees
.11
S(M)E 10-250 employees
.12
Large orgs
Larger variability in capability of equipment
Greater chance that necessary software is not present or up
to date
Likely to have more capable equipment
Likely to have required software installed or easily obtainable
More likely to have IT support
Number of substances used
.14
Using small n. of substances
.15
Using medium n. of substances
.16
Using large n. of substances
.17
Industry sectors
.18
Using
.19 Indirect users
.20
.21
HSE
Fit3 policy stakeholders
.22
Fit3 program deliverers
.23
System builders
.24
.25
Service provider
Chemicals providers
Developing changes in corporate style
Developing regular changes to technical content
May have slightly different configuration to the final host of
the system
75
Require easy & fast changes & additions to certain content
**
Link
s
Ratin
g
Pote
nt
rqur ial
mn
& sy ts
ste
spec m
s
Direct users
.4
.13
Ana
ly
User | Stakeholder groups
(NB. Groups can overlap)
sis
3.3. Technical environment*
.1
.2
.3
**
Link
s
Rati
ng
Pote
nt
rqur ial
mnt
syste s &
m
spec
s
Direct users
COSHH-E (paper) user groups
non-owners
.4
owner-users
.5
owner-non-users
.6
.7
.8
COSHH Experience
No experience
Experienced
.9
.10
Organisation size
(S)ME <10 employees
.11
S(M)E 10-250 employees
.12
Large orgs
.13
Ana
ly
User | Stakeholder groups
(NB. Groups can overlap)
sis
3.4. Psycho-social & organisational environment*
Anonymity - fear of inspection NB potential conflict with need Provide proof of anonymity eg. Short msg on data privacy &
to integrate with HSE site
link to 'Privacy policy'
Person responsible for COSHH very likely to have dedicated
role in safety;
Likely to have greater resources to perform COSHH;
Likely to be more constrained by established organisational
methods
Number of substances used
.14
Using small n. of substances
.15
Using medium n. of substances
.16
Using large n. of substances
.17
Industry sectors
.18
Using
.19 Indirect users
.20
.21
HSE
Fit3 policy stakeholders
.22
Fit3 program deliverers
.23
System builders
.24
.25
Service provider
Chemicals providers
Seek consistent corporate image
More integration with HSE web site
Seek to reassure users their data will not be used for ANY
other means, especially inspection
Develop & provide 'Privacy policy'
76
3.4.20
.1
.2
.3
Direct users
COSHH-E (paper) user groups
non-owners
.4
owner-users
.5
owner-non-users
.6
.7
.8
COSHH Experience
No experience
Experienced
.9
.10
Organisation size
(S)ME <10 employees
.11
S(M)E 10-250 employees
.12
Large orgs
.13
Learn about COSHH
Learn about COSHH-E
Obtain easy COSHH
Check own assessments
Update own assessments
Learn about COSHH
Check / update knowledge
Number of substances used
.14
Using small n. of substances
.15
Using medium n. of substances
.16
Using large n. of substances
.17
Industry sectors
.18
Using
.19 Indirect users
.20
.21
HSE
Fit3 policy stakeholders
.22
Fit3 program deliverers
.23
System builders
.24
.25
Service provider
Chemicals providers
77
s
g* *
Link
Ratin
Goa
ls
User | Stakeholder groups
(NB. Groups can overlap)
Pote
nt
rqur ial
mn
syste ts &
m
spe
cs
3.5. User group goals
5.5
APPENDIX 5: LIST OF ISSUES
The list of issues is developed from the expert/heuristics evaluations and the cognitive
walkthrough sessions.
5.5.1
Appendix 5.1: Expert/heuristics evaluations
The list is divided into the following subject areas:
(1)
Functionality
(2)
Interaction & Graphical Interface
(3)
Structuring & navigation
(4)
Language
(5)
Error handling, error tolerance & flexibility
(6)
Help & Learning
Contents of issues list
1. Functionality
81
1.1. Clarity of introduction to the functionality of the system...................................... 81
1.2. Defining the ‘process’ to be assessed ................................................................. 82
1.3. Dealing with ‘Mixtures’ ......................................................................................... 83
1.4. Dealing with liquids + solids in the same task ..................................................... 84
1.5. Dealing with ‘split’ tasks....................................................................................... 85
1.6. Entering chemical names .................................................................................... 86
1.7. Continuity between sessions ............................................................................... 87
1.8. Security of advice sheets..................................................................................... 87
1.9. Expiry of assessment........................................................................................... 88
1.10. Apparent relevance of the advice sheets to the user......................................... 88
1.11. Forced review of tasks on returning to an assessment ..................................... 89
1.12. Quantity applied to task rather than substance ................................................. 89
1.13. Advice offered part way through assessment for ‘E’ rated substances ............. 90
1.14. Re-entry of identical data required for same substance .................................... 91
1.15. Duplicated advice sheets................................................................................... 91
78
2. Interaction & graphical interface issues
92
2.1. Home page ‘Home’ btn ........................................................................................ 92
2.2. Clickable COSHH-E Logo.................................................................................... 93
2.3. Top menu bar items............................................................................................. 93
2.4. Design of the ‘Go’ button ..................................................................................... 94
2.5. Duplicate Help link ............................................................................................... 94
2.6. Multiple routes into Help system.......................................................................... 95
2.7. No distinction between types of tasks.................................................................. 96
2.8. Side menu bar ..................................................................................................... 97
2.9. User not warned of PDF file download ................................................................ 98
2.10. Unnecessary information presented to all users................................................ 99
2.11. Page link in ‘Index to Terms’ page does not have a meaningful name ........... 100
2.12. Navigation in Help – Page identification .......................................................... 100
2.13. Flexibility in entering Risk Phrases .................................................................. 101
2.14. Context Help – Two different Help subjects presented together ..................... 102
2.15. Slow loading of R-Phrase page ....................................................................... 103
2.16. Speed of page delivery from server................................................................. 103
2.17. Layout of text & radio buttons .......................................................................... 104
3. Structuring & Navigation issues
105
3.1. Navigation structuring in Help – Use of ‘Back’ & ‘Next’ page links .................... 105
3.2. Numbered steps on the start page .................................................................... 106
3.3. Maintaining user awareness of where they are in the assessment
process .......................................................................................................... 107
3.4. Navigation after re-entry into the system using the assessment code .............. 108
3.5. Lack of a Privacy Policy document .................................................................... 108
3.6. Link to printing information................................................................................. 109
3.7. Identical advice in sheets................................................................................... 109
4. Language issues
110
4.1. “CHIP and ‘dangerous for supply’”..................................................................... 110
4.2. Use of term ‘Direct advice’................................................................................. 110
4.3. Text suggests advice sheets are offered on the current page........................... 111
4.4. Use of term ‘Assessment’ .................................................................................. 111
4.5. Instruction for selecting tasks ............................................................................ 112
4.6. Over-complex use of language.......................................................................... 113
4.7. Use of terms ‘Chemical’ and ‘Substance’ .......................................................... 114
4.8. Help on Help ...................................................................................................... 115
79
5. Error handling, error tolerance & flexibility
116
5.1. ‘Page expired’ on using ‘Back’ link .................................................................... 116
5.2. Error after selecting ‘Re-Assess’ ....................................................................... 117
5.3. Re-Assess & Delete functions: no ability to cancel............................................ 118
5.4. Hazard Override ................................................................................................ 118
5.5. Removing tasks ................................................................................................. 119
5.6. Confusion over number of chemicals being assessed (after re-entering
the system) .................................................................................................... 120
5.7. Re-entering system & adding a task.................................................................. 121
5.8. Editing the ‘Process name’ ................................................................................ 122
5.9. Boiling point entry .............................................................................................. 123
6. Help & Learning
124
6.1. Understanding the task-based approach........................................................... 124
6.2. Different learning needs of users....................................................................... 125
6.3. Support for ‘expert’ users................................................................................... 125
6.4. Overall function of the system ........................................................................... 126
6.5. Context of COSHH-E......................................................................................... 126
6.6. The 8 Steps to good practice for COSHH is not included on site ...................... 127
6.7. Explanation of what is needed before starting................................................... 128
6.8. Help suggests gasses are not included in COSHH-E........................................ 129
80
1. Functionality
Substance-based Assessment route
1.1. Clarity of introduction to the functionality of the system
Location:
Home.asp
User Group:
All new users who have not used paper COSHH-E or e-COSHH-E before.
Issue:
The home page may be establishing incorrect expectations of functionality in the user’s mind
Issue:
The home page does not convey the fact that the advice that will be provided comes from a set of existing advice sheets, & that this
system will help to choose the right ones & form a package of the most appropriate advice sheets.
Relvnt reqmnt: Users to understand quickly exactly what they will get from the system
Relvnt reqmnt: Users to be able to relate their goals with what the system has to offer as outputs
Recomm'n:
Make it clear at the outset that the system attempts to find the most appropriate advice sheets for the user
81
1.2. Defining the ‘process’ to be assessed
Location:
‘Process and Tasks’ page (UnitOperations.asp)
User Group:
All users, especially those new to COSHH-E & e-COSHH-E
Issue:
The user might define a process which the system is later unable to cope with ie. they may make the process too broad in scope, & then
find that the system cannot accommodate the complexity of the substances/tasks scenario they have included within the ‘process’
Issue:
The user might define a process which involves too many tasks & substances, which the system can support, but which leads the user
into a state of confusion or an assessment which is too long.
Issue:
There is little control over how the user defines the scope of their ‘process’
Issue:
The example which is provided on the page encourages definition of a too broad & complex process – someone may describe the job
they are doing in too broad terms
Issue:
Giving the option to leave this blank could conflict with the user’s perception of the need to complete this entry box. The user may not
understand why it is ok to leave this blank.
Relvnt reqmnt: The system should support user goals & lead the user away from creating errors or confusion
Relvnt reqmnt: Cognitive load should be minimised
Recomm'n:
If the current system architecture remains, it should support the user in defining their process in a way which prevents the system being
unable to cope with the scenario, & prevents over-complex assessments.
Recomm'n:
Entering the process name should be compulsory, not optional.
82
1.3. Dealing with ‘Mixtures’
Location:
‘How many chemicals are you using’ page (StateMixDetails.asp)
User Group:
Any user dealing with substance mixtures & performing a mixing task
Issue:
Confusion between the task of mixing & the two options presented. The user may be asked about mixtures made before carrying out a
mixing task.
Issue:
If the user wants to enter a mixture plus a single substance for a particular task, this is not possible.
Relvnt reqmnt: System should match user expectations & mental models
Relvnt reqmnt: System should minimise cognitive load
Relvnt reqmnt: System should match with & support user goals
Recomm'n:
There should be flexibility to match the way the user wants to use the system
Recomm'n:
System to support all scenarios of mixtures, or prevent user getting into a situation where their scenario of use cannot be
accommodated ie. at the ‘process’ definition stage.
83
1.4. Dealing with liquids + solids in the same task
Location:
‘Chemical or product name’ page (ChemicalCAS.asp)
User Group:
Any user who is using a solid & liquid in the same task, but not as a mixture
Issue:
If the user has defined a process which includes both a solid & a liquid (eg. mixing cement with waterproofer) the system cannot cope
with the two together, & produces an error message.
Issue:
The error message seems to appear even if dealing with separate substances not in a mixture ie. if the user previously entered the
number of chemicals rather than chemicals in a mixture
Issue:
If the user cannot enter the substances as they are actually using them, they must find a way to work around the restriction in the
system
Issue:
The restriction occurs because the system currently cannot provide separate advice for solids & liquids if presented together at this
stage in the assessment
Issue:
The user was not previously warned that, in defining their ‘Process’, they must treat the use of liquids & solids as separate processes.
Issue:
The list of tasks on the ‘Process & Tasks’ page (UnitOperations.asp) includes ‘Mixing’ which therefore invites the user to include both
solids & liquids in the same process, if they are mixing a solid with a liquid. The system then cannot support this.
Relvnt reqmnt: The user should not have to try & fit their aims around limitations of the system; The system should fit user goals & circumstances
Relvnt reqmnt: Cognitive load should be minimised
Recomm'n:
If keeping the existing basic structure, forewarn the user to separate solids & liquids
Recomm'n:
If redesigning, allow the user to enter the data for both solids & liquids, then provide separate advice for each.
84
1.5. Dealing with ‘split’ tasks
Location:
‘Specific’ advice route (substance based assessment)
User Group:
Any users performing two or more of the same tasks in one process.
Issue:
If a user is performing a process that involves two transfer operations, for example, split by performing another task in-between, then it
becomes difficult to make the system meet this scenario. If the user is transferring two substances, mixing them & then transferring or
storing the result, the user can find that the system does not meet their scenario of chemical use & tasks. This partly depends on the
scope of the ‘process’ that the user defines before starting. If the scope is too wide, they may encounter problems. It becomes difficult
to make the system fit the user scenario.
Issue:
There is a particular restriction in the system whereby if you are transferring before & after mixing, the system cannot deal with
transfer of both a mixture & individual chemicals.
Issue:
For the user, there is considerable cognitive load required to try & understand how to make the system deal with mixtures
Issue:
Although the above issues may not affect the final advice given, to the user it will appear at this stage that the system cannot meet their
needs & goals.
Relvnt reqmnt: System should minimise cognitive load by matching user expectations & mental models of how they need to achieve their goals
Relvnt reqmnt: System should match user goals
Recomm'n:
This appears to be due to the ‘Process’ & ‘Task’ based approach, which would require restructuring to avoid this.
85
1.6. Entering chemical names
Location:
‘Chemical or Product Name’ page (ChemicalCAS.asp)
User Group:
Anybody entering chemical names
Issue:
Users are told that it does not matter if they get the chemical name exactly right. At this stage in the assessment this caused slight
confusion for the assessors because it is difficult to understand why the exact name is not required.
Issue:
User potentially loses some confidence in the system.
Issue:
Some users expected the system to recognise the substances they started to type in, & then to complete the entry for them.
Relvnt reqmnt: Functionality should fit user expectations with regard to completing their goals
Recomm'n:
It is not necessary to tell the user that the exact name doesn’t matter
Recomm'n:
It may be better just to ask the user to get the name as correct as possible
Recomm'n:
A correct name may be important for the record keeping, especially if similarly named products are in use
86
1.7. Continuity between sessions
Location:
The ‘How many chemicals are you using?’ page (StateMixDetails.asp)
User Group:
Those who need to exit the system & return after entering tasks
Issue:
If user returns to the system by entering their assessment code, the previously entered number of chemicals will not have been saved,
where they have not completed the entire assessment routine.
Issue:
The paper based system has the advantage that the user can easily interrupt the process to gather extra data for the assessment eg. data
on number of chemicals and the MSDS information.
Relvnt reqmnt: Where users might need to interrupt their assessment session to gather data from other sources, the system should allow them to return
to the same point easily.
Relvnt reqmnt: There should be no loss of data between user sessions
Recomm'n:
Maintain continuity when leaving the session.
Recomm'n:
If possible, capture a user exit & warn of lost data if exiting before completion.
1.8. Security of advice sheets
Location:
PDF resource area of server
User Group:
Users who just want copies of the advice sheets
Issue:
All the advice sheets are available via search engines such as Google, so they can be downloaded without going through the proper
assessment routine to select appropriate sheets.
Relvnt reqmnt: HSE may want the sheets only to be available through the assessment system
Recomm'n:
Make secure if this is important to HSE
87
1.9. Expiry of assessment
Location:
Server
User Group:
All users
Issue:
Assessments expire & are removed after 30 days
Issue:
A user might return to an assessment near the end of the expiry time, and have their assessment deleted when they need it
Relvnt reqmnt: User should not loose data until they want to
Recomm'n:
Consider making the assessment expire if the user does not return to the assessment within 30 days. This should allow users to keep
their assessment if they are working on COSHH-E over a distributed length of time, but still allow finished assessments to be removed.
1.10. Apparent relevance of the advice sheets to the user
Location:
All advice sheets
User Group:
All users who have not used the paper COSHH-E
Issue:
The advice sheets provide advice which is sometimes very specific, but does not necessarily relate to the users own circumstances.
This can make it difficult for the user to relate to the advice, or possibly cause them to think that the advice is not right for them even
though the general principles probably are.
Issue:
Users are very likely to have the wrong expectations of what the system will provide ie. they are likely to expect advice more specific
to their circumstances. The system does not currently establish the correct expectations at the start.
Issue:
Disappointment with the resulting advice sheets offered is very likely unless users understand the limitations of the advice sheets.
Relvnt reqmnt: Users should understand what they will get from the system, & have a realistic understanding of how their goals will be met.
Recomm'n:
The system should meet user expectations, or user expectations should be corrected near the start of the system if possible ie. by use of
the correct learning path for different users.
Recomm'n:
Users should understand how to interpret advice if it is not specifically about their own circumstances.
88
1.11. Forced review of tasks on returning to an assessment
Location:
All pages from StateMixDetails.asp
User Group:
All using substance based advice route
Issue:
After returning to an assessment, user is forced to review all previously completed tasks to get to the task where they want to continue.
Issue:
During the review of the tasks, the ‘Re-Assess’ & ‘Delete’ options appear, when the primary goal of the user is just to return to their
previous point of exit.
Relvnt reqmnt: System should support user in achieving their major goals as efficiently as possible
Recomm'n:
System should support exit & return to where the user left off, without the extensive review process.
1.12. Quantity applied to task rather than substance
Location:
Task and quantity related pages, and page structuring
User Group:
Those who use quantities of substances from more than one size category (small / medium / large) in the same task / process
Issue:
When user enters the quantity used, this is applied to all substances listed for the task. This assumes that the user uses similar
quantities of all substances for a given task. This will result in the user only getting advice for only one range of quantity, when they
conceivably could be dealing with two or three.
Issue:
The user may lose confidence that they have been given the right advice in terms of the quantity being handled.
Issue:
This is a function of the task based design.
Relvnt reqmnt: User should be able to understand how to enter their particular scenarios of substance use within the restrictions of the system, or the
system meet their individual scenario needs.
Recomm'n:
To fit users’ scenarios of use, be tolerant of users’ different ways of using the system, or help user to remain within the restrictions of
the system ie. to separate substances which use very different quantities (if the task-based approach is used).
89
1.13. Advice offered part way through assessment for ‘E’ rated substances
Location:
Hazard classification results pages
User Group:
Users with ‘E’ rated substances
Issue:
Important advice on dealing with ‘E’ hazard rated substances is provided part way through the assessment, and is not provided at the
end with the advice sheets.
Issue:
The inability to provide this advice at the end of the assessment is partly because of the task-based approach. The warning may only
apply to one substance out of several listed for the same task.
Relvnt reqmnt:
Recomm'n:
Important warnings should be provided along with the downloaded advice
Recomm'n:
A warning sheet dealing with Haz ‘E’ substances could be produced and added to the download list
90
1.14. Re-entry of identical data required for same substance
Location:
Task loop from UnitOperations.asp to advice sheets
User Group:
All using substance based advice route
Issue:
If using the same substances for more than one task, the user must re-enter some or all of the same substance information for each task
Issue:
NB. Some of the data could vary from task to task eg. quantity used
Relvnt reqmnt: System to require the minimum of effort, & minimise re-entry of the same data
Recomm'n:
Either support re-entry of data by auto-fill or preferably structure the system so that only the differences between two tasks, using the
same substance, need to be provided by the user.
Recomm'n:
Support substance-led assessment, & capture only the substance information that is different between tasks
Industry-based route
1.15. Duplicated advice sheets
Location:
AdviceSummary.asp
User Group:
All those who might select duplicate sheets
Issue:
In some areas of the industry advice, it is possible to select the same advice sheet twice, resulting in it appearing twice in the final list
of advice
Relvnt reqmnt: Prevent confusion; meet user needs efficiently as possible.
Recomm'n:
Use an interface which shows where the same advice sheets apply to different industry selection.
Recomm'n:
Preferably remove duplication in combination with the above.
91
2. Interaction & graphical interface issues
Substance-based Assessment route
2.1. Home page ‘Home’ btn
Location:
http://www.coshh-essentials.org.uk/Home.asp
User Group:
All new users
Issue:
Home button is clickable but performs no function when user is on home page.
Relvnt reqmnt: Inactive buttons should indicate they are inactive
Recomm'n:
Remove from home page or show that it is inactive, or show when the user is on the home page
92
2.2. Clickable COSHH-E Logo
Location:
http://www.coshh-essentials.org.uk/Home.asp, + most other pages
User Group:
All new users who have not clicked this logo before
Issue:
Clickable logo. Displayed on home page yet links to home page ie. goes nowhere when clicked at home page
Issue:
Does not convey its function to the user ie. return to home page
Relvnt reqmnt: Unnecessary item; clickable items should perform a function; clickable items should be obviously clickable; clickable items should
make their function clear to the user
Recomm'n:
Remove hyperlink;
Recomm'n:
or Remove item;
Recomm'n:
Don’t use as ‘Home page’ link anywhere on system, use the ‘home page’ button
2.3. Top menu bar items
Location:
On all pages except Help pages
User Group:
Most relevance to: New users; Those with low tolerance of IT;
Issue:
Text in menu bar – some items clickable and some not
Relvnt reqmnt: Functions should be immediately obvious; User must use mouse to discover items are clickable or not clickable
Recomm'n:
Clickable items to look like menu buttons, text to look obviously only text
93
2.4. Design of the ‘Go’ button
Location:
Most pages
User Group:
All users, especially new users
Issue:
The ‘Go’ button would generally be expected to be labelled ‘Next’ and to look like a standard interactive button.
Issue:
The ‘Back’ link relates to the ‘Go’ button, but is in a different format
Relvnt reqmnt: Navigation options should be as clear & intuitive as possible to the user
Relvnt reqmnt: The most familiar interactive windows systems components should be used
Relvnt reqmnt: Interface design should be consistent
Recomm'n:
‘Go’ & ‘Back’ buttons should match
Recomm'n:
‘Go’ button should be labelled ‘Next’
Recomm'n:
Both buttons should obey familiar design parameters for interactive buttons
2.5. Duplicate Help link
Location:
On most pages
User Group:
Most relevant to: Experienced users not requiring Help system
Issue:
Constant access to Help system via two Help links not necessary
Relvnt reqmnt: Interface should be as uncluttered as possible; two items performing the same function;
Recomm'n:
Make decision on best location of Help link, and keep.
94
2.6. Multiple routes into Help system
Location:
Home page & EntryOptions.htm
User Group:
New users to the site
Issue:
Users are taken to the same Help page (Introduction1.htm) when they click the ‘Help’ interactions on the Home page and also when
they click the ‘Step1’ on the EntryOptions.htm page. The ‘Worked Example’ also leads into the Help system. This means users can
find themselves revisiting the same pages by different routes.
Relvnt reqmnt: Routes into Help & Training areas need to be clearly structured for the user to understand where links will take them, and where they
have already been.
Recomm'n:
Any ‘Worked Example’ buttons should be clearly part of the Help system, rather than looking like a separate entity.
Recomm'n:
‘Help’ & ‘Worked Example’ buttons would be better labelled ‘Help – Contents’ & ‘Help – Worked Example’
95
2.7. No distinction between types of tasks
Location:
‘Process and tasks’ page (UnitOperations.asp)
User Group:
All users, especially those unfamiliar with COSHH-E
Issue:
Some tasks apply to handling substances, while others apply to using & processing substances, but the two are not distinguished
Issue:
Making no distinction between the two types of tasks makes it more difficult for the user to try & apply the tasks to their own situation
Relvnt reqmnt: Minimise cognitive load on the user
Recomm'n:
Group tasks into handling & using/processing tasks.
96
2.8. Side menu bar
Location:
Home page & others
User Group:
All users
Issue:
All buttons share the same visual priority on the page, but some are much more important than others
Issue:
Some buttons do not explain their function well, especially the lower half.
Relvnt reqmnt: Provide visual priority to the most important interface items
Relvnt reqmnt: Minimise visual clutter & adopt minimalist design
Recomm'n:
Re-think the upper section to match the goals of different user types
Recomm'n:
The lower section of buttons could be grouped under some type of ‘related links’ section
97
2.9. User not warned of PDF file download
Location:
http://www.coshh-essentials.org.uk/help/Glossary.htm
User Group:
Anybody accessing the “Control of Substances Hazardous to Health (COSHH) Regulations” link on the “Index to Terms” page
Issue:
Links to HTML pages and PDF files appear the same.
Issue:
There is no indication that the above link leads to a PDF file (indg136.pdf) which takes longer to download and requires the PDF
reader software.
Issue:
Users must have Acrobat v1.4 or later
Issue:
On lower specification computers, downloading PDF files and loading of the PDF viewer can take some time to complete.
Relvnt reqmnt: User should know the difference between HTML links and links to other formats eg. PDF
Recomm'n:
Distinguish PDF links with a PDF logo + file size
Recomm'n:
Have a summary html page which explains the content of indg136.pdf so that users can make a decision to download or not
Recomm'n:
Make it clear to the user that the Acrobat software is required for this link, and which version of Acrobat is required.
98
2.10. Unnecessary information presented to all users
Location:
‘How harmful’ page of the substance advice route (RPhrase.asp)
User Group:
All users
Issue:
This page presents a considerable amount of text to the user in the ‘Important note’. This text is not relevant to all users, only those
who find that they have risk phrases that are not listed. Because the text is listed as an important note, most users will read it & try to
digest it, with some finding that it is not currently relevant to them at this point.
Relvnt reqmnt: Adopt a minimalist design for both visual appearance & use of text
Relvnt reqmnt: Present only information for the user to complete their current task
Relvnt reqmnt: Minimise cognitive load on the user
Recomm'n:
Rather than include this explanation, possibly include ALL risk phrases, then put the ones which COSHH-E does not use in a separate
area. Attach a link which presents an explanation only if it is needed.
99
2.11. Page link in ‘Index to Terms’ page does not have a meaningful name
Location:
Glossary.htm
User Group:
Anybody using the ‘Index to Terms’ page
Issue:
The ‘Index to Terms’ page includes a navigation link called ‘Index’ which is potentially confusing.
Issue:
The user would be unsure about the function of the ‘Index’ link.
Issue:
The ‘Index’ link leads to a page which is not titled ‘Index’
Relvnt reqmnt: The function of navigation links should be clear to the user at all times; the user should know where they are & where they can go.
Recomm'n:
The ‘Index’ link should be more meaningful and accurate eg. named ‘Help – Contents’
2.12. Navigation in Help – Page identification
Location:
Introduction1.htm
User Group:
Anybody using the Help system
Issue:
Main Help contents page is not given a meaningful title.
Relvnt reqmnt: The user should always be aware of where they are in the system;
Recomm'n:
Main page is titled ’Help – Contents’ or something similar
100
2.13. Flexibility in entering Risk Phrases
Location:
RPhrase.asp
User Group:
Any users wishing to enter RPhrases not listed
Issue:
Users concerned about not being able to enter some risk phrases because they don’t appear on the list.
Issue:
Risk phrases that e-COSHH-E doesn’t deal with are handled by a long text explanation
Issue:
Risk phrases that e-COSHH-E doesn’t deal with are handled by omitting them
Relvnt reqmnt: Users should be allowed the flexibility to enter data in the way they want to
Recomm'n:
Include the omitted risk phrases in the list, but present user with an explanation of why they are not used in the assessment when they
check the phrase box
Recomm'n:
Carry the omitted risk phrases through to the Advice Summary sheet so the user has a record of what phrases were not handled.
Recomm'n:
Provide further advice on how user should deal with other phrases
101
2.14. Context Help – Two different Help subjects presented together
Location:
BoilingPoint.htm & VapourPressure.htm
User Group:
Anybody accessing the ‘Index to terms’ via links in the system for the terms ‘Volatility’, ‘Vapour pressure’ or ‘Boiling point’
Issue:
When clicking on a help link for these terms, additional information is presented in the same help screen ie. definition for two terms is
provided in one page. In the example below, clicking on ‘Boiling point’ gives you a description of ‘Volatility’.
Link to Glossary term ‘Boiling point’
Resulting ‘Index to terms’ information includes Volatility
Relvnt reqmnt: Context help should provide just the help indicated by the link; User should be presented only with information relevant to the current
task which is being focussed on.
Recomm'n:
Separate these help items
102
2.15. Slow loading of R-Phrase page
Location:
‘How harmful?’ page (RPhrase.asp)
User Group:
All users in substance-based advice route, especially those with more complex assessments
Issue:
Response time from server unacceptably slow (typical delays >1min eg. 1:01, 1:06, 1:05, 01:09, 01:05, 01:00, 01:01)
Issue:
Assessors found the delay so long that many thought the system had crashed
Issue:
Sometimes the system appears unresponsive (no ‘waving flag’ or ‘Page loading..’ or progress bar in Internet Explorer) while waiting
for the RPhrase page to load.
Issue:
No progress bar to show time remaining
Relvnt reqmnt: Wait times of this length should provide a progress bar to the user, indicating realistic progress
Relvnt reqmnt: System should not appear unresponsive for such long periods without some feedback to the user
Relvnt reqmnt: System should not be frustrating to use
Recomm'n:
Reduce loading speed
Recomm'n:
Provide realistic indication of progress, if delay is over 2s
Industry-based ‘Direct’ advice route
2.16. Speed of page delivery from server
Location:
All ‘Direct Advice’ route pages
User Group:
All users
Issue:
Speed of delivery of pages is very slow. This can become irritating given the number of pages that need to be gone through
Relvnt reqmnt: Minmise latency; System to be as quick & efficient to use as possible
Recomm'n:
Restructure &/or speed up page delivery
103
2.17. Layout of text & radio buttons
Location:
Most of the ‘Direct’ industry advice pages
User Group:
All users
Issue:
Radio & check boxes should be as close to their related text as possible.
Issue:
The text items are closer to each other than to their radio buttons
Issue:
The layout creates 2 strong visual groups: a group of text items & a group of radio buttons
Relvnt reqmnt: Make pages as easy to read as possible
Relvnt reqmnt: Minimise cognitive load on the user
Recomm'n:
Place radio buttons near to text, & put more space between text items. Essentially, the space between text & radio button should be
less than the space between two text items. Text will need to be right justified.
Recomm'n:
Alternatively, link text to radio button by continuing the horizontal underline
104
3. Structuring & Navigation issues
3.1. Navigation structuring in Help – Use of ‘Back’ & ‘Next’ page links
Location:
Introduction2.htm to introduction5.htm & WorkedExample1.htm to WorkedExample9.htm
User Group:
Anybody entering the ‘Worked example’ help or ‘Guidance’ help from somewhere other than the main Help contents page
Issue:
If user has entered these help pages from other links in the system, then the ‘Back’ link will not take them back to the page they came
from, but take them to a page they might never have visited.
Issue:
If user is not familiar with these series of linked help pages, they will not know that ‘Back’ & ‘Next’ refer to the next/previous pages in
a series of linked pages, ie. these links will not be meaningful.
---------------------------------------------------------------------------------------------------------------------------------------------------------Issue:
The user will only understand the linked nature of the pages if they enter via the overview of these pages which is presented in the
Help main contents page (Introduction1.htm).
Relvnt reqmnt: User should understand where they are and where they can go in the system; Links & buttons should be meaningful to the user
Recomm'n:
If the user is to be linked into these series of Help pages part way through the series of pages, there should be a visible index showing
them where they are and where the ‘Back’ and ‘Next’ links will take them.
105
3.2. Numbered steps on the start page
Location:
‘Getting Started page’ (EntryOptions.htm)
User Group:
All new users
Issue:
Numbering implies sequenced steps to the user, whereas the text describes a number of options.
Relvnt reqmnt: Users should clearly understand their options for proceeding
Recomm'n:
Remove numbering & give options clear titles rather than descriptive sentences
Recomm'n:
Different user groups should be guided to the appropriate option (eg. new experienced; new inexperienced; revisiting; etc)
106
3.3. Maintaining user awareness of where they are in the assessment process
Location:
All pages in the ‘Task loop’ of substance based advice route
User Group:
All users of the substance based advice route with more than one task to assess
Issue:
The third line down where the current task is identified, is not distinct in comparison with the information on the rest of the page.
Some assessors found that they did not have a good idea of where they were in the process of assessment.
Issue:
The progress bar at the top of the page does not provide an indication of the looping required to go through all the tasks, therefore this
information is important to the user.
Issue:
It normally takes the user some time to establish that the tasks they ticked in the previous page (UnitOperations.asp) are being dealt
with one at a time in a loop.
Issue:
When reaching the end of the first task loop, assessors were surprised to be taken back for another round in the next task, because their
concept of where they were in the entire process was not clear.
Relvnt reqmnt: User to know where they are & where they are going
Recomm'n:
System to indicate the looping through the tasks more clearly
Recomm'n:
System to make the connection between the tasks selected in the previous page, and the current task being dealt with, more clear.
Recomm'n:
Making the ‘Task (1 of 12) Transferring’ into a clear, bold sub-title for the page may be a solution to improving clarity
107
3.4. Navigation after re-entry into the system using the assessment code
Location:
‘Process & Tasks’ page (UnitOperations.asp) & pages thereafter
User Group:
Any user re-entering the system to continue assessments
Issue:
After re-entering the system, you cannot navigate to a specific place in the assessment, for example the download page
Issue:
After re-entering the system, you can’t continue where you left off without passing through all the previous tasks to get to the one you
need to continue from or edit
Issue:
Because of the delay in loading various pages, the total delay encountered when trying to continue a session is excessively long
Relvnt reqmnt: The system should allow the user to navigate to previously completed pages, without passing through all the already completed tasks
Relvnt reqmnt: The system should allow continuation of sessions from where the user left off
Recomm'n:
Allow user to at least select the task that they wish to revisit or continue from
Recomm'n:
Preferably allow user to navigate nearer to the exact point where they wish to get to (after re-entering)
Recomm'n:
The faster the system can load the pages, the more acceptable it will be to have to pass through the pages to get to the required one
3.5. Lack of a Privacy Policy document
Location:
Not applicable
User Group:
Users who are especially concerned about HSE knowing their substance use.
Issue:
No dedicated Privacy Policy document could be found, explaining to the user the protection and anonymity of their data. The privacy
policy is alluded to in various areas of the help information, but it would be expected that there would be a dedicated area for this for
easy access by users for who it is important.
Relvnt reqmnt: Maximise use, confidence in, and acceptance of, the system.
108
Industry ‘Direct’ advice route
3.6. Link to printing information
Location:
Production Industries page (Production.asp)
User Group:
All users seeking printing COSHH information
Issue:
Clicking the ‘Printing’ radio button sends the users directly to another site without warning
Relvnt reqmnt: Users need to be warned before being redirected off the site; external links should be identified to the user
Recomm'n:
Either have an internal page with the link + explanation, or use a different interaction from a radio button
3.7. Identical advice in sheets
Location:
Industry Advice – Woodwork – WD01 to WD05
User Group:
Woodworkers
Issue:
The system is structured such that if the user selects several different woodworking machines, the same advice is provided under
different advice sheet titles.
Issue:
The user may resent spending time downloading several identical advice sheets.
Issue:
The assessment advice may be more powerful because each woodworking machine type has a relevant graphic of the machinery
Issue:
The user is very likely to comment on the identical advice for each machine
Relvnt reqmnt: The system should minimise time required to use it
Recomm'n:
A single, more generic advice sheet on wood dusts rather than advice for specific machines might be considered
109
4. Language issues
4.1. “CHIP and ‘dangerous for supply’”
Location:
CHIP.htm
User Group:
Anybody accessing the explanation of the CHIP regulations in the Help system
Issue:
The term “dangerous for supply” is not explained. The title will not be meaningful to most users.
Relvnt reqmnt: Clear use of language to reduce mental effort in use.
Recomm'n:
Create more meaningful title
Recomm'n:
Explain ‘dangerous for supply’
4.2. Use of term ‘Direct advice’
Location:
All presences of the term ‘Direct Advice’
User Group:
All users
Issue:
‘Direct Advice’ is not likely to convey the correct meaning to the user
Relvnt reqmnt: Accessible, meaningful language
Recomm'n:
Replace with a term such as ‘Industry sector advice’ or ‘Industry specific advice’ or sililar
Recomm'n:
Test the meaningfulness of various terms with different user groups
110
4.3. Text suggests advice sheets are offered on the current page
Location:
Summary.asp
User Group:
All users
Issue:
The text on the Summary page suggests that the advice sheets are currently offered on the page
Relvnt reqmnt: Make language accessible and clear
Recomm'n:
This might say ‘..offered to you after clicking the ‘Next’ button..’ or ‘offered to you in the next step..’ or similar
4.4. Use of term ‘Assessment’
Location:
All presences of the term ‘Assessment’
User Group:
COSHH experts to COSHH novices
Issue:
Term ‘Assessment’ having different meaning to users with different backgrounds, possibly leading to an incorrect idea about what the
system will do for the user
Relvnt reqmnt: Terminology right for all user types
Recomm'n:
’Possibly a help link to explain what the phrase encompasses in the system, and where the system fits into a complete ‘Risk
Assessment’ as defined by HSE
111
4.5. Instruction for selecting tasks
Location:
‘Process and tasks’ page (UnitOperations.asp)
User Group:
All those using the substance-based advice route.
Issue:
The instruction does not make it clear that the tasks being referred to relate to the substances, not necessarily to the process that the
user has in their head.
Issue:
The assessors had to spend some time & cognitive effort to realise that the instruction is referring to tasks that are done with the
substance itself, not just tasks that are done as part of the ‘process’ that they might have entered previously.
Relvnt reqmnt: Language must ensure the user forms the correct mental model of what the list of tasks are referring to
Recomm'n:
This might be better worded along the lines of “You may be doing one or more of the following tasks with the substances that you
use…”
Recomm'n:
The task descriptions themselves might convey the correct meaning better if they were grouped into tasks done using the product, and
tasks handling the product (see entry under
112
4.6. Over-complex use of language
Location:
‘How harmful’ page of the substance advice route (RPhrase.asp)
User Group:
All users of the substance based advice route
Issue:
This text was found to be complex to digest, placing cognitive load on the user.
Relvnt reqmnt: Cognitive load on the user should be minimised
Relvnt reqmnt: Language should be as simple as possible.
Recomm'n:
This text should be simplified.
113
4.7. Use of terms ‘Chemical’ and ‘Substance’
Location:
Throughout the site where these terms are used
User Group:
All users, especially those new to COSHH & COSHH-E
Issue:
There is a loss of consistency in places where the terms ‘Chemical’ & ‘Substance’ are used.
Issue:
Although the site says it provides steps to control chemicals, it also provides steps to control some substances other than chemicals
Issue:
Users who need advice on substances such as wood dust & flour may believe that the site cannot help them
Relvnt reqmnt: Consistent use of language & terms
Recomm'n:
Needs to be dealt with by COSHH experts, to decide the extent of the issue & if there is a reasonable solution.
114
4.8. Help on Help
Location:
introduction1.htm
User Group:
Anyone wanting help on help
Issue:
Help text on how to use help links is not clearly worded.
Relvnt reqmnt: Clear easy to read instructions
Recomm'n:
Click the underlined text for help
115
5. Error handling, error tolerance & flexibility
5.1. ‘Page expired’ on using ‘Back’ link
Location:
All pages with a ‘Back’ link
User Group:
All
Issue:
On occasions a ‘page expired’ error message was received from the server, even though the ‘Back’ link was selected only shortly after
entering the page with the ‘Back’ link
Issue:
The page expired event can occur even if the user has moved on to a new page in the system recently
Relvnt reqmnt: The user should be aware of any time limits to their task
Relvnt reqmnt: The system should allow novice users time to learn the system & run at their own pace
Recomm'n:
Reset the time-out when the users interacts with the server in any way, rather than setting an overall time-out at the start of the
assessment
Recomm'n:
Give the user some warning of an impending timeout
Recomm'n:
Before timeout occurs, instruct users what to do to continue without receiving a ‘page expired’ message
Recomm'n:
Instead of automatic timeout, present the user with a dialog that says something to the effect “COSHH Essentials has noticed that you
have not used the system for a while (X Minutes). Would you like to continue with the session?” > Yes / No. If ‘No’, then timeout
session, if ‘Yes’ then renew the timer.
116
5.2. Error after selecting ‘Re-Assess’
Location:
Possibly the RPhrase.asp page, but not certain
User Group:
Any user wanting to re-assess a substance for a particular task
Issue:
After selecting ‘Re-Assess’ radio button for a substance, the following page was received, which appears to be corrupted
Relvnt reqmnt: System robustness
Relvnt reqmnt: System to be as fast & efficient as possible for the user
Recomm'n:
Find out if this is a common error, or a one-off glitch. If frequent, it may need to be cured.
117
5.3. Re-Assess & Delete functions: no ability to cancel
Location:
‘Chemical or Product Name’ page (ChemicalCAS.asp)
User Group:
Any using the substance-based advice route
Issue:
After selecting ‘Re-Assess’ or ‘Delete’ option, the user cannot cancel either of these operations
Issue:
The only way to clear this without going through with the action is to go back to the previous page & forward again
Relvnt reqmnt: System should tolerate user errors, & always allow user to cancel an operation, especially a significant operation
Recomm'n:
‘Radio’ buttons as used here are mutually exclusive but generally do not allow deselection. Either a ‘Cancel’ type of radio button
should be added, or preferably a pair of mutually excusive check-boxes.
5.4. Hazard Override
Location:
HazardClassification.asp
User Group:
Any user that attempts to use the hazard override routine
Issue:
Some hazard groups can’t be overridden, yet the system does not check for this until the user has clicked to the override link, read all
the information that is presented and then tried to override. Only after trying the 2nd override link on the HazardClassificationOverride
page is the user told that this is not possible.
Relvnt reqmnt: The system should not lead the user into functions that it already knows cannot be carried out.
Recomm'n:
The system should disable the override link when override is not appropriate.
118
5.5. Removing tasks
Location:
‘Process and tasks’ page (UnitOperations.asp)
User Group:
All users of the substance-based advice route
Issue:
Tasks cannot be removed after selecting them and starting the assessment process (even if the task which needs removing has not been
assessed)
Relvnt reqmnt: The system should be tolerant of such errors, & allow the user to correct them
Relvnt reqmnt: Users should be able to ‘play’ with the system without running into errors that cannot be corrected
Recomm'n:
Allow users to de-select tasks that have not yet been assessed
119
5.6. Confusion over number of chemicals being assessed (after re-entering the system)
Location:
Chemicals loop, Substance-based advice.
User Group:
Anybody re-entering the system after completing one substance entry
Issue:
Having completed the entry of one chemical of two, then exiting and re-entering e-COSHH-E, the system provides confusing feedback
to the user – the system states ‘You are using 2 chemicals’ while also saying ‘Chemical name (1 of 1).
Relvnt reqmnt: Consistency and logical interface
Recomm'n:
If possible, system should say ‘(2 of 2).
120
5.7. Re-entering system & adding a task
Location:
RPhrase.asp & UnitOperations.asp
User Group:
All users of substance based advice route
Issue:
System returned this error when re-entering an assessment to add a task to the assessment:
Relvnt reqmnt: Flexibility to allow user to learn the system & make mistakes (in this case to add another task)
Recomm'n:
Allow addition of extra tasks after re-entering the system
121
5.8. Editing the ‘Process name’
Location:
UnitOperations.asp
User Group:
Any users wanting to re-define their process name
Issue:
It is possible to change the ‘Process name’ that has been previously entered, and the name persists through the current assessment, but
it is not written into the server database. When reloading the assessment, the name has reverted to the previous original entry
Relvnt reqmnt: Users should be allowed to correct their errors
Relvnt reqmnt: System should not loose edits performed by user
Recomm'n:
Write edited entry to database
122
5.9. Boiling point entry
Location:
‘Form: How much of the chemical can get into the air’ page (DustinessVolatility.asp (Liquid))
User Group:
All users needing to enter boiling point
Issue:
Site feedback suggested that some users have the need to enter a value less than 20C, or do not understand that this would be
considered a gas, even though they might use it in a controlled environment which keeps it liquid
Issue:
Users are not provided with an explanation of why their substance is not handled by COSHH-E
Relvnt reqmnt: System should allow users to enter information flexibly, in the way they want to enter it
Relvnt reqmnt: Error messages should inform the user why their data entry was unacceptable, & provide guidance to a solution
Recomm'n:
If COSHH-E cannot support gasses as liquids, this should be filtered out before the user gets this far
Recomm'n:
System should inform users why substances that boil below 20C are not supported by COSHH-E
Recomm'n:
Users should be given advice about how to assess their gas in liquid form ie. point to alternative advice
123
6. Help & Learning
6.1. Understanding the task-based approach
Location:
UnitOperations.asp
User Group:
All using the substance based advice route
Issue:
The paper based system provides an important piece of knowledge that is not included in the web version:
This paragraph contains key information which sets up the users mental picture of how the system will operate and what it will deliver.
Relvnt reqmnt: Users to have a correct mental model of the system, and realistic expectations of what will be delivered.
Relvnt reqmnt: The system is understandable and quick to understand
Recomm'n:
Provide a higher level of understanding, as provided by the paper system.
124
6.2. Different learning needs of users
Location:
All Help pages
User Group:
All users
Issue:
System needs to support different learning paths for different types of users (COSHH expert; COSHH beginner; COSHH-E paper
system user)
Relvnt reqmnt: System to support learning paths suitable for different user groups with different expertise & experience in COSHH
Relvnt reqmnt: System to support a wide range of users with different backgrounds from different industries
Recomm'n:
Identify & support appropriate learning paths to meet learning objectives of different user types, so that they receive the level of
learning & support that is correct for them
6.3. Support for ‘expert’ users
Location:
Substance based advice route
User Group:
Expert users
Issue:
The system is mainly designed for novice COSHH-E users, & does not support more advanced users who do not need so much help
Relvnt reqmnt: Flexibility to leave behind the learning help of highly supported route for a more efficient route
Recomm'n:
Provide less on-page help, & more provide optional help
125
6.4. Overall function of the system
Location:
Entire system
User Group:
All users, especially those who are not familiar with COSHH regulations
Issue:
Although the best advice sheets available are being selected for the user, the advice sheets may only be providing some input to a full
COSHH assessment.
Issue:
e-COSHH-E gives a strong impression to the user that it will fulfil all their requirements under COSHH regulations.
Relvnt reqmnt: Users need to be aware of where e-COSHH-E fits into the scheme of their major goal ie. to meet their obligations under COSHH regs
Recomm'n:
Provide the learning support required to use the advice sheets as part of a full approach to controlling harmful substances, & make it
clear to the user how e-COSHH-E fits into a full approach
6.5. Context of COSHH-E
Location:
User Group:
New users, with low awareness of COSHH
Issue:
A way is needed to simply put e-COSHH-E in the context of the requirements of COSHH ie. show what it does in terms of the 8 steps
for complying with COSHH.
Issue:
This user group needs more support in understanding where e-COSHH-E fits into the totality of dealing with the regs.
Relvnt reqmnt: Users should understand how the system fits into their wider goals of controlling haz substances.
Recomm'n:
126
6.6. The 8 Steps to good practice for COSHH is not included on site
Location:
Not applicable
User Group:
With some or no knowledge of COSHH
Issue:
The 8 steps guidance is not included on the site, but forms important background for beginners.
Relvnt reqmnt: Advancing knowledge of COSHH to be able to make best use of the system
Recomm'n:
Integrate into structured on-site learning or provide link to HSE source.
127
6.7. Explanation of what is needed before starting
Location:
Help system
User Group:
New users to e-COSHH-E
Issue:
There is no quick explanation of what is needed before starting.
Relvnt reqmnt: Adequate support for learning for new users
Recomm'n:
Base the learning section on a Training Needs Analysis and Learning Objectives
128
6.8. Help suggests gasses are not included in COSHH-E
Location:
FAQ page
User Group:
Any users for who the advice sheets on fumes are relevant
Issue:
FAQ suggests gasses are not covered by COSHH-E, but some users may consider fumes as gasses, which are covered by some advice
sheets
Relvnt reqmnt: Consistency
Recomm'n:
Update FAQ to cover certain fumes
129
5.5.2
Appendix 5.2: Cognitive walkthrough sessions
The cognitive walkthrough session logs consist of lists of events that triggered related issues.
Similar events and issues have been grouped together from the different sessions, and given
descriptive theme titles.
Event theme: User introducing themselves to the system
1. Event:
Issue:
/Home.asp – User attempting to find background info on COSHH legal
requirements & what to do.
User found no obvious way to the info required eg Brief Guide to COSHH PDF.
User initially tried to use ‘HSE’ link.
Notes: User subsequently went to ‘About COSHH Essentials’ button and then found a link
in that page to Brief Guide to COSHH.
2. Event:
Issue:
/Home.asp - User searching for COSHH info & legal duties – clicked on link to
example of advice.
Link to PDF advice sheet was dead (file not found msg).
Notes: This link leads to a 347k PDF file but doesn’t tell the user that it is a PDF or a fairly
significant size – slow links will take a minute or two to download.
3. Event:
/Home.asp – User explores Help system for info on COSHH
Issue:
No apparent relevant advice in Help system – nothing directly related to COSHH.
Notes:
User went on to try ‘About COSHH Ess’ link instead
4. Event:
Issue:
/Home.asp – User clicks on ‘home’ button.
Button leads to nowhere.
130
Notes: Home page not titled as Home.
Notes: User might click on this to check they are starting at Home page.
5. Event:
/simplecoshh.pdf – user opened Simple Guide to COSHH from /about.asp
Issue:
User had some difficulties tracking through the PDF document with the viewer in
‘view by page’ mode ie. Not continuous scrolling of pages – pages jump to next
page if user tries to raise the bottom of the page up the screen to view it.
Issue:
User had to search through PDF doc to find info on steps to take to meet COSHH
Notes: Can Acrobat be controlled ie. Made to open in continuous page scrolling mode?
Notes: PDF document designed to be a paper doc – not designed for internet. This doc
required some wading through to get to the steps required to meet COSHH regs.
6. Event:
About.asp/Indg136.PDF – User accesses link to ‘Brief Guide to COSHH’ from
About page
Issue:
No info on type & size of this file link
Issue:
Info is not easy to digest before starting the system – not in easy format.
Issue:
No indication that this is essential reading – if important, say so.
Notes: The system assumes it is possible to use e-COSHH-E without any background on
COSHH, but system is difficult to understand if don’t have background
Notes: File is 354kb
7. Event:
Issue:
About.asp - After using ‘About’ page user wishes to start
User has to return to Home page, then to start page
Notes: User sought to start from the About page
131
8. Event:
Issue:
/help/introduction2.htm – reading about help on COSHH-E
Blue box text difficult to read. Contrast & colour blindness issues
.
Notes:
9. Event:
Issue:
/help/introduction1.htm Searching for info on COSHH in help pages
Initial impression - a lot of text to take in
Notes:
10. Event: Searching for info on COSHH in help pages, having tried some of the links under
‘Guidance’…
Issue:
Advice seems very general. No distilled, quick to digest advice
Notes: NB Link to the help page is titled ‘Find out what the law says and how COSHH
Essentials works’ but then actual linked page does not use these titles.
11. Event: /help/introduction2.htm User tried this link during search for more info on COSHH
Issue:
Leads to in-depth technical information. Too much information. Was looking for
the basics.
132
Notes: Maybe the key words here (when scan read) are ‘how the system works’ – user
might expect basic info on how system operates rather than deep technical detail.
Not likely to be of interest to the majority.
12. Event: User searching for info on COSHH
Issue:
A lot of layers of information to go through, with no structured navigation.
Notes: Card sorting might be useful for organising the help system information in more
meaningful ways ie. to achieve user goals for information.
13. Event: /help/introduction2.htm - User tried following link …
Issue:
Link only brings up this dialog…
Notes: A lot more info to process. No indication that this is an external link. Might be
better to explain that info is either available in paper format (link to HSE pubs) or
HSE Direct.
User may want hard copy anyway rather than using HSE Direct.
133
14. Event: /help/Introduction3.htm - User accesses link to Brief Guide to COSHH
Issue:
Large amount of info to wade through just to get the basics.
Notes: No indication this is a PDF file (will require Acrobat).
No indication of size (for slow connections).
If there is a required reading list as a primer to using COSHH, this should be
explained.
NB Brief Guide to COSHH also includes links/refs to other docs = another depth.
Most useful info in this doc could be turned into HTML ie. the risk assessment
process chart:
15. Event: User expectations before starting
Issue:
User expects to be able to enter a chemical substance into the system & retrieve
exposure limits & control advice for that substance.
Issue:
Expects a means to record the substances in use & produce a suitable risk
assessment record.
Notes:
134
Event theme: User at EntryOptions.htm (titled “Getting started”)
16. Event: /EntryOptions.htm – User attracted to ‘Direct Advice’
Issue:
User had wrong idea about what Direct Advice would be
Notes: Good that it attracted user, but for wrong reasons
17. Event: /EntryOptions.htm - User decided to ignore advice on getting help & tried to start a
new assessment, hopeful that the system will take them through successfully.
Issue:
User ignored advice to view the background help because they wanted to achieve
task quickly.
Notes: There was no means to provide absolutely essential info to the user ie. Use of the
help system was optional.
Notes: The help introduction pages leave the user to browse a hefty amount of text.
18. Event: /EntryOptions.htm – User chooses option 2
Issue:
User misses out on any background info
Notes:
135
19. Event: www.coshh-essentials.org.uk/EntryOptions.htm – User at the start page.
Issue:
States four choices, but raised a Qn in user’s mind about whether these are stages
to follow.
Notes: Start page needs to tie in with defined user goals & user types– see requirements
analyses.
20. Event: /EntryOptions.htm - User about to start new assessment – reads instruction text…
Issue:
Does everybody know what a SDS is?
Notes: May need background to know this ie. Brief Guide to COSHH.
21. Event: EntryOptions.htm – User back at the entry page after clicking on ‘Start’ button on
home page
Issue:
User has to hope that this will lead to useful information, but term ‘Direct advice’
is fairly meaningless to them. The term ‘Industry specific advice’ was later
suggested as more meaningful.
Notes: This issues previously picked up and exact same phrase suggested.
136
Event theme: User at the ‘Process and tasks’ page
22. Event: /UnitOperations.asp – User enters into the ‘Process & Tasks’ page
Issue:
User expecting to be able to enter substance names but is presented with a choice
of tasks which has conflicted with their expectations.
Notes: Paper system does start with the substance identification.
23. Event: /UnitOperations.asp – User reads instructions for the ‘Process & Tasks’ page &
attempts to select appropriate tasks
Issue:
User presented with a limited number of tasks, most of which are difficult to relate
to. User thinking in terms of their job tasks.
Notes: Instructions do not relate these tasks to the substances in use
24. Event: /UnitOperations.asp – User trying to choose ‘tasks’
Issue:
User thinking in terms of job & process tasks, rather than picking out tasks with
chemicals
Issue:
What about if there are several separate ‘Transferring’ tasks – is it OK to group
them?
Issue:
Weighing – user wondering about measuring out liquids, weighing is wrong term
Notes: This list of tasks appears strange to the user because they are not explained as being
related to subject areas of advice.
137
25. Event: /UnitOperations.asp – User trying to understand the term ‘Process’
Issue:
Term ‘Process’ is ambiguous, & there are no boundaries to what constitutes a
process.
Issue:
Looking at this page, user begins to assume that process has to be broken down
into sub-processes = difficult to do at the computer at this stage in time
Notes:
26. Event: /UnitOperations.asp – User wants to know how to save their work
Issue:
No feedback on this currently given
Notes: ID number given later in next page
27. Event: /UnitOperations.asp – User wants to return Home briefly
Issue:
Returned to Home page but lost assessment
Notes:
28. Event: /UnitOperations.asp – user trying to understand and complete step 1.
Issue:
Seemed to take some time to understand what section 1. is for – what is supposed
to be entered here & why can it be left blank.
Notes: Instructions say please complete the following, but then says you can leave blank.
No instruction should be necessary if completion is self explanatory.
29. Event: /UnitOperations.asp – user trying to understand and complete step 2.
Issue:
Takes some time to understand why these particular tasks are listed & what the
definitions are.
138
Notes: These are tasks which apply to the substance.
How will different users define ‘Process’ & ‘Task’ in their own minds?
NB Maybe something like ‘Advice on doing the following tasks with chemicals, is
available)…
If these terms are not immediately relevant/familiar to the user, this requires a
rairly high amount of effort to read through the explanations.
30. Event: /UnitOperations.asp – User completing task selection
Issue:
User wondering what the ‘none of the above’ is for.
Notes:
31. Event: /images/ChemicalName.gif – user on /UnitOperations.asp page
Issue:
User hoping to be able to use nav bar to actually navigate (rather than just as a
progress indicator)
Notes: Is this feasibly technically with the current system ie. the way it cycles through the
tasks and chemicals sections?
139
Event theme: User entering chemical number and state
32. Event: /StateMixDetails.asp – User completing this entry/interpreting entry options
Issue:
Entry choice 1. appeared to the user to be for if you are not sure what the
component chems in the substances are, and choice 2. appeared to be if you knew
more about the substance. These boxes have therefore been misinterpreted
slightly.
Notes: What if you use a mixture plus other single chemicals in the task?
33. Event: /ChemicalCAS.asp – user has previously entered number of chems & is now
attempting to enter chemical names
Issue:
User feels a lack of confidence that the system will properly recognise the chemical
they have named – the fact that the exact name is not required conflicts with the
user’s idea of what the system should be doing.
Issue:
User concerned that the system might not correctly recognise the name they have
entered ie. user has the belief that the exact chem name will be used.
Issue:
User expected a lookup database which would recognise the chemical name they
start to enter.
Notes: How can users be given the confidence that the system is reliable without using
exact correct chem names?
140
Notes: NB the help link:
exact name is not needed.
leads to an explanation of wht an
34. Event: /ChemicalCAS.asp – User has entered names and clicked ‘Go’
Issue:
Long delay from server causes user to re-click the ‘Go’ button
Notes: Is delay due to building the R-Phrase page from a database?
Notes: Are repeat clicks captured? Unnacceptable delay if repeating many times.
35. Event: /ChemicalCAS.asp – User waiting for next page, scanning screen
Issue:
User has only just noticed the blue box at the top of the page showing x Task of X.
User alarmed at the idea of having to go through several tasks in the same way
(especially if for the same chems).
Notes: Is this system an efficient means to achieve the task?
36. Event: /ChemicalCAS.asp – user wondering what to do if single chem & mixture need to
be entered
Issue:
System cannot accept a single chem plus a mixture.
Notes: System cannot accept a liquid & a solid. This is not because of an error in the
system, but because the system cannot provide separate help for two different types
of substance.
141
37. Event: /ChemicalCAS.asp – User progressing through the ‘Wizard’ interface
Issue:
User feeling slightly lost in the depth of the ‘Wizard’ interface. Although the blue
box at the top of the screen & the progress bar both provide feedback about what
the user has entered and where they are in the system, user still felt the interface
was very linear
Notes: Wizard interfaces are very linear – tend to be suited to not too many steps.
38. Event: /ChemicalCAS.asp – User clicked ‘Go’ & waiting for next page
Issue:
Return of the Risk Phrases page is slow.
Notes: Pages are built on the fly, and the more interface items there are eg. checkboxes, the
slower it appears to be.
Event theme: User entering risk phrases & obtaining hazard rating
39. Event: /RPhrase.asp – User enters the Risk phrases selection page & scans the page for
what to do.
Issue:
User complains of yet more text to comprehend.
Notes: The important notice is not relevant to all users – could trigger it from the non-listed
phrases ie. include the phrases & then if ticked, trigger the message.
40. Event: /RPhrase.asp – User hovers cursor over the risk phrases
Issue:
The pop-up sidebar help was not expected – there is no indication that this help
was going to appear
142
Notes: How to indicate that this help is available? Mini help icons? simple instruction?
Notes: The help system is very useful once the user realises what is going on, but there is
nothing to make the help obvious; the user must figure it out.
41. Event: /RPhrase.asp – User scanning & interpreting the page
Issue:
The sentence in bold was not immediately understandable without examining the
the list of risk phrases below to understand the term ‘groupings’.
Issue:
Big block of text which needed reading several times in order to understand. Also
there is a mixture of ‘Dos & Don’t’s’ in the text, & a mixture of messages.
Notes: Might be better re-phrased something like ‘Some phrases are listed in groups – it is
important to select the group as they appear on the SDS or label.’
Notes: Would be easier to read as bullet points. Also, msg about phrases which don’t
appear is not relevant to some users – is there a better way to capture this eg. list
other phrases in a side box with a help link or simply have a help link which says “I
have phrases which are not listed here”
143
42. Event: /RPhraseList.asp – User tries link to full list of phrases
Issue:
User could not see the point of having the list when then the mouse-over sidebar
help is available on the phrases.
Notes: ‘Click here’ links are not necessary.
43. Event: /RPhrase.asp – User interpreting the risk phrases page & what to do
Issue:
User liked the fact that the instruction tells you where in the SDS to find the RPhrases
Issue:
Risk phrases page very wordy – a lot to wade through & absorb. Use of bullet
points considered to be a better idea.
Notes: This could be shortened to a link to help ‘I have risk phrases which aren’t listed
here’ and then give detailed help – this text is not relevant to all users.
NB Instruction asked user to ‘enter numbers’ but in fact they are ‘selecting’
numbers.
144
44. Event: /RPhrase.asp – User selecting risk phrases
Issue:
User liked the instant feedback help boxes explaining the RPs
Notes: This help responds fast and works well.
45. Event: /RPhrase.asp – User navigates to bottom pf page & selects ‘here’ link
Issue:
Link leads to same help as the sidebar RPhrase help – provided no extra info.
Notes: ‘Click here’ type of links are generally frowned upon. link description implies that
the sidebar help is incomplete. Should perhaps read ‘List of risk phrase
explanations’ & position it near the sidebar help box.
46. Event: /RPhrase.asp - User stops to consider how they are progressing with the system &
how far they have yet to go – this is just one chemical of one task.
Issue:
User feels that there are lots of levels to go through to get to the end.
Notes:
47. Event: /RPhrase.asp – User is returned to risk phrases for the second of two chems in the
task.
Issue:
User expected advice to be related to the chem-task-quantity etc
Issue:
Progress bar has returned back several steps
Notes: Users expectations are not being met ie. expected advice related to an individual
chem.
48. Event: /HazardClassification.asp - User enters the hazard rating page
Issue:
User did not find it immediately obvious what this rating bar meant ie. the actual
classification was not immediately clear
145
Notes: Dark lettering is being used to highlight the active item which does not obey normal
convention for highlighting, although it is bolder and larger, in this case that could
just be because ‘special cases’ are always marked as larger. Surrounding the whole
box would indicate better.
49. Event: /HazardClassification.asp – User comprehending text
Issue:
More text to take in.
Notes: NB1 users tend to read text fully the first time around through a wizard interface,
but not subsequently = may not read this if it appears in subsequent assessments
after not appearing previously.
Notes: NB2 this extra text constitutes control advice, yet it is given here instead of at the
end with the other advice.
50. Event: /HazardClassification.asp – User investigating the use of alternative chemicals
Issue:
User led off into a protracted tangent to try & get info.
Notes: There is no link to the paper version of this, only the electronic version requiring
subscription (at a cost which is not likely to be considered at this point).
Notes: This takes user away from the core task in a potentially time consuming tangent.
Also, this advice does not appear to be available at the end with the other advice.
51. Event: /HazardClassification.asp – User expectations of what the expert override is about
Issue:
User expected sources of expert advice & what to do, but got more detailed text.
Notes:
52. Event: /HazardClassification.asp – User reads the sentence about ‘expert override’
Issue:
User doesn’t know what the definition of an expert is or if they are one.
146
Issue:
User questions whether an expert would be using COSHH-E
Notes: This probably needs a help link before actually going to the override page.
53. Event: expert override page – User clicks to decline the override but is not sent back to the
classification page, but sent on to the next step
Issue:
Confusion for the user about what has happened
Notes: If the user selects ‘back’ at this point, there is an error.
54. Event: /HazardClassification.asp - User enters the Hazard Group classification page &
reads instructions.
Issue:
User reads notice about overriding the hazard classification but wonders what an
‘expert’ is defined as and when this overriding feature is applicable.
Notes: Which users does this feature apply to?
55. Event: HazardClassification.asp – user has entered this page after completing info for a
class ‘E –special’ type of chemical.
Issue:
Disappointment with the link to external site for expert advice. Leads to a home
page not a list of experts or contact details or what to do.
Issue:
Disappointment with HSE Direct link
Notes: The ‘Seven steps….’ document links to HSE direct but requires subscription. There
is no link been made to HSE books for info about the paper option.
Notes: Link appears to be within the system, not to an external link. looks like it leads to a
pathway to override the rating.
147
Event theme: User assessing Dustiness/Volatility and Usage
56. Event: /DustinessVolatility.asp – User attempting to fill in the technical details
Issue:
User now realises that the SDS is required, having missed this requirement at the
start. User suggests a list of what is required before starting would have been
useful.
Notes: Requirement for the SDS is mentioned right at the start, but SDSs are not explained
and this requirement is buried in the EntryOptions start menu (with no explanation
of SDS).
57. Event: /DustinessVolatility.asp – User completed entries & ready to proceed
Issue:
User reports ‘Go’ button on ‘wrong side’ ie. on the left of screen instead of the
right.
Issue:
User nearly clicked the wrong ‘Go’ button because there are two (went for the one
at the bottom)
Notes: Leaving the system idle was not tested (eg to get SDS info) – presumably there is a
time out, but this is not clear – need to test this.
Notes: Leaving the system to retrieve SDS info was not tested ie. state of the system on
return.
58. Event: DustinessVolatility Sol.asp - User enters the ‘Form – How dusty’ page & explores
the definitions of ‘Dusty’. and the Low Medium High categories.
Issue:
All the underlined links lead to the same help screen with the same definitions –
this was unexpected to the user.
Notes: Why the use of the word ‘Form:’ in the title when it is not used elsewhere?
Notes: Seems to default to ‘Hight’ being selected – is this intended?
Notes: Next page dealing with quantities adopts a different (& considered better) format
for the help links.
148
59. Event: DustinessVolatility.asp(Solid) – user attempts to complete ‘Volatility’ section
Issue:
User concerned that to do this requires the SDS and may need to contact supplier if
the SDS does not help.
Notes: These substances are rare – could they therefore be dealt with differently? What
about the most common examples?
Notes: NB if user leaves session and re-enters, they enter the system in a different state to
which they left ie. they don’t return to where they left off. if the leaving/returning
scenario is quite common then this becomes more of an issue. If can’t be solved
then likelihood of leaving the system could be minimised by preparing user more
ie. make user aware of what is needed if possible.
60. Event: /UsageInfo.asp (Liquid) – user attempting to enter a period less than one day eg.
one day a fortnight. User looks at Help info.
Issue:
Help info is intelligible, so user does not know at this point what to do.
Issue:
Loss of confidence in the system.
Issue:
User cannot enter less than one day.
Notes: Inflexibility in this part of the system means that the user has to think hard about
how to make the system fit their circumstances
149
Notes: User does not know what the consequences are of not putting in the usage exactly
according to their circumstances. In fact the ‘break-point’ is 30 mins use in any
one day – it may therefore be easier to ask the user to enter if the average use is
more or less than 30mins per day averaged over, say, 1 month.
Notes: NB this raises the issue of how many tasks you do with the chemical affecting
length of exposure during the day – several tasks could add together to go over
30mins exposure per day, but is this accounted for in the system? Tech basis for
COSHH-E says “The hypothesis underlying COSHH Essentials is that an increase
in Control Approach (1 to 2, or 2 to 3) affords at least a 10-fold increase in
protection from dusts and vapours. Were you to exceed an 8-hour limit value tenfold over 30 minutes, with no further exposure, that would equate to the limit value
over 300 minutes, i.e. less than 8 hours.”
61. Event: UsageInfo.asp – User examining the page ready to complete.
Issue:
User commented on the fact that there is no list of the chemicals which need to be
done ie. this is one of several chems to be assessed for this task.
Notes: Info box at top of page just lists the number of chems being used (for the task).
Event theme: User at Summary page and collecting advice
62. Event: /Summary.asp – Summary page - User enters this page
Issue:
Expecting summary advice on the chemical, but user asked to proceed on to next
chemical
Notes:
63. Event: /Summary.asp – User observes progress bar.
Issue:
Expects advice to come next for this chemical because the progress bar indicates
this.
Notes: Advice is geared up to the most hazardous chemical, therefore there is a risk that
the user has entered substances which use different machines etc, but the advice
will be applied to both.
Notes: Progress bar does not indicate the looping onto the next chemical.
150
64. Event: /Summary.asp – user pauses to assess their situation/location in the process
Issue:
User questioning why more info is being asked for when a rating for the chemical
has been established.
Notes: Indicates a conflict between user expectations and the way the system functions –
did not expect to be grouping the substances. User trying to build a model of what
is happening to understand.
Notes: Users generally don’t want to have to build a mental model of what is happening,
and ideally shouldn’t have to.
65. Event: /Summary.asp – User tries to click on progress bar to get to advice
Issue:
Progress bar is not active
Notes: Wanted to override the restriction of the interface to get to the advice rather than go
onto next chemical.
66. Event: Summary.asp – user enters this page and reads info & instructions.
Issue:
Summary at this stage seems irrelevant to user – not sure why given it. User
hoping they will be told what to do next.
Notes: Text says user should now print off control guidance sheets, but these are not
known to the user at this stage & don’t appear on this page.
67. Event: /Summary.asp – User enters & reads Final summary page (after 2 chems)
Issue:
Summary page implies advice sheets are offered on this page. Only after going to
the bottom of the page to do realise you have to click ‘Go’ and get to the next page.
Notes:
68. Event: /ControlApproach.asp – User evaluating what is offered on this page
Issue:
Commented on amount of text on page again being a large amount.
Issue:
User expected each chemical to be listed with its risk rating and measures to
control the risks, but final page did not meet this expectation. Expected all layed
out in a table.
Issue:
User has low confidence that this will form a proper risk assessment.
Notes:
151
69. Event: /ControlApproach.asp – User opening some of the PDF advice sheets
Issue:
User feels that all advice is very general, more than expected
Notes: User not understanding that specific advice is not necessary, but that generic advice
can be applied because control approaches are the same for similar risk subs &
tasks..
70. Event: ControlApproach.asp – user enters this page
Issue:
User expecting info on what levels are acceptable for the particular chemicals in
use. advice appears very general, not specific to own situation.
Notes:
Event theme: User entering ‘Direct Advice’ option
71. Event: /IndustryOptions.asp – ‘Direct Advice’ Production vs. Service – user enters this
page & studies what is required
Issue:
Had different expectation of what ‘Direct Advice’ would provide. Expected to
type in a topic area and receive advice.
Issue:
User did not immediately understand what was required here – initially found that
this top-level layer of description was hard to relate to.
Notes: Term ‘Direct Advice’ does not describe this section well for many users.
Notes: Note some users could cross these categories and will need to go through system
twice.
Notes: This top level of description requires some users to think about their category when
they had might never have considered these categories before
72. Event: /IndustryOptions.asp – ‘Direct Advice’ – Clicked ‘Go’ after selection
Issue:
Very slow system response
Issue:
User re-clicking ‘Go’ button
Issue:
User thinking that something has gone wrong.
Notes: Response time far too slow
Notes: Are second-clicks of the ‘Go’ button captured to prevent re-sending the page again?
152
73. Event: IndustryOptions.asp – user enters this page and goes through decision process.
User tries to see if both options can be selected.
Issue:
User doesn’t understand at this stage what to do if they consider themselves to be
in both categories.
Notes: Categories relate to the types of advice sheets on offer, not the user’s situation.
74. Event: WorkOptions.asp (Production) - User enters page and examines
Issue:
User notes the different format of the progress bar & that it is not active ie.
functional navigation bar
Issue:
User notes you can’t see whole process to come on progress bar
Notes: Possibly the progress bar should have generic titles
Notes: Progress can’t be predicted because the titles depend on the user selection
75. Event: /AcvtivityOptions.asp (Woodwork) – user evaluating how this page meets their
needs & expectations
Issue:
User does not understand why tasks relate to woodworking rather than using
chemicals - User expecting advice on chemicals – advice is on substances ie.
mainly wood dusts
Issue:
User not expecting such a limited list of tasks
Issue:
Tasks not entirely applicable to user circumstances – was led to believe they would
get info relevant to them
153
Notes: The site focuses on chemicals which means that advice on other substances eg.
wood dusts is unexpected, and user would not use COSHH-E to get advice on
these
Notes: User is not made aware that the ‘tasks’ they are ticking are in fact directly related to
advice sheets – user is left blind through the beginning of the process. There is no
model for the user as to how this system works.
Notes: User not forewarned that there is a limited range of tasks
76. Event: Activity Options.asp – user enters page from WorkOptions.asp
Issue:
12Two items appeared in the nav bar after only one step onto the next page
Notes:
77. Event: User enters Activity Options.asp
Issue:
User looking for more activity descriptions – limited choices + if also spraying,
then would have to enter a different industry area to get the advice
Notes: Choices limited by the advice sheets available ie. relate to sheets rather than the
user. User may need to understand the limited nature of the advice sheets to start
with.
78. Event: User accesses PDF ‘Direct Advice’ sheets
Issue:
Advice seems very general and only partially fulfils their need.
Notes: Given the delays in the response of the system, final results can be disappointing
154
5.6
APPENDIX 6: USABILITY HEURISTICS LIST
General heuristics:
- Status of the system conveyed to the user at all
times
-
Use of familiar real-world analogies &
language
-
User can keep control of the system, & can
use the system on their terms
-
Matching with user goals
-
Help & documentation
-
Latency
-
Internal & external consistency
-
Minimise cognitive load & effort
Specific criteria:
- Feedback provided to the user at all times
- Feedback provided in timely way
- User kept aware & informed
- User knows where they are and where they can
go at all times
- User knows the consequences of any actions
that cannot be undone
- User is kept aware of any time limits
- Links to off-site and on-site info should be
distinguished
- Be meaningful to a very wide range of
professionals from hairdressers to car mechanics
to corporate H&S officers
- Use appropriate levels of language
- Use real-world phrases & concepts for
interacting with the user
- User should have scope to make errors &
change them; Support Undo & Redo
- System errors & dead-ends avoided
- System error messages meaningful, helpful &
provides constructive instructions for recovery
- Continuity of sessions
- Users should quickly become aware of how the
system will meet their needs
- Users should quickly understand how to go
about achieving their goals
- Focused on user tasks & goals
- Cascaded, cross-linked help system
- Indexed & searchable
- Response times need to be fast
- Waits >2s should have a progress bar & active
animating icon
- Waits of ½-2s should have animated hourglass
- Trap multiple clicks if immediate feedback if
not present upon clicking an item
- Consistent terms, interface components & style
within the system
- Consistency with corporate identities
- Consistency with Windows conventions
- Consistency with established standards
- Intuitive interface
- Low-level language
- Minimise need for help & explanation
- Easy access to required info for task
- No need to search
- Less important items less prominent
- Minimalist design to reduce competition for
users’ attention
- Maintained consistency
- Appropriate conspicuity of interface
components/uniformity
- Conformity with expectations
- Efficiency in meeting users’ goals
155
-
-
Flexibility
-
Pleasure of use
-
Accessibility
-
156
Navigation & system structure to be highly
visible & match users’ typical mental model of
how it should work
Easily learnable
Adaptable to learning need & experience
Adaptable to different circumstances of use
Adaptable to different user system speed &
screen size
Error tolerant
Special needs – see accessibility
Visual appeal
Minimalist design
Sparing use of colour
Text size; Colour; Audible feedback
Place interactions in consistent area
5.7
APPENDIX 7: ISO 13407 / TR 18529 SOFTWARE DEVELOPMENT
PROCESS
1
Ensure HCD content in system strategy
1.1
1.2
1.3
1.4
1.5
Represent stakeholders
Collect market intelligence
Define and plan system strategy
Collect market feedback
Analyse trends in users
2
Plan and manage the HCD process
2.1
2.2
2.3
2.4
2.5
2.6
2.7
2.8
Consult stakeholders
Identify and plan user involvement
Select human-centred methods and techniques
Ensure a human-centred approach within the team
Plan human-centred design activities
Manage human-centred activities
Champion human-centred approach
Provide support for human-centred design
3
Specify the stakeholder and organisational requirements
3.1
3.2
3.3
3.4
3.5
3.6
Clarify and document system goals
Analyse stakeholders
Assess risk to stakeholders
Define the use of the system
Generate the stakeholder and organisational requirements
Set quality in use objectives
4
Understand & specify the context of use
4.1
4.2
4.3
4.4
4.5
Identify and document user’s tasks
Identify and document significant user attributes
Identify and document organisational environment
Identify and document technical environment
Identify and document physical environment
5
Produce design solutions
5.1
5.2
5.3
5.4
5.5
5.6
5.7
5.8
Allocate functions
Produce composite task model
Explore system design
Use existing knowledge to develop design solutions
Specify system and use
Develop prototypes
Develop user training
Develop user support
6
Evaluate designs against requirements
6.1
6.2
6.3
6.4
6.5
6.6
Specify and validate context of evaluation
Evaluate early prototypes in order to define the requirements for the system
Evaluate prototypes in order to improve the design
Evaluate the system to check that the stakeholder and organisational requirements have been met
Evaluate the system in order to check that the required practice has been followed
Evaluate the system in use in order to ensure that it continues to meet organisational and user needs
7
Introduce and operate the system
7.1
7.2
7.3
7.4
7.5
7.6
Management of change
Determine impact on organisation and stakeholders
Customisation and local design
Deliver user training
Support users in planned activities
Ensure conformance to workplace ergonomic legislation
157
5.8
APPENDIX 8: A PLAN FOR A USER-CENTRED DESIGN APPROACH
158
5.9
APPENDIX 9: VISITOR ACCESS COUNTS (JULY 2005)
Site Name
America Online (AOL)
no-dns-yet.demon.co.uk
mail.lolacars.com
gtw-13.nhs.uk
82-41-192-137.cable.ubr02.livi.blueyonder.co.uk
213.146.131.202
dsl-217-155-185-53.zen.co.uk
193.130.130.58
82-34-238-67.cable.ubr04.gill.blueyonder.co.uk
unallocated.star.net.uk
no-dns.as5587.net
81.144.164.242
tewseng.plus.com
194.74.54.58
mail.polymerlabs.com
62.25.106.209
glfd-cache-8.server.ntli.net
193.132.24.231
195.92.40.49
cache.aramiska.net
nott-cache-13.server.ntli.net
host81-137-53-139.in-addr.btopenworld.com
www.buildinginsustainability.co.uk
host217-40-26-121.in-addr.btopenworld.com
62.172.209.50
vpi.gotadsl.co.uk
manc-cache-12.server.ntli.net
mrsint-adsl.demon.co.uk
weblock.lubrizol.com
brhm-cache-8.server.ntli.net
leasedchennai-199.214.200.static.vsnl.net.in
leic-cache-6.server.ntli.net
193.120.131.27
193.130.87.58
bluebellhouse.plus.com
raptor.warwickshire.gov.uk
hide-217.nhs.uk
ptr-62.221.219.82.rev.exa.net.uk
parent-proxy-msd6-nat.zqwdrqsz.com
ip02.taunton.adsl.gxn.net
wwwproxy.qinetiq.com
ne-ssingh5.hh.med.ic.ac.uk
brnt-cache-8.server.ntli.net
midd-cache-7.server.ntli.net
wsatkins.co.uk
cmbg-cache-5.server.ntli.net
sgpx.saint-gobain.com
dyn000-020.mdx.ac.uk
138.scansafe.net
129.230.252.1
sotn-cache-6.server.ntli.net
dh111.public.mod.uk
brig-cache-5.server.ntli.net
159
Total Requests
3,685
2,948
2,804
2,786
2,557
1,812
1,732
1,632
1,069
979
962
937
918
907
872
863
863
828
779
777
766
694
682
621
617
609
593
555
547
520
511
499
491
466
464
458
402
356
347
330
302
291
286
278
272
261
254
237
234
220
217
216
215
Visits
2,247
233
88
393
66
58
20
51
19
68
28
21
19
30
22
110
27
23
117
34
81
16
13
28
13
13
56
18
28
59
15
43
13
24
15
15
22
13
82
21
30
13
40
32
15
26
44
22
15
25
18
40
21
Site Name
webcacheM05b.cache.pol.co.uk
lutn-cache-10.server.ntli.net
leed-cache-8.server.ntli.net
195.8.190.131
195.6.68.157
83.217.170.147
dh112.public.mod.uk
testme.essexcc.gov.uk
bagu-cache-7.server.ntli.net
host161.eaton.com
host81-130-200-33.in-addr.btopenworld.com
hide-139.nhs.uk
cdif-cache-5.server.ntli.net
clydesdaleforge.plus.com
81.168.121.50
hudd-cache-2.server.ntli.net
renf-cache-8.server.ntli.net
194.60.85.4
194.168.231.2
host5.businesslink.org
134-146-0-27.shell.com
212.137.45.109
193.132.159.170
pete-cache-5.server.ntli.net
217.33.145.164
mailgate.coal.com
194.200.145.5
colc-cache-5.server.ntli.net
hide-232.nhs.uk
dh212.public.mod.uk
webcacheB12a.cache.pol.co.uk
194.152.88.110
dh210.public.mod.uk
dh114.public.mod.uk
egspd42404.ask.com
host217-37-197-161.in-addr.btopenworld.com
195.144.131.12
gateway.devonport.co.uk
host81-139-35-237.in-addr.btopenworld.com
145.8.163.25
parent-proxy-msd5-nat.zqwdrqsz.com
dh214.public.mod.uk
www.bromsgrove.gov.uk
166.REVERSE.67.17.81.in-addr.arpa
82.152.48.70
65.100.90.164
server3.business2www.com
Total Requests
214
210
208
203
187
185
184
181
175
166
155
154
145
144
143
136
134
132
127
124
123
112
112
111
109
105
104
103
103
101
100
98
97
96
93
89
88
71
71
55
50
39
33
29
23
16
16
160
Visits
13
27
22
19
15
16
37
17
28
15
13
16
23
14
14
15
20
15
13
62
17
15
13
17
15
21
18
15
14
31
14
26
27
28
31
32
13
27
14
13
25
17
33
29
23
16
13
5.10
APPENDIX 10: PAGE REQUESTS FOR INDUSTRY ADVICE SHEETS
Page requests for industry advice sheets
Service & Retail
Foundry
Bakery & Milling
SR01.pdf
44
FD01.pdf
10
FL01.pdf
27
SR02.pdf
88
FD02.pdf
9
FL02.pdf
18
SR03.pdf
10
FD03.pdf
10
FL03.pdf
14
SR04.pdf
1,684
FD04.pdf
23
FL04.pdf
13
SR05.pdf
8
FD05.pdf
2
FL05.pdf
13
SR06.pdf
6
FD06.pdf
6
FL06.pdf
8
SR07.pdf
35
FD07.pdf
16
FL07.pdf
11
SR08.pdf
12
FD08.pdf
10
Total requests
104
SR09.pdf
33
FD09.pdf
3
SR10.pdf
11
FD10.pdf
10
SR11.pdf
87
FD11.pdf
2
MR01.pdf
51
SR12.pdf
34
FD12.pdf
10
MR02.pdf
45
SR13.pdf
69
FD13.pdf
7
MR03.pdf
31
SR14.pdf
152
FD14.pdf
92
MR04.pdf
19
SR15.pdf
38
Total requests
210
MR05.pdf
66
SR16.pdf
33
Total requests
212
SR17.pdf
8
SR18.pdf
38
WD01.pdf
47
Rubber manufacture
SR19.pdf
44
WD02.pdf
32
RB01.pdf
6
SR20.pdf
16
WD03.pdf
25
RB02.pdf
6
SR21.pdf
26
WD04.pdf
15
RB03.pdf
6
SR22.pdf
31
WD05.pdf
16
RB04.pdf
8
SR23.pdf
35
WD06.pdf
21
RB05.pdf
10
SR24.pdf
125
WD07.pdf
21
RB06.pdf
3
SR25.pdf
6
WD08.pdf
25
RB07.pdf
10
SR26.pdf
25
WD09.pdf
16
Total requests
49
SR27.pdf
24
Total requests
218
Total requests
Motor repair
Woodworking
2722
161
6
REFERENCES
The Control of Substances Hazardous to Health (COSHH) Regulations 2002 (as
amended). Approved Code of Practice and guidance L5 (Fifth edition), HSE Books* 2005.
ISBN 0 7176 2981 3.
COSHH: A brief guide to the Regulations 3rd.ed (INDG136REV3). HSE Books* 2005.
ISBN 0 7176 2982 1.
Available for free download (indg136.pdf) at
http://www.hse.gov.uk/pubns/chindex.htm.
COSHH essentials: Easy steps to control chemicals. Control of Substances Hazardous to
Health Regulations (Second edition) (HSG193). HSE Books* 2003. ISBN 0 7176 2737 3
COSHH Essentials online: www.coshh-essentials.org.uk
Cope, M T (2006). Human Factors issues in using the electronic COSHH-Essentials
(internet-based) system: Summary discussion of main findings (HSL report ERG/06/23).
Health & Safety Laboratory, Harpur Hill Business Park, Buxton, Derbyshire. SK17 9JN.
BS 7830:1996: Guide to the design and preparation of on-screen documentation for users
of application software. BSI**. ISBN 0 580 25778 9.
BS ISO/IEC 9126:2001: Information technology – Software product quality (Parts 2-4).
BSI**. ISBN 0 580 36526 3.
ISO/IEC 9127:1988: Information processing systems – User documentation and cover
information for consumer software packages. ISO***.
ISO 9241:1992-1998 (Parts 10-17): Ergonomic requirements for office work with visual
display terminals (VDTs). ISO***.
BS EN ISO 13407:1999: Human-centred design processes for interactive systems. BSI**.
ISBN 0 580 32618 7.
BS EN ISO 14915:2002: Software ergonomics for multimedia user interfaces. BSI**.
ISBN 0 580 40923 6.
BS ISO/IEC 15910:1999: Information technology Ð Software user documentation process.
BSI**. ISBN 0 580 34733 8.
BS ISO/IEC 18019:2004: Software and system engineering -- Guidelines for the design
and preparation of user documentation for application software. BSI**. ISBN 0 580
43290 4.
ISO-TR (Technical Report) 18529:2000: Ergonomics -- Ergonomics of human-system
interaction -- Human-centred lifecycle process descriptions. ISO***.
BS ISO/IEC 25062:2006: Software engineering -- Software product Quality Requirements
and Evaluation (SQuaRE) – Common Industry Format (CIF) for usability test reports.
BSI**. ISBN 0 580 47058 X.
Nielsen, J. (1994b). Heuristic evaluation. In Nielsen, J., and Mack, R.L. (Eds.), Usability
Inspection Methods, John Wiley & Sons, New York, NY.
162
*HSE Books, PO Box 1999, Sudbury, Suffolk CO10 2WA. Tel: 01787 881165 Fax: 01787
313995. Website: www.hsebooks.co.uk.
**British Standards Institute, 389 Chiswick High Road, London, W4 4AL, United Kingdom.
Telephone: +44 (0)20 8996 9001 Fax: +44 (0)20 8996 7001. Email: [email protected]. Web site: http://www.bsonline.bsi-global.com/server/index.jsp.
***http://www.iso.org/iso/store.htm
163