Use Case Evaluation (UCE): A Method for Early Usability Evaluation in Software Development Kasper Hornbæk
Department of Computer Science
University of Copenhagen
Rune Thaarup Høegh, Michael Bach Pedersen & Jan Stage
Department of Computer Science,
Aalborg University
Use Case Evaluation (UCE): A Method for Early Usability Evaluation in Software Development Kasper Hornbæk
Department of Computer Science
University of Copenhagen
Rune Thaarup Høegh, Michael Bach Pedersen & Jan Stage
Department of Computer Science,
Aalborg University
Aim and Motivation
Use Case Evaluation (UCE): Usability evaluation based on use cases
Usability problems are cheaper to solve early in the development process
Identifying usability problems early in the process is difficult with the current software development practice
Usability work is usually separated from core software development activities
Most usability work takes place late in the software development process
Use cases
Available early in the development process
Already part of many development methods
Valuable means for integrating usability in the software development process
Use Case Evaluation (UCE)
Fully dressed use cases (Cockburn) are recommended
List of guidelines assist the inspection (11)
Based on heuristics from Heuristic Evaluation (9)
Supplemented with guidelines from other methods (2)
Evaluation product
Assessment of the usability of the system; a list of usability problems
Assessment of the quality of the use cases
Usability Problem
Definition: ”An aspect of the system that will hinder or delay the user in completing a task, be difficult or impossible for the user to understand, or cause the user to be frustrated”.
Example of Fully Dressed Use Case (partial)
Procedure for Inspection of Use Cases
One or more evaluators
Brainstorm
Use cases inspected one by one
Note problems that may be predicted
Systematic inspection based on 11 predefined guidelines
Use cases inspected one by one
Note problems that may predicted while employing the guidelines
Where may a guideline be breached?
Fruitful to go over all use cases at least twice
Asses the overall quality of each of the use cases
If more than one evaluator
Create a joint problem list
Guidelines for Use Case Evaluation
Empirical Study I
4 evaluators with 2-8 years of experience in HCI after obtaining masters degree
Health care application (HealthMonitor)
Monitors elderly persons’ medical conditions in their home
Four fully dressed use cases was described for the HealthMonitor
Avg. 472 words long and consisted 6-19 steps
Evaluators received descriptions of
The experimental procedure to be followed
The UCE method
The four use cases
An explanation of the HealthMonitor’s general use context
Empirical Study II
Usability problems reported by
Title
Place(s) where found
Related use case
Guideline breached
Severity rating (cosmetic, serious or critical)
Matching of problems into a joint problem list
General assessment of the use cases
Evaluators opinions on using the UCE method
Comparison with think-aloud usability evaluation of the HealthMonitor
Five user sessions
Analysed by Instant Data Analysis (IDA)
Analysed by conventional video based analysis
Results I
Results II
Discussion
Large portion of usability problems identified through both UCE and conventional method
Several other usability problems were assessed as being useful
Additional benefits from inspection based on use cases
Early focus on usability issues in a natural way
May uncover and emphasize non-functional requirements
May improve overall quality of the use cases
Still need to be empirically documented
Potential improvements on UCE:
Style of writing use cases
Used guidelines
Inspection across use cases for inconsistencies
Limitations & Further Work
Did not assess impact of UCE evaluation in a real-life context
Not a strict experiment
Participants not randomly assigned to think-aloud or UCE
Partly conducted by authors who also had developed the method
Follow-up study by other researchers necessary
Possibly with non-expert participants
Despite the limitations
Our paper suggest that inspection of use cases may help introduce effective usability evaluation early in the software development process
Comments