Field tests attempt to put new interfaces to work in realistic environments for a
fixed trial period. Field tests can be made more fruitful if logging software is used
to capture error, command, and help frequencies plus productivity measures.
Game designers pioneered the can-you-break-this approach to usability testing by
providing energetic teenagers with the challenge of trying to beat new games.
This destructive testing approach, in which the users try to find fatal flaws in the
system, or otherwise to destroy it, has been used in other projects and should be
For all its success, usability testing does have at least two serious limitations: it
emphasizes first-time usage and has limited coverage of the interface features.
These and other concerns have led design teams to supplement usability testing
with the varied forms of expert reviews.
Written user surveys are a familiar, inexpensive and generally acceptable
companion for usability tests and expert reviews.
The keys to successful surveys are clear goals in advance and then development
of focused items that help attain the goals.
Survey goals can be tied to the components of the Objects and Action Interface
model of interface design. Users could be asked for their subjective impressions
about specific aspects of the interface such as the representation of:
o task domain objects and actions
o syntax of inputs and design of displays.
Other goals would be to ascertain
o users background (age, gender, origins, education, income)
o experience with computers (specific applications or software packages,
length of time, depth of knowledge)
o job responsibilities (decision-making influence, managerial roles,
o personality style (introvert vs. extrovert, risk taking vs. risk aversive, early
vs. late adopter, systematic vs. opportunistic)
o reasons for not using an interface (inadequate services, too complex, too
o familiarity with features (printing, macros, shortcuts, tutorials)
o their feeling state after using an interface (confused vs. clear, frustrated vs.
in-control, bored vs. excited).
Online surveys avoid the cost of printing and the extra effort needed for
distribution and collection of paper forms.
Many people prefer to answer a brief survey displayed on a screen, instead of
filling in and returning a printed form, although there is a potential bias in the