In this paper, the authors rethink the way HCI researchers use and perceive so-called usability evaluation. The main message is: “the choice of evaluation methodology – if any – must arise from and be appropriate for the actual problem or research question under consideration.” In their rethinking, they looked at the heavy push for usability evaluation from academia and research communities. Then they explained why usability evaluation – a scientific method – can yield weak science. Further, they discussed how usability evaluation, when used inappropriately, will jeopardize potential innovative designs or emerging technology. Finally, they summed up with a few to-dos for HCI researchers to embrace usability evaluation in a way that avoids most of the drawbacks aforementioned.
What to do:
- Usability evaluation is just one of the many methods that comprise our user-centered design toolkit, and that it should be used only when appropriate;
- A usability evaluation at a particular point in our design cycle might/might not produce anything meaningful;
- Stop the blanket insistence on usability evaluation (think about alternatives);
- The formulatic way we used to do usability evaluation might result in weak science;
- It is important to look to other disciplines to consider how they judge design worthiness.
One should get rid of the idea of usability evaluation. It is more important to identify what is the problem, what is the solution, how can I know how the problem is solved by this solution.