Complementary role of two evaluation methods in the usability and accessibility evaluation of a non-standard system

Abstract

Usability, which is generally defined in terms of application effectiveness, efficiency and user satisfaction, is one of the focus areas of human-computer interaction (HCI). Accessibility is the design of systems that can be perceived, understood and used by people with varying abilities. Although accessibility concerns are aimed at making systems usable for people with disabilities, support for direct accessibility, the built-in redundancies in an application that enable as many people as possible to utilize it without system modifications, is beneficial to people with or without disabilities. Different usability evaluation methods (UEMs) are available. Selecting between the various methods can be influenced by the type of system being evaluated. The Digital Doorway (DD), a non-standard computer system deployed to promote computer literacy amongst underprivileged communities in South Africa, was evaluated using the heuristic evaluation method and a field usability study. The heuristic evaluation method revealed a large number of usability and direct accessibility-related problems, some of which could be classified as low-severity problems. The field study showed additional problems that affected the successful completion of user tasks. Since a number of these were a direct consequence of the context of use, they were not recognized as problems by expert evaluators. The study showed that the heuristic evaluation method can be optimized by complementing it with another method that involves user participation and is, preferably, carried out in the intended context of use.