Information technology security management (ITSM) tools support a number of goals, such as protection of networks and data, detection of threats, and management of users and their access. Because of this breadth of activity, ITSM involves technical complexity, as well as requires collaboration and information sharing among diverse stakeholders. Usability is an important characteristic of ITSM tools, although there is little consensus on how to evaluate the usability of these resources.

For this study, Jaferian et al. developed criteria for evaluating the usability of tools that support ITSM. The criteria take the form of a set of principles rather than fixed evaluation measures. These principles – or heuristics – describe common properties of usable interfaces, phrased in language that is easy to understand and open to interpretation. General rules such as these can prompt evaluators to find problems a user might encounter in a real world context.

The heuristic tool is developed from a review of literature on usability and real world problems with ITSM tools. These problems were then interpreted in light of a theory of human activities. After combining similar issues according to the theory, 7 heuristics for evaluating ITSM prevailed, each of which is supported and elaborated by more specific ideas.

To evaluate the usability heuristics 28 participants applied the tool to various scenarios describing ITSM situations. Some of the participants used the ITSM heuristic tool, while others used a heuristic tool designed by Nielson that is commonly used in other domains of human computer interaction. The results of evaluations that were supported by each tool were compared. When compared with the Nielson tool evaluations, uses of the ITSM tool identified more problems and more severe problems, and also led to fewer false positives (such as problems that appeared to be usability concerns but in fact stemmed from technical constraints of the program).

The Nielson heuristics led evaluators to find additional issues not identified with the new tool; given this, a combination of the heuristics may be appropriate in order to evaluate ITSM usability. There is a correlation between the evaluators’ number of years experience in Human Computer Interaction with the number of problems reported, but not with the severity of problems identified. Although an evaluator with more experience may be able to point out a higher number of problems, identifying potentially serious issues with usability may something non-experts can support.

This study demonstrates that it is possible to evaluate usability of ITSM tools and - given that severe problems with usability can be identified and addressed – this process is important to effectively supporting ITSM.