Developing usability heuristics with PROMETHEUS: A case study in virtual learning environments

Ismael Figueroa, Cristhy Jiménez, Hector Allende-Cid, Paul Leger

Research output: Contribution to journalArticlepeer-review

17 Scopus citations


Heuristic evaluation is one of the most widely-used methods for evaluating the usability of a software product. Proposed in 1990 by Nielsen and Molich, it consists in having a small group of evaluators performing a systematic revision of a system under a set of guiding principles known as usability heuristics. Although Nielsen's 10 usability heuristics are used as the de facto standard in the process of heuristic evaluation, recent research has provided evidence not only for the need of custom domain specific heuristics, but also for the development of methodological processes to create such sets of heuristics. In this work we apply the PROMETHEUS methodology, recently proposed by the authors, to develop the VLEs heuristics: a novel set of usability heuristics for the domain of virtual learning environments. In addition to the development of these heuristics, our research serves as further empirical validation of PROMETHEUS. To validate our results we performed an heuristic evaluation using both VLEs and Nielsen's heuristics. Our design explicitly controls the effect of evaluator variability by using a large number of evaluators. Indeed, for both sets of heuristics the evaluation was performed independently by 7 groups of 5 evaluators each. That is, there were 70 evaluators in total, 35 using VLEs and 35 using Nielsen's heuristics. In addition, we perform rigorous statistical analyses to establish the validity of the novel VLEs heuristics. The results show that VLEs perform better than Nielsen's heuristics, finding more problems, which are also more relevant to the domain, as well as satisfying other quantitative and qualitative criteria. Finally, in contrast to evaluators using Nielsen's heuristics, evaluators using VLEs heuristics reported greater satisfaction regarding utility, clarity, ease of use, and need of additional elements.

Original languageEnglish
Pages (from-to)132-142
Number of pages11
JournalComputer Standards and Interfaces
StatePublished - Jul 2019


  • Evaluation methods
  • Heuristics
  • Methodology
  • Prometheus
  • Usability
  • Virtual learning environments


Dive into the research topics of 'Developing usability heuristics with PROMETHEUS: A case study in virtual learning environments'. Together they form a unique fingerprint.

Cite this