Ideas for refining the recommendation for testing

This section outlines first impressions, which have been derived from the use of the usability testing methods presented in report R 2.5.2, during the field trials testing usability of the Shared Space Tools (especially Shared Space GUI).
The ideas presented here cannot be easily generalised, since the only partners who executed usability tests where FH-OOE and EVTEK. Some general ideas have been received also from other partners. The ideas are presented below.

1. 1. In general it might be good to have an example session presentation in WK’s or WK modules where someone who has executed usability tests, presents an example of a usability test or evaluations s/he has executed on KP-Lab tools, prototypes or mock-ups. This kind of presentation could include for example the following topics:

  • a. Describe the purpose of the test,
  • b. Why the particular method was used,
  • c. How the test was planned, organise, executed and analysed
  • d. Present the challenges, lacks and mistakes occurred/made during the execution.

This would help the distribution and the selection of testing methods as well the executions of the tests/evaluations. 2. 2. Another request has been to provide more guidance or templates for reporting. This request is also in reaction to the different formats that the reports were received in. Many of them were lacking the purpose of test, an understandable description of the method and execution. In addition, most were missing the credibility statement of the tests. In the report 2.5.2 there is a template(s) for collecting the usability problems, but it might be worthwhile to have a template for reporting the test(s) and results. One template example could be to report the test and result according to the KP-Lab heuristics.

3. 3. KP-Lab heuristics were used in different manners. The heuristics were used as guidelines for the issues that the system should provide, or in case the test was concentrating on a specific usability aspect (for example consistency and error messages) the appropriate parts of KP-Lab heuristics were taken as a guide. The KP-Lab heuristics were also used for categorising the found usability problems.

4. Checklists of the system’s functions have also been up every now and then as a good addition to the different kind of tests during different phases of the development. The uses of checklists seem to fall more into the grey testing area between usability tests and technical system or module tests. Despite this, it is seen to be a worthwhile idea to prepare checklists of the functionalities of the different modules (tools) developed in KP-Lab. These checklist then can be used by WK members to check that all functionalities according to requirements are present, technical partners can test that all functionalities work properly and for usability tests the checklists can work as background material for knowing what functionalities the system/modules/tool actually has. A suggestion for the place of the checklists has been KP-Lab wiki (an example of checklist NOTE: it is not complete and has only been created to support EVTEK’s inside practices.)

5. To enable reuse and learning from the previously executed usability tests and evaluations it is suggested that the testing documentations are in future linked to the particular WK they belong to as well as to KP-Lab wiki (Category Of Feedback Testing Data, In this manner, a library of the tests and methods would be gathered under the task T2.4, still the tests and results would also be available in the appropriate WK.

6. Collaborative creation of “usability questionnaires” was needed. This kind of practice could be organised under T 2.4 and have the following parts:
  • Generating common background questions
  • Finding if there are similar purposes for testing and generate common questions for these test purposes with modifications to fit the content
  • In cases of special context or purpose particular sets of questions could be generated, which would amount to a library of specific questions. These three different parts could them be reused and mixed to fit further needs that arrive.
  • The generated questionnaire parts should include descriptions of:
  • a) Purpose of test (what the answers are thought to answer for),
  • b) Context (before, during, after use, etc.),
  • c) Analysing methods of the answers

7. The previously made recommendations (Report R 2.5.2) and the cycle of the methods should be checked if it fits the new WP2 description of the work of the WKs.

The previously made recommendation can be found from Report R. 2.5.2, which can be downloaded from:

M24 Guideline And Template Working Area

This page is a category under: usability and under Category Of Recommendations
  Page Info My Prefs Log in
This page (revision-14) last changed on 18:24 25-Mar-2017 by merja.

Referenced by

JSPWiki v2.4.102