Document type: KP-Lab internal document
Product name and version: “SMAT” annotation tool,, Nov2008 version
Organisers of the test: Karolinska Institutet, Klas Karlgren
Date of the test: Nov 2008 – Jan 2009
Date of the report: 19.01.2009
Editor: Klas Karlgren
Contacts: Klas Karlgren
Emails: -
Purpose/objectives of the test: Usability evaluation
Method 1:Usability inspection; heuristic evaluation combined with an informal cognitive walkthrough
Number and type of participants: 1
Tasks (if task are used): The main task concerned getting started with annotating a movie clip

Executive Summary

Summary of the most important observations of potential problems (n.b. this is a formative evaluation which only addressed potential problems, not advantages):

  • It is difficult to quickly getting started in using the program– users do not easily arrive at a page which they are expecting to get to, namely the page where they can start annotating. Instead they appear to have to go numerous steps of logging in, “structuring” and “configuring” and choosing “sequences”, “phases”, “scenarios” “members” with insufficient information about why these steps are required and what they are needed for.
  • The structure is too complex – or rather, users are forced to step through long paths through the structure before they can start anotating.
  • Layout could be more consistent sizewise and moving buttons closer to the functions/text that they control or activate would render a more logical layout (examples below). Some pages are much smaller than others not contributing to a consistent design of the tool. Other pages are almost identical to each other with hardly discernable differences as to which tabs are provided and a risk is that users fail to see the differences. A consistent size would be preferable and in both cases, several of the parts of the system could be merged in order to achieve a simpler structure.
  • Unnecessary frames on each page gives a cluttered layout.
  • The tool requires that video clips have the flv format which causes some problems since it is not a very common format and therefore demands time-consuming conversions of files.
  • It is difficult to get an overview of video files that are uploaded because they end up in folders which are placed in other folders. Also, interacting with these becomes unnecessarily cumbersome.
  • Carrying out the annotating itself is too cumbersome now (it was simpler in the earlier versions) since users need to constantly pause the video and press a “record” button and moreover switch between tabs while annotating. The layout of the annotation page could also be modified in order to support swifter annotation.
  • After annotating there is not a clear sense of what happens to the annotated video. The list of videos does not indicate whether videos have been annotated or not or by whom.
  • Interaction with the tool often requires unnecessary steps since important functionality is hidden on tabs which cannot be reached because they are inactive (grayed). They can only be reached by opening several folders and clicking particular files. For this reason, links in the system do not lead to the pages to which users expect to be arriving at.
  • Language could be checked and simplified in order to make the tool easier to grasp. Terms that apparently refer to different things sometimes have names that are so similar that it becomes difficult to keep them apart.
  • The difference between “adding” and “uploading” video is not obvious.
  • Likewise, “configuring” and “structuring” sound similar but mean different things and the terms risk being difficult to distinguish
  • Moreover, there are differences between “sequences”, “phases”, and “scenarios” which similarly can be confused.
• Some of the language could be checked (why are e.g., small and capital first letters mixed in some headers, cf. “add Content”, “upload Content”?)
  • Lists of video files (“content items”) and users (“members”) showing information about these should also be the place where they are modified. Now presentation of information is sometimes done in one location but the same information is modified in another. If both were done in the same place the tool would become simpler.

This page is a category under: usability and under Category Of Feedback Testing Data
  Page Info My Prefs Log in
This page (revision-2) last changed on 18:24 25-Mar-2017 by merja.

Referenced by
Category Of Feedb...

JSPWiki v2.4.102