Announcements

M28-M36 Usability testing procedures

Direct link to the page for usability test reports:
http://kplab.evtek.fi:8080/wiki/Wiki.jsp?page=CategoryOfFeedbackTestingData

The general idea is to organise tests according to the three usability-levels presented during the spring GA 2008. More detailed information on the levels, further elaboration on the aims, means and division of work is available at: Usability levels

In brief the presented and accepted procedures for the different levels are as follows:
Tests on the interface level, including evaluation of basic Graphical User Interface elements and functions as specified are initiated and carried out either by technical partners themselves or in close collaboration between technical partners and the respective Working Knot. In any case Working Knots are responsible for carrying out a Functional testing initiated by the technical partners not later than 2 weeks ahead of the release (see the road maps of WK).

Tests on the interaction level are aimed to test whether the tools are in line with the high-level requirements (i.e. whether they provide appropriate support to carry out the respective user tasks). Usability tests on this level are executed by the pedagogical and professional partners interested in using these tools. The main aim of the Working Knots is to co-ordinate the tests and collect and integrate the received feedback. To allow for a lightweight but systematic analysis of usability issues on this level the usability group has produced pre- and post-questionnaires to be used during piloting of KP-Lab tools. These questionnaires are meant to supplement other tests on the interaction level.

Finally, tests on the activity level, which are aimed to assess the utility of the tool on how well the tools support actual knowledge practices in the real situation (field) are organised, executed and reported mainly by the pedagogical and professional partners. Results of these studies mainly feed back into the refinement and specification of high-level requirements.

The reporting of any kind of usability tests should be done according to the template provided by the usability group (which is available Reporting template ). The reports of the test should be linked to the Usability pages in the KP-Lab wiki (Pages to link the reporting of the tests ). The actual report can be located where it is seen to be fit to place, but the link has to be provided in the usability KP-Lab wikipages.

In case you have any questions on the different usability levels, the described procedures, problems in selecting testing methods appropriate in your case etc., please get directly in touch with the contact persons of the usability group (e.g., Merja Bauters merjab@evtek.fi, Christoph Richter Christoph.Richter@fh-hagenberg.at).

Important addresses:

Post-questionnaire at

The pre-and post-questionnaires, slides presented in the GA are in the attachment as zip-format. You have to uncompress the file. Also this mail itself is in the attachment as doc-format.

If you have any questions please ask!

Very Best
Merja

T 2.4 Meetings

Category Of Usability Heuristics

T2.4.1 Testing and refining the usability heuristics developed so far and implement and evaluate the respective testing procedures.

Table of Contents

Introduction

The KP-Lab heuristics are meant to enable the minimum usability over all the tools developed in KP-Lab. The Heuristics follow the manner of Nielsen’s Heuristics and should be used in the same way, which also means that the KP-Lab heuristics should not be the only evaluation/testing used, but one within other means to evaluate and test (cf. Nielsen 2005b). Below you will find the KP-Lab Heuristics and the main points of executing an expert (heuristic) evaluation. In appendix 1, a template that can be used when executing the evaluation is presented enable, the results of which can be feedback for the technical partners.

Pages referring to this category:

KP Lab Heuristics From M 12
R.2.10 KP Lab Heuristics
R.2.10 KP Lab Hueristics
R.2.6 Usability Guidelines And Recommendations For KP Lab Tools And Methods Releases 1 M 18




This page is a category under: usability

OLD Category Of Evalution Criteria

T2.4.2 Definition of criteria and procedures for usability testing in relation to trialogical practices, taking into account different contexts of use. It is carried out in close cooperation with WP3, including joint meetings and review of reports.
Table of Contents

Introduction

Previously (task force on usability time) seen task for the now called T2.4.2 Definition of criteria and procedures for usability testing in relation to trialogical practices..
We do not have any ready-made answer to these issues yet but we are going to develop more specific “trialogical criteria” by:
  1. making general design principles of trialogical learning more specific in relation to the tool use (criteria developed in collaboration with WP3)
  2. developing more specific usability criteria in relation to various pedagogical and working life practices and specific aims (criteria developed in collaboration with WP8, WP9, and WP10).
New description to be provided according to the new WP2 Description of work.

Below the link to different versions can be found.

Pages that are realted to this category:
Edited Version Of The Evalution Criteria Under Development 060407
T 2.4.2 Definition Of Criteria And Procedures For Usability Testing In Relation To Trialogical Practices


This page is a category under: usability

OLD Category Of Recommendations

Table of Contents

Introduction

The former task force on usability described the first version of recommendations for testing and evaluation. These will be further updated and refined by the T2.4 Development and Implementation of Usability Criteria and Testing-Procedures.

M 24 Recommedations:

  • See attachment for word document templates of the pre- and post-questions (pre-questions(info) and post-questions(info)) Just click the links the docs are down loaded
  • The usability levels to help to understand the different testing and evaluation possibilities

R.2.10 Usability guidelines and recommendations for KP-Lab tools and methods, release 3 (M27)


Pages referring to this category:

Draft Common Questionnaire
M 24 Background Questionnaires
M 24 Common Questionnaire 2
M 24 Guideline And Template Working Area
M 24 Recommendations For Usability Testing Methods
M 24 Reporting Template
M 24 Usability Levels Table
R.2.6 Usability Guidelines And Recommendations M 18
Recommendations M 12




This page is a category under: usability

Category Of Feedback Testing Data

T2.4.3 Specifying and implementing means for collecting general end-user feedback.
Table of Contents

Introduction

Collection of the Usability tests and links to different usability test executed in KP-Lab.
The idea is to have a brief of the study as a wiki page and the full report as an attachment. The brief of the usability study can be copied from here: UsabilityreportBriefWikipage. The full reporting template in WOrd format, which is placed as an attachment to the wiki page can be found here

Testing and evaluations of tools of the KP-environment

Shared Space

M12

Pedagogically related usability issues M12
Using heuristics in evaluating M12
M12 Beta SSp UU usability test of M12 Beta Shared Space
M12 all tests and evaluations of M12 Shared Space (as well as recommendations and heuristics)Report 2.6

M22- M24

M24 Mediawiki preliminary test on the under development Mediawiki for KP-Lab
M 24 SSpA Mock-up (screen shot) Heuristic Evaluation raw material in the attachment below (raw files in compressed archive format i.e., zip-format(info))
M24 Usability report SSp Test
M 24 Heuristic evaluation (Nielsen hueristics)
M24 Shared Space goaround usability test
Usability Report: Think aloud Shared Space M 24

M 28

Visual Model Editor
M28VMEUsabilityReportFHOOE012008

Chat
M28UsabilityReportFHOOE022008Chat

To Do List
M28 Expert Evaluation on To-Do List

M2T (Meeting Management Tool)
M28 Expert EvaluationOn Meeting Management Tool
M28 HUJI 02-2008 Diagnostic Usability Test Template ready just click to add the report

Tag Vocabulary Editor and Tailored View
M28 Expert Evaluation Vocabulary Editor and Tailored View

Chat
M28 Expert Evaluation Chat

M 32

Process View


Real-Time Awareness
M32 Usability report on Real-Time Awareness

Visual Model Editor
M 35 Usability Report on Visual Modelling Editor UH

Chat
M 32 Usability Report on ToDo-List and Chat-UU 01-2008


ToDo List
M 32 Usability Report on ToDo-List and Chat-UU 01-2008

Tailored View
M 32 Usability Report on Tailored view-UH 01-2008


Content View
Usability report EVTEK 01-2008 M30-M31 Diagnostic evaluation


M2T (Meeting Management Tool)

M 36

Process View
M36UsabilityReportOnProcessView071008
M36UsabilityReportOnProcessView131008
M36UsabilityReportOnProcessView141008

Semantic Search
M36UsabilityReportOnSemanticSearch
Usability Report EVTEK 03-2008 M33-M35 Diagnostic evaluation2

Wiki
UsabilityReportEVTEK052009FunctionalTesting
Usability Report UH 05-2009 Diagnostic Evaluation

Community View
Usability report FH-OOE 09-2008 Expert Evaluation
Usability Report FH-OOE 10-2008 Diagnostic Evaluation


Awareness
Usability Report UH 02-2008 M33-M35 Diagnostic Evaluation

Google Docs
Usability Report UH 02-2008 M33-M35 Diagnostic Evaluation
Usability Report UU 01-2008 M33-M35 Diagnostic Evaluation includes: Usability Report UU 01-2008 M33-M35 Diagnostic Evaluation of Google calendar & again ToDo list

Google Calendar
Usability Report EVTEK 03-2008 M33-M35 Diagnostic evaluation
Usability Report UU 01-2008 M33-M35 Diagnostic Evaluation
Usability Report FH-OOE-09 2009 Expert evaluation

Help
UsabilityReportEVTEK052009FunctionalTesting
Usability Report UU 03-2009 Expert Evaluation]

Visual modelling language editor
Usability Report UH 05-2009, Diagnostic Evaluation
Usability Report FH-OOE 08-2009, Field Trial
Usability Report FH-OOE 11-2010, Expert Evaluation

Data Export
Usability Report UU 03-2009 Expert Evaluation
Usability Report UH 05-2009, Diagnostic Evaluation
Usability Report FH-OOE-09 2009 Expert evaluation

SCORM Import/Export in KPE
Usability Report FH-OOE 07-2009, Expert evaluation

SMAT

M28 Expert Evaluation on Semantic Multimedia Annotation Tool
M32ExpertEvaluationOfTheSemanticMultimediaAnnotationToolSMAT
M36Usability Evaluation of the SMAT annotation tool KI-02-2008
M36 KI 02-2008 Diagnostic usability Multi-user collaborative annotation
M36 EVTEK 03-2008 Diagnostic Usability Multi-user Text Audio
M36 UniNe 01-2008 Diagnostic usability Multi-user Text Audio
M36 Annotation scenario management KI 02-2008 Diagnostic usability test with domain experts

Activity System Design Tool (former Change Lab)

M24 ASDT GUI-Level Usability Study, Heuristic Evaluation
M24 ASDT GUI-Level Usability Study, User Tests
Please modify the below, how you have executed them but keep the codes, so that when reporting next time we know which studies have been e.g., combined.
MXX SSp plugin, UH_POYRY 01-2008 Expert Evaluation
MXX SSp plugin UH & POYRY 02-2008 Usability workshop with about 10 users
MXX SMAT integration UH & POYRY 03-2008 Expert Evaluation
MXX SMAT integration UH & POYRY 04-2008 Usability workshop with about 10 users
MXX CL analysis UH & POYRY 05-2009 Expert Evaluation
MXX CL analysis UH & POYRY 06-2009 Usability workshop with about 10 potential end-users
MXX H/P awareness utilization UH & POYRY 05-2009 Expert Evaluation
MXX H/P awareness utilization UH & POYRY 06-2009 Usability workshop with about 10 potential end-users

CASS

Development phase CASS-Query testing on December 2007
CASS Q Usability Report-Expert Evaluation June 2008
CASS Memo UH 04-2008 Diagnostic Evaluation
CASS Memo KI 04-2008 Expert Evaluation

Map-It

M24 HUJI 01-2008 Diagnostic usability test with teachers
M32 HUJI 02-2008 Diagnostic Usability Test

Field trial tests

Shared Space

M24

M24 Experience of the mind map structure of the content presentation in shared space
M24 Interviews of Shared space use and questionnaires
M24 Diagnostic evaluation during the exploration of course content for research purposes
M24 Usability report FH-OOE 03-2008
M24 Usability report FH-OOE 04-2008

M32

M32 Chat usability report from field Metropolia 02-2008
M32 Tag Vocabulary editor usability report from field Metropolia 02-2008

M36

Usability Report EVTEK 02-5-2008 Wiki Googlecalendar Communityview Help

Activity System Design Tool (former Change Lab)


Semantic Multimedia Anntoation Tool

EVTEK 05-2009 Field Trial Multiuser Text Audio Scenario Video

Map-It

M12 Mapit
Related thesis material to be announced later.
M24 Map-ItHUJI - mapit 1st, small scale trial
M24 Map-ItHUJI - map-it trial with teachers

M32

ELTE 01-2008FieldTrial M32-35

End-User Feedback

Suggestion and Experiences after M 32

Suggestion and Experiences after M 36

Pages referring to this category:
Activity System Design Tools Experiences Suggestions
CASSQ Usability Report Expert Evaluation June 2008
Development Phase CASS Query Testing On December 2007
Experiences Suggestions
Expert Evaluation On Semantic Multimedia Annotation Tool
M 12 Beta S Sp UU Usability Test
M 24 ASDTGUI Level Usability Study Heuristic Evaluation
M 24 ASDTGUI Level Usability Study User Tests
M 24 CLTGUI Level Usability Study
M 24 Diagnostic Evaluation During The Exploration Of Course Content For Research Purposes
M 24 Experience Of The Mind Map Structure Of The Content Presentation In Shared Space
M 24 Heuristic Evaluation
M 24 Interviews Of Shared Space Use And Questionnaires
M 24 Mediawiki Preliminary Test
M 24 S Sp A Mock Up Heuristic Evaluation
M 24 Shared Space Goaround Usability Test
M 24 Usability Report FHOOE 032008
M 24 Usability Report FHOOE 042008
M 24 Usability Report S Sp Test
M 28 Expert Evaluation Chat
M 28 Expert Evaluation On Meeting Management Tool
M 28 Expert Evaluation On Semantic Multimedia Annotation Tool
M 28 Expert Evaluation On To Do List
M 28 Expert Evaluation Vocabulary Editor And Tailored View
M 28 HUJI 022008 Diagnostic Usability Test
M 28 Usability Report FHOOE 022008 Chat
M 28 VME Usability Report FHOOE 012008
M 30 Expert Evaluation Of The Semantic Multimedia Annotation Tool SMAT
M 32 Chat Usability Report From Field Metropolia 022008
M 32 Experiences Suggestions
M 32 Expert Evaluation Of The Semantic Multimedia Annotation Tool SMAT
M 32 Tag Vocabulary Editor Usability Report From Field Metropolia 022008
M 32 Usability Report On Process View 071008
M 32 Usability Report On Process View 131008
M 32 Usability Report On Process View 141008
M 32 Usability Report On Real Time Awareness
M 32 Usability Report On Semantic Search
M 32 Usability Report On Tailored View UH 012008
M 32 Usability Report On To Do List And Chat UU 012008
M 35 Usability Report On Visual Modelling Editor UH
M 36 Activity System Design Tools Experiences Suggestions
M 36 Experiences Suggestions
M 36 Map It Experiences Suggestions
M 36 Semantic Multimedia Annotation Tool Experiences Suggestions
M 36 Usability Evaluation Of The SMAT Annotation Tool KI 022008
M 36 Usability Report On Process View 071008
M 36 Usability Report On Process View 131008
M 36 Usability Report On Process View 141008
M 36 Usability Report On Semantic Search
Map It Experiences Suggestions
R.2.6 Usability Guidelines And Recommendations M 18
Semantic Multimedia Annotation Tool Experiences Suggestions
Usability Report EVTEK 0252008 Wiki Googlecalendar Communityview Help
Usability Report EVTEK 032008 M 33 M 35 Diagnostic Evaluation 2
Usability Report EVTEK 052009 Functional Testing
Usability Report EVTEK 072009 Functional Testing
Usability Report FHOOE 052009 Expert Evaluation
Usability Report FHOOE 072009 Expert Evaluation
Usability Report FHOOE 082009 Field Trial
Usability Report FHOOE 092008 Expert Evaluation
Usability Report FHOOE 092009 Expert Evaluation
Usability Report FHOOE 112010 Expert Evaluation
Usability Report Think Aloud Shared Space M 24
Usability Report UH 022008 M 33 M 35 Diagnostic Evaluation
Usability Report UH 052009 Diagnostic Evaluation
Usability Report UU 012008 M 33 M 35 Diagnostic Evaluation
Usability Report UU 032009 Expert Evaluation
Usabilityreport Brief Wikipage
VME Usability Report FHOOE 012008



This page is a category under: usability


Refering Pages

Activity System Design Tools Experiences Suggestions
CASSQ Usability Report Expert Evaluation June 2008
Category Of Evalution Criteria
Category Of Feedback Testing Data
Category Of Recommendations
Category Of Usability Heuristics
Development Phase CASS Query Testing On December 2007
Draft Common Questionnaire
Edited Version Of The Evalution Criteria Under Development 060407
Experiences Suggestions
Expert Evaluation On Semantic Multimedia Annotation Tool
GA Meeting Memo T 2.427.02.2008
KP Lab Heuristics From M 12
M 12 Beta S Sp UU Usability Test
M 24 ASDTGUI Level Usability Study Heuristic Evaluation
M 24 ASDTGUI Level Usability Study User Tests
M 24 Background Questionnaires
M 24 CLTGUI Level Usability Study
M 24 Common Questionnaire 2
M 24 Diagnostic Evaluation During The Exploration Of Course Content For Research Purposes
M 24 Experience Of The Mind Map Structure Of The Content Presentation In Shared Space
M 24 Guideline And Template Working Area
M 24 Heuristic Evaluation
M 24 Interviews Of Shared Space Use And Questionnaires
M 24 Mediawiki Preliminary Test
M 24 Recommendations For Usability Testing Methods
M 24 Reporting Template
M 24 S Sp A Mock Up Heuristic Evaluation
M 24 Shared Space Goaround Usability Test
M 24 Usability Levels Table
M 24 Usability Report FHOOE 032008
M 24 Usability Report FHOOE 042008
M 24 Usability Report S Sp Test
M 28 Expert Evaluation Chat
M 28 Expert Evaluation On Meeting Management Tool
M 28 Expert Evaluation On Semantic Multimedia Annotation Tool
M 28 Expert Evaluation On To Do List
M 28 Expert Evaluation Vocabulary Editor And Tailored View
M 28 HUJI 022008 Diagnostic Usability Test
M 28 Usability Report FHOOE 022008 Chat
M 28 VME Usability Report FHOOE 012008
M 30 Expert Evaluation Of The Semantic Multimedia Annotation Tool SMAT
M 32 Chat Usability Report From Field Metropolia 022008
M 32 Experiences Suggestions
M 32 Expert Evaluation Of The Semantic Multimedia Annotation Tool SMAT
M 32 Tag Vocabulary Editor Usability Report From Field Metropolia 022008
M 32 Usability Report On Process View 071008
M 32 Usability Report On Process View 131008
M 32 Usability Report On Process View 141008
M 32 Usability Report On Real Time Awareness
M 32 Usability Report On Semantic Search
M 32 Usability Report On Tailored View UH 012008
M 32 Usability Report On To Do List And Chat UU 012008
M 35 Usability Report On Visual Modelling Editor UH
M 36 Activity System Design Tools Experiences Suggestions
M 36 Experiences Suggestions
M 36 Map It Experiences Suggestions
M 36 Semantic Multimedia Annotation Tool Experiences Suggestions
M 36 Usability Evaluation Of The SMAT Annotation Tool KI 022008
M 36 Usability Report On Process View 071008
M 36 Usability Report On Process View 131008
M 36 Usability Report On Process View 141008
M 36 Usability Report On Semantic Search
Map It Experiences Suggestions
Meeting Memo T 2.425.01.2008
R.2.10 KP Lab Heuristics
R.2.10 KP Lab Hueristics
R.2.6 Usability Guidelines And Recommendations For KP Lab Tools And Methods Releases 1 M 18
R.2.6 Usability Guidelines And Recommendations M 18
Recommendations M 12
Semantic Multimedia Annotation Tool Experiences Suggestions
T 2.4.2 Definition Of Criteria And Procedures For Usability Testing In Relation To Trialogical Practices
T 2.4 Meeting Minutes 26.09.2007
T 2.4 Meetings
Usability Report EVTEK 0252008 Wiki Googlecalendar Communityview Help
Usability Report EVTEK 032008 M 33 M 35 Diagnostic Evaluation 2
Usability Report EVTEK 052009 Functional Testing
Usability Report EVTEK 072009 Functional Testing
Usability Report FHOOE 052009 Expert Evaluation
Usability Report FHOOE 072009 Expert Evaluation
Usability Report FHOOE 082009 Field Trial
Usability Report FHOOE 092008 Expert Evaluation
Usability Report FHOOE 092009 Expert Evaluation
Usability Report FHOOE 112010 Expert Evaluation
Usability Report Think Aloud Shared Space M 24
Usability Report UH 022008 M 33 M 35 Diagnostic Evaluation
Usability Report UH 052009 Diagnostic Evaluation
Usability Report UU 012008 M 33 M 35 Diagnostic Evaluation
Usability Report UU 032009 Expert Evaluation
Usabilityreport Brief Wikipage
VME Usability Report FHOOE 012008
Wiki Category

Links to help, FAQ and sandbox

Help for KP-Lab Wiki
FAQ
Sandbox for exploring things
You can do experiments like the below one just to find out what can be done and how.
This is a text an in the middle this is a link to a NewPage and in my sentence I use the word Collective Flow[1] because bla bla bla...


[#1] here is the defintion for Collective Flow:

Socially shared positive feeling that emerges across intensive trialogical efforts. The participants become so excited about the shared trialogical object that they forget to worry whether they are as smart, beautiful or successful than their fellow participants. The state of collective flow is temporary and may be accompanied by frustrations and tensions. Well functioning groups provide such intense experiences of collective flow that the participants may never forget it.

See also Flow.


Back to the Trialogical Glossary



This is a WikiCategory
  Page Info My Prefs Log in
This page (revision-52) last changed on 18:24 25-Mar-2017 by merja.
 
JSPWiki v2.4.102
[RSS]