Usability Report FHOOE 02-2008 - Chat
Document type: KP-Lab internal document
Product name and version: Chat prototype (M28) on Mielikki: One-to-One and Context-based Chat
Organisers of the test: Eva Zöserl
Date of the test: 04 July 2008
Date of the report: 07 August 2008, 24 September 2008 (update)
Contacts: Eva Zöserl
This report gives an overview of a diagnostic evaluation carried out on July 4th, 2008, using the Chat prototype (M28 release) which was integrated in the Shared Space Application on Mielikki. Both, the one-to-one chat as well as the context-based chat were available. Since the one-to-one chat did not work properly at the time of the evaluation, it was decided that the participants get a general picture of the one-to-one chat and then start focusing on the context-based chat. The objectives of this evaluation have been to test the usability and utility of the functionalities implemented so far in a real world context. The outcomes of this evaluation are intended to provide feedback on the quality of the functionalities developed so far but also to better understand users’ needs and their implications for further development.
All in all, the evaluation took about 1 hour and has been carried out 3 times with groups of two persons per evaluation. 5 of 6 persons were bachelor students at FH-OOE. One person has already finished the degree programme “Engineering for Computer-based Learning” and is working at the campus in the position of an eLearning coordinator since 2 years. To assess the usability and utility of the Chat tool in a real world context, the participants were asked to carry out a group task. Within the scope of the group task the participants had to create a context-based chat which is related to an empty note content item. The context-based chat was used for communication and coordination issues, while the note was used for documenting the outcomes of their chat discussion.
Methods used included pre- and post-questionnaires as well as observations made by the instructor. Additionally, all relevant content items like the context-based chat protocol and the note in which the participants entered their results have been part of the evaluation. Besides this, all user interactions have been recorded by the usability testing tool MORAE.
All in all the evaluation revealed the following strength and weaknesses:
The possibilities of directly relating a chat conversation to certain content items in order to contextualize the conversation as well as saving these conversations were seen as main strengths. Additionally, it was positively mentioned that the background (content view) is still accessible while chatting and that it is possible to additionally open content items. All in all the context-based chat is seen as a useful tool which has the capability of facilitating collaboration on documents and tasks.
On the other hand the participants identified one main weakness which rather belongs to the Note Editor tool. While it was agreed that working on a document (e.g. note content item) and the simultaneous use of a chat are important for collaborative creation of documents, it was not possible to see the content of a note when another user was editing this note at the same time. It was claimed that it is important to access or at least to see the current content of an item while it is being edited. Simultaneous communication and coordination would then be easier for the chat partners.
Most participants just answered that the system was not working correctly than providing concrete usability feedback. Nevertheless strengths like communicating with people that are not enrolled in the space (e.g. external experts) as well as communicating privately within the working environment were identified.
A weakness of the current one-to-one chat is the lack of information about available contacts. This information (e.g. user profile) is useful to search and contact external people. In addition, some participants were irritated about the amount of available users when they added a contact. They merely expected users who are enrolled in the space. The lack of private chats with more than two users was also criticized.
In conclusion the following main recommendations were derived:
- Ensure the simultaneous display of the chat as well as the content of content items (in our case: notes). For example, when a chat partner is currently editing a note, the other chat partner should at least see the current content of the note. This would ease the communication and coordination when they are chatting about and simultaneously working on a certain content item.
- Always show the context of the chat, even when the user hides relations.
- Pre-define different colors for different users.
- Provide formatting functionality within the chat (font, size, color).
- Let the user be able to insert breaks in his chat text.
- Use a consistent and familiar area for changing the chat window position or provide an explicit button.
- Use unique names for newly created chats in order to avoid confusion. By default the chat should not have the same title as the content item to which the chat belongs.
- Provide a familiar button to minimize the chat window.
- Display the first and the last name or at least simply the username. Additionally, provide further information about the users when clicking onto the username (e.g. link to the profile).
- Make the initiation of context-based chats more obvious.
- Provide awareness features: Users should see if there are currently discussions without opening the chat itself.
- Provide further information about users that can be added to the personal contact list (e.g. link to the profile).
- Make the initiation of one-to-one chats more obvious. The small icon of the contact list is difficult to find. Moreover, allow initiating one-to-one chats also in the user community view.
- Allow private chats with more than two users (private group chats).
This page is a category under: usability
and under Category Of Feedback Testing Data