The below questions do not take into account what discussed about the different levels of asking questions for example from the extended Pilots (to be planned).

Please add comments/addition, suggestion etc. directly in the wiki page, in italics or using colour cooding xxx .
E.g. merja (14.01.): I think question should be formulated as follows xxxx.
E.g. merja (14.01.): I think question should be formulated as follows xxxx.
You can also use strike through
e.g. a) Which way did you like the content view?

Dead line for the Commenting, suggesting corrections additions to very draft of common questions by Thursday 17.01

Usability related question areas - very draft!

eva (18.01.): I think we need to consider and define in which context the questionnaire should/can be used and for what purpose. In which of the following contexts: pilot, extended pilot, research case, usability trial (see Glossary).
merja: As I mentioned above these questions do not considered to be straight forward used in the extended pilots. These have been thought to be used inside KP-Lab. If some questions or parts make sense to use in the extended pilots we can discuss it. It was also mentioned in the virtual meeting that maybe there should be a separate questionnaire for the usability and for pedagogical "issues" which would be tied to the "package" we offer for those who are willing to take KP-Lab tools and/or practices into use, i.e., we offer training and support and they have to report whatever we come up with.

Questions of the background given in the beginning of the course (setting)

This set of questions should be given in the start of the course (or at least not too late from the start). We need these kind of questions (for design) to know the tools that the students are used to work with for being able to use the interesting and appropriate ideas these tools hold, i.e. the look and feel of the tools, what kind of functions and in what way the tools offer these, etc. It tells what the students are habituated with. Therefore to get this information from others also and not only from EVTEK would be helpful.
These kind of questions were asked in summer from students from Helsinki University and it worked well. These questions have been formed together with the designers, to be able to pinpoint the "categories/groups" of tolls that provide relevant information for the designers.

Background of the users’ knowledge and previous practices or uses of technologies What do people actually know about existing technologies?

1. Have you been using in studies or work any of the following “tools” (mark which ones)
a. Chats (e.g. Skype, MSM, GoogleTalk)
b. Editing Wikipages (not only using e.g.
c. Working/studying environments (such as Blackboard, WEBCT, Moodle, etc)

2. What is favourite manner to communicate/keep contact with fellow students if face-to-face is not possible (list from the most preferred to the least...or something like that)
a. E-mail
b. Chat
c. Through virtual meetings
d. Xxxx

Crina: 3. I would suggest a question directed at working on common objects: What program(s)/tool(s) did you use before when writing/making an assignment in collaboration/a group assignment?
Crina: 4. Maybe also a question about experiences with existing technologies: To which extent did the tools you used until now support your work in the courses you attended? Please explain?
Crina: How do you see the role of technology in this course?
Crina: How do you consider technological tools can support learning during this course/the making of the assignment (depends on the set –up of the course).

Minna (23.1.08) By the way, should we formulate the questions so that they apply also to other settings than courses? Not all usability testing is done related to some course.

General questions about the course (UH)
These questions we (EVTEK) will send (in the week 4) for the students to answer (so we added the questions that UH might want to use also.)
merja: I did not know before the meeting what are the other partners going to do with their pedagogical questions, so this was sort of n example of the possibility to the questions could be delivered also int he same time, without taking sides which way arrange the questions - first ped. then usab....or some other manner....As I understood, you will all use separately what you want if you use questions at all, or something like that.

1. What kind of thoughts and expectations do you have about the course, its goals and forms of studying?
2. What do you personally want to achieve by taking part in the course?

eva (18.01.): How do these questions relate to usability (overlaps with wp8?)? When asking general questions then I would prefer to ask general questions that take the tool and the context (the course) into account.
Minna (23.1.08) Actually I would delete these questions from this usablitity questions set. Including them here related to the concrete situation that Merja's course in EVTEK was starting. We had already decided that we will not use those pre-questions at all that were planned in WP8 last spring. But since Merja was going to give a usability pre-questionnaire for students anyway, we decided to add these two questions there. But this was perhaps only our special case and does not relate to other settings that might have other kind of pre-testing or pre-questioning. Delete these from here?.

Questions given in the end of the course (setting)

Minna (23.1.08) I would suggest to first ask about general impressions of the new technology; there might be something that we cannot anticipate that might have been central in students' experience. In our WP8 questions we had the following questions about technology, perhaps some of them is useful as such or in a modified form:
Which technologies and tools were the most important for you in this course and for which purposes did you use them?
Which features of these tools have been most helpful and which have been least helpful?
Can you think of any additional functionalities of the tools that would have been valuable for you?

3. Idea of questions: Is the conventional/habitual way the one that people want to stick with?
a) Which way did you like the content view?
b) Which way did you like the comments tool?
c) Which way did you like the community tool?
d) Which way did you like the vocabulary versus free tagging?
e) Which way did you like the awareness features (e.g. session log, online information, etc)?

Other formulation idea fro these questions:

  • What is your (general) impression of the …? or
  • What can you tell about the …? or
  • Could you tell what your (general) impression was about…?

4.Idea of questions: What does the system remind them of?

eva (18.01.): For me the purpose of this information is not clear. In which way does this information help us?
merja: I am not sure if it help us/you, but it does help the designers to know to what the students associate the tools with. Which further defines the conventions of what the students are used to and thus helps to guide the design of the UI and function logic of KP-Lab tools. As mentioned above, these questions have been generated together with designers. The formulation of the questions is totally open thus, all the ideas above and below are very welcome to make these easier to understand...

a) After the use of the Shared Space application (KP-Environment) did it remind you of any other tool you have previously used?
b) if yes, what tool and how it was similar?

General that might have an interest to ped partners:
1. Idea of questions: Do students or teachers find anything useful about the systems they used?
Crina: Do you consider that the SSp application was useful during this course?

a) Please describe to what the Shared Space application (KP-Environment) was useful?
b) Was it better than any other tool you have been using before in this type of project work (change the term to fit whatever your students happened to be doing)?
Crina: Do you consider SSP better (tricky word) than other tools you used before for performing this type of tasks/assignments? Can you explain why yes/not?
c) What kind of “things” were now possible but were not been before?
Crina: What kind of activities/ actions/tasks/collaboration etc. did SSp make possible and which were not supported by other/old tools?

eva (18.01.): Here we could also integrate general questions that address functionality which is seen to be missing related to the context of use.
merja: It is a bit like pre- and postquestions since the first set of questions are given in the start of the course and the second the above is given in the end of the course.

Intermediate questions that can be delivered to the students

e.g. in cases that focus on usability and include additional usability methods such as observations, videotaping, log-analyses
merja: The below set of question we (EVTEK at least) have been thinking of using with selected group of students (and a teacher - not me) that has been followed more tightly (e.g., by observing and/or video taping). These questions could be used as guideline for interview or as a questionnaire. Therefore, the idea with these is not to be given to all students. We see that these questions combined with the other methods help to see how the "feeling" that the tools and use provide might change during the course.

Idea of questions: Practices/uses of the tools

1. Which of the Shared Space Application tools have you been using /did you (and your group) use during the course?
a) commenting
b) note editor
b-1) If using note editor and commenting: which one was better for you and why?
Crina: b-1) If using note editor and commenting: which one served better your purposes and actions when working on the assignment/project? Please, explain.

c) Did you use semantic tagging?
Crina: c-1) If yes, what for?
c-1) If yes, how did you use it and did it make sense to use? How did it help you?
Crina: c-2) Did you find it easy to use? Was it useful when working on your assignment/project?
d) Did you find the awareness features (i.e., session log, information on who is on-line, history of the actions and notification ….) useful?
d-1) If they where useful, tell why and how they were useful
Crina: Did you find them (name them here) useful? Can you explain why yes/not?
d-2) Was something missing from the awareness features that you would have needed? Describe what these features could be.
e) Did you use wiki?
e-1) If yes, what did you use it for?
Crina: e-2) What was your impression/experience with it (something like this, about the way they experienced working with the tool)

2. Idea of questions: How much do they incorporate technology into their collaborative work?
merja: We are concentrating here with parts that we definitely need some information and not try to cover all that exists in SSpA. That would be nearly an endless list.
a) Did using the Shared Space application (KP-Environment) help you to organize your work? Please, explain how not or how it did.
b) How did the use of the application help or support you (and your team) in producing the reports/deliverables/etc together?
Crina: How did SSp support you and your team/group in your collaborative work on common object (report, deliverable, project, etc)
c) Did you use any of the tools in the application to brainstorm, share and develop ideas together?
c-1) If so, what tools did you use for any of the activities: brainstorming, sharing and developing and for which purpose did you use them?
Crina: c-2) In which way did this tools support your work (something like this…)?
d) How did the tools help you to do iterations (re-use, change or built upon your previous productions of documents, mock-ups, timetables etc)?
e) How using wiki help you to write “documents” together? Crina: (I would suggest “in collaboration”)

eva (18.01.): As already proposed, I would also integrate questions about what was missing in which context (activity).
for us the context is known, which is included into the report, especially since these sets of questions were thought to be used with other methods in the context.

What EVTEK can ask from students since they have used the previous system

1. Idea of questions: How do the new improvements/tools seem to have been implemented?
a) Was the commenting functionality better now than in the previous version or any other (tell which other tool) commenting tool you have used?
b) Was the overall appearance of the Shared Space application (KP-Environment) better, more structured now than in the previous version?
c) Was the process view improved in the way you suggested last spring?
d) Did the tagging make more sense in this version in comparison to the previous version?

This page is a category under: usability and under Category Of Recommendations
  Page Info My Prefs Log in
This page (revision-25) last changed on 18:24 25-Mar-2017 by merja.

Referenced by

JSPWiki v2.4.102