One comment on “User Evaluation – The SUS Results

  1. We are using the standard SUS test as well for our thesis, but we are concerned on how to exactly interpret the results. You seem very confident in drawing specific conclusions after performing the test. For example, you say:

    “It seems that some functionality, primarily the Challenge Tracks, feel out of place or are simply not integrated well enough.”

    How do you know this? Did you ask the users to motivate their answers? Or did you combine the results from a think aloud session while paper prototyping with the results from a SUS questionnaire?

    Either way, one of the main critiques/suggestions we got with our first evaluation methodology was that SUS is too vague as an evaluation tool. We had to specifically come with some form of evaluation that is less general and more focused on what we were doing. We can strongly advise you to do the same. In your case, that means finding an evaluation method that ties in closely with gamification. The best way to search for something like that would be to look at other gamification projects and see how they evaluate their applications. We are aware of how tight schedules can be and doing some literature study (on evaluation methods) now might seem like a step backwards but it really isn’t.

    We’re also not implying you haven’t thought of this before, but the current state of your blog does not reflect your thoughts on this.

    To summarize: Testing for usability with SUS is great, but not enough. We (and I guess everybody from the HCI-group) would be very interested in seeing a blogpost about other evaluation methods you want to use (specifically to test some aspects of the gamification), and what results they yield. Good luck :)!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s