I’ve been moderating user tests for several years now and the one thing I’ve learned is that no two design projects are the same. This means that they always require their own unique discussion guide for use during user testing. That discussion guide is vital because it includes the questions, tasks and themes that you, as the moderator, need to understand in order to reach your client’s target objective. As the name suggests, it guides the discussion between the moderator and the user during the testing process.
Obviously there’s no one-size-fits-all when it comes to writing a discussion guide but there are some things you should always include to get the best possible results. I thought it would be useful to share them and so, without further ado, here are what I think are the 5 key elements to include in your discussion guide:
Here at Natural Interaction, we have the words *remember to ask why* at the top of every discussion guide we write.
Why? Because you can gain so much knowledge by including that little three-letter word: why. During a test you can see what the user is doing, but you don’t always know why they’re choosing to do that thing a certain way. By making sure you ask that question during every one of your sessions you’ll be able to identify trends or discrepancies in the way different people use the site. This is by far the most important point in this blog - even if you don’t bother reading past this point, please remember to ask why!
Obviously, some people will do exactly this without needing to be prompted. Great! But some people will lean more towards telling you what they think you want to hear. This is especially true if they think you designed the site. We’re wired to be polite, to not offend people.
Often, users are out of their comfort zone during a moderation session. This means they’re more likely to over think what they’re saying instead of just responding naturally to what’s happening. Saying “don’t worry about offending me, just say what you really think” usually helps break the ice and reminds them we really do value their thoughts!
We'll provide the insight you need to improve the usability, satisfaction and performance of your website or appFind out more
Just like your real life customers, users will very often get sidetracked or accidentally click on things that aren’t supposed to be part of the test. So it’s really, REALLY important they’re able to easily return to whatever it was they were doing before. The website should give them an easy, visible way of doing this without them needing to click click click on the back button.
Once you’ve designed this easy, visible way of going back to the home page, the next step is to test it. It might not actually be as easy or visible to the user as it seems to you. Users will get frustrated very quickly if they feel lost on a website and will often give up on it altogether if they can’t achieve their goals quickly and easily.
Separate to them actually completing tasks, when they first arrive on a page it’s a really good idea to ask them what they’d do next. This is to ensure they really understand the purpose of the page they’re on and are not just reeling off what they can see in front of them.
It’s obviously also a good way of testing whether they actually know how to proceed to the next step of a journey / next page etc. I shouldn’t need to explain why they need to know this though - if you don’t know then you’re probably barking up the wrong tree career-wise!
While you may get someone who feels they’ve said everything they need to say, often this question gives your participant a good opportunity to mention anything they didn’t get a chance to during the test, or expand further on a previous comment/issue they’d had earlier on.
When you study the participant journey as a whole (there was a really good talk at UX Bristol on this that I wanted to link to but I can’t remember what year / who gave it) often the participant will remember things post-session that they wished they’d mentioned during the test. While this isn’t a fool-proof way to avoid this, it does help the participant get some closure & gives another opportunity for you to gain some good feedback. Although there’s nothing stopping you giving them your email address in case they think of anything later on either!
So there you have it: 5 key elements to include in your discussion guide. Everyone will have an opinion on makes a good discussion guide, but these fundamentals are a good place to start. Happy writing! Oh, and if you find it too difficult, drop us a line and we can always write one for you!