The 3 most important things I've learned about moderating research
June 4, 2018
1. No two sessions are the same
This seems like an obvious one. Yet, I never understood how much a session could vary purely based on who your participant is. This is why knowing who your target audience is so important.
For example, I could be moderating research tests with two women who, on the surface, tick the same boxes: the same age and same household income for example. But one is a working professional and the other is a stay at home mother. The working professional breezes through the website, completing tasks easily and full of praise for the design. Her 45-minute session scheduled could be done in half the time. When it comes to the stay at home mother, she struggles. She hates the design. She can’t find what she’s looking for. Our 45-minute session turns into 60 minutes plus even though we're looking at the same website and the same discussion guide.
Two completely different sessions. It’s fascinating. This is why we test - because although we can assume, based on our experience, the only way to truly know the answer is through research with real users. And in the spirit of avoiding assumptions, it’s worth mentioning that you could have two near-identical participants, same profession/demographic, who’ll go on to have completely different experiences. It really does vary from person to person.
2. My responsibility for setting the tone & the participant's overall experience
Building on what I've already said about my hypothetical participant who struggled, as you can imagine, those hour-long sessions are a SLOG for the participant. So it’s down to me, as the moderator, to try and make the experience outside of the usability as pleasant as possible. We have a policy of letting participants struggle to do things themselves at least twice before intervening. This can be difficult to watch, but the suffering of one participant will potentially help us make it easy for hundreds of thousands of future users, on the real thing. My guidance can help a participant remember some positives from the situation. Even if the website was impossible to use and they ended up completely frustrated by it.
This starts by gauging their mood and the kind of person they are. I like joking around to make them feel comfortable, but if they seem introverted or are in a bad mood then this isn’t appropriate. If they mention they’ve had a bad day beforehand I’ll ask why so that they can vent. I try to remember any personal details they mention about themselves to bring up later. That way they know I’ve actually paid attention to what they’re saying.
I also try not to interrupt or talk over them. This can’t always be avoided, so if it happens I make sure I apologise, ask them to continue and try hard not to do it again. Paying close attention to their conversational pattern can help too. Participants ultimately want to be heard, so I try to be the best listener possible.
I’m not going to pretend that every session is sunshine and rainbows. But I do enjoy watching our assumptions being proved right, wrong, and everything in between. When I first began, I was SO nervous because whilst I’m good at pretending otherwise, I’m actually fairly shy. That, along with trying to avoid any technical hitches, made it extremely nerve-wracking, to begin with!
After a few moderating research sessions, I found that it’s nothing to worry about. Aside from sticking to the discussion guide and avoiding leading the participant through the session, there’s no real pressure on the moderator, except to be pleasant. And it’s so interesting. You meet lots of different people from all walks of life and you get to watch a design succeed or fail in real time. You get proved right and you get proved wrong.
That hour spent with your participant is actually invaluable because can save so much time later on in the design process. It ensures you don’t build the wrong thing and that's why user research really is the backbone of UX design. And if you don’t believe that then you’re doing it wrong.