Whereby or Lookback for remote user testing?
You plan to do remote user-testing or interviews, and you wonder which tooling to use… Here’s my view!
Enter the contestants! #
Lookback is a tool specifically for remote usability-testing. You can use it for lve testing (an interviewer is present during the test) or self testing (people go through a list of tasks in their own time). Lookback records the screen, face and audio of the participants while they do the test, which you can watch back later. Observers can see a live-stream and chat with you as interviewer — your participants won’t see how many observers there are. It’s possible to write notes in Lookback and have timestamps linked to the video. Furthermore, there’s a smartphone app so you can use it to user test your own apps.
Whereby #
Whereby is a tool to facilitatie online meetings. It acts as a virtual meeting-room. It’s not specifically built for remote usability testing.
So. Who’s the winner? #
Looking at the description I gave for both tools, it seems like there’s not contest: Lookback is the clear winner.
However…
- Lookback needs a plugin and even then onlyworks on Chrome. There’s no way to get it to work on Firefox or Apple Safari. That is a drawback! We need to tell all our research-participants they have to install Chrome and a plugin. Some of our participants are on locked-down computers and cannot install anything themselves. In comparison: Whereby runs on Firefox, Chrome and Safari — without any installation
- Lookback forces participants to have a microphone. I like to call my participants using a regular phone call first, it’s reliable. During the call I setup the video connection. Sometimes it just won’t work and I use the phone for audio and an online tool only for the screenshare. With Lookback this approach is impossible. If a participant doesn’t have a microphone, it’s impossible to set up video or screenshare. I know this might sound like an ‘edge-case’, but for me this happens quite a lot. You’ll be happy to know that Whereby will work without camera- or microphone-access! Lookback Participate on smartphone is even worse: it won’t let you start a call without a headset
In Dutch it says: “Share your screen, choose what you want to share”, I selected “Your entire screen”. So why is the Share-button (bottom right) not active??? It’s a very confusing interaction-pattern in Google Chrome
- Lookback screensharing is complicated. In my last usability-test, 9 out of 10 participants could not get through the three ‘easy-steps’ of Lookback without my help. This is not entirely Lookback’s fault: setting up screensharing in Chrome is convoluted, you need to click the screen you want to share even if you only have 1 screen, otherwise the button share screen won’t be activated . Whereby’s screensharing in Chrome is equally difficult to setup, but the saving grace here is that you can do it after setting up the rest of the call. Participants will first experience the happy “it works!” -feeling, before we struggle going through the screenshare-setup together
- Lookback is expensive. Lookback costs $50 per month, per researcher. If you want to export notes and video, it costs $100 per month, per researcher. That’s crazy! Whereby is free. If you want to lock the room (a good idea), it costs $9 per month. If you want to record the call, there’s an added $5 per month.
There’s more annoyances (why do my participants need to enter an email-address to participate? Why can’t I chat or send a link to my participants during a call? Why is the flow of calling participants needlessly convoluted?Why the need for the“lookback-floating window” — participants continously lose or close it, etc.), but these are minor problems.
No more Lookback, ever? #
I’m not saying never. Lookback has several saving graces. Selftesting. Easier for observers. Smartphone testing. Making notes in Lookback is very nice, with their automatic timestamping. Streaming is very reliable, even with a lot of observers. If the connection is bad on your end, you can rewatch the stream in good quality. But to me the participant-experience is more important, and that just doesn’t work.
What do you think?
So how do you user test with Whereby anyway? #
To end this article on a positive note, I’ll give a quick rundown on my setup for usability testing and interviews in Whereby:
- Recruit participants
- by screener or from a predefined list / participant pool
- Call participants by phone for planning a date/time
- I sometimes email them with a form to pick a date/time, but calling is much nicer: you immediately know if there are any problems. Always confirm through an email!
- Trial-run!
- A 2 minute call in a Whereby-room, trying out the participants’ connection, microphone and webcam. If necessary, we try out the screensharing as well.
- Run the test
- I wait 3 minutes to see if participants ‘show up’ in the Whereby-room. If not: I call them using a normal phone. If there are problems with their internet-connection or microphone, we keep the audio-connection through phone and run screenshare and/or video through Whereby. I record video through Quicktime or Lookback — the advantage with the latter is that the team can observe immediately
As always, thanks in advance for your feedback!
Most of my remarks will apply to other tooling as well. Use what’s most comfortable for you ;)
In short:
- Zoom, WebEx and Teams: it’s possible to use it without installing an app, but they make it difficult to do. I find that a hurdle for my participants
- Skype: you need an account and there are different types of Skype (business and personal) that can’t ‘talk’ to eachother (Skype for business is being phased out for Teams, though)
- Google Meet: Google keeps changing the name and functionality of this product (Hangout? Google Talk?). Besides that, the interface is complicated