Skip to content
  • Official
  • Beginner
  • English
  • 10 steps · 2 quizzes

Usability Testing Workshop: Run Your First Test

Usability testing workshop: write unbiased tasks, moderate think-aloud, watch a mock session, then find the pattern in five sessions and rate severity.

  • Free
  • No account needed
  • Stop any time, pick up where you left off
Workshopy.io

WorkshopyOfficial

78 lessons · used by 2 teachers

For teachers

Teaching this? Run it live with your class.

  • Students join with a code — no accounts
  • See who’s done and who’s stuck, step by step
  • It’s your copy — change anything before class

Sign in to copy it to your workspace. It’s free.

Your path

10 steps · 2 quizzes · open any step, or read it here

  1. Start here: why did nobody book the room?Start hereThis usability testing workshop starts with a problem every product team meets: a new form went live, everyone on the team found it easy, and still people get stuck. Watch five people try it, one after another, and see where they stop.InteractiveHands-on
    Read this step

    This usability testing workshop starts with a problem every product team meets: a new form went live, everyone on the team found it easy, and still people get stuck. Watch five people try it, one after another, and see where they stop.

    Interactive · Five sessions, one form: where do people get stuck? — try it in the lesson

    Each path ends where someone got stuck. Two places stopped almost everyone, and five short sessions were enough to find them. In this lesson you learn to run that test yourself.

    What you'll do

    • Choose five participants and write tasks that don't give the answer away.
    • Moderate a think-aloud session without helping or leading.
    • Watch a mock session and note what went wrong, as it happens.
    • Find the pattern across five sessions and rate each problem by severity.

    By the end you can

    • Plan and run a first usability test on a real product with five people.
    • Turn five sessions' notes into a short, ranked list of what to fix.

    You need a browser. No research background.

  2. What a usability test is (and isn't)A usability test is simple: a real user of the product tries to do a real task with it while you watch what they do. You don't ask whether they like it. You look at where they hesitate, where they make a wrong turn, and whether they finish.InteractiveHands-on
    Read this step

    A usability test is simple: a real user of the product tries to do a real task with it while you watch what they do. You don't ask whether they like it. You look at where they hesitate, where they make a wrong turn, and whether they finish.

    That separates it from methods that collect opinions: surveys, focus groups, preference tests ("which logo looks more trustworthy?"). Opinions are useful, but people are poor judges of what they will do, and they often rate a screen as easy right after struggling with it. Behaviour is the evidence.

    Three things make it a test:

    1. Representative participants: people who would really use it, not the team that built it.
    2. Realistic tasks: goals they would really have, in their words.
    3. Observation: someone (or a recording) captures what they do and say.

    Try it: sort the six cases below. Two of them are usability tests, and they look quite different from each other.

    Interactive · Usability test or something else? — try it in the lesson

    Takeaway: if nobody is using the product to do something while you watch, it isn't a usability test.

  3. Plan the test: goal, people, moderated or notA first test needs a one-page plan, not a research project. Write down: The goal: one question the test answers. "Can employees book a room for a meeting without help?" is a goal; "test the new form" is not.InteractiveHands-on
    Read this step

    A first test needs a one-page plan, not a research project. Write down:

    • The goal: one question the test answers. "Can employees book a room for a meeting without help?" is a goal; "test the new form" is not.
    • The participants: who the real users are, and how many. Five people from one user group is the usual start (the five-users step explains why).
    • The tasks: three to five, written as goals (the next step).
    • Where and how: in a room, on a video call, or unmoderated with a tool.
    • What success looks like for each task: the room is booked, at the right time, once.

    Moderated or unmoderated?

    Moderated Unmoderated
    Who is there A moderator, in the room or on a call Nobody: the participant follows written tasks while a tool records screen and voice
    Strength You can ask "what did you expect?" at the moment it matters Fast and cheap for many people, at any hour
    Weakness Takes your time, one session at a time No follow-up questions; unclear tasks can't be rescued
    Good for A first test, early designs, complex tasks Checking a fix, clear tasks, comparing many people

    For a first test, run it moderated: you will learn more from watching five people live than from reading fifty recordings.

    Who to recruit

    Recruit people who will really use the product, with a mix of how often and how expertly they use it. Leave out anyone who built it, designed it, ordered the test or administers it: they know how it is supposed to work, so they can't show you where it doesn't. Offer a small thank-you for their time; sessions of 30 to 60 minutes are typical.

    Try it: the test is of the company's new room-booking form. Pick your five.

    Interactive · Pick five participants for the room-booking test — try it in the lesson

    Takeaway: the people you test with decide what you can find. Real users, nobody from inside.

  4. Write tasks that don't give the answer awayA task is the instruction you read to the participant. Most first tests go wrong here: the task tells people where to click, so the test only shows that people can follow instructions. A good task: Is a goal, not a route.InteractiveHands-on
    Read this step

    A task is the instruction you read to the participant. Most first tests go wrong here: the task tells people where to click, so the test only shows that people can follow instructions.

    A good task:

    • Is a goal, not a route. "Book a room for six people this Friday at 2 pm" instead of "click Book a room, then the date field".
    • Doesn't use the screen's words. If the button says Guests, the task says "three colleagues are joining". Matching words turn the test into a search for a label.
    • Gives the details a real person would have: the date, the number of people, a discount code from their manager.
    • Has a clear end, so you and they know when it is done.
    • Asks them to do, not to judge. "Do you like the form?" is a question for afterwards, not a task.

    Read each task aloud once, then hand it over on a card so nobody has to remember it.

    Try it: four draft tasks, each giving too much away. Choose the rewrite you would use.

    Interactive · Rewrite the leading tasks — try it in the lesson

    Takeaway: describe the situation and the goal; let the participant find the way.

  5. Think-aloud: let them talk, don't helpIn a think-aloud test, participants say what they are thinking while they work: what they are looking for, what they expect, what surprises them.InteractiveHands-on
    Read this step

    In a think-aloud test, participants say what they are thinking while they work: what they are looking for, what they expect, what surprises them. The method comes from cognitive psychology (Ericsson and Simon's protocol analysis); in usability testing it is the most common way to hear why someone does what you see them do.

    Before the first task, say three things:

    1. "We're testing the form, not you. You can't do anything wrong."
    2. "Please think aloud: say what you're looking at and thinking, even small things."
    3. "I can't help during the tasks, because I want to see how it works without me. Afterwards I'll answer anything."

    During the session your job is to stay out of the way:

    • When they go quiet, nudge neutrally: "What are you thinking?", "Keep talking."
    • When they ask you something, hand it back: "What would you expect?"
    • Never explain, hint or name a control. If you must step in so they can go on, note the task as not completed.
    • Watch what they do more than what they say. "That was easy" after two wrong tries is a quote; the two wrong tries are the finding.

    Try it: four moments from a session. What do you say?

    Interactive · What do you say when the participant stops? — try it in the lesson

    Takeaway: a moderator who helps ends the test. Neutral prompts, questions handed back, help only after.

  6. Check your planQuiz · 4 questions
    Read the questions

    Question 1. Which of these is a usability test?

    • A survey asking 300 users how easy the app is, from 1 to 7
    • Five customers each try to change their delivery address while a moderator watches
    • A focus group discussing which features to build next
    • The developer clicking through the new screen before release

    Question 2. You want to test a new expense form with five people next week and ask them why they hesitate. Which kind of test fits best?

    • Moderated, so you can ask follow-up questions at the moment it matters
    • Unmoderated, because it is always more accurate
    • A survey, because it reaches more people
    • Neither: test it with the team that built it

    Question 3. Which task is written best?

    • Click Settings, then Notifications, and turn off the weekly email.
    • Do you find the notification settings easy to use?
    • You get a summary email every Monday that you never read. Stop it from coming.
    • Look around the app for a while.

    Question 4. A participant has been silent for 30 seconds, looking at the screen. What does the moderator say?

    • "The button you need is on the right."
    • "What are you thinking right now?"
    • "Is this part confusing?"
    • Nothing: the session should be stopped
  7. Watch a mock session: what to noteDuring a session one person moderates and, ideally, a second person takes notes. Good notes are observations: what the participant did and said, where, and when. Interpretations ("the date field is badly designed") come later, when you compare sessions.InteractiveHands-on
    Read this step

    During a session one person moderates and, ideally, a second person takes notes. Good notes are observations: what the participant did and said, where, and when. Interpretations ("the date field is badly designed") come later, when you compare sessions.

    For every problem, write:

    • The time, so you can find it in the recording.
    • Where: the field, button or page.
    • What they did: the clicks, the wrong turns, the retries.
    • What they said: a short quote in their words.
    • The outcome: recovered, finished differently, or gave up.

    A problem is anything that slows the person down, confuses them or leads to a wrong result. A smooth moment isn't a problem, and neither is an opinion on its own.

    Try it: Noa, a sales rep, books a room for Friday. Play her session and note each moment that is a problem.

    Interactive · Watch a mock session and note the problems — try it in the lesson

    Takeaway: note what you see and hear, with a time. The "why" and the fix come later.

    Check your work · Noa said one thing out loud that contradicted what she did. Paste her last words. — check it in the lesson

  8. Why five users: the curve and its limitsHow many people do you need?InteractiveHands-on
    Read this step

    How many people do you need? In 1993 Jakob Nielsen and Thomas Landauer looked at usability studies and found that the share of problems a test finds follows a simple curve:

    $$\text{found}(n) = 1 - (1 - L)^n$$

    where $n$ is the number of participants and $L$ the share of problems one participant shows you. Across their studies $L$ averaged about 31%. With that value five users find $1 - 0.69^5 \approx 0.84$ of the problems (Nielsen rounds it to 85%), which is where the advice "test with five users" comes from. Each extra person mostly repeats what you saw.

    Try it: run sessions one at a time on the room-booking form. Its problems are illustrative: some hit most people, some almost nobody. Watch where the curve flattens, then try a new group of people.

    Interactive · Run sessions one by one and watch the pattern appear — try it in the lesson

    What the curve doesn't promise:

    • 31% is an average, not a law. On a form with many rare problems, $L$ is lower and five people find less. Laura Faulkner (2003) drew groups of five from 60 participants: on average they found about 85% of the problems, but some groups found only 55%. With ten people, no group found less than about 80%.
    • Rare problems stay hidden. A problem one person in twenty meets will usually not appear in five sessions, however serious it is.
    • Five per user group. If two groups use the product very differently (say, people who book rooms and the admins who set them up), test each group; Nielsen suggests three or four people per group when there are several.
    • It's for finding problems, not measuring. "Four of five failed" says fix it; it is not a percentage you can report as a rate.

    Nielsen's own advice is to spend the budget on several small tests rather than one big one: test five, fix what you found, test five more on the fixed version.

    Takeaway: five sessions find the problems most people meet. Fix them and test again, rather than adding more people to one round.

  9. Find the pattern across five sessionsAfter five sessions you have five sets of notes. Before you discuss solutions, put them on one sheet: a row for each observation, a column for each participant, a mark where that person met it.InteractiveHands-on
    Read this step

    After five sessions you have five sets of notes. Before you discuss solutions, put them on one sheet: a row for each observation, a column for each participant, a mark where that person met it. (Some teams call this a "rainbow spreadsheet"; sticky notes grouped on a wall work the same way.)

    Then read it row by row:

    • A problem three or more of five met is a pattern. It's almost certainly not about one person.
    • A problem one or two people met is still real. Keep it; its severity decides whether to fix it now.
    • A wish ("it would be nice to see a photo of the room") isn't a problem. Put it on an ideas list.
    • Merge rows that are the same problem in different words before counting.

    Try it: here are the five sessions of the room-booking test on one sheet. Mark every row.

    Interactive · Five sessions on one sheet: find the pattern — try it in the lesson

    Takeaway: count before you discuss. The pattern tells you what to fix first; your memory of the most vivid session doesn't.

  10. Rate severity and write the reportNot every problem is equally bad. Jakob Nielsen's widely used scale rates each one from 0 to 4InteractiveHands-on
    Read this step

    Not every problem is equally bad. Jakob Nielsen's widely used scale rates each one from 0 to 4:

    Rating Name Meaning
    0 Not a problem Nobody struggled
    1 Cosmetic Noticed, but changes nothing anyone does
    2 Minor Slows people down; they recover
    3 Major Causes a wrong result or a lot of lost time for some people
    4 Catastrophe Stops most people or gives a wrong result; fix before release

    Three things go into a rating: frequency (how many met it), impact (slowed down, or a wrong result) and persistence (once, or every time they use it). A problem that hits one person in five but makes them miss a meeting can outrank one that hits everyone for ten seconds.

    Try it: rate four problems from the booking test and watch your fix list sort itself.

    Interactive · Rate four problems from the room-booking test — try it in the lesson

    The report: one page

    Keep it short enough that the team reads it the same day:

    1. What you tested and with whom: the goal, five participants, the tasks.
    2. Task results: who completed each task, and who needed help.
    3. The problems, worst first: for each, the severity, how many met it, one quote or clip, and a suggested fix.
    4. What worked well: so it isn't changed by accident.
    5. Next: what to fix, and when you will test the fixed version.

    Takeaway: rate each problem by how many met it, how bad it was and how often it happens. Then fix, and test again.

Comments

You’ll sign in with Google before it’s posted.Comments are public — don’t name your students.
0 / 1,500

No comments yet. Be the first to say how it went.

Using it

Workshopy shares this lesson under CC BY 4.0. Anyone may use and change it, with credit to you. Anyone can learn it here, free. A copy in your workspace is yours to change and run live; your students never need an account.

Versions

  1. 1

    Version 1Current

    · First published

Copies are made from the current version. A copy you already have doesn’t change.

More from Workshopy

All 78 lessons