Test Usability Frequently

Usability tests are the most effective way to test a product's user experience. Continuous testing verifies verifies that the product and team are focused on solving people's real problems and creating a great user experience for the product. We've found that having a testing plan early is the best way to consistently run user interviews and usability tests during the project. A starting point for the project's interviews and testing should be biweekly. This sets the expectation that interviews and testing are needed and the team should discuss if that plan will work for their individual project. Testing should always accommodate the users and project needs.

We're testing the software, not you.

Usability is the measure of how easy it is for a user to reach an outcome.

We think about usability testing similarly to test-driven development: writing falsifiable outcomes for users. Our outcomes are written in the form of a script that's written in the same way as a jobs story (opens new window).

When I arrive at work I want to review the team's status updates so I can help any blocked teammates.

Once the script is written, we find testers.

The most representative candidates are going to be sourced from our existing userbase. Send out a tweet, add a banner to our site, or add a link to our newsletter. Even pre-launched projects have a mailing list to use.

When we have trouble sourcing or aren't interested in existing users, craigslist can be effective to find candidates. Our office manager puts a posting on craigslist, schedules them to come into our office, and pays them $30 for their time after the test.

We have a simple template for finding people on craigslist.

Title: Get a $25 gift card to help us test our iPhone app

We're looking for people to use our iPhone app and give us feedback.

We'll have you test our iPhone app by performing a handful of tasks, giving us feedback as you do so. You should be an iPhone user but you don't need to be any kind of expert; in fact, we'd prefer people who consider themselves average users.

We'll need about 30 minutes of your time and will give you a $25 gift card. The sessions will take place this Friday, [date], between 9am and 4pm.

Our office, you can find out more about our company by visiting our website: https://www.codecarrot.net.

If you are interested, please click the link below and answer a few questions:

[link to form]

After the tester has arrived, we introduce ourselves and explain the process. There's no need to be strictly formal, we want them to be at ease. To have a relaxed user test, it's important to remind the user of a few things:

• "We are looking for your honest feedback."
• "None of what you're about to see was made by me. There's no way you can hurt my feelings."
• "We're testing the software, not you. It's not user testing, it's usability testing."

If we are filming them, we ask them to sign a simple consent form.

Thank you for participating in our usability research. We will be recording your session to allow CodeCarrot team members who are unable to be here today to observe your session and benefit from your comments.

Please read the statement below and sign where indicated.

> I understand that my usability session will be recorded.

> I grant CodeCarrot permission to use this recording for internal use only, for the purpose of improving the software being tested.

Name
>

Signature
>

Date
>

Once underway, we ask them to say out loud what they're thinking as they're using the software, which can feel unnatural but is important. While running the session, the only reasons to speak are to get them to talk out loud again, ask questions, and provide them with outcomes to achieve.

We should avoid leading questions such as "Was this task difficult for you?" but should ask follow-up questions. For example, if someone says "That's awesome!", we shouldn't silently pat ourselves on the back. We should say "Why is it awesome for you?" We are seeking understanding.

After the session, we look back at our notes to identify the top handful of problems and fix them before the next round of usability tests. If there's a problem revealed with the navigation, there's a temptation to totally re-do it. We try to resist that urge and come up with less drastic changes.