How to get started with User Testing

This page displays take away information I have saved from articles read focused on Testing, Usability, Accessibility. I use this as a reference point for research in the Testing, Usability, Accessibility category.

Take Aways:

  • The goal of user testing is to identify any usability problems, collect qualitative and quantitative data, and determine the participant’s satisfaction with the product. The earlier issues are identified and fixed, the less expensive the fixes will be.

  • Recruiting users
    - Instead of waiting until you’re ready to start conducting user tests, start collecting interested users now. This way you have a bucket of people to call when you’re ready.
    - Set up was a form to capture leads for UX research. If they opt in to our list, we have permission to email them about upcoming user testing sessions. We can also reach out to them for surveys, prototype feedback, interviews, and field visits.
    Phone number. This comes in handy when you need to confirm with the person that they still intend to show up for user testing.
    Technologies used. This helps us filter our list down to the type of persona we’re looking for certain tests.
    Local to San Francisco. We use this to help us find good candidates who can attend user tests in person instead of remote.
    - Ethnio, a service made specifically to handle user research recruiting and scheduling.

  • Plan your tests
    - the type of tests we have fall into 3 categories:
    What are they doing today? Interview style research to help us understand their current workflow or problems.
    Would they use this? Hybrid of interview and design artifacts to get their reaction to something we’re working on.
    Can they use this? Usability test on a clickable prototype or coded solution. We make heavy use of InVision prototypes here to make sure we’re confident about our solutions before we develop anything.
    - The test lead works with the broader team to align on what we want to test and prioritizes the key things they want to focus on
    - Once we understand the key things we want to test, we’ll work together to come up with a high-level script and tasks of how might be best to test this.
    - When we’re testing clickable prototypes, we use Sketch and InVision, and we use Abstract to manage our design files.
    - Invision naming example (Deployment [User Test 09-09-2017 ])

  • Scheduling
    - Using a research coordinator - Recruiting and scheduling takes up a lot of time. If you have this ability definitely use it to focus on the other areas in the process.
    - Our goal for scheduling is to have 5 user tests in a day with 1-2 backups planned. We have 5 slots, each lasting one hour: 10am, 11:30am, 1pm, 2:30pm, and 4pm, with a 30-minute buffer between each to prepare for the next.
    - These tools let us create a page with time slots, and then our users choose a time that works for them. They also help us manage messaging and reminders for the appointments.

  • Running the tests
    - On the day we actually do the user test, we greet the user, have them sign a waiver, and bring them into the testing lab.
    - We record the sessions and hook it up to a big screen so we can watch. We have no more than 2 people in a room: the test lead and either another designer, PM, or developer who takes notes.
    - The script we put together serves as a guideline but adapt as the path goes in other directions.
    - Finish by asking them if they have any questions, and if they could wave a magic wand and have any feature today, what would it be
    - After the test, express your thanks and send a gift card (we typically offer Amazon gift card incentives).

  • Report your findings
    - Take your raw notes, create a report, try to keep it brief, and highlight the main issues you discovered.
    - We share our reports in our wiki. The report includes:
    Who was tested (photos, names, titles, companies)
    What was tested (links to prototypes/designs)
    List of issues including topic, description of issue, recommendation on how to fix, priority
    Other observations and feature requests
    Links to videos (stored in Google Drive)

  • Action items
    - The most important thing: take action on what you discover during user testing.
    - If you can show clips of users struggling, it’s an easier sell to developers, stakeholders, or whoever needs convincing.