Guide to product testing

This page displays take away information I have saved from articles read focused on Testing, Usability, Accessibility. I use this as a reference point for research in the Testing, Usability, Accessibility category.

Take Aways:

  • I want to start by saying that we’re testing [the product], and not you. You can’t do or say anything wrong, and nothing is going to hurt our feelings.

  • When we’re in the very beginning of a new project and validating a product idea through testing, we’re miles away from proper usability testing (which we do later in the project). This is just one of the reasons we don’t refer to these sessions as usability testing or user testing—we instead call it product testing.

  • Start testing the very first week of the project
    - Sprints are great when kicking off new initiatives, testing new product ideas and features, etc. We end the workshop by creating a medium-fidelity prototype that we then test with users. We want to validate the direction we’re heading right at the start.

  • Do more than ‘just’ usability testing
    - Concept: Validate the product idea with the desired target audience
    - Features: Prioritize features (especially when building an MVP)
    - Information architecture: Test out different approaches to information architecture
    - User flow: How proposed flows affect the user experience and behavior
    - Visual design: A series of tests related to visual design, voice and tone, readability, branding, naming, micro-copy, etc.

  • 5 users per target group
    - Identify people who match your demographics (or even better: match your personas and then screen for behavioral traits, attitudes, and goals that match those of your users.
    - We tend to always have 6 humans recruited (plus floaters available for morning and afternoon sessions). The tendency to do more than 5 comes from experience—sometimes users who show up aren’t a good match, sometimes there are equipment issues, or there’s a fire drill (it’s happened more than once!).

  • Do it more than once
    - The most important thing is to guide those tests towards a desired outcome.

  • Take care of the details
    - We’ve seen completely different testing results when one or more details weren’t executed with care.
    - Use real content in your prototype.
    - Know thy flow. It’s always important to know what’s the primary focus on each screen so that the study goals can be properly validated.
    - Know your hardware. Set yourself up for success and don’t let “typical” hardware behaviors get in the way. I’m talking about screensavers, brightness levels, notifications, other apps, etc.
    - Know your software. Prototyping tools and behind the scenes interactions can sometimes conflict with intended actions for our users.