Archive for February, 2008

Usability Testing 101

So how do you do this elusive thing called a usability test? ClickZ and Digital Web Magazine have writeups on it,  and User Interface Engineering has an article on the Common Usability Testing Mistakes.  

There are different approaches, depending on what stage of development you’re in, but the basic premise stays the same:

  1. Develop a test plan. The “meat” of the plan is the tasks you’ll have the users attempt. 4 to 6 tasks is a reasonable number. Don’t try to include too many, or your test gets too long. Too few, and you don’t cover enough ground. Here’s a Usability Test Plan Template you can use.
  2. Find some representative users. They can be friends or relatives, but they need to be from your target user group. Some people have temp agencies hire people for testing, but I find it easier to just start asking around. (Make sure you have lots!) Most people are very willing to participate–all you have to do is ask. Dana Chiswell provides some useful tips on choosing participants.
  3. Perform the test. Sit down with the users, one at at time, and have them attempt the tasks in your test plan. Ask them to think out loud as they go, so you can understand what they’re thinking. You may be tempted to show them how it works–don’t! It’s critical that you let them struggle through it on their own so you can see where the software doesn’t make sense for them. According to Jakob Nielsen, you only need 5 users–I find you need anywhere from 3 up to 10–once you start hearing the same story from each user, you can quit.
  4. Observe, ask questions, and learn. Some people aren’t very talkative, so you have to ask a few probing questions to get their thoughts. It’s helpful to record the session using something like Morae or WebEx. Use a webcam to capture facial expressions, if you can. Little video clips from the test can be an invaluable (and extremely convincing) communication tool!
  5. Give the participant a gift. It’s important to thank your participants. Some people pay them. I think it’s nice to give them a gift. Make the gift proportional to the value of their time.
  6. Analyze the results. Look for common ground amongst participants. Where did they struggle? What frustrated them? What did they like about it?
  7. Present the results. Always, always, always start the presentation with “the good stuff”, but warn your audience that you’re going to give them the bad news next. They to know your intent is to help build a better product, not to cut them down. It’s important that they know there will be some bad news, though, and when it’s coming. Use video clips from the test to underscore your findings. The clips are often painful for developers to watch, but they get the point across.

I really like what John M says about Usability testing:

“Whilst planning for Usability Testing, its easy to constrain it to be more of a “Validation” type technique. Usability testing & the information gathered from the exercise, should serve to make informed design & development decisions right from the outset, thereby acting in more of a “preventative” role.   The idea in the case of Usability testing is to test early & test often. Usability testing lets the design and development teams identify problems before they are deeply entrenched. The earlier those problems are found and fixed, the less expensive the fixes are. As the project progresses, it becomes more and more difficult and expensive to make major design changes. The more you test and change based on what you learn, the more confident you can be that your application will meet your objectives and your users’ needs when it is released.”

 So, with a little usability testing, you can avoid situations like this:


When do you start usability? What process to follow?

What development process should we use to ensure the product (or web site or software) is usable? When should you start usability? I think I’ve said before that you should start usability right at the beginning of the process. It should be integrated right into the systems engineering process or any other development process you’re using. Bringing a usability person on at the end to “bless” your project is NOT going to work. I wouldn’t want my name on any project that approaches usability this way–yet this seems to be how most projects want to do usability. Bringing usability in late is, admittedly, better than never, but to really benefit from usability practices, you have to use it throughout the entire development process, right from the start of the project.

There are established processes for usability and many different tools you can use along the way. and NASA both have process models. James Williams Helms of Virginia Tech wrote a paper on the LUCID/Star Usability process model.

Here is the process in a nutshell:

  1. Research and Planning: this part involves researching your user community, gathering statistics, usability testing the existing site/product/software, performing surveys, interviewing users, developing personas, and determining the requirements and goals for the site/product/software. Make sure you understand your users’ goals, motivations, and work flows. Know what they come to your site for, what tasks they are trying to complete, what information they seek. This is critical.
  2. Conceptual Design & UI Structure: Use card sorts to help you with your UI structure. Sketch out pages, show them to users and see if they understand them. Modify as appropriate. Remember what Frank Lloyd Wright said:

    “You can use an eraser on the drafting table or a sledge hammer on the construction site.”

    Now is the time when changes are cheap, so take full advantage of it. Don’t try to lock down your design too soon. Iterate your design with feedback from real users. Present the feedback from your tests to management and explain the logic behind your design. Don’t be afraid to push back if management asks for something that doesn’t make sense, or if they want to change something on a whim. They expect us to deliver the best solution–we have to tell them what approach we should use, and why, to get the best possible product.

  3. Usability Test Detailed Design & Usability Test: For a web site, you might start to mock up some pages now, and develop the flow of the site, but don’t start programming too soon. Use simple HTML hyperlinks or even Powerpoint to show the flow. Let users hack away at your prototype (via usability tests) and revise the design appropriately. Be open to change, if it’s needed. This is the time to change, before you’ve started programming. Programming is expensive. Making changes later will cost much more. Make sure management understands that. Talk a lot about the user experience, use statistics, show user feedback. Our management wants our end users to be happy, and most of the people on the leadership team have said to me (in interviews) that their opinion doesn’t count–it’s the users’ opinions that count. Make sure you make it count by giving the users plenty of opportunity to express their opinion along the way.
  4. Develop & Usability Test: Now that you’ve iterated the UI design, you can start making it work. Hopefully your usability tests to this point have uncovered a lot of the gotchas. But still, do more usability testing as you go, especially when you run into questions about how something should work. It’s best to just test it. Always include usability test results in your management meetings. They show very clearly the reasons for every design decision. Let management make the decisions, but give them enough information to make the decisions that make sense for the user.
  5. QA test, etc. and Launch: Usability testing doesn’t ensure you’ve caught all the end cases, the bugs, etc. Make sure you test for bugs and handle errors with care. Treat the user with respect, and make sure all error messages are in user (not programmer) terminology, that they are kind and they blame the software, not the user, for the error (even when it is the user’s fault). Nobody likes a product, web site or software that makes them feel stupid.

That’s pretty much it in a nutshell. Notice usability isn’t just a usability blessing at the end–it’s involved from the very start, when you’re doing user research, developing personas, formulating your requirements. Don’t get stuck in “the way we’ve always done it.”

Projects that incorporate usability practices get meaningful results–I’ve seen metrics reported like improvements in sales of 100 to 700%. Notice there’s a wide range there. Projects that incorporate usability from the very beginning get much better results (closer to 700%). Projects that invite usability at the end aren’t going to see as much difference.

The same amount of effort, applied at the right time in the process, can have a huge impact.

Another tool for building click-through UIs

Thanks to my good friend Carla, I was drawn to a LinkedIn Answer today that helped me answer the predicament I’m finding myself in. I need a good tool for mocking up my UI screens, quickly. I could use something like PowerPoint, but I’d like something that helps me work faster, with pre-built widgets I can drag and drop onto the screen. I’d been looking at iRise, but I was frustrated that there weren’t any simple drawing tools in there. I want to be able to draw a box!

So I’m downloading the free trial of Axure today to try it out. It’s got the drawing tools I was looking for, at a fraction of the price of iRise. I’ve got a set of screens to mock up in lightning speed and this looks like it will do the trick!

Anybody out there got experience with it?

 Thanks, Carla! You’re awesome! 🙂

Wells Fargo violates bank account security rules

Okay, this is way off the topic of usability, but I am so angry about this I just have to blog about it. Wells Fargo is my bank and for the most part, I love them. Their online banking system is very usable (which I love, except I wish they’d give me a more direct way to get to my billpay) and overall I’m really happy with them. Their staff is really friendly when I walk into the bank and they really go the extra mile to get me the information I need, even if it’s the details on a deposit that happened over a year ago.

But their auto finance department needs to take bank account security 101. If I’m even a few days late on my car payment, they have a rude person call me and harrass me to do my payment over the phone. I don’t really doubt that it’s Wells Fargo calling me, but I have no way of actually confirming that, so I always tell them I don’t want to do my payment over the phone and I will take care of it online. They get really angry about that. Today the lady on the phone actually called me “weird”. Would she give her account information to anyone who happened to call and say they were from Wells Fargo?

Banks have been trying to educate consumers to NOT give your account information out to callers. You should never, never, never give out your bank account number to someone who calls you, no matter who they say they are. Only if you initiated the call is it okay to do that. It just blows me away that Wells Fargo, one of the largest banks out there, would knowingly ask consumers to do this, perpetuating the problems with account fraud we have. And the fact that this woman thought I was “weird” because I wouldn’t give her my personal information and bank account number means that the message is not getting across to consumers, either. People are willingly giving them their account information over the phone when they have no way of confirming that it actually IS Wells Fargo calling.

 I tried to walk into a Wells Fargo bank to complain about it, but they told me they don’t have anything to do with auto financing and I would have to call their auto finance department. There is a problem there, but I don’t blame the innocent bank staff. I blame the idiot managers in the account finance department who are:

  1. Hiring rude people to harrass people about their car payments. If my account was months overdue, yeah, it’s okay to call, but not after a couple days. If you’re going to call me when my payment is a couple days late, you should be really, really nice about it.
  2. Asking people to give out personal information over the phone with no verification that they are who they say they are. By doing this, they are teaching consumers habits that only increases banking risk and makes it easier for fraudsters to hijack people’s bank accounts and steal identities.

It’s maddening and it’s got to stop!

A Portrait of American Consumerism

Back to the subject of Americans and our overabundance of Stuff, I ran across an article about Chris Jordan’s photography on I think I’d seen some of his photography before, but Chris takes statistics about our consumer-istic lifestyle and turns them into giant, visual works of art that convey the scale of our overabundant consumeristic nature. It’s another well-done, impactful view of the Story of Stuff.

As Shea on Ecogeek says:

 Chris thinks that the green movement is happening, but is stalling out. He used the analogy of the finish of a bike race – everyone is waiting for the other person to make the first move toward the finish line. He also believes that the green movement is hampered by the lack of cool. Michael Jordan changed the face of basketball fashion overnight when he showed up to the game wearing baggy shorts. Although people like Al Gore and Paul Hawkin are necessary to the environmental movement, they lack the cool factor of a green Michael Jordan.

I’ve experienced this first-hand when I go to the store and use my reusable shopping bags. Everyone looks at me like I’m a little crazy. A few comment on how cool my bags are, but for them it’s all in the design of the bag, not the concept of reducing our consumerism and saving the environment. When I use my flourescent “Live Green” bags with the cool design, everyone thinks I’m cool. When I pull out the bunched up, plain-jane eco-sack from my purse, they look at me like I’ve grown a third head or something (but it’s so much more convenient for a forgetful, busy mom!)

I guess I’ll try to use the “cool” bags more often, to get the message across to other shoppers to consider reducing their use. I wish we weren’t so shallow, though.