Earthling

Note: The below is an archived entry from Earthling, formerly EarthLink's official blog. The blog itself has been decommissioned and is no longer updated, and comments are trackbacks are no longer accepted.

Here's another update from the team developing EarthLink's new start page. Part 1 set the stage, explained what the project is about, and asked for your thoughts on start page priorities. In Part 2, the team reacted to your feedback and gave us an update. Now, in part 3, Tom talks about usability testing, and how it helps to shape the product:

Last week, we had our second round of usability testing on the new personal start page. Our first round of testing was in the Pasadena office, and this time, we held the testing in Atlanta. In fact, we had the honor of taking the Atlanta office's shiny new usability lab out on its maiden voyage. We had some technical difficulties for the first session -- bad feedback in the audio tech sense, rather than the usability sense -- but after that, she drove like a dream.
Usability testing can mean several different things, depending on what you're testing and how you're testing it. In this case, the usability testing went like this:

One by one, we brought Web users (generally known here as
"participants") into a windowless room with a computer, a desk, two chairs, two cameras and a microphone.

The lead usability engineer for the new start page project sat with each participant in the room and brought up a series of browser windows on the computer. Each participant saw our (mostly) working version of the start page and several mockups of how the start page might look when the final design is finished.

For about an hour, each participant took the working version out for a spin, poked at the mockups, and talked about what he or she did and didn't like.

Meanwhile, in the next room, the rest of us watched it all on three big flat panel monitors. Two of the monitors show the user, from different angles, and the third shows what the user is doing on the computer screen.

We had eight different sessions, over two days.

The juiciest bits of a session were usually when the participant took a first crack at the working start page. More often than not in this type of testing, at least one thing that seems clear to the project team isn't intuitive at all to fresh eyes.

During the first day of testing, user after user was tripping over our "edit mode" for one particular feature. I don't want to spill the beans on what we're up to, but I'll say that this particular feature allows you to integrate your favorite Web sites into the start page. The funny thing here is that most participants had the right expectations of what they could and couldn't do, and had the drive to personalize the feature, but were completely bamboozled by the editing interface.
Enter the iterative development process to save the day. We went straight from the last session of the day to our daily team meeting, to discuss what we had seen. We came up with a few quick fixes to make the edit mode more intuitive, and the developers actually worked the changes into the working site that afternoon. When the next session started the following morning, we had something new to test.

This version went over better, and spurred more feedback that steered us towards a completely new approach to solving the problem. If you've ever participated in this type of testing and wondered if the feedback really meant anything, take it to heart that if often makes a huge difference.

My favorite part of each session usually came at the very end. Barry would excuse himself and walk over to the control room to see if we have any follow-up questions, leaving the participant alone to do whatever he or she wants. It's very telling what the participant does when no one is prodding them. When they voluntarily go back to the working site to keep playing with a particular feature, we know we're on to something.

Earthling Home

Comments