July 2011 – Nina Mehta

User testing is a seemingly giant mystery. Having formerly worked as a journalist, asking mountains of questions to someone I’ve never met before is like second nature for me. I moved to ‘Silicon Valley’, land of the startups, a couple of months ago and have since been getting asked a lot about user testing from engineering friends. It’s possible to start putting your product in front of people without having a UX homie in the house.Below are 11 practices I follow that help inform better product design. I learned and did these things at myGengo, RockMelt and in graduate school. The list below is not holistic and may not work for your team or product. Use your best judgement, it’s qualitative research. And designers, I invite you to critique these points.

Test the product, not the person. “Thanks so much for coming in, we really really appreciate it and value your feedback. Our product is still in the early stages and we want to find out what is confusing and what doesn’t work. You can quit any time and the more things you can find wrong with it, the better. It’s also helpful if you talk outloud while you’re using [product]. It helps us know what you’re thinking and when you’re stuck.”
Be an excellent listener. Be humble. If you disgaree with an opinion, keep it to yourself and make a note of it on paper.
Think of the guest coming in as a person, not a user. I avoid the word ‘user’ as much as I can but sometimes it creeps in. Get to know them, discover who they are as people. Just hang out for a few minutes until you’re both comfortable.
Take notes on paper, not on your laptop or phone; it makes you the ‘magical computer person’ more accessible and human. It’s ok if you need a computer open to chat with people in another room or do some recording. Digitizing notes afterwards is a good idea.
Avoid leading questions that impose a value. Avoid: “Do you think this is a good color?” Instead ask “What do you think about this color?”
Know what you want out of the session. What features are you testing? There’s a fine balance between keeping the session open ended and getting what you want out of it. But in the end, you are leading the session. If you are going down a path where you are learning something interesting, follow it and probe with questions. If you find yourself in a time suck, take back the lead. Make note of body language and facial expressions.
Ask about expectations: “It seems like that button was hard for you to find, where did you expect to see it?”
Don’t be scared of the people coming in. They’re just people. But, like sharks, they’ll sense your fear.
Aim for a half an hour session padded with questions at the front and the end. Eventually,both of you will get tired. Pad a little time in between sessions for you to recenter.
Debrief after the session with your guest and the team. Thank them graciously for their time. Ask if you can send follow up questions later. I like to organize my findings as follows (and props to Matt Beebe for this structure)
- a. engineering bugs (broken button)
- b. design bugs (misaligned pixels)
- c. backburner (possible problems, feature requests, things that need deeper thinking or data analysis)
Have fun

Tips from uxperts to help you get started

Observing the User Experience: A practitioner’s Guide to User Research
User Interface Engineer: How to do a UX Test
Why you only need to test with 5 users
My post about user-centered versus user-led design. Just because users demand sparkly kittens on your submit buttons doesn’t mean it’s the right decision.

Quite simply no. Dave Winer recently wrote “web browsers are done. Feature complete.” I refute this for a reason I perceive to be quite obvious: browsers must keep changing because people keep changing. Their needs, their wants, their economies, their mood and their lifestyles all influence what needs to be done next for the web.

URLS
How we get to wherever we’re going on the web has already changed. When the web formally and broadly platform for how we distribute information and communicate it will become more obvious why browsers have a long way to go. With Icann’s announcement to increase domain suffixes available, we could potentially travel to http://fanta.coke or http://maps.google instead of a traditional .com, .org, .gov suffix. Chrome released the omni bar which merged manual URL entry and the search for content. Browsers (on many devices) serve as portals to information and the web as as a platform for Saas then there is much work to be done. URLs are fading to the background as a primary way to navigate to information we seek.

Saas
At myGengo, I worked on a web-based UI for translators (not shown). Before they were either using Microsoft Word or a plain text editor. Professional translators who do mountains of translation use professional tools like MemoQ (right). The software allows translators to work relatively efficiently but a lot of UX needs are unmet. I worked on this problem. My goal was to improve what myGengo had already built and weave in the powerful features these pieces of software have. I needed to do it in a way that makes sense and moves the language, translation and communication industry forward. All of course, to be done within the browser.

Universal Logins
I worked on the alpha launch of RockMelt. What their service does better than any other browser is the universal login. It’s front facing, it’s required, it gives users more context and it’s quite clear what happens as a result of the login. If or when they (or Chrome or Firefox) leverage universal logins beyond personalized data synced, I think we’ll see more fluid way to explore the web. When we can eliminate or reduce login barriers while still offering personal security and stability and clear communication to users what has been done, we will move the web forwards.

The CR-48 Chrome Notebook (rigt) seems to be working towards this goal I reviewed the Chromebook in April. To promote an idea that the web is where we can do all of our computing is an idea they are working to materialize. It has a ways to go but the philosophy is not wrong.

‘RockMelt is just a bunch of plugins’
I get a lot of questions about RockMelt. They recently raised $30 million in funding and people ask me why. They tell me it’s either too noisy or it’s the same as all the apps or plugins they’ve added to Chrome or Firefox. How many people know what a browser is, how many know what a plugin or app is? I don’t have the numbers but my prediction is quite low.

When people ask about Rockmelt, I say:

RockMelt users LOVE the RockMelt. First and foremost. And it’s true.
Their team and projects are well managed and organize.
The team had a diverse (technical and cultural) background whom are wonderful, smart people.

Anyway, let’s say it is a browser with a bunch of plugins patched on top. What’s the harm in doing a little legwork for your users? A browser with features for people who love consuming and producing content on the web is not a bad idea. It would be interesting to build a browser business where you just package up features and distribute them to niche audiences. Do you think cheese heads in Greenbay would love an ESPN browser? Or the brokers on Wallstreet, with a Bloomberg browser? Is their an audience for the Sudoku, Crossword, Plants vs Zombines browser? I’ve been wanting to see RockMelt is opening up a door here.

Mobile
The obvious bridge for mobile and desktop browsers has been syncing content. I’m excited for the next stages where we have smart and helpful geo-location services, better ways to communicate fluidly across platforms (and Facebook has done a good job with this so far) and brilliant ways to find exactly who or what I’m looking for (even if it’s something to suck time while I wait) depending on what I’m doing and how I’ve been browsing.

I can understand why Winer wrote the post he wrote. But I would like to learn from him how he thinks technology, economy and culture will more forward without the (desktop, mobile, wrist watch, taxi cab, restaurant menu, etc) browser being a primary portal for business and play. Look to for Globe Trekker (who makes a good discussion about HTML standards) and Adaptive Path for alternate perspectives. I welcome yours in the comments.

What Winer’s post does do well is motivate an important conversation in a public space. If the internet really was some kind of information super highway, Winer would be sitting in the back seat asking “are we there yet?”