The Art of Guerilla Usability Testing

The post The Art of Guerilla Usability Testing appeared first on UX Booth.

Guerrilla usability testing is a powerful technique. Designer Martin Belam describes it as “the art of pouncing on lone people in cafes and public spaces, [then] quickly filming them whilst they use a website for a couple of minutes.” Let’s skip the pouncing part and instead focus on its subtleties, including how to obtain and share feedback with our team.

I recently worked on a quickstart project in which my team was asked to build a responsive website in a short amount of time. We were given very little time to code (let alone conduct research) for the endeavor, yet by employing guerilla usability testing along the way we collected feedback on the brand position. Eventually, we aligned our designs to both customer expectations and business goals.

Once a week throughout the project, we tested different kinds of prototypes to bring the business’s ideas to life. For example, while mid-development, we sketched a mobile version of the site on index cards and did a quick assessment. This revealed navigational problems (which guided us to rethink a key point in the customer journey) and even ended up shaping a bit of the brand’s media material. What’s more, guerilla usability testing opened our stakeholders’ eyes so that they challenged their own, innate assumptions about “the user.”

We iterated through our design ideas using lo-fi techniques like paper prototyping. Sketch by Chris Cheshire.

The bottom line? Guerilla usability testing presented itself as an easy-to-perform technique for refining the user experience. It helped us validate (and invalidate) critical assumptions at cheap cost and with rapid speed.

Breaking it down

It’s hard to see the magic that guerrilla usability testing affords and not want in on the action, right? Here are some basic questions to consider before getting started:

1. What shall we test?
2. Where will we test?
3. With whom will we test? and, of course,
4. How will we test?

What shall we test?

One of the best parts about this kind of testing is that it can be done with almost anything, from concepts drawn on the back of napkins to fully functioning prototypes. Steve Krug recommends testing things earlier than we think we should and I agree – get out of the building as soon as possible.

Test what the product could be so as to shape what the product should be. Even loosely defined UI sketches can be a great way to evaluate a future product. In fact, recent research shows that lower-fidelity prototypes can be more valuable concerning both high and low-level user interactions.

Where do we test?

Where we conduct tests affects how we perform and document our work. For instance, if we’re testing a new mobile app for a retail chain, we might go to the store itself and walk the aisles; if we’re working on “general” office software, we might test it with coworkers in a different part of the office; etc. The point is: let context drive the work.

With whom do we test?

When designing for the mass market, it’s easy enough to ask friendly looking strangers if they have a couple minutes to spare. Public spaces and shopping centers present some of the best places to do this on account of the sheer amount of foot traffic they receive (as well the relaxed nature of the environment). With more specific user sets, however, it’s useful to target subjects based on their context (see above): a mixture of location and behavior.

Coffeeshops are great because you’ll often find test subjects from varying cultural backgrounds and different age ranges.

How do we test?

Testing is fairly straightforward: have participants talk aloud as they perform tasks. Use the think-aloud protocol to test overall product comprehension rather than basic task completion. The key is to watch customers fiddle with a product and silently evaluate its usability. As Sarah Harrison explains, “Observing users is like flossing–people know they’re supposed to do it every day, but they don’t. So just do it. It’s not a big deal.”
Always start with open-ended, non-leading questions like:

1. What do you make of this?
2. What would you do here?
3. How would you do [that]?

By answering these kinds of questions, participants tell a loose story in which they explain how they perceive a product. Along the way, we can generate ideas for how to improve things in the next iteration.

Employing the technique

Guerrilla usability testing is very much about adapting to the situation. That said, here are some helpful hints that I find consistently work in different international contexts:

1. Beware the implicit bias. While coffeeshops are a great place to find test participants, focusing on people who frequent them introduces bias to our work. Simply acknowledging this implicit bias can help designers neutralise subjective experiences and account for individual differences. Remember to target different genders and be fair in who you approach.
2. Explain what’s going on. Designers should be honest about who we are, why we’re testing, and what sort of feedback we’re looking to receive. Oftentimes, it’s best to do this with a release form, so that people are fully aware of the implications of their participation – like if it’s going to just be used internally versus shared globally at conferences. These sort of release forms, while tedious to carry around, help establish trust.
3. Be ethical. Of course, being honest doesn’t mean we need to be fully transparent. Sometimes it’s useful to skip certain information, like if we worked on the product they’re testing. Alternatively, we might tell white lies about the purpose of a study. Just make sure to always tell the truth at the end of each session: trust is essential to successful collaboration.
4. Make it casual. Lighten up tests by offering cups of coffee and/or meals in exchange for people’s time. Standing in line or ordering with a test subject is a great opportunity to ask questions about their lifestyle and get a better feel for how a test might go.
5. Be participatory. Break down barriers by getting people involved: ask them to draw – on a napkin or piece of notebook paper, for example – what they might expect to see on the third or fourth screen of a UI flow. This doesn’t have to be a full-blown user interface necessarily, just a rough concept of what’s in their head. You never know what you’ll learn by fostering imagination.
6. Don’t lead participants. When you sense confusion, ask people what’s going through their head. Open them up by prodding, saying “I don’t know. What do you think?”. People in testing situations often can feel as though they are being tested (as opposed to the product itself), and therefore can start to apologise or shut down.
7. Keep your eyes peeled. It’s important to encapsulate passing thoughts for later analysis. Ethnographic observation is one good way to capture what you were thinking of during tests. Don’t get too hung up about formalised notes though, most of the time your scribbles will work just fine. It’s about triggering memories, not showing it off at an academic conference.
8. Capture the feedback. A key part of any testing process is capturing what we’ve learned. While the way in which we do this is definitely a personal choice, there are a few preferred tools available: apps like Silverback or UX Recorder collect screen activity along with a test subject’s facial reaction. Other researchers build their own mobile rigs. The important part to remember here is to use tools that fit your future sharing needs.
9. Be a timecop. Remember, this isn’t a usability lab with paid users. Be mindful of how much time you spend with test subjects and always remind them that they can leave at any point during the test. The last thing you’d want is a grumpy user skewing your feedback.

Sharing the feedback

Conducting the tests is only half the battle, of course. To deliver compelling and relevant results from guerilla usability tests, designers need to strategically decide how we’ll share our findings with our colleagues.

When analysing and preparing captured feedback, always consider your audience. The best feedback is the kind that understands stakeholders and kickstarts important conversations between them. For example, developers who need to evaluate bugs will have different needs than executives who want to prioritise new features.
Next, when delivering feedback, align it with your audience’s expectations. Try editing clips in iMovie or making slides in PowerPoint. Your co-workers are probably as busy as you, so an edited down “trailer” that highlights relevant results or a bullet-point summary along with powerful quotes is always a good method to keep people listening.

Go guerilla

At the end of the day, guerilla usability testing comes in many forms. There’s no perfection to the art. It is unashamedly and unapologetically impromptu. Consider making up your own approach as you go: learn by doing.
Note: Thanks to Andrew for providing lots of feedback on early drafts of this article.

Related reading

• “Guerilla testing” by UK’s Government Digital Services
• “Guerrilla Usability Testing” by Andy Budd
• “10 tips for ambush guerrilla user testing” by Martin Belam
• “Recording Mobile Usability Tests” by Jenn Downs
• “Failing Fast: Getting Projects Out of the Lab” by Tony Hillerson, Alan Lewis, Scott Green, Ryan Stewart, Randy Rieland
• “Rocket Surgery Made Easy” by Steve Krug
• “Talking Aloud is Not the Same as Thinking Aloud” by Mike Hughes

Advice from a former Apple director who coined the term ‘user experience’

June 6, 2013  |  by Taylor Soper

Having worked on Apple’s User Interface Technologies and introducing the term “user experience” to company execs in the early 90s, Mitch Stein knows a thing or two about how humans interact with computers.

“The term ‘user experience’ is more than just aesthetics to me,” Stein said. “We have relationships with our technology. User experience is not just eye candy — it promotes a positive relationship between humans and technology.”

Stein spoke earlier this week at the Hacker News Meetup in Seattle, sharing insight from his time Apple, where he managed key portions of five OS releases and projects like Finder, Installer and Chooser. Stein also spearhead numerous UI and architectural innovations at places like IBM, Adobe and Oracle.

“I like to work where the atoms meet the bits, where real world and technology meet,” he said. Here’s a few tips from Stein, whose InfoPortal Prototype was once featured at the “Workspheres” exhibition at the Museum of Modern Art in New York City.


Know the creative cycle: Assimilate, Innovate, Iterate

“This is key. First, assimilate: You don’t ask the user what they want — you go out and live with them and literally become the user. You do it with a wide-angle lens. You do it not just to tackle the problem you think you’re solving —you need to understand the culture they live in, what motivates them, that sort of stuff. I know that sounds touchy feel-y, but it really works. 

Once you have that, you go back and take everything else into consideration: business, competition, materials, make vs. buy decisions, potential partnership opportunities. At this point, you can start innovating. That’s where the magic happens. 

Once you do all that, you damn well better go back and show the folks and see if they go “wow, that’s cool,” or not. That’s the iteration phase. Then, like the shampoo bottle says, it’s lather, rinse, repeat. Everything is a cycle.”

Failure isn’t an option — it’s a requirement. The trick is to fail faster and learn from it.

“Failing is required. The two most dangerous people in the world are those who have nothing to lose and those who have one big success and think they are hot shit and they can do no wrong. I respect people with multiple successes. Those people are open-minded and have learned from their failures. 

In a company where failure is not tolerated, the innovators and people willing to take risks get frustrated or fired. People who are left are just afraid of their own shadow and they cannot do anything but incremental progress because they are too scared. Actually, at one point, one of the companies I was at had monthly parties called “Fail Party.” We would make fun of and celebrate the kind of people that had the guts to do stuff. It was to show everyone that as long as you learn from your mistakes and get benefit from it — even if it’s to learn that you shouldn’t do anything like that in the future — that’s totally acceptable.”

Key UX areas of focus and important traits of excellent UX practitioners  

Stein’s top UX areas of focus:

• Personalization
• Cognitive resonance (sense of place)
• Context
• Scalability
• Assistance
• Integrated Information Access

Stein’s most important traits of excellent UX practitioners:

• Empathy
• Diverse experience and interests (empowers associative thinking)
• Ability to deal with uncertainty and managing risk (not avoiding it)
• Profound laziness
• Balance of confidence and open-mindedness 

How I love Marks & Spencer!

Check out the beautiful tea packaging.

Photography inspiration!

10 steps to an engaging interactive user experience

By Irene Pereyra on December 05, 2011

Irene Pereyra, director of UX and strategy at digital agency Fantasy Interactive, rounds up 10 top tips to help you deliver an amazing interactive user experience

These days, building websites or applications that attract and retain customers has become somewhat of a science. For people who aren’t well versed in the digital space, I often compare the work that I do as a UX designer to an architect. Like the architect who builds your home, my UX team builds a comprehensive blueprint, which outlines every single detail of the site’s features and functionality.

But it’s not a one-shot deal. Getting to an intuitive and engaging user interaction requires many steps. Here’s my top 10 tips to help you deliver an amazing interactive experience for your users

1. Design for the user, really

Back when online interaction was still in its infancy, and not much thought had been given to whom we were designing for, users were all too willing to spend their time learning the interaction required to complete tasks on websites. If users were confused, people often assumed they just weren’t tech savvy or well-informed on how to navigate the internet. As more and more websites, mobile devices and tablets started popping up; users weren’t as willing or patient to “learn” on their own. Nowadays, you’ll see more users becoming frustrated and even angry when they feel a product, application or website is substandard – and rightfully so.

It’s tempting to design with your own preferences and tastes in mind. But that won’t help users complete tasks on the site if they have a whole different set of preferences and needs. Think about what users want to do and help them complete those tasks in the easiest and most intuitive way possible. Are they browsing? Searching? Gaming? Watching video? Trying to complete a task? Looking for specific content? It’s the UX team’s job to look at the entire experience holistically and make sure that users’ needs are always met.

It’s this “design for the user” strategy that shaped every decision we made on Civil War 150. The biggest challenge with this project was the huge amount of Civil War related facts and statistics. To make it fun and relatable, we used colorful infographics to guide users through the top 150 Civil War related topics including Who They Were, Weapons of War, How They Died, 5 Deadliest Battles, Paying for the War and West Point Warriors. By giving users what they want, learning about the Civil War becomes richly engaging, informative and fun experience for hardcore Civil War fanatics and 7th grade history students alike.

2. Do your homework

Listen and absorb. The more conversations you have with clients, the better informed you’ll be. Dive deep into every piece of documentation, research their field, examine all content with a fine-tooth comb, understand the client’s goals, document thoroughly all of the client’s wishes (no matter how small) and talk to as many people across as many departments as possible. Only then will you have the most complete and accurate picture and be prepared to move to the next phase of the project.

Another crucial part of listening comes from doing a thorough analysis of what competitors are doing in the same space. Are there any innovators that you can learn from? Have they made any mistakes you want to avoid? Is there one universal component that ties all of them together? Were there any missed opportunities? Use the answers to these questions as inspiration in your own project.

The types of sites you may look at during this analysis phase can vary dramatically. You could even look at sites that sell cat food as points of reference when designing an application for audio equipment. It’s still relative and can be helpful because the user behaviour could very well be the same for cat food sites and audio equipment sites. Either way, you can learn valuable lessons from UX best practices across completely different industries and form factors.

3. Be an advocate for the user

We often think of the user as our client, though it’s not entirely true. In any project, there are sets of business objectives that need to be met and it’s the UX designer’s responsibility to meet those objectives, while at the same time informing the client about the user’s needs. That’s why the greatest digital projects are often those where there is a perfect equilibrium between the client’s objectives and the user’s needs. Sit on that fence, and balance well.

4. Forget about Nancy, think user types

Personas are vital when it comes to structuring the content. Look at all the content holistically and think about what people are trying to accomplish. Doing this helps prioritise the content and allows the site to be structured around the user’s goals. But traditional personas – “Nancy, who is 28-35 years old, drives an economy car, has a four-year-old PC she primarily uses for email, earns between $30K-$50K a year and wants to comparison shop for a cheap airfare to visit her mother in Florida” – won’t offer much insight into the user’s actual behaviour.

Instead, group basic User Types into categories according to what they want to do on the site such as “browsing,” “comparison shopping,” “killing time,” “looking for specific content.” These groupings will provide you with much more useful insights about why users come to sites or applications, the context of use (where and how), what content they’re seeking and how much time they have. In turn, you’ll be better equipped to design the website or application around their behaviour patterns, thus making their fictional names, ages, professions and income levels irrelevant.

5. Less really is more

You may think this is obvious and doesn’t need further explanation. But most sites and applications still manage to get it wrong. The key is to cut down tasks required by users to the bare minimum. I can’t stress this enough. Get rid of all that extra clutter that doesn’t add value, or worse, distracts and confuses the user. Know exactly how you want users to travel through your site or application and then guide the user as if you were holding their hand through the entire process. Again, users want things to be as simple, worry-free and fast as possible. If they can see what’s coming next before even clicking on something, they’ll be happy users.

To give you an example of using the “less is more” strategy, we worked together with Google on a project called 20 Things I Learned About the Web and Browsers. The challenge was to take the tactile experience of reading a book and reinvent it for online users. Using interactive features such as instant search, animated canvas page flips, enhanced canvas illustrations, offline mode, bookmarking and lights out mode, we made it simple, fun and informative for users to learn about the web and browsers. By focusing on what and how users wanted to travel through the website, learning about the web and browsers became enjoyable and informative for users.

6. Pretend you’re working for Fisher Price

Our CEO David Martin has this saying – and it’s really resonated for me. “All interaction should feel like Fisher Price.” In other words, when you make things bulky and oversized (like most children’s toys) and design digital experiences for “fat fingers,” it will automatically be easier to use. So how do we translate this to designing engaging and interactive user experiences?

Rather than using the standard input fields, radio buttons or checkboxes, try using big buttons, jumbo sliders and giant input fields. You’ll see user engagement increase and bounce rates drop. Isn’t that what all UX designers strive for?

Labeling is also extremely important. Whenever you ask users to provide information, try to use cheeky, simple, and to-the-point terminology so that it will feel less like a hassle. The result will be that users will feel more emotionally compelled to complete the process. And that could mean a boost in signups, web traffic, online sales and ROI.

When a 9.0 earthquake struck Japan earlier this year, earthquake relief efforts and support poured in from around the world. Working with Google, we created a platform for the global community to share their Messages for Japan. Taking a cue from Fisher Price, the site featured big buttons to group messages into categories of “Love” and “Hope”, as well as giant input fields asking users to “Write Your Message” and “Make a Donation.”

7. Take cues from tablets

Because you’re already limited by the amount of real estate on tablets, the need to simplify interactions is even greater. Ask yourself if your design would work on a tablet. If the answer is yes, you already have the two basic building blocks in place for a strong user experience: clear hierarchy and intuitive way finding.

These are the same questions we asked when we teamed up with CBSNews.com to create an elegant and visually rich online news experience for viewers of America’s #1 news program, 60 Minutes.

8. Design your UX

Adding placeholders for copy (lorem ipsum) next to some gray boxes underneath a row of navigation links does not constitute design. Visual hierarchy, content grouping, spacing, positioning and size are all things that should be solved in wireframes before a visual designer even sets eyes on it. If in your wireframes you’re working within the actual site or application width, and 12px text in your template is actually 12px in design, you’re on track.

9. Collaborate with all departments

User experience design alone is not enough to make great work and it will surely not provide all the answers. Listen, collaborate and become the liaison between the client, the user and the rest of your internal team. Only then can you create the best possible experience for the user while still meeting all business objectives.

Remember, you’re not alone. UX designers, visual designers and interactive developers all have a hand in making a project a success – it truly is a collaborative and multi-disciplinary effort. When it’s a shared passion and everybody pitches in with their level of expertise and voices their opinions, magic happens.

10. Don’t grade your own homework

The chance of you hitting the bulls eye with your first shot is very slim so be prepared to design iteratively as you gather more information about the performance of your site or application. One of the things that I am adamant about is that you should only do quantitative analysis on your own work. Tracking the performance and understanding where people are dropping off is extremely important and should be done in-house. But a third party should always conduct the qualitative user data analysis so you can truly have an objective testing environment. Doing this in-house is much like grading your own homework and won’t give you the depth of truth and insight that you’re looking for. If you want real objective answers, let someone else do the user testing and take those learnings into your next iteration.

Black & White vs Color

I'm on the fence with this particular image. Usually I can look at a image and right away I know if it's going to be black & white or color. I think this image fits both but maybe I'm leaning more on the black & white. click on the thumbnails for a larger view. Thoughts?

It was taken in the beautiful Point Reyes National seashore! Be sure to pay this lovely landscape a visit. Truly an Awe moment!

To see in color is a delight for the eye but to see in black and white is a delight for the soul.
- Andri Cauldwell

Sounds of 2011

Hotly-tipped singer songwriter Liam Bailey will make his major label debut with the release of new single You Better Leave Me on Polydor Records on March 13th. The track is something I've been waiting for! It's a refreshing modern spin on neo soul. Until then catch the video here!



Also get a free download of the track I belong here