Can You Crowdsource Your User Experience Research?

Mostafa Fawzy, Hemera, Thinkstock

In one sense, almost any type of user research is crowdsourced—you’re talking to people and using that information to improve your system. But in a true sense, crowdsourcing is more than just collecting information, it’s collaborating on it. We want to have real conversations, not one-time emailed suggestions without followups. So here’s a few tidbits on crowdsourcing User Experience (UX) for your site, mobile app, API or whatever else you’ve got cooking:

Crowdsource your bug testing

One way to crowdsource both ideas for new features and Quality Assurance on what you already have is to turn your “to-do” list inside out. Rather than keeping it private and internal, you make it transparent and invite others to help you catalog issues and requests. This public crowdsourcing of design ideas worked spectacularly well for Data.gov, who created a list of issues to be fixed on their GitHub site, and then invited their community—friends and critics—to contribute. Here’s a handy Usability Case Study they wrote about the experience.

drical, iStock, Thinkstock

The API Usability Testing Program that I am proud to be a part of with 18F uses a similar idea to collect issues discovered during our product evaluations. When we evaluated the National Renewable Energy Laboratory’s homepage, they created a page to track all of the usability problems that were discovered. And we created a page to collect these expanding lists of public-facing issues, and to help both us and other agencies keep up to speed on this way of community.

Crowdsource your template

Another method is to create a list of criteria for people to evaluate your site on, and then share this link with either a list of targeted users, or with as many people as you can get. This list can include critical user tasks on your site, or specific areas you want feedback on like a Contact Us page, accessibility, or readability of content. Our program has been beta-testing a few Expert Evaluation sheets, created in Google docs, that we share online and ask people to fill in true crowdsourced fashion. You can find our Expert Evaluation Worksheet in our Usability Starter Kit (scroll down).

The Mobile Application Testing Program at GSA uses federal crowdsourcing to discover bugs and compatibility issues on mobile systems as well.

Design workshops and events

Some User Experience professionals take crowdsourcing to the next logical step—from conversations with customers to literally “co-designing” it with them. This co-creation can take on many forms, from the hack-a-thons and design-a-thons that are more and more common these days, to a Design Studio format often used with both users and stakeholders.

Here, you create an event, either live or virtual, in which you focus on ideas for a product, actual designs, or both, and work in tandem with your stakeholders in driving the creative process. These workshops require skill to pull off, and will require skilled facilitators to keep the process moving in the right direction. But for many, this technique is the best way to create something you are very comfortable that users will understand, because they were literally involved in creating it.

Crowdsourcing UX tools

You may have to create with your customers, but you need some help pulling it off. There are myriad tools that allow you to do early stage Ideation with your customers, or do online usability testing, surveys, or card sorts.

What UX crowdsourcing can’t give you

All of these techniques (and especially the tools) promise a lot, but they can’t give you everything. A few limitations to keep in mind:

1. Some skills required—Facilitating a crowdsourcing effort (and especially a design event) takes a lot of practice. And tools are generally only as good as the person using them.
2. Mystery users—Depending on how far you go with your crowdsourcing, you may be getting information from people who don’t represent your users, or even people from different countries and cultures who have different expectations from the Web.
3. Quantitative Metrics—You may be able to get crucial completion and conversation rates with your tools or templates, or you might not.
4. Follow-ups—What did people mean when they said your website looked “commercial?” Do you have a method of following up and getting more in-depth replies?

In the end, crowdsourcing is a good way to expand your information collection efforts and try and use new tools and techniques. Look into it, and see if it’s right for your organization and product.