The Content Corner: User Research for Complex Systems

My office is preparing to embark on a complete redesign of a 10-year-old system that averages 20,000 users a month. The success and adoption of the new system design and the product as a whole will be heavily determined by how well our team translates users’ needs. Providing a good user experience will also play a critical role in reducing struggles long-time users may encounter with a new system.

Note: Due to the early stages of this project and various procurement concerns, I am leaving out some of the specifics but still felt that this practical discussion of user research could be beneficial.

Web Application Versus Website=Big Difference

One of the key distinctions that will most likely make this the most complex site redesign I have been involved with is the complexity of content this site provides. It is far more than an informational website, but is a Web-based service or application much closer to installed software. Most of us, especially from a content creation and UX perspective work on sites that deliver information and the most complicated site interaction is search. The level of user interaction generally goes something like: “I need to find X. OK, I found X. Thanks.”

Any one of my system’s nine distinct user types will have very specific tasks that they want to complete once logged on to the system. These can include:

• Uploading PDFs
• Creating custom searches
• Assigning user rights to colleagues
• Creating and adding content to custom or system-generated folders
• Adding notes
• Rating content on a five star system, etc.

And while a typical website user will also have a specific task at hand (generally to find information) the range of complex tasks offered by my system requires an even deeper exploration of the user’s needs and the many paths they could take to reach a particular goal.

ABC: Always Be Communicating

Because of this level of complexity and the importance of this critical, national judiciary system, we are structuring this project to begin and end with the user. My hope is that by the time our new system launches, users will be tired of hearing from us. I’ve never heard of a project failing due to over-communication with their stakeholders.

We’ve recently kicked off the very early stages of this project by meeting with a user group we work closely with and is involved with several other user groups that regularly use the system, including our largest user group, those applying for positions.

This initial meeting’s purpose was essentially to lay the groundwork for the project and discuss everyone’s hopes, dreams and goals for a new system. It was also a critical first step in helping all of our users to break away from the conceptions and familiarity of our 10-year-old system.

This is a critical issue that probably doesn’t get enough research and discussion. I’m familiar with the concepts behind change management and freezing in preparation for code or system changes, but this doesn’t quite address the issue of user familiarity and comfort with an existing system. Our main hope is that if we do our jobs right, our users will never even miss the old system.

Getting to Know You

Let’s get back to the initial user group and some of the UX methods that we are going to employ for this project.

Focus Groups

Because of the makeup of our users and the early stage of the project, we are going to start our process with some formal and informal focus groups. The goal of these is to better understand some of the high level expectations and conceptions of the current system and how it translates to the actual environment or reality. Over 10 years with few major changes, it is very possible that the process our system was built to facilitate and the actual process have diverged significantly. A focus group will help us get a nice overview of user needs without delving too deep into actual interfaces or functionality. This is also helpful in our early stages when we haven’t started the full procurement process and are far away from even having paper wireframes.

User Interviews

The next step in our user research helps us get more detailed about the user and how the user performs certain tasks. Again, the level of actual system interaction can vary. I have done numerous user interviews where we focus much more on tasks the user needs to achieve rather than how they do “X” in the system. Due to the complexity of my system and user task possibilities, I’m looking for user interviews to help understand a task’s start and finish removed from any system interaction. My hope is that understanding at this level can help us build the new system around a user’s journey or process.

Usability Testing

The final step of our usability research will be to actually watch users interact with both the current system and with mockups or wireframes of a future system.

The efficacy of performing usability tests on existing systems has always been a concern of mine, but one of the things I am hoping to get out of these tests is to learn about any existing pain points or workarounds that long-time users have. I’ve always recognized that soon after a system is implemented the next important phase is users figuring out how to make it do the things they actually want to do (meaning the system for whatever reason isn’t actually addressing their needs). From past experiences, I also don’t know if you ever waste time by watching users interact with a website or system that you administer. There are always surprises and users doing things that completely change your way of looking at your old, familiar website.

Knowledge is Power

By wrapping ourselves in the warm blanket of user research and combining that with frequent communication, we hope to understand our complex set of users as best as possible. And via this understanding I want to start the process of creating a new system by completely divorcing it from programming or coding but focusing on tasks. I’d rather look at user stories borrowed from the world of agile programming that are centered around what a user needs to accomplish.

In our environment with various users and disparate needs and tasks, distilling all we learn from our user research into the powerful statement of “As a (user type), I want (feature, function), so I can (task/reason)” will be extremely valuable. It can really help maintain focus on the user’s needs which can easily be lost when developing a complex system within the federal government (or anywhere). There are so many conflicting priorities that the users can often get lost in the need to meet budgets, timelines, or political concerns. And while we all live in the real world, as much as possible we should still err on the side of the user.