Using Focus Groups to Make Better Videos

There’s what you expect your audience to think, and then there’s what your audience is actually thinking. Sometimes, these can be entirely different. But, you won’t know unless you test it.

For the release of the 2014 Consumer Action Handbook (CAH), the Federal Citizen Information Center’s marketing team piloted a series of videos. The videos intended to showcase the expertise of the CAH’s editor-in-chief, Marietta Jelks. Using letters received from the public asking consumer questions, Marietta gave her advice, trying to help not only the writer, but other members of the public who may have similar problems.

Five different “Ask Marietta” videos were created, using two different formats. They were distributed via USA.gov social media channels, the USA.gov blog, and through PR Newswire. After their release, the marketing team was bursting with ideas of what could be done better or differently.

But before committing more time and resources to the project, it was necessary to get the reaction from the the actual public to the videos and their overall concept—not just what we assumed was the public perception.

Recruiting

To do this, we recruited participants for two focus groups. One was an in-person session at GSA, and the other was a virtual discussion held using GoToMeeting. Because roughly 70% of communication is non-verbal, the in-person session was vital so we could see people’s body language and their immediate reactions to the materials. However, it also limited us to only people in the local area.

To get more of a nationwide spread in participants, we created the virtual session, so even if they weren’t physically here, we could still get their input and read their body language and inflections in their voice, etc. Adhering to OMB rules, we could only use nine non-federal employees for the focus groups, but were unlimited in how many feds could participate. We created a script to keep us on task—here’s a generic version (Word document, 25K, 11 pages, Nov. 2014) of it.

Our recruiting efforts yielded six people for the in-person session (5 non-feds, 1 fed, and 2 no-shows), and five people for the virtual session (3 non-feds, 2 feds). Initially we were disappointed in the no-shows, but found that six made for a comfortable discussion. We felt that any more than that, the conversation may have become uneven, with a few people dominating the discussion, and more difficult to include the others. During the virtual session, there were a few technical issues in the beginning, some participants were unfamiliar with working their webcams or GoToMeeting, but they were ironed out quickly and were of no consequence.

Participants were asked to view two of the videos, and some social media messaging used for promotional purposes. Both groups had spirited, insightful, and engaged discussions giving great feedback. Following the discussion portion, quantitative data was collected from each participant regarding not only demographics, but ratings for each video, measuring their feelings about the “Ask Marietta” concept as a whole.

What we learned

This information, along with qualitative data from the sessions was compiled into a thorough report. Some of the reactions of the focus groups were very unexpected, others were not, but either way it gave the team solid measurements (rather than assumptions) on which to build the “Ask Marietta” brand, make changes, and produce an overall better product to serve citizens.

As advice to others looking to do this sort of research, I’d recommend keeping the groups on the smaller side—six seemed comfortable, and any more may present difficulties. Ask participants for their level of education beforehand. It occurred to us later that it was likely those in our groups had a college degree—which is helpful, but not necessarily reflective of our audience as a whole.

Also follow up with your participants to thank them for their help, and see if maybe they’d be willing to come back for a future session, after changes have been made to your product. We look forward to hearing how their thoughts may or may not have changed after we make some modifications to the “Ask Marietta” concept.Colleen Bayus is a Marketing Specialist on the USA.gov (FCIC) Marketing Team at the U.S. General Services Administration.