{
    "version" : "https://jsonfeed.org/version/1",
    "content" : "news",
    "type" : "single",
    "title" : "Lessons Learned: Evaluating Video Content |Digital.gov",
    "description": "Lessons Learned: Evaluating Video Content",
    "home_page_url" : "/preview/gsa/digitalgov.gov/cm-topics-button-component/","feed_url" : "/preview/gsa/digitalgov.gov/cm-topics-button-component/2016/12/01/lessons-learned-evaluating-video-content/index.json","item" : [
    {"title" :"Lessons Learned: Evaluating Video Content","summary" : "If you were to perform research on the value proposition of training videos, you would notice that opinions are split on their efficacy. Despite all the tools that are out there that can help you evaluate video quality, views, and drop-off, there are some things that should be considered in the analysis of your organization’s","date" : "2016-12-01T12:00:23-04:00","date_modified" : "2024-04-02T09:45:13-04:00","authors" : {"tim-jakubowski" : "Tim Jakubowski"},"topics" : {
        
            "content-strategy" : "Content Strategy",
            "multimedia" : "Multimedia",
            "product-and-project-management" : "Product and project management",
            "research" : "Research",
            "user-experience" : "User Experience"
            },"branch" : "cm-topics-button-component",
      "filename" :"2016-12-01-lessons-learned-evaluating-video-content.md",
      
      "filepath" :"news/2016/12/2016-12-01-lessons-learned-evaluating-video-content.md",
      "filepathURL" :"https://github.com/GSA/digitalgov.gov/blob/cm-topics-button-component/content/news/2016/12/2016-12-01-lessons-learned-evaluating-video-content.md",
      "editpathURL" :"https://github.com/GSA/digitalgov.gov/edit/cm-topics-button-component/content/news/2016/12/2016-12-01-lessons-learned-evaluating-video-content.md","slug" : "lessons-learned-evaluating-video-content","url" : "/preview/gsa/digitalgov.gov/cm-topics-button-component/2016/12/01/lessons-learned-evaluating-video-content/","content" :"\u003cp\u003eIf you were to perform research on the value proposition of training videos, you would notice that opinions are split on their efficacy. Despite all the tools that are out there that can help you evaluate video quality, views, and drop-off, there are some things that should be considered in the analysis of your organization’s videos. \u003cdiv class=\"image\"\u003e\n  \u003cimg\n    src=\"https://s3.amazonaws.com/digitalgov/_legacy-img/2017/03/600-x-400-Time-to-evaluate-concept-clock-donskarpo-iStock-Thinkstock-500832345.jpg\"\n    alt=\"Stopwatch with the words, Time to Evaluate.\"/\u003e\u003c/div\u003e\n\n\u003c/p\u003e\n\u003cp\u003eAs a member of the Service Design practice at the Consumer Financial Protection Bureau (CFPB), I was tasked with a research project evaluating how non-consumers interact with the CFPB in regards to complaint data. This audience included companies, federal and state level government agencies, and congressional offices. Each group accesses a “portal” to either respond to complaints (companies) or to have access to complaint data (congressional offices and government agencies). The tasks that are performed in each respective portal vary dependent upon responsibilities.\u003c/p\u003e\n\u003cp\u003eEach portal was comprised of an interface that was not highly intuitive to navigate, but was supported with a manual of how to use the portal, FAQs (which somewhat mirrored the manual), and “training videos.” When it came to evaluating the videos, I did not have access to any analytics on usage or if they were viewed completely, and surveys from a year prior did not capture feedback on them either. As I began analyzing, I started having some thoughts on videos in general.\u003c/p\u003e\n\u003ch2 id=\"determine-why-the-videos-were-created-in-the-first-place\"\u003eDetermine why the videos were created in the first place\u003c/h2\u003e\n\u003cp\u003eWas it something users even wanted? Is your UI and content too hard to understand in the first place and that’s why you created a video? It turns out that on our end, one business unit would train all new portal users once they were on-boarded. This practice was not scalable at a growing agency, so they decided to create videos. It was not something that the CFPB had ever received feedback on, but rather a business desire.\u003c/p\u003e\n\u003cp\u003eI would say think holistically — make sure that the video is not trying to solve another problem your users are having that will still exist once you create the video.\u003c/p\u003e\n\u003ch2 id=\"if-possible-leverage-any-end-user-interviews\"\u003eIf possible, leverage any end user interviews\u003c/h2\u003e\n\u003cp\u003eWith my timeline and Paperwork Reduction Act (PRA) limitations, I was limited to nine interviews. I took a sample that varied in entity size, complexity, and regulatory-facing experience. The interviews encompassed a variety of topics and probed about what the user does when he/she does not know how to utilize the portal. It turns out that every single entity was using the manual, but would have preferred screenshots in it. If the manual was unclear, they ended up calling that business unit anyway. Each user said he/she didn’t have time for videos, and had never even bothered viewing them.\u003c/p\u003e\n\u003ch2 id=\"consider-the-cost-of-maintenance\"\u003eConsider the cost of maintenance\u003c/h2\u003e\n\u003cp\u003eDepending on the content, there is potential that the video could become outdated quite quickly. When I was reviewing the videos, I noticed that they were done at a time prior to some incremental enhancements, and therefore were not 100% accurate. At a relatively new bureau where things can evolve quickly, the potential for further change was high. In terms of the investment of maintaining these, we realized it would be much more cost-effective to just include screenshots in the manual, and republish it more quickly than a video.\u003c/p\u003e\n\u003cp\u003eOverall, evaluation of training videos really depends on your project and the limitations you have, but I cannot stress the importance of understanding of why it was created in the first place and what problem the video is trying to solve. The solution may not be in the video, but could lie elsewhere.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eThis article is the result of a thread on our User Experience Community listserv. See how other agencies are working with \u003ca href=\"/preview/gsa/digitalgov.gov/cm-topics-button-component/topics/content-strategy/\"\u003econtent\u003c/a\u003e and \u003ca href=\"/preview/gsa/digitalgov.gov/cm-topics-button-component/topics/multimedia/\"\u003evideo\u003c/a\u003e, and find out how to join our \u003ca href=\"/preview/gsa/digitalgov.gov/cm-topics-button-component/communities/\"\u003eCommunities\u003c/a\u003e, like \u003ca href=\"/preview/gsa/digitalgov.gov/cm-topics-button-component/communities/video-production/\"\u003eDigital Audio/Video Production and Strategy\u003c/a\u003e and \u003ca href=\"/preview/gsa/digitalgov.gov/cm-topics-button-component/communities/user-experience/\"\u003eUser Experience\u003c/a\u003e.\u003c/em\u003e\n\u003cem\u003eIf you’re interested in writing for \u003ca href=\"https://digital.gov\"\u003eDigitalGov\u003c/a\u003e, learn about the \u003ca href=\"/preview/gsa/digitalgov.gov/cm-topics-button-component/resources/open-opportunities/\"\u003eOpen Opportunities program\u003c/a\u003e and check out our assignments (for feds with .gov or .mil email accounts only). If your agency’s team has tech-related lessons learned, case studies, digital or mobile product news, strategy success stories, etc. to share, please \u003ca href=\"/preview/gsa/digitalgov.gov/cm-topics-button-component/about/contribute/\"\u003ereview our guidelines\u003c/a\u003e and \u003ca href=\"/preview/gsa/digitalgov.gov/cm-topics-button-component/about/\"\u003econtact DigitalGov\u003c/a\u003e to submit an article idea.\u003c/em\u003e\u003c/p\u003e\n"}
  ]
}