{
    "version" : "https://jsonfeed.org/version/1",
    "content" : "news",
    "type" : "single",
    "title" : "The Right Tools for the Job: Re-Hosting DigitalGov Search to a Dynamic Infrastructure Environment |Digital.gov",
    "description": "The Right Tools for the Job: Re-Hosting DigitalGov Search to a Dynamic Infrastructure Environment",
    "home_page_url" : "/preview/gsa/digitalgov.gov/bc-archive-content-3/","feed_url" : "/preview/gsa/digitalgov.gov/bc-archive-content-3/2016/08/18/the-right-tools-for-the-job-re-hosting-digitalgov-search-to-a-dynamic-infrastructure-environment/index.json","item" : [
    {"title" :"The Right Tools for the Job: Re-Hosting DigitalGov Search to a Dynamic Infrastructure Environment","summary" : "This is the first post of a 5-part series. DigitalGov Search is a commercial-grade search engine provided as a shared-service by the United States General Services Administration. We power about 2,300 search configurations for hundreds of federal, state, and local government agencies. Using our platform, agencies can easily configure a search experience for the public that","date" : "2016-08-18T10:00:29-04:00","date_modified" : "2025-01-27T19:42:55-05:00","authors" : {"dmccleskey" : "Dawn Pointer McCleskey"},"topics" : {
        
            "cloud-and-infrastructure" : "Cloud and infrastructure",
            "content-strategy" : "Content strategy",
            "product-and-project-management" : "Product and project management",
            "search" : "Search"
            },"branch" : "bc-archive-content-3",
      "filename" :"2016-08-18-the-right-tools-for-the-job-re-hosting-digitalgov-search-to-a-dynamic-infrastructure-environment.md",
      
      "filepath" :"news/2016/08/2016-08-18-the-right-tools-for-the-job-re-hosting-digitalgov-search-to-a-dynamic-infrastructure-environment.md",
      "filepathURL" :"https://github.com/GSA/digitalgov.gov/blob/bc-archive-content-3/content/news/2016/08/2016-08-18-the-right-tools-for-the-job-re-hosting-digitalgov-search-to-a-dynamic-infrastructure-environment.md",
      "editpathURL" :"https://github.com/GSA/digitalgov.gov/edit/bc-archive-content-3/content/news/2016/08/2016-08-18-the-right-tools-for-the-job-re-hosting-digitalgov-search-to-a-dynamic-infrastructure-environment.md","slug" : "the-right-tools-for-the-job-re-hosting-digitalgov-search-to-a-dynamic-infrastructure-environment","url" : "/preview/gsa/digitalgov.gov/bc-archive-content-3/2016/08/18/the-right-tools-for-the-job-re-hosting-digitalgov-search-to-a-dynamic-infrastructure-environment/","content" :"\u003cblockquote\u003e\n\u003cp\u003e\u003cem\u003eThis is the first post of a 5-part series.\u003c/em\u003e\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003e\u003ca href=\"https://search.gov\"\u003eDigitalGov Search\u003c/a\u003e is a commercial-grade search engine provided as a shared-service by the United States \u003ca href=\"http://www.gsa.gov/\"\u003eGeneral Services Administration\u003c/a\u003e. We power about 2,300 search configurations for hundreds of federal, state, and local government agencies. Using our platform, agencies can easily configure a search experience for the public that brings together resources from across their many publishing platforms: websites, blogs and feeds, social media, and government-specific resources like rules and notices from the Federal Register, and posts from USAJobs.\u003c/p\u003e\n\u003cp\u003eCustomers include the \u003ca href=\"https://search.whitehouse.gov/search?query=bo+and+sunny\u0026amp;op=Search\u0026amp;affiliate=wh\"\u003eWhite House\u003c/a\u003e, \u003ca href=\"http://search.digitalgov.gov/customers.html\"\u003e12 Department-level websites\u003c/a\u003e, all branches of the military, the \u003ca href=\"http://find.irs.gov/search?utf8=%E2%9C%93\u0026amp;affiliate=irs\u0026amp;query=tax+extension\u0026amp;commit=Search\"\u003eInternal Revenue Service\u003c/a\u003e, \u003ca href=\"http://nasasearch.nasa.gov/search?utf8=%E2%9C%93\u0026amp;affiliate=nasa\u0026amp;query=juno\"\u003eNASA\u003c/a\u003e, and many more. We partner with agencies continually to improve the public’s ability to find the information they need on government websites, and streamline agency staff workflows for managing their searches. Launched originally as the \u003ca href=\"http://web.archive.org/web/20001109105200/http://www.firstgov.gov\"\u003esearch box on FirstGov.gov\u003c/a\u003e, we now serve 260 million queries a year across the government space, with peak throughput over 59 transactions per second. Our service is a big, mature system. Our move to a new infrastructure had to be perfect, and invisible.\u003c/p\u003e\n\u003cdiv class=\"image\"\u003e\n  \u003cimg\n    src=\"https://s3.amazonaws.com/digitalgov/_legacy-img/2016/08/600-x-330-White-House-DG-Search-menu.jpg\"\n    alt=\"Screen capture from a Kindle device of DigitalGov Search on the White House website.\"/\u003e\u003c/div\u003e\n\n\n\u003cp\u003eIn 2016, we began migrating the DigitalGov Search infrastructure to Amazon Web Services’ (AWS) Elastic Compute Cloud (EC2). Given the tight window of opportunity – seven months from project launch to migration – and the velocity constraints of our small team size, we elected to forklift the application into the new infrastructure as-is, and leave for a subsequent phase any consideration of refactoring to adopt additional modern practices like containerization. Even so, we were facing a challenging combination of requirements:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003eThe DigitalGov Search EC2 infrastructure needs to support the addition, removal, or replacement of servers on-the-fly without interruption of service, at a significant reduction of costs for the government;\u003c/li\u003e\n\u003cli\u003eDigitalGov Search customers – government departments and agencies – need the option to direct their users to our results page “masked” with their own domain name, such as search.justice.gov, rather than the default domain of search.usa.gov;\u003c/li\u003e\n\u003cli\u003eCustomers using domain masks need those domains to be served over https; and\u003c/li\u003e\n\u003cli\u003eThe DNS for all search.usa.gov records are required to be signed by DNSSEC.\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eMeeting requirements 1, 2, and 3 together would be straightforward. Meeting requirements 2, 3, and 4 together would also have been straightforward. Satisfying the first and last requirements together required fancy footwork, a new open-source software feature and a two-tier DNS server architecture that we’ll describe in detail in this series of posts.\u003c/p\u003e\n\u003cp\u003eWhile this series of posts will discuss the requirements above separately, the build and migration was handled in three phases:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003eInsource our content delivery network (CDN) and web application firewall (WAF), initially routing traffic to our old data centers, but still reducing monthly costs significantly\u003c/li\u003e\n\u003cli\u003eBuild our servers and application within AWS\u003c/li\u003e\n\u003cli\u003eCut over our read-only data center, allow dust to settle; then cut over our read-write data center, allow dust to settle; shut off old infrastructure\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eIf you have questions or would like to discuss particulars, \u003ca href=\"mailto:search@gsa.gov\"\u003ewe would love to hear from you\u003c/a\u003e.\u003c/p\u003e\n\u003ch3 id=\"series\"\u003e\n  \u003cem\u003eThis is the first post in a five part series. Stay tuned for the following posts:\u003c/em\u003e\n\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"/preview/gsa/digitalgov.gov/bc-archive-content-3/2016/09/02/quality-speed-and-lower-costs-yes-you-can-have-it-all/\"\u003eQuality, Speed, and Lower Costs: Yes, You Can Have It All\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"/preview/gsa/digitalgov.gov/bc-archive-content-3/2016/09/06/a-domain-by-any-other-name-cnames-wildcard-records-and-another-level-of-indirection/\"\u003eA Domain by Any Other Name: CNAMES, Wildcard Records and Another Level of Indirection\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"/preview/gsa/digitalgov.gov/bc-archive-content-3/2016/09/07/lets-encrypt-those-cnames-shall-we/\"\u003eLet’s Encrypt those CNAMES, Shall We?\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"/preview/gsa/digitalgov.gov/bc-archive-content-3/2016/09/12/dnssec-vs-elastic-load-balancers-the-zone-apex-problem/\"\u003eDNSSEC vs. Elastic Load Balancers: the Zone Apex Problem\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n"}
  ]
}