DigitalGov Search: Our Open Source Strategy

At DigitalGov Search, we keep an eye on on our what our government counterparts are up to, both in the U.S. and other countries. We recently came across Gov.UK’s philosophy on and approach to coding in the open. It caught our attention and we realized we should also articulate our open source strategy.

Use and Contribute to Open Source Projects

Since 2010, we’ve embraced and leveraged open source software to build our site search service for federal, state, and local government websites. This use of open source has allowed us to experience enormous growth over the past few years. In March 2013 alone, over 24 million searchers received results from our service—a four-fold increase since March 2010.

Our search service is now a complex system made up of many moving parts, including providing type-ahead search suggestions, serving search results, fetching, indexing, and caching content, and providing analytics.

Each of these parts is compiled into our codebase and, as we use open source components for our system, we contribute back to the projects.

Code in the Open

We recently began to unravel our monolithic codebase so that we can share individual pieces of our code. To borrow the phrase from Gov.UK, we’re now coding in the open.

We recently released the code for our Jobs and Recalls API servers. They’re our first foray into coding in the open. The source code for these API servers is in our GitHub repo and is available for anyone to see and contribute to.

The data products for this code are also open and available for anyone to consume on our Developer hub.

These two servers and their underlying data now operate outside of our core search codebase.

Following this same model, moving forward, we plan to:

• Share first—For every new feature, we’ll write the code so that anyone can make use of the code, not just us. If the public community contributes to the codebase, we’ll be able to improve this feature without taxing our developers.
• Expose APIs—We’ll expose our data products as APIs so that anyone can make use of the data, not just searchers on a government websites.
• Be our own customer—We’ll use our own public code and data just like everyone else. We’ll call our own API servers to integrate the data within our search results pages.

Make Things Open to Make Things Better

We agree with Gov.UK that “to make things open makes things better.”

We have finite resources and we don’t want to lose our focus on serving our agency customers and improving visitors’ search experience on government websites. So, we won’t be spending a lot of time to build or support a vibrant community around our code.

That said, we hope that exposing the pieces of our system will be useful to someone somewhere. We’ll continue to provide the “ingredients” of our search service so that others will be able to make use of the code and data in ways that we could never imagine.

And, We’re Not Alone

We’re not alone. Other federal agencies have embraced the approach of coding in the open and have GitHub repos. Below are a just a few of our many favorites.

• The Consumer Financial Protection Bureau has embraced open source, including serving their public data sets through qu.
• USA.gov recently released the code for their iPhone app.
• The White House was in the forefront of using open source software in the federal government with their use of Drupal. They’re continuing to lead by example by opening up the code for their We the People petitions and iOS and Android mobile apps.Editor’s Note: Interested in learning more about search from the DigitalGov Search team? Register for the Search Is the New Big Data (in-person training).