Software and Apps Challenges
Challenge and prize competitions are one path that federal agencies take to drive innovation and solve mission-centric problems—whether technical, scientific, or creative. One type of competition is software and apps challenges, where solvers are asked to develop specific software or other code-based technical solutions, such as websites, mobile applications, or algorithms.
Here you’ll find tips on running a software/apps challenge, resources, examples and information about online platforms you can use to host your competition.
Definition
A software/apps challenge is when an organization asks the public to create software that may use a particular data set or have a set of functionalities outlined by the seeker, in order to solve an existing problem or draw attention to open and available datasets.
Tips for Running a Software/Apps Challenge
- Decide what kind of solution you are seeking. The platform that serves you best can vary depending on whether you are only seeking proposals, prototypes, or fully developed applications or code.
- Depending on complexity, a series of small challenges may be preferable to a large one that demands more time and resources.
- Consider two-phase challenges, focused first on ideas or proposals and then on actual prototypes or functioning applications or solutions.
- Know whether you want to catalyze the creation of open source code and software that others can build upon, or if it is necessary for your agency to own the intellectual property rights (or license them).
- Determine what prizes will motivate your solvers: Do you have access to key people in the industry who can be judges? Could you offer time with a venture capital or business consulting firm? Is the prize money enough to get people to invest their time? Are you asking competitors to stretch themselves and be innovative?
- Consider the role of the public when choosing a platform. You may want the public to have input in the decision through voting, or give feedback on submissions to promote further development.
- Communicate with the participants. In the rules, it is best to clearly outline what it is you are seeking, available datasets, how you intend to use the software or solutions developed, etc. so competitors can develop better solutions and they will know how their contributions will be used.
- Determine from the beginning and state in the challenge rules details about technology ownership, parameters for submissions, who qualifies to enter, how the entries will be judged and how the prizes will be awarded.
- Please see the multiple resources listed below. There are so many ways to run apps challenges, that experts have created pages of plans and details that will further your understanding and goals. Choose what is best for your agency needs and goals.
Resources
Expert Panel: Collaborative Innovation, Apps Challenges. Case Foundation 2012
Panel Discussion: Collaborative Innovation, Algorithm and Big Data Challenges
Best practices: Marketing an apps challenge
Best practices: Post-competition
Lessons learned, sample rules, and press clips
ChallengePost: App Contest Best Practices
Examples
Apps for Vehicles The “Apps for Vehicles” challenge, part of the Department of Energy’s “Energy Data Initiative,” was designed to improve safety and fuel efficiency through data innovation. Entrants were asked to use vehicle open data, such as engine speed, brake position, and distance covered to create novel ways to improve safety and efficiency. One of the winning entries, MyCarma, uses the gas mileage data of each individual user to create a personal fuel economy sticker, and can offer other makes/models that may save money in the future based on the user’s driving style. The DOE awarded $16,000 in prizes to multiple winners for the first phase of the competition.
DTRA Algorithm Challenge The Defense Threat Reduction Agency issued this challenge to develop an algorithm with the goal of specific identification of organisms from a stream of DNA sequences. An incredible 2,700+ entries were submitted in hopes of winning a portion of the $1 million in prize money.
CMS Provider Screening Challenge
The Centers for Medicare and Medicaid Services issued this challenge to develop a system to screen Medicaid providers in order to validate credentials, authenticate identity, and check for sanctions, among others. The solution needed to be able to function in a multi-state and multi-program capacity. CMS offered between $500,000 and $600,000 to the winning entrants. The unique aspect of this competition was dividing it into more than 100 small, discrete challenges.
“Apps for Energy” is a mobile application challenge sponsored by the U.S. Department of Energy seeking apps that help consumers make the most of Green Button electricity use data. Green Button is a public data source for energy use, so the challenge was initiated with the hopes of developing applications that would help consumers better manage their energy usage. The winning application, Leafully, was developed not only to inform consumers about their personal energy usage but also to put that information into meaningful terms to educate users about their energy footprint. The U.S. Department of Energy awarded $100,000 in prizes to eight winners, ranging from $6,000 to $30,000.
The U.S. Department of the Treasury sponsored the MyMoneyAppUp challenge for the development of applications to help citizens take a more active role in controlling and shaping their financial futures. It was a two-part challenge with a total prize purse of $27,500 in which the public was asked to contribute ideas in the first phase, and create the software applications in phase two.
The FCC challenged the developers to use hyper-local government and public data to create applications that helps Americans have greater access to such personalized information and foster a richer connection between the public and their local or tribal governments, regardless of geography, race, or economic status. The FCC offered $100,000 in prizes ranging from $1,000 to $30,000 for the best overall application.
Challenge.gov contains more examples of software/apps challenges.
Criteria for Choosing a Platform
Here are some criteria for choosing a software or apps challenge platform.
General
- Does this tool have (or has it had) any federal government clients?
- Is the tool 508 compliant?
- Do they have a federally-compatible Terms of Service agreement?
- How many developers visit the site? How does the site reach out to software developers and Web designers?
- How much does it cost per challenge? For an enterprise licence? Is a free trial available?
Functionality
- Does this tool have the ability to display submissions in a gallery for public viewing?
- Is there the ability to keep submissions private?
Citizens can:
- Easily create an account with an email address
- Share the challenge or their submission with friends and family via social media
- Easily submit their entries
- Receive email updates on the challenge
- Ask the agency questions and receive feedback
Federal agencies can:
- Brand the challenge with agency seal, photos, video, etc.
- Keep submissions private, make them all public, or a mix of the two
- Easily receive and answer questions from entrants
- Conduct judging online, or easily download submissions
- Have a public voting component
- Offer a prize at the end for best app and publicize
Online Platforms and Tools
The following list of tools and challenge platforms is provided as a resource and is not an endorsement for any company or technology. Those companies that are on the GSA Schedule 541-4G for Challenges & Competition Services are noted with (GS) next to the company name. Those that have a federal-compatible terms of service are noted with a (ToS) next to the company name.
ChallengePost (GS)
ChallengePost provides a user-friendly platform that allows for a range of public participation, from the submission and display of challenge entries, to community discussions, and public voting. Clients include the Federal Trade Commission (FTC) and U.S. Department of Agriculture (USDA). The platform has reviewing features that challenge managers can use to select which entries to showcase. It provides both public and private judging tools.
ChallengePost powers the Challenge.gov list of federal challenge competitions and hosted a range of federal competitions for the first three years of Challenge.gov. This platform is currently available to federal customers at no cost for software and application competitions.
InnoCentive (GS)
InnoCentive has run more than 1,600 challenges. Government clients include NASA, the State Department, DoD, EPA, the Air Force Research Lab, and the Defense Threat Reduction Agency. Challenges can be purchased a la carte, or in bundles. InnoCentive posts the challenge, answers questions from solvers, transfers prize money to winners, and messages all who are registered on the site about each new challenge.
Kaggle Kaggle is the world’s largest community of data scientists. They compete with each other to solve complex problems, and the top competitors get invited to consult on interesting projects from some of the world’s biggest companies through Kaggle Connect. Kaggle has a proven track-record of solving real-world problems across a diverse array of industries including life sciences, financial services, energy, information technology, and retail.
TopCoder (GS)
A Web community of software developers and digital designers. Challenge sponsors pose a problem to be solved by the community. This platform has been used by the Department of Defense and NASA among others to host a number of challenges that require expertise in software development and Web design.If you are a vendor who provides a software/apps challenge platform and would like to be included in this list, contact Karen Trebon via email.
