Posts tagged open

Resources for Developers

In line with the White House’s Digital Government Strategy, we’re making our code and data more open.

APIs for Search.gov Customers

These APIs are available for use on official government websites only. You must be a Search.gov customer. Sign in is required.

i14y—This API allows you to send content directly from your content management system (CMS) into Search.gov for real-time indexing. Instructions can be found under Admin Center > YourSite > Content > i14y. Request your site be activated for i14y to enable this section.

Search Results API—This API exposes all relevant results “modules” in a single JSON call, including: web results, best bets, health topics, job openings, recent tweets, recent news, recent videos, Federal Register documents, and related searches. Use it to pull search results from our service to display on your agency’s website or mobile applications. Instructions can be found under Admin Center > YourSite > Activate Search.

Type-ahead API—This API exposes the type-ahead suggestions that often appear below your search box as searchers enter their search terms. Instructions can be found under Admin Center > YourSite > Activate Search.

APIs and Data Feeds for the Public

Jobs API—Taps into a list of current jobs openings in federal agencies. Jobs are searchable by keyword, location, agency, schedule, or any combination of these.

Visit USA.gov/Developer for a full list of USA.gov’s APIs and data feeds available to the public.

Source Code (Public Github Repositories)

ASIS (Advanced Social Image Search)—The source code that runs our image search. ASIS indexes images from Flickr, Instagram, and media RSS to provide a combined search API.

i14y—The source code that runs our search engine for agencies’ published content. i14y indexes agencies’ published content in real time, for search through our regular search channels.

Jobs—The source code that runs our jobs search and Jobs API. Indexes current jobs openings in federal agencies.

Non-.gov URLs—The source code that runs our non-.gov URLs API. Indexes all government URLs that don’t end in .gov or .mil.

Punchcard—The repository of synonyms, protected words, stop words, localizations, and other vocabularies that we use to improve the precision, recall, and usability of search results.

search.digitalgov.gov—Pages and layout for our website, https://search.gov.

Site Links—The source code that “decorates” organic web results to provide additional, value-added links to help searchers find what they’re looking for. Currently provides one-click links to EDGAR filings for relevant SEC.gov results. Also published as a Ruby gem at https://rubygems.org/gems/sitelink_generator.

Unique Child Attribute—activerecord-validate_unique_child_attribute is an ActiveRecord extension to enforce uniqueness validations when accepting nested attributes. Works around Rails issue #4568.

CMS Modules and Plugins

These modules and plugins were developed by Search.gov customers, not us, but we’re linking to them here so you have easy access to them. Use their respective platforms to connect with their developers and submit issues.

Drupal Module—Allows you to connect your Drupal website to your Search.gov search configuration. The module also supports realtime indexing via Search.gov’s i14y content indexing API. Check out our help docs here.

WordPress Plugin—Starter code - contributions welcome! This plugin allows you to override the default WordPress search and connect your WordPress-powered website to your Search.gov search configuration.

Search Is the New Big Data

Search is easy, right? You type a term in a search box and the exact page you’re looking for appears at the top of the list of results. But search is hard and has many shades of grey.

On April 10, 2014, Loren Siebert, our senior search architect, presented on:

  • Complexities of recall and precision,
  • Popular open source search technologies, and
  • “Search magic” like stemming, synonyms, fuzziness, and stopwords.

Download the slide deck and visit the resources below to learn more.


Page last reviewed or updated:

Our Open Source Strategy

We keep an eye on on our what our government counterparts are up to, both in the U.S. and other countries. We first came across Gov.UK’s philosophy on and approach to coding in the open a couple of years ago. It caught our attention and we realized we should also articulate our open source strategy.

Use and Contribute to Open Source Projects

Since 2010, we’ve embraced and leveraged open source software to build our site search service for federal, state, and local government websites. This use of open source has allowed us to experience enormous growth over the past few years. In July 2014 alone, over 23 million searchers received results from our service—a five-fold increase since July 2010.

Our search service is now a complex system made up of many moving parts, including providing type-ahead search suggestions, serving search results, fetching, indexing, and caching content, and providing analytics.

Each of these parts is compiled into our codebase and, as we use open source components for our system, we contribute back to the projects.

Code in the Open

We recently began to unravel our monolithic codebase so that we can share individual pieces of our code. To borrow the phrase from Gov.UK, we’re now coding in the open.

We recently released the code for our social media image, jobs and recalls API servers. They’re our first foray into coding in the open. The source code for these API servers is in our GitHub repo and is available for anyone to see and contribute to.

The data products for the jobs and recalls code are also open and available for anyone to consume on our Developer hub.

These three servers and their underlying data now operate outside of our core search codebase.

Following this same model, moving forward, we plan to:

  • Share first—For every new feature, we’ll write the code so that anyone can make use of the code, not just us. If the public community contributes to the codebase, we’ll be able to improve this feature without taxing our developers.
  • Expose APIs—We’ll expose our data products as APIs so that anyone can make use of the data, not just searchers on a government websites.
  • Be our own customer—We’ll use our own public code and data just like everyone else. We’ll call our own API servers to integrate the data within our search results pages.

Make Things Open to Make Things Better

We agree with Gov.UK that “to make things open makes things better.”

We have finite resources and we don’t want to lose our focus on serving our agency customers and improving visitors’ search experience on government websites. So, we won’t be spending a lot of time to build or support a vibrant community around our code.

That said, we hope that exposing the pieces of our system will be useful to someone somewhere. We’ll continue to provide the “ingredients” of our search service so that others will be able to make use of the code and data in ways that we could never imagine.

And, We’re Not Alone

We’re not alone. Other federal agencies have embraced the approach of coding in the open and have GitHub repos. Below are a just a few of our many favorites.


Page last reviewed or updated: