Posts tagged api

Resources for Developers

In line with the White House’s Digital Government Strategy, we’re making our code and data more open.

APIs for Search.gov Customers

These APIs are available for use on official government websites only. You must be a Search.gov customer. Sign in is required.

Search Results API—This API exposes all relevant results “modules” in a single JSON call, including: web results, best bets, health topics, job openings, recent tweets, recent news, recent videos, Federal Register documents, and related searches. Use it to pull search results from our service to display on your agency’s website or mobile applications. Instructions can be found under Admin Center > YourSite > Activate Search.

Type-ahead API—This API exposes the type-ahead suggestions that often appear below your search box as searchers enter their search terms. Instructions can be found under Admin Center > YourSite > Activate Search.

i14y—This API is still available, but we encourage the use of sitemap-based indexing instead. Read more about our indexing process here. The i14y API allows you to send content directly from your content management system (CMS) into Search.gov for real-time indexing.

APIs and Data Feeds for the Public

USAJOBS API—Taps into a list of current job openings in federal agencies. The Search.gov Jobs API has been deprecated.

USA.gov/Developer—Full list of USA.gov’s APIs and data feeds available to the public.

api.data.gov—Portal for access to APIs from across the federal government.

Source Code (Public Github Repositories)

ASIS (Advanced Social Image Search)—The source code that runs our image search. ASIS indexes images from Flickr, Instagram, and media RSS to provide a combined search API.

i14y—The source code that runs our search engine for agencies’ published content. i14y indexes agencies’ published content in real time, for search through our regular search channels.

Non-.gov URLs—The source code that runs our non-.gov URLs API. Indexes all government URLs that don’t end in .gov or .mil.

Punchcard—The repository of synonyms, protected words, stop words, localizations, and other vocabularies that we use to improve the precision, recall, and usability of search results.

search.digitalgov.gov—Pages and layout for our website, https://search.gov.

Site Links—The source code that “decorates” organic web results to provide additional, value-added links to help searchers find what they’re looking for. Currently provides one-click links to EDGAR filings for relevant SEC.gov results. Also published as a Ruby gem at https://rubygems.org/gems/sitelink_generator.

Unique Child Attribute—activerecord-validate_unique_child_attribute is an ActiveRecord extension to enforce uniqueness validations when accepting nested attributes. Works around Rails issue #4568.

CMS Modules and Plugins

These modules and plugins were developed by Search.gov customers, not us, but we’re linking to them here so you have easy access to them. Use their respective platforms to connect with their developers and submit issues.

Drupal Module—Allows you to connect your Drupal website to your Search.gov search configuration. The module also supports realtime indexing via Search.gov’s i14y content indexing API. Check out our help docs here.

WordPress Plugin—Starter code - contributions welcome! This plugin allows you to override the default WordPress search and connect your WordPress-powered website to your Search.gov search configuration.

api open

i14y Content Indexing API Documentation

Search.gov Home > APIs for Developers > i14y API

Important Note: April, 2018. i14y does not visit your content to do full-text scraping of your content. For new implementations, the Search.gov team recommends you index your content with us not using the i14y API, but rather by publishing a comprehensive xml sitemap, which we can use to index your content. Read more.

i14y Github repo (External link)

What is i14y?

The Search.gov i14y Content Indexing API allows you to send content directly from your content management system (CMS) into Search.gov for real-time indexing. By hooking into your CMS workflow, you can immediately create, update, and delete the associated documents in our search indexes via this API. Even if you don’t have a CMS, you can develop software to publish your content in a way that meets the i14y API specifications below.

Ruby/Python/Node clients are welcome.

Before you Begin

  1. Set-up a search site with our service via Search.gov
  2. Request that your search site be enabled for i14y by emailing our team.
  3. Set up an i14y “drawer” in our system. Full instructions are available in our help manual. You’ll give your drawer a handle, and you’ll need to enter this drawer handle in the username field below after you click the Authorize button.
  4. Get your i14y secret token. You’ll need to enter this token as your password. The token can be found in our system in: Search Admin Center > Manage Content > i14y Drawers, select the option Show.

Note: If you are a developer who doesn’t have access to our site configurations, please email us.

i14y api

Getting Started with i14y

Search.gov Home > APIs for Developers > i14y API

Important Note: April, 2018 - For new implementations, the Search.gov team recommends you index your content with us not using the i14y API, but rather by publishing a comprehensive xml sitemap, which we can use to index your content. Read more.

i14y Github repo (External link)

Technical Documentation

What is i14y?

i14y is a content indexing API that allows you to send your content directly to our indexes. When your site is enabled for i14y, it gives you complete control over what searchers see. No commercial web results will be served: instead, searchers will see content exactly as you’ve sent it to us.

How Do I Use i14y?

We currently have a Drupal module (External link) that will hook your CMS into i14y. You can also check out our help docs for working with the Drupal module.

If you use a different CMS, or don’t have one at all, we recommend you focus on publishing a comprehensive xml sitemap, which we would leverage to index your content.

Important Note: i14y does not visit your content to do full-text scraping of your content. For new implementations, the Search.gov team recommends you index your content with us not using the i14y API, but rather by publishing a comprehensive xml sitemap, which we can use to index your content. Read more.

Checklist to Go Live with i14y

Step 1. Connect one of your sites to i14y

i14y needs to be enabled in the back end for a site to be able to receive content through it. You can

  • Add a brand new search site,
  • Use the Clone Site tool to copy one of your existing site’s settings to a new site (YourSite > Dashboard > Clone Site), or
  • use an existing site.

Email us and let us know which site you want enabled for i14y. Please note: once we turn on i14y for a site, we no longer serve search results from a commercial web index. Therefore, we recommend that you set up a test site for i14y, so that your customers will continue to get results until you are fully set up and ready to move your i14y index to your production search site.

Step 2. Add a Drawer

An i14y Drawer is an index receiving content via the i14y API. We’ve called them Drawers because, like drawers in a filing cabinet, multiple indexes can be included in a single configuration to scope the entirety of a site’s search.

After we have enabled i14y for your search site, a new page will appear: Admin Center > YourSite > Manage Content > i14y Drawers.

Click Add i14y Drawer in the upper right corner. Create a handle for your drawer - the handle must be all lowercase alphanumeric, all one word, and can include underscores but no other special characters (e.g., agency_drawer_handle2).

The Drawer Description is optional.

If you have more than one website or domain that will be sending content, add a separate Drawer for each of these sites. We also recommend setting up separate drawers for staging or test content, so you can easily remove that content from your search site when you are ready to go live.

Step 3. Fill your Drawer

After you have created your Drawer(s), click Show from the i14y Drawers list to find that Drawer’s secret API key. Use this secret key with your drawer handle in your API call or your CMS module to send your content to the right place.

You are now ready to add content to your drawer. You can do this in two ways:

  • Use i14y directly. View our Github repo (External link) or Technical Documentation for more information.
  • Use a module plugged into your CMS. At this time there is a Drupal module (External link) available. Help docs for the module are here.
  • Important Note: i14y does not visit your content to do full-text scraping of your content. For new implementations, the Search.gov team recommends you index your content with us not using the i14y API, but rather by publishing a comprehensive xml sitemap, which we can use to index your content. Read more.

Step 4. Review your index

You can view the number of documents indexed for each drawer on the main i14y Drawers list, and when the most recent document was received. Click Show to view documents within a particular drawer. We display the most recent 1,000 items that were sent to the drawer. You can also search for keywords in the documents’ text, titles, and descriptions (Note: you cannot currently search for URLs).

Note: We use the Domains section to scope search results - if the domain(s) of your i14y content are not listed in the Domains section, that content will not appear on your search results page.

We send success and / or failure codes in response to your API call, so if the number of documents in our index doesn’t match what you sent, check those response codes.

If you experience difficulty sending documents to i14y, it is possible your firewall is not letting you communicate with the i14y server. Check out our cURL test commands or view the full i14y documentation.

We can attach each i14y drawer to multiple search configurations: if you have a drawer that you’d like to use for multiple search sites, email us.

Caution: A pop-up message will appear when you hit Remove on a drawer: please review this pop-up message carefully. If you remove a drawer that is only associated with one search configuration, the drawer and its contents will be deleted from our system. If the drawer is attached to multiple search configurations, it will only be removed from the search configuration you are currently on. The pop-up message will indicate what type of drawer you have.

If you accidentally delete a drawer, you will need to set-up a new drawer and resend the content - we are unable to retrieve deleted drawers. If you accidentally remove a shared drawer but it is still associated with other search configurations, we can re-attach it to your site. Contact us for assistance.

Once you have your index populated, you will set up the rest of your search as you would for a traditional Search.gov site:

If you have any RSS content that will not be sent to your i14y drawer, you can add those feeds as well.

Update your website’s search box form code to point to affiliate=youri14yenabledsitehandle.

Terms of Use

By accessing the i14y API, you agree to USA.gov’s Terms of Service for Developer Resources.

Did you know? i14y is hacker shorthand for “interoperability”, because there are 14 characters between the first and last letters. i14y can also be shorthand for Independence Day.

i14y api manage-content

cURL Commands for i14y Testing

Important Note: April, 2018. i14y does not visit your content to do full-text scraping of your content. For new implementations, the Search.gov team recommends you index your content with us not using the i14y API, but rather by publishing a comprehensive xml sitemap, which we can use to index your content. Read more.

If you experience difficulty sending documents to our index via i14y, it is possible your firewall is not letting you communicate with the i14y server. Adding a test document to your i14y drawer can help you diagnose a firewall issue. Windows users may need to install cURL (External link) in order to run this test from the command line.

Adding the Test Document

From the command line, enter:

curl "https://i14y.usa.gov/api/v1/documents" -XPOST -H "Content-Type:application/json" -u your_drawer_handle:your_secret_token -d '{"document_id":"1", "title":"Test Document", "path": "http://www.gov.gov/cms/doc1.html", "created": "2015-05-12T22:35:09Z", "description":"The summary of the document goes here.", "content":"This is placeholder text, and in a real document would be paragraphs long.", "promote": false, "language" : "en", "tags" : "tag1, another tag"}'

Note: you need to replace your_drawer_handle with your i14y drawer handle, found in the i14y drawers section in the Search Admin Center (Search.gov Home > Admin Center > YourSite > Manage Content > i14y Drawers). You will also need to replace your_secret_token with the drawer’s secret token, which can be found by hitting “Show” in the 14y drawer list. The drawer handle and token should be separated by a colon (:) with no spaces on either side of the colon.

The above command returns JSON structured like this:

{
"status":200,
"developer_message":"OK",
"user_message":"Your document was successfully created."
}

If you do not see a 200 status, contact our team.

After successfully sending a document, you should see an increase (by 1) in the number of documents in your i14y drawer.

Removing the Test Document

Once you have successfully added the test document to your drawer, you will need to delete it, or it will appear in your site’s search results.

From the command line, enter:

curl "https://i14y.usa.gov/api/v1/documents/1" -XDELETE -u your_drawer_handle:your_secret_token

The above command returns JSON structured like this:

{
"status":200,
"developer_message":"OK",
"user_message":"Your document was successfully deleted."
}

Resources: Read tips on Getting Started With i14y, or view the full i14y technical documentation.

i14y api manage-content

Displaying Type-ahead Suggestions on Your Agency's Website

Note: this information is for customers who wish to display type-ahead (autocomplete) suggestions on their agency’s website. If you are only looking to turn on type-ahead suggestions on your search results page, please review the instructions on our Display Overview page.

There are two ways to get type-ahead suggestions to appear on your agency’s website.

JavaScript Snippet

We offer a JavaScript snippet that is placed on your pages before the closing </body> tag.

You must be a Search.gov customer to use this feature. Full instructions and code can be found under Admin Center > YourSite > Activate Search > Code Snippets.

The code will look similar to:

<script type="text/javascript">
//<![CDATA[
      var usasearch_config = { siteHandle:"YourSiteHandle" };

      var script = document.createElement("script");
      script.type = "text/javascript";
      script.src = "//search.usa.gov/javascripts/remote.loader.js";
      document.getElementsByTagName("head")[0].appendChild(script);

//]]>
</script>

Type-ahead API

The Type-ahead API exposes your site’s type-ahead suggestions. Calling the API directly alllows you to show type-ahead suggestions without adding the JavaScript code.

This API is available for use on official government websites only. You must be a Search.gov customer to access the instructions. Sign in is required.

Keys and instructions can be found under Admin Center > YourSite > Activate Search > Type-ahead API Instructions.

Troubleshooting: Suggestions are derived from the searches performed on your website. For additional details, please review the Type-ahead API section in the Search Admin Center.

api activate-search autocomplete

Our Open Source Strategy

We keep an eye on on our what our government counterparts are up to, both in the U.S. and other countries. We first came across Gov.UK’s philosophy on and approach to coding in the open a couple of years ago. It caught our attention and we realized we should also articulate our open source strategy.

Use and Contribute to Open Source Projects

Since 2010, we’ve embraced and leveraged open source software to build our site search service for federal, state, and local government websites. This use of open source has allowed us to experience enormous growth over the past few years. In July 2014 alone, over 23 million searchers received results from our service—a five-fold increase since July 2010.

Our search service is now a complex system made up of many moving parts, including providing type-ahead search suggestions, serving search results, fetching, indexing, and caching content, and providing analytics.

Each of these parts is compiled into our codebase and, as we use open source components for our system, we contribute back to the projects.

Code in the Open

We recently began to unravel our monolithic codebase so that we can share individual pieces of our code. To borrow the phrase from Gov.UK, we’re now coding in the open.

We recently released the code for our social media image, jobs and recalls API servers. They’re our first foray into coding in the open. The source code for these API servers is in our GitHub repo and is available for anyone to see and contribute to.

The data products for the jobs and recalls code are also open and available for anyone to consume on our Developer hub.

These three servers and their underlying data now operate outside of our core search codebase.

Following this same model, moving forward, we plan to:

  • Share first—For every new feature, we’ll write the code so that anyone can make use of the code, not just us. If the public community contributes to the codebase, we’ll be able to improve this feature without taxing our developers.
  • Expose APIs—We’ll expose our data products as APIs so that anyone can make use of the data, not just searchers on a government websites.
  • Be our own customer—We’ll use our own public code and data just like everyone else. We’ll call our own API servers to integrate the data within our search results pages.

Make Things Open to Make Things Better

We agree with Gov.UK that “to make things open makes things better.”

We have finite resources and we don’t want to lose our focus on serving our agency customers and improving visitors’ search experience on government websites. So, we won’t be spending a lot of time to build or support a vibrant community around our code.

That said, we hope that exposing the pieces of our system will be useful to someone somewhere. We’ll continue to provide the “ingredients” of our search service so that others will be able to make use of the code and data in ways that we could never imagine.

And, We’re Not Alone

We’re not alone. Other federal agencies have embraced the approach of coding in the open and have GitHub repos. Below are a just a few of our many favorites.

open api about-us

Page last reviewed or updated: