Posts tagged api

Resources for Developers

In line with the White House’s Digital Government Strategy, we’re making our code and data more open.

APIs for Search.gov Customers

These APIs are available for use on official government websites only. You must be a Search.gov customer. Sign in is required.

i14y—This API allows you to send content directly from your content management system (CMS) into Search.gov for real-time indexing. Instructions can be found under Admin Center > YourSite > Content > i14y. Request your site be activated for i14y to enable this section.

Search Results API—This API exposes all relevant results “modules” in a single JSON call, including: web results, best bets, health topics, job openings, recent tweets, recent news, recent videos, Federal Register documents, and related searches. Use it to pull search results from our service to display on your agency’s website or mobile applications. Instructions can be found under Admin Center > YourSite > Activate Search.

Type-ahead API—This API exposes the type-ahead suggestions that often appear below your search box as searchers enter their search terms. Instructions can be found under Admin Center > YourSite > Activate Search.

APIs and Data Feeds for the Public

Jobs API—Taps into a list of current jobs openings in federal agencies. Jobs are searchable by keyword, location, agency, schedule, or any combination of these.

Visit USA.gov/Developer for a full list of USA.gov’s APIs and data feeds available to the public.

Source Code (Public Github Repositories)

ASIS (Advanced Social Image Search)—The source code that runs our image search. ASIS indexes images from Flickr, Instagram, and media RSS to provide a combined search API.

i14y—The source code that runs our search engine for agencies’ published content. i14y indexes agencies’ published content in real time, for search through our regular search channels.

Jobs—The source code that runs our jobs search and Jobs API. Indexes current jobs openings in federal agencies.

Non-.gov URLs—The source code that runs our non-.gov URLs API. Indexes all government URLs that don’t end in .gov or .mil.

Punchcard—The repository of synonyms, protected words, stop words, localizations, and other vocabularies that we use to improve the precision, recall, and usability of search results.

search.digitalgov.gov—Pages and layout for our website, https://search.gov.

Site Links—The source code that “decorates” organic web results to provide additional, value-added links to help searchers find what they’re looking for. Currently provides one-click links to EDGAR filings for relevant SEC.gov results. Also published as a Ruby gem at https://rubygems.org/gems/sitelink_generator.

Unique Child Attribute—activerecord-validate_unique_child_attribute is an ActiveRecord extension to enforce uniqueness validations when accepting nested attributes. Works around Rails issue #4568.

CMS Modules and Plugins

These modules and plugins were developed by Search.gov customers, not us, but we’re linking to them here so you have easy access to them. Use their respective platforms to connect with their developers and submit issues.

Drupal Module—Allows you to connect your Drupal website to your Search.gov search configuration. The module also supports realtime indexing via Search.gov’s i14y content indexing API. Check out our help docs here.

WordPress Plugin—Starter code - contributions welcome! This plugin allows you to override the default WordPress search and connect your WordPress-powered website to your Search.gov search configuration.

i14y Content Indexing API Documentation

Search.gov Home > APIs for Developers > i14y API

Important Note: April, 2018. i14y does not visit your content to do full-text scraping of your content. For new implementations, the Search.gov team recommends you index your content with us not using the i14y API, but rather by publishing a comprehensive xml sitemap, which we can use to index your content. Read more.

i14y Github repo (External link)

What is i14y?

The Search.gov i14y Content Indexing API allows you to send content directly from your content management system (CMS) into Search.gov for real-time indexing. By hooking into your CMS workflow, you can immediately create, update, and delete the associated documents in our search indexes via this API. Even if you don’t have a CMS, you can develop software to publish your content in a way that meets the i14y API specifications below.

Ruby/Python/Node clients are welcome.

Before you Begin

  1. Set-up a search site with our service via Search.gov
  2. Request that your search site be enabled for i14y by emailing our team.
  3. Set up an i14y “drawer” in our system. Full instructions are available in our help manual. You’ll give your drawer a handle, and you’ll need to enter this drawer handle in the username field below after you click the Authorize button.
  4. Get your i14y secret token. You’ll need to enter this token as your password. The token can be found in our system in: Search Admin Center > Manage Content > i14y Drawers, select the option Show.

Note: If you are a developer who doesn’t have access to our site configurations, please email us.

Getting Started with i14y

Search.gov Home > APIs for Developers > i14y API

Important Note: April, 2018 - For new implementations, the Search.gov team recommends you index your content with us not using the i14y API, but rather by publishing a comprehensive xml sitemap, which we can use to index your content. Read more.

i14y Github repo (External link)

Technical Documentation

What is i14y?

i14y is a content indexing API that allows you to send your content directly to our indexes. When your site is enabled for i14y, it gives you complete control over what searchers see. No commercial web results will be served: instead, searchers will see content exactly as you’ve sent it to us.

How Do I Use i14y?

We currently have a Drupal module (External link) that will hook your CMS into i14y. You can also check out our help docs for working with the Drupal module.

If you use a different CMS, or don’t have one at all, we recommend you focus on publishing a comprehensive xml sitemap, which we would leverage to index your content.

Important Note: i14y does not visit your content to do full-text scraping of your content. For new implementations, the Search.gov team recommends you index your content with us not using the i14y API, but rather by publishing a comprehensive xml sitemap, which we can use to index your content. Read more.

Checklist to Go Live with i14y

Step 1. Connect one of your sites to i14y

i14y needs to be enabled in the back end for a site to be able to receive content through it. You can

  • Add a brand new search site,
  • Use the Clone Site tool to copy one of your existing site’s settings to a new site (YourSite > Dashboard > Clone Site), or
  • use an existing site.

Email us and let us know which site you want enabled for i14y. Please note: once we turn on i14y for a site, we no longer serve search results from a commercial web index. Therefore, we recommend that you set up a test site for i14y, so that your customers will continue to get results until you are fully set up and ready to move your i14y index to your production search site.

Step 2. Add a Drawer

An i14y Drawer is an index receiving content via the i14y API. We’ve called them Drawers because, like drawers in a filing cabinet, multiple indexes can be included in a single configuration to scope the entirety of a site’s search.

After we have enabled i14y for your search site, a new page will appear: Admin Center > YourSite > Manage Content > i14y Drawers.

Click Add i14y Drawer in the upper right corner. Create a handle for your drawer - the handle must be all lowercase alphanumeric, all one word, and can include underscores but no other special characters (e.g., agency_drawer_handle2).

The Drawer Description is optional.

If you have more than one website or domain that will be sending content, add a separate Drawer for each of these sites. We also recommend setting up separate drawers for staging or test content, so you can easily remove that content from your search site when you are ready to go live.

Step 3. Fill your Drawer

After you have created your Drawer(s), click Show from the i14y Drawers list to find that Drawer’s secret API key. Use this secret key with your drawer handle in your API call or your CMS module to send your content to the right place.

You are now ready to add content to your drawer. You can do this in two ways:

  • Use i14y directly. View our Github repo (External link) or Technical Documentation for more information.
  • Use a module plugged into your CMS. At this time there is a Drupal module (External link) available. Help docs for the module are here.
  • Important Note: i14y does not visit your content to do full-text scraping of your content. For new implementations, the Search.gov team recommends you index your content with us not using the i14y API, but rather by publishing a comprehensive xml sitemap, which we can use to index your content. Read more.

Step 4. Review your index

You can view the number of documents indexed for each drawer on the main i14y Drawers list, and when the most recent document was received. Click Show to view documents within a particular drawer. We display the most recent 1,000 items that were sent to the drawer. You can also search for keywords in the documents’ text, titles, and descriptions (Note: you cannot currently search for URLs).

Note: We use the Domains section to scope search results - if the domain(s) of your i14y content are not listed in the Domains section, that content will not appear on your search results page.

We send success and / or failure codes in response to your API call, so if the number of documents in our index doesn’t match what you sent, check those response codes.

If you experience difficulty sending documents to i14y, it is possible your firewall is not letting you communicate with the i14y server. Check out our cURL test commands or view the full i14y documentation.

We can attach each i14y drawer to multiple search configurations: if you have a drawer that you’d like to use for multiple search sites, email us.

Caution: A pop-up message will appear when you hit Remove on a drawer: please review this pop-up message carefully. If you remove a drawer that is only associated with one search configuration, the drawer and its contents will be deleted from our system. If the drawer is attached to multiple search configurations, it will only be removed from the search configuration you are currently on. The pop-up message will indicate what type of drawer you have.

If you accidentally delete a drawer, you will need to set-up a new drawer and resend the content - we are unable to retrieve deleted drawers. If you accidentally remove a shared drawer but it is still associated with other search configurations, we can re-attach it to your site. Contact us for assistance.

Once you have your index populated, you will set up the rest of your search as you would for a traditional Search.gov site:

If you have any RSS content that will not be sent to your i14y drawer, you can add those feeds as well.

Update your website’s search box form code to point to affiliate=youri14yenabledsitehandle.

Terms of Use

By accessing the i14y API, you agree to USA.gov’s Terms of Service for Developer Resources.


Did you know? i14y is hacker shorthand for “interoperability”, because there are 14 characters between the first and last letters. i14y can also be shorthand for Independence Day.

cURL Commands for i14y Testing

Important Note: April, 2018. i14y does not visit your content to do full-text scraping of your content. For new implementations, the Search.gov team recommends you index your content with us not using the i14y API, but rather by publishing a comprehensive xml sitemap, which we can use to index your content. Read more.

If you experience difficulty sending documents to our index via i14y, it is possible your firewall is not letting you communicate with the i14y server. Adding a test document to your i14y drawer can help you diagnose a firewall issue. Windows users may need to install cURL (External link) in order to run this test from the command line.

Adding the Test Document

From the command line, enter:

curl "https://i14y.usa.gov/api/v1/documents" -XPOST -H "Content-Type:application/json" -u your_drawer_handle:your_secret_token -d '{"document_id":"1", "title":"Test Document", "path": "http://www.gov.gov/cms/doc1.html", "created": "2015-05-12T22:35:09Z", "description":"The summary of the document goes here.", "content":"This is placeholder text, and in a real document would be paragraphs long.", "promote": false, "language" : "en", "tags" : "tag1, another tag"}'

Note: you need to replace your_drawer_handle with your i14y drawer handle, found in the i14y drawers section in the Search Admin Center (Search.gov Home > Admin Center > YourSite > Manage Content > i14y Drawers). You will also need to replace your_secret_token with the drawer’s secret token, which can be found by hitting “Show” in the 14y drawer list. The drawer handle and token should be separated by a colon (:) with no spaces on either side of the colon.

The above command returns JSON structured like this:

{
"status":200,
"developer_message":"OK",
"user_message":"Your document was successfully created."
}

If you do not see a 200 status, contact our team.

After successfully sending a document, you should see an increase (by 1) in the number of documents in your i14y drawer.

Removing the Test Document

Once you have successfully added the test document to your drawer, you will need to delete it, or it will appear in your site’s search results.

From the command line, enter:

curl "https://i14y.usa.gov/api/v1/documents/1" -XDELETE -u your_drawer_handle:your_secret_token

The above command returns JSON structured like this:

{
"status":200,
"developer_message":"OK",
"user_message":"Your document was successfully deleted."
}

Resources: Read tips on Getting Started With i14y, or view the full i14y technical documentation.

Showing Results Within Your Agency's Website

We offer an API that allows agencies to display results from our service on their website. Instead of searchers going to our hosted search results page, agencies can use the “Search Results API” to pull results from our service and display the results within their own pages. Agencies that want to maintain an exact match of their headers, footers, and other branding are encouraged to use this API.

The API exposes all relevant results “modules” in a single JSON call, including:

Note about Web Results: To retrieve web results through this API, you must either use our indexes (populated via the i14y API or via RSS), or you must purchase an API key from Bing.

This API is available for use on official government websites only. You must be a Search.gov customer to access the instructions. Sign in is required.

Instructions can be found under Admin Center > YourSite > Activate Search > Search Results API Instructions. The Search Results API key can be found under Admin Center > YourSite > Activate Search > API Access Key.

Displaying Type-ahead Suggestions on Your Agency's Website

Note: this information is for customers who wish to display type-ahead (autocomplete) suggestions on their agency’s website. If you are only looking to turn on type-ahead suggestions on your search results page, please review the instructions on our Display Overview page.

There are two ways to get type-ahead suggestions to appear on your agency’s website.

JavaScript Snippet

We offer a JavaScript snippet that is placed on your pages before the closing </body> tag.

You must be a Search.gov customer to use this feature. Full instructions and code can be found under Admin Center > YourSite > Activate Search > Code Snippets.

The code will look similar to:

<script type="text/javascript">
//<![CDATA[
      var usasearch_config = { siteHandle:"YourSiteHandle" };

      var script = document.createElement("script");
      script.type = "text/javascript";
      script.src = "//search.usa.gov/javascripts/remote.loader.js";
      document.getElementsByTagName("head")[0].appendChild(script);

//]]>
</script>

Type-ahead API

The Type-ahead API exposes your site’s type-ahead suggestions. Calling the API directly alllows you to show type-ahead suggestions without adding the JavaScript code.

This API is available for use on official government websites only. You must be a Search.gov customer to access the instructions. Sign in is required.

Keys and instructions can be found under Admin Center > YourSite > Activate Search > Type-ahead API Instructions.


Troubleshooting: Suggestions are derived from the searches performed on your website. For additional details, please review the Type-ahead API section in the Search Admin Center.

Jobs API

Search.gov Home > APIs for Developers > Jobs API

Helping job seekers land a job with the government.

Overview

Federal, state, and local government agencies are hiring and have thousands of job openings across the country, posted on USAJobs.gov and local government jobs sites.

This Jobs API allows you to tap into a list of current jobs openings with the government. Jobs are searchable by keyword, location, agency, schedule, or any combination of these.

This API is great for job websites or applications, job banks, and career counseling and placement applications.

Using the API

The current default version of the API is v3. To explicitly specify an older version of the API, see the Jobs API code documentation.

This Jobs API accepts a GET request.

Jobs are searchable by keyword, location, agency, schedule, or any combination of these.

The base URL is https://jobs.search.gov/jobs/search.json?parameters

Sample API requests follow.

All nursing jobs https://jobs.search.gov/jobs/search.json?query=nursing+jobs

Nursing jobs in NY https://jobs.search.gov/jobs/search.json?query=nursing+jobs+in+ny

Nursing jobs with a state government agency https://jobs.search.gov/jobs/search.json?query=nursing+jobs&tags=state

Nursing jobs with a federal government agency https://jobs.search.gov/jobs/search.json?query=nursing+jobs&tags=federal

Nursing jobs with Veterans Affairs in NY https://jobs.search.gov/jobs/search.json?query=nursing+jobs+with+veterans+affairs+in+ny

Part-time nursing jobs with Veterans Affairs in NY https://jobs.search.gov/jobs/search.json?query=parttime+nursing+jobs+with+veterans+affairs+in+ny

Parameters

Seven parameters are accepted.

  1. query
  2. organization_ids (comma-separated)
  3. hl (for highlighting)
  4. size
  5. from
  6. tags (comma-separated)
  7. lat_lon

Query

Attempts to extract as much “signal” as possible from the input text. Handles word variants, so a search on “nursing jobs” will find a job titled “nurse practitioner” and “RN.” When parts of the query parameter are used to search against the position title, the results are ordered by relevance. When no query parameter is specified, they are ordered by date with the most recent listed first. For generic jobs searches (e.g., ‘jobs’, ‘internships’) where the lat_lon parameter is passed in, the results will be sorted by distance to the searcher’s geographic location.

Sample query formats follow.

All jobs (by date or by proximity)

  • jobs
  • positions
  • vacancies
  • opportunities
  • postings

Location-based (by date)

  • jobs in fulton
  • jobs in fulton, md
  • jobs in fulton, maryland
  • jobs in md
  • pentagon jobs
  • jobs fayetteville
  • fayetteville nc jobs
  • jobs atlanta georgia

Agency-based (by date)

  • job opportunities at the cia
  • jobs at the treasury dept
  • tsa job openings
  • va jobs # will match jobs in Virginia

Keyword-based (by relevance)

  • nursing jobs
  • summer internship position

Schedule-based (by date)

  • part-time jobs
  • full-time positions

Combinations (by relevance)

  • part-time security guard job openings at the tsa in los angeles, ca

Organization_ids

A comma-separated string specifying which federal, state, or local agencies to use as a filter.

For federal agencies, the ID is based on USAJobs’ agency schema. Two letter codes are used to span entire departments, while four letter codes are generally used for independent agencies or agencies within a department.

For state and local agencies, a sample of the format follows.

State of Virginia US-VA

State of Virginia Department of Taxation US-VA:DEPT-TAX

Fairfax County, VA US-VA:COUNTY-FAIRFAX

Fairfax County Sheriff US-VA:COUNTY-FAIRFAX:SHERIFF

City of Fairfax, VA US-VA:COUNTY-FAIRFAX:CITY-FAIRFAX

Air Force or State of Virginia AF,US-VA

If you specify organization_ids in your API call but the searcher types in another agency, the searchers’ agency overrides the organization_ids parameter. For example, VA jobs are returned when you specify Air Force jobs, but the searcher looks for jobs at the VA (https://jobs.search.gov/jobs/search.json?query=jobs+at+the+va&organization_ids=AF).

Hl [highlighting]

No highlighting is included by default. Use ‘hl=1’ to highlight terms in the position title that match terms in the user’s search. Highlighted terms are surrounded with <em> tags.

Size

Specifies how many results are returned (up to 100 at a time).

From

Specifies the starting record.

Tags

A comma-separated string specifying the level of government. Current tags are federal, state, county, and city. To get federal and state listings, you would pass in federal,state for the tags parameter.

Lat_lon

Comma-separated pair denoting the position of the searcher looking for a job. For example, ‘lat_lon=37.783333,-122.416667’ is the value for San Francisco, CA. For generic job searches (e.g., ‘jobs’, ‘internships’) where this parameter is passed in, the Jobs API server will sort jobs results and return those that are closest to the searcher’s location. For jobs that have multiple locations associated with the position, the distance to the closest location is used to determine the overall geo-distance for the job. Note that for keyword-based job searches (e.g., ‘jobs in NY’, ‘nursing jobs’), this parameter has no effect and is ignored.

Data Sources

We’ve tailored this Jobs API for our use in our search results. Data are normalized across government job listings and we don’t include all jobs from all data sources. So, we encourage you to use the original data sources directly.

Federal government jobs are from USAJobs. The USAJobs API for job opportunity announcements is at https://data.usajobs.gov/Rest.

State and local government jobs are from cities, counties, and state agencies that use NeoGov (External link), such as jobs with the the city of Boulder, Colorado.

Are we missing your jobs data? Email us at search@support.digitalgov.gov to let us know where you publish the jobs data for your federal, state, or local government agency. We’ll accept your jobs data via FTP or any machine-readable format (such as an RSS feed or API).

What it Returns

Below is a sample JSON response for https://jobs.search.gov/jobs/search.json?query=nursing+jobs+with+veterans+affairs+in+albany+ny.

[
  {
    id: "usajobs:340537800",
	position_title: "Nursing Assistant Float Pool",
	organization_name: "Veterans Affairs, Veterans Health Administration",
	rate_interval_code: "PA",
	minimum: 31315,
	maximum: 40706,
	start_date: "2013-03-27",
	end_date: "2013-05-31",
	locations: [
		"Albany, NY"
	],
	url: "https://www.usajobs.gov/GetJob/ViewDetails/340537800"
  },
  {
    id: "usajobs:342782900",
	position_title: "Registered Nurse - Endoscopy",
	organization_name: "Veterans Affairs, Veterans Health Administration",
	rate_interval_code: "PA",
	minimum: 0,
	maximum: 0,
	start_date: "2013-05-02",
	end_date: "2013-05-13",
	locations: [
		"Albany, NY"
	],
	url: "https://www.usajobs.gov/GetJob/ViewDetails/342782900"
  }
]

Code Samples

We “eat our own dog food” and use this Jobs API to display the job openings on the government websites that use Search.gov.

See the sample results page below that shows current jobs openings across the Department of Homeland Security for a search on jobs on DHS.gov.

Open Source Code

The server code that runs this API is on Github. If you’re a Rails developer, fork our Jobs Server API repo to add features (such as additional datasets) or fix bugs.

Better yet, send us a pull request to contribute your changes to our project and improve this API for everyone who is using it.

Feedback

Email us at search@support.digitalgov.gov with any suggestions about what you’d like to see in the next version of API.

Terms of Use

By accessing the Jobs API, you agree to USA.gov’s Terms of Service for Developer Resources.

Our Open Source Strategy

We keep an eye on on our what our government counterparts are up to, both in the U.S. and other countries. We first came across Gov.UK’s philosophy on and approach to coding in the open a couple of years ago. It caught our attention and we realized we should also articulate our open source strategy.

Use and Contribute to Open Source Projects

Since 2010, we’ve embraced and leveraged open source software to build our site search service for federal, state, and local government websites. This use of open source has allowed us to experience enormous growth over the past few years. In July 2014 alone, over 23 million searchers received results from our service—a five-fold increase since July 2010.

Our search service is now a complex system made up of many moving parts, including providing type-ahead search suggestions, serving search results, fetching, indexing, and caching content, and providing analytics.

Each of these parts is compiled into our codebase and, as we use open source components for our system, we contribute back to the projects.

Code in the Open

We recently began to unravel our monolithic codebase so that we can share individual pieces of our code. To borrow the phrase from Gov.UK, we’re now coding in the open.

We recently released the code for our social media image, jobs and recalls API servers. They’re our first foray into coding in the open. The source code for these API servers is in our GitHub repo and is available for anyone to see and contribute to.

The data products for the jobs and recalls code are also open and available for anyone to consume on our Developer hub.

These three servers and their underlying data now operate outside of our core search codebase.

Following this same model, moving forward, we plan to:

  • Share first—For every new feature, we’ll write the code so that anyone can make use of the code, not just us. If the public community contributes to the codebase, we’ll be able to improve this feature without taxing our developers.
  • Expose APIs—We’ll expose our data products as APIs so that anyone can make use of the data, not just searchers on a government websites.
  • Be our own customer—We’ll use our own public code and data just like everyone else. We’ll call our own API servers to integrate the data within our search results pages.

Make Things Open to Make Things Better

We agree with Gov.UK that “to make things open makes things better.”

We have finite resources and we don’t want to lose our focus on serving our agency customers and improving visitors’ search experience on government websites. So, we won’t be spending a lot of time to build or support a vibrant community around our code.

That said, we hope that exposing the pieces of our system will be useful to someone somewhere. We’ll continue to provide the “ingredients” of our search service so that others will be able to make use of the code and data in ways that we could never imagine.

And, We’re Not Alone

We’re not alone. Other federal agencies have embraced the approach of coding in the open and have GitHub repos. Below are a just a few of our many favorites.


Page last reviewed or updated: