Skip to main content
U.S. flag

An official website of the United States government

Dot gov

Official websites use .gov
A .gov website belongs to an official government organization in the United States.


Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

May 2018 Newsletter


Twitter announced changes to their API, which will affect our ability to index tweets. Also, join us for a technical SEO webinar May 29!

Changes coming to Twitter’s API

Twitter may be limiting access to their API this summer 

Many of you have listed Twitter handles in your search sites, in order to show relevant tweets in your results pages. Twitter announced they will be deprecating the API we use to index your tweets, and replace it with two APIs: a very limited API and an unlimited "enterprise" API. The new APIs have been reported as unfinished, and there is no pricing published for the enterprise API.

The deprecation of the current API was originally set for June 19, but has been postponed for an unspecified period of time.

What’s the impact on your search?

We currently index close to two thousand twitter accounts. We will investigate options, but the likely outcome is that when they deprecate their current API we will no longer show Tweets in our search results. There would be no action for you to take. If you have strong feelings about this potential change, please reach out to us and we can discuss.

Technical SEO Webinar May 30

Join us to learn how search engines index your sites

Our upcoming technical webinar on how search engines index your websites will now be held on May 29, at 1pm ET. Learn about how crawlers take in content, and how they interact with XML sitemaps and robots.txt files.

Did you miss the Intro session a couple weeks ago? Watch the recording (60 minutes, YouTube)</td>

SEO Pro Tip

Use the robots meta tag to manage* indexing of individual pages

Many websites have topic pages that list all the items they have for that particular area, such as These pages are helpful to crawlers, which will use the links on them to discover new pages to add to the search index. And while it’s great to include those new pages in the search results, the topic page itself is not very helpful as a search result.

To block indexing of these topic pages, place a noindex robots meta tag in the <head> of the page. On the topic pages, we want crawlers not to index the page, but still follow the links on it, so our robots meta tag looks like this:

`<meta name="robots" content="noindex, follow">`

Release Notes

Want to learn about the latest features, fixes, and focuses of the Search team? We post monthly Release Notes on our website.

Read the Latest Notes

May 01, 2018