Skip to main content
U.S. flag

An official website of the United States government

Dot gov

Official websites use .gov
A .gov website belongs to an official government organization in the United States.


Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

April 2018 Newsletter


Two webinars coming in May - an intro session and one about how search engines work. Read tips about avoiding spider traps in your site.

Two Webinars in May

Come learn more about search engines, and how to make the best use of

May 3 (1pm-2pm ET): An Introduction to

New to our service or want a refresher? Join us for a general tour of the Admin Center and an overview of search optimization good practices. Register now

May 10 (1pm-2pm ET): How Search Engines Index Your Websites

A more advanced session looking at how search engines work, and the role of xml sitemaps and robots.txt files. If you are new, we recommend you attend the Introduction session first. Register now

Is your website a spider trap?

Sometimes dynamic link development can cause search engine spiders to get caught in an infinite loop of links inside your website. For instance, if a page has a ‘real’ URL, but can be accessed through a number of other links with different query parameters, each one of these links will look new to the spider and it will keep crawling and crawling and crawling. Allowing search engines to get around your site easily is a ranking factor. Read more

Note: links to articles are not an endorsement of the website.

XML Sitemaps and Robots.txt files: Quick Tips

  • XML Sitemaps advertise to search engines what content is on your website.

Tip: Be sure the URLs in your sitemap match the protocol of the sitemap itself and the pages on the list

Robots.txt files help search engines find your sitemaps and indicate what you do and do not want indexed.

Tip: Be sure that the sitemap for a given subdomain is listed on the robots.txt file for that subdomain.

Release Notes

Want to learn about the latest features, fixes, and focuses of the Search team? We post monthly Release Notes on our website.

Read the Latest Notes

April 01, 2018