Posts tagged help-manual

Adding a Custom System Alert to Your Results Page

Search.gov Home > Admin Center > YourSite > Display > Search Page Alert

There may be times when you want to add a system alert to your search results page. For instance, during a site redesign, you may wish to inform searchers that your site is in transition.

Use the Search Page Alert feature to add a text message to your search results page, which will appear at all times above all search results, regardless of the query. Enter a title and text for the alert, set the status to active, and hit Save to turn on your alert. The appearance of the alert is not customizable.

See, for example, the custom alert on Defense.gov announcing their migration to a new system.

Custom System Alert on Defense.gov


Did you know? See also our closely related feature Best Bets, which allows you to recommend individual results or sets of results, which will appear only when a searcher’s query matches the text of their titles, descriptions, or keywords.

How to Add Your Instagram Pictures to Our Index

Search.gov Home > Admin Center > YourSite > Manage Content > Instagram

ALERT: Instagram now requires accounts to grant permission to index their images via an integration between systems. Since this integration will not be possible for Search.gov in the foreseeable future, we’re exploring other options. Our Instagram index was last updated in June 2016. Any images in our index prior to that date will continue to be shown on your search results page, as long as you do not remove your Instagram account from the Admin Center. If you remove your account, any photos in our index will be permanently deleted from our system. This help manual page is for historical reference only.

Tell Us About Your Instagram Account

Provide us with the username for your Instagram Account.

When you’re logged into Instagram, you can see your username at https://instagram.com/accounts/login/?next=/accounts/edit/ (External link).

For example, the Instagram username for the Department of Labor is usdol (External link).

Opt to Display Your Instagram Pictures

When you add the username for your Instagram account, we’ll automatically index all of the pictures in your account.

On the image results page, we’ll display the pictures from your Instagram account by default. If you’d like to backfill them with the standard image results from your website, email us and we’ll turn on your web images for you.

See the sample results page below that shows image results displayed on DOL.gov for a search on minimum wage.

Image results from Flickr & Instagram


Did you know? Do you have a multimedia gallery on your website for your agency’s photos, images, videos, podcasts, or other multimedia content? Do you use a content management system, database, or media RSS (MRSS) feed to power this gallery? You can index MRSS feeds so that your multimedia content is automatically included in your search results.

Did you know? You can also tell us about your Flickr photostream. Note that searchers see interspersed results from both Flickr and Instagram. If you have a lot of duplicate images in the two services, consider listing only one in the Admin Center.

Troubleshooting tip: Flickr and Instagram results appear on the newly redesigned results page only. Email us at search@support.digitalgov.gov if you’re ready to turn on the new results page.

Analyzing Your Monthly Reports

Search.gov Home > Admin Center > YourSite > Analytics > Monthly Reports

The monthly report gives a bird’s-eye view of the number of queries and clicks each month.

Data are shown for the present month-to-date by default. You can change the time period by selecting a different month or year and re-generating the report.

You can also toggle between filtered and unfiltered views of the data by clicking on the filter icon () in the top navigation bar. The filtered data represent our best effort to show you real searches performed by your site’s visitors. The unfiltered data include nearly all searches and clicks. Only known spiders (such as Bingbot and Googlebot) are excluded.

Monthly Usage Stats

Data for the total number of queries and clicks. Use this report to find and analyze the so-called short head (External link) of your most popular search queries and clicks, which are typically the same as your site’s top tasks.

Queries with No Results

A list of queries that returned no results. Sometimes, searchers look for information on the wrong website or use different words than what are used on the page, and so they get no results. Use this list of Queries with No Results to help searchers get to the information they’re looking for. You can set up Best Bets, or a Routed Query. You can also customize the No Results message displayed on your results page.

Impressions and Clicks by Module

Detailed data for the total number of impressions and clicks and your clickthru rate (CTR) are presented for each “module” on the search results page. We also provide the average CTR across all Search.gov customers so you can see how your rate compares to the average rate. The data in the table is sorted in descending order from the most to least number of clicks.

Use this report to inform which modules you opt to display.

Download Top Queries

View a detailed report of the number of times searchers have input specific terms and phrases by clicking on the CSV (comma-separated values) link to download the list as a text file. Once you’ve downloaded the CSV file, you can easily import it into Excel (External link) or another speadsheet program to analyze the data.

Read Understanding Your Users’ Needs By Analyzing Search Terms for tips on how to create a semi-automated report for analyzing the data in this CSV file on a regular basis.

Definitions

All data presented on this page (and other pages in the Admin Center) are IP-deduped to exclude bots and other noise in an attempt to accurately represent searchers’ intent on your site. Some definitions follow.

Queries: Number of times a search query (that is, a word or string of words) was entered in the search box by a unique searcher.

Clicks. Number of times a searcher clicked on one of your pages within the search results for a particular query.

Impressions: Number of times a module is displayed, whether it is clicked on or not. Not all modules are displayed for all queries. Each time a module is displayed it is counted as one impression.

Clickthru Rate (CTR): The rate (expressed in a percentage) at which searchers click on a module. This rate is calculated by dividing the total number of clicks by the total number of impressions. CTR is useful to measure the performance of specific campaigns, such as Best Bets for seasonal events.

Raw count: Total number of times the query was submitted. The raw count is provided in the downloadable CSV files only.

IP-deduped count: Total number of times the query was submitted by any one IP address. This excludes bots and other traffic that send in a query multiple times from one IP address. It is often a more accurate representation of “real,” human traffic.


Did you know? On the first of each month, we email you a report with data on the previous month’s queries, clicks, and top search terms.

Did you know? The Site Overview provides a snapshot of what has been happening on your site in the past day or so.

Editing Your Site Settings

Search.gov Home > Admin Center > YourSite > Dashboard > Settings

Display Name

Tell us the name of your website. Searchers see your site name on the results page, such as in your page title or Best Bets. We recommend using the plain language name for your agency or site, such as Social Security Administration.

Homepage URL

Tell us the homepage URL of your website. Enter a fully qualified URL, including https://, https://www, etc.

We use this homepage URL to (a) populate your primary domain and (b) link the default logo on your results page to your homepage.

Correct Syntax

  • https://www.dol.gov
  • https://gobierno.usa.gov
  • https://stemcells.nih.gov

Incorrect Syntax

  • usa.gov (use https://www.usa.gov)
  • www.nasa.gov (use https://www.nasa.gov)
  • https://www.sec.gov/litigation.shtml (use the higher-level folder, https://www.sec.gov/litigation)
  • https://www.gsa.gov/portal/category/100000?utm_source=OCM&utm_medium=print-radio&utm_term=&utm_campaign=shortcuts (use the higher-level domain, https://www.gsa.gov)
  • http://www.doc.gov (use the domain that resolves, https://www.commerce.gov)

Site Handle

You created your site handle when you originally added your site. Searchers can see your site handle in the browser’s address bar. We recommend making this handle short and unique, such as ssa. If you’d like to edit your site handle, email us at search@support.digitalgov.gov.

Note: If you change the site handle on a live search site, you will lose any analytics that have been gathered.

Site name and handle on SocialSecurity.gov's search results page

Site ID

We assign a unique, system-generated numeric ID to your site. We use this ID in our back-end code. You can see this ID in the Admin Center URL for your site.

Site Language

You chose your site’s language (English, Spanish, or one of the other 69 supported languages when you first added your site. If you’d like to edit your site’s language, email us.

Delete

After you’ve set up a site, you may want to delete it. Use caution. This action permanently deletes your site and its associated analytics and indexes. We cannot restore deleted sites.


Did you know? Click the pushpin icon in the top navigation bar to change the default site that appears when you first log in to the Admin Center.

View a Snapshot of What's Happening on Your Site Today

Search.gov Home > Admin Center > YourSite > Dashboard > Site Overview

The Site Overview page provides a snapshot of what has been happening on your site in the past day. All of the analytics in the Admin Center are updated in real-time, which means that you instantaneously see the searches and clicks as they’re performed on your site.

You can opt to receive this snapshot as a daily email by clicking on the envelope icon () in the top navigation bar next to your selected site.

You can also toggle between filtered and unfiltered views of the data by clicking on the filter icon () in the top navigation bar. The filtered data represent our best effort to show you real searches performed by your site’s visitors. The unfiltered data include nearly all searches and clicks. Only known spiders (such as Bingbot and Googlebot) are excluded.

Top Queries

A list of the 10 most popular searches on your site. If you don’t have any searches, you’ll see a “Not enough query data available” message.

The data are processed to present “real” searches by humans (that is, the data are de-duped by IP address to remove bot traffic and other noise).

Top Clicked URLs

A list of the top 10 pages that were clicked thru by searchers on your search results page. If you don’t have any clicks, you’ll see a “Not enough query data available” message.

A list of the top 10 changing URLs that were visited by users on your website. If there are any pages that are trending, they’ll appear here.

Use this report to identify newly popular pages on your website. Investigate why pages are trending, if the reason is not immediately apparent.

You’ll only see the trending URLs list if you have our Javascript snippet on your web pages. Data are updated every few minutes, so you may see URLs come and go quickly.

A list of the top 10 search terms with the greatest gain between yesterday and today. If there are any queries that are trending, they’ll appear here.

Use this report to identify newly popular terms. Create new content or update existing content to ensure it’s current, accurate, and complete.

Queries with No Results

A list of the top 10 queries that returned no results.

Use this data to help searchers find your content by adding a Best Bet, updating your existing web pages, or both.

For example, on one agency’s website, a dozen searches for frostline returned no results. The frost line—also known as depth of frost or freezing—is the depth to which the groundwater in soil is expected to freeze. The agency’s pages on the topic referred to this concept as depth of freezing.

Top Queries with Low Click Thrus

A list of the top 10 searches with low click thru rates. If there are any frequently searched terms that returned results but didn’t get a click at least 20% of the time, they’ll appear here.

Use this alert about low click thrus to identify issues with coverage. You may opt to create a Best Bet, add pages that may be missing from the results, or do both to improve recall. Or, you may opt to incorporate language from these popular search terms into your page titles and descriptions, or enter common search terms as keywords, verbatim.

This Month’s Totals to Date

The total number of queries and clicks for the present month-to-date. A click is recorded each time a searcher clicks on a results link.

A graph of your site’s total search queries over time is also presented to provide an overview of how your traffic has trended from the time your agency started using our service (but no further back than the last 13 months). Hover over the trend line to see the total number of queries in a given month.


Did you know? The Monthly Report gives a bird’s-eye view of the number of queries and clicks each month.

Did you know? Click the pushpin icon in the top navigation bar to change the default site that appears when you first log in to the Admin Center.

Filtering Tags

Search.gov Home > Admin Center > YourSite > Manage Content > Domains > Advanced > Filter Tags

Note: this feature is only available to sites that are enabled for i14y.

Tags are used in the full text searching of i14y documents. If a tag is added to a document, the document will appear in search results when a query matches the tag, even if the term does not appear in the document’s full text.

Use the Filter Tags feature to scope your search site based on tags. You can exclude documents that have a given tag from all search results. Or, you can require that a tag is present in order for a document to be shown.

For example:

  • You can have a “News Search” site that will only show results that have the tag “news”.
  • You can exclude all documents that have the “blog” tag from your search results, if you don’t want them to appear.

Add Your RSS Feeds to Our Index

Search.gov Home > Admin Center > YourSite > Manage Content > RSS

Would you like searchers to be able to search your news or multimedia content? Would you like them to be able to narrow results by date?

Tell us the locations and names of your RSS feeds. We index all new and updated content on your feeds within minutes.

Step 1. Tell Us About Your RSS Feeds

We love feeds! Tell us about all of your RSS feeds, even if you don’t opt to show them in step 2 below. Feeds are the fastest and most reliable way for us to learn about your new and updated content.

Select the option to add a new RSS feed.

Name

Create a name for the feed. Within this ‘bucket’ you can list a single RSS feed or many feeds. For example, you can opt to list all of your agency’s press releases separately (such as Police News, Fire News, EMS News, etc.) or you can list all three under one general name (such as News).

Feed type

Select the type of feed. The default option is RSS (for text content like press releases or blogs). Change it to media RSS (External link) for multimedia content like images and videos.

Each item within a RSS feed must include a title, link, description, and publication date.

Each item within a media RSS feed must also include both a media:content URL (External link) to specify a direct URL to the media object and a media:thumbnail URL (External link) to specify a URL to the object’s thumbnail.

Any items missing a required element won’t display in your search results.

Sample RSS Item with All Required Elements

<item>
<title>
Statement from Agriculture Secretary Tom Vilsack Regarding World Organization for Animal Health (OIE) Upgrade of United States&apos; BSE Risk Status
</title>
<pubDate>May 29, 2013 00:00:00 CDT</pubDate>
<link>
http://www.usda.gov/wps/portal/usda/usdahome?contentid=2013/05/0106.xml&contentidonly=true
</link>
<description>
WASHINGTON, May 29, 2013–Agriculture Secretary Tom Vilsack made the following statement about notification received today from the World Organization for Animal Health (OIE) upgrading the United States&apos; risk classification for bovine spongiform encephalopathy (BSE) to negligible risk:
</description>
</item>

Sample Media RSS Item with All Required Elements

<item>
<title type="html">
<![CDATA[ Great Lakes Beach Health ]]>
</title>
<link>
http://gallery.usgs.gov/photos/05_24_2013_gkb4Erq11X_05_24_2013_0
</link>
<guid>
http://gallery.usgs.gov/photos/05_24_2013_gkb4Erq11X_05_24_2013_0
</guid>
<pubDate>Fri, 24 May 2013 00:00:00 EDT</pubDate>
<media:description type="html">
<![CDATA[ As schools close for the year and summer weather beckons, many recreationalists head to the Great Lakes' public beaches. However, these coastal areas can become 	contaminated with disease-causing bacteria that threaten public health, disrupt water 	recreation, and pay a toll on the Great Lakes economies that depend on summer tourism. ]]>
</media:description>
<media:thumbnail url="http://gallery.usgs.gov/images/05_24_2013/gkb4Erq11X_05_24_2013/thumbs/CoastalEco_KPrzybyla_kelly18.JPG"/>
<media:content type="" url="http://gallery.usgs.gov/images/05_24_2013/gkb4Erq11X_05_24_2013/large/CoastalEco_KPrzybyla_kelly18.JPG"/>
</item>

Step 2. Opt to Show as a Facet, Inline Module, or Both

Allow searchers to see inline results for recent, relevant RSS results across all of your RSS feeds by turning on the News module on the Display Overview page. When a searcher’s query matches the title of an RSS article published within the past four months, the article appears in the News module. Very recent news results (less than five days) appear at the top of the page and less recent news results appear at the bottom. Up to three articles are displayed. You can edit the default module title, News, on the Display Overview page.

Allow searchers to narrow results to a specific feed by turning on the option to show the facet on the Display Overview page. To ensure searchers don’t encounter too many dead ends, we recommend showing only feeds with a significant amount of content as a facet.

Step 3. Check Your Search Results Page

Module

If you opted to show your RSS feed(s) as a module, searchers will see inline news results. It will highlight the three most recent, relevant results across all of your RSS feeds.

RSS Inline Module Example

Facet

If you opted to show your RSS feed(s) as a facet, searchers can narrow their results to see only RSS-based content by clicking on the facet.

Within the RSS-based results, searchers can opt to limit results to the last hour, day, week, month, or year, or they can set a custom date range. They also can sort results in descending order by relevance (best match first) or date (most recent first).

Results count

We show the number of results returned for searches against your feed(s).

RSS Facet Example with Results Count

Step 4. Check the Status of Your Feeds

We use color coding to indicate each feed’s status.

RSS status messages and colors

Color Status
Green    Feed indexed, no errors      
Yellow Pending Indexing
Red Feed indexed, but error in one or more items
   

Click on the name of any feed with an error to see more detailed information about it. Possible error messages follow.

  • 404 Not Found
  • Feed looks empty
  • Description can’t be blank
  • Title can’t be blank
  • Missing link field
  • Missing pubDate field
  • Link is not a valid URL
  • Linked URL does not exist (HTTP 404)

Troubleshooting tip: We support RSS 2.0 and Atom feeds. Learn more and validate your feeds at:

Troubleshooting tip: We index the content on your RSS feeds from the time you input them in the Admin Center. To backfill historical content, temporarily modify your RSS feeds to return more results. Leave this larger feed in place for one hour. You can do this during off-hours and you don’t need to coordinate with us.

Did you know? You can set up a search box on your website that limits results to your feed.

Start with the standard form snippet on the Code Snippets page under the Activate Search tab. Change the form action to action="https://search.usa.gov/search/news and add the following line to limit the results to your feed.

<input type="hidden" name="channel" value="###">

The value is the number for your feed ID, which is visible in the URL when you edit your feed in the Admin Center.

Did you know? You can click on the ‘Preview’ option to see the content we have indexed for each of your RSS feeds.

Did you know? When you provide us with your YouTube channel, we’ll automatically index the RSS feed for your YouTube channel.

Did you know? For any feeds that you’ve extended with a contributor, publisher, or subject Dublin Core (External link) property, searchers may narrow results by these facets in the sidebar on the results page.

Working with i14y Drawers

Search.gov Home > Admin Center > YourSite > Manage Content > i14y Drawers

An i14y Drawer is an index receiving content via the i14y API. We’ve called them Drawers because, like drawers in a filing cabinet, multiple indexes can be included in a single configuration to scope the entirety of a site’s search.

Information on working with our Drupal module is here.

Step 1. Make sure your site is connected to i14y

If your site Admin Center has the following page: Admin Center > YourSite > Manage Content > i14y Drawers, your site is connected to i14y. If you don’t see this page in the Admin Center, email us. You can also read about Getting Started with i14y.

Step 2. Add one or more Drawers

On Admin Center > YourSite > Manage Content > i14y Drawers, Click Add i14y Drawer in the upper right corner. If you have more than one website that will be sending content, add a separate Drawer for each of these sites.

Handle

Let us know what the handle for your drawer should be - the handle must be all lowercase alphanumeric, all one word, and can include underscores but no other special characters (e.g., agency_drawer_handle2).

The Drawer Description is optional

Step 3. Send Content to Fill your Drawer

After you have created your Drawer(s), click Show from the i14y Drawers list to find that Drawer’s secret API key. Use this secret key with your drawer handle in your API call or your CMS module to send your content to the right place.

Step 4. Review your Index

You can view the number of documents indexed for each drawer on the main i14y Drawers list, and when the most recent document was received. Click Show to view documents within a particular drawer. We display the most recent 1,000 items that were sent to the drawer. You can also search for keywords in the documents’ text, titles, and descriptions (Note: you cannot currently search for URLs).

Note: We use the Domains section to scope search results - if the domain(s) of your i14y content are not listed in the Domains section, that content will not appear on your search results page.

We send success and / or failure codes in response to your API call, so if the number of documents in our index doesn’t match what you sent, check those response codes.

Note: If you experience difficulty sending documents to i14y, it is possible your firewall is not letting you communicate with the i14y server. Check out our cURL test commands or view the full i14y documentation.


Caution: A pop-up message will appear when you hit Remove on a drawer: please review this pop-up message carefully. If you remove a drawer that is only associated with one search configuration, the drawer and its contents will be deleted from our system. If the drawer is attached to multiple search configurations, it will only be removed from the search configuration you are currently on. The pop-up message will indicate what type of drawer you have.

If you accidentally delete a drawer, you will need to set-up a new drawer and resend the content - we are unable to retrieve deleted drawers. If you accidentally remove a shared drawer but it is still associated with other search configurations, we can re-attach it to your site. Contact us for assistance.


Did you know? We can attach each i14y drawer to multiple search configurations: if you have a drawer that you’d like to use for multiple search sites, email us.

Adding Supplemental Content to Your Search Configuration

Search.gov Home > Admin Center > YourSite > Manage Content > Domains > Advanced

Is content missing from your search results? Do you want to have RSS content appear in your main page search results?

We offer two ways for you to tell us about content that you want us to fetch and include in our web index: via RSS feed (Supplemental Feed) or manually (via the Supplemental URLs section).

Note for customers using Collections: We serve Collections results from commercial indexes; the instructions below only apply to your main search results.

Note to our i14y customers: Supplemental content will not appear for customers using our i14y content indexing API.

Adding Content via Supplemental Feed

You can use an RSS feed to add URLs. The feed is useful if you’d like to automate the process, add multiple URLs, or both.

Enter the URL of your RSS feed. Click Submit. We’ll fetch each URL in your feed and index the title, description (optional), and the full text of the document/webpage for the items you provide. Please be sure to follow our schema, and note that we will only index the items that are listed in the feed.

You can see the list of URLs we have indexed from your feed by viewing the Supplemental URLs section.

Delete a URL added via RSS feed by deleting the item from your RSS feed. We’ll remove the URL from our index the next time we fetch your RSS feed (we fetch saved feeds once per day, in the evening in Eastern Time; for brand new feeds, we fetch as soon as you hit Submit on the feed URL).

You can also delete all of the URLs added via the RSS feed by deleting the RSS feed itself.

Note: We can’t restore URLs that don’t exist within your current feed. Please don’t remove items from the feed unless you want them to be removed from the index. You can add a feed that contains up to 1,000 items on your own. If you have a feed that will include more than 1,000 items, please contact us.

Adding Content via the Supplemental URLs Section

You can also manually add a specific URL on the Supplemental URLs page. We’ll fetch each URL you add manually and we’ll index the title, description, and the full text of the document/webpage for the link you provide. Note: You can’t manually add a Supplemental URL if it has already been added by via Supplemental feed.

Delete a manually added URL by selecting Remove within the list of supplemental URLs.

The source column in the crawl report shows how you added the URL, via an RSS feed or manually.

How Supplemental Content Appears on Your Search Results Page

We display titles as they are provided in the feed or as you entered them manually.

We display descriptions as follows:

  • If a searcher’s term matches terms in the description you provided: We’ll display the description exactly as you provided it.
  • If a searcher’s term only matches terms in the full text of the document: We’ll display snippets taken from the full text.

By default, supplemental content will appear after all Bing commercial search results have been shown. For example, a search on DigitalGov.gov for “serverless architecture” displays results from Bing’s index on page 1.

After hitting ‘next’ to view page 2, supplemental content is served.

Page 1:

Commercial Results are Displayed on Page One of the DigitalGov.gov Search Results Page

Page 2:

Supplemental Feed Results are Displayed on Page Two of the DigitalGov.gov Search Results Page

If there are no commercial results for a query but there are supplemental content results, we will display the supplemental content on page one.

Getting Supplemental Content To Always Appear First

If you would like your supplemental content to always appear first in search results, please contact our team. Searchers will first be served any supplemental content results, and are then offered the chance to “search again,” which will lead them to commercial index results.

Please note: if you have any regular RSS feeds set up in your search site, this regular RSS content will also appear first with the supplemental content, prior to commercial results. If content is present in both types of feed, duplicates will appear in your results.

In the example below, a Supplemental URL appears first, and clicking “Try your search again” will lead to a page of Bing results.

A Supplemental URL is Displayed on Page One of the DigitalGo Search Search Results Page

Getting Supplemental Content To Be Your Only Search Results

If you would like your supplemental content to be the only results served from your site, please contact our team. Searchers will not be offered the chance to “search again” on a commercial index after they exhaust your supplemental content results (see above example).

Please note: if you have any regular RSS feeds set up in your search site, this regular RSS content will also appear with the supplemental content. If content is present in both types of feed, duplicates will appear in your results.


Troubleshooting tip: We support RSS 2.0 and Atom feeds. Learn more and validate your feeds at:

Troubleshooting tip: Are you seeing an error message in the crawled URLs report for your PDF that says, “No content found in document”? Your PDF is likely an image-only, non-searchable file that was created from a paper document using a scanner. See the resources below for more information on how to create searchable PDF files.

Troubleshooting tip: Are you seeing an error message in the crawled URLs report that says your page is taking too long to load? Use Pingdom Tools Full Page Test (External link) to test the load time of your page, analyze it, and find the bottlenecks.

Did you know? To help the public find your web pages when they search on Bing.com, we notify Bing about any URLs you add. While this helps with search engine optimization (SEO), it is not a cure-all. You should also register for and use commercial search engines’ webmaster tools.

How to Edit Your Domains

Search.gov Home > Admin Center > YourSite > Manage Content > Domains

Enter the applicable domains, subdomains, or folders to return results specific to your website. You do not need to include http:// as we recognize domains with or without it.

Entire Site(s)

  • List yoursite.gov (without www) to search your entire site, including all subdomains.
  • List yoursite.gov and yoursite2.gov to search multiple sites.

Specific Subdomains

  • List www.yoursite.gov to include only the primary ‘www’ domain.
  • List subdomain.yoursite.gov to include subdomains.

Specific Directories

List yoursite.gov/folder to include folders. Note: this will include all sub-folders (if they exist), i.e. yoursite.gov/folder/folder2.

You cannot specify individual URLs.

Searches that start three or more subfolders deep cannot use our default configuration. Email us for assistance.


Troubleshooting Tip: You can exclude specific subdomains or folders using any of the following options.

  1. List the excluded subdomains or folders in your robots.txt (External link) file. This blocks them from all crawlers that follow the robots exclusion protocol, including Google.com and Bing.com.

  2. Exclude them by omission. For example, to exclude subdomain3, list only subdomain1.yoursite.gov, subdomain2.yoursite.gov, and subdomain4.yoursite.gov.

  3. Email us at search@support.digitalgov.gov to filter them from your site’s search results. Note that this option should be used only if the other options aren’t feasible as it can have an undesired effect on other features such as the display of spelling suggestions and relevance ranking.

Troubleshooting Tip: You can remove or filter specific URLs using any of the following options.

  1. Remove the page from your site and return a 404 (Not Found) 401 (Gone), or 301 (Permanently Moved) HTTP status.

  2. Add a noindex robots metatag or x-robots tag (External link). This blocks them from all crawlers that follow the robots exclusion protocol, including Google.com and Bing.com.

  3. Request the page’s removal using the Block URLs option in Bing Webaster Tools (External link) and Remove URLs option in Google Webmaster Tools (External link).

  4. Use the Filter URLs option under the Advanced tab. Note that this option should be used only if the other options aren’t feasible as it can have an undesired effect on other features such as the display of spelling suggestions and relevance ranking.

Troubleshooting Tip: Are specific web pages missing from your Bing web results? You can tell us about any missing URLs via an RSS feed or manually (the Supplemental Feed and Supplemental URLs options under the Advanced tab, respectively). We’ll fetch and index the pages and append them to your web results.