Posts tagged release-notes
Improvements
- Increasing indexing coverage We now index pages with only title metadata. Previously, we required pages include either content or description metadata in order to be indexed. This was to ensure we indexed pages with more complete information. However, this limited relevant content included on Javascript pages and similar client-side rendered pages.
- As you include title metadata, it’s important keep them short and on-topic without packing in repeated information such as agency name. If you want to include your agency name, we recommend you place it after the page title. More information on metadata and tags you should include is available here.
- Improving USAJOBS Integration Common government terms “federal” and “gobierno” no longer trigger our USAJOBS integration. Previously, queries that included these terms triggered our integration with USAJOBS to display relevant postings. However, these queries often resulted in no or irrelevant job postings.
- Service upgrades We continued to upgrade components of our service, including ElasticSearch, Rails, and Ruby to improve the overall performance of our indexing system.
Fixes
- Language parsing bug fix We fixed a bug that parsed non-English pages (e.g. Greek) incorrectly or not at all. This particularly affected pages containing mathematical formulas and equations.
Improvements
- 90 day Account Deactivations: We now deactivate accounts for users who haven’t logged into Search.gov in the last 90 days. If you are unable to login should contact us via search@support.digitalgov.gov to reactivate your accounts. Contractors who do not use .gov or .mil email addresses will be asked to have their government POC re-confirm their status.
- Improved stability: We stabilized search performance by removing unnecessary content from our indices to free up memory.
- Service upgrades: We continued to upgrade components of our service, including ElasticSearch, Rails, and Ruby to improve the overall performance of our indexing system.
- Non-.gov Government Domains Inventory: We decommissioned the old tool we used to manage the inventory of domains in use by federal agencies outside of the .gov and .mil top level domains. We transferred the data to GitHub, where it will be easier for agency staff to contribute to the inventory. More information is available on our website.
Fixes
- Image search result bug fix: We fixed a bug that prevented images sent to our index via Media RSS from being displayed in search results.
- Video search results bug fix: We fixed a bug that prevented videos from appearing in some search results pages.
Improvements
- Sitelimit parameter support
- Now our customers can specify that a search box only returns results from a particular part of their site. For example, for site www.example.gov, you want a search box on your site to only return results from
www.example.gov/history. By adding the appropriate URL parameter and value, you can limit results: &sitelimit=www.example.gov/history.
- You can also specify multiple values for the sitelimit. See this example from the National Park Service.
- Keep in mind that the sitelimit must be for a domain within your site’s index. That is, if the Domains list includes
www.example.gov, the sitelimit parameter cannot be www.agency.gov/history.
- This feature is an alternative to our Collections feature.
- Service upgrades
- We continued to upgrade components of our service, including ElasticSearch, Rails, and Ruby to improve the overall performance of our indexing system.
Fixes
Improvements
- We launched updated Freshness ranking for customers indexed by Search.gov. The fresher a page the more likely it is to be promoted. Previously, we only factored the date a page is created into its freshness. Now, we first factor the date a page was last modified and then, the date a page is created. This new feature highlights the importance of ensuring that metadata on your site’s pages and sitemap is accurate. Here is more information about how we rank your search results: https://search.gov/manual/ranking-factors.html
- We continued to upgrade components of our service, including ElasticSearch, Rails, and Ruby to improve the overall performance of our indexing system.
Fixes and Chores
- Fixed a bug that made it difficult to view sidebar items on the search results page when hovering over individual links. Now, the experience when hovering over these links is fully accessible.
Improvements
- We’ve continued to work on improving our recently launched click-based relevancy to boost search results relevance ranking for our indexing customers. As always, it’s important to periodically review your site’s Best Bets and keep your sitemaps up-to-date with latest
<lastmod> date metadata.
- We continued to upgrade components of our service, including ElasticSearch, Rails, and Ruby to improve the overall performance of our indexing system.
- We launched a new Search Site Launch Guide to make it easier to go live with Search.gov:
- https://search.gov/manual/site-launch-guide.html
- Released new documentation to make it easier to maintain your indexed content with Search.gov:
- https://search.gov/manual/indexing-with-searchgov.html
Fixes and Chores
- We mitigated potential security vulnerabilities associated with Ruby gems.
- We applied high priority Ubuntu patches.
Improvements
- We’ve added Click counts as a ranking factor for customers indexed by Search.gov. We look at the URLs that represent 75% of all clicks on search results, and give those a boost. This is the “fat head” that comes before the “long tail.” As always, following best practices will help results stay relevant:
- Use Best Bets to promote frequently visited pages that are not bubbling to the top of results on their own
- Periodically review and update Best Bets
- Maintain an up-to-date and complete sitemap with updated dates
Fixes, Upgrades, Misc
- Fixed an issue with the type-ahead feature on customer search boxes
- Fixed an issue with disappearing search icon on search boxes
- Continued with Rails upgrade for our applications
Improvements
- After integrating directly with the new USAJOBS API, we worked on additional tuning of trigger words to avoid false positive job-related searches
- We have improved search performance by caching repeat queries made to our data store
- We updated our content parser to accept some non-standard HTML tags, and to ignore any content within
<nav> and <footer> elements
Fixes, Upgrades, Misc
- We upgraded Ruby on our
search-gov repo
- We increased the processing power on the servers that support our primary web index
- We reindexed our primary index into more Elasticsearch shards
- We decreased the cookie timeout for Admin Center sessions
- We made the failed password reset alert language more ambiguous, so people will no longer be able to tell whether the email address has an account
- We fixed a bug in our MRSS photo indexer
Improvements
- We integrated with Bing v7 and transitioned our customers to this newer version.
- We now will index content on a domain even if the root of that domain lists a different domain as the canonical domain. For example,
https://publications.sampleagency.gov may list https://www.sampleagency.gov as the canonical domain, but still serve content from https://publications.sampleagency.gov/reports/first_report.pdf. We can now index https://publications.sampleagency.gov/reports/first_report.pdf`.
- We updated our job search location feature to show more job openings, and cleaned up how we send job queries to the USAJOBS api to get more results.
- We now automatically review URLs for reindexing, checking for 404s and 301s. We’re doing this every 30 days to begin with, and will adjust that timeframe as needed.
Fixes
- We upgraded the Ruby version on the repo for our search.gov website, and
asis, our image indexing repo.
- We upgraded the
activejob Ruby gem across repos.
Improvements
- We made several updates to our Chef cookbooks to further harden our operating system, including backend password policies, package configuration, and OS configuration.
- We shifted our model for supporting domain masks for hosted search results pages to leverage CAA records.
Fixes
- We fixed a gnarly bug in Elasticsearch that made queries containing very common words, like “the”, behave as if there were no results.
Improvements
- We integrated directly with the new USAJOBS API. This means that we
- are now querying their system at query time, rather than building an index of their job postings within our own system and querying that at query time.
- have reconfigured what information our jobs searches include in the full query that we send to USAJOBS
- have increased the geographic radius we’ll look at when a user searches on a jobs-related term. The radius is now 75 miles from the user’s general location.
- we are now always providing a link to USAJOBS.gov if someone has searched for a jobs-related term, even if there are no jobs located near the searcher.
- We now support indexing TXT files. There are more TXT files on government websites than you would have thought!
Fixes
- We fixed a link in the Jobs module that led searchers to a broken USAJOBS.gov page.
- We now deduplicate sitemap URLs so we will not try to index the same content more than once.
- We updated Ruby gems: Loofah, Rack, FFI.
- We upgraded Ruby.