internal,static: prevent crawlers from indexing versioned pages

Some search queries are surfacing results for versioned
pkg.go.dev URLs. This change updates the robots meta tag
to skip pages that are not the latest minor version.

Fixes golang/go#54958.

Change-Id: Ib5b98f9e41a594f2a8ad8980338ac2b5d7f6fca0
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/429675
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Suzy Mueller <suzmue@golang.org>
Run-TryBot: Jamal Carvalho <jamal@golang.org>
2 files changed
tree: dac698e2df88ac4d89e985ab7913540afb217fec
  1. cmd/
  2. deploy/
  3. devtools/
  4. doc/
  5. internal/
  6. migrations/
  7. static/
  8. tests/
  9. third_party/
  10. .dockerignore
  11. .eslintignore
  12. .eslintrc.yaml
  13. .gitignore
  14. .prettierignore
  15. .prettierrc.yaml
  16. .stylelintignore
  17. .stylelintrc.yaml
  18. all.bash
  19. CONTRIBUTING.md
  20. go.mod
  21. go.sum
  22. jest.config.js
  23. LICENSE
  24. package-lock.json
  25. package.json
  26. PATENTS
  27. README.md
  28. tsconfig.json
README.md

Pkg.go.dev

Go Reference

A site for discovering Go packages

Pkg.go.dev is a website for discovering and evaluating Go packages and modules.

You can check it out at https://pkg.go.dev.

Requirements

Pkgsite requires Go 1.19 to run. The last commit that works with Go 1.18 is 9ffe8b928e4fbd3ff7dcf984254629a47f8b6e63. The last commit that works with Go 1.17 is 4d836c6a652cde92f433967680dfd6171a91ec12.

Issues

If you want to report a bug or have a feature suggestion, please first check the known issues to see if your issue is already being discussed. If an issue does not already exist, feel free to file an issue.

For answers to frequently asked questions, see pkg.go.dev/about.

You can also chat with us on the #pkgsite Slack channel on the Gophers Slack.

Contributing

We would love your help!

Our canonical Git repository is located at go.googlesource.com/pkgsite. There is a mirror of the repository at github.com/golang/pkgsite.

To contribute, please read our contributing guide.

License

Unless otherwise noted, the Go source files are distributed under the BSD-style license found in the LICENSE file.

Links