internal/worker: load shed based on DB lock contention

If many DB processes are waiting for locks, shed load.

This is in addition to the existing load-shedding rule based on zip
size.

I conducted an informal experiment to see how this worked. I queued up
82 versions of github.com/aws/aws-sdk-go for processing. That module
has a small zip, so no shedding occurs because of zip size, but it
takes some time to process.

Without lock-based load-shedding, there were 157 "max serialization"
errors. Most of the time there were many active fetches in progress,
almost all waiting for locks.

With lock-based load-shedding, there were only 48 "max serialization"
errors. Most fetches completed quickly.

For golang/go#48010

Change-Id: I0cee02b9c4085a8bc187d803eaca2f30ddd378b5
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/349312
Trust: Jonathan Amsterdam <jba@google.com>
Run-TryBot: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Jamal Carvalho <jamal@golang.org>
Reviewed-by: Julie Qiu <julie@golang.org>
1 file changed
tree: df972013954056f8beef6701429f6096abaad56f
  1. cmd/
  2. devtools/
  3. doc/
  4. internal/
  5. migrations/
  6. static/
  7. tests/
  8. third_party/
  9. .dockerignore
  10. .eslintignore
  11. .eslintrc.yaml
  12. .gitignore
  13. .prettierignore
  14. .prettierrc.yaml
  15. .stylelintignore
  16. .stylelintrc.yaml
  17. all.bash
  18. cloudbuild.yaml
  19. CONTRIBUTING.md
  20. go.mod
  21. go.sum
  22. jest.config.js
  23. LICENSE
  24. package-lock.json
  25. package.json
  26. PATENTS
  27. README.md
  28. tsconfig.json
README.md

Pkg.go.dev

Go Reference

A site for discovering Go packages

Pkg.go.dev is a website for discovering and evaluating Go packages and modules.

You can check it out at https://pkg.go.dev.

Issues

If you want to report a bug or have a feature suggestion, please first check the known issues to see if your issue is already being discussed. If an issue does not already exist, feel free to file an issue.

For answers to frequently asked questions, see go.dev/about.

You can also chat with us on the #pkgsite Slack channel on the Gophers Slack.

Contributing

We would love your help!

Our canonical Git repository is located at go.googlesource.com/pkgsite. There is a mirror of the repository at github.com/golang/pkgsite.

To contribute, please read our contributing guide.

License

Unless otherwise noted, the Go source files are distributed under the BSD-style license found in the LICENSE file.

Links