| commit | bb6da5216eba144f9891c1c0dd2228fdbfa45163 | [log] [tgz] |
|---|---|---|
| author | Jonathan Amsterdam <jba@google.com> | Wed Sep 01 07:22:46 2021 -0400 |
| committer | Jonathan Amsterdam <jba@google.com> | Wed Sep 01 14:16:41 2021 +0000 |
| tree | 230b553713bffd364321db456594aa70026c8dca | |
| parent | 6e28a50e59803c0b1f05ca9e18a38d33b9e2868b [diff] |
internal/{worker/fetch}: include DB activity in load-shedding
Move the load-shedding logic to the worker and have it span both the
fetch and processing of the module (as previously) as well as
inserting it into the database.
This is a more accurate estimation of load, since running a lot of
concurrent queries definitely slows down processing.
Most of the time this won't make much difference, but under high load,
such as when processing multiple large modules, it will reduce DB
contention and should result in greater throughput.
For golang/go#48010
Change-Id: I7d0922e02d00182e867fd3b29fc284c32ecab5ee
Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/346749
Trust: Jonathan Amsterdam <jba@google.com>
Run-TryBot: Jonathan Amsterdam <jba@google.com>
TryBot-Result: kokoro <noreply+kokoro@google.com>
Reviewed-by: Julie Qiu <julie@golang.org>
Pkg.go.dev is a website for discovering and evaluating Go packages and modules.
You can check it out at https://pkg.go.dev.
If you want to report a bug or have a feature suggestion, please first check the known issues to see if your issue is already being discussed. If an issue does not already exist, feel free to file an issue.
For answers to frequently asked questions, see go.dev/about.
You can also chat with us on the #pkgsite Slack channel on the Gophers Slack.
We would love your help!
Our canonical Git repository is located at go.googlesource.com/pkgsite. There is a mirror of the repository at github.com/golang/pkgsite.
To contribute, please read our contributing guide.
Unless otherwise noted, the Go source files are distributed under the BSD-style license found in the LICENSE file.