| commit | 222c0a398c751966e6ec20546fd0dffcd44e9595 | [log] [tgz] |
|---|---|---|
| author | Rob Findley <rfindley@google.com> | Tue Jun 09 21:41:22 2020 -0400 |
| committer | Robert Findley <rfindley@google.com> | Fri Jun 12 02:09:55 2020 +0000 |
| tree | 30b6a17890780741b8d23dcbbfeaaa41d3bcf6c5 | |
| parent | 2b3b7078c6cbbea6fc9e20341b5261a6a0b363fd [diff] |
internal/postgres: fail hedged search on the first failed searcher Waiting for subsequent searchers to complete after the first one fails behaves poorly in overload scenarios. For example, one of the ways that overload can manifest itself is by connection exhaustion or Postgres actively refusing connections. Let's assume this is happening, and Postgres randomly refuses half the connections. Now let's assume that we have a particular query for a rare term that is much faster and cheaper when using the reverse index. To be concrete, let's say that 'deep' search (using the index) costs 0.1, and 'popular' search (doing a linear scan) costs 10 -- the units don't matter. Normally, the expected query cost of this search is approximately 0.2: 0.1 for the deep search, and around 0.1 for the popular search that was simultaneously occurring until deep completed. If queries are randomly refused with a 50% probability, 25% the time this query is going to cost 0.1, because popular search was refused and deep succeeds, 25% of the time it's going to cost 10 (!), because deep search was refused and we continued to wait for popular search to complete, 25% of the time it costs ~0 because both searches fail, and 25% of the time it costs 0.2 as usual, because they both succeed. That means that in this particular scenario, our query went from an expected cost of 0.2, to an expected cost of ~2.6, an order of magnitude more! Not a good thing to happen while the database is already overloaded. Instead, let's error out the search on the first error from any searcher. A test is added to exercise the behavior of hedgedSearch in the presence of errors. Also fix some miscellaneous stale comments along the way. Change-Id: Id0496b3d402a403a17160c63edaacad537302284 Reviewed-on: https://team-review.git.corp.google.com/c/golang/discovery/+/766809 CI-Result: Cloud Build <devtools-proctor-result-processor@system.gserviceaccount.com> Reviewed-by: Jonathan Amsterdam <jba@google.com>
Pkg.go.dev is a website for discovering and evaluating Go packages and modules.
You can check it out at https://pkg.go.dev.
Pkg.go.dev launched in November 2019, and is currently under active development by the Go team.
Here's what we are currently working on:
Design updates: We have some design changes planned for pkg.go.dev, to address UX feedback that we have received. You can expect a more cohesive search and navigation experience coming soon. We plan to share these designs for feedback once they are ready.
Godoc.org redirect: Longer term, we are working towards redirecting godoc.org traffic to pkg.go.dev. We know that there are features available on godoc.org that users want to see on pkg.go.dev, and we want to ensure that we address these. We’ve been keeping track of issues related to redirecting godoc.org traffic on Go issue #39144. These issues will be prioritized in the next few months. We also plan to continue improving our license detection algorithm.
Search improvements: We’ll be improving our search experience based on feedback in Go issue #37810, to make it easier for users to find the dependencies they are looking for and make better decisions around which ones to import.
We encourage everyone to begin using pkg.go.dev today for all of their needs and to file feedback! You can redirect all of your requests from godoc.org to pkg.go.dev, by clicking Always use pkg.go.dev at the top of any page on godoc.org.
If you want to report a bug or have a feature suggestion, please first check the known issues to see if your issue is already being discussed. If an issue does not already exist, feel free to file an issue.
For answers to freqently asked questions, see go.dev.
You can also chat with us on the #tools slack channel on the Gophers slack.
We would love your help!
Our canonical Git repository is located at go.googlesource.com/pkgsite. There is a mirror of the repository at github.com/golang/pkgsite.
To contribute, please read our contributing guide.
Unless otherwise noted, the Go source files are distributed under the BSD-style license found in the LICENSE file.