| commit | 9b1c9bd2b0c3df16a819a464b4ecf410b0557a48 | [log] [tgz] |
|---|---|---|
| author | Jonathan Amsterdam <jba@google.com> | Thu Jun 17 10:17:27 2021 -0400 |
| committer | Jonathan Amsterdam <jba@google.com> | Fri Jun 18 17:41:02 2021 +0000 |
| tree | 570f90bd90906f4be7094cfb62530de4ee1d9bea | |
| parent | 2cfdcb5d9ede961efdbbbc790ee3c7f2722ef1fa [diff] |
internal/database: add ability to run queries incrementally Postgres lets you define a cursor and fetch rows from it as top-level statements, without needing to write any PS-SQL. See https://www.postgresql.org/docs/11/sql-declare.html and https://www.postgresql.org/docs/11/sql-fetch.html for details. Use this feature to define RunQueryIncrementally, which repeatedly fetches query rows in batches until it runs out or the passed function says it's done. If grouping by module paths is enabled, use RunQueryIncrementally with a very large limit to read rows until we've seen a page's worth of module paths. This CL doesn't handle pagination correctly. Any page after the first is going to be wrong. Change-Id: Idf8233160b0cf74412a688e1a6b95f4f2b720008 Reviewed-on: https://go-review.googlesource.com/c/pkgsite/+/329469 Trust: Jonathan Amsterdam <jba@google.com> Run-TryBot: Jonathan Amsterdam <jba@google.com> TryBot-Result: kokoro <noreply+kokoro@google.com> Reviewed-by: Julie Qiu <julie@golang.org>
Pkg.go.dev is a website for discovering and evaluating Go packages and modules.
You can check it out at https://pkg.go.dev.
If you want to report a bug or have a feature suggestion, please first check the known issues to see if your issue is already being discussed. If an issue does not already exist, feel free to file an issue.
For answers to frequently asked questions, see go.dev/about.
You can also chat with us on the #pkgsite Slack channel on the Gophers Slack.
We would love your help!
Our canonical Git repository is located at go.googlesource.com/pkgsite. There is a mirror of the repository at github.com/golang/pkgsite.
To contribute, please read our contributing guide.
Unless otherwise noted, the Go source files are distributed under the BSD-style license found in the LICENSE file.