design/25530-sumdb: clarify a few things about database access
Updates to address questions in CL 173637.
Change-Id: I5f57cc0613a2a80221f997c634f2953989c2f9db
Reviewed-on: https://go-review.googlesource.com/c/proposal/+/173917
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
diff --git a/design/25530-sumdb.md b/design/25530-sumdb.md
index 2ec929d..98f10cc 100644
--- a/design/25530-sumdb.md
+++ b/design/25530-sumdb.md
@@ -180,7 +180,7 @@
The Go checksum database will run at `https://sum.golang.org/` and serve the following endpoints:
- `/latest` will serve a signed tree size and hash for the latest log.
-
+
- `/lookup/M@V` will serve the log record number for the entry about module M version V,
followed by the data for the record (that is, the `go.sum` lines for module M version V)
and a signed tree hash for a tree that contains the record.
@@ -189,7 +189,7 @@
authenticating it against the signed tree hash
and authenticating the signed tree hash against the client's
timeline of signed tree hashes.
-
+
- `/tile/H/L/K[.p/W]` will serve a [log tile](https://research.swtch.com/tlog#serving_tiles).
The optional `.p/W` suffix indicates a partial log tile with only `W` hashes.
Clients must fall back to fetching the full tile if a partial tile is not found.
@@ -250,12 +250,14 @@
That variable will default to the `sum.golang.org` server.
The `go` command will cache the latest signed tree size and tree hash
-in `$GOPATH/pkg/sumdb/sum.golang.org/latest`.
-It will cache lookup results and tiles in
-`$GOPATH/pkg/mod/download/cache/sumdb/sum.golang.org/lookup/path@version`
-and `$GOPATH/pkg/mod/download/cache/sumdb/sum.golang.org/tile/H/L/K[.W]`.
+in `$GOPATH/pkg/sumdb/<sumdb-name>/latest`.
+It will cache lookup results and tiles in
+`$GOPATH/pkg/mod/download/cache/sumdb/<sumdb-name>/lookup/path@version`
+and `$GOPATH/pkg/mod/download/cache/sumdb/<sumdb-name>/tile/H/L/K[.W]`.
+(More generally, `https://<sumdb-URL>` is cached
+in `$GOPATH/pkg/mod/download/cache/sumdb/<sumdb-URL>`.)
This way, `go clean -modcache` deletes cached lookup results and tiles
-but not the latest signed tree hash, which should be preserved for
+but not the latest signed tree hash, which should be preserved for
detection of timeline inconsistency.
No `go` command (only a manual `rm -rf $GOPATH/pkg`)
will wipe out the memory of the latest observed tree size and hash.
@@ -277,17 +279,19 @@
- `GONOPROXY=prefix1,prefix2,prefix3` sets a list of module path prefixes,
possibly containing globs, that should not be proxied.
For example:
-
+
GONOPROXY=*.corp.google.com,rsc.io/private
will bypass the proxy for the modules foo.corp.google.com, foo.corp.google.com/bar, rsc.io/private, and rsc.io/private/bar,
though not rsc.io/privateer (the patterns are path prefixes, not string prefixes).
-- `GOSUMDB=<database-key>` sets the Go checksum database to use.
+- `GOSUMDB=<sumdb-key>` sets the Go checksum database to use,
+ where `<sumdb-key>` is a verifier key as defined in
+ [package note](https://godoc.org/golang.org/x/exp/notary/internal/note#hdr-Verifying_Notes).
- `GONOSUMDB=prefix1,prefix2,prefix3` sets a list of module path prefixes,
again possibly containing globs, that should not be looked up using the database.
-
+
We expect that corporate environments may fetch all modules, public and private,
through an internal proxy;
`GONOSUMDB` allows them to disable checksum database lookups for
@@ -400,20 +404,20 @@
so that any available proxy that proxies the database
can be a gossip source.
If we add a `go fetch-latest-chccksum-log-from-goproxy` (obviously not the final name)
-and
+and
GOPROXY=https://other.proxy/ go fetch-latest-checksum-log-from-goproxy
succeeds, then the client and other.proxy are seeing the same log.
Compared to the original scenario of a single checksum database with
-no transparent log, the use of a single transparent log
+no transparent log, the use of a single transparent log
and the ability to proxy the database and gossip improves
detection of attacks so much that there is little incremental
security benefit to adding the complexity of multiple notaries.
At some point in the future, it might make sense for the
Go ecosystem to support using multiple databases,
-but to begin with we have opted for the simpler
+but to begin with we have opted for the simpler
(but still reasonably secure) ecosystem design
of a single database.
@@ -474,7 +478,7 @@
corporate computer management systems already provide
ways to preset environment variables.)
-#### Private Module SHA256s
+#### Private Module SHA256s
Another possibility to reduce exposure is to support and
use by default an alternate lookup `/lookup/SHA256(module)@version`,
@@ -493,7 +497,7 @@
That operation—for now, let's call it `go notify <modulepath>`—would
need to be run just once ever across the whole Go ecosystem
for each module path.
-Most likely the author would do it, perhaps as part of the
+Most likely the author would do it, perhaps as part of the
still-hypothetical `go release` command,
or else the first user of the module would need to do it
(perhaps thinking carefully about being the first-ever user of the module!).
@@ -505,7 +509,7 @@
it would have to look up the given version tag for all K of them
before returning an answer. This seems needlessly expensive
and of little practical benefit.
-(An attacker might even create a long list of module paths
+(An attacker might even create a long list of module paths
that collide with a popular module, just to slow down requests.)
The SHA256 + `go notify` scheme is not part of this proposal today,
@@ -534,7 +538,7 @@
The `go` command will also cache database lookup results
(reauthenticating them against cached tiles at each use),
-so that using a single computer to
+so that using a single computer to
upgrade the version of a particular dependency used by N different modules
will result in only one database lookup, not N.
That further reduces the strength of any usage signal.
@@ -568,7 +572,7 @@
not make it past the proxy.
The database endpoints are designed for cacheability,
so that a proxy can avoid making any request more than once.
-Requests for new versions of modules would still need to be
+Requests for new versions of modules would still need to be
relayed to the database.
We anticipate that there will be many proxies available
