cmd/go: default to GOPROXY=https://proxy.golang.org and GOSUMDB=sum.golang.org
This CL changes the default module download and module verification mechanisms
to use the Go module mirror and Go checksum database run by Google.
See https://proxy.golang.org/privacy for the services' privacy policy.
(Today, that URL is a redirect to Google's standard privacy policy,
which covers these services as well. If we publish a more specific
privacy policy just for these services, that URL will be updated to
display or redirect to it.)
See 'go help modules' and 'go help modules-auth' for details (added in this CL).
To disable the mirror and checksum database for non-public modules:
go env -w GONOPROXY=*.private.net,your.com/*
go env -w GONOSUMDB=*.private.net,your.com/*
(If you are using a private module proxy then you'd only do the second.)
If you run into problems with the behavior of the go command when using
the Go module mirror or the Go checksum database, please file issues at
https://golang.org/issue/new, so that we can address them for the
Go 1.13 release.
For #25530.
This CL also documents GONOPROXY.
Fixes #32056.
Change-Id: I2fde82e071742272b0842efd9580df1a56947fec
Reviewed-on: https://go-review.googlesource.com/c/go/+/178179
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
diff --git a/src/cmd/go/alldocs.go b/src/cmd/go/alldocs.go
index 1a7eff2..26fb337 100644
--- a/src/cmd/go/alldocs.go
+++ b/src/cmd/go/alldocs.go
@@ -2020,21 +2020,6 @@
//
// Module proxy protocol
//
-// The go command by default downloads modules from version control systems
-// directly, just as 'go get' always has. The GOPROXY environment variable allows
-// further control over the download source. If GOPROXY is unset, is the empty string,
-// or is the string "direct", downloads use the default direct connection to version
-// control systems. Setting GOPROXY to "off" disallows downloading modules from
-// any source. Otherwise, GOPROXY is expected to be a comma-separated list of
-// the URLs of module proxies, in which case the go command will fetch modules
-// from those proxies. For each request, the go command tries each proxy in sequence,
-// only moving to the next if the current proxy returns a 404 or 410 HTTP response.
-// The string "direct" may appear in the proxy list, to cause a direct connection to
-// be attempted at that point in the search.
-//
-// No matter the source of the modules, downloaded modules must match existing
-// entries in go.sum (see 'go help modules' for discussion of verification).
-//
// A Go module proxy is any web server that can respond to GET requests for
// URLs of a specified form. The requests have no query parameters, so even
// a site serving from a fixed file system (including a file:/// URL)
@@ -2591,16 +2576,43 @@
//
// Module downloading and verification
//
-// The go command checks downloads against known checksums,
-// to detect unexpected changes in the content of any specific module
-// version from one day to the next. See 'go help module-auth' for details.
+// The go command can fetch modules from a proxy or connect to source control
+// servers directly, according to the setting of the GOPROXY environment
+// variable (see 'go help env'). The default setting for GOPROXY is
+// "https://proxy.golang.org", the Go module mirror run by Google.
+// See https://proxy.golang.org/privacy for the service's privacy policy.
+// If GOPROXY is set to the string "direct", downloads use a direct connection
+// to source control servers. Setting GOPROXY to "off" disallows downloading
+// modules from any source. Otherwise, GOPROXY is expected to be a comma-separated
+// list of the URLs of module proxies, in which case the go command will fetch
+// modules from those proxies. For each request, the go command tries each proxy
+// in sequence, only moving to the next if the current proxy returns a 404 or 410
+// HTTP response. The string "direct" may appear in the proxy list,
+// to cause a direct connection to be attempted at that point in the search.
+// Any proxies listed after "direct" are never consulted.
//
-// The go command can fetch modules from a proxy instead of connecting
-// to source control systems directly, according to the setting of the GOPROXY
-// environment variable.
+// The GONOPROXY environment variable is a comma-separated list of
+// glob patterns (in the syntax of Go's path.Match) of module path prefixes
+// that should always be fetched directly, ignoring the GOPROXY setting.
+// For example,
//
-// See 'go help goproxy' for details about the proxy and also the format of
-// the cached downloaded packages.
+// GONOPROXY=*.corp.example.com,rsc.io/private
+//
+// forces a direct connection to download modules with path prefixes matching
+// either pattern, including "git.corp.example.com/xyzzy", "rsc.io/private",
+// and "rsc.io/private/quux".
+//
+// The 'go env -w' command (see 'go help env') can be used to set these variables
+// for future go command invocations.
+//
+// No matter the source of the modules, the go command checks downloads against
+// known checksums, to detect unexpected changes in the content of any specific
+// module version from one day to the next. This check first consults the current
+// module's go.sum file but falls back to the Go checksum database.
+// See 'go help module-auth' for details.
+//
+// See 'go help goproxy' for details about the proxy protocol and also
+// the format of the cached downloaded packages.
//
// Modules and vendoring
//
@@ -2778,18 +2790,17 @@
// database requires giving the public key explicitly. The URL defaults to
// "https://" followed by the database name.
//
-// GOSUMDB defaults to "sum.golang.org" when GOPROXY="https://proxy.golang.org"
-// and otherwise defaults to "off". NOTE: The GOSUMDB will later default to
-// "sum.golang.org" unconditionally.
+// GOSUMDB defaults to "sum.golang.org", the Go checksum database run by Google.
+// See https://sum.golang.org/privacy for the service's privacy policy.
//
// If GOSUMDB is set to "off", or if "go get" is invoked with the -insecure flag,
-// the checksum database is never consulted, but at the cost of giving up the
-// security guarantee of verified repeatable downloads for all modules.
-// A better way to bypass the checksum database for specific modules is
-// to use the GONOSUMDB environment variable.
+// the checksum database is not consulted, and all unrecognized modules are
+// accepted, at the cost of giving up the security guarantee of verified repeatable
+// downloads for all modules. A better way to bypass the checksum database
+// for specific modules is to use the GONOSUMDB environment variable.
//
// The GONOSUMDB environment variable is a comma-separated list of
-// patterns (in the syntax of Go's path.Match) of module path prefixes
+// glob patterns (in the syntax of Go's path.Match) of module path prefixes
// that should not be compared against the checksum database.
// For example,
//
@@ -2799,6 +2810,9 @@
// either pattern, including "git.corp.example.com/xyzzy", "rsc.io/private",
// and "rsc.io/private/quux".
//
+// The 'go env -w' command (see 'go help env') can be used to set these variables
+// for future go command invocations.
+//
//
// Testing flags
//
diff --git a/src/cmd/go/internal/cfg/cfg.go b/src/cmd/go/internal/cfg/cfg.go
index 2d17d10..77d8bab 100644
--- a/src/cmd/go/internal/cfg/cfg.go
+++ b/src/cmd/go/internal/cfg/cfg.go
@@ -303,13 +303,6 @@
return v
}
- // Proxy is off by default for now.
- // TODO(rsc): Remove this condition, turning it on always.
- // (But do NOT do this without approval from rsc.)
- if true {
- return "direct"
- }
-
return "https://proxy.golang.org"
}
@@ -319,13 +312,6 @@
return v
}
- // Checksum database is off by default except when GOPROXY is proxy.golang.org.
- // TODO(rsc): Remove this condition, turning it on always.
- // (But do NOT do this without approval from rsc.)
- if !strings.HasPrefix(GOPROXY, "https://proxy.golang.org") {
- return "off"
- }
-
return "sum.golang.org"
}
diff --git a/src/cmd/go/internal/modfetch/fetch.go b/src/cmd/go/internal/modfetch/fetch.go
index 817f765..d40d2c6 100644
--- a/src/cmd/go/internal/modfetch/fetch.go
+++ b/src/cmd/go/internal/modfetch/fetch.go
@@ -702,18 +702,17 @@
database requires giving the public key explicitly. The URL defaults to
"https://" followed by the database name.
-GOSUMDB defaults to "sum.golang.org" when GOPROXY="https://proxy.golang.org"
-and otherwise defaults to "off". NOTE: The GOSUMDB will later default to
-"sum.golang.org" unconditionally.
+GOSUMDB defaults to "sum.golang.org", the Go checksum database run by Google.
+See https://sum.golang.org/privacy for the service's privacy policy.
If GOSUMDB is set to "off", or if "go get" is invoked with the -insecure flag,
-the checksum database is never consulted, but at the cost of giving up the
-security guarantee of verified repeatable downloads for all modules.
-A better way to bypass the checksum database for specific modules is
-to use the GONOSUMDB environment variable.
+the checksum database is not consulted, and all unrecognized modules are
+accepted, at the cost of giving up the security guarantee of verified repeatable
+downloads for all modules. A better way to bypass the checksum database
+for specific modules is to use the GONOSUMDB environment variable.
The GONOSUMDB environment variable is a comma-separated list of
-patterns (in the syntax of Go's path.Match) of module path prefixes
+glob patterns (in the syntax of Go's path.Match) of module path prefixes
that should not be compared against the checksum database.
For example,
@@ -722,5 +721,8 @@
disables checksum database lookups for modules with path prefixes matching
either pattern, including "git.corp.example.com/xyzzy", "rsc.io/private",
and "rsc.io/private/quux".
+
+The 'go env -w' command (see 'go help env') can be used to set these variables
+for future go command invocations.
`,
}
diff --git a/src/cmd/go/internal/modfetch/proxy.go b/src/cmd/go/internal/modfetch/proxy.go
index 605c72c..5f0432c 100644
--- a/src/cmd/go/internal/modfetch/proxy.go
+++ b/src/cmd/go/internal/modfetch/proxy.go
@@ -32,21 +32,6 @@
UsageLine: "goproxy",
Short: "module proxy protocol",
Long: `
-The go command by default downloads modules from version control systems
-directly, just as 'go get' always has. The GOPROXY environment variable allows
-further control over the download source. If GOPROXY is unset, is the empty string,
-or is the string "direct", downloads use the default direct connection to version
-control systems. Setting GOPROXY to "off" disallows downloading modules from
-any source. Otherwise, GOPROXY is expected to be a comma-separated list of
-the URLs of module proxies, in which case the go command will fetch modules
-from those proxies. For each request, the go command tries each proxy in sequence,
-only moving to the next if the current proxy returns a 404 or 410 HTTP response.
-The string "direct" may appear in the proxy list, to cause a direct connection to
-be attempted at that point in the search.
-
-No matter the source of the modules, downloaded modules must match existing
-entries in go.sum (see 'go help modules' for discussion of verification).
-
A Go module proxy is any web server that can respond to GET requests for
URLs of a specified form. The requests have no query parameters, so even
a site serving from a fixed file system (including a file:/// URL)
diff --git a/src/cmd/go/internal/modload/help.go b/src/cmd/go/internal/modload/help.go
index c1685ff..96fec84 100644
--- a/src/cmd/go/internal/modload/help.go
+++ b/src/cmd/go/internal/modload/help.go
@@ -328,16 +328,43 @@
Module downloading and verification
-The go command checks downloads against known checksums,
-to detect unexpected changes in the content of any specific module
-version from one day to the next. See 'go help module-auth' for details.
+The go command can fetch modules from a proxy or connect to source control
+servers directly, according to the setting of the GOPROXY environment
+variable (see 'go help env'). The default setting for GOPROXY is
+"https://proxy.golang.org", the Go module mirror run by Google.
+See https://proxy.golang.org/privacy for the service's privacy policy.
+If GOPROXY is set to the string "direct", downloads use a direct connection
+to source control servers. Setting GOPROXY to "off" disallows downloading
+modules from any source. Otherwise, GOPROXY is expected to be a comma-separated
+list of the URLs of module proxies, in which case the go command will fetch
+modules from those proxies. For each request, the go command tries each proxy
+in sequence, only moving to the next if the current proxy returns a 404 or 410
+HTTP response. The string "direct" may appear in the proxy list,
+to cause a direct connection to be attempted at that point in the search.
+Any proxies listed after "direct" are never consulted.
-The go command can fetch modules from a proxy instead of connecting
-to source control systems directly, according to the setting of the GOPROXY
-environment variable.
+The GONOPROXY environment variable is a comma-separated list of
+glob patterns (in the syntax of Go's path.Match) of module path prefixes
+that should always be fetched directly, ignoring the GOPROXY setting.
+For example,
-See 'go help goproxy' for details about the proxy and also the format of
-the cached downloaded packages.
+ GONOPROXY=*.corp.example.com,rsc.io/private
+
+forces a direct connection to download modules with path prefixes matching
+either pattern, including "git.corp.example.com/xyzzy", "rsc.io/private",
+and "rsc.io/private/quux".
+
+The 'go env -w' command (see 'go help env') can be used to set these variables
+for future go command invocations.
+
+No matter the source of the modules, the go command checks downloads against
+known checksums, to detect unexpected changes in the content of any specific
+module version from one day to the next. This check first consults the current
+module's go.sum file but falls back to the Go checksum database.
+See 'go help module-auth' for details.
+
+See 'go help goproxy' for details about the proxy protocol and also
+the format of the cached downloaded packages.
Modules and vendoring
diff --git a/src/cmd/go/testdata/script/mod_sumdb_golang.txt b/src/cmd/go/testdata/script/mod_sumdb_golang.txt
index ca040c5..0eb0fc8 100644
--- a/src/cmd/go/testdata/script/mod_sumdb_golang.txt
+++ b/src/cmd/go/testdata/script/mod_sumdb_golang.txt
@@ -1,16 +1,16 @@
-[!net] skip
-
+# Test default GOPROXY and GOSUMDB
env GOPROXY=
env GOSUMDB=
go env GOPROXY
-stdout '^direct$'
+stdout '^https://proxy.golang.org$'
go env GOSUMDB
-stdout '^off$'
+stdout '^sum.golang.org$'
env GOPROXY=https://proxy.golang.org
go env GOSUMDB
stdout '^sum.golang.org$'
# download direct from github
+[!net] skip
env GOSUMDB=sum.golang.org
env GOPROXY=direct
go get -m rsc.io/quote
