blob: e788cb79dfb2d852d8f23be3177f04831b835860 [file] [log] [blame] [view]
Bryan C. Mills0d7c8e02018-07-16 15:50:05 -04001# Go 1.11 Modules
Bryan C. Mills75f81732018-07-16 14:15:15 -04002
thepudds720dd652018-09-23 15:05:57 -04003Go 1.11 includes preliminary support for versioned modules as proposed [here](https://golang.org/design/24301-versioned-go). Modules are an [experimental](https://research.swtch.com/vgo-accepted) opt-in feature in Go 1.11, with the hope of incorporating feedback and finalizing the feature for Go 1.12. Even though the details may change, future releases will support modules defined using Go 1.11 or `vgo`.
Bryan C. Mills75f81732018-07-16 14:15:15 -04004
thepudds720dd652018-09-23 15:05:57 -04005The recent work by the Go team on versioned modules started outside of the main Go repository with the `vgo` tool, but on July 12, 2018 support for versioned modules landed in the main repository ([announcement thread](https://groups.google.com/d/msg/golang-dev/a5PqQuBljF4/61QK4JdtBgAJ)), and [Go 1.11 was released](https://groups.google.com/d/msg/golang-nuts/-yv9VlfsFCg/lPX_DUJnEQAJ) on August 24, 2018.
thepuddsf398d512018-07-22 18:10:33 -04006
thepudds720dd652018-09-23 15:05:57 -04007Please provide feedback on modules via [existing or new issues](https://github.com/golang/go/wiki/Modules#github-issues) and via [experience reports](https://github.com/golang/go/wiki/ExperienceReports).
thepudds634f3412018-07-28 15:27:09 -04008
thepuddsfc0364b2018-08-02 22:10:15 -04009## Table of Contents
10
thepuddse4474f22018-08-02 22:26:17 -040011The remaining content on this page is organized as follows:
thepudds720dd652018-09-23 15:05:57 -040012* [Quick Start Example](https://github.com/golang/go/wiki/Modules#quick-start-example)
thepuddsfc0364b2018-08-02 22:10:15 -040013* [New Concepts](https://github.com/golang/go/wiki/Modules#new-concepts)
thepuddsfc0364b2018-08-02 22:10:15 -040014 * [Modules](https://github.com/golang/go/wiki/Modules#modules)
thepudds0e8012e2018-08-05 12:24:11 -040015 * [go.mod](https://github.com/golang/go/wiki/Modules#gomod)
thepuddsfc0364b2018-08-02 22:10:15 -040016 * [Version Selection](https://github.com/golang/go/wiki/Modules#version-selection)
thepudds14331902018-08-05 12:09:17 -040017 * [Semantic Import Versioning](https://github.com/golang/go/wiki/Modules#semantic-import-versioning)
thepudds885ec5c2018-09-24 21:46:42 -040018* [How to Use Modules](https://github.com/golang/go/wiki/Modules#how-to-use-modules)
19 * [How to Install and Activate Module Support](https://github.com/golang/go/wiki/Modules#how-to-install-and-activate-module-support)
20 * [How to Define a Module](https://github.com/golang/go/wiki/Modules#how-to-define-a-module)
21 * [How to Upgrade and Downgrade Dependencies](https://github.com/golang/go/wiki/Modules#how-to-upgrade-and-downgrade-dependencies)
22 * [How to Prepare for a Release](https://github.com/golang/go/wiki/Modules#how-to-prepare-for-a-release)
thepuddsfc0364b2018-08-02 22:10:15 -040023* [Additional Resources](https://github.com/golang/go/wiki/Modules#additional-resources)
24* [Changes Since the Initial Vgo Proposal](https://github.com/golang/go/wiki/Modules#changes-since-the-initial-vgo-proposal)
25* [GitHub Issues](https://github.com/golang/go/wiki/Modules#github-issues)
thepudds885ec5c2018-09-24 21:46:42 -040026* [FAQs — Most Common](https://github.com/golang/go/wiki/Modules#faqs-most-common)
thepuddsf4456932018-09-07 01:51:55 -040027 * [How are versions marked as incompatible?](https://github.com/golang/go/wiki/Modules#how-are-versions-marked-as-incompatible)
28 * [When do I get old behavior vs. new module-based behavior?](https://github.com/golang/go/wiki/Modules#when-do-i-get-old-behavior-vs-new-module-based-behavior)
29 * [Why does installing a tool via 'go get' fail with error 'cannot find main module'?](https://github.com/golang/go/wiki/Modules#why-does-installing-a-tool-via-go-get-fail-with-error-cannot-find-main-module)
30 * [How can I track tool dependencies for a module?](https://github.com/golang/go/wiki/Modules#how-can-i-track-tool-dependencies-for-a-module)
thepudds122bf9b2018-09-07 02:04:45 -040031 * [What is the status of module support in IDEs, editors and standard tools like goimports, gorename, etc.?](https://github.com/golang/go/wiki/Modules#what-is-the-status-of-module-support-in-ides-editors-and-standard-tools-like-goimports-gorename-etc)
thepudds885ec5c2018-09-24 21:46:42 -040032* [FAQs — Additional Control](https://github.com/golang/go/wiki/Modules#faqs-additional-control)
thepuddsf4456932018-09-07 01:51:55 -040033 * [What community tooling exists for working with modules?](https://github.com/golang/go/wiki/Modules#what-community-tooling-exists-for-working-with-modules)
thepudds122bf9b2018-09-07 02:04:45 -040034 * [When should I use the 'replace' directive?](https://github.com/golang/go/wiki/Modules#when-should-i-use-the-replace-directive)
thepuddsf4456932018-09-07 01:51:55 -040035 * [Can I work entirely outside of VCS on my local filesystem?](https://github.com/golang/go/wiki/Modules#can-i-work-entirely-outside-of-vcs-on-my-local-filesystem)
36 * [How do I use vendoring with modules? Is vendoring going away?](https://github.com/golang/go/wiki/Modules#how-do-i-use-vendoring-with-modules-is-vendoring-going-away)
thepudds122bf9b2018-09-07 02:04:45 -040037 * [Are there "always on" module repositories and enterprise proxies?](https://github.com/golang/go/wiki/Modules#are-there-always-on-module-repositories-and-enterprise-proxies)
38 * [Can I control when go.mod gets updated and when the go tools use the network to satisfy dependencies?](https://github.com/golang/go/wiki/Modules#can-i-control-when-gomod-gets-updated-and-when-the-go-tools-use-the-network-to-satisfy-dependencies)
thepuddsf4456932018-09-07 01:51:55 -040039 * [How do I use modules with CI systems such as Travis or CircleCI?](https://github.com/golang/go/wiki/Modules#how-do-i-use-modules-with-ci-systems-such-as-travis-or-circleci)
thepudds885ec5c2018-09-24 21:46:42 -040040* [FAQs — go.mod and go.sum](https://github.com/golang/go/wiki/Modules#faqs-gomod-and-gosum)
thepudds58207da2018-09-17 09:57:11 -040041 * [Why does 'go mod tidy' record indirect and test dependencies in my 'go.mod'?](https://github.com/golang/go/wiki/Modules#why-does-go-mod-tidy-record-indirect-and-test-dependencies-in-my-gomod)
thepudds7f6103b2018-09-17 10:03:25 -040042 * [Is 'go.sum' a lock file? Why does 'go.sum' include information for module versions I am no longer using?](https://github.com/golang/go/wiki/Modules/#is-gosum-a-lock-file-why-does-gosum-include-information-for-module-versions-i-am-no-longer-using)
thepuddsd86f5aa2018-09-17 10:21:31 -040043 * [Should I still add a 'go.mod' file if I do not have any dependencies?](https://github.com/golang/go/wiki/Modules#should-i-still-add-a-gomod-file-if-i-do-not-have-any-dependencies)
thepudds885ec5c2018-09-24 21:46:42 -040044* [FAQs — Semantic Import Versioning](https://github.com/golang/go/wiki/Modules#faqs-semantic-import-versioning)
thepudds885ec5c2018-09-24 21:46:42 -040045 * [Why must major version numbers appear in import paths?](https://github.com/golang/go/wiki/Modules#additional-frequently-asked-questions)
46 * [Why are major versions v0, v1 omitted from import paths?](https://github.com/golang/go/wiki/Modules#additional-frequently-asked-questions)
thepudds7afc40d2018-09-24 22:06:51 -040047 * [What are some implications of tagging my project with major version v0, v1, or making breaking changes with v2+?](https://github.com/golang/go/wiki/Modules#what-are-some-implications-of-tagging-my-project-with-major-version-v0-v1-or-making-breaking-changes-with-v2)
thepudds885ec5c2018-09-24 21:46:42 -040048* [FAQs — Minimal Version Selection](https://github.com/golang/go/wiki/Modules#faqs-minimal-version-selection)
49 * [Won't minimal version selection keep developers from getting important updates?](https://github.com/golang/go/wiki/Modules#additional-frequently-asked-questions)
50* [FAQs — Possible Problems](https://github.com/golang/go/wiki/Modules#faqs-possible-problems)
thepudds122bf9b2018-09-07 02:04:45 -040051 * [Why does 'go build' require gcc, and why are prebuilt packages such as net/http not used?](https://github.com/golang/go/wiki/Modules#why-does-go-build-require-gcc-and-why-are-prebuilt-packages-such-as-nethttp-not-used)
thepuddsf4456932018-09-07 01:51:55 -040052 * [Do modules work with relative imports like `import "./subdir"`?](https://github.com/golang/go/wiki/Modules#do-modules-work-with-relative-imports-like-import-subdir)
thepudds885ec5c2018-09-24 21:46:42 -040053* [FAQs — Miscellaneous](https://github.com/golang/go/wiki/Modules#faqs-miscellaneous)
thepudds122bf9b2018-09-07 02:04:45 -040054 * [How did the 'go mod' commands change in go1.11beta3?](https://github.com/golang/go/wiki/Modules#how-did-the-go-mod-commands-change-in-go111beta3)
thepuddsfc0364b2018-08-02 22:10:15 -040055
thepudds720dd652018-09-23 15:05:57 -040056## Quick Start Example
Bryan C. Mills75f81732018-07-16 14:15:15 -040057
thepudds720dd652018-09-23 15:05:57 -040058The details are covered in the remainder of this page, but here is a simple example of creating a module from scratch.
Bryan C. Mills0d7c8e02018-07-16 15:50:05 -040059
thepudds720dd652018-09-23 15:05:57 -040060Create a directory outside of your GOPATH:
AJ ONeal21fc7f12018-09-20 14:06:57 -060061```
thepudds720dd652018-09-23 15:05:57 -040062$ mkdir -p /tmp/scratchpad/hello
63$ cd /tmp/scratchpad/hello
AJ ONeal21fc7f12018-09-20 14:06:57 -060064```
65
thepudds720dd652018-09-23 15:05:57 -040066Initialize a new module:
AJ ONeal21fc7f12018-09-20 14:06:57 -060067```
thepudds720dd652018-09-23 15:05:57 -040068$ go mod init github.com/you/hello
69
70go: creating new go.mod: module github.com/you/hello
71```
72
73Write your code:
74```
75$ cat <<EOF > hello.go
76package main
77
78import (
79 "fmt"
80 "rsc.io/quote"
81)
82
83func main() {
84 fmt.Println(quote.Hello())
85}
86EOF
87```
88
89Build and run:
90```
91$ go build
92$ ./hello
93
94Hello, world.
95```
96
97Note your `go.mod` file includes explicit versions for your dependencies:
98```
99$ cat go.mod
100
101module github.com/you/hello
102
103require rsc.io/quote v1.5.2
AJ ONeal21fc7f12018-09-20 14:06:57 -0600104```
105
Bryan C. Mills75f81732018-07-16 14:15:15 -0400106## New Concepts
107
thepudds345f5622018-09-06 08:55:00 -0400108These sections provide a high-level introduction to the main new concepts. For more details and rationale, please see [the official proposal document](https://golang.org/design/24301-versioned-go), this 40-minute introductory [video by Russ Cox describing the philosophy behind the design](https://www.youtube.com/watch?v=F8nrpe0XWRg&list=PLq2Nv-Sh8EbbIjQgDzapOFeVfv5bGOoPE&index=3&t=0s), or the more detailed initial [vgo blog series](https://research.swtch.com/vgo).
thepudds0e8012e2018-08-05 12:24:11 -0400109
Bryan C. Mills75f81732018-07-16 14:15:15 -0400110### Modules
111
Baruch Sadogursky43df1132018-08-28 07:18:04 -0600112A *module* is a collection of related Go packages that are versioned together as a single unit. Most often, a single version-control repository corresponds exactly to a single module, but alternatively, a single version-control repository can hold multiple modules.
Bryan C. Mills75f81732018-07-16 14:15:15 -0400113
thepuddsdef8ca72018-08-28 17:26:39 -0400114Modules record precise dependency requirements and create reproducible builds.
thepuddscc84ca22018-08-28 17:24:14 -0400115
thepuddsdef8ca72018-08-28 17:26:39 -0400116Modules must be [semantically versioned](https://semver.org/) in the form `v(major).(minor).(patch)`, such as `v0.1.0`, `v1.2.3`, or `v3.0.1`. The leading `v` is required. If using Git, [tag](https://git-scm.com/book/en/v2/Git-Basics-Tagging) released commits with their versions. Public and private module repositories and proxies are becoming available (see for example FAQ [below](https://github.com/golang/go/wiki/Modules#are-there-always-on-module-repositories-and-enterprise-proxies)).
thepudds80d05d32018-08-13 07:38:09 -0400117
thepudds0e8012e2018-08-05 12:24:11 -0400118### go.mod
119
thepuddsdef8ca72018-08-28 17:26:39 -0400120A module is defined by a tree of Go source files with a `go.mod` file in the tree's root directory. Module source code may be located outside of GOPATH.
thepudds0e8012e2018-08-05 12:24:11 -0400121
thepudds8ff45442018-08-26 12:45:52 -0400122All of the packages in a module share a common prefix -- the *module path*. The `go.mod` file defines the module path via the `module` directive. For example, if you are defining a module for two packages `example.com/my/thing/foo` and `example.com/my/thing/bar`, the first line in your `go.mod` file typically would be `module example.com/my/thing`.
thepudds0e8012e2018-08-05 12:24:11 -0400123
thepuddse508c052018-08-15 12:57:00 -0400124Module files may include comments and will look familiar to a Go programmer. Here is an example `go.mod` file:
thepudds0e8012e2018-08-05 12:24:11 -0400125
Bryan C. Mills75f81732018-07-16 14:15:15 -0400126```
thepudds72779d82018-08-15 12:48:53 -0400127module github.com/my/module/v3
Bryan C. Mills75f81732018-07-16 14:15:15 -0400128
129require (
thepudds72779d82018-08-15 12:48:53 -0400130 github.com/some/dependency v1.2.3
131 github.com/another/dependency v0.1.0
thepudds8ff45442018-08-26 12:45:52 -0400132 github.com/additional/dependency/v4 v4.0.0
Bryan C. Mills75f81732018-07-16 14:15:15 -0400133)
134```
135
thepuddsc53293c2018-08-15 13:05:09 -0400136There are four directives: `module`, `require`, `exclude`, `replace`.
Bryan C. Mills75f81732018-07-16 14:15:15 -0400137
thepudds343420b2018-09-07 09:59:22 -0400138`exclude` and `replace` directives only operate on the current (“main”) module. `exclude` and `replace` directives in modules other than the main module are ignored when building the main module. The `replace` and `exclude` statements therefore allow the main module complete control over its own build, without also being subject to complete control by dependencies. (See FAQ [below](https://github.com/golang/go/wiki/Modules#when-should-i-use-the-replace-directive) for an example of using `replace` directives).
Bryan C. Mills75f81732018-07-16 14:15:15 -0400139
thepudds0e8012e2018-08-05 12:24:11 -0400140In Go source code, packages are imported using the full path including the module, for example:
thepuddse508c052018-08-15 12:57:00 -0400141 * `import "example.com/my/module/v2/pkg/foo"` to import `foo` from the v2 version of module `example.com/my/module`.
Bryan C. Mills75f81732018-07-16 14:15:15 -0400142
Bryan C. Mills0d7c8e02018-07-16 15:50:05 -0400143### Version Selection
Bryan C. Mills75f81732018-07-16 14:15:15 -0400144
thepudds0e8012e2018-08-05 12:24:11 -0400145If you add a new import to your source code that is not yet covered by a `require` in `go.mod`, any go command run (e.g., 'go build') will automatically look up the proper module and add the *highest* version of that new direct dependency to your module's `go.mod` as a `require` directive. For example, if your new import corresponds to dependency M whose latest tagged release version is `v1.2.3`, your module's `go.mod` will end up with `require M v1.2.3`, which indicates module M is a dependency with allowed version >= v1.2.3 (and < v2, given v2 is considered incompatible with v1).
thepudds5af74db2018-08-01 18:14:36 -0400146
thepudds7806bc72018-08-29 00:16:41 -0400147The *minimal version selection* algorithm is used to select the versions of all modules used in a build. For each module in a build, the version selected by minimal version selection is always the semantically *highest* of the versions explicitly listed by a `require` directive in the main module or one of its dependencies.
Bryan C. Mills75f81732018-07-16 14:15:15 -0400148
thepudds738d95c2018-08-29 08:37:13 -0400149As an example, if your module depends on module A which has a `require D v1.0.0`, and your module also depends on module B which has a `require D v1.1.1`, then minimal version selection would choose `v1.1.1` of D to include in the build (given it is the highest listed `require` version). This selection of `v1.1.1` remains consistent even if some time later a `v1.2.0` of D becomes available. This is an example of how the modules system provides 100% reproducible builds. When ready, the module author or user might choose to upgrade to the latest available version of D or choose an explicit version for D.
thepuddsc164d5a2018-08-03 13:04:28 -0400150
thepudds91ffcea2018-08-02 23:55:11 -0400151For a brief rationale and overview of the minimal version selection algorithm, [see the "High Fidelity Builds" section](https://github.com/golang/proposal/blob/master/design/24301-versioned-go.md#update-timing--high-fidelity-builds) of the official proposal, or see the [more detailed `vgo` blog series](https://research.swtch.com/vgo).
thepudds5077edf2018-08-02 09:01:12 -0400152
thepudds6ae9d6f2018-08-15 15:52:57 -0400153To see a list of the selected module versions (including indirect dependencies), use `go list -m all`.
Bryan C. Mills75f81732018-07-16 14:15:15 -0400154
thepudds0168d012018-08-02 22:24:15 -0400155See also the ["How to Upgrade and Downgrade Dependencies"](https://github.com/golang/go/wiki/Modules#how-to-upgrade-and-downgrade-dependencies) section below and the ["How are versions marked as incompatible?"](https://github.com/golang/go/wiki/Modules#how-are-versions-marked-as-incompatible) FAQ below.
thepudds5af74db2018-08-01 18:14:36 -0400156
thepudds14331902018-08-05 12:09:17 -0400157### Semantic Import Versioning
158
thepudds01c74342018-08-25 10:49:21 -0400159For many years, the official Go FAQ has included this advice on package versioning:
160
161> "Packages intended for public use should try to maintain backwards compatibility as they evolve. The Go 1 compatibility guidelines are a good reference here: don't remove exported names, encourage tagged composite literals, and so on. If different functionality is required, add a new name instead of changing an old one. If a complete break is required, create a new package with a new import path."
162
163The last sentence is especially important. With Go 1.11 modules, that advice is formalized into the _import compatibility rule_:
164
165> "If an old package and a new package have the same import path,
166> the new package must be backwards compatible with the old package."
167
168Recall [semantic versioning](https://semver.org/) requires a major version change when a v1 or higher package makes a backwards incompatible change. The result of following both the import compatibility rule and semantic versioning is called _semantic import versioning_.
169
thepudds05465dd2018-09-23 16:22:19 -0400170As a result of semantic import versioning, code opting in to Go modules **must comply with these rules**:
thepudds23d29522018-08-25 13:00:46 -0400171* Follow [semantic versioning](https://semver.org/) (with tags such as `v1.2.3`).
thepudds656e8352018-08-21 15:32:45 -0400172* If the module is version v2 or higher, the major version of the module _must_ be included in both the module path in the `go.mod` file (e.g., `module example.com/my/mod/v2`) and the package import path (e.g., `import "example.com/my/mod/v2/foo"`).
173* If the module is version v0 or v1, do _not_ include the major version in either the module path or the import path.
174
thepudds05465dd2018-09-23 16:22:19 -0400175In general, packages with different import paths are different packages, including if different import paths are due to different major versions. Thus `example.com/my/mod/foo` is a different package than `example.com/my/mod/v2/foo`, and both may be imported in a single build, which among other benefits helps with diamond dependency problems and also allows a v1 module to be implemented in terms of its v2 replacement or vice versa.
thepudds14331902018-08-05 12:09:17 -0400176
thepuddsaa43ea42018-08-15 11:52:47 -0400177See the ["Module compatibility and semantic versioning"](https://tip.golang.org/cmd/go/#hdr-Module_compatibility_and_semantic_versioning) section of the tip documentation for more details.
178
thepudds05465dd2018-09-23 16:22:19 -0400179This section so far has been focused on code that opts in to modules. However, putting major versions in import paths for v2+ modules could create incompatibilities with older versions of Go, or with code that has not yet opted in to modules. To help with this, Go versions 1.9.7+, 1.10.3+ and 1.11 have been [updated](https://go-review.googlesource.com/c/go/+/109340) so that code built with those releases can properly consume v2+ modules without requiring modification of pre-existing code. (When relying on this updated mechanism, a package that has _not_ opted in to modules would _not_ include the major version in the import path for any imported v2+ modules. In contrast, a package that _has_ opted in to modules _must_ include the major version in the import path for any imported v2+ modules).
thepudds0e8012e2018-08-05 12:24:11 -0400180
thepudds05465dd2018-09-23 16:22:19 -0400181For the exact mechanics required to release a v2+ module, please see the ["Releasing Modules (v2 or Higher)"](https://github.com/golang/go/wiki/Modules#releasing-modules-v2-or-higher) section below.
thepudds14331902018-08-05 12:09:17 -0400182
thepudds885ec5c2018-09-24 21:46:42 -0400183## How to Use Modules
184
185### How to Install and Activate Module Support
thepudds720dd652018-09-23 15:05:57 -0400186
187To use modules, two install options are:
188* [Install the latest Go 1.11 release](https://golang.org/dl/).
189* [Install the Go toolchain from source](https://golang.org/doc/install/source) on the `master` branch.
190
191Once installed, you can then activate module support in one of two ways:
192* Invoke the `go` command in a directory outside of the `$GOPATH/src` tree, with a valid `go.mod` file in the current directory or any parent of it and the environment variable `GO111MODULE` unset (or explicitly set to `auto`).
193* Invoke the `go` command with `GO111MODULE=on` environment variable set.
194
thepudds885ec5c2018-09-24 21:46:42 -0400195### How to Define a Module
Bryan C. Mills75f81732018-07-16 14:15:15 -0400196
thepuddsfd489a12018-08-01 02:47:49 -0400197To create a `go.mod` for an existing project:
Bryan C. Mills75f81732018-07-16 14:15:15 -0400198
thepudds6f427542018-07-31 13:33:16 -04001991. Navigate to the root of the module's source tree and activate module mode in the `go` command:
Bryan C. Mills75f81732018-07-16 14:15:15 -0400200
Bryan C. Mills0d7c8e02018-07-16 15:50:05 -0400201 ```
202 $ cd $GOPATH/src/<project path>
203 $ export GO111MODULE=on
204 ```
Bryan C. Mills75f81732018-07-16 14:15:15 -0400205
thepudds6f427542018-07-31 13:33:16 -0400206 or:
Bryan C. Mills75f81732018-07-16 14:15:15 -0400207
Bryan C. Mills0d7c8e02018-07-16 15:50:05 -0400208 ```
209 $ cd <project path outside $GOPATH/src>
210 ```
Bryan C. Mills75f81732018-07-16 14:15:15 -0400211
thepudds6f427542018-07-31 13:33:16 -04002122. Create the initial module definition and write it to the `go.mod` file:
Bryan C. Mills75f81732018-07-16 14:15:15 -0400213
Bryan C. Mills0d7c8e02018-07-16 15:50:05 -0400214 ```
thepudds6f434ef2018-08-13 13:29:51 -0400215 $ go mod init
Bryan C. Mills0d7c8e02018-07-16 15:50:05 -0400216 ```
Bryan C. Mills75f81732018-07-16 14:15:15 -0400217
thepudds9bdb2722018-07-27 07:02:48 -0400218 This step converts from any existing [`dep`](https://github.com/golang/dep) `Gopkg.lock` file or from any of the other [nine total supported dependency formats](https://tip.golang.org/pkg/cmd/go/internal/modconv/?m=all#pkg-variables), adding require statements to match the existing configuration.
Bryan C. Mills75f81732018-07-16 14:15:15 -0400219
thepudds8d39bc32018-08-13 13:34:49 -0400220 If `go mod` cannot automatically determine an appropriate module path (e.g., if running outside of VCS), or if you need to otherwise override that path, you can supply the module path as follows:
Bryan C. Mills75f81732018-07-16 14:15:15 -0400221
Bryan C. Mills0d7c8e02018-07-16 15:50:05 -0400222 ```
thepuddsf1054312018-08-01 01:30:34 -0400223 $ go mod init example.com/my/module/v2
Bryan C. Mills0d7c8e02018-07-16 15:50:05 -0400224 ```
Bryan C. Mills75f81732018-07-16 14:15:15 -0400225
thepudds74241a12018-07-31 13:31:00 -04002263. Build the module. This will automatically add missing or unconverted dependencies as needed to satisfy imports for this particular build invocation:
Bryan C. Mills0d7c8e02018-07-16 15:50:05 -0400227
228 ```
thepudds74241a12018-07-31 13:31:00 -0400229 $ go build ./...
Bryan C. Mills0d7c8e02018-07-16 15:50:05 -0400230 ```
thepudds6f427542018-07-31 13:33:16 -04002314. Test the module as configured to ensure that it works with the selected versions:
Bryan C. Mills0d7c8e02018-07-16 15:50:05 -0400232
233 ```
234 $ go test ./...
235 ```
236
thepuddsd5e66ee2018-08-15 13:20:27 -04002375. (Optional) Run the tests for your module plus the tests for all direct and indirect dependencies to check for incompatibilities:
Bryan C. Mills0d7c8e02018-07-16 15:50:05 -0400238
239 ```
thepudds4b3d8932018-07-27 06:50:53 -0400240 $ go test all
Bryan C. Mills0d7c8e02018-07-16 15:50:05 -0400241 ```
Bryan C. Mills75f81732018-07-16 14:15:15 -0400242
thepudds0168d012018-08-02 22:24:15 -0400243Prior to tagging a release, see the ["How to Prepare for a Release"](https://github.com/golang/go/wiki/Modules#how-to-prepare-for-a-release) section below.
thepudds74241a12018-07-31 13:31:00 -0400244
thepudds432bda32018-08-07 05:22:33 -0400245For more information on all of these topics, the primary entry point to the official modules documentation is [available on tip.golang.org](https://tip.golang.org/cmd/go/#hdr-Modules__module_versions__and_more).
246
thepudds88919642018-08-02 21:51:04 -0400247## How to Upgrade and Downgrade Dependencies
Bryan C. Mills75f81732018-07-16 14:15:15 -0400248
thepudds4b3d8932018-07-27 06:50:53 -0400249Day-to-day adding, removing, upgrading, and downgrading of dependencies should be done using 'go get', which will automatically update the `go.mod` file.
250
thepudds61c12ce2018-07-27 08:04:34 -0400251In addition, go commands like 'go build', 'go test', or even 'go list' will automatically add new dependencies as needed to satisfy imports (updating `go.mod` and downloading the new dependencies).
thepuddsb05a8442018-07-26 08:18:42 -0400252
thepudds927647d2018-07-26 09:03:39 -0400253To upgrade to the latest version for all transitive dependencies of the current module:
thepudds5af74db2018-08-01 18:14:36 -0400254 * run `go get -u` to use the latest *minor or patch* releases
255 * run `go get -u=patch` to use the latest *patch* releases
thepuddsee879c52018-07-25 12:08:34 -0400256
thepudds8f75c6c2018-08-01 09:18:37 -0400257To upgrade or downgrade to a more specific version, 'go get' allows version selection to be overridden by adding an @version suffix or "module query" to the package argument, such as `go get github.com/gorilla/mux@v1.6.2`, `go get github.com/gorilla/mux@e3702bed2`, or `go get github.com/gorilla/mux@'<v1.6.2'`.
258
thepuddsff8f0f82018-08-25 13:07:59 -0400259`go get github.com/gorilla/mux` obtains the latest version with a [semver](https://semver.org/) tag. Using a branch name such as `go get github.com/gorilla/mux@master` is one way to obtain the latest commit regardless of whether or not it has a semver tag.
260
261In general, module queries that do not resolve to a semver tag will be recorded as [pseudo-versions](https://tip.golang.org/cmd/go/#hdr-Pseudo_versions) in the `go.mod` file.
thepudds72779d82018-08-15 12:48:53 -0400262
thepudds90d81be2018-08-26 12:21:40 -0400263Modules are capable of consuming packages that have not yet opted into modules. Modules can also consume packages that do not yet have any proper semver tags (in which case they will be recorded using pseudo-versions in `go.mod`).
264
thepudds8f75c6c2018-08-01 09:18:37 -0400265See the ["Module-aware go get"](https://tip.golang.org/cmd/go/#hdr-Module_aware_go_get) and ["Module queries"](https://tip.golang.org/cmd/go/#hdr-Module_queries) sections of the tip documentation for more information on the topics here.
thepuddsb05a8442018-07-26 08:18:42 -0400266
thepuddsd5e66ee2018-08-15 13:20:27 -0400267After upgrading or downgrading any dependencies, you may then want to run the tests again for all packages in your build (including direct and indirect dependencies) to check for incompatibilities:
thepuddsee879c52018-07-25 12:08:34 -0400268 ```
thepudds4b3d8932018-07-27 06:50:53 -0400269 $ go test all
thepuddsee879c52018-07-25 12:08:34 -0400270 ```
thepuddsf398d512018-07-22 18:10:33 -0400271
thepudds88919642018-08-02 21:51:04 -0400272## How to Prepare for a Release
thepudds61aba5c2018-07-27 07:10:40 -0400273
thepudds05465dd2018-09-23 16:22:19 -0400274### Releasing Modules (All Versions)
275
thepuddsdf13b282018-08-01 00:38:40 -0400276Best practices for creating a release of a module are expected to emerge as part of the initial modules experiment. Many of these might end up being automated by a [future 'go release' tool](https://github.com/golang/go/issues/26420).
thepudds61aba5c2018-07-27 07:10:40 -0400277
thepudds88919642018-08-02 21:51:04 -0400278Some current suggested best practices to consider prior to tagging a release:
thepudds61aba5c2018-07-27 07:10:40 -0400279
thepudds885ec5c2018-09-24 21:46:42 -0400280* Run `go mod tidy` to possibly prune any extraneous requirements (as described [here](https://tip.golang.org/cmd/go/#hdr-Maintaining_module_requirements)) and also ensure your current go.mod reflects all possible build tags/OS/architecture combinations (as described [here](https://github.com/golang/go/issues/25971#issuecomment-399091682)).
thepuddscb4e8b52018-08-28 15:01:45 -0400281 * In contrast, other commands like `go build` and `go test` will not remove dependencies from `go.mod` that are no longer required and only update `go.mod` based on the current build invocation's tags/OS/architecture.
thepuddsf918d5c2018-07-31 13:36:22 -0400282
thepuddsd8674ab2018-08-15 15:22:10 -0400283* Run `go test all` to test your module (including running the tests for your direct and indirect dependencies) as a way of validating that the currently selected packages versions are compatible.
thepuddsd5e66ee2018-08-15 13:20:27 -0400284 * The number of possible version combinations is exponential in the number of modules, so in general you cannot expect your dependencies to have tested against all possible combinations of their dependencies.
thepudds61aba5c2018-07-27 07:10:40 -0400285 * As part of the modules work, `go test all` has been [re-defined to be more useful](https://research.swtch.com/vgo-cmd) to include all the packages in the current module, plus all the packages they depend on through a sequence of one or more imports, while excluding packages that don't matter in the current module.
thepudds61aba5c2018-07-27 07:10:40 -0400286
thepuddsabb30ff2018-08-15 09:32:02 -0400287* Typically the `go.sum` file should be checked in along with the `go.mod` file.
288 * `go.sum` contains the expected cryptographic checksums of the content of specific module versions.
thepudds3d101f02018-08-15 14:19:10 -0400289 * If someone clones your repository and downloads your dependencies using the go command, they will receive an error if there is any mismatch between their downloaded copies of your dependencies and the corresponding entries in your `go.sum`.
290 * In addition, `go mod verify` checks that the on-disk cached copies of module downloads still match the entries in `go.sum`.
thepudds2b5afbd2018-08-29 00:35:43 -0400291 * Note that `go.sum` is not a lock file as used in some alternative dependency management systems. (`go.mod` provides enough information for reproducible builds).
thepuddsb98d7cf2018-08-15 09:39:44 -0400292 * See the ["Module downloading and verification"](https://tip.golang.org/cmd/go/#hdr-Module_downloading_and_verification) section of the tip documentation for more details. See very brief [rationale here](https://twitter.com/FiloSottile/status/1029404663358087173). See possible future extensions being discussed for example in [#24117](https://github.com/golang/go/issues/24117) and [#25530](https://github.com/golang/go/issues/25530).
thepuddsabb30ff2018-08-15 09:32:02 -0400293
thepudds05465dd2018-09-23 16:22:19 -0400294### Releasing Modules (v2 or Higher)
295
296If you are releasing a v2 or higher module, please first review the discussion in the ["Semantic Import Versioning" ](https://github.com/golang/go/wiki/Modules#semantic-import-versioning) section above, which includes why major versions are included in the module path and import path for v2+ modules, as well as how Go versions 1.9.7+ and 1.10.3+ have been updated to simplify that transition.
297
298There are two ways to release a v2 or higher module. Using the example of creating a `v3.0.0` release, the two options are:
299
3001. **Most common approach**: Update the `go.mod` file to include a `/v3` at the end of the module path. Update import statements within the module to also use `/v3` (e.g., `import "github.com/my/module/v3/foo"`). Tag the release with `v3.0.0`.
301 * Go versions 1.9.7+, 1.10.3+, and 1.11 are able to properly consume and build a v2+ module created using this approach without requiring updates to consumer code that has not yet opted in to modules (as described in the the ["Semantic Import Versioning"](https://github.com/golang/go/wiki/Modules#semantic-import-versioning) section above).
302 * A community tool [github.com/marwan-at-work/mod](https://github.com/marwan-at-work/mod) helps automate this procedure. See the [repository](https://github.com/marwan-at-work/mod) or the [community tooling FAQ](https://github.com/golang/go/wiki/Modules#what-community-tooling-exists-for-working-with-modules) below for an overview.
303 * To avoid confusion with this approach, consider putting the `v3.*.*` commits for the module on a separate v3 branch.
304 * If instead you have been previously releasing on master and would prefer to tag `v3.0.0` on master, that is a viable option, but consider creating a v1 branch for any future v1 bug fixes.
305
3062. **Less common approach**: Create a new `v3` subdirectory (e.g., `my/module/v3`) and place a new `go.mod` file in that subdirectory. The module path must end with `/v3`. Copy or move the code into the `v3` subdirectory. Update import statements within the module to also use `/v3` (e.g., `import "github.com/my/module/v3/foo"`). Tag the release with `v3.0.0`.
307 * This provides greater backwards compatibility. In particular, Go versions older than 1.9.7 and 1.10.3 are also able to properly consume and build a v2+ module created using this approach.
thepuddsd0ef5ad2018-08-15 12:07:14 -0400308
thepudds8390d032018-07-22 23:37:00 -0400309## Additional Resources
thepuddsf398d512018-07-22 18:10:33 -0400310
thepudds8390d032018-07-22 23:37:00 -0400311### Documentation and Proposal
312
thepuddsfd489a12018-08-01 02:47:49 -0400313* Official documentation:
314 * Latest [HTML documentation for modules on tip.golang.org](https://tip.golang.org/cmd/go/#hdr-Modules__module_versions__and_more)
315 * Run `go help modules` for more about modules. (This is the main entry point for modules topics via `go help`)
316 * Run `go help mod` for more about the `go mod` command.
317 * Run `go help module-get` for more about the behavior of `go get` when in module-aware mode.
318 * Run `go help goproxy` for more about the module proxy, including a pure file-based option via a `file:///` URL.
thepudds592f7432018-07-26 10:03:47 -0400319* The initial ["Go & Versioning"](https://research.swtch.com/vgo) series of blog posts on `vgo` by Russ Cox (first posted February 20, 2018)
320* Official [golang.org blog post introducing the proposal](https://blog.golang.org/versioning-proposal) (March 26, 2018)
321 * This provides a more succinct overview of the proposal than the full `vgo` blog series, along with some of the history and process behind the proposal
thepudds8390d032018-07-22 23:37:00 -0400322* Official [Versioned Go Modules Proposal](https://golang.org/design/24301-versioned-go) (last updated March 20, 2018)
323
324### Introductory Material
325
thepudds7afc40d2018-09-24 22:06:51 -0400326* Example based 35 minute introductory video ["What are Go modules and how do I use them?"](https://www.youtube.com/watch?v=6MbIzJmLz6Q&list=PL8QGElREVyDA2iDrPNeCe8B1u7li5S6ep&index=5&t=0s) ([slides](https://talks.godoc.org/github.com/myitcv/talks/2018-08-15-glug-modules/main.slide#1)) by Paul Jolly (August 15, 2018)
thepudds8390d032018-07-22 23:37:00 -0400327* Introductory blog post ["Taking Go Modules for a Spin"](https://dave.cheney.net/2018/07/14/taking-go-modules-for-a-spin) by Dave Cheney (July 14, 2018)
thepudds8390d032018-07-22 23:37:00 -0400328* Introductory [Go Meetup slides on modules](https://docs.google.com/presentation/d/1ansfXN8a_aVL-QuvQNY7xywnS78HE8aG7fPiITNQWMM/edit#slide=id.g3d87f3177d_0_0) by Chris Hines (July 16, 2018)
thepudds7afc40d2018-09-24 22:06:51 -0400329* Introductory 40 minute video ["The Principles of Versions in Go"](https://www.youtube.com/watch?v=F8nrpe0XWRg&list=PLq2Nv-Sh8EbbIjQgDzapOFeVfv5bGOoPE&index=3&t=0s) from GopherCon Singapore by Russ Cox (May 2, 2018)
thepudds8390d032018-07-22 23:37:00 -0400330 * Succinctly covers the philosophy behind the design of versioned Go modules, including the three core principles of "Compatibility", "Repeatability", and "Cooperation"
331
332### Additional Material
333
thepudds345f5622018-09-06 08:55:00 -0400334* Blog post ["Using Go modules with vendor support on Travis CI"](https://arslan.io/2018/08/26/using-go-modules-with-vendor-support-on-travis-ci/) by Fatih Arslan (August 26, 2018)
335* Blog post ["Go Modules and CircleCI"](https://medium.com/@toddkeech/go-modules-and-circleci-c0d6fac0b000) by Todd Keech (July 30, 2018)
thepudds592f7432018-07-26 10:03:47 -0400336* Blog post ["The vgo proposal is accepted. Now what?"](https://research.swtch.com/vgo-accepted) by Russ Cox (May 29, 2018)
thepudds8390d032018-07-22 23:37:00 -0400337 * Includes summary of what it means that versioned modules are currently an experimental opt-in feature
thepudds345f5622018-09-06 08:55:00 -0400338* Blog post on [how to build go from tip and start using go modules](https://carolynvanslyck.com/blog/2018/07/building-go-from-source/) by Carolyn Van Slyck (July 16, 2018)
thepudds8390d032018-07-22 23:37:00 -0400339
thepudds88919642018-08-02 21:51:04 -0400340## Changes Since the Initial Vgo Proposal
341
thepuddsa7d30542018-08-03 18:02:38 -0400342As part of the proposal, prototype, and beta processes, there have been over 400 issues created by the overall community. Please continue to supply feedback.
343
344Here is a partial list of some of the larger changes and improvements, almost all of which were primarily based on feedback from the community:
thepudds88919642018-08-02 21:51:04 -0400345
346* Top-level vendor support was retained rather than vgo-based builds ignoring vendor directories entirely ([discussion](https://groups.google.com/d/msg/golang-dev/FTMScX1fsYk/uEUSjBAHAwAJ), [CL](https://go-review.googlesource.com/c/vgo/+/118316))
thepudds0168d012018-08-02 22:24:15 -0400347* Backported minimal module-awareness to allow older Go versions 1.9.7+ and 1.10.3+ to more easily consume modules for v2+ projects ([discussion](https://github.com/golang/go/issues/24301#issuecomment-371228742), [CL](https://golang.org/cl/109340))
thepudds42f1dea2018-08-03 18:07:15 -0400348* Allowed vgo to use v2+ tags by default for pre-existing packages did not yet have a go.mod (recent update in related behavior described [here](https://github.com/golang/go/issues/25967#issuecomment-407567904))
thepudds0168d012018-08-02 22:24:15 -0400349* Added support via command `go get -u=patch` to update all transitive dependencies to the latest available patch-level versions on the same minor version ([discussion](https://research.swtch.com/vgo-cmd), [documentation](https://tip.golang.org/cmd/go/#hdr-Module_aware_go_get))
thepudds88919642018-08-02 21:51:04 -0400350* Additional control via environmental variables (e.g., GOFLAGS in [#26585](https://github.com/golang/go/issues/26585), [CL](https://go-review.googlesource.com/c/go/+/126656))
thepudds55989cc2018-08-05 20:39:02 -0400351* Finer grain control on whether or not go.mod is allowed to be updated, how vendor directory is used, and whether or not network access is allowed (e.g., -mod=readonly, -mod=vendor, GOPROXY=off; related [CL](https://go-review.googlesource.com/c/go/+/126696) for recent change)
thepudds0168d012018-08-02 22:24:15 -0400352* Added more flexible replace directives ([CL](https://go-review.googlesource.com/c/vgo/+/122400))
353* Added additional ways to interrogate modules (for human consumption, as well as for better editor / IDE integration)
354* The UX of the go CLI has continued to be refined based on experiences so far (e.g., [#26581](https://github.com/golang/go/issues/26581), [CL](https://go-review.googlesource.com/c/go/+/126655))
thepudds856618c2018-09-07 00:20:35 -0400355* Additional support for warming caches for use cases such as CI or docker builds via `go mod download` ([#26610](https://github.com/golang/go/issues/26610#issuecomment-408654653))
thepudds88919642018-08-02 21:51:04 -0400356* **Most likely**: better support for installing specific versions of programs to GOBIN ([#24250](https://github.com/golang/go/issues/24250#issuecomment-377553022))
357
thepudds88919642018-08-02 21:51:04 -0400358## GitHub Issues
thepudds8390d032018-07-22 23:37:00 -0400359
thepuddsdf13b282018-08-01 00:38:40 -0400360* [Currently open module issues](https://golang.org/issues?q=is%3Aopen+is%3Aissue+label:modules)
361* [Closed module issues](https://golang.org/issues?q=is%3Aclosed+is%3Aissue+label:modules)
362* [Closed vgo issues](https://github.com/golang/go/issues?page=3&q=-label%3Amodules+vgo+is%3Aclosed)
thepuddse01f40e2018-08-01 01:05:48 -0400363* Submit a [new module issue](https://github.com/golang/go/issues/new?title=cmd%2Fgo%3A%20%3Cfill%20this%20in%3E) using 'cmd/go:' as the prefix
364
thepudds7afc40d2018-09-24 22:06:51 -0400365## FAQs — Most Common
thepuddse01f40e2018-08-01 01:05:48 -0400366
367### How are versions marked as incompatible?
368
thepuddsd9442592018-08-02 09:44:49 -0400369The `require` directive allows any module to declare that it should be built with version >= x.y.z of a dependency D (which may be specified due to incompatibilities with version < x.y.z of module D). Empirical data suggests [this is the dominant form of constraints used in `dep` and `cargo`](https://twitter.com/_rsc/status/1022590868967116800). In addition, the top-level module in the build can `exclude` specific versions of dependencies or `replace` other modules with different code. See the full proposal for [more details and rationale](https://github.com/golang/proposal/blob/master/design/24301-versioned-go.md).
thepuddse01f40e2018-08-01 01:05:48 -0400370
thepudds601d2312018-08-06 23:26:24 -0400371One of the key goals of the versioned modules proposal is to add a common vocabulary and semantics around versions of Go code for both tools and developers. This lays a foundation for future capabilities to declare additional forms of incompatibilities, such as possibly:
thepudds0bb0f8d2018-08-01 01:51:19 -0400372* declaring deprecated versions as [described](https://research.swtch.com/vgo-module) in the initial `vgo` blog series
thepuddse01f40e2018-08-01 01:05:48 -0400373* declaring pair-wise incompatibility between modules in an external system as discussed for example [here](https://github.com/golang/go/issues/24301#issuecomment-392111327) during the proposal process
thepudds00c21b72018-09-07 01:00:11 -0400374* declaring pair-wise incompatible versions or insecure versions of a module after a release has been published. See for example the on-going discussion in [#24031](https://github.com/golang/go/issues/24031#issuecomment-407798552) and [#26829](https://github.com/golang/go/issues/26829)
thepuddse01f40e2018-08-01 01:05:48 -0400375
thepudds758e6082018-08-25 15:02:30 -0400376### When do I get old behavior vs. new module-based behavior?
377
378In general, modules are opt-in for Go 1.11, so by design old behavior is preserved by default.
379
380Summarizing when you get the old 1.10 status quo behavior vs. the new opt-in modules-based behavior:
381
382* Inside GOPATH — defaults to old 1.10 behavior (ignoring modules)
thepudds038b77e2018-08-28 12:20:18 -0400383* Outside GOPATH while inside a file tree with a `go.mod` — defaults to modules behavior
thepudds758e6082018-08-25 15:02:30 -0400384* GO111MODULE environment variable:
thepuddse24c7712018-08-25 15:09:57 -0400385 * unset or `auto` — default behavior above
386 * `on` — force module support on regardless of directory location
387 * `off` — force module support off regardless of directory location
thepudds758e6082018-08-25 15:02:30 -0400388
thepudds320b6242018-09-06 23:52:46 -0400389### Why does installing a tool via `go get` fail with error `cannot find main module`?
thepudds758e6082018-08-25 15:02:30 -0400390
thepudds868cbbc2018-09-06 23:59:14 -0400391This occurs when you have set `GO111MODULE=on`, but are not inside of a file tree with a `go.mod` when you run `go get`.
thepudds758e6082018-08-25 15:02:30 -0400392
thepudds856618c2018-09-07 00:20:35 -0400393The simplest solution is to leave `GO111MODULE` unset (or equivalently explicitly set to `GO111MODULE=auto`), which avoids this error.
thepuddse24c7712018-08-25 15:09:57 -0400394
thepudds868cbbc2018-09-06 23:59:14 -0400395Recall one of the primary reason modules exist is to record precise dependency information. This dependency information is written to your current `go.mod`. If you are not inside of a file tree with a `go.mod` but you have told the `go get` command to operate in module mode by setting `GO111MODULE=on`, then running `go get` will result in the error `cannot find main module` because there is no `go.mod` available to record dependency information.
thepuddse24c7712018-08-25 15:09:57 -0400396
thepudds868cbbc2018-09-06 23:59:14 -0400397Solution alternatives include:
thepudds586c5c02018-08-25 21:37:33 -0400398
thepudds856618c2018-09-07 00:20:35 -04003991. Leave `GO111MODULE` unset (the default, or explicitly set `GO111MODULE=auto`), which results in friendlier behavior. This will give you Go 1.10 behavior when you are outside of a module and hence will avoid `go get` reporting `cannot find main module`.
thepudds758e6082018-08-25 15:02:30 -0400400
thepudds320b6242018-09-06 23:52:46 -04004012. Leave `GO111MODULE=on`, but as needed disable modules temporarily and enable Go 1.10 behavior during `go get`, such as via `GO111MODULE=off go get example.com/cmd`. This can be turned into a simple script or shell alias such as `alias oldget='GO111MODULE=off go get'`
402
4033. Create a temporary `go.mod` file that is then discarded. This has been automated by a [simple shell script](https://gist.github.com/rogpeppe/7de05eef4dd774056e9cf175d8e6a168) by [@rogpeppe](https://github.com/rogpeppe). This script allows version information to optionally be supplied (usage: `vgoget example.com/cmd[@version]`).
404
4054. Create a `go.mod` you use to track your globally installed tools, such as in `~/global-tools/go.mod`, and `cd` to that directory prior to running `go get` or `go install` for any globally installed tools.
406
4075. Create a `go.mod` for each tool in separate directories, such as `~/tools/gorename/go.mod` and `~/tools/goimports/go.mod`, and `cd` to that appropriate directory prior to running `go get` or `go install` for the tool.
408
409This current limitation will be resolved. However, the primary issue is that modules are currently opt-in, and a full solution will likely wait until GO111MODULE=on becomes the default behavior. See [#24250](https://github.com/golang/go/issues/24250#issuecomment-377553022) for more discussion, including this comment:
410
411> This clearly must work eventually. The thing I'm not sure about is exactly what this does as far as the version is concerned: does it create a temporary module root and go.mod, do the install, and then throw it away? Probably. But I'm not completely sure, and for now I didn't want to confuse people by making vgo do things outside go.mod trees. Certainly the eventual go command integration has to support this.
412
413This FAQ has been discussing tracking _globally_ installed tools.
thepudds758e6082018-08-25 15:02:30 -0400414
thepudds93bae8f2018-08-25 22:39:46 -0400415If instead you want to track the tools required by a _specific_ module, see the next FAQ.
thepudds758e6082018-08-25 15:02:30 -0400416
417### How can I track tool dependencies for a module?
418
419If you:
420 * want to use a go-based tool (e.g. stringer) while working on a module, and
421 * want to ensure that everyone is using the same version of that tool while tracking the tool's version in your module's `go.mod` file
422
423then one currently recommended approach is to add a `tools.go` file to your module with a `// +build tools` build constraint as shown in [this comment in #25922](https://github.com/golang/go/issues/25922#issuecomment-412992431).
424
425The brief rationale (also from #25922):
426
427> I think the tools.go file is in fact the best practice for tool dependencies, certainly for Go 1.11.
thepuddse24c7712018-08-25 15:09:57 -0400428>
thepudds758e6082018-08-25 15:02:30 -0400429> I like it because it does not introduce new mechanisms.
thepuddse24c7712018-08-25 15:09:57 -0400430>
thepudds758e6082018-08-25 15:02:30 -0400431> It simply reuses existing ones.
432
thepuddsb488c3a2018-09-02 12:26:20 -0400433### What is the status of module support in IDEs, editors and standard tools like goimports, gorename, etc?
434
435Support for modules is starting to land in editors and IDEs.
436
437For example:
438* **GoLand**: currently has full support for modules outside and inside GOPATH, including completion, syntax analysis, refactoring, navigation as described [here](https://blog.jetbrains.com/go/2018/08/24/goland-2018-2-2-is-here/).
439* **VS Code**: work is in progress and looking for contribtors to help. Tracking issue is [#1532](https://github.com/Microsoft/vscode-go/issues/1532).
440* **Atom with go-plus**: tracking issue is [#761](https://github.com/joefitzgerald/go-plus/issues/761).
441* **vim with vim-go**: initial support for syntax highlighting and formatting `go.mod` has [landed](https://github.com/fatih/vim-go/pull/1931). Broader support tracked in [#1906](https://github.com/fatih/vim-go/issues/1906).
442* **emacs with go-mode.el**: tracking issue in [#237](https://github.com/dominikh/go-mode.el/issues/237).
443
444The status of other tools such as goimports, guru, gorename and similar tools is being tracked in an umbrella issue [#24661]( https://github.com/golang/go/issues/24661). Please see that umbrella issue for latest status.
445
446Some tracking issues for particular tools includes:
447* **gocode**: tracking issue in [mdempsky/gocode/#46](https://github.com/mdempsky/gocode/issues/46). Note that `nsf/gocode` is recommending people migrate off of `nsf/gocode` to `mdempsky/gocode`.
448* **goimports**: possible work-in-progress approach in [CL 128362](https://go-review.googlesource.com/c/tools/+/128362).
449* **go-tools** (tools by dominikh such as staticcheck, megacheck, gosimple): sample tracking issue [dominikh/go-tools#328](https://github.com/dominikh/go-tools/issues/328).
450
451In general, even if your editor, IDE or other tools have not yet been made module aware, much of their functionality should work with modules if you are using modules inside GOPATH and do `go mod vendor` (because then the proper dependencies should be picked up via GOPATH).
452
thepudds636e98d2018-09-07 01:38:02 -0400453The full fix is to move programs that load packages off of `go/build` and onto `golang.org/x/tools/go/packages`, which understands how to locate packages in a module-aware manner. This will likely eventually become `go/packages`.
454
thepudds7afc40d2018-09-24 22:06:51 -0400455## FAQs — Additional Control
456
thepudds636e98d2018-09-07 01:38:02 -0400457### What community tooling exists for working with modules?
458
459The community is starting to build tooling on top of modules. For example:
460
461* [github.com/rogpeppe/gohack](https://github.com/rogpeppe/gohack)
462 * A new community tool to automate and greatly simplify `replace` and multi-module workflows, including allowing you to easily modify one of your dependencies
463 * For example, `gohack example.com/some/dependency` automatically clones the appropriate repository and adds the necessary `replace` directives to your `go.mod`
464 * Remove all gohack replace statements with `gohack -u`
465 * The project is continuing to expand to make other module-related workflows easier
466* [github.com/marwan-at-work/mod](https://github.com/marwan-at-work/mod)
467 * Command line tool to automatically upgrade/downgrade major versions for modules
468 * Automatically adjusts `go.mod` files and related import statements in go source code
469 * Helps with upgrades, or when first opting in to modules with a v2+ package
470* [github.com/goware/modvendor](https://github.com/goware/modvendor)
471 * Helps copy additional files into the `vendor/` folder, such as shell scripts, .cpp files, .proto files, etc.
472
473### When should I use the replace directive?
474
thepudds0adc8292018-09-07 10:05:32 -0400475* As described in the ['go.mod' concepts section above](https://github.com/golang/go/wiki/Modules#gomod), `replace` directives provide additional control in the top-level `go.mod` for what is actually used to satisfy a dependency found in the Go source or go.mod files, while `replace` directives in modules other than the main module are ignored when building the main module.
476* The `replace` directive allows you to supply another import path that might be another module located in VCS (GitHub or elsewhere), or on your local filesystem with a relative or absolute file path. The new import path from the `replace` directive is used without needing to update the import paths in the actual source code.
477* One sample use case is if you need to fix or investigate something in a dependency, you can have a local fork and add the something like the following in your top-level `go.mod`:
thepudds636e98d2018-09-07 01:38:02 -0400478 * `replace example.com/original/import/path => /your/forked/import/path`
479* `replace` also allows the top-level module control over the exact version used for a dependency, such as:
thepudds12d4be02018-09-24 00:23:47 -0400480 * `replace example.com/some/dependency => example.com/some/dependency v1.2.3`
thepudds636e98d2018-09-07 01:38:02 -0400481* `replace` also can be used to inform the go tooling of the relative or absolute on-disk location of modules in a multi-module project, such as:
482 * `replace example.com/project/foo => ../foo`
thepudds343420b2018-09-07 09:59:22 -0400483* **Note**: in general, you can specify a version to the left of the `=>` in a replace directive, but typically it is less sensitive to change if you omit that (e.g., as done in the examples above).
thepudds636e98d2018-09-07 01:38:02 -0400484* See the [tip documentation](https://tip.golang.org/cmd/go/#hdr-Edit_go_mod_from_tools_or_scripts) for more details.
thepudds122bf9b2018-09-07 02:04:45 -0400485* [github.com/rogpeppe/gohack](https://github.com/rogpeppe/gohack) makes these types of workflows much easier. See the [repository](https://github.com/rogpeppe/gohack) or the immediately prior FAQ for an overview.
thepudds636e98d2018-09-07 01:38:02 -0400486
487### Can I work entirely outside of VCS on my local filesystem?
488
489Yes. VCS is not required.
490
thepudds32421f02018-09-07 02:14:16 -0400491This is very simple if you have a single module you want to edit at a time, and you can place the file tree containing the single `go.mod` in a convenient location.
thepudds636e98d2018-09-07 01:38:02 -0400492
493If you want to have multiple inter-related modules on your local disk that you want to edit at the same time, then `replace` directives are one approach. Here is a sample `go.mod` that uses a `replace` with a relative path to point the `hello` module at the on-disk location of the `goodbye` module (without relying on any VCS):
494
495```
496module example.com/me/hello
497
498require (
499 example.com/me/goodbye v0.0.0
500)
501
502replace example.com/me/goodbye => ../goodbye
503```
504As shown in this example, if outside of VCS you can use `v0.0.0` as the version in the `require` directive. Note that the `require` directive is needed. (A common mistake is to include `replace foo => ../foo` without having a corresponding `require foo v0.0.0`).
505
506A small runnable example is shown in this [thread](https://groups.google.com/d/msg/golang-nuts/1nYoAMFZVVM/eppaRW2rCAAJ).
thepuddsb488c3a2018-09-02 12:26:20 -0400507
thepudds00c21b72018-09-07 01:00:11 -0400508### How do I use vendoring with modules? Is vendoring going away?
thepuddse01f40e2018-08-01 01:05:48 -0400509
thepudds00c21b72018-09-07 01:00:11 -0400510The initial series of `vgo` blog posts did propose dropping vendoring entirely, but [feedback](https://groups.google.com/d/msg/golang-dev/FTMScX1fsYk/uEUSjBAHAwAJ) from the community resulted in retaining support for vendoring.
511
512In brief, to use vendoring with modules:
513* `go mod vendor` resets the main module's vendor directory to include all packages needed to build and test all of the module's packages based on the state of the go.mod files and Go source code.
514* By default, go commands like `go build` ignore the vendor directory when in module mode.
515* The `-mod=vendor` flag (e.g., `go build -mod=vendor`) instructs the go commands to use the main module's top-level vendor directory to satisfy dependencies. The go commands in this mode therefore ignore the dependency descriptions in go.mod and assume that the vendor directory holds the correct copies of dependencies. Note that only the main module's top-level vendor directory is used; vendor directories in other locations are still ignored.
Filippo Valsorda6a7f7732018-09-08 01:55:06 -0400516* Some people will want to routinely opt-in to vendoring by setting a `GOFLAGS=-mod=vendor` environment variable.
thepudds00c21b72018-09-07 01:00:11 -0400517
518Older versions of Go such as 1.10 understand how to consume a vendor directory created by `go mod vendor`, so vendoring is one way to provide dependencies to older versions of Go that do not fully understand modules.
519
520If you are considering using vendoring, it is worthwhile to read the ["Modules and vendoring"](https://tip.golang.org/cmd/go/#hdr-Modules_and_vendoring) and ["Make vendored copy of dependencies"](https://tip.golang.org/cmd/go/#hdr-Make_vendored_copy_of_dependencies) sections of the tip documentation.
521
thepudds122bf9b2018-09-07 02:04:45 -0400522### Are there "always on" module repositories and enterprise proxies?
523
524Publicly hosted "always on" immutable module repositories and optional privately hosted proxies and repositories are becoming available.
525
526For example:
527* [Project Athens](https://github.com/gomods/athens): Open source project in the works and looking for contributors.
528* [JFrog Artifactory](https://jfrog.com/artifactory/): Commercial offering. Support for Go 1.11 modules started with release 5.11 as described [here](https://jfrog.com/blog/goproxy-artifactory-go-registries/) and [here](https://www.jfrog.com/confluence/display/RTF/Go+Registry).
529
530Note that you are not required to run a proxy. Rather, the go tooling in 1.11 has added optional proxy support via [GOPROXY](https://tip.golang.org/cmd/go/#hdr-Module_proxy_protocol) to enable more enterprise use cases (such as greater control), and also to better handle situations such as "GitHub is down" or people deleting GitHub repositories.
531
532### Can I control when go.mod gets updated and when the go tools use the network to satisfy dependencies?
thepudds00c21b72018-09-07 01:00:11 -0400533
534By default, a command like `go build` will reach out to the network as needed to satisfy imports.
535
536Some teams will want to disallow the go tooling from touching the network at certain points, or will want greater control regarding when the go tooling updates `go.mod`, how dependencies are obtained, and how vendoring is used.
537
538The go tooling provides a fair amount of flexibility to adjust or disable these default behaviors, including via `-mod=readonly`, `-mod=vendor`, `GOFLAGS`, `GOPROXY=off`, `GOPROXY=file:///filesystem/path`, `go mod vendor`, and `go mod download`.
539
540The details on these options are spread throughout the official documentation. One community attempt at a consolidated overview of knobs related to these behaviors is [here](https://github.com/thepudds/go-module-knobs/blob/master/README.md), which includes links to the official documentation for more information.
541
542### How do I use modules with CI systems such as Travis or CircleCI?
543
544The simplest approach is likely just setting the environment variable `GO111MODULE=on`, which should work with most CI systems.
545
546However, it can be valuable to run tests in CI on Go 1.11 with modules enabled as well as disabled, given some of your users will not have yet opted in to modules themselves. Vendoring is also a topic to consider.
547
548The following two blog posts cover these topics more concretely:
549
550* ["Using Go modules with vendor support on Travis CI"](https://arslan.io/2018/08/26/using-go-modules-with-vendor-support-on-travis-ci/) by Fatih Arslan
551* ["Go Modules and CircleCI"](https://medium.com/@toddkeech/go-modules-and-circleci-c0d6fac0b000) by Todd Keech
552
thepudds7afc40d2018-09-24 22:06:51 -0400553## FAQs — go.mod and go.sum
554
thepudds58207da2018-09-17 09:57:11 -0400555### Why does 'go mod tidy' record indirect and test dependencies in my 'go.mod'?
thepudds00c21b72018-09-07 01:00:11 -0400556
thepudds58207da2018-09-17 09:57:11 -0400557In general, the modules system records precise dependency requirements in your `go.mod`. (For more details, see the [go.mod concepts](https://github.com/golang/go/wiki/Modules#gomod) section above or the [go.mod tip documentation](https://tip.golang.org/cmd/go/#hdr-The_go_mod_file)).
thepudds00c21b72018-09-07 01:00:11 -0400558
thepudds58207da2018-09-17 09:57:11 -0400559`go mod tidy` updates your current `go.mod` to include the transitive dependencies needed for tests in _all_ active modules, not just the main module: if a test fails (e.g., via `go test all`), we must know which dependencies were used in order to reproduce the failure.
thepudds00c21b72018-09-07 01:00:11 -0400560
thepudds58207da2018-09-17 09:57:11 -0400561`go mod tidy` also ensures your current `go.mod` reflects the dependency requirements for all possible combinations of OS, architecture, and build tags (as described [here](https://github.com/golang/go/issues/25971#issuecomment-399091682)). In contrast, other commands like `go build` and `go test` only update `go.mod` to provide the packages imported by the requested packages under the current `GOOS`, `GOARCH`, and build tags (which is one reason `go mod tidy` might add requirements that were not added by `go build` or similar).
thepudds00c21b72018-09-07 01:00:11 -0400562
Bryan C. Mills85d7baf2018-09-07 09:14:59 -0400563If a dependency of your module does not itself have a `go.mod` (e.g., because the dependency has not yet opted in to modules itself), or if its `go.mod` file is missing one or more of its dependencies (e.g., because the module author did not run `go mod tidy`), then the missing transitive dependencies will be added to _your_ module's requirements, along with an `// indirect` comment to indicate that the dependency is not from a direct import within your module.
thepudds00c21b72018-09-07 01:00:11 -0400564
Bryan C. Mills85d7baf2018-09-07 09:14:59 -0400565If you are curious why a particular module is showing up in your `go.mod`, you can run `go mod why -m <module>` to [answer](https://tip.golang.org/cmd/go/#hdr-Explain_why_packages_or_modules_are_needed) that question. Other useful tools for inspecting requirements and versions include `go mod graph` and `go list -m all`.
thepudds00c21b72018-09-07 01:00:11 -0400566
thepudds58207da2018-09-17 09:57:11 -0400567### Is 'go.sum' a lock file? Why does 'go.sum' include information for module versions I am no longer using?
568
thepudds7f6103b2018-09-17 10:03:25 -0400569No, `go.sum` is not a lock file. For validation purposes, `go.sum` contains the expected cryptographic checksums of the content of specific module versions. See the ["How to prepare a release"](https://github.com/golang/go/wiki/Modules#how-to-prepare-for-a-release) section above or the ["Module downloading and verification"](https://tip.golang.org/cmd/go/#hdr-Module_downloading_and_verification) section in the tip documentation for more details.
thepudds58207da2018-09-17 09:57:11 -0400570
571In part because `go.sum` is not a lock file, it retains cryptographic checksums for module versions even after you stop using a module or particular module version. This allows validation of the checksums if you later resume using something, which provides additional safety.
572
thepuddsd86f5aa2018-09-17 10:21:31 -0400573In addition, your module's `go.sum` records checksums for all direct and indirect dependencies used in a build (and hence your `go.sum` will frequently have more modules listed than your `go.mod`).
574
575### Should I still add a 'go.mod' file if I do not have any dependencies?
576
577Yes. This supports working outside of GOPATH, helps communicate to the ecosystem that you are opting in to modules, and in addition the `module` directive in your `go.mod` serves as a definitive declaration of the identify of your code (which is one reason why import comments might eventually be deprecated). Of course, modules are purely an opt-in capability in Go 1.11.
578
thepudds7afc40d2018-09-24 22:06:51 -0400579## FAQs — Semantic Import Versioning
580
581### Why must major version numbers appear in import paths?
582
583Please see the discussion on the semantic import versioning and the import compatibility rule in the ["Semantic Import Versioning"](https://github.com/golang/go/wiki/Modules#semantic-import-versioning) concepts section above. See also the [blog post announcing the proposal](https://blog.golang.org/versioning-proposal), which talks more about the motivation and justification for the import compatibility rule.
584
585### Why are major versions v0, v1 omitted from import paths?"
586
587Please see the question "Why are major versions v0, v1 omitted from import paths?" in the earlier [FAQ from the official proposal discussion](https://github.com/golang/go/issues/24301#issuecomment-371228664).
588
thepudds856618c2018-09-07 00:20:35 -0400589### What are some implications of tagging my project with major version v0, v1, or making breaking changes with v2+?
thepudds1da29f62018-08-05 11:19:52 -0400590
thepudds856618c2018-09-07 00:20:35 -0400591In response to a comment about *"k8s does minor releases but changes the Go API in each minor release"*, Russ Cox made the following [response](https://github.com/kubernetes/kubernetes/pull/65683#issuecomment-403705882) that highlights some implications for picking v0, v1, vs. frequently making breaking changes with v2, v3, v4, etc. with your project:
thepudds1da29f62018-08-05 11:19:52 -0400592
593> I don't fully understand the k8s dev cycle etc, but I think generally the k8s team needs to decide/confirm what they intend to guarantee to users about stability and then apply version numbers accordingly to express that.
594>
595> * To make a promise about API compatibility (which seems like the best user experience!) then start doing that and use 1.X.Y.
596> * To have the flexibility to make backwards-incompatible changes in every release but allow different parts of a large program to upgrade their code on different schedules, meaning different parts can use different major versions of the API in one program, then use X.Y.0, along with import paths like k8s.io/client/vX/foo.
597> * To make no promises about API compatible and also require every build to have only one copy of the k8s libraries no matter what, with the implied forcing of all parts of a build to use the same version even if not all of them are ready for it, then use 0.X.Y.
598
thepudds162ccfe2018-08-05 11:27:29 -0400599On a related note, Kubernetes has some atypical build approaches (currently including custom wrapper scripts on top of godep), and hence Kubernetes is an imperfect example for many other projects, but it will likely be an interesting example as [Kubernetes moves towards adopting Go 1.11 modules](https://github.com/kubernetes/kubernetes/pull/64731#issuecomment-407345841).
600
thepudds7afc40d2018-09-24 22:06:51 -0400601## FAQs — Minimal Version Selection
602
603### Won't minimal version selection keep developers from getting important updates?
604
605Please see the question "Won't minimal version selection keep developers from getting important updates?" in the earlier [FAQ from the official proposal discussion](https://github.com/golang/go/issues/24301#issuecomment-371228664).
606
607## FAQs — Possible Problems
608
thepudds856618c2018-09-07 00:20:35 -0400609### Why does `go build` require gcc, and why are prebuilt packages such as net/http not used?
610
611In short:
612
613> Because the pre-built packages are non-module builds and can’t be reused. Sorry. Disable cgo for now or install gcc.
614
615This is only an issue when opting in to modules (e.g., via `GO111MODULE=on`). See [#26988](https://github.com/golang/go/issues/26988#issuecomment-417886417) for additional discussion.
616
617### Do modules work with relative imports like `import "./subdir"`?
618
619No. See [#26645](https://github.com/golang/go/issues/26645#issuecomment-408572701), which includes:
620
621> In modules, there finally is a name for the subdirectory. If the parent directory says "module m" then the subdirectory is imported as "m/subdir", no longer "./subdir".
622
thepudds7afc40d2018-09-24 22:06:51 -0400623## FAQs — Miscellaneous
624
thepudds84465052018-09-07 01:09:12 -0400625### How did the `go mod` commands change in `go1.11beta3`?
626
627In go1.11beta3, there was a significant change for the `go mod` commands. Older material and blogs might still use the older commands from before the change. See the [tip documentation](https://tip.golang.org/cmd/go/#hdr-Module_maintenance) as well as two snippets from the [CL](https://go-review.googlesource.com/c/go/+/126655) briefly covering the rationale and the list of new vs. old commands:
628```
629The current "go mod" command does too many things.
630
631It looks like "everything you might want to do with modules"
632which causes people to think all module operations go through
633"go mod", which is the opposite of the seamless integration we're
634going for. In particular too many people think "go mod -require"
635and "go get" are the same.
636```
637and:
638```
639split "go mod" into multiple subcommands:
640
641 go mod edit # old go mod -require ...
642 go mod fix # old go mod -fix
643 go mod graph # old go mod -graph
644 go mod init # old go mod -init
645 go mod tidy # old go mod -sync
646 go mod vendor # old go mod -vendor
647 go mod verify # old go mod -verify
648
649Splitting out the individual commands makes both the docs
650and the implementations dramatically easier to read.
651It simplifies the command lines
652(go mod -init -module m is now 'go mod init m')
653and allows command-specific flags.
654```
655
thepuddse01f40e2018-08-01 01:05:48 -0400656