blob: 2686b63a291a7f085438cb2107c2ece9536e7441 [file] [log] [blame] [view]
Bryan C. Mills0d7c8e02018-07-16 15:50:05 -04001# Go 1.11 Modules
Bryan C. Mills75f81732018-07-16 14:15:15 -04002
thepuddsa569e2a2018-12-29 17:43:34 -05003Go 1.11 includes preliminary support for versioned modules as proposed [here](https://golang.org/design/24301-versioned-go). Modules are an experimental opt-in feature in Go 1.11, with the [plan](https://blog.golang.org/modules2019) of incorporating feedback and finalizing the feature for Go 1.13. Even though some details may change, future releases will support modules defined using Go 1.11 or 1.12.
Bryan C. Mills75f81732018-07-16 14:15:15 -04004
thepudds43da1062018-12-23 10:32:47 -05005The initial prototype (`vgo`) was [announced](https://research.swtch.com/vgo) in February 2018. In July 2018, support for versioned modules [landed](https://groups.google.com/d/msg/golang-dev/a5PqQuBljF4/61QK4JdtBgAJ) in the main repository. Go 1.11 was released in August 2018.
thepuddsf398d512018-07-22 18:10:33 -04006
thepudds720dd652018-09-23 15:05:57 -04007Please provide feedback on modules via [existing or new issues](https://github.com/golang/go/wiki/Modules#github-issues) and via [experience reports](https://github.com/golang/go/wiki/ExperienceReports).
thepudds634f3412018-07-28 15:27:09 -04008
thepuddsfc0364b2018-08-02 22:10:15 -04009## Table of Contents
10
thepuddsdc17cc62018-12-21 15:03:23 -050011The "Quick Start" and "New Concepts" sections are particularly important for someone who is starting to work with modules. The "How to..." sections cover more details on mechanics. The largest quantity of content on this page is in the FAQs answering more specific questions; it can be worthwhile to at least skim the FAQ one-liners listed here.
thepuddsdc9d7bc2018-12-13 13:14:12 -050012
13* [Quick Start](https://github.com/golang/go/wiki/Modules#quick-start)
thepudds4d09f912019-01-24 11:03:05 -050014 * [Example](https://github.com/golang/go/wiki/Modules#example)
15 * [Daily Workflow](https://github.com/golang/go/wiki/Modules#daily-workflow)
thepuddsfc0364b2018-08-02 22:10:15 -040016* [New Concepts](https://github.com/golang/go/wiki/Modules#new-concepts)
thepuddsfc0364b2018-08-02 22:10:15 -040017 * [Modules](https://github.com/golang/go/wiki/Modules#modules)
thepudds0e8012e2018-08-05 12:24:11 -040018 * [go.mod](https://github.com/golang/go/wiki/Modules#gomod)
thepuddsfc0364b2018-08-02 22:10:15 -040019 * [Version Selection](https://github.com/golang/go/wiki/Modules#version-selection)
thepudds14331902018-08-05 12:09:17 -040020 * [Semantic Import Versioning](https://github.com/golang/go/wiki/Modules#semantic-import-versioning)
thepudds885ec5c2018-09-24 21:46:42 -040021* [How to Use Modules](https://github.com/golang/go/wiki/Modules#how-to-use-modules)
22 * [How to Install and Activate Module Support](https://github.com/golang/go/wiki/Modules#how-to-install-and-activate-module-support)
23 * [How to Define a Module](https://github.com/golang/go/wiki/Modules#how-to-define-a-module)
24 * [How to Upgrade and Downgrade Dependencies](https://github.com/golang/go/wiki/Modules#how-to-upgrade-and-downgrade-dependencies)
thepudds1adc5432018-10-05 00:18:57 -040025 * [How to Prepare for a Release (All Versions)](https://github.com/golang/go/wiki/Modules#how-to-prepare-for-a-release)
26 * [How to Prepare for a Release (v2 or Higher)](https://github.com/golang/go/wiki/Modules#releasing-modules-v2-or-higher)
thepuddsb865b442018-12-23 12:03:38 -050027* [Migrating to Modules](https://github.com/golang/go/wiki/Modules#migrating-to-modules)
thepuddsfc0364b2018-08-02 22:10:15 -040028* [Additional Resources](https://github.com/golang/go/wiki/Modules#additional-resources)
29* [Changes Since the Initial Vgo Proposal](https://github.com/golang/go/wiki/Modules#changes-since-the-initial-vgo-proposal)
30* [GitHub Issues](https://github.com/golang/go/wiki/Modules#github-issues)
Bryan C. Mills4635c4e2018-09-28 14:16:38 -040031* [FAQs](https://github.com/golang/go/wiki/Modules#faqs)
thepuddsf4456932018-09-07 01:51:55 -040032 * [How are versions marked as incompatible?](https://github.com/golang/go/wiki/Modules#how-are-versions-marked-as-incompatible)
Jean de Klerk105df932018-12-19 14:27:25 -080033 * [Can two modules depend on each other?](https://github.com/golang/go/wiki/Modules#can-two-modules-depend-on-each-other-cyclical-import)
Jean de Klerkdfae7f12018-12-19 14:26:51 -080034 * [Can a module depend on a different version of itself?](https://github.com/golang/go/wiki/Modules#can-a-module-depend-on-a-different-version-of-itself)
thepuddsf4456932018-09-07 01:51:55 -040035 * [When do I get old behavior vs. new module-based behavior?](https://github.com/golang/go/wiki/Modules#when-do-i-get-old-behavior-vs-new-module-based-behavior)
36 * [Why does installing a tool via 'go get' fail with error 'cannot find main module'?](https://github.com/golang/go/wiki/Modules#why-does-installing-a-tool-via-go-get-fail-with-error-cannot-find-main-module)
37 * [How can I track tool dependencies for a module?](https://github.com/golang/go/wiki/Modules#how-can-i-track-tool-dependencies-for-a-module)
thepudds122bf9b2018-09-07 02:04:45 -040038 * [What is the status of module support in IDEs, editors and standard tools like goimports, gorename, etc.?](https://github.com/golang/go/wiki/Modules#what-is-the-status-of-module-support-in-ides-editors-and-standard-tools-like-goimports-gorename-etc)
thepudds979093a2018-09-24 22:14:19 -040039* [FAQs — Additional Control](https://github.com/golang/go/wiki/Modules#faqs--additional-control)
thepuddsf4456932018-09-07 01:51:55 -040040 * [What community tooling exists for working with modules?](https://github.com/golang/go/wiki/Modules#what-community-tooling-exists-for-working-with-modules)
thepudds122bf9b2018-09-07 02:04:45 -040041 * [When should I use the 'replace' directive?](https://github.com/golang/go/wiki/Modules#when-should-i-use-the-replace-directive)
thepuddsf4456932018-09-07 01:51:55 -040042 * [Can I work entirely outside of VCS on my local filesystem?](https://github.com/golang/go/wiki/Modules#can-i-work-entirely-outside-of-vcs-on-my-local-filesystem)
43 * [How do I use vendoring with modules? Is vendoring going away?](https://github.com/golang/go/wiki/Modules#how-do-i-use-vendoring-with-modules-is-vendoring-going-away)
thepudds122bf9b2018-09-07 02:04:45 -040044 * [Are there "always on" module repositories and enterprise proxies?](https://github.com/golang/go/wiki/Modules#are-there-always-on-module-repositories-and-enterprise-proxies)
45 * [Can I control when go.mod gets updated and when the go tools use the network to satisfy dependencies?](https://github.com/golang/go/wiki/Modules#can-i-control-when-gomod-gets-updated-and-when-the-go-tools-use-the-network-to-satisfy-dependencies)
thepuddsf4456932018-09-07 01:51:55 -040046 * [How do I use modules with CI systems such as Travis or CircleCI?](https://github.com/golang/go/wiki/Modules#how-do-i-use-modules-with-ci-systems-such-as-travis-or-circleci)
thepudds979093a2018-09-24 22:14:19 -040047* [FAQs — go.mod and go.sum](https://github.com/golang/go/wiki/Modules#faqs--gomod-and-gosum)
thepudds58207da2018-09-17 09:57:11 -040048 * [Why does 'go mod tidy' record indirect and test dependencies in my 'go.mod'?](https://github.com/golang/go/wiki/Modules#why-does-go-mod-tidy-record-indirect-and-test-dependencies-in-my-gomod)
thepudds7f6103b2018-09-17 10:03:25 -040049 * [Is 'go.sum' a lock file? Why does 'go.sum' include information for module versions I am no longer using?](https://github.com/golang/go/wiki/Modules/#is-gosum-a-lock-file-why-does-gosum-include-information-for-module-versions-i-am-no-longer-using)
thepuddsd86f5aa2018-09-17 10:21:31 -040050 * [Should I still add a 'go.mod' file if I do not have any dependencies?](https://github.com/golang/go/wiki/Modules#should-i-still-add-a-gomod-file-if-i-do-not-have-any-dependencies)
thepuddse4ddd272018-10-09 10:16:32 -040051 * [Should I commit my 'go.sum' file as well as my 'go.mod' file?](https://github.com/golang/go/wiki/Modules#should-i-commit-my-gosum-file-as-well-as-my-gomod-file)
thepudds979093a2018-09-24 22:14:19 -040052* [FAQs — Semantic Import Versioning](https://github.com/golang/go/wiki/Modules#faqs--semantic-import-versioning)
53 * [Why must major version numbers appear in import paths?](https://github.com/golang/go/wiki/Modules#why-must-major-version-numbers-appear-in-import-paths)
54 * [Why are major versions v0, v1 omitted from import paths?](https://github.com/golang/go/wiki/Modules#why-are-major-versions-v0-v1-omitted-from-import-paths)
thepudds7afc40d2018-09-24 22:06:51 -040055 * [What are some implications of tagging my project with major version v0, v1, or making breaking changes with v2+?](https://github.com/golang/go/wiki/Modules#what-are-some-implications-of-tagging-my-project-with-major-version-v0-v1-or-making-breaking-changes-with-v2)
thepuddse8762f62018-12-07 17:33:00 -050056 * [Can a module consume a package that has not opted in to modules?](https://github.com/golang/go/wiki/Modules#can-a-module-consume-a-package-that-has-not-opted-in-to-modules)
57 * [Can a module consume a v2+ package that has not opted into modules? What does '+incompatible' mean?](https://github.com/golang/go/wiki/Modules#can-a-module-consume-a-v2-package-that-has-not-opted-into-modules-what-does-incompatible-mean)
58 * [How are v2+ modules treated in a build if modules support is not enabled? How does "minimal module compatibility" work in 1.9.7+, 1.10.3+, and 1.11?](https://github.com/golang/go/wiki/Modules#how-are-v2-modules-treated-in-a-build-if-modules-support-is-not-enabled-how-does-minimal-module-compatibility-work-in-197-1103-and-111)
Jean de Klerk2c3c07d2019-02-25 12:28:53 -070059* [FAQs — Multi-Module Repositories](https://github.com/golang/go/wiki/Modules#faqs--multi-module-repositories)
60 * [What are multi-module repositories?](https://github.com/golang/go/wiki/Modules#what-are-multi-module-repositories)
61 * [Is it possible to add a module to a multi-module repository?](https://github.com/golang/go/wiki/Modules#is-it-possible-to-add-a-module-to-a-multi-module-repository)
62 * [Is it possible to remove a module from a multi-module repository?](https://github.com/golang/go/wiki/Modules#is-it-possible-to-remove-a-module-from-a-multi-module-repository)
63 * [Can a module depend on an internal/ in another?](https://github.com/golang/go/wiki/Modules#can-a-module-depend-on-an-internal-in-another)
thepudds979093a2018-09-24 22:14:19 -040064* [FAQs — Minimal Version Selection](https://github.com/golang/go/wiki/Modules#faqs--minimal-version-selection)
65 * [Won't minimal version selection keep developers from getting important updates?](https://github.com/golang/go/wiki/Modules#wont-minimal-version-selection-keep-developers-from-getting-important-updates)
thepudds885ec5c2018-09-24 21:46:42 -040066* [FAQs — Possible Problems](https://github.com/golang/go/wiki/Modules#faqs-possible-problems)
thepuddsa2751192018-12-07 10:41:52 -050067 * [What are some general things I can spot check if I am seeing a problem?](https://github.com/golang/go/wiki/Modules#what-are-some-general-things-i-can-spot-check-if-i-am-seeing-a-problem)
thepudds192b1f52018-12-09 08:55:19 -050068 * [What can I check if I am not seeing the expected version of a dependency?](https://github.com/golang/go/wiki/Modules#what-can-i-check-if-i-am-not-seeing-the-expected-version-of-a-dependency)
thepuddsa2751192018-12-07 10:41:52 -050069 * [Why am I getting an error 'cannot find module providing package foo'?](https://github.com/golang/go/wiki/Modules#why-am-i-getting-an-error-cannot-find-module-providing-package-foo)
thepudds93dda5a2018-12-07 10:43:54 -050070 * [Why does 'go mod init' give the error 'cannot determine module path for source directory'?](https://github.com/golang/go/wiki/Modules#why-does-go-mod-init-give-the-error-cannot-determine-module-path-for-source-directory)
thepuddsbf4b37d2018-12-20 02:25:10 -050071 * [I have a problem with a complex dependency that has not opted in to modules. Can I use information from its current dependency manager?](https://github.com/golang/go/wiki/Modules#i-have-a-problem-with-a-complex-dependency-that-has-not-opted-in-to-modules-can-i-use-information-from-its-current-dependency-manager)
thepudds122bf9b2018-09-07 02:04:45 -040072 * [Why does 'go build' require gcc, and why are prebuilt packages such as net/http not used?](https://github.com/golang/go/wiki/Modules#why-does-go-build-require-gcc-and-why-are-prebuilt-packages-such-as-nethttp-not-used)
thepuddsf4456932018-09-07 01:51:55 -040073 * [Do modules work with relative imports like `import "./subdir"`?](https://github.com/golang/go/wiki/Modules#do-modules-work-with-relative-imports-like-import-subdir)
Sylvain Laurent5b4c4e02018-10-03 17:02:28 +020074 * [Some needed files may not be present in populated vendor directory](https://github.com/golang/go/wiki/Modules#some-needed-files-may-not-be-present-in-populated-vendor-directory)
thepuddsfc0364b2018-08-02 22:10:15 -040075
thepuddsdc9d7bc2018-12-13 13:14:12 -050076## Quick Start
Bryan C. Mills75f81732018-07-16 14:15:15 -040077
thepudds4d09f912019-01-24 11:03:05 -050078#### Example
79
thepudds720dd652018-09-23 15:05:57 -040080The details are covered in the remainder of this page, but here is a simple example of creating a module from scratch.
Bryan C. Mills0d7c8e02018-07-16 15:50:05 -040081
thepudds720dd652018-09-23 15:05:57 -040082Create a directory outside of your GOPATH:
AJ ONeal21fc7f12018-09-20 14:06:57 -060083```
thepudds720dd652018-09-23 15:05:57 -040084$ mkdir -p /tmp/scratchpad/hello
85$ cd /tmp/scratchpad/hello
AJ ONeal21fc7f12018-09-20 14:06:57 -060086```
87
thepudds720dd652018-09-23 15:05:57 -040088Initialize a new module:
AJ ONeal21fc7f12018-09-20 14:06:57 -060089```
thepudds720dd652018-09-23 15:05:57 -040090$ go mod init github.com/you/hello
91
92go: creating new go.mod: module github.com/you/hello
93```
94
95Write your code:
96```
97$ cat <<EOF > hello.go
98package main
99
100import (
101 "fmt"
102 "rsc.io/quote"
103)
104
105func main() {
106 fmt.Println(quote.Hello())
107}
108EOF
109```
110
111Build and run:
112```
113$ go build
114$ ./hello
115
116Hello, world.
117```
118
thepuddsf5091652018-12-13 13:34:55 -0500119The `go.mod` file was updated to include explicit versions for your dependencies, where `v1.5.2` here is a [semver](https://semver.org) tag:
thepudds720dd652018-09-23 15:05:57 -0400120```
121$ cat go.mod
122
123module github.com/you/hello
124
125require rsc.io/quote v1.5.2
AJ ONeal21fc7f12018-09-20 14:06:57 -0600126```
127
thepudds4d09f912019-01-24 11:03:05 -0500128#### Daily Workflow
129
130Note there was no `go get` required in the example above.
thepuddsdc17cc62018-12-21 15:03:23 -0500131
132Your typical day-to-day workflow can be:
133
134* Add import statements to your `.go` code as needed.
135* Standard commands like `go build` or `go test` will automatically add new dependencies as needed to satisfy imports (updating `go.mod` and downloading the new dependencies).
136* When needed, more specific versions of dependencies can be chosen with commands such as `go get foo@v1.2.3`, `go get foo@master`, `go get foo@e3702bed2`, or by editing `go.mod` directly.
thepuddsdc9d7bc2018-12-13 13:14:12 -0500137
138A brief tour of other common functionality you might use:
139
thepuddsc9292912018-12-21 16:28:35 -0500140* `go list -m all` — View final versions that will be used in a build for all direct and indirect dependencies ([details](https://github.com/golang/go/wiki/Modules#version-selection))
141* `go list -u -m all` — View available minor and patch upgrades for all direct and indirect dependencies ([details](https://github.com/golang/go/wiki/Modules#how-to-upgrade-and-downgrade-dependencies))
142* `go get -u` or `go get -u=patch` — Update all direct and indirect dependencies to latest minor or patch upgrades ([details](https://github.com/golang/go/wiki/Modules#how-to-upgrade-and-downgrade-dependencies))
143* `go build ./...` or `go test ./...` — Build or test all packages in the module when run from the module root directory ([details](https://github.com/golang/go/wiki/Modules#how-to-define-a-module))
144* `go mod tidy` — Prune any no-longer-needed dependencies from `go.mod` and add any dependencies needed for other combinations of OS, architecture, and build tags ([details](https://github.com/golang/go/wiki/Modules#how-to-prepare-for-a-release))
145* `replace` directive or `gohack` — Use a fork, local copy or exact version of a dependency ([details](https://github.com/golang/go/wiki/Modules#when-should-i-use-the-replace-directive))
146* `go mod vendor` — Optional step to create a `vendor` directory ([details](https://github.com/golang/go/wiki/Modules#how-do-i-use-vendoring-with-modules-is-vendoring-going-away))
thepuddsdc9d7bc2018-12-13 13:14:12 -0500147
thepuddsdc17cc62018-12-21 15:03:23 -0500148After reading the next four sections on "New Concepts", you will have enough information to get started with modules for most projects. It is also useful to review the [Table of Contents](https://github.com/golang/go/wiki/Modules#table-of-contents) above (including the FAQ one-liners there) to familiarize yourself with the list of more detailed topics.
149
Bryan C. Mills75f81732018-07-16 14:15:15 -0400150## New Concepts
151
thepuddsb9b02482018-12-19 10:42:59 -0500152These sections provide a high-level introduction to the main new concepts. For more details and rationale, please see this 40-minute introductory [video by Russ Cox describing the philosophy behind the design](https://www.youtube.com/watch?v=F8nrpe0XWRg&list=PLq2Nv-Sh8EbbIjQgDzapOFeVfv5bGOoPE&index=3&t=0s), the [official proposal document](https://golang.org/design/24301-versioned-go), or the more detailed initial [vgo blog series](https://research.swtch.com/vgo).
thepudds0e8012e2018-08-05 12:24:11 -0400153
Bryan C. Mills75f81732018-07-16 14:15:15 -0400154### Modules
155
thepudds43da1062018-12-23 10:32:47 -0500156A *module* is a collection of related Go packages that are versioned together as a single unit.
Bryan C. Mills75f81732018-07-16 14:15:15 -0400157
thepuddsdef8ca72018-08-28 17:26:39 -0400158Modules record precise dependency requirements and create reproducible builds.
thepuddscc84ca22018-08-28 17:24:14 -0400159
Jean de Klerk64727de2019-02-25 12:39:27 -0700160Most often, a version control repository contains exactly one module defined in the repository root. ([Multiple modules are supported in a single repository](https://github.com/golang/go/wiki/Modules#faqs--multi-module-repositories), but typically that would result in more work on an on-going basis than a single module per repository).
thepudds43da1062018-12-23 10:32:47 -0500161
Jean de Klerk084a2542019-02-25 12:47:55 -0700162Summarizing the relationship between repositories, modules, and packages:
163* A repository contains one or more Go modules.
164* Each module contains one or more Go packages.
165* Each package consists of one or more Go source files in a single directory.
166
167 Modules must be semantically versioned according to [semver](https://semver.org/), usually in the form `v(major).(minor).(patch)`, such as `v0.1.0`, `v1.2.3`, or `v1.5.0-rc.1`. The leading `v` is required. If using Git, [tag](https://git-scm.com/book/en/v2/Git-Basics-Tagging) released commits with their versions. Public and private module repositories and proxies are becoming available (see FAQ [below](https://github.com/golang/go/wiki/Modules#are-there-always-on-module-repositories-and-enterprise-proxies)).
168
thepudds0e8012e2018-08-05 12:24:11 -0400169### go.mod
170
Jean de Klerk084a2542019-02-25 12:47:55 -0700171A module is defined by a tree of Go source files with a `go.mod` file in the tree's root directory. Module source code may be located outside of GOPATH. There are four directives: `module`, `require`, `replace`, `exclude`.
thepudds0e8012e2018-08-05 12:24:11 -0400172
thepuddsb9b02482018-12-19 10:42:59 -0500173Here is an example `go.mod` file defining the module `github.com/my/thing`:
thepudds0e8012e2018-08-05 12:24:11 -0400174
Bryan C. Mills75f81732018-07-16 14:15:15 -0400175```
thepuddsd293f292018-10-04 23:45:35 -0400176module github.com/my/thing
Bryan C. Mills75f81732018-07-16 14:15:15 -0400177
178require (
thepudds72779d82018-08-15 12:48:53 -0400179 github.com/some/dependency v1.2.3
thepudds725fdf32018-10-04 22:50:12 -0400180 github.com/another/dependency/v4 v4.0.0
Bryan C. Mills75f81732018-07-16 14:15:15 -0400181)
182```
183
Jean de Klerk30001cb2019-02-25 12:43:52 -0700184A module declares its identity in its `go.mod` via the `module` directive, which provides the _module path_. The import paths for all packages in a module share the module path as a common prefix. The module path and the relative path from the `go.mod` to a package's directory together determine a package's import path.
185
186For example, if you are creating a module for a repository `github.com/my/repo` that will contain two packages with import paths `github.com/my/repo/foo` and `github.com/my/repo/bar`, then the first line in your `go.mod` file typically would declare your module path as `module github.com/my/repo`, and the corresponding on-disk structure could be:
thepuddsd293f292018-10-04 23:45:35 -0400187
188```
thepuddsae770a22018-12-10 18:06:32 -0500189repo/
thepuddsd293f292018-10-04 23:45:35 -0400190├── go.mod
191├── bar
192│   └── bar.go
193└── foo
194 └── foo.go
195```
196
thepuddsb9b02482018-12-19 10:42:59 -0500197In Go source code, packages are imported using the full path including the module path. For example, if a module declared its identity in its `go.mod` as `module example.com/my/module`, a consumer could do:
198```
thepudds3c945452018-12-19 12:24:46 -0500199import "example.com/my/module/mypkg"
thepuddsb9b02482018-12-19 10:42:59 -0500200```
thepudds3c945452018-12-19 12:24:46 -0500201This imports package `mypkg` from the module `example.com/my/module`.
Bryan C. Mills75f81732018-07-16 14:15:15 -0400202
thepuddsb9b02482018-12-19 10:42:59 -0500203`exclude` and `replace` directives only operate on the current (“main”) module. `exclude` and `replace` directives in modules other than the main module are ignored when building the main module. The `replace` and `exclude` statements therefore allow the main module complete control over its own build, without also being subject to complete control by dependencies. (See FAQ [below](https://github.com/golang/go/wiki/Modules#when-should-i-use-the-replace-directive) for discussion of when to use a `replace` directive).
Bryan C. Mills75f81732018-07-16 14:15:15 -0400204
Bryan C. Mills0d7c8e02018-07-16 15:50:05 -0400205### Version Selection
Bryan C. Mills75f81732018-07-16 14:15:15 -0400206
thepuddsc6470ba2018-12-05 11:47:23 -0500207If you add a new import to your source code that is not yet covered by a `require` in `go.mod`, most go commands like 'go build' and 'go test' will automatically look up the proper module and add the *highest* version of that new direct dependency to your module's `go.mod` as a `require` directive. For example, if your new import corresponds to dependency M whose latest tagged release version is `v1.2.3`, your module's `go.mod` will end up with `require M v1.2.3`, which indicates module M is a dependency with allowed version >= v1.2.3 (and < v2, given v2 is considered incompatible with v1).
thepudds5af74db2018-08-01 18:14:36 -0400208
thepudds7806bc72018-08-29 00:16:41 -0400209The *minimal version selection* algorithm is used to select the versions of all modules used in a build. For each module in a build, the version selected by minimal version selection is always the semantically *highest* of the versions explicitly listed by a `require` directive in the main module or one of its dependencies.
Bryan C. Mills75f81732018-07-16 14:15:15 -0400210
thepudds738d95c2018-08-29 08:37:13 -0400211As an example, if your module depends on module A which has a `require D v1.0.0`, and your module also depends on module B which has a `require D v1.1.1`, then minimal version selection would choose `v1.1.1` of D to include in the build (given it is the highest listed `require` version). This selection of `v1.1.1` remains consistent even if some time later a `v1.2.0` of D becomes available. This is an example of how the modules system provides 100% reproducible builds. When ready, the module author or user might choose to upgrade to the latest available version of D or choose an explicit version for D.
thepuddsc164d5a2018-08-03 13:04:28 -0400212
thepudds91ffcea2018-08-02 23:55:11 -0400213For a brief rationale and overview of the minimal version selection algorithm, [see the "High Fidelity Builds" section](https://github.com/golang/proposal/blob/master/design/24301-versioned-go.md#update-timing--high-fidelity-builds) of the official proposal, or see the [more detailed `vgo` blog series](https://research.swtch.com/vgo).
thepudds5077edf2018-08-02 09:01:12 -0400214
thepudds6ae9d6f2018-08-15 15:52:57 -0400215To see a list of the selected module versions (including indirect dependencies), use `go list -m all`.
Bryan C. Mills75f81732018-07-16 14:15:15 -0400216
thepudds0168d012018-08-02 22:24:15 -0400217See also the ["How to Upgrade and Downgrade Dependencies"](https://github.com/golang/go/wiki/Modules#how-to-upgrade-and-downgrade-dependencies) section below and the ["How are versions marked as incompatible?"](https://github.com/golang/go/wiki/Modules#how-are-versions-marked-as-incompatible) FAQ below.
thepudds5af74db2018-08-01 18:14:36 -0400218
thepudds14331902018-08-05 12:09:17 -0400219### Semantic Import Versioning
220
thepudds01c74342018-08-25 10:49:21 -0400221For many years, the official Go FAQ has included this advice on package versioning:
222
223> "Packages intended for public use should try to maintain backwards compatibility as they evolve. The Go 1 compatibility guidelines are a good reference here: don't remove exported names, encourage tagged composite literals, and so on. If different functionality is required, add a new name instead of changing an old one. If a complete break is required, create a new package with a new import path."
224
thepudds725fdf32018-10-04 22:50:12 -0400225The last sentence is especially important — if you break compatibility, you should change the import path of your package. With Go 1.11 modules, that advice is formalized into the _import compatibility rule_:
thepudds01c74342018-08-25 10:49:21 -0400226
227> "If an old package and a new package have the same import path,
228> the new package must be backwards compatible with the old package."
229
thepuddsc6470ba2018-12-05 11:47:23 -0500230Recall [semver](https://semver.org/) requires a major version change when a v1 or higher package makes a backwards incompatible change. The result of following both the import compatibility rule and semver is called _Semantic Import Versioning_, where the major version is included in the import path — this ensures the import path changes any time the major version increments due to a break in compatibility.
thepudds01c74342018-08-25 10:49:21 -0400231
thepuddsc6470ba2018-12-05 11:47:23 -0500232As a result of Semantic Import Versioning, code opting in to Go modules **must comply with these rules**:
233* Follow [semver](https://semver.org/) (with tags such as `v1.2.3`).
thepudds4b4f5352018-12-19 12:30:55 -0500234* If the module is version v2 or higher, the major version of the module _must_ be included as a `/vN` at the end of the module paths used in `go.mod` files (e.g., `module github.com/my/mod/v2`, `require github.com/my/mod/v2 v2.0.0`) and in the package import path (e.g., `import "github.com/my/mod/v2/mypkg"`).
thepudds656e8352018-08-21 15:32:45 -0400235* If the module is version v0 or v1, do _not_ include the major version in either the module path or the import path.
236
thepuddsc6470ba2018-12-05 11:47:23 -0500237In general, packages with different import paths are different packages. For example, `math/rand` is a different package than `crypto/rand`. This is also true if different import paths are due to different major versions appearing in the import path. Thus `example.com/my/mod/mypkg` is a different package than `example.com/my/mod/v2/mypkg`, and both may be imported in a single build, which among other benefits helps with diamond dependency problems and also allows a v1 module to be implemented in terms of its v2 replacement or vice versa.
thepudds14331902018-08-05 12:09:17 -0400238
thepuddsc6470ba2018-12-05 11:47:23 -0500239See the ["Module compatibility and semantic versioning"](https://golang.org/cmd/go/#hdr-Module_compatibility_and_semantic_versioning) section of the `go` command documentation for more details on Semantic Import Versioning, and see https://semver.org for more about semantic versioning.
thepudds7eb5d6f2018-09-24 22:48:23 -0400240
thepuddsc6470ba2018-12-05 11:47:23 -0500241This section so far has been focused on code that has opted in to modules and imports other modules. However, putting major versions in import paths for v2+ modules could create incompatibilities with older versions of Go, or with code that has not yet opted in to modules. To help with this, there are three important transitional special cases or exceptions to the behavior and rules described above. These transitional exceptions will become less important over time as more packages opt in to modules.
thepuddsaa43ea42018-08-15 11:52:47 -0400242
thepuddsc6470ba2018-12-05 11:47:23 -0500243**Three Transitional Exceptions**
244
2451. **gopkg.in**
246
247 Existing code that uses import paths starting with `gopkg.in` (such as `gopkg.in/yaml.v1` and `gopkg.in/yaml.v2`) can continue to use those forms for their module paths and import paths even after opting in to modules.
248
2492. **'+incompatible' when importing non-module v2+ packages**
250
thepuddsdc17cc62018-12-21 15:03:23 -0500251 A module can import a v2+ package that has not opted in to modules itself. A non-module v2+ package that has a valid v2+ [semver](https://semver.org) tag will be recorded with an `+incompatible` suffix in the importing module's `go.mod` file. The `+incompatible` suffix indicates that even though the v2+ package has a valid v2+ [semver](semver.org) tag such as `v2.0.0`, the v2+ package has not actively opted in to modules and hence that v2+ package is assumed to have _not_ been created with an understanding of the implications of Semantic Import Versioning and how to use major versions in import paths. Therefore, when operating in [module mode](https://github.com/golang/go/wiki/Modules#when-do-i-get-old-behavior-vs-new-module-based-behavior), the `go` tool will treat a non-module v2+ package as an (incompatible) extension of the v1 version series of the package and assume the package has no awareness of Semantic Import Versioning, and the `+incompatible` suffix is an indication that the `go` tool is doing so.
thepuddsc6470ba2018-12-05 11:47:23 -0500252
2533. **"Minimal module compatibility" when module mode is not enabled**
254
255 To help with backwards compatibility, Go versions 1.9.7+, 1.10.3+ and 1.11 have been updated to make it easier for code built with those releases to be able to properly consume v2+ modules _without_ requiring modification of pre-existing code. This behavior is called "minimal module compatibility", and it only takes effect when full [module mode](https://github.com/golang/go/wiki/Modules#when-do-i-get-old-behavior-vs-new-module-based-behavior) is disabled for the `go` tool, such as if such as you have set `GO111MODULE=off` in Go 1.11, or are using Go versions 1.9.7+ or 1.10.3+. When relying on this "minimal module compatibility" mechanism in Go 1.9.7+, 1.10.3+ and 1.11, a package that has _not_ opted in to modules would _not_ include the major version in the import path for any imported v2+ modules. In contrast, a package that _has_ opted in to modules _must_ include the major version in the import path to import any v2+ modules (in order to properly import the v2+ module when the `go` tool is operating in full module mode with full awareness of Semantic Import Versioning).
thepudds0e8012e2018-08-05 12:24:11 -0400256
thepudds05465dd2018-09-23 16:22:19 -0400257For the exact mechanics required to release a v2+ module, please see the ["Releasing Modules (v2 or Higher)"](https://github.com/golang/go/wiki/Modules#releasing-modules-v2-or-higher) section below.
thepudds14331902018-08-05 12:09:17 -0400258
thepudds885ec5c2018-09-24 21:46:42 -0400259## How to Use Modules
260
261### How to Install and Activate Module Support
thepudds720dd652018-09-23 15:05:57 -0400262
263To use modules, two install options are:
264* [Install the latest Go 1.11 release](https://golang.org/dl/).
265* [Install the Go toolchain from source](https://golang.org/doc/install/source) on the `master` branch.
266
267Once installed, you can then activate module support in one of two ways:
268* Invoke the `go` command in a directory outside of the `$GOPATH/src` tree, with a valid `go.mod` file in the current directory or any parent of it and the environment variable `GO111MODULE` unset (or explicitly set to `auto`).
269* Invoke the `go` command with `GO111MODULE=on` environment variable set.
270
thepudds885ec5c2018-09-24 21:46:42 -0400271### How to Define a Module
Bryan C. Mills75f81732018-07-16 14:15:15 -0400272
thepuddsfd489a12018-08-01 02:47:49 -0400273To create a `go.mod` for an existing project:
Bryan C. Mills75f81732018-07-16 14:15:15 -0400274
thepudds7eb5d6f2018-09-24 22:48:23 -04002751. Navigate to the root of the module's source tree outside of GOPATH:
Bryan C. Mills75f81732018-07-16 14:15:15 -0400276
Bryan C. Mills0d7c8e02018-07-16 15:50:05 -0400277 ```
thepudds7eb5d6f2018-09-24 22:48:23 -0400278 $ cd <project path outside $GOPATH/src> # e.g., cd ~/projects/hello
Bryan C. Mills0d7c8e02018-07-16 15:50:05 -0400279 ```
thepudds7eb5d6f2018-09-24 22:48:23 -0400280 Note that outside of GOPATH, you do not need to set `GO111MODULE` to activate module mode.
Bryan C. Mills75f81732018-07-16 14:15:15 -0400281
thepudds7eb5d6f2018-09-24 22:48:23 -0400282 Alternatively, if you want to work in your GOPATH:
Bryan C. Mills75f81732018-07-16 14:15:15 -0400283
Bryan C. Mills0d7c8e02018-07-16 15:50:05 -0400284 ```
thepudds7eb5d6f2018-09-24 22:48:23 -0400285 $ export GO111MODULE=on # manually active module mode
286 $ cd $GOPATH/src/<project path> # e.g., cd $GOPATH/src/you/hello
Bryan C. Mills0d7c8e02018-07-16 15:50:05 -0400287 ```
Bryan C. Mills75f81732018-07-16 14:15:15 -0400288
thepudds6f427542018-07-31 13:33:16 -04002892. Create the initial module definition and write it to the `go.mod` file:
Bryan C. Mills75f81732018-07-16 14:15:15 -0400290
Bryan C. Mills0d7c8e02018-07-16 15:50:05 -0400291 ```
thepudds6f434ef2018-08-13 13:29:51 -0400292 $ go mod init
Bryan C. Mills0d7c8e02018-07-16 15:50:05 -0400293 ```
thepudds9bdb2722018-07-27 07:02:48 -0400294 This step converts from any existing [`dep`](https://github.com/golang/dep) `Gopkg.lock` file or from any of the other [nine total supported dependency formats](https://tip.golang.org/pkg/cmd/go/internal/modconv/?m=all#pkg-variables), adding require statements to match the existing configuration.
Bryan C. Mills75f81732018-07-16 14:15:15 -0400295
thepudds3c948ab2018-12-11 08:59:09 -0500296 `go mod init` will often be able to use auxiliary data (such as VCS meta-data) to automatically determine the appropriate module path, but if `go mod init` states it can not automatically determine the module path, or if you need to otherwise override that path, you can supply the [module path](https://github.com/golang/go/wiki/Modules#gomod) as an optional argument to `go mod init`, for example:
Bryan C. Mills75f81732018-07-16 14:15:15 -0400297
Bryan C. Mills0d7c8e02018-07-16 15:50:05 -0400298 ```
thepudds3c948ab2018-12-11 08:59:09 -0500299 $ go mod init github.com/my/repo
Bryan C. Mills0d7c8e02018-07-16 15:50:05 -0400300 ```
Bryan C. Mills75f81732018-07-16 14:15:15 -0400301
thepuddse15da1b2018-11-25 12:58:59 -0500302 Note that if your dependencies include v2+ modules, or if you are initializing a v2+ module, then after running `go mod init` you might also need to edit your `go.mod` and `.go` code to add `/vN` to import paths and module paths as described in the ["Semantic Import Versioning"](https://github.com/golang/go/wiki/Modules#semantic-import-versioning) section above. This applies even if `go mod init` automatically converted your dependency information from `dep` or other dependency managers. (Because of this, after running `go mod init`, you typically should not run `go mod tidy` until you have successfully run `go build ./...` or similar, which is the sequence shown in this section).
303
thepuddsbb4f9f02018-12-13 13:24:11 -05003043. Build the module. When executed from the root directory of a module, the `./...` pattern matches all the packages within the current module. `go build` will automatically add missing or unconverted dependencies as needed to satisfy imports for this particular build invocation:
Bryan C. Mills0d7c8e02018-07-16 15:50:05 -0400305
306 ```
thepudds74241a12018-07-31 13:31:00 -0400307 $ go build ./...
Bryan C. Mills0d7c8e02018-07-16 15:50:05 -0400308 ```
thepudds6f427542018-07-31 13:33:16 -04003094. Test the module as configured to ensure that it works with the selected versions:
Bryan C. Mills0d7c8e02018-07-16 15:50:05 -0400310
311 ```
312 $ go test ./...
313 ```
314
thepuddsd5e66ee2018-08-15 13:20:27 -04003155. (Optional) Run the tests for your module plus the tests for all direct and indirect dependencies to check for incompatibilities:
Bryan C. Mills0d7c8e02018-07-16 15:50:05 -0400316
317 ```
thepudds4b3d8932018-07-27 06:50:53 -0400318 $ go test all
Bryan C. Mills0d7c8e02018-07-16 15:50:05 -0400319 ```
Bryan C. Mills75f81732018-07-16 14:15:15 -0400320
thepudds0168d012018-08-02 22:24:15 -0400321Prior to tagging a release, see the ["How to Prepare for a Release"](https://github.com/golang/go/wiki/Modules#how-to-prepare-for-a-release) section below.
thepudds74241a12018-07-31 13:31:00 -0400322
thepuddsc6470ba2018-12-05 11:47:23 -0500323For more information on all of these topics, the primary entry point to the official modules documentation is [available on golang.org](https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more).
thepudds432bda32018-08-07 05:22:33 -0400324
thepudds88919642018-08-02 21:51:04 -0400325## How to Upgrade and Downgrade Dependencies
Bryan C. Mills75f81732018-07-16 14:15:15 -0400326
thepuddsb9b02482018-12-19 10:42:59 -0500327Day-to-day upgrading and downgrading of dependencies should be done using 'go get', which will automatically update the `go.mod` file. Alternatively, you can edit `go.mod` directly.
thepudds4b3d8932018-07-27 06:50:53 -0400328
thepudds61c12ce2018-07-27 08:04:34 -0400329In addition, go commands like 'go build', 'go test', or even 'go list' will automatically add new dependencies as needed to satisfy imports (updating `go.mod` and downloading the new dependencies).
thepuddsb05a8442018-07-26 08:18:42 -0400330
thepuddsb9b02482018-12-19 10:42:59 -0500331To view available minor and patch upgrades for all direct and indirect dependencies, run `go list -u -m all`.
332
333To upgrade to the latest version for all direct and indirect dependencies of the current module:
thepudds5af74db2018-08-01 18:14:36 -0400334 * run `go get -u` to use the latest *minor or patch* releases
335 * run `go get -u=patch` to use the latest *patch* releases
thepuddsee879c52018-07-25 12:08:34 -0400336
thepuddse12e42c2018-12-19 13:58:59 -0500337To upgrade or downgrade to a more specific version, 'go get' allows version selection to be overridden by adding an @version suffix or ["module query"](https://golang.org/cmd/go/#hdr-Module_queries) to the package argument, such as `go get foo@v1.6.2`, `go get foo@e3702bed2`, or `go get foo@'<v1.6.2'`.
thepudds8f75c6c2018-08-01 09:18:37 -0400338
thepuddsafeb4122019-02-26 09:00:13 -0500339`go get foo` updates to the latest version with a [semver](https://semver.org/) tag, or the the latest known commit if there are no semver tags ([details](https://golang.org/cmd/go/#hdr-Module_aware_go_get)). `go get foo` is equivalent to `go get foo@latest` — in other words, `@latest` is the default if no `@` version is specified.
340
341Note that a `-u` in `go get -u foo` and `go get -u foo@latest` also upgrades all the direct and indirect dependencies of `foo`. A common starting point is instead `go get foo` or `go get foo@latest` without a `-u` (and after things are working, consider `go get -u=patch foo`, `go get -u foo`, `go get -u`, or `go get -u=patch`).
thepuddscf232d22018-11-02 17:58:22 -0400342
thepuddsc6470ba2018-12-05 11:47:23 -0500343Using a branch name such as `go get foo@master` is one way to obtain the latest commit regardless of whether or not it has a semver tag.
thepuddsff8f0f82018-08-25 13:07:59 -0400344
345In general, module queries that do not resolve to a semver tag will be recorded as [pseudo-versions](https://tip.golang.org/cmd/go/#hdr-Pseudo_versions) in the `go.mod` file.
thepudds72779d82018-08-15 12:48:53 -0400346
thepuddsb9b02482018-12-19 10:42:59 -0500347See the ["Module-aware go get"](https://golang.org/cmd/go/#hdr-Module_aware_go_get) and ["Module queries"](https://golang.org/cmd/go/#hdr-Module_queries) sections of the `go` command documentation for more information on the topics here.
thepudds90d81be2018-08-26 12:21:40 -0400348
thepuddsb9b02482018-12-19 10:42:59 -0500349Modules are capable of consuming packages that have not yet opted into modules, including recording any available semver tags in `go.mod` and using those semver tags to upgrade or downgrade. Modules can also consume packages that do not yet have any proper semver tags (in which case they will be recorded using pseudo-versions in `go.mod`).
thepuddsb05a8442018-07-26 08:18:42 -0400350
thepuddsd5e66ee2018-08-15 13:20:27 -0400351After upgrading or downgrading any dependencies, you may then want to run the tests again for all packages in your build (including direct and indirect dependencies) to check for incompatibilities:
x1ah2e74f822018-11-22 14:45:57 +0800352
thepuddsee879c52018-07-25 12:08:34 -0400353 ```
thepudds4b3d8932018-07-27 06:50:53 -0400354 $ go test all
thepuddsee879c52018-07-25 12:08:34 -0400355 ```
thepuddsf398d512018-07-22 18:10:33 -0400356
thepudds88919642018-08-02 21:51:04 -0400357## How to Prepare for a Release
thepudds61aba5c2018-07-27 07:10:40 -0400358
thepudds05465dd2018-09-23 16:22:19 -0400359### Releasing Modules (All Versions)
360
thepuddsdf13b282018-08-01 00:38:40 -0400361Best practices for creating a release of a module are expected to emerge as part of the initial modules experiment. Many of these might end up being automated by a [future 'go release' tool](https://github.com/golang/go/issues/26420).
thepudds61aba5c2018-07-27 07:10:40 -0400362
thepudds88919642018-08-02 21:51:04 -0400363Some current suggested best practices to consider prior to tagging a release:
thepudds61aba5c2018-07-27 07:10:40 -0400364
thepudds885ec5c2018-09-24 21:46:42 -0400365* Run `go mod tidy` to possibly prune any extraneous requirements (as described [here](https://tip.golang.org/cmd/go/#hdr-Maintaining_module_requirements)) and also ensure your current go.mod reflects all possible build tags/OS/architecture combinations (as described [here](https://github.com/golang/go/issues/25971#issuecomment-399091682)).
thepuddscb4e8b52018-08-28 15:01:45 -0400366 * In contrast, other commands like `go build` and `go test` will not remove dependencies from `go.mod` that are no longer required and only update `go.mod` based on the current build invocation's tags/OS/architecture.
thepuddsf918d5c2018-07-31 13:36:22 -0400367
thepuddsd8674ab2018-08-15 15:22:10 -0400368* Run `go test all` to test your module (including running the tests for your direct and indirect dependencies) as a way of validating that the currently selected packages versions are compatible.
thepuddsd5e66ee2018-08-15 13:20:27 -0400369 * The number of possible version combinations is exponential in the number of modules, so in general you cannot expect your dependencies to have tested against all possible combinations of their dependencies.
thepudds61aba5c2018-07-27 07:10:40 -0400370 * As part of the modules work, `go test all` has been [re-defined to be more useful](https://research.swtch.com/vgo-cmd) to include all the packages in the current module, plus all the packages they depend on through a sequence of one or more imports, while excluding packages that don't matter in the current module.
thepudds61aba5c2018-07-27 07:10:40 -0400371
thepuddse4ddd272018-10-09 10:16:32 -0400372* Ensure your `go.sum` file is committed along with your `go.mod` file. See [FAQ below](https://github.com/golang/go/wiki/Modules#should-i-commit-my-gosum-file-as-well-as-my-gomod-file) for more details and rationale.
thepuddsabb30ff2018-08-15 09:32:02 -0400373
thepuddsf75aafd2018-12-03 22:04:28 -0500374Your new module release becomes available to consumers once you push the new release tags.
375
thepudds05465dd2018-09-23 16:22:19 -0400376### Releasing Modules (v2 or Higher)
377
378If you are releasing a v2 or higher module, please first review the discussion in the ["Semantic Import Versioning" ](https://github.com/golang/go/wiki/Modules#semantic-import-versioning) section above, which includes why major versions are included in the module path and import path for v2+ modules, as well as how Go versions 1.9.7+ and 1.10.3+ have been updated to simplify that transition.
379
Komu Wairagu45290032018-12-14 09:52:47 +0300380Note that if you are adopting modules for the first time for a pre-existing repository or set of packages that have already been tagged `v2.0.0` or higher before adopting modules, then the [recommended best practice](https://github.com/golang/go/issues/25967#issuecomment-422828770) is to increment the major version when first adopting modules. For example, if you are the author of `foo`, and the latest tag for the `foo` repository is `v2.2.2`, and `foo` has not yet adopted modules, then the best practice would be to use `v3.0.0` for the first release of `foo` to adopt modules (and hence the first release of `foo` to contain a `go.mod` file). Incrementing the major version in this case provides greater clarity to consumers of `foo`, allows for additional non-module patches or minor releases on the v2 series of `foo` if needed, and provides a strong signal for a module-based consumer of `foo` that different major versions result if you do `import "foo"` and a corresponding `require foo v2.2.2+incompatible`, vs. `import "foo/v3"` and a corresponding `require foo/v3 v3.0.0`. (Note that this advice regarding incrementing the major version when first adopting modules does _not_ apply to pre-existing repos or packages whose latest versions are v0.x.x or v1.x.x).
thepuddsce129982018-12-03 12:50:28 -0500381
thepuddsf75aafd2018-12-03 22:04:28 -0500382There are two alternative mechanisms to release a v2 or higher module. Note that with both techniques, the new module release becomes available to consumers when the module author pushes the new tags. Using the example of creating a `v3.0.0` release, the two options are:
thepudds05465dd2018-09-23 16:22:19 -0400383
thepudds026d4d22018-12-03 15:13:41 -05003841. **Major branch**: Update the `go.mod` file to include a `/v3` at the end of the module path in the `module` directive (e.g., `module github.com/my/module/v3`). Update import statements within the module to also use `/v3` (e.g., `import "github.com/my/module/v3/mypkg"`). Tag the release with `v3.0.0`.
thepudds05465dd2018-09-23 16:22:19 -0400385 * Go versions 1.9.7+, 1.10.3+, and 1.11 are able to properly consume and build a v2+ module created using this approach without requiring updates to consumer code that has not yet opted in to modules (as described in the the ["Semantic Import Versioning"](https://github.com/golang/go/wiki/Modules#semantic-import-versioning) section above).
386 * A community tool [github.com/marwan-at-work/mod](https://github.com/marwan-at-work/mod) helps automate this procedure. See the [repository](https://github.com/marwan-at-work/mod) or the [community tooling FAQ](https://github.com/golang/go/wiki/Modules#what-community-tooling-exists-for-working-with-modules) below for an overview.
387 * To avoid confusion with this approach, consider putting the `v3.*.*` commits for the module on a separate v3 branch.
thepudds16d85892019-03-13 12:06:32 -0400388 * **Note:** creating a new branch is _not_ required. If instead you have been previously releasing on master and would prefer to tag `v3.0.0` on master, that is a viable option. (However, be aware that introducing an incompatible API change in `master` can cause issues for non-modules users who issue a `go get -u` given the `go` tool is not aware of [semver](https://semver.org) prior to Go 1.11 or when [module mode](https://github.com/golang/go/wiki/Modules#when-do-i-get-old-behavior-vs-new-module-based-behavior) is not enabled in Go 1.11+).
thepudds05465dd2018-09-23 16:22:19 -0400389
thepudds026d4d22018-12-03 15:13:41 -05003902. **Major subdirectory**: Create a new `v3` subdirectory (e.g., `my/module/v3`) and place a new `go.mod` file in that subdirectory. The module path must end with `/v3`. Copy or move the code into the `v3` subdirectory. Update import statements within the module to also use `/v3` (e.g., `import "github.com/my/module/v3/mypkg"`). Tag the release with `v3.0.0`.
thepudds05465dd2018-09-23 16:22:19 -0400391 * This provides greater backwards compatibility. In particular, Go versions older than 1.9.7 and 1.10.3 are also able to properly consume and build a v2+ module created using this approach.
thepudds57f8e212018-12-07 11:16:10 -0500392 * A more sophisticated approach here could exploit type aliases (introduced in Go 1.9) and forwarding shims between major versions residing in different subdirectories. This can provide additional compatibility and allow one major version to be implemented in terms of another major version, but would entail more work for a module author. An in-progress tool to automate this is `goforward`. Please see [here](https://golang.org/cl/137076) for more details and rationale, along with a functioning initial version of `goforward`.
thepuddsd0ef5ad2018-08-15 12:07:14 -0400393
Bryan C. Mills4635c4e2018-09-28 14:16:38 -0400394See https://research.swtch.com/vgo-module for a more in-depth discussion of these alternatives.
395
thepuddsb865b442018-12-23 12:03:38 -0500396## Migrating to Modules
thepudds365c05a2018-12-07 06:50:29 -0500397
thepudds28f85512018-12-07 11:49:19 -0500398This section attempts to briefly enumerate the major decisions to be made when migrating to modules as well as list other migration-related topics. References are generally provided to other sections for more details.
thepudds365c05a2018-12-07 06:50:29 -0500399
thepuddsba32a3a2018-12-07 11:28:49 -0500400This material is primarily based on best practices that have emerged from the community as part of the modules experiment; this is therefore a work-in-progress section that will improve over time as the community gains more experience.
thepudds365c05a2018-12-07 06:50:29 -0500401
thepudds28f85512018-12-07 11:49:19 -0500402Summary:
thepudds365c05a2018-12-07 06:50:29 -0500403
404* The modules system is designed to allow different packages in the overall Go ecosystem to opt in at different rates.
thepuddsb865b442018-12-23 12:03:38 -0500405* Packages that are already on version v2 or higher have more migration considerations, primarily due to the implications of [Semantic Import versioning](https://github.com/golang/go/wiki/Modules#semantic-import-versioning).
406* New packages and packages on v0 or v1 have substantially fewer considerations when adopting modules.
thepuddsb0ad9142018-12-07 11:21:44 -0500407* Modules defined with Go 1.11 can be used by older Go versions (although the exact Go versions depends on the strategy used by the main module and its dependencies, as outlined below).
thepudds365c05a2018-12-07 06:50:29 -0500408
409Migration topics:
410
411#### Automatic Migration from Prior Dependency Managers
412
413 * `go mod init` automatically translates the required information from [dep, glide, govendor, godep and 5 other pre-existing dependency managers](https://tip.golang.org/pkg/cmd/go/internal/modconv/?m=all#pkg-variables) into a `go.mod `file that produces the equivalent build.
414 * If you are creating a v2+ module, be sure your `module` directive in the converted `go.mod` includes the appropriate `/vN` (e.g., `module foo/v3`).
415 * Note that if you are importing v2+ modules, you might need to do some manual adjustments after an initial conversion in order to add `/vN` to the `require` statements that `go mod init` generates after translating from a prior dependency manager. See the ["How to Define a Module"](https://github.com/golang/go/wiki/Modules#how-to-define-a-module) section above for more details.
416 * In addition, `go mod init` will not edit your `.go` code to add any required `/vN` to import statements. See the ["Semantic Import versioning"](https://github.com/golang/go/wiki/Modules#semantic-import-versioning) and ["Releasing Modules (v2 or Higher)"](https://github.com/golang/go/wiki/Modules#releasing-modules-v2-or-higher) sections above for the required steps, including some options around community tools to automate the conversion.
417
418#### Providing Dependency Information to Older Versions of Go
419
thepudds28f85512018-12-07 11:49:19 -0500420 * Older versions of Go understand how to consume a vendor directory created by `go mod vendor`. Therefore, vendoring is one way for a module to provide dependencies to older versions of Go that do not fully understand modules. See the [vendoring FAQ](https://github.com/golang/go/wiki/Modules#how-do-i-use-vendoring-with-modules-is-vendoring-going-away) and the `go` command [documentation](https://tip.golang.org/cmd/go/#hdr-Modules_and_vendoring) for more details.
thepudds365c05a2018-12-07 06:50:29 -0500421
thepuddsb9b02482018-12-19 10:42:59 -0500422#### Updating Pre-Existing Install Instructions
423
424 * Pre-modules, it is common for install instructions to include `go get -u foo`. If you are publishing a module `foo`, consider dropping the `-u` in instructions for modules-based consumers.
425 * `-u` asks the `go` tool to upgrade all the direct and indirect dependencies of `foo`.
426 * A module consumer might choose to run `go get -u foo` later, but there are more benefits of ["High Fidelity Builds"](https://github.com/golang/proposal/blob/master/design/24301-versioned-go.md#update-timing--high-fidelity-builds) if `-u` is not part of the initial install instructions. See ["How to Upgrade and Downgrade Dependencies"](https://github.com/golang/go/wiki/Modules#how-to-upgrade-and-downgrade-dependencies) for more details.
427 * `go get -u foo` does still work, and can still be a valid choice for install instructions.
428 * In addition, `go get foo` is not strictly needed for a module-based consumer.
429 * Simply adding an import statement `import "foo"` is sufficient. (Subsequent commands like `go build` or `go test` will automatically download `foo` and update `go.mod` as needed).
430 * Module-based consumers will not use a `vendor` directory by default.
431 * When module mode is enabled in the `go` tool, `vendor` is not strictly required when consuming a module (given the information contained in `go.mod` and the cryptographic checksums in `go.sum`), but some pre-existing install instructions assume the `go` tool will use `vendor` by default. See the [vendoring FAQ](https://github.com/golang/go/wiki/Modules#how-do-i-use-vendoring-with-modules-is-vendoring-going-away) for more details.
432 * Install instructions that include `go get foo/...` might have issues in some cases (see discussion in [#27215](https://github.com/golang/go/issues/27215#issuecomment-427672781)).
433
thepudds28f85512018-12-07 11:49:19 -0500434#### Incrementing the Major Version When First Adopting Modules with v2+ Packages
thepudds365c05a2018-12-07 06:50:29 -0500435
436* If you have packages that have already been tagged v2.0.0 or higher before adopting modules, then the recommended best practice is to increment the major version when first adopting modules. For example, if you are on `v2.0.1` and have not yet adopted modules, then you would use `v3.0.0` for the first release that adopts modules. See the ["Releasing Modules (v2 or Higher)"](https://github.com/golang/go/wiki/Modules#releasing-modules-v2-or-higher) section above for more details.
437
thepudds365c05a2018-12-07 06:50:29 -0500438#### v2+ Modules Allow Multiple Major Versions Within a Single Build
439
440* If a module is on v2 or higher, an implication is that multiple major versions can be in a single build (e.g., `foo` and `foo/v3` might end up in a single build).
thepudds28f85512018-12-07 11:49:19 -0500441 * This flows naturally from the rule that "packages with different import paths are different packages".
Alex Haynes17601472019-02-14 17:00:08 -0600442 * When this happens, there will be multiple copies of package-level state (e.g., package-level state for `foo` and package-level state for `foo/v3`) as well as each major version will run its own `init` function.
thepudds28f85512018-12-07 11:49:19 -0500443 * This approach helps with multiple aspects of the modules system, including helping with diamond dependency problems, gradual migration to new versions within large code bases, and allowing a major version to be implemented as a shim around a different major version.
thepudds365c05a2018-12-07 06:50:29 -0500444* See the "Avoiding Singleton Problems" section of https://research.swtch.com/vgo-import or [#27514](https://github.com/golang/go/issues/27514) for some related discussion.
445
446#### Modules Consuming Non-Module Code
447
448 * Modules are capable of consuming packages that have not yet opted into modules, with the appropriate package version information recorded in the importing module's `go.mod`. Modules can consume packages that do not yet have any proper semver tags. See FAQ [below](https://github.com/golang/go/wiki/Modules#can-a-module-consume-a-package-that-has-not-opted-in-to-modules) for more details.
449 * Modules can also import a v2+ package that has not opted into modules. It will be recorded with an `+incompatible` suffix if the imported v2+ package has valid semver tags. See FAQ [below](https://github.com/golang/go/wiki/Modules#can-a-module-consume-a-v2-package-that-has-not-opted-into-modules-what-does-incompatible-mean) for more details.
450
451#### Non-Module Code Consuming Modules
452
453 * **Non-module code consuming v0 and v1 modules**:
454 * Code that has not yet opted in to modules can consume and build v0 and v1 modules (without any requirement related to the Go version used).
455
456 * **Non-module code consuming v2+ modules**:
457
458 * Go versions 1.9.7+, 1.10.3+ and 1.11 have been updated so that code built with those releases can properly consume v2+ modules without requiring modification of pre-existing code as described in the ["Semantic Import versioning"](https://github.com/golang/go/wiki/Modules#semantic-import-versioning) and ["Releasing Modules (v2 or Higher)"](https://github.com/golang/go/wiki/Modules#releasing-modules-v2-or-higher) sections above.
459
460 * Go versions prior to 1.9.7 and 1.10.3 can consume v2+ modules if the v2+ module was created following the "Major subdirectory" approach outlined in the ["Releasing Modules (v2 or Higher)"](https://github.com/golang/go/wiki/Modules#releasing-modules-v2-or-higher) section.
461
462#### Strategies for Authors of Pre-Existing v2+ Packages
463
thepuddsb865b442018-12-23 12:03:38 -0500464For authors of pre-existing v2+ packages considering opting in to modules, one way to summarize the alternative approaches is as a choice between three top-level strategies . Each choice then has follow-on decisions and variations (as outlined above). These alternative top-level strategies are:
thepudds365c05a2018-12-07 06:50:29 -0500465
4661. **Require clients to use Go versions 1.9.7+, 1.10.3+, or 1.11+**.
467
thepudds35618662018-12-07 12:04:28 -0500468 The approach uses the "Major Branch" approach and relies on the "minimal module awareness" that was backported to 1.9.7 and 1.10.3. See the ["Semantic Import versioning"](https://github.com/golang/go/wiki/Modules#semantic-import-versioning) and ["Releasing Modules (v2 or Higher)"](https://github.com/golang/go/wiki/Modules#releasing-modules-v2-or-higher) sections above for more details.
thepudds365c05a2018-12-07 06:50:29 -0500469
4702. **Allow clients to use even older Go versions like Go 1.8**.
471
472 This approach uses the "Major Subdirectory" approach and involves creating a subdirectory such as `/v2` or `/v3`. See the ["Semantic Import versioning"](https://github.com/golang/go/wiki/Modules#semantic-import-versioning) and ["Releasing Modules (v2 or Higher)"](https://github.com/golang/go/wiki/Modules#releasing-modules-v2-or-higher) sections above for more details.
473
4743. **Wait on opting in to modules**.
475
476 In this strategy, things continue to work with client code that has opted in to modules as well as with client code that has not opted in to modules. As time goes by, Go versions 1.9.7+, 1.10.3+, and 1.11+ will be out for an increasingly longer time period, and at some point in the future, it becomes more natural or client-friendly to require Go versions 1.9.7+/1.10.3+/1.11+, and at that point in time, you can implement strategy 1 above (requiring Go versions 1.9.7+, 1.10.3+, or 1.11+) or even strategy 2 above (though if you are ultimately going to go with strategy 2 above in order to support older Go versions like 1.8, then that is something you can do now).
thepudds365c05a2018-12-07 06:50:29 -0500477
thepudds8390d032018-07-22 23:37:00 -0400478## Additional Resources
thepuddsf398d512018-07-22 18:10:33 -0400479
thepudds8390d032018-07-22 23:37:00 -0400480### Documentation and Proposal
481
thepuddsfd489a12018-08-01 02:47:49 -0400482* Official documentation:
thepuddsb9b02482018-12-19 10:42:59 -0500483 * Latest [HTML documentation for modules on golang.org](https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more)
thepuddsfd489a12018-08-01 02:47:49 -0400484 * Run `go help modules` for more about modules. (This is the main entry point for modules topics via `go help`)
485 * Run `go help mod` for more about the `go mod` command.
486 * Run `go help module-get` for more about the behavior of `go get` when in module-aware mode.
487 * Run `go help goproxy` for more about the module proxy, including a pure file-based option via a `file:///` URL.
thepudds592f7432018-07-26 10:03:47 -0400488* The initial ["Go & Versioning"](https://research.swtch.com/vgo) series of blog posts on `vgo` by Russ Cox (first posted February 20, 2018)
489* Official [golang.org blog post introducing the proposal](https://blog.golang.org/versioning-proposal) (March 26, 2018)
490 * This provides a more succinct overview of the proposal than the full `vgo` blog series, along with some of the history and process behind the proposal
thepudds8390d032018-07-22 23:37:00 -0400491* Official [Versioned Go Modules Proposal](https://golang.org/design/24301-versioned-go) (last updated March 20, 2018)
492
493### Introductory Material
494
thepuddsb9b02482018-12-19 10:42:59 -0500495* Introductory 40 minute video ["The Principles of Versions in Go"](https://www.youtube.com/watch?v=F8nrpe0XWRg&list=PLq2Nv-Sh8EbbIjQgDzapOFeVfv5bGOoPE&index=3&t=0s) from GopherCon Singapore by Russ Cox (May 2, 2018)
496 * Succinctly covers the philosophy behind the design of versioned Go modules, including the three core principles of "Compatibility", "Repeatability", and "Cooperation"
thepudds7afc40d2018-09-24 22:06:51 -0400497* Example based 35 minute introductory video ["What are Go modules and how do I use them?"](https://www.youtube.com/watch?v=6MbIzJmLz6Q&list=PL8QGElREVyDA2iDrPNeCe8B1u7li5S6ep&index=5&t=0s) ([slides](https://talks.godoc.org/github.com/myitcv/talks/2018-08-15-glug-modules/main.slide#1)) by Paul Jolly (August 15, 2018)
thepudds8390d032018-07-22 23:37:00 -0400498* Introductory blog post ["Taking Go Modules for a Spin"](https://dave.cheney.net/2018/07/14/taking-go-modules-for-a-spin) by Dave Cheney (July 14, 2018)
thepudds8390d032018-07-22 23:37:00 -0400499* Introductory [Go Meetup slides on modules](https://docs.google.com/presentation/d/1ansfXN8a_aVL-QuvQNY7xywnS78HE8aG7fPiITNQWMM/edit#slide=id.g3d87f3177d_0_0) by Chris Hines (July 16, 2018)
Mikael Johanssone37cb5c2018-12-19 20:39:13 +0100500* Introductory 30 minute video ["Intro to Go Modules and SemVer"](https://www.youtube.com/watch?v=aeF3l-zmPsY) by Francesc Campoy (Nov 15, 2018)
thepudds8390d032018-07-22 23:37:00 -0400501
502### Additional Material
503
thepudds345f5622018-09-06 08:55:00 -0400504* Blog post ["Using Go modules with vendor support on Travis CI"](https://arslan.io/2018/08/26/using-go-modules-with-vendor-support-on-travis-ci/) by Fatih Arslan (August 26, 2018)
505* Blog post ["Go Modules and CircleCI"](https://medium.com/@toddkeech/go-modules-and-circleci-c0d6fac0b000) by Todd Keech (July 30, 2018)
thepudds592f7432018-07-26 10:03:47 -0400506* Blog post ["The vgo proposal is accepted. Now what?"](https://research.swtch.com/vgo-accepted) by Russ Cox (May 29, 2018)
thepudds8390d032018-07-22 23:37:00 -0400507 * Includes summary of what it means that versioned modules are currently an experimental opt-in feature
thepudds345f5622018-09-06 08:55:00 -0400508* Blog post on [how to build go from tip and start using go modules](https://carolynvanslyck.com/blog/2018/07/building-go-from-source/) by Carolyn Van Slyck (July 16, 2018)
thepudds8390d032018-07-22 23:37:00 -0400509
thepudds88919642018-08-02 21:51:04 -0400510## Changes Since the Initial Vgo Proposal
511
thepuddsa7d30542018-08-03 18:02:38 -0400512As part of the proposal, prototype, and beta processes, there have been over 400 issues created by the overall community. Please continue to supply feedback.
513
514Here is a partial list of some of the larger changes and improvements, almost all of which were primarily based on feedback from the community:
thepudds88919642018-08-02 21:51:04 -0400515
516* Top-level vendor support was retained rather than vgo-based builds ignoring vendor directories entirely ([discussion](https://groups.google.com/d/msg/golang-dev/FTMScX1fsYk/uEUSjBAHAwAJ), [CL](https://go-review.googlesource.com/c/vgo/+/118316))
thepudds0168d012018-08-02 22:24:15 -0400517* Backported minimal module-awareness to allow older Go versions 1.9.7+ and 1.10.3+ to more easily consume modules for v2+ projects ([discussion](https://github.com/golang/go/issues/24301#issuecomment-371228742), [CL](https://golang.org/cl/109340))
thepudds42f1dea2018-08-03 18:07:15 -0400518* Allowed vgo to use v2+ tags by default for pre-existing packages did not yet have a go.mod (recent update in related behavior described [here](https://github.com/golang/go/issues/25967#issuecomment-407567904))
thepudds0168d012018-08-02 22:24:15 -0400519* Added support via command `go get -u=patch` to update all transitive dependencies to the latest available patch-level versions on the same minor version ([discussion](https://research.swtch.com/vgo-cmd), [documentation](https://tip.golang.org/cmd/go/#hdr-Module_aware_go_get))
thepudds88919642018-08-02 21:51:04 -0400520* Additional control via environmental variables (e.g., GOFLAGS in [#26585](https://github.com/golang/go/issues/26585), [CL](https://go-review.googlesource.com/c/go/+/126656))
thepudds55989cc2018-08-05 20:39:02 -0400521* Finer grain control on whether or not go.mod is allowed to be updated, how vendor directory is used, and whether or not network access is allowed (e.g., -mod=readonly, -mod=vendor, GOPROXY=off; related [CL](https://go-review.googlesource.com/c/go/+/126696) for recent change)
thepudds0168d012018-08-02 22:24:15 -0400522* Added more flexible replace directives ([CL](https://go-review.googlesource.com/c/vgo/+/122400))
523* Added additional ways to interrogate modules (for human consumption, as well as for better editor / IDE integration)
524* The UX of the go CLI has continued to be refined based on experiences so far (e.g., [#26581](https://github.com/golang/go/issues/26581), [CL](https://go-review.googlesource.com/c/go/+/126655))
thepudds856618c2018-09-07 00:20:35 -0400525* Additional support for warming caches for use cases such as CI or docker builds via `go mod download` ([#26610](https://github.com/golang/go/issues/26610#issuecomment-408654653))
thepudds88919642018-08-02 21:51:04 -0400526* **Most likely**: better support for installing specific versions of programs to GOBIN ([#24250](https://github.com/golang/go/issues/24250#issuecomment-377553022))
527
thepudds88919642018-08-02 21:51:04 -0400528## GitHub Issues
thepudds8390d032018-07-22 23:37:00 -0400529
thepuddsdf13b282018-08-01 00:38:40 -0400530* [Currently open module issues](https://golang.org/issues?q=is%3Aopen+is%3Aissue+label:modules)
thepudds7eb5d6f2018-09-24 22:48:23 -0400531* [Closed module issues](https://github.com/golang/go/issues?q=is%3Aclosed+is%3Aissue+label%3Amodules+sort%3Aupdated-desc)
532* [Closed vgo issues](https://github.com/golang/go/issues?q=-label%3Amodules+vgo+is%3Aclosed+sort%3Aupdated-desc)
thepuddse01f40e2018-08-01 01:05:48 -0400533* Submit a [new module issue](https://github.com/golang/go/issues/new?title=cmd%2Fgo%3A%20%3Cfill%20this%20in%3E) using 'cmd/go:' as the prefix
534
thepuddsbfe95152018-12-01 09:44:05 -0500535## FAQs
thepuddse01f40e2018-08-01 01:05:48 -0400536
537### How are versions marked as incompatible?
538
thepuddsd9442592018-08-02 09:44:49 -0400539The `require` directive allows any module to declare that it should be built with version >= x.y.z of a dependency D (which may be specified due to incompatibilities with version < x.y.z of module D). Empirical data suggests [this is the dominant form of constraints used in `dep` and `cargo`](https://twitter.com/_rsc/status/1022590868967116800). In addition, the top-level module in the build can `exclude` specific versions of dependencies or `replace` other modules with different code. See the full proposal for [more details and rationale](https://github.com/golang/proposal/blob/master/design/24301-versioned-go.md).
thepuddse01f40e2018-08-01 01:05:48 -0400540
thepudds601d2312018-08-06 23:26:24 -0400541One of the key goals of the versioned modules proposal is to add a common vocabulary and semantics around versions of Go code for both tools and developers. This lays a foundation for future capabilities to declare additional forms of incompatibilities, such as possibly:
thepudds0bb0f8d2018-08-01 01:51:19 -0400542* declaring deprecated versions as [described](https://research.swtch.com/vgo-module) in the initial `vgo` blog series
thepuddse01f40e2018-08-01 01:05:48 -0400543* declaring pair-wise incompatibility between modules in an external system as discussed for example [here](https://github.com/golang/go/issues/24301#issuecomment-392111327) during the proposal process
thepudds00c21b72018-09-07 01:00:11 -0400544* declaring pair-wise incompatible versions or insecure versions of a module after a release has been published. See for example the on-going discussion in [#24031](https://github.com/golang/go/issues/24031#issuecomment-407798552) and [#26829](https://github.com/golang/go/issues/26829)
thepuddse01f40e2018-08-01 01:05:48 -0400545
thepudds758e6082018-08-25 15:02:30 -0400546### When do I get old behavior vs. new module-based behavior?
547
548In general, modules are opt-in for Go 1.11, so by design old behavior is preserved by default.
549
550Summarizing when you get the old 1.10 status quo behavior vs. the new opt-in modules-based behavior:
551
552* Inside GOPATH — defaults to old 1.10 behavior (ignoring modules)
thepudds038b77e2018-08-28 12:20:18 -0400553* Outside GOPATH while inside a file tree with a `go.mod` — defaults to modules behavior
thepudds758e6082018-08-25 15:02:30 -0400554* GO111MODULE environment variable:
thepuddse24c7712018-08-25 15:09:57 -0400555 * unset or `auto` — default behavior above
556 * `on` — force module support on regardless of directory location
557 * `off` — force module support off regardless of directory location
thepudds758e6082018-08-25 15:02:30 -0400558
Jean de Klerkdfae7f12018-12-19 14:26:51 -0800559### Can two modules depend on each other (cyclical import)?
560Yes. Two packages however may not depend on each other (this is a build constraint).
561
562### Can a module depend on a different version of itself?
563A module can depend on a different major version of itself: by-and-large, this is comparable to depending on a different module.
564
thepudds320b6242018-09-06 23:52:46 -0400565### Why does installing a tool via `go get` fail with error `cannot find main module`?
thepudds758e6082018-08-25 15:02:30 -0400566
thepudds868cbbc2018-09-06 23:59:14 -0400567This occurs when you have set `GO111MODULE=on`, but are not inside of a file tree with a `go.mod` when you run `go get`.
thepudds758e6082018-08-25 15:02:30 -0400568
thepudds856618c2018-09-07 00:20:35 -0400569The simplest solution is to leave `GO111MODULE` unset (or equivalently explicitly set to `GO111MODULE=auto`), which avoids this error.
thepuddse24c7712018-08-25 15:09:57 -0400570
thepudds868cbbc2018-09-06 23:59:14 -0400571Recall one of the primary reason modules exist is to record precise dependency information. This dependency information is written to your current `go.mod`. If you are not inside of a file tree with a `go.mod` but you have told the `go get` command to operate in module mode by setting `GO111MODULE=on`, then running `go get` will result in the error `cannot find main module` because there is no `go.mod` available to record dependency information.
thepuddse24c7712018-08-25 15:09:57 -0400572
thepudds868cbbc2018-09-06 23:59:14 -0400573Solution alternatives include:
thepudds586c5c02018-08-25 21:37:33 -0400574
thepudds856618c2018-09-07 00:20:35 -04005751. Leave `GO111MODULE` unset (the default, or explicitly set `GO111MODULE=auto`), which results in friendlier behavior. This will give you Go 1.10 behavior when you are outside of a module and hence will avoid `go get` reporting `cannot find main module`.
thepudds758e6082018-08-25 15:02:30 -0400576
thepudds320b6242018-09-06 23:52:46 -04005772. Leave `GO111MODULE=on`, but as needed disable modules temporarily and enable Go 1.10 behavior during `go get`, such as via `GO111MODULE=off go get example.com/cmd`. This can be turned into a simple script or shell alias such as `alias oldget='GO111MODULE=off go get'`
578
thepuddsb9b02482018-12-19 10:42:59 -05005793. Create a temporary `go.mod` file that is then discarded. This has been automated by a [simple shell script](https://gist.github.com/rogpeppe/7de05eef4dd774056e9cf175d8e6a168) by [@rogpeppe](https://github.com/rogpeppe). This script allows version information to optionally be supplied via `vgoget example.com/cmd[@version]`. (This can be a solution for avoiding the error `cannot use path@version syntax in GOPATH mode`).
thepudds320b6242018-09-06 23:52:46 -0400580
thepuddsb9b02482018-12-19 10:42:59 -05005814. `gobin` is a module-aware command to install and run main packages. By default, `gobin` installs/runs main packages without first needing to manually create a module, but with the `-m` flag it can be told to use an existing module to resolve dependenci
582es. Please see the `gobin` [README](https://github.com/myitcv/gobin#usage) and [FAQ](https://github.com/myitcv/gobin/wiki/FAQ) for details and additional use cases.
thepudds320b6242018-09-06 23:52:46 -0400583
thepudds9f0f5152018-11-30 09:01:04 -05005845. Create a `go.mod` you use to track your globally installed tools, such as in `~/global-tools/go.mod`, and `cd` to that directory prior to running `go get` or `go install` for any globally installed tools.
585
5866. Create a `go.mod` for each tool in separate directories, such as `~/tools/gorename/go.mod` and `~/tools/goimports/go.mod`, and `cd` to that appropriate directory prior to running `go get` or `go install` for the tool.
thepudds320b6242018-09-06 23:52:46 -0400587
588This current limitation will be resolved. However, the primary issue is that modules are currently opt-in, and a full solution will likely wait until GO111MODULE=on becomes the default behavior. See [#24250](https://github.com/golang/go/issues/24250#issuecomment-377553022) for more discussion, including this comment:
589
590> This clearly must work eventually. The thing I'm not sure about is exactly what this does as far as the version is concerned: does it create a temporary module root and go.mod, do the install, and then throw it away? Probably. But I'm not completely sure, and for now I didn't want to confuse people by making vgo do things outside go.mod trees. Certainly the eventual go command integration has to support this.
591
592This FAQ has been discussing tracking _globally_ installed tools.
thepudds758e6082018-08-25 15:02:30 -0400593
thepudds93bae8f2018-08-25 22:39:46 -0400594If instead you want to track the tools required by a _specific_ module, see the next FAQ.
thepudds758e6082018-08-25 15:02:30 -0400595
596### How can I track tool dependencies for a module?
597
598If you:
thepuddsbfe95152018-12-01 09:44:05 -0500599 * want to use a go-based tool (e.g. `stringer`) while working on a module, and
thepudds758e6082018-08-25 15:02:30 -0400600 * want to ensure that everyone is using the same version of that tool while tracking the tool's version in your module's `go.mod` file
601
thepuddsbfe95152018-12-01 09:44:05 -0500602then one currently recommended approach is to add a `tools.go` file to your module that includes import statements for the tools of interest (such as `import _ "golang.org/x/tools/cmd/stringer"`), along with a `// +build tools` build constraint. The import statements allow the `go` command to precisely record the version information for your tools in your module's `go.mod`, while the `// +build tools` build constraint prevents your normal builds from actually importing your tools.
thepudds758e6082018-08-25 15:02:30 -0400603
thepuddsbfe95152018-12-01 09:44:05 -0500604For a concrete example of how to do this, please see this ["Go Modules by Example" walkthrough](https://github.com/go-modules-by-example/index/blob/master/010_tools/README.md).
605
606A discussion of the approach along with an earlier concrete example of how to do this is in [this comment in #25922](https://github.com/golang/go/issues/25922#issuecomment-412992431).
607
608The brief rationale (also from [#25922](https://github.com/golang/go/issues/25922#issuecomment-402918061)):
thepudds758e6082018-08-25 15:02:30 -0400609
610> I think the tools.go file is in fact the best practice for tool dependencies, certainly for Go 1.11.
thepuddse24c7712018-08-25 15:09:57 -0400611>
thepudds758e6082018-08-25 15:02:30 -0400612> I like it because it does not introduce new mechanisms.
thepuddse24c7712018-08-25 15:09:57 -0400613>
thepudds758e6082018-08-25 15:02:30 -0400614> It simply reuses existing ones.
615
thepuddsb488c3a2018-09-02 12:26:20 -0400616### What is the status of module support in IDEs, editors and standard tools like goimports, gorename, etc?
617
618Support for modules is starting to land in editors and IDEs.
619
620For example:
621* **GoLand**: currently has full support for modules outside and inside GOPATH, including completion, syntax analysis, refactoring, navigation as described [here](https://blog.jetbrains.com/go/2018/08/24/goland-2018-2-2-is-here/).
thepudds5dd795a2018-09-24 23:57:51 -0400622* **VS Code**: work is in progress and looking for contributors to help. Tracking issue is [#1532](https://github.com/Microsoft/vscode-go/issues/1532). An initial beta is described in the [VS Code module status wiki page](https://github.com/Microsoft/vscode-go/wiki/Go-modules-support-in-Visual-Studio-Code).
thepuddsb488c3a2018-09-02 12:26:20 -0400623* **Atom with go-plus**: tracking issue is [#761](https://github.com/joefitzgerald/go-plus/issues/761).
624* **vim with vim-go**: initial support for syntax highlighting and formatting `go.mod` has [landed](https://github.com/fatih/vim-go/pull/1931). Broader support tracked in [#1906](https://github.com/fatih/vim-go/issues/1906).
625* **emacs with go-mode.el**: tracking issue in [#237](https://github.com/dominikh/go-mode.el/issues/237).
626
627The status of other tools such as goimports, guru, gorename and similar tools is being tracked in an umbrella issue [#24661]( https://github.com/golang/go/issues/24661). Please see that umbrella issue for latest status.
628
629Some tracking issues for particular tools includes:
630* **gocode**: tracking issue in [mdempsky/gocode/#46](https://github.com/mdempsky/gocode/issues/46). Note that `nsf/gocode` is recommending people migrate off of `nsf/gocode` to `mdempsky/gocode`.
thepuddsb488c3a2018-09-02 12:26:20 -0400631* **go-tools** (tools by dominikh such as staticcheck, megacheck, gosimple): sample tracking issue [dominikh/go-tools#328](https://github.com/dominikh/go-tools/issues/328).
632
633In general, even if your editor, IDE or other tools have not yet been made module aware, much of their functionality should work with modules if you are using modules inside GOPATH and do `go mod vendor` (because then the proper dependencies should be picked up via GOPATH).
634
thepudds636e98d2018-09-07 01:38:02 -0400635The full fix is to move programs that load packages off of `go/build` and onto `golang.org/x/tools/go/packages`, which understands how to locate packages in a module-aware manner. This will likely eventually become `go/packages`.
636
thepudds7afc40d2018-09-24 22:06:51 -0400637## FAQs — Additional Control
638
thepudds636e98d2018-09-07 01:38:02 -0400639### What community tooling exists for working with modules?
640
641The community is starting to build tooling on top of modules. For example:
642
643* [github.com/rogpeppe/gohack](https://github.com/rogpeppe/gohack)
644 * A new community tool to automate and greatly simplify `replace` and multi-module workflows, including allowing you to easily modify one of your dependencies
645 * For example, `gohack example.com/some/dependency` automatically clones the appropriate repository and adds the necessary `replace` directives to your `go.mod`
thepuddsc4a1c1e2018-12-04 09:53:19 -0500646 * Remove all gohack replace statements with `gohack undo`
thepudds636e98d2018-09-07 01:38:02 -0400647 * The project is continuing to expand to make other module-related workflows easier
648* [github.com/marwan-at-work/mod](https://github.com/marwan-at-work/mod)
649 * Command line tool to automatically upgrade/downgrade major versions for modules
650 * Automatically adjusts `go.mod` files and related import statements in go source code
651 * Helps with upgrades, or when first opting in to modules with a v2+ package
652* [github.com/goware/modvendor](https://github.com/goware/modvendor)
653 * Helps copy additional files into the `vendor/` folder, such as shell scripts, .cpp files, .proto files, etc.
654
655### When should I use the replace directive?
656
thepudds0adc8292018-09-07 10:05:32 -0400657* As described in the ['go.mod' concepts section above](https://github.com/golang/go/wiki/Modules#gomod), `replace` directives provide additional control in the top-level `go.mod` for what is actually used to satisfy a dependency found in the Go source or go.mod files, while `replace` directives in modules other than the main module are ignored when building the main module.
658* The `replace` directive allows you to supply another import path that might be another module located in VCS (GitHub or elsewhere), or on your local filesystem with a relative or absolute file path. The new import path from the `replace` directive is used without needing to update the import paths in the actual source code.
659* One sample use case is if you need to fix or investigate something in a dependency, you can have a local fork and add the something like the following in your top-level `go.mod`:
thepudds636e98d2018-09-07 01:38:02 -0400660 * `replace example.com/original/import/path => /your/forked/import/path`
661* `replace` also allows the top-level module control over the exact version used for a dependency, such as:
thepudds12d4be02018-09-24 00:23:47 -0400662 * `replace example.com/some/dependency => example.com/some/dependency v1.2.3`
thepudds636e98d2018-09-07 01:38:02 -0400663* `replace` also can be used to inform the go tooling of the relative or absolute on-disk location of modules in a multi-module project, such as:
664 * `replace example.com/project/foo => ../foo`
thepuddsc4a1c1e2018-12-04 09:53:19 -0500665* In general, you have the option of specifying a version to the left of the `=>` in a replace directive, but typically it is less sensitive to change if you omit that (e.g., as done in all of the `replace` examples above).
thepudds3c948ab2018-12-11 08:59:09 -0500666* **Note**: for direct dependencies, a `require` directive is needed even when doing a `replace`. For example, if `foo` is a direct dependency, you cannot do `replace foo => ../foo` without a corresponding `require` for `foo`. (If you are not sure what version to use in the `require` directive, you can often use `v0.0.0` such as `require foo v0.0.0`; see [#26241](https://golang.org/issue/26241)).
thepudds12903282018-12-04 10:21:46 -0500667* You can confirm you are getting your expected versions by running `go list -m all`, which shows you the actual final versions that will be used in your build including taking into account `replace` statements.
thepuddsc4a1c1e2018-12-04 09:53:19 -0500668* See the ['go mod edit' documentation](https://golang.org/cmd/go/#hdr-Edit_go_mod_from_tools_or_scripts) for more details.
669* [github.com/rogpeppe/gohack](https://github.com/rogpeppe/gohack) makes these types of workflows much easier, especially if your goal is to have mutable checkouts of dependencies of a module. See the [repository](https://github.com/rogpeppe/gohack) or the immediately prior FAQ for an overview.
670* See the next FAQ for the details of using `replace` to work entirely outside of VCS.
thepudds636e98d2018-09-07 01:38:02 -0400671
672### Can I work entirely outside of VCS on my local filesystem?
673
674Yes. VCS is not required.
675
thepuddsc4a1c1e2018-12-04 09:53:19 -0500676This is very simple if you have a single module you want to edit at a time outside of VCS (and you either have only one module in total, or if the other modules reside in VCS). In this case, you can place the file tree containing the single `go.mod` in a convenient location. Your `go build`, `go test` and similar commands will work even if your single module is outside of VCS (without requiring any use of `replace` in your `go.mod`).
thepudds636e98d2018-09-07 01:38:02 -0400677
678If you want to have multiple inter-related modules on your local disk that you want to edit at the same time, then `replace` directives are one approach. Here is a sample `go.mod` that uses a `replace` with a relative path to point the `hello` module at the on-disk location of the `goodbye` module (without relying on any VCS):
679
680```
681module example.com/me/hello
682
683require (
684 example.com/me/goodbye v0.0.0
685)
686
687replace example.com/me/goodbye => ../goodbye
688```
thepuddsc4a1c1e2018-12-04 09:53:19 -0500689As shown in this example, if outside of VCS you can use `v0.0.0` as the version in the `require` directive. Note that as mentioned in the prior FAQ, the `require` directive is needed here. (`replace example.com/me/goodbye => ../goodbye` does not yet work without a corresponding `require example.com/me/goodbye v0.0.0`; this might change in the future with [#26241](https://golang.org/issue/26241)).
thepudds636e98d2018-09-07 01:38:02 -0400690
691A small runnable example is shown in this [thread](https://groups.google.com/d/msg/golang-nuts/1nYoAMFZVVM/eppaRW2rCAAJ).
thepuddsb488c3a2018-09-02 12:26:20 -0400692
thepudds00c21b72018-09-07 01:00:11 -0400693### How do I use vendoring with modules? Is vendoring going away?
thepuddse01f40e2018-08-01 01:05:48 -0400694
thepudds00c21b72018-09-07 01:00:11 -0400695The initial series of `vgo` blog posts did propose dropping vendoring entirely, but [feedback](https://groups.google.com/d/msg/golang-dev/FTMScX1fsYk/uEUSjBAHAwAJ) from the community resulted in retaining support for vendoring.
696
697In brief, to use vendoring with modules:
698* `go mod vendor` resets the main module's vendor directory to include all packages needed to build and test all of the module's packages based on the state of the go.mod files and Go source code.
699* By default, go commands like `go build` ignore the vendor directory when in module mode.
700* The `-mod=vendor` flag (e.g., `go build -mod=vendor`) instructs the go commands to use the main module's top-level vendor directory to satisfy dependencies. The go commands in this mode therefore ignore the dependency descriptions in go.mod and assume that the vendor directory holds the correct copies of dependencies. Note that only the main module's top-level vendor directory is used; vendor directories in other locations are still ignored.
Filippo Valsorda6a7f7732018-09-08 01:55:06 -0400701* Some people will want to routinely opt-in to vendoring by setting a `GOFLAGS=-mod=vendor` environment variable.
thepudds00c21b72018-09-07 01:00:11 -0400702
703Older versions of Go such as 1.10 understand how to consume a vendor directory created by `go mod vendor`, so vendoring is one way to provide dependencies to older versions of Go that do not fully understand modules.
704
705If you are considering using vendoring, it is worthwhile to read the ["Modules and vendoring"](https://tip.golang.org/cmd/go/#hdr-Modules_and_vendoring) and ["Make vendored copy of dependencies"](https://tip.golang.org/cmd/go/#hdr-Make_vendored_copy_of_dependencies) sections of the tip documentation.
706
thepudds122bf9b2018-09-07 02:04:45 -0400707### Are there "always on" module repositories and enterprise proxies?
708
709Publicly hosted "always on" immutable module repositories and optional privately hosted proxies and repositories are becoming available.
710
711For example:
712* [Project Athens](https://github.com/gomods/athens): Open source project in the works and looking for contributors.
Alexei Vainshteinc41c5932018-10-18 13:54:22 +0300713* [JFrog Artifactory](https://jfrog.com/artifactory/): Commercial offering. Support for Go 1.11 modules started with release 5.11 as described [here](https://jfrog.com/blog/goproxy-artifactory-go-registries/) and [here](https://www.jfrog.com/confluence/display/RTF/Go+Registry). From Artifactory version 6.2.0, please use [JFrog CLI 1.20.2](https://www.jfrog.com/confluence/display/CLI/CLI+for+JFrog+Artifactory#CLIforJFrogArtifactory-BuildingGoPackages) and above.
Jeevanandam Mf632a5f2018-12-15 17:03:26 -0800714* [THUMBAI](https://thumbai.app) - Open source project - Go Mod Proxy server and Go Vanity Import Path server
thepudds122bf9b2018-09-07 02:04:45 -0400715
716Note that you are not required to run a proxy. Rather, the go tooling in 1.11 has added optional proxy support via [GOPROXY](https://tip.golang.org/cmd/go/#hdr-Module_proxy_protocol) to enable more enterprise use cases (such as greater control), and also to better handle situations such as "GitHub is down" or people deleting GitHub repositories.
717
718### Can I control when go.mod gets updated and when the go tools use the network to satisfy dependencies?
thepudds00c21b72018-09-07 01:00:11 -0400719
720By default, a command like `go build` will reach out to the network as needed to satisfy imports.
721
722Some teams will want to disallow the go tooling from touching the network at certain points, or will want greater control regarding when the go tooling updates `go.mod`, how dependencies are obtained, and how vendoring is used.
723
724The go tooling provides a fair amount of flexibility to adjust or disable these default behaviors, including via `-mod=readonly`, `-mod=vendor`, `GOFLAGS`, `GOPROXY=off`, `GOPROXY=file:///filesystem/path`, `go mod vendor`, and `go mod download`.
725
726The details on these options are spread throughout the official documentation. One community attempt at a consolidated overview of knobs related to these behaviors is [here](https://github.com/thepudds/go-module-knobs/blob/master/README.md), which includes links to the official documentation for more information.
727
728### How do I use modules with CI systems such as Travis or CircleCI?
729
730The simplest approach is likely just setting the environment variable `GO111MODULE=on`, which should work with most CI systems.
731
732However, it can be valuable to run tests in CI on Go 1.11 with modules enabled as well as disabled, given some of your users will not have yet opted in to modules themselves. Vendoring is also a topic to consider.
733
734The following two blog posts cover these topics more concretely:
735
736* ["Using Go modules with vendor support on Travis CI"](https://arslan.io/2018/08/26/using-go-modules-with-vendor-support-on-travis-ci/) by Fatih Arslan
737* ["Go Modules and CircleCI"](https://medium.com/@toddkeech/go-modules-and-circleci-c0d6fac0b000) by Todd Keech
738
thepudds7afc40d2018-09-24 22:06:51 -0400739## FAQs — go.mod and go.sum
740
thepudds58207da2018-09-17 09:57:11 -0400741### Why does 'go mod tidy' record indirect and test dependencies in my 'go.mod'?
thepudds00c21b72018-09-07 01:00:11 -0400742
thepuddse392e8e2018-09-29 10:22:51 -0400743The modules system records precise dependency requirements in your `go.mod`. (For more details, see the [go.mod concepts](https://github.com/golang/go/wiki/Modules#gomod) section above or the [go.mod tip documentation](https://tip.golang.org/cmd/go/#hdr-The_go_mod_file)).
thepudds00c21b72018-09-07 01:00:11 -0400744
thepuddse392e8e2018-09-29 10:22:51 -0400745`go mod tidy` updates your current `go.mod` to include the dependencies needed for tests in your module — if a test fails, we must know which dependencies were used in order to reproduce the failure.
thepudds00c21b72018-09-07 01:00:11 -0400746
thepudds58207da2018-09-17 09:57:11 -0400747`go mod tidy` also ensures your current `go.mod` reflects the dependency requirements for all possible combinations of OS, architecture, and build tags (as described [here](https://github.com/golang/go/issues/25971#issuecomment-399091682)). In contrast, other commands like `go build` and `go test` only update `go.mod` to provide the packages imported by the requested packages under the current `GOOS`, `GOARCH`, and build tags (which is one reason `go mod tidy` might add requirements that were not added by `go build` or similar).
thepudds00c21b72018-09-07 01:00:11 -0400748
thepudds23256d12018-09-29 10:31:34 -0400749If a dependency of your module does not itself have a `go.mod` (e.g., because the dependency has not yet opted in to modules itself), or if its `go.mod` file is missing one or more of its dependencies (e.g., because the module author did not run `go mod tidy`), then the missing transitive dependencies will be added to _your_ module's requirements, along with an `// indirect` comment to indicate that the dependency is not from a direct import within your module.
750
751Note that this also means that any missing test dependencies from your direct or indirect dependencies will also be recorded in your `go.mod`. (An example of when this is important: `go test all` runs the tests of _all_ direct and indirect dependencies of your module, which is one way to validate that your current combination of versions work together. If a test fails in one of your dependencies when you run `go test all`, it is important to have a complete set of test dependency information recorded so that you have reproducible `go test all` behavior).
thepuddse392e8e2018-09-29 10:22:51 -0400752
thepuddsdced9eb2018-11-29 06:37:22 -0500753Another reason you might have `// indirect` dependencies in your `go.mod` file is if you have upgraded (or downgraded) one of your indirect dependencies beyond what is required by your direct dependencies, such as if you ran `go get -u` or `go get foo@1.2.3`. The go tooling needs a place to record those new versions, and it does so in your `go.mod` file (and it does not reach down into your dependencies to modify _their_ `go.mod` files).
754
755In general, the behaviors described above are part of how modules provide 100% reproducible builds and tests by recording precise dependency information.
thepudds00c21b72018-09-07 01:00:11 -0400756
Matthew Calea913f0e2018-10-23 15:58:35 -0500757If you are curious as to why a particular module is showing up in your `go.mod`, you can run `go mod why -m <module>` to [answer](https://tip.golang.org/cmd/go/#hdr-Explain_why_packages_or_modules_are_needed) that question. Other useful tools for inspecting requirements and versions include `go mod graph` and `go list -m all`.
thepudds00c21b72018-09-07 01:00:11 -0400758
thepudds58207da2018-09-17 09:57:11 -0400759### Is 'go.sum' a lock file? Why does 'go.sum' include information for module versions I am no longer using?
760
thepuddsc3ee9b52018-12-06 08:30:43 -0500761No, `go.sum` is not a lock file. The `go.mod` files in a build provide enough information for 100% reproducible builds.
762
763For validation purposes, `go.sum` contains the expected cryptographic checksums of the content of specific module versions. See the [FAQ below](https://github.com/golang/go/wiki/Modules#should-i-commit-my-gosum-file-as-well-as-my-gomod-file) for more details on `go.sum` (including why you typically should check in `go.sum`) as well as the ["Module downloading and verification"](https://tip.golang.org/cmd/go/#hdr-Module_downloading_and_verification) section in the tip documentation.
thepudds58207da2018-09-17 09:57:11 -0400764
765In part because `go.sum` is not a lock file, it retains cryptographic checksums for module versions even after you stop using a module or particular module version. This allows validation of the checksums if you later resume using something, which provides additional safety.
766
thepuddsd86f5aa2018-09-17 10:21:31 -0400767In addition, your module's `go.sum` records checksums for all direct and indirect dependencies used in a build (and hence your `go.sum` will frequently have more modules listed than your `go.mod`).
768
thepuddse4ddd272018-10-09 10:16:32 -0400769### Should I commit my 'go.sum' file as well as my 'go.mod' file?
Paul Jollyb64be362018-10-08 12:15:43 +0100770
Bryan C. Mills4aa4c782019-03-02 14:22:57 -0500771Typically your module's `go.sum` file should be committed along with your `go.mod` file.
Paul Jollyb64be362018-10-08 12:15:43 +0100772
thepuddse4ddd272018-10-09 10:16:32 -0400773* `go.sum` contains the expected cryptographic checksums of the content of specific module versions.
774* If someone clones your repository and downloads your dependencies using the go command, they will receive an error if there is any mismatch between their downloaded copies of your dependencies and the corresponding entries in your `go.sum`.
775* In addition, `go mod verify` checks that the on-disk cached copies of module downloads still match the entries in `go.sum`.
776* Note that `go.sum` is not a lock file as used in some alternative dependency management systems. (`go.mod` provides enough information for reproducible builds).
777* See very brief [rationale here](https://twitter.com/FiloSottile/status/1029404663358087173) from Filippo Valsorda on why you should check in your `go.sum`. See the ["Module downloading and verification"](https://tip.golang.org/cmd/go/#hdr-Module_downloading_and_verification) section of the tip documentation for more details. See possible future extensions being discussed for example in [#24117](https://github.com/golang/go/issues/24117) and [#25530](https://github.com/golang/go/issues/25530).
Paul Jollyb12f08b2018-10-08 12:18:21 +0100778
thepuddsbfe95152018-12-01 09:44:05 -0500779### Should I still add a 'go.mod' file if I do not have any dependencies?
780
Dave Hallidayafb676f2018-12-20 17:44:39 -0500781Yes. This supports working outside of GOPATH, helps communicate to the ecosystem that you are opting in to modules, and in addition the `module` directive in your `go.mod` serves as a definitive declaration of the identity of your code (which is one reason why import comments might eventually be deprecated). Of course, modules are purely an opt-in capability in Go 1.11.
thepuddsbfe95152018-12-01 09:44:05 -0500782
thepudds7afc40d2018-09-24 22:06:51 -0400783## FAQs — Semantic Import Versioning
784
785### Why must major version numbers appear in import paths?
786
thepuddsc6470ba2018-12-05 11:47:23 -0500787Please see the discussion on the Semantic Import Versioning and the import compatibility rule in the ["Semantic Import Versioning"](https://github.com/golang/go/wiki/Modules#semantic-import-versioning) concepts section above. See also the [blog post announcing the proposal](https://blog.golang.org/versioning-proposal), which talks more about the motivation and justification for the import compatibility rule.
thepudds7afc40d2018-09-24 22:06:51 -0400788
789### Why are major versions v0, v1 omitted from import paths?"
790
791Please see the question "Why are major versions v0, v1 omitted from import paths?" in the earlier [FAQ from the official proposal discussion](https://github.com/golang/go/issues/24301#issuecomment-371228664).
792
thepuddse8762f62018-12-07 17:33:00 -0500793### What are some implications of tagging my project with major version v0, v1, or making breaking changes with v2+?
794
795In response to a comment about *"k8s does minor releases but changes the Go API in each minor release"*, Russ Cox made the following [response](https://github.com/kubernetes/kubernetes/pull/65683#issuecomment-403705882) that highlights some implications for picking v0, v1, vs. frequently making breaking changes with v2, v3, v4, etc. with your project:
796
797> I don't fully understand the k8s dev cycle etc, but I think generally the k8s team needs to decide/confirm what they intend to guarantee to users about stability and then apply version numbers accordingly to express that.
798>
799> * To make a promise about API compatibility (which seems like the best user experience!) then start doing that and use 1.X.Y.
800> * To have the flexibility to make backwards-incompatible changes in every release but allow different parts of a large program to upgrade their code on different schedules, meaning different parts can use different major versions of the API in one program, then use X.Y.0, along with import paths like k8s.io/client/vX/foo.
801> * To make no promises about API compatible and also require every build to have only one copy of the k8s libraries no matter what, with the implied forcing of all parts of a build to use the same version even if not all of them are ready for it, then use 0.X.Y.
802
803On a related note, Kubernetes has some atypical build approaches (currently including custom wrapper scripts on top of godep), and hence Kubernetes is an imperfect example for many other projects, but it will likely be an interesting example as [Kubernetes moves towards adopting Go 1.11 modules](https://github.com/kubernetes/kubernetes/pull/64731#issuecomment-407345841).
804
thepudds10bb4b32018-12-05 18:59:16 -0500805### Can a module consume a package that has not opted in to modules?
806
807Yes.
808
809If a repository has not opted in to modules but has been tagged with valid [semver](https://semver.org) tags (including the required leading `v`), then those semver tags can be used in a `go get`, and a corresponding semver version will be record in the importing module's `go.mod` file. If the repository does not have any valid semver tags, then the repository's version will be recorded with a ["pseudo-version"](https://golang.org/cmd/go/#hdr-Pseudo_versions) such as ` v0.0.0-20171006230638-a6e239ea1c69` (which includes a timestamp and a commit hash, and which are designed to allow a total ordering across versions recored in `go.mod` and to make it easier to reason about which recorded versions are "later" than another recorded version).
810
811For example, if the latest version of package `foo` is tagged `v1.2.3` but `foo` has not itself opted in to modules, then running `go get foo` or `go get foo@v1.2.3` from inside module M will be recorded in module M's `go.mod` file as:
812
813```
814require foo v1.2.3
815```
816
817The `go` tool will also use available semver tags for a non-module package in additional workflows (such as `go list -u=patch`, which upgrades the dependencies of a module to available patch releases, or `go list -u -m all`, which shows available upgrades, etc.).
818
819Please see the next FAQs for additional details related to v2+ packages that have not opted in to modules.
820
821### Can a module consume a v2+ package that has not opted into modules? What does '+incompatible' mean?
822
thepudds3b6d8aa2018-12-07 16:21:51 -0500823Yes, a module can import a v2+ package that has not opted into modules, and if the imported v2+ package has a valid [semver](semver.org) tag, it will be recorded with an `+incompatible` suffix.
thepudds10bb4b32018-12-05 18:59:16 -0500824
825**Additional Details**
826
thepudds3b6d8aa2018-12-07 16:21:51 -0500827Please be familiar with the material in the ["Semantic Import Versioning"](https://github.com/golang/go/wiki/Modules#semantic-import-versioning) section above.
828
829It is helpful to first review some core principles that are generally useful but particularly important to keep in mind when thinking about the behavior described in this FAQ.
830
831The following core principles are _always_ true when the `go` tool is operating in module mode (e.g., `GO111MODULE=on`):
832
Grant Wu74201812019-03-12 14:07:58 -04008331. A package's import path defines the identity of the package.
thepudds3b6d8aa2018-12-07 16:21:51 -0500834 * Packages with _different_ import paths are treated as _different_ packages.
835 * Packages with the _same_ import path are treated as the _same_ package (and this is true _even if_ the VCS tags say the packages have different major versions).
8362. An import path without a `/vN` is treated as a v1 or v0 module (and this is true _even if_ the imported package has not opted in to modules and has VCS tags that say the major version is greater than 1).
8373. The module path (such as `module foo/v2`) declared at the start of a module's `go.mod` file is both:
838 * the definitive declaration of that module's identity
839 * the definitive declaration of how that module must be imported by consuming code
840
841As we will see in the next FAQ, these principles are not always true when the `go` tool is _not_ in module mode, but these principles are always true when the `go` tool _is_ in module mode.
842
843In short, the `+incompatible` suffix indicates that principle 2 above is in effect when the following are true:
844* an imported package has not opted in to modules, and
845* its VCS tags say the major version is greater than 1, and
846* principle 2 is overriding the VCS tags – the import path without a `/vN` is treated as a v1 or v0 module (even though the VCS tags say otherwise)
847
thepuddsa3eb3002018-12-19 14:16:33 -0500848When the `go` tool is in module mode, it will assume a non-module v2+ package has no awareness of Semantic Import Versioning and treat it as an (incompatible) extension of the v1 version series of the package (and the `+incompatible` suffix is an indication that the `go` tool is doing so).
thepudds3b6d8aa2018-12-07 16:21:51 -0500849
850**Example**
thepudds10bb4b32018-12-05 18:59:16 -0500851
thepuddse8762f62018-12-07 17:33:00 -0500852Suppose:
thepudds10bb4b32018-12-05 18:59:16 -0500853* `oldpackage` is a package that predates the introduction of modules
854* `oldpackage` has never opted in to modules (and hence does not have a `go.mod` itself)
855* `oldpackage` has a valid semver tag `v3.0.1`, which is its latest tag
856
857In this case, running for example `go get oldpackage@latest` from inside module M will record the following in module M's `go.mod` file:
858
859```
860require oldpackage v3.0.1+incompatible
861```
862
thepudds3b6d8aa2018-12-07 16:21:51 -0500863Note that there is no `/v3` used at the end of `oldpackage` in the `go get` command above or in the recorded `require` directive – using `/vN` in module paths and import paths is a feature of [Semantic Import Versioning](https://github.com/golang/go/wiki/Modules#semantic-import-versioning), and `oldpackage` has not signaled its acceptance and understanding of Semantic Import Versioning given `oldpackage` has not opted into modules by having a `go.mod` file within `oldpackage` itself. In other words, even though `oldpackage` has a [semver](https://semver.org) tag of `v3.0.1`, `oldpackage` is not granted the rights and responsibilities of [Semantic Import Versioning](https://github.com/golang/go/wiki/Modules#semantic-import-versioning) (such as using `/vN` in import paths) because `oldpackage` has not yet stated its desire to do so.
thepudds10bb4b32018-12-05 18:59:16 -0500864
865The `+incompatible` suffix indicates that the `v3.0.1` version of `oldpackage` has not actively opted in to modules, and hence the `v3.0.1` version of `oldpackage` is assumed to _not_ understand Semantic Import Versioning or how to use major versions in import paths. Therefore, when operating in [module mode](https://github.com/golang/go/wiki/Modules#when-do-i-get-old-behavior-vs-new-module-based-behavior), the `go` tool will treat the non-module `v3.0.1` version of `oldpackage` as an (incompatible) extension of the v1 version series of `oldpackage` and assume that the `v3.0.1` version of `oldpackage` has no awareness of Semantic Import Versioning, and the `+incompatible` suffix is an indication that the `go` tool is doing so.
866
867The fact that the the `v3.0.1` version of `oldpackage` is considered to be part of the v1 release series according to Semantic Import Versioning means for example that versions `v1.0.0`, `v2.0.0`, and `v3.0.1` are all always imported using the same import path:
868
869```
870import "oldpackage"
871```
872
873Note again that there is no `/v3` used at the end of `oldpackage`.
874
thepudds3b6d8aa2018-12-07 16:21:51 -0500875In general, packages with different import paths are different packages. In this example, given versions `v1.0.0`, `v2.0.0`, and `v3.0.1` of `oldpackage` would all be imported using the same import path, they are therefore treated by a build as the same package (again because `oldpackage` has not yet opted in to Semantic Import Versioning), with a single copy of `oldpackage` ending up in any given build. (The version used will be the semantically highest of the versions listed in any `require` directives; see ["Version Selection"](https://github.com/golang/go/wiki/Modules#version-selection)).
thepudds10bb4b32018-12-05 18:59:16 -0500876
thepudds3b6d8aa2018-12-07 16:21:51 -0500877If we suppose that later a new `v4.0.0` release of `oldpackage` is created that adopts modules and hence contains a `go.mod` file, that is the signal that `oldpackage` now understands the rights and responsibilities of Semantic Import Versioning, and hence a module-based consumer would now import using `/v4` in the import path:
thepudds10bb4b32018-12-05 18:59:16 -0500878
879```
880import "oldpackage/v4"
881```
882
883and the version would be recorded as:
884
885```
886require oldpackage/v4 v4.0.0
887```
888
889`oldpackage/v4` is now a different import path than `oldpackage`, and hence a different package. Two copies (one for each import path) would end up in a module-aware build if some consumers in the build have `import "oldpackage/v4"` while other consumers in the same build have `import "oldpackage"`. This is desirable as part of the strategy to allow gradual adoption of modules. In addition, even after modules are out of their current transitional phase, this behavior is also desirable to allow gradual code evolution over time with different consumers upgrading at different rates to newer versions (e.g., allowing different consumers in a large build to choose to upgrade at different rates from `oldpackage/v4` to some future `oldpackage/v5`).
890
891### How are v2+ modules treated in a build if modules support is not enabled? How does "minimal module compatibility" work in 1.9.7+, 1.10.3+, and 1.11?
892
893When considering older Go versions or Go code that has not yet opted in to modules, Semantic Import Versioning has significant backwards compatibility implications related to v2+ modules.
894
895As described in the ["Semantic Import Versioning"](https://github.com/golang/go/wiki/Modules#semantic-import-versioning) section above:
896* a module that is version v2 or higher must include a `/vN` in its own module path declared in its `go.mod`.
897* a module-based consumer (that is, code that has opted in to modules) must include a `/vN` in the import path to import a v2+ module.
898
899However, the ecosystem is expected to proceed at varying paces of adoption for modules and Semantic Import Versioning.
900
901As described in more detail in the ["How to Release a v2+ Module"](https://github.com/golang/go/wiki/Modules#releasing-modules-v2-or-higher) section, in the "Major Subdirectory" approach, the author of a v2+ module creates subdirectories such as `mymodule/v2` or `mymodule/v3` and moves or copies the approriate packages underneath those subdirectories. This means the traditional import path logic (even in older Go releases such as Go 1.8 or 1.7) will find the appropriate packages upon seeing an import statement such as `import "mymodule/v2/mypkg"`. Hence, packages residing in a "Major Subdirectory" v2+ module will be found and used even if modules support is not enabled (whether that is because you are running Go 1.11 and have not enabled modules, or because you are running a older version like Go 1.7, 1.8, 1.9 or 1.10 that does not have full module support). Please see the ["How to Release a v2+ Module"](https://github.com/golang/go/wiki/Modules#releasing-modules-v2-or-higher) section for more details on the "Major Subdirectory" approach.
902
903The remainder of this FAQ is focused on the "Major Branch" approach described in the ["How to Release a v2+ Module"](https://github.com/golang/go/wiki/Modules#releasing-modules-v2-or-higher) section. In the "Major Branch" approach, no `/vN` subdirectories are created and instead the module version information is communicated by the `go.mod` file and by applying semver tags to commits (which often will be on `master`, but could be on different branches).
904
905In order to help during the current transitional period, "minimal module compatibility" was [introduced](https://go-review.googlesource.com/c/go/+/109340) to Go 1.11 to provide greater compatibility for Go code that has not yet opted in to modules, and that "minimal module compatibility" was also backported to Go 1.9.7 and 1.10.3 (where those versions are effectively always operating with full module mode disabled given those older Go versions do not have full module support).
906
thepudds10bb4b32018-12-05 18:59:16 -0500907The primary goals of "minimal module compatibility" are:
908
9091. Allow older Go versions 1.9.7+ and 1.10.3+ to be able to more easily compile modules that are using Semantic Import Versioning with `/vN` in import paths, and provide that same behavior when [module mode](https://github.com/golang/go/wiki/Modules#when-do-i-get-old-behavior-vs-new-module-based-behavior) is disabled in Go 1.11.
910
9112. Allow old code to be able to consume a v2+ module without requiring that old consumer code to immediately change to using a new `/vN` import path when consuming a v2+ module.
912
thepudds7e5e2cd2018-12-07 17:18:37 -05009133. Do so without relying on the module author to create `/vN` subdirectories.
914
thepudds10bb4b32018-12-05 18:59:16 -0500915**Additional Details – "Minimal Module Compatibility"**
916
917"Minimal module compatibility" only takes effect when full [module mode](https://github.com/golang/go/wiki/Modules#when-do-i-get-old-behavior-vs-new-module-based-behavior) is disabled for the `go` tool, such as if you have set `GO111MODULE=off` in Go 1.11, or are using Go versions 1.9.7+ or 1.10.3+.
918
thepudds57f8e212018-12-07 11:16:10 -0500919When a v2+ module author has _not_ created `/v2` or `/vN` subdirectories and you are instead relying on the "minimal module compatibility" mechanism in Go 1.9.7+, 1.10.3+ and 1.11:
thepudds10bb4b32018-12-05 18:59:16 -0500920
921* A package that has _not_ opted in to modules would _not_ include the major version in the import path for any imported v2+ modules.
thepudds7e5e2cd2018-12-07 17:18:37 -0500922* In contrast, a package that _has_ opted in to modules _must_ include the major version in the import path to import any v2+ modules.
thepudds10bb4b32018-12-05 18:59:16 -0500923 * If a package has opted in to modules, but does not include the major version in the import path when importing a v2+ modules, it will not import a v2+ version of that module when the `go` tool is operating in full module mode. (A package that has opted in to modules is assumed to "speak" Semantic Import Versioning. If `foo` is a module with v2+ versions, then under Semantic Import Versioning saying `import "foo"` means import the v1 Semantic Import Versioning series of `foo`).
924* The mechanism used to implement "minimal module compatibility" is intentionally very narrow:
925 * The entirety of the logic is – when operating in GOPATH mode, an unresolvable import statement containing a `/vN` will be tried again after removing the `/vN` if the import statement is inside code that has opted in to modules (that is, import statements in `.go` files within a tree with a valid `go.mod` file).
926 * The net effect is that an import statement such as `import "foo/v2"` within code that lives inside of a module will still compile correctly in GOPATH mode in 1.9.7+, 1.10.3+ and 1.11, and it will resolve as if it said `import "foo"` (without the `/v2`), which means it will use the version of `foo` that resides in your GOPATH without being confused by the extra `/v2`.
thepudds7e5e2cd2018-12-07 17:18:37 -0500927 * "Minimal module compatibility" does not affect anything else, including it does not the affect paths used in the `go` command line (such as arguments to `go get` or `go list`).
928* This transitional "minimal module awareness" mechanism purposefully breaks the rule of "packages with different import paths are treated as different packages" in pursuit a very specific backwards compatibility goal – to allow old code to compile unmodified when it is consuming a v2+ module. In slightly more detail:
929 * It would be a more burdensome for the overall ecosystem if the only way for old code to consume a v2+ module was to first change the old code.
930 * If we are not modifying old code, then that old code must work with pre-module import paths for v2+ modules.
931 * On the other hand, new or updated code opting in to modules must use the new `/vN` import for v2+ modules.
932 * The new import path is not equal to old import path, yet both are allowed to work in a single build, and therefore we have two different functioning import paths that resolve to the same package.
933 * For example, when operating in GOPATH mode, `import "foo/v2"` appearing in module-based code resolves to the same code residing in your GOPATH as `import "foo"`, and the build ends up with one copy of `foo` – in particular, whatever version is on disk in GOPATH. This allows module-based code with `import "foo/v2"` to compile even in GOPATH mode in 1.9.7+, 1.10.3+ and 1.11.
934* In contrast, when the `go` tool is operating in full module mode:
935 * There are no exceptions to the rule "packages with different import paths are different packages" (including vendoring has been refined in full module mode to also adhere to this rule).
936 * For example, if the `go` tool is in full module mode and `foo` is a v2+ module, then `import "foo"` is asking for a v1 version of `foo` vs. `import "foo/v2"` is asking for a v2 version of `foo`.
thepudds10bb4b32018-12-05 18:59:16 -0500937
Jean de Klerk2c3c07d2019-02-25 12:28:53 -0700938## FAQS — Multi-Module Repositories
939
940### What are multi-module repositories?
941
942A multi-module repository is a repository that contains multiple modules, each with its own go.mod file. Each module starts at the directory containing its go.mod file, and contains all packages from that directory and its subdirectories recursively, excluding any subtree that contains another go.mod file.
943
944Each module has its own version information. Version tags for modules below the root of the repository must include the relative directory as a prefix. For example, consider the following repository:
945
946```
947my-repo
948|____foo
949| |____rop
950| | |____go.mod
951```
952
953The tag for version 1.2.3 of module "my-repo/foo/rop" is "foo/rop/v1.2.3".
954
955Typically, the path for one module in the repository will be a prefix of the others. For example, consider this repository:
956
957```
958my-repo
959|____go.mod
960|____bar
961|____foo
962| |____rop
963| |____yut
964|____mig
965| |____go.mod
966| |____vub
967```
968![Fig. A top-level module's path is a prefix of another module's path.](https://github.com/jadekler/module-testing/blob/master/imagery/multi_module_repo.png)
969
970_Fig. A top-level module's path is a prefix of another module's path._
971
972This repository contains two modules. However, the module "my-repo" is a prefix of the path of the module "my-repo/mig".
973
974Adding modules, removing modules, and versioning modules in such a configuration require considerable care and deliberation, so it is almost always easier and simpler to manage a single-module repository rather than multiple modules in an existing repository.
975
976### Is it possible to add a module to a multi-module repository?
977
978Yes. However, there are two classes of this problem:
979
980The first class: the package to which the module is being added to is not in version control yet (a new package). This case is straightforward: add the package and the go.mod in the same commit, tag the commit, and push.
981
982The second class: the path at which the module is being added is in version control and contains one or more existing packages. This case requires a considerable amount of care. To illustrate, consider again the following repository (now in a github.com location to simulate the real-world better):
983
984```
985github.com/my-repo
986|____go.mod
987|____bar
988|____foo
989| |____rop
990| |____yut
991|____mig
992| |____vub
993```
994
995Consider adding module "github.com/my-repo/mig". If one were to follow the same approach as above, the package /my-repo/mig could be provided by two different modules: the old version of "github.com/my-repo", and the new, standalone module "github.com/my-repo/mig. If both modules are active, importing "github.com/my-repo/mig" would cause an “ambiguous import” error at compile time.
996
997The way to get around this is to make the newly-added module depend on the module it was "carved out" from, at a version after which it was carved out.
998
999Let's step through this with the above repository, assuming that "github.com/my-repo" is currently at v1.2.3:
1000
10011. Add github.com/my-repo/mig/go.mod:
1002
1003 ```
1004 cd path-to/github.com/my-repo/mig
1005 go mod init github.com/my-repo/mig
1006
1007 # Note: if "my-repo/mig" does not actually depend on "my-repo", add a blank
1008 # import.
1009 # Note: version must be at or after the carve-out.
1010 go mod edit -require github.com/myrepo@v1.3
1011 ```
1012
10131. `git commit`
10141. `git tag v1.3.0`
10151. `git tag mig/v1.0.0`
10161. Next, let's test these. We can't `go build` or `go test` naively, since the go commands would try to fetch each dependent module from the module cache. So, we need to use replace rules to cause `go` commands to use the local copies:
1017
Jean de Klerk35251072019-02-25 12:33:48 -07001018 ```
Jean de Klerk2c3c07d2019-02-25 12:28:53 -07001019 cd path-to/github.com/my-repo/mig
1020 go mod edit -replace github.com/my-repo@v1.3.0=../
1021 go test ./...
1022 go mod edit -dropreplace github.com/my-repo@v1.3.0
1023 ```
1024
10251. `git push origin master v1.2.4 mig/v1.0.0` push the commit and both tags
1026
1027Note that in the future [golang.org/issue/28835](https://github.com/golang/go/issues/28835) should make the testing step a more straightforward experience.
1028
1029Note also that code was removed from module "github.com/my-repo" between minor versions. It may seem strange to not consider this a major change, but in this instance the transitive dependencies continue to provide compatible implementations of the removed packages at their original import paths.
1030
1031### Is it possible to remove a module from a multi-module repository?
1032
1033Yes, with the same two cases and similar steps as above.
1034
1035### Can a module depend on an internal/ in another?
1036
1037Yes. Packages in one module are allowed to import internal packages from another module as long as they share the same path prefix up to the internal/ path component. For example, consider the following repository:
1038
1039```
1040my-repo
1041|____go.mod
1042|____internal
1043|____foo
1044| |____go.mod
1045```
1046
1047Here, package foo can import /my-repo/internal as long as module "my-repo/foo" depends on module "my-repo". Similarly, in the following repository:
1048
1049```
1050my-repo
1051|____internal
1052| |____go.mod
1053|____foo
1054| |____go.mod
1055```
1056
1057Here, package foo can import my-repo/internal as long as module "my-repo/foo" depends on module "my-repo/internal". The semantics are the same in both: since my-repo is a shared path prefix between my-repo/internal and my-repo/foo, package foo is allowed to import package internal.
1058
thepudds7afc40d2018-09-24 22:06:51 -04001059## FAQs — Minimal Version Selection
1060
1061### Won't minimal version selection keep developers from getting important updates?
1062
1063Please see the question "Won't minimal version selection keep developers from getting important updates?" in the earlier [FAQ from the official proposal discussion](https://github.com/golang/go/issues/24301#issuecomment-371228664).
1064
1065## FAQs — Possible Problems
1066
thepudds8cca0ba2018-12-07 10:32:11 -05001067### What are some general things I can spot check if I am seeing a problem?
1068
1069* Double-check that modules are enabled by running `go env` to confirm it does not show an empty value for the read-only `GOMOD` variable.
1070 * Note: you never set `GOMOD` as a variable because it is effectively read-only debug output that `go env` outputs.
thepudds192b1f52018-12-09 08:55:19 -05001071 * If you are setting `GO111MODULE=on` to enable modules, double-check that it is not accidentally the plural `GO111MODULES=on`. (People sometimes naturally include the `S` because the feature is often called "modules").
thepudds8cca0ba2018-12-07 10:32:11 -05001072* If vendoring is expected to be used, double-check check that the `-mod=vendor` flag is being passed to `go build `or similar, or that `GOFLAGS=-mod=vendor` is set.
1073 * Modules by default ignore the `vendor` directory unless you ask the `go` tool to use `vendor`.
thepudds8cca0ba2018-12-07 10:32:11 -05001074* It is frequently helpful to check `go list -m all` to see the list of actual versions selected for your build
1075 * `go list -m all` usually gives you more detail compared to if you were to instead just look a `go.mod` file.
thepudds6a3214d2018-12-14 08:59:46 -05001076* If running `go get foo` fails in some way, or if `go build` is failing on a particular package `foo`, it often can be helpful to check the output from `go get -v foo` or `go get -v -x foo`:
thepudds192b1f52018-12-09 08:55:19 -05001077 * In general, `go get` will often provide more a detailed error message than `go build`.
1078 * The `-v` flag to `go get` asks to print more verbose details, though be mindful that certain "errors" such as 404 errors _might_ be expected based on how a remote repository was configured.
1079 * If the nature of the problem is still not clear, you can also try the more verbose `go get -v -x foo`, which also shows the git or other VCS commands being issued. (If warranted, you can often execute the same git commands outside of the context of the `go` tool for troubleshooting purposes).
thepudds8cca0ba2018-12-07 10:32:11 -05001080* You can check to see if you are using a particularly old git version
1081 * Older versions of git were a common source of problems for the `vgo` prototype and Go 1.11 beta, but much less frequently in the GA 1.11.
thepudds192b1f52018-12-09 08:55:19 -05001082* The module cache in Go 1.11 can sometimes cause various errors, primarily if there were previously network issues or multiple `go` commands executing in parallel (see [#26794](https://github.com/golang/go/issues/26794), which is addressed for Go 1.12). As a troubleshooting step, you can copy $GOPATH/pkg/mod to a backup directory (in case further investigation is warranted later), run `go clean -modcache`, and then see whether the original problem persists.
thepudds8cca0ba2018-12-07 10:32:11 -05001083* If you are using Docker, it can be helpful to check if you can reproduce the behavior outside of Docker (and if the behavior only occurs in Docker, the list of bullets above can be used as a starting point to compare results between inside Docker vs. outside).
1084
thepudds192b1f52018-12-09 08:55:19 -05001085The error you are currently examining might be a secondary issue caused by not having the expected version of a particular module or package in your build. Therefore, if the cause of a particular error is not obvious, it can be helpful to spot check your versions as described in the next FAQ.
1086
thepudds584c4bb2018-12-07 12:23:45 -05001087### What can I check if I am not seeing the expected version of a dependency?
thepudds8cca0ba2018-12-07 10:32:11 -05001088
thepudds584c4bb2018-12-07 12:23:45 -050010891. A good first step is to run `go mod tidy`. There is some chance this might resolve the issue, but it will also help put your `go.mod` file into a consistent state with respect to your `.go` source code, which will help make any subsequent investigation easier.
thepudds8cca0ba2018-12-07 10:32:11 -05001090
thepudds584c4bb2018-12-07 12:23:45 -050010912. The second step usually should be to check `go list -m all` to see the list of actual versions selected for your build. `go list -m all` shows you the final selected versions, including for indirect dependencies and after resolving versions for any shared dependencies. It also shows the outcome of any `replace` and `exclude` directives.
thepudds8cca0ba2018-12-07 10:32:11 -05001092
thepudds3a7131b2018-12-13 13:46:33 -050010933. A good next step can be to examine the output of `go mod graph` or `go mod graph | grep <module-of-interest>`. `go mod graph` prints the module requirement graph (including taking into account replacements). Each line in the output has two fields: the first column is a consuming module, and the second column is one of that module's requirements (including the version required by that consuming module). This can be a quick way to see which modules are requiring a particular dependency, including when your build has a dependency that has different required versions from different consumers in your build (and if that is the case, it is important to be familiar with the behavior described in the ["Version Selection"](https://github.com/golang/go/wiki/Modules#version-selection) section above).
thepudds8cca0ba2018-12-07 10:32:11 -05001094
thepudds584c4bb2018-12-07 12:23:45 -05001095`go mod why -m <module>` can also be useful here, although it is typically more useful for seeing why a dependency is included at all (rather than why a dependency ends up with a particular version).
thepudds8cca0ba2018-12-07 10:32:11 -05001096
thepudds8197fef2018-12-12 09:19:32 -05001097`go list` provides many more variations of queries that can be useful to interrogate your modules if needed. One example is the following, which will show the exact versions used in your build excluding test-only dependencies:
1098```
1099go list -deps -f '{{with .Module}}{{.Path}} {{.Version}}{{end}}' ./... | sort -u
1100```
thepudds8cca0ba2018-12-07 10:32:11 -05001101
thepudds192b1f52018-12-09 08:55:19 -05001102A more detailed set of commands and examples for interrogating your modules can be seen in a runnable "Go Modules by Example" [walkthough](https://github.com/go-modules-by-example/index/tree/master/018_go_list_mod_graph_why).
1103
1104One cause of unexpected versions can be due to someone having created an invalid or unexpected `go.mod` file that was not intended, or a related mistake (for example: a `v2.0.1` version of module might have incorrectly declared itself to be `module foo` in its `go.mod` without the required `/v2`; an import statement in `.go` code intended to import a v3 module might be be missing the required `/v3`; a `require` statement in a `go.mod` for a v4 module might be be missing the required `/v4`). Therefore, if the cause of a particular issue you are seeing is not obvious, it can be worthwhile to first re-read the material in the ["go.mod"](https://github.com/golang/go/wiki/Modules#gomod) and ["Semantic Import Versioning"](https://github.com/golang/go/wiki/Modules#semantic-import-versioning) sections above (given these include important rules that modules must follow) and then take a few minutes to spot check the most relevant `go.mod` files and import statements.
thepudds8cca0ba2018-12-07 10:32:11 -05001105
1106### Why am I getting an error 'cannot find module providing package foo'?
1107
1108This is a general error message that can occur for several different underlying causes.
1109
1110In some cases, this error is simply due to a mistyped path, so the first step likely should be to double-check for incorrect paths based on the details listed in the error message.
1111
thepudds192b1f52018-12-09 08:55:19 -05001112If you have not already done so, a good next step is often to try `go get -v foo` or `go get -v -x foo`:
thepudds8cca0ba2018-12-07 10:32:11 -05001113* In general, `go get` will often provide more a detailed error message than `go build`.
thepudds192b1f52018-12-09 08:55:19 -05001114* See the first troubleshooting FAQ in this section [above](https://github.com/golang/go/wiki/Modules#what-are-some-general-things-i-can-spot-check-if-i-am-seeing-a-problem) for more details.
thepudds8cca0ba2018-12-07 10:32:11 -05001115
1116Some other possible causes:
1117
thepuddsbb4f9f02018-12-13 13:24:11 -05001118* You might see the error `cannot find module providing package foo` if you have issued `go build` or `go build .` but do not have any `.go` source files in the current directory. If this is what you are encountering, the solution might be an alternative invocation such as `go build ./...` (where the `./...` expands out to match all the packages within the current module). See [#27122](https://github.com/golang/go/issues/27122).
thepudds8cca0ba2018-12-07 10:32:11 -05001119
thepudds192b1f52018-12-09 08:55:19 -05001120* The module cache in Go 1.11 can cause this error, including in the face of network issues or multiple `go` commands executing in parallel. This is resolved in Go 1.12. See the first troubleshooting FAQ in this section [above](https://github.com/golang/go/wiki/Modules#what-are-some-general-things-i-can-spot-check-if-i-am-seeing-a-problem) for more details and possible corrective steps.
thepudds8cca0ba2018-12-07 10:32:11 -05001121
thepudds93dda5a2018-12-07 10:43:54 -05001122### Why does 'go mod init' give the error 'cannot determine module path for source directory'?
thepudds8cca0ba2018-12-07 10:32:11 -05001123
1124`go mod init` without any arguments will attempt to guess the proper module path based on different hints such as VCS meta data. However, it is not expected that `go mod init` will always be able to guess the proper module path.
1125
thepuddsa2751192018-12-07 10:41:52 -05001126If `go mod init` gives you this error, those heuristics were not able to guess, and you must supply the module path yourself (such as `go mod init github.com/you/hello`).
thepudds8cca0ba2018-12-07 10:32:11 -05001127
thepuddsca39a402018-12-20 02:24:08 -05001128### I have a problem with a complex dependency that has not opted in to modules. Can I use information from its current dependency manager?
1129
1130Yes. This requires some manual steps, but can be helpful in some more complex cases.
1131
1132When you run `go mod init` when initializing your own module, it will automatically convert from a prior dependency manager by translating configuration files like `Gopkg.lock`, `glide.lock`, or `vendor.json` into a `go.mod` file that contains corresponding `require` directives. The information in a pre-existing `Gopkg.lock` file for example usually describes version information for all of your direct and indirect dependencies.
1133
1134However, if instead you are adding a new dependency that has not yet opted in to modules itself, there is not a similar automatic conversion process from any prior dependency manager that your new dependency might have been using. If that new dependency itself has non-module dependencies that have had breaking changes, then in some cases that can cause incompatibility problems. In other words, a prior dependency manager of your new dependency is not automatically used, and that can cause problems with your indirect dependencies in some cases.
1135
1136One approach is to run `go mod init` on your problematic non-module direct dependency to convert from its current dependency manager, and then use the `require` directives from the resulting temporary `go.mod` to populate or update the `go.mod` in your module.
1137
1138For example, if `github.com/some/nonmodule` is a problematic direct dependency of your module that is currently using another dependency manager, you can do something similar to:
1139
1140```
thepudds6f4a0262019-02-26 13:56:53 -05001141$ git clone -b v1.2.3 https://github.com/some/nonmodule /tmp/scratchpad/nonmodule
thepuddsca39a402018-12-20 02:24:08 -05001142$ cd /tmp/scratchpad/nonmodule
1143$ go mod init
1144$ cat go.mod
1145```
1146
thepudds6f4a0262019-02-26 13:56:53 -05001147The resulting `require` information from the temporary `go.mod` can be manually moved into the actual `go.mod` for your module, or you can consider using https://github.com/rogpeppe/gomodmerge, which is a community tool targeting this use case. In addition, you will want to add a `require github.com/some/nonmodule v1.2.3` to your actual `go.mod` to match the version that you manually cloned.
1148
1149A concrete example of following this technique for docker is in this [#28489 comment](https://github.com/golang/go/issues/28489#issuecomment-454795390), which illustrates getting a consistent set of versions
1150of docker dependencies to avoid case sensitive issues between `github.com/sirupsen/logrus` vs. `github.com/Sirupsen/logrus`.
thepuddsca39a402018-12-20 02:24:08 -05001151
thepudds584c4bb2018-12-07 12:23:45 -05001152### Why does 'go build' require gcc, and why are prebuilt packages such as net/http not used?
thepudds856618c2018-09-07 00:20:35 -04001153
1154In short:
1155
1156> Because the pre-built packages are non-module builds and can’t be reused. Sorry. Disable cgo for now or install gcc.
1157
1158This is only an issue when opting in to modules (e.g., via `GO111MODULE=on`). See [#26988](https://github.com/golang/go/issues/26988#issuecomment-417886417) for additional discussion.
1159
1160### Do modules work with relative imports like `import "./subdir"`?
1161
1162No. See [#26645](https://github.com/golang/go/issues/26645#issuecomment-408572701), which includes:
1163
1164> In modules, there finally is a name for the subdirectory. If the parent directory says "module m" then the subdirectory is imported as "m/subdir", no longer "./subdir".
1165
Sylvain Laurent5b4c4e02018-10-03 17:02:28 +02001166### Some needed files may not be present in populated vendor directory
1167
thepudds83477c02018-12-07 08:56:31 -05001168Directories without `.go` files are not copied inside the `vendor` directory by `go mod vendor`. This is by design.
Sylvain Laurent5b4c4e02018-10-03 17:02:28 +02001169
thepudds83477c02018-12-07 08:56:31 -05001170In short, setting aside any particular vendoring behavior – the overall model for go builds is that the files needed to build a package should be in the directory with the `.go` files.
thepudds7afc40d2018-09-24 22:06:51 -04001171
thepudds83477c02018-12-07 08:56:31 -05001172Using the example of cgo – modifying C source code in other directories will not trigger a rebuild, and instead your build will use stale cache entries. The cgo documentation now [includes](https://go-review.googlesource.com/c/go/+/125297/5/src/cmd/cgo/doc.go):
thepudds84465052018-09-07 01:09:12 -04001173
thepudds83477c02018-12-07 08:56:31 -05001174> Note that changes to files in other directories do not cause the package
1175to be recompiled, so _all non-Go source code for the package should be
1176stored in the package directory_, not in subdirectories.
thepudds84465052018-09-07 01:09:12 -04001177
thepudds584c4bb2018-12-07 12:23:45 -05001178A community tool https://github.com/goware/modvendor allows you to easily copy a complete set of .c, .h, .s, .proto or other files from a module into the `vendor` director. Although this can be helpful, some care must be taken to make sure your go build is being handled properly in general (regardless of vendoring) if you have files needed to build a package that are outside of the directory with the `.go` files.
Dmitry Savintsev660f1e32018-10-12 13:49:21 +02001179
thepudds83477c02018-12-07 08:56:31 -05001180See additional discussion in [#26366](https://github.com/golang/go/issues/26366#issuecomment-405683150).
thepudds6a3214d2018-12-14 08:59:46 -05001181
1182An alternative approach to traditional vendoring is to check in the module cache. It can end up with similar benefits as traditional vendoring and in some ways ends up with a higher fidelity copy. This approach is explained as a "Go Modules by Example" [walkthrough](https://github.com/go-modules-by-example/index/blob/master/012_modvendor/README.md).