Author(s): Conrad Irwin
Last updated: 2023-05-16
Discussion at https://golang.org/issue/48429.
Authors of go modules frequently use tools that are written in go and distributed as go modules. Although go has good support for managing dependencies imported into their programs, the support for tools used during development is comparatively weak.
To make it easier for go developers to use tools written in go
go.mod should gain a new directive that lets module authors define which tools are needed.
It is desirable that all collaborators on a given project use the same version of tools to avoid the output changing slightly on different people’s machines. This comes up particularly with tools like linters (where changes overtime may change whether or not the code is considered acceptable) and code generation (where the generated code must be assumed to match the version of the library that is linked).
The currently recommended approach to this is to create a file called
tools.go that imports the package containing the tools to make the dependencies visible to the module graph. To hide this file from the compiler, it is necessary to exclude it from builds by adding an unused build tag such as
//go:build tools. To hide this file from other packages that depend on your module, it must be put in its own package inside your module.
This approach is quite fiddly to use correctly, and still has a few downsides:
go run golang.org/x/tools/cmd/stringer, and so projects often contain wrapper scripts.
go run relinks tools every time they are run, which may be noticeably slow.
People work around this by either globally installing tools, which may lead to version skew, or by installing and using third party tooling (like accio) to manage their tools instead.
go.mod gains a new directive:
This acts exactly as though you had a correctly set up
tools.go that contains
As with other directives, multiple
tool directives can be parenthesized:
go 1.22 tool ( golang.org/x/tools/cmd/stringer ./cmd/migrate )
Is equivalent to:
go 1.22 tool golang.org/x/tools/cmd/stringer tool ./cmd/migrate
To allow automated changes
go mod edit will gain two new parameters:
-tool path/to/package and
-droptool path/to/package that add and remove
tool directives respectively.
To allow users to easily add new tools,
go get will gain a new parameter:
go get is run with the
-tool parameter, then it will download the specified package and add it to the module graph as it does today. Additionally it will add a new
tool directive to the current module’s
If you combine the
-tool flag with the
@none version, then it will also remove the
tool directive from your
go tool is run in module mode with an argument that does not match a go builtin tool, it will search the current
go.mod for a tool directive that matches the last path segment and compile and run that tool similarly to
For example if your go.mod contains:
tool golang.org/x/tools/cmd/stringer require golang.org/x/tools v0.9.0
go tool stringer will act similarly to
go run email@example.com, and
go tool with no arguments will also list
stringer as a known tool.
In the case that two tool directives end in the same path segment,
go tool X will error. In the case that a tool directive ends in a path segment that corresponds to a builtin go tool, the builtin tool will be run. In both cases you can use
go tool path/to/package to specify what you want unconditionally.
The only difference from
go run is that
go tool will cache the built binary in
$GOCACHE/tool/<current-module-path>/<TOOLNAME>. Subsequent runs of
go tool X will then check that the built binary is up to date, and only rebuild it if necessary to speed up re-using tools.
When the go cache is trimmed, any tools that haven't been used in the last five days will be deleted. Five days was chosen arbitrarily as it matches the expiry used for existing artifacts. Running
go clean -cache will also remove all of these binaries.
We will add a new metapackage
tools that contains all of the tools in the current modules
This would allow for the following operations:
# Install all tools in GOBIN go install tools # Build and cache tools so `go tool X` is fast: go build tools # Update all tools to their latest versions. go get tools # Install all tools in the bin/ directory go build -o bin/ tools
This proposal tries to improve the workflow of go developers who use tools packaged as go modules while developing go modules. It deliberately does not try and solve the problem of versioning arbitrary binaries: anything not distributed as a go module is out of scope.
There were a few choices that needed to be made, explained below:
require directives in
go.mod allows us to do this without introducing a separate dependency tree or resolution path. This also means that you can use
replace directives to control the dependencies used when building your tools.
go tool X allows go to handle versioning for you, unlike installing binaries to your path.
go run today) as tools are likely to be reused. Reusing the existing go cache and expiry allows us to do this in a best-effort way without filling up the users’ disk if they develop many modules with a large number of tools.
go tool X always defaults to the tool that ships with the go distribution in case of conflict, so that it always acts as you expect.
There’s no language change, however we are changing the syntax of
go.mod. This should be ok, as the file-format was designed with forward compatibility in mind.
If go adds tools to the distribution in the future that conflict with tools added to projects’
go.mod files, this may cause compatibility issues in the future. I think this is likely not a big problem in practice, as I expect new tools to be rare. Experience from using
$PATH as a shared namespace for executables suggests that name conflicts in binaries can be easily avoided in practice.
I plan to work on this for go1.22.
This should probably not do anything special with workspaces. Because tools must be present in the
require directives of a module, there is no easy way to make them work at a workspace level instead of a module level.
It might be possible to try and union all tools in all modules in the workspace, but I suggest we defer this to future work if it’s desired.