cmd/go/internal/vgo: load packages in parallel

This is a large CL. The best way to read it is probably to treat
vgo/load.go as if it were a newly-written file, not by reading diffs.

The main motivation here is to do package loading in parallel,
including the on-demand downloading of dependency zip files.
Running the load in parallel requires making the structure a bit
cleaner than it was before: the old recursive traversal could
retraverse sections of the graph if it decided that now it needed
tests and their dependencies, and so on. To parallelize, the work
had to be split up into items visited exactly once, and the result
is a bit cleaner.

Most of vgo/search.go has been absorbed into load.go's ImportPaths.
Previously, there was an odd recursion where ImportPaths's loader
callback could call into MatchAll/MatchALL, which created and ran
a new loader before returning back to the original loader's callback.
The new structure more properly expresses the dependency of
the loader on matching, which in turn depends on the current state
of the loader (namely buildList).

The magic "ALL" package pattern is gone. Code that needs that
functionality now has a proper entry point (vgo.LoadALL).

Finally, isolate all the "what happened?" loader state consulted
after the fact into a single struct. The build list remains separate,
at least for now.

The cleaner structure sets us up well for the next CL.

Fixes golang/go#25162.

Change-Id: I963f0effd7314f7e77eaad058774dbd2586c9cd2
Reviewed-on: https://go-review.googlesource.com/121303
Reviewed-by: Bryan C. Mills <bcmills@google.com>
12 files changed
tree: 28f0c0fc517d4bac38261f6ab942c8dc8219ae0d
  1. vendor/
  2. AUTHORS
  3. codereview.cfg
  4. CONTRIBUTING.md
  5. CONTRIBUTORS
  6. LICENSE
  7. main.go
  8. PATENTS
  9. README.md
README.md

Versioned Go Prototype (vgo)

This repository holds a prototype of what the go command might look like with integrated support for package versioning.

See research.swtch.com/vgo for documents about the design.

Download/Install

Use go get -u golang.org/x/vgo.

You can also manually git clone the repository to $GOPATH/src/golang.org/x/vgo.

Report Issues / Send Patches

See CONTRIBUTING.md.

This is still a very early prototype. You are likely to run into bugs. Please file bugs in the main Go issue tracker, golang.org/issue, and put the prefix x/vgo: in the issue title.

Thank you.