cmd/go: parallelize module load operations

Loading information about modules not yet cached
on the local system is dominated by network I/O time.
Load the information in parallel to overlap this I/O.

As a test case, I am using a real (not synthetic) but unpublished
repo with 90 transitive dependencies. In that repo,
I've measured the time it takes for

	rm -rf $GOPATH/src/v && vgo mod -vendor

Before this CL, that operation averages about 220 seconds.

Adding caching of already-looked-up information
in modfetch/repo.go drops the average time to about 170 seconds.

Changing the MVS requirement scans to run in parallel
drops the average time to about 65 seconds.
This CL includes Bryan's comments for mvs.Reqs
from CL 116235, extended to document the new parallel
access requirements.

This CL also adds the build -x flag to 'vgo mod' so that it's
possible to see all the git commands being executed.

Fixes golang/go#24316.

Change-Id: Ia8228e850c6c60d8fa80d2fb3b6773c70e86137b
Reviewed-on: https://go-review.googlesource.com/119055
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
7 files changed
tree: 370a24762b5ea19e5a43961c2483548d8adb097d
  1. vendor/
  2. AUTHORS
  3. codereview.cfg
  4. CONTRIBUTING.md
  5. CONTRIBUTORS
  6. LICENSE
  7. main.go
  8. PATENTS
  9. README.md
README.md

Versioned Go Prototype (vgo)

This repository holds a prototype of what the go command might look like with integrated support for package versioning.

See research.swtch.com/vgo for documents about the design.

Download/Install

Use go get -u golang.org/x/vgo.

You can also manually git clone the repository to $GOPATH/src/golang.org/x/vgo.

Report Issues / Send Patches

See CONTRIBUTING.md.

This is still a very early prototype. You are likely to run into bugs. Please file bugs in the main Go issue tracker, golang.org/issue, and put the prefix x/vgo: in the issue title.

Thank you.