cmd/go: allow go get with arbitrary URLs

This CL permits using arbitrary, non-VCS-qualified URLs as
aliases for fully VCS-qualified and/or well-known code hosting
sites.

Example 1) A VCS-qualified URL can now be shorter.

Before:
$ go get camlistore.org/r/p/camlistore.git/pkg/blobref
After:
$ go get camlistore.org/pkg/blobref

Example 2) A custom domain can be used as the import,
referencing a well-known code hosting site.

Before:
$ go get github.com/bradfitz/sonden
After:
$ go get bradfitz.com/pkg/sonden

The mechanism used is a <meta> tag in the HTML document
retrieved from fetching:

    https://<import>?go-get=1  (preferred)
    http://<import>?go-get=1   (fallback)

The meta tag should look like:

<meta name="go-import" content="import-alias-prefix vcs full-repo-root">

The full-repo-root must be a full URL root to a repository containing
a scheme and *not* containing a ".vcs" qualifier.

The vcs is one of "git", "hg", "svn", etc.

The import-alias-prefix must be a prefix or exact match of the
package being fetched with "go get".

If there are multiple meta tags, only the one with a prefix
matching the import path is used. It is an error if multiple
go-import values match the import prefix.

If the import-alias-prefix is not an exact match for the import,
another HTTP fetch is performed, at the declared root (which does
*not* need to be the domain's root).

For example, assuming that "camlistore.org/pkg/blobref" declares
in its HTML head:

<meta name="go-import" content="camlistore.org git https://camlistore.org/r/p/camlistore" />

... then:

$ go get camlistore.org/pkg/blobref

... looks at the following URLs:

   https://camlistore.org/pkg/blobref?go-get=1
   http://camlistore.org/pkg/blobref?go-get=1
   https://camlistore.org/?go-get=1
   http://camlistore.org/?go-get=1

Ultimately it finds, at the root (camlistore.org/), the same go-import:

<meta name="go-import" content="camlistore.org git https://camlistore.org/r/p/camlistore" />

... and proceeds to trust it, checking out git //camlistore.org/r/p/camlistore at
the import path of "camlistore.org" on disk.

Fixes #3099

R=r, rsc, gary.burd, eikeon, untheoretic, n13m3y3r, rsc
CC=golang-dev
https://golang.org/cl/5660051
5 files changed