gopls/integration/govim: update build steps to capture artifacts
Govim integration tests generate a number of artifacts, including both
the govim and gopls logs, that can be useful in debugging failures. This
change updates our cloud build configuration to capture these artifacts,
along with several other minor improvements.
Notably artifacts are uploaded to GCS as a separate build step, so that
we have the potential to use its granular permission model for sharing
these artifacts. Right now, this requires temporarily swallowing the
exit code of `go test` so that the build can proceed.
Also:
- Update govim to a newer version; we still can't use latest as there
isn't a tagged version that contains the requisite flag change.
- Alter the test harness to run tests from the github.com/govim/govim
module root.
- Switch use a major version label when referring to the test harness
build step, to allow for breaking changes (such as the one made
here).
- Add a missing copyright header to run_local.sh.
- Update run_local.sh to work with the modified harness.
- Update documentation accordingly.
Change-Id: Ie5ddaf54e775371a36163f98c1beb90c217be931
Reviewed-on: https://go-review.googlesource.com/c/tools/+/214577
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
diff --git a/gopls/integration/govim/Dockerfile b/gopls/integration/govim/Dockerfile
index c3a0bde..cb278db 100644
--- a/gopls/integration/govim/Dockerfile
+++ b/gopls/integration/govim/Dockerfile
@@ -6,11 +6,19 @@
# distros, so we build from their base image.
FROM govim/govim:go1.13.5_vim_v8.1.2414_v1
-# Get a copy of govim in order to run its integration tests. We use a pinned
-# version so that this build is repeatable, and so that we're not sensitive to
-# test breakages in govim.
+# We use a pinned version of govim so that this build is repeatable, and so
+# that we're not sensitive to test breakages in govim.
# TODO(findleyr): Once a version of govim has been tagged containing
# https://github.com/govim/govim/pull/629, switch this to @latest.
-ENV GOPROXY=https://proxy.golang.org GOPATH=/go VIM_FLAVOR=vim
-WORKDIR /src/mod
-RUN go mod init mod && go get -t github.com/govim/govim@v0.0.27-0.20191220164001-63ce556bb69e
+ENV GOPROXY=https://proxy.golang.org GOPATH=/go VIM_FLAVOR=vim \
+ GOVIM_VERSION=v0.0.27-0.20200108180210-6236474c3ac7
+WORKDIR /src
+
+# Get govim, as well as all dependencies for the ./cmd/govim tests. In order to
+# use the go command's module version resolution, we use `go mod download` for
+# this rather than git. Another approach would be to use a temporary module and
+# `go get -t -u github.com/govim/govim`, but that may cause test dependencies
+# to be upgraded.
+RUN GOVIM_DIR=$(go mod download -json github.com/govim/govim@$GOVIM_VERSION | jq -r .Dir) && \
+ cp -r $GOVIM_DIR /src/govim && cd /src/govim && \
+ go list -test -deps ./cmd/govim
diff --git a/gopls/integration/govim/README.md b/gopls/integration/govim/README.md
index 005ccff..c3ae925 100644
--- a/gopls/integration/govim/README.md
+++ b/gopls/integration/govim/README.md
@@ -5,17 +5,24 @@
## Running on GCP
-To run these integration tests in Cloud Build (assuming the `gcloud` command is
-configured for a valid GCP project):
+To run these integration tests in Cloud Build, use the following steps. Here
+we assume that `$PROJECT` is a valid GCP project and `$BUCKET` is a cloud
+storage bucket owned by that project.
- `cd` to the root directory of the tools project.
-- (at least once per GCP project) Build the test harness:
+- (at least once per GCP project) Build the test harness:
```
-$ gcloud builds submit --config=gopls/integration/govim/cloudbuild.harness.yaml
+$ gcloud builds submit \
+ --project="${PROJECT}" \
+ --config=gopls/integration/govim/cloudbuild.harness.yaml \
+ --substitutions=_RESULT_BUCKET="${BUCKET}"
```
-- Run the integration tests:
+- Run the integration tests:
```
-$ gcloud builds submit --config=gopls/integration/govim/cloudbuild.yaml
+$ gcloud builds submit \
+ --project="${PROJECT}" \
+ --config=gopls/integration/govim/cloudbuild.yaml \
+ --substitutions=_RESULT_BUCKET="${BUCKET}"
```
## Running locally
diff --git a/gopls/integration/govim/cloudbuild.harness.yaml b/gopls/integration/govim/cloudbuild.harness.yaml
index 974a66b..0deb8af 100644
--- a/gopls/integration/govim/cloudbuild.harness.yaml
+++ b/gopls/integration/govim/cloudbuild.harness.yaml
@@ -6,7 +6,11 @@
# for gopls. See README.md for instructions on how to use this.
steps:
- name: 'gcr.io/cloud-builders/docker'
- args: ['build', '-t', 'gcr.io/$PROJECT_ID/govim-harness',
+ args: ['build',
+ # To allow for breaking changes to this test harness, tag with a major
+ # version number.
+ '-t', 'gcr.io/$PROJECT_ID/govim-harness:latest',
+ '-t', 'gcr.io/$PROJECT_ID/govim-harness:2',
# It is assumed that this build is running from the root directory of the
# tools repository.
'-f', 'gopls/integration/govim/Dockerfile',
diff --git a/gopls/integration/govim/cloudbuild.yaml b/gopls/integration/govim/cloudbuild.yaml
index 2cf2c7f..a19c787 100644
--- a/gopls/integration/govim/cloudbuild.yaml
+++ b/gopls/integration/govim/cloudbuild.yaml
@@ -4,20 +4,42 @@
# Build gopls, and run the govim integration tests. See README.md for
# instructions on how to use this.
+
+substitutions:
+ # This bucket must be owned by the the GCP project executing the build. If
+ # you are running this from your own project, override using --substitutions.
+ _RESULT_BUCKET: 'golang-gopls_integration_tests'
+
steps:
+ # Build gopls from source, to use with the govim integration tests.
- name: 'golang:1.13'
env: ['GOPROXY=https://proxy.golang.org']
dir: 'gopls'
args: ['go', 'build']
- - name: 'gcr.io/$PROJECT_ID/govim-harness'
- # Work in the dummy module created in the test harness, that requires a
- # pinned version of github.com/govim/govim.
- dir: '/src/mod'
- # The below setting currently causes too much noise on STDERR.
- # TODO(findleyr): look into govim changes that make it easier to capture
- # individual logs as build artifacts.
- # env: ['GOVIM_TESTSCRIPT_STDERR=true']
- args: ['go', 'test',
- 'github.com/govim/govim/cmd/govim',
- # Specify the path to the gopls binary built in step 0.
- '-gopls', '/workspace/gopls/gopls']
+
+ # Run the tests. Note that the script in this step does not return the exit
+ # code from `go test`, but rather saves it for use in the final step after
+ # uploading artifacts.
+ - name: 'gcr.io/$PROJECT_ID/govim-harness:2'
+ dir: '/src/govim'
+ env:
+ - GOVIM_TESTSCRIPT_WORKDIR_ROOT=/workspace/artifacts
+ - VIM_FLAVOR=vim
+ args: ['/workspace/gopls/integration/govim/run_tests_for_cloudbuild.sh']
+
+ # The govim tests produce a large number of artifacts; tarball/gzip to reduce
+ # roundtrips and save space.
+ - name: 'ubuntu'
+ args: ['tar', '-czf', 'artifacts.tar.gz', 'artifacts']
+
+ # Upload artifacts to GCS.
+ - name: 'gcr.io/cloud-builders/gsutil'
+ args: ['cp', 'artifacts.tar.gz', 'gs://${_RESULT_BUCKET}/govim/${BUILD_ID}/artifacts.tar.gz']
+
+ # Exit with the actual exit code of the integration tests.
+ - name: 'ubuntu'
+ args: ['bash', 'govim_test_result.sh']
+
+# Write build logs to the same bucket as artifacts, so they can be more easily
+# shared.
+logsBucket: 'gs://${_RESULT_BUCKET}'
diff --git a/gopls/integration/govim/run_local.sh b/gopls/integration/govim/run_local.sh
index 35f3c8d..9cbf629 100755
--- a/gopls/integration/govim/run_local.sh
+++ b/gopls/integration/govim/run_local.sh
@@ -1,5 +1,9 @@
#!/bin/bash -e
+# Copyright 2019 The Go Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style
+# license that can be found in the LICENSE file.
+
# Run govim integration tests against a local gopls.
# TODO(findleyr): this script assumes that docker may be run without sudo.
# Update it to escalate privileges if and only if necessary.
@@ -25,7 +29,7 @@
temp_gopls_name=$(basename "${temp_gopls}")
docker run --rm -t \
-v "${tools_dir}:/src/tools" \
- -w "/src/mod" \
+ -w "/src/govim" \
gopls-govim-harness \
- go test github.com/govim/govim/cmd/govim \
+ go test ./cmd/govim \
-gopls "/src/tools/gopls/${temp_gopls_name}"
diff --git a/gopls/integration/govim/run_tests_for_cloudbuild.sh b/gopls/integration/govim/run_tests_for_cloudbuild.sh
new file mode 100755
index 0000000..47a9cfd
--- /dev/null
+++ b/gopls/integration/govim/run_tests_for_cloudbuild.sh
@@ -0,0 +1,16 @@
+#!/bin/bash
+
+# Copyright 2020 The Go Authors. All rights reserved.
+# Use of this source code is governed by a BSD-style
+# license that can be found in the LICENSE file.
+
+# This script runs govim integration tests but always succeeds, instead writing
+# their result to a file so that any test failure can be deferred to a later
+# build step. We do this so that we can capture govim test artifacts regardless
+# of the test results.
+
+# Substitute the locally built gopls binary for use in govim integration tests.
+go test ./cmd/govim -gopls /workspace/gopls/gopls
+
+# Stash the error, for use in a later build step.
+echo "exit $?" > /workspace/govim_test_result.sh
