cmd/coordinator: use 'go tool dist test -k' on release branches
Right now, on release branches, testing is stopped very soon after
encountering the first package with a failing test. For example:
[...]
FAIL cmd/go/internal/modfetch 17.181s
ok cmd/go/internal/modfetch/codehost 5.474s
FAIL
2019/11/07 02:35:34 Failed: exit status 1
Ideally we should not have failing tests on release branches for long,
but that is not the current state. Until we reach that state, make
coordinator run 'go tool dist test' with -k flag so that it keeps
testing all remaining packages even if one fails, and report complete
test results in the log.
That way, a package that fails early doesn't make it completely
impossible to see if other package tests pass or fail.
Updates golang/go#14305
Updates golang/go#36181
Change-Id: Ia674005ae45c6b9dbffa6f5b56d60b7ed38f6b34
Reviewed-on: https://go-review.googlesource.com/c/build/+/211678
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
diff --git a/cmd/coordinator/coordinator.go b/cmd/coordinator/coordinator.go
index 45fcb80..7365bb7 100644
--- a/cmd/coordinator/coordinator.go
+++ b/cmd/coordinator/coordinator.go
@@ -2059,6 +2059,19 @@
func (st *buildStatus) HasBuildlet() bool { return atomic.LoadInt32(&st.hasBuildlet) != 0 }
+// useKeepGoingFlag reports whether this build should use -k flag of 'go tool
+// dist test', which makes it keep going even when some tests have failed.
+func (st *buildStatus) useKeepGoingFlag() bool {
+ // For now, keep going for post-submit builders on release branches,
+ // because we prioritize seeing more complete test results over failing fast.
+ // Later on, we may start doing this all post-submit builders on all branches.
+ // See golang.org/issue/14305.
+ //
+ // TODO(golang.org/issue/36181): A more ideal long term solution is one that reports
+ // a failure fast, but still keeps going to make all other test results available.
+ return !st.isTry() && strings.HasPrefix(st.branch, "release-branch.go")
+}
+
func (st *buildStatus) isTry() bool { return st.trySet != nil }
func (st *buildStatus) isSlowBot() bool {
@@ -3236,6 +3249,9 @@
if st.conf.CompileOnly {
args = append(args, "--compile-only")
}
+ if st.useKeepGoingFlag() {
+ args = append(args, "-k")
+ }
args = append(args, names...)
var buf bytes.Buffer
t0 := time.Now()
