Running tests with the
-json flag or passing test output through
go tool test2json1 will produce a stream of JSON events. Each event specifies an action, such as
output, etc. An event may specify what test it belongs to. The VSCode Go test controller must capture these events in order to notify VSCode of test output and lifecycle events.
Processing test events generated by
TestXxx(*testing.T) functions is easy. Events with an empty
Test field can be ignored, and all other events have a meaningful
Action field. Output is recorded, and run/pass/fail/skip events are converted to VSCode test API events.
go#37555 did require special handling, but that only appeared in Go 1.14 and was backported to 1.14.1.
Test events generated by
BenchmarkXxx(*testing.B) functions require significantly more processing. If a benchmark fails or is skipped, the
Action fields are populated appropriately. Otherwise,
Test is empty and
Action is always
output. Thus, nominal lifecycle events (run/pass) must be deduced purely from test output. When a benchmark begins, an output such as
BenchmarkFooBar\n is produced. When a benchmark completes, an output such as
BencmarkFooBar-4 123456 123.4 ns/op 123 B/op 12 allocs/op is produced. No explicit
pass events are generated. Thus:
BenchmarkFooBar\n is seen, the benchmark will be marked as running
Thus, a benchmark that does not produce results (and does not fail or skip) will never produce an event indicating that it has completed. Benchmarks that call
(*testing.B).Run will not produce results. In practice, this means that any incomplete benchmarks must be explicitly marked as passed once
go test returns.