design/2981: coverage
Added coverage result to JSON output
Renamed {Benchmark,Test}Result.Output to Log, to make it even more
obvious.
Specified that examples are treated as tests.
Change-Id: I80ae1c813ea21508a9f2093fb5e1ea586fc2fa8a
Reviewed-on: https://go-review.googlesource.com/15704
Reviewed-by: Minux Ma <minux@golang.org>
diff --git a/design/2981-go-test-json.md b/design/2981-go-test-json.md
index 71efb8f..9555345 100644
--- a/design/2981-go-test-json.md
+++ b/design/2981-go-test-json.md
@@ -60,7 +60,10 @@
* Add `func Test(f func(*T)) TestResult`
to be consistent with `func Benchmark(f func(*B)) BenchmarkResult`.
This is not required for JSON output.
- * Add `Name`, `Output` and `Procs` fields to `BenchmarkResult`.
+ * Add `Name`, `Log` and `Procs` fields to `BenchmarkResult`.
+ * Add `type CoverageResult`.
+ * In type `Cover`, change `CoveredPackages` field type from `string` to
+ `[]string`. This type is not covered by Go 1 compatibility guidelines.
Type definitions and details below.
@@ -78,25 +81,35 @@
SKIP
)
-// The results of a test run.
+// The results of a test or an example.
type TestResult struct {
- Name string
- State TestState
- T time.Duration // The total time taken.
- Output string // The log created by calling (*T).Log and (*T).Logf.
+ Name string
+ State TestState
+ T time.Duration // The total time taken.
+ Log string // The log created by calling (*T).Log and (*T).Logf.
}
// Test runs a test function and returns results.
func Test(f func(*T)) TestResult
type BenchmarkResult struct {
- Name string
- Procs int // The value of runtime.GOMAXPROCS for this benchmark run.
- Output string // The log created by calling (*B).Log and (*B).Logf.
+ Name string
+ Procs int // The value of runtime.GOMAXPROCS for this benchmark run.
+ Log string // The log created by calling (*B).Log and (*B).Logf.
// existing fields
}
+// CoverageResult is aggregated code coverage info.
+// It is used for `go test` JSON output.
+// To get full coverage info, use -coverprofile flag in go test.
+type CoverageResult struct {
+ Mode string
+ TotalStatements int64
+ ActiveStatements int64
+ CoveredPackages []string
+}
+
// result is used for test binary JSON output format.
// It is not added to the public API.
//
@@ -105,6 +118,7 @@
type result struct {
Test *TestResult `json:",omitempty"`
Benchmark *BenchmarkResult `json:",omitempty"`
+ Coverage *CoverageResult `json:",omitempty"`
}
```
@@ -125,11 +139,18 @@
"Name": "TestBar",
"State": "PASS",
"T": 1000000,
- "Output": "some test output\n"
+ "Log": "some test output\n"
}
}
{
- "Benchmark": {
+ "Test": {
+ "Name": "Example1",
+ "State": "PASS",
+ "T": 1000000,
+ }
+}
+{
+ "Benchmark": {
"Name": "BenchmarkBar",
"State": "PASS",
"T": 1000000,
@@ -139,6 +160,16 @@
"MemBytes": 0
}
}
+{
+ "Coverage": {
+ "Mode": "set",
+ "TotalStatements": 1000,
+ "ActiveStatements": 900,
+ "CoveredPackages": [
+ "example.com/foobar"
+ ]
+ }
+}
```
### `go test`
@@ -152,6 +183,7 @@
Test *TestResult `json:",omitempty"`
Benchmark *BenchmarkResult `json:",omitempty"`
+ Coverage *CoverageResult `json:",omitempty"`
Stdout string `json:",omitempty"` // Unrecognized stdout of the test binary.
Stderr string `json:",omitempty"` // Stderr output line of the test binary.
}
@@ -179,7 +211,15 @@
"Name": "TestBar",
"State": "PASS",
"T": 1000000,
- "Output": "some test output\n"
+ "Log": "some test output\n"
+ }
+}
+{
+ "Package": "example.com/foobar",
+ "Test": {
+ "Name": "Example1",
+ "State": "PASS",
+ "T": 1000000,
}
}
{
@@ -194,6 +234,18 @@
"MemBytes": 0
}
}
+{
+ "Package": "example.com/foobar",
+ "Coverage": {
+ "Mode": "set",
+ "TotalStatements": 1000,
+ "ActiveStatements": 900,
+ "CoveredPackages": [
+ "example.com/foobar"
+ ]
+ }
+}
+
```
## Rationale
@@ -236,7 +288,9 @@
## Compatibility
-The API changes are fully backwards compatible.
+The only backwards incompatibility is changing `testing.Cover.CoveredPackages`
+field type, but `testing.Cover` is not covered by Go 1 compatibility
+guidelines.
## Implementation
@@ -249,9 +303,9 @@
1. Add `type TestResult`, `type TestStatus` and
`func Test(f func(*T)) TestResult` to package `testing`.
Modify `testing.tRunner` to create `TestResult`.
-1. Add `-test.json` flag to the `testing` package.
- Modify `testing.(*T).report` and `testing.RunBenchmarks` functions
- to print JSON if `-test.json` is specified.
+1. Add `-test.json` flag and `type CoverageResult` to the `testing` package.
+ Modify `(*T).report`, `RunBenchmarks`, `coverReport` and `runExample`
+ functions to print JSON if `-test.json` is specified.
If `-test.verbose` is specified, print verbose messages to stderr.
1. Add `-json` flag to `go test`.
If specified, pass `-test.json` to test binaries.
@@ -262,4 +316,4 @@
The goal is to get agreement on this proposal and to complete the work
before the 1.6 freeze date.
-[testStreamOutput]: https://github.com/golang/go/blob/0b248cea169a261cd0c2db8c014269cca5a170c4/src/cmd/go/test.go#L361-L369
\ No newline at end of file
+[testStreamOutput]: https://github.com/golang/go/blob/0b248cea169a261cd0c2db8c014269cca5a170c4/src/cmd/go/test.go#L361-L369
