14313-benchmark-format: minor refreshes This updates some links and some stale information in this design doc. This is slightly odd because we use this design doc also as a specification, which creates some tension between whether it should continue to capture the state of the world at the time or be kept fully up-to-date. I tried to take a light touch in this CL. Change-Id: Ifa3cf7d70e0f6a65cd9184f76f449d5618211f63 Reviewed-on: https://go-review.googlesource.com/c/proposal/+/357529 Reviewed-by: Michael Knyszek <mknyszek@google.com>
diff --git a/design/14313-benchmark-format.md b/design/14313-benchmark-format.md index 908ed8d..3fb3ef7 100644 --- a/design/14313-benchmark-format.md +++ b/design/14313-benchmark-format.md
@@ -56,31 +56,33 @@ BenchmarkEncodeDigitsCompress1e5-8 1 12301511 ns/op 8.13 MB/s BenchmarkEncodeDigitsCompress1e6-8 1 137962041 ns/op 7.25 MB/s -The testing package always reports ns/op, and each benchmark can request the addition of MB/s (throughput) and also B/op and allocs/op (allocation rates). +The testing package always reports ns/op, and directly supports the +addition of MB/s (throughput) and also B/op and allocs/op (allocation +rates). +Benchmarks can report additional metrics with any custom unit using +[`B.ReportMetric`](https://pkg.go.dev/testing#B.ReportMetric). ### Benchmark processors Multiple tools have been written that process this format, -most notably [benchcmp](https://godoc.org/golang.org/x/tools/cmd/benchcmp) -and its more statistically valid successor [benchstat](https://godoc.org/rsc.io/benchstat). -There is also [benchmany](https://godoc.org/github.com/aclements/go-misc/benchmany)'s plot subcommand +most notably [benchcmp](https://pkg.go.dev/golang.org/x/tools/cmd/benchcmp) +and its more statistically valid successor [benchstat](https://pkg.go.dev/golang.org/x/perf/cmd/benchstat). +There is also [benchmany](https://pkg.go.dev/github.com/aclements/go-misc/benchmany)'s plot subcommand and likely more unpublished programs. ### Benchmark runners Multiple tools have also been written that generate this format. In addition to the standard Go testing package, -[compilebench](https://godoc.org/rsc.io/compilebench) +[compilebench](https://pkg.go.dev/golang.org/x/tools/cmd/compilebench) generates this data format based on runs of the Go compiler, and Austin's unpublished shellbench generates this data format after running an arbitrary shell command. -The [golang.org/x/benchmarks/bench](https://golang.org/x/benchmarks/bench) benchmarks -are notable for _not_ generating this format, -which has made all analysis of those results +The [golang.org/x/benchmarks](https://golang.org/x/benchmarks) benchmarks +are notable for _not_ originally generating this format, +which made all analysis of those results more complex than we believe it should be. -We intend to update those benchmarks to generate the standard format, -once a standard format is defined. Part of the motivation for the proposal is to avoid the need to process custom output formats in future benchmarks.