---
title: PerformanceMonitoring
---

The Go project monitors the performance characteristics of the Go implementation
as well as that of subrepositories like golang.org/x/tools.

## Benchmarks

`golang.org/x/benchmarks/cmd/bench` is the entrypoint for our performance tests.
For Go implementations, this runs both the
[Sweet](https://golang.org/x/benchmarks/sweet) (end-to-end benchmarks)
and [bent](https://golang.org/x/benchmarks/cmd/bent) (microbenchmarks)
benchmarking suites.

For the `golang.org/x/tools` project, it runs the repository's benchmarks.

These benchmarks can all be invoked manually, as can `cmd/bench`, but using both
Sweet and bent directly will likely offer a better user experience.
See their documentation for more details.

## Performance testing principles

### Change with the times

Our set of benchmarks is curated.
It is allowed to change over time.
Sticking to a single benchmark set over a long period of time can easily land
us in a situation where we're optimizing for the wrong thing.

### Always perform a comparison

We never report performance numbers in isolation, and only relative to some
baseline.
This strategy comes from the fact that comparing performance data taken far
apart in time, even on the same hardware, can result in a lot of noise that
goes unaccounted for.
The state of a machine or VM on one day is likely to be very different than
the state of a machine or VM on the next day.

We refer to the tested version of source code as the "experiment" and the
baseline version of source code as the "baseline."

## Presubmit

Do you have a Gerrit change that you want to run against our benchmarks?

Select a builder containing the word `perf` in the "Choose Tryjobs" dialog that
appears when selecting a [SlowBot](/wiki/SlowBots).

There are two kinds of presubmit builders for performance testing:
- `perf_vs_parent`, which measures the performance delta of a change in isolation.
- `perf_vs_tip`, which measures the performance delta versus the current
  tip-of-tree for whichever repository the change is for.
  (Remember to rebase your change(s) before using this one!)

There's a third special presubmit builder for the tools repository as well which
contains the string `perf_vs_gopls_0_11`.
This measures the performance delta versus the `release-branch-gopls.0.11` branch
of the tools repository.

## Postsubmit

The [performance dashboard](http://perf.golang.org/dashboard) provides
continuous monitoring of benchmark performance for every commit that is made to
the main Go repository and other subrepositories.
The dashboard, more specifically, displays graphs showing the change in certain
performance metrics (also called "units") over time for different benchmarks.
Use the navigation interface at the top of the page to explore further.

The [regressions page](https://perf.golang.org/dashboard/?benchmark=regressions)
displays all benchmarks in order of biggest regression to biggest improvement,
followed by all benchmarks for which there is no statistically clear answer.

On the graphs, red means regression, blue means improvement.

### Baselines

In post-submit, the baseline version for Go repository performance tests is
automatically determined.
For performance tests against changes on release branches, the baseline is always
the latest release for that branch (for example, the latest minor release for
Go 1.21 on `release-branch.go1.21`).
For performance tests against tip-of-tree, the baseline is always the latest
overall release of Go.
This is indicated by the name of the builder that produces these benchmark
esults, which contains the string `perf_vs_release`.
What this means is that on every minor release of Go, the baseline shifts.
These baseline shifts can be observed in the [per-metric view](#per-metric-view).

Performance tests on subrepositories typically operate against some known
long-term fixed baseline.
For the tools repository, it's the tip of the `release-branch-gopls.0.11`
branch.

### Per-metric view

Click on any graph's performance metric name to view a more detailed timeline
of the performance deltas for that metric.

![Image displaying which link to click to reach the per-metric
page.](images/performance-monitoring-per-metric-link.png)

This view is particularly useful for identifying the culprit behind a regression
and for pinpointing the source of an improvement.

Sometimes, a performance change happens because a benchmark has changed or
because the baseline version being used as changed.
This view also displays information about the baseline versions and the version
of `golang.org/x/benchmarks` that was used to produce the results to help identify
when this happens.