cmd/compile: adjust start heap size

TLDR
- not-huge increase to default starting heap boost,
- small improvement in build performance,
- remove concurrency dependence of starting heap,
- aligns RSS behavior with GOMEMLIMIT,
- adds a gcflags=-d=gcstart=N (N -> N MiB) flag for
  people who want to trade a lot of memory for a
  little build performance improvement.

This removes concurrency (-c flag) sensitivity and increases the
nominal default to 128MiB.

Refactored the startheap code into a separate file, to make it
easier to extract and reuse.

Added sensitivity to concurrency=1 and GOMEMLIMIT!="" (in addition
to existing GOGC!=""), those disable the default starting heap boost
because the compiler-invoker has indicated either a desire to control
the GC or a desire to run in minimum memory(or both).

Adds a -d flag gcstart=N (N is number of MiB) for
tinkering/experiments. This always enables the starting heap.
(`GOGC=XXX` and `-d=gcstart=YYY` will use `GOGC=XXX` after starting
heap size is achieved.)

Derated the "boost" obtained by a factor of .70 so that
`-d=gcstart=2000` yields the same RSS as `GOMEMLIMIT=2000MiB`

(Actually adjusts the boost with a high-low breakpoint.)

The parent, with concurrency sensitivity, provided 64MB of plain
boost.  Derating reduces the effects of boosting the starting heap
slightly.  The benchmark here shows that maintaining 64MB results in
a minor regression, while increasing it to 128MB produces a slight
improvement, and does not grow the RSS versus 64MB.

```
        │   parent    │                sh64                │               sh128                │              sh1024               │
        │   sec/op    │   sec/op     vs base               │   sec/op     vs base               │   sec/op    vs base               │
std       10.164 ± 1%   10.527 ± 1%  +3.57% (p=0.000 n=50)   10.084 ± 1%  -0.79% (p=0.000 n=50)   9.631 ± 1%  -5.24% (p=0.000 n=50)
compile    21.05 ± 1%    20.78 ± 0%  -1.28% (p=0.000 n=50)    20.74 ± 1%  -1.46% (p=0.000 n=50)   20.77 ± 0%  -1.32% (p=0.001 n=50)
ast        20.45 ± 1%    20.39 ± 1%       ~ (p=0.334 n=50)    20.44 ± 0%       ~ (p=0.818 n=50)   20.11 ± 1%  -1.65% (p=0.000 n=50)
geomean    16.35         16.46       +0.65%                   16.23       -0.76%                  15.90       -2.75%

        │   parent    │                sh64                │               sh128                │               sh1024               │
        │ user-sec/op │ user-sec/op  vs base               │ user-sec/op  vs base               │ user-sec/op  vs base               │
std        66.06 ± 0%    69.74 ± 0%  +5.56% (p=0.000 n=50)    64.68 ± 0%  -2.09% (p=0.000 n=50)    59.51 ± 0%  -9.91% (p=0.000 n=50)
compile    84.69 ± 1%    82.54 ± 0%  -2.53% (p=0.000 n=50)    82.63 ± 0%  -2.43% (p=0.000 n=50)    82.66 ± 1%  -2.40% (p=0.000 n=50)
ast        59.41 ± 0%    58.84 ± 1%  -0.95% (p=0.011 n=50)    59.48 ± 1%       ~ (p=0.341 n=50)    57.13 ± 1%  -3.83% (p=0.000 n=50)
geomean    69.27         69.71       +0.63%                   68.25       -1.47%                   65.50       -5.44%

        │   parent   │                sh64                │               sh128               │              sh1024               │
        │ sys-sec/op │ sys-sec/op   vs base               │ sys-sec/op  vs base               │ sys-sec/op  vs base               │
std       9.599 ± 1%   10.031 ± 1%  +4.50% (p=0.000 n=50)   9.513 ± 1%  -0.90% (p=0.014 n=50)   9.359 ± 1%  -2.50% (p=0.000 n=50)
compile   6.813 ± 1%    6.740 ± 1%  -1.08% (p=0.017 n=50)   6.716 ± 1%  -1.42% (p=0.006 n=50)   6.696 ± 1%  -1.72% (p=0.000 n=50)
ast       4.315 ± 1%    4.291 ± 1%       ~ (p=0.781 n=50)   4.296 ± 1%       ~ (p=0.792 n=50)   4.279 ± 2%       ~ (p=0.124 n=50)
geomean   6.559         6.620       +0.93%                  6.499       -0.92%                  6.449       -1.68%

        │     parent     │                 sh64                  │                 sh128                 │                 sh1024                  │
        │ peak-RSS-bytes │ peak-RSS-bytes  vs base               │ peak-RSS-bytes  vs base               │ peak-RSS-bytes  vs base                 │
std         257.1Mi ± 1%     257.2Mi ± 1%       ~ (p=0.754 n=50)     257.0Mi ± 0%       ~ (p=0.570 n=50)     605.6Mi ± 0%  +135.59% (p=0.000 n=50)
compile    1007.2Mi ± 1%    1004.3Mi ± 0%       ~ (p=0.064 n=50)    1007.4Mi ± 0%       ~ (p=0.348 n=50)    1009.4Mi ± 1%         ~ (p=0.598 n=50)
ast         1.848Gi ± 0%     1.842Gi ± 0%       ~ (p=0.079 n=50)     1.824Gi ± 0%  -1.25% (p=0.000 n=50)     1.856Gi ± 0%    +0.47% (p=0.000 n=50)
geomean     788.3Mi          786.8Mi       -0.19%                    785.0Mi       -0.41%                    1.027Gi        +33.37%
```

Updates #73044

Change-Id: I6359642a94b396e696dd57e64ed1f2c4cf178475
Reviewed-on: https://go-review.googlesource.com/c/go/+/724441
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
4 files changed
tree: 04cf02876710e965a3a9acd7b7a620ce7780012f
  1. .github/
  2. api/
  3. doc/
  4. lib/
  5. misc/
  6. src/
  7. test/
  8. .gitattributes
  9. .gitignore
  10. codereview.cfg
  11. CONTRIBUTING.md
  12. go.env
  13. LICENSE
  14. PATENTS
  15. README.md
  16. SECURITY.md
README.md

The Go Programming Language

Go is an open source programming language that makes it easy to build simple, reliable, and efficient software.

Gopher image Gopher image by Renee French, licensed under Creative Commons 4.0 Attribution license.

Our canonical Git repository is located at https://go.googlesource.com/go. There is a mirror of the repository at https://github.com/golang/go.

Unless otherwise noted, the Go source files are distributed under the BSD-style license found in the LICENSE file.

Download and Install

Binary Distributions

Official binary distributions are available at https://go.dev/dl/.

After downloading a binary release, visit https://go.dev/doc/install for installation instructions.

Install From Source

If a binary distribution is not available for your combination of operating system and architecture, visit https://go.dev/doc/install/source for source installation instructions.

Contributing

Go is the work of thousands of contributors. We appreciate your help!

To contribute, please read the contribution guidelines at https://go.dev/doc/contribute.

Note that the Go project uses the issue tracker for bug reports and proposals only. See https://go.dev/wiki/Questions for a list of places to ask questions about the Go language.