tree 231dc272118af71c6a943421cbdc39a3fc294f33
parent f62e16ca4165a3ae6024315bdec53b340c447b6d
author Dominik Honnef <dominik@honnef.co> 1554904022 +0200
committer Dominik Honnef <dominik@honnef.co> 1556075463 +0000

go/packages: deduplicate file parsing

When loading packages from source, many files are being parsed
repeatedly, for example due to test variants. While the median number of
times a file gets parsed is 2, it is significantly higher (up to 28
times) when parsing the standard library, because of test variant
shenanigans.

By caching file contents and their parsed representations we can cut
down on processing time and garbage produced. When loading individual
packages or 3rd party projects, the effect is rather small. However when
loading the entire standard library, the effect is substantial.

name       old time/op    new time/op    delta
Jaeger-8      2.95s ± 7%     2.84s ± 8%     ~     (p=0.089 n=10+10)
Std-8         4.96s ± 7%     4.23s ± 3%  -14.62%  (p=0.000 n=9+9)
Strconv-8     892ms ±34%     877ms ±21%     ~     (p=0.853 n=10+10)

name       old alloc/op   new alloc/op   delta
Jaeger-8     1.22GB ± 0%    1.21GB ± 0%   -0.84%  (p=0.000 n=10+10)
Std-8        2.57GB ± 0%    2.20GB ± 0%  -14.61%  (p=0.000 n=10+8)
Strconv-8     201MB ± 1%     200MB ± 1%     ~     (p=0.105 n=10+10)

name       old allocs/op  new allocs/op  delta
Jaeger-8      12.7M ± 0%     12.4M ± 0%   -1.82%  (p=0.000 n=9+10)
Std-8         26.4M ± 0%     17.3M ± 0%  -34.62%  (p=0.000 n=9+9)
Strconv-8     1.94M ± 0%     1.91M ± 0%   -1.50%  (p=0.000 n=10+10)

When loading std, peak RSS decreases from 1.96 GB to 1.57 GB.

While we're here, we simplify our ParseFile implementation. The contract
of ParseFile specifies that implementations must use src for parsing,
and use filename only for display purposes. As such, we mustn't ever
call it with a nil src, making the check for a nil src in our own
implementation superfluous.

Change-Id: I33daac20fc52ccdb3187a336633f712d01b71d86
Reviewed-on: https://go-review.googlesource.com/c/tools/+/171377
Run-TryBot: Ian Cottrell <iancottrell@google.com>
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>