go /
net /
84f0e6f92b10139f986b1756e149a7d9de270cdc http2: fix lock contention slowdown due to gracefulShutdownCh
gracefulShutdownCh is shared by all connections in a server. When a
server accumulates many connections (e.g., 5000 in the kubemark-5000
benchmark), we have 5000 serverConn.serve goroutines selecting on this
channel. This means 5000 goroutines hammer the channel's lock, which
causes severe lock contention.
The fix in this CL is to make a local proxy for gracefulShutdownCh in
each connection so that each connection selects on gracefulShutdownCh
at most once per connection rather than once per serverConn.serve loop
iteration.
This fix is intended to be backported quickly into Go 1.8.2. The
downside of this fix is 2KB extra stack usage per connection. A better
fix will be implemented in Go 1.9.
Unfortunately, I have been unable to reproduce this problem locally.
This fix was verified by the kubernetes team. See:
https://github.com/kubernetes/kubernetes/issues/45216#issuecomment-300830243
Updates golang/go#20302
Change-Id: I19ab19268a6ccab9b6e9dffa0cfbc89b8c7d0f19
Reviewed-on: https://go-review.googlesource.com/43455
Run-TryBot: Tom Bergan <tombergan@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
1 file changed