tree dfcee76c31f70eec5c2ceb750e3a6e5f7a66f5ac
parent 242b6b35177ec3909636b6cf6a47e8c2c6324b5d
author Tom Bergan <tombergan@google.com> 1494624463 -0700
committer Brad Fitzpatrick <bradfitz@golang.org> 1495570330 +0000

[release-branch.go1.8] http2: fix lock contention slowdown due to gracefulShutdownCh

gracefulShutdownCh is shared by all connections in a server. When a
server accumulates many connections (e.g., 5000 in the kubemark-5000
benchmark), we have 5000 serverConn.serve goroutines selecting on this
channel. This means 5000 goroutines hammer the channel's lock, which
causes severe lock contention.

The fix in this CL is to make a local proxy for gracefulShutdownCh in
each connection so that each connection selects on gracefulShutdownCh
at most once per connection rather than once per serverConn.serve loop
iteration.

This fix is intended to be backported quickly into Go 1.8.2. The
downside of this fix is 2KB extra stack usage per connection. A better
fix will be implemented in Go 1.9.

Unfortunately, I have been unable to reproduce this problem locally.
This fix was verified by the kubernetes team. See:
https://github.com/kubernetes/kubernetes/issues/45216#issuecomment-300830243

Updates golang/go#20302

Change-Id: I19ab19268a6ccab9b6e9dffa0cfbc89b8c7d0f19
Reviewed-on: https://go-review.googlesource.com/43455
Run-TryBot: Tom Bergan <tombergan@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
(cherry picked from commit d3ede01617fc9ddf0827da5a431705cf91cf563d)
Reviewed-on: https://go-review.googlesource.com/43459
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Chris Broadfoot <cbro@golang.org>