_content/talks/2014/gotham-context.slide - website - Git at Google

 Cancellation, Context, and Plumbing
 GothamGo 2014

 Sameer Ajmani
 sameer@golang.org

 * Video

 This talk was presented at GothamGo in New York City, November 2014.

 .link http://vimeo.com/115309491 Watch the talk on Vimeo

 * Introduction

 In Go servers, each incoming request is handled in its own goroutine.

 Handler code needs access to request-specific values:

 - security credentials
 - request deadline
 - operation priority

 When the request completes or times out, its work should be canceled.

 * Cancellation

 Abandon work when the caller no longer needs the result.

 - user navigates away, closes connection
 - operation exceeds its deadline
 - when using hedged requests, cancel the laggards

 Efficiently canceling unneeded work saves resources.

 * Cancellation is advisory

 Cancellation does not stop execution or trigger panics.

 Cancellation informs code that its work is no longer needed.

 Code checks for cancellation and decides what to do:
 shut down, clean up, return errors.

 * Cancellation is transitive

 .image gotham-context/transitive.svg

 * Cancellation affects all APIs on the request path

 Network protocols support cancellation.

 - HTTP: close the connection
 - RPC: send a control message

 APIs above network need cancellation, too.

 - Database clients
 - Network file system clients
 - Cloud service clients

 And all the layers atop those, up to the UI.

 *Goal:* provide a uniform cancellation API that works across package boundaries.

 * Cancellation APIs

 Many Go APIs support cancellation and deadlines already.

 Go APIs are synchronous, so cancellation comes from another goroutine.

 Method on the connection or client object:

   // goroutine #1
   result, err := conn.Do(req)

   // goroutine #2
   conn.Cancel(req)

 Method on the request object:

   // goroutine #1
   result, err := conn.Do(req)

   // goroutine #2
   req.Cancel()

 * Cancellation APIs (continued)

 Method on the pending result object:

   // goroutine #1
   pending := conn.Start(req)
   ...
   result, err := pending.Result()

   // goroutine #2
   pending.Cancel()


 Different cancellation APIs in each package are a headache.

 We need one that's independent of package or transport:

   // goroutine #1
   result, err := conn.Do(x, req)

   // goroutine #2
   x.Cancel()

 * Context

 A `Context` carries a cancellation signal and request-scoped values to all functions running on behalf of the same task.  It's safe for concurrent access.

 .code gotham-context/interface.go /type Context/,/^}/

 *Idiom:* pass `ctx` as the first argument to a function.

   import "golang.org/x/net/context"

   // ReadFile reads file name and returns its contents.
   // If ctx.Done is closed, ReadFile returns ctx.Err immediately.
   func ReadFile(ctx context.Context, name string) ([]byte, error)

 Examples and discussion in [[http://blog.golang.org/context][blog.golang.org/context]].

 * Contexts are hierarchical

 `Context` has no `Cancel` method; obtain a cancelable `Context` using `WithCancel`:

 .code gotham-context/interface.go /WithCancel /,/func WithCancel/

 Passing a `Context` to a function does not pass the ability to cancel that `Context`.

   // goroutine #1
   ctx, cancel := context.WithCancel(parent)
   ...
   data, err := ReadFile(ctx, name)

   // goroutine #2
   cancel()

 Contexts form a tree, any subtree of which can be canceled.

 * Why does Done return a channel?

 Closing a channel works well as a broadcast signal.

 _After_the_last_value_has_been_received_from_a_closed_channel_c,_any_receive_from_c_will_succeed_without_blocking,_returning_the_zero_value_for_the_channel_element._

 Any number of goroutines can `select` on `<-ctx.Done()`.

 Examples and discussion in in [[http://blog.golang.org/pipelines][blog.golang.org/pipelines]].

 Using `close` requires care.

 - closing a nil channel panics
 - closing a closed channel panics

 `Done` returns a receive-only channel that can only be canceled using the `cancel` function returned by `WithCancel`.  It ensures the channel is closed exactly once.

 * Context values

 Contexts carry request-scoped values across API boundaries.

 - deadline
 - cancellation signal
 - security credentials
 - distributed trace IDs
 - operation priority
 - network QoS label

 RPC clients encode `Context` values onto the wire.

 RPC servers decode them into a new `Context` for the handler function.

 * Replicated Search

 Example from [[https://go.dev/talks/2012/concurrency.slide][Go Concurrency Patterns]].

 .code gotham-context/first.go /START1/,/STOP1/

 Remaining searches may continue running after First returns.

 * Cancelable Search

 .code gotham-context/first-context.go /START1/,/STOP1/

 * Context plumbing

 *Goal:* pass a `Context` parameter from each inbound RPC at a server through the call stack to each outgoing RPC.

 .code gotham-context/before.go /START/,/END/

 * Context plumbing (after)

 .code gotham-context/after.go /START/,/END/

 * Problem: Existing and future code

 Google has millions of lines of Go code.

 We've retrofitted the internal RPC and distributed file system APIs to take a Context.

 Lots more to do, growing every day.

 * Why not use (something like) thread local storage?

 C++ and Java pass request state in thread-local storage.

 Requires no API changes, but ...
 requires custom thread and callback libraries.

 Mostly works, except when it doesn't. Failures are hard to debug.

 Serious consequences if credential-passing bugs affect user privacy.

 "Goroutine-local storage" doesn't exist, and even if it did,
 request processing may flow between goroutines via channels.

 We won't sacrifice clarity for convenience.

 * In Go, pass Context explicitly

 Easy to tell when a Context passes between functions, goroutines, and processes.

 Invest up front to make the system easier to maintain:

 - update relevant functions to accept a `Context`
 - update function calls to provide a `Context`
 - update interface methods and implementations

 Go's awesome tools can help.

 * Automated refactoring

 *Initial*State:*

 Pass `context.TODO()` to outbound RPCs.

 `context.TODO()` is a sentinel for static analysis tools. Use it wherever a `Context` is needed but there isn't one available.

 *Iteration:*

 For each function `F(x)` whose body contains `context.TODO()`,

 - add a `Context` parameter to `F`
 - update callers to use `F(context.TODO(),`x)`
 - if the caller has a `Context` available, pass it to `F` instead

 Repeat until `context.TODO()` is gone.

 * Finding relevant functions

 The [[http://godoc.org/golang.org/x/tools/cmd/callgraph][golang.org/x/tools/cmd/callgraph]] tool constructs the call graph of a Go program.

 It uses whole-program pointer analysis to find dynamic calls (via interfaces or function values).

 *For*context*plumbing:*

 Find all functions on call paths from `Context` _suppliers_ (inbound RPCs) to `Context` _consumers_ (`context.TODO`).

 * Updating function calls

 To change add all `F(x)` to `F(context.TODO(),`x)`:

 - define `FContext(ctx,`x)`
 - `F(x)` → `FContext(context.TODO(),`x)`
 - change `F(x)` to `F(ctx,`x)`
 - `FContext(context.TODO(),`x)` → `F(context.TODO(),`x)`
 - remove `FContext(ctx,`x)`

 * gofmt -r

 Works well for simple replacements:

   gofmt -r 'pkg.F(a) -> pkg.FContext(context.TODO(), a)'

 But this is too imprecise for methods.  There may be many methods named M:

   gofmt -r 'x.M(y) -> x.MContext(context.TODO(), y)'

 We want to restrict the transformation to specific method signatures.

 * The eg tool

 The [[http://godoc.org/golang.org/x/tools/cmd/eg][golang.org/x/tools/cmd/eg]] tool performs precise example-based refactoring.

 The `before` expression specifies a pattern and the `after` expression its replacement.

 To replace `x.M(y)` with `x.MContext(context.TODO(),`y)`:

 .code gotham-context/eg.go

 * Dealing with interfaces

 We need to update dynamic calls to `x.M(y)`.

 If `M` called via interface `I`, then `I.M` also needs to change.  The eg tool can update call sites with receiver type `I`.

 When we change `I`, we need to update all of its implementations.

 Find types assignable to `I` using [[http://godoc.org/golang.org/x/tools/go/types][golang.org/x/tools/go/types]].

 More to do here.

 * What about the standard library?

 The Go 1.0 compatibility guarantee means we will not break existing code.

 Interfaces like `io.Reader` and `io.Writer` are widely used.

 For Google files, used a currying approach:

   f, err := file.Open(ctx, "/gfs/cell/path")
   ...
   fio := f.IO(ctx)  // returns an io.ReadWriteCloser that passes ctx
   data, err := ioutil.ReadAll(fio)

 For versioned public packages, add `Context` parameters in a new API version and provide `eg` templates to insert `context.TODO()`.

 More to do here.

 * Conclusion

 Cancellation needs a uniform API across package boundaries.

 Retrofitting code is hard, but Go is tool-friendly.

 New code should use `Context`.

 Links:

 - [[http://golang.org/x/net/context][golang.org/x/net/context]] - package
 - [[http://blog.golang.org/context][blog.golang.org/context]] - blog post
 - [[http://golang.org/x/tools/cmd/eg][golang.org/x/tools/cmd/eg]] - eg tool
	Cancellation, Context, and Plumbing
	GothamGo 2014

	Sameer Ajmani
	sameer@golang.org

	* Video

	This talk was presented at GothamGo in New York City, November 2014.

	.link http://vimeo.com/115309491 Watch the talk on Vimeo

	* Introduction

	In Go servers, each incoming request is handled in its own goroutine.

	Handler code needs access to request-specific values:

	- security credentials
	- request deadline
	- operation priority

	When the request completes or times out, its work should be canceled.

	* Cancellation

	Abandon work when the caller no longer needs the result.

	- user navigates away, closes connection
	- operation exceeds its deadline
	- when using hedged requests, cancel the laggards

	Efficiently canceling unneeded work saves resources.

	* Cancellation is advisory

	Cancellation does not stop execution or trigger panics.

	Cancellation informs code that its work is no longer needed.

	Code checks for cancellation and decides what to do:
	shut down, clean up, return errors.

	* Cancellation is transitive

	.image gotham-context/transitive.svg

	* Cancellation affects all APIs on the request path

	Network protocols support cancellation.

	- HTTP: close the connection
	- RPC: send a control message

	APIs above network need cancellation, too.

	- Database clients
	- Network file system clients
	- Cloud service clients

	And all the layers atop those, up to the UI.

	Goal: provide a uniform cancellation API that works across package boundaries.

	* Cancellation APIs

	Many Go APIs support cancellation and deadlines already.

	Go APIs are synchronous, so cancellation comes from another goroutine.

	Method on the connection or client object:

	// goroutine #1
	result, err := conn.Do(req)

	// goroutine #2
	conn.Cancel(req)

	Method on the request object:

	// goroutine #1
	result, err := conn.Do(req)

	// goroutine #2
	req.Cancel()

	* Cancellation APIs (continued)

	Method on the pending result object:

	// goroutine #1
	pending := conn.Start(req)
	...
	result, err := pending.Result()

	// goroutine #2
	pending.Cancel()


	Different cancellation APIs in each package are a headache.

	We need one that's independent of package or transport:

	// goroutine #1
	result, err := conn.Do(x, req)

	// goroutine #2
	x.Cancel()

	* Context

	A `Context` carries a cancellation signal and request-scoped values to all functions running on behalf of the same task. It's safe for concurrent access.

	.code gotham-context/interface.go /type Context/,/^}/

	Idiom: pass `ctx` as the first argument to a function.

	import "golang.org/x/net/context"

	// ReadFile reads file name and returns its contents.
	// If ctx.Done is closed, ReadFile returns ctx.Err immediately.
	func ReadFile(ctx context.Context, name string) ([]byte, error)

	Examples and discussion in [[http://blog.golang.org/context][blog.golang.org/context]].

	* Contexts are hierarchical

	`Context` has no `Cancel` method; obtain a cancelable `Context` using `WithCancel`:

	.code gotham-context/interface.go /WithCancel /,/func WithCancel/

	Passing a `Context` to a function does not pass the ability to cancel that `Context`.

	// goroutine #1
	ctx, cancel := context.WithCancel(parent)
	...
	data, err := ReadFile(ctx, name)

	// goroutine #2
	cancel()

	Contexts form a tree, any subtree of which can be canceled.

	* Why does Done return a channel?

	Closing a channel works well as a broadcast signal.

	_After_the_last_value_has_been_received_from_a_closed_channel_c,_any_receive_from_c_will_succeed_without_blocking,_returning_the_zero_value_for_the_channel_element._

	Any number of goroutines can `select` on `<-ctx.Done()`.

	Examples and discussion in in [[http://blog.golang.org/pipelines][blog.golang.org/pipelines]].

	Using `close` requires care.

	- closing a nil channel panics
	- closing a closed channel panics

	`Done` returns a receive-only channel that can only be canceled using the `cancel` function returned by `WithCancel`. It ensures the channel is closed exactly once.

	* Context values

	Contexts carry request-scoped values across API boundaries.

	- deadline
	- cancellation signal
	- security credentials
	- distributed trace IDs
	- operation priority
	- network QoS label

	RPC clients encode `Context` values onto the wire.

	RPC servers decode them into a new `Context` for the handler function.

	* Replicated Search

	Example from [[https://go.dev/talks/2012/concurrency.slide][Go Concurrency Patterns]].

	.code gotham-context/first.go /START1/,/STOP1/

	Remaining searches may continue running after First returns.

	* Cancelable Search

	.code gotham-context/first-context.go /START1/,/STOP1/

	* Context plumbing

	Goal: pass a `Context` parameter from each inbound RPC at a server through the call stack to each outgoing RPC.

	.code gotham-context/before.go /START/,/END/

	* Context plumbing (after)

	.code gotham-context/after.go /START/,/END/

	* Problem: Existing and future code

	Google has millions of lines of Go code.

	We've retrofitted the internal RPC and distributed file system APIs to take a Context.

	Lots more to do, growing every day.

	* Why not use (something like) thread local storage?

	C++ and Java pass request state in thread-local storage.

	Requires no API changes, but ...
	requires custom thread and callback libraries.

	Mostly works, except when it doesn't. Failures are hard to debug.

	Serious consequences if credential-passing bugs affect user privacy.

	"Goroutine-local storage" doesn't exist, and even if it did,
	request processing may flow between goroutines via channels.

	We won't sacrifice clarity for convenience.

	* In Go, pass Context explicitly

	Easy to tell when a Context passes between functions, goroutines, and processes.

	Invest up front to make the system easier to maintain:

	- update relevant functions to accept a `Context`
	- update function calls to provide a `Context`
	- update interface methods and implementations

	Go's awesome tools can help.

	* Automated refactoring

	InitialState:*

	Pass `context.TODO()` to outbound RPCs.

	`context.TODO()` is a sentinel for static analysis tools. Use it wherever a `Context` is needed but there isn't one available.

	Iteration:

	For each function `F(x)` whose body contains `context.TODO()`,

	- add a `Context` parameter to `F`
	- update callers to use `F(context.TODO(),`x)`
	- if the caller has a `Context` available, pass it to `F` instead

	Repeat until `context.TODO()` is gone.

	* Finding relevant functions

	The [[http://godoc.org/golang.org/x/tools/cmd/callgraph][golang.org/x/tools/cmd/callgraph]] tool constructs the call graph of a Go program.

	It uses whole-program pointer analysis to find dynamic calls (via interfaces or function values).

	Forcontextplumbing:

	Find all functions on call paths from `Context` _suppliers_ (inbound RPCs) to `Context` _consumers_ (`context.TODO`).

	* Updating function calls

	To change add all `F(x)` to `F(context.TODO(),`x)`:

	- define `FContext(ctx,`x)`
	- `F(x)` → `FContext(context.TODO(),`x)`
	- change `F(x)` to `F(ctx,`x)`
	- `FContext(context.TODO(),`x)` → `F(context.TODO(),`x)`
	- remove `FContext(ctx,`x)`

	* gofmt -r

	Works well for simple replacements:

	gofmt -r 'pkg.F(a) -> pkg.FContext(context.TODO(), a)'

	But this is too imprecise for methods. There may be many methods named M:

	gofmt -r 'x.M(y) -> x.MContext(context.TODO(), y)'

	We want to restrict the transformation to specific method signatures.

	* The eg tool

	The [[http://godoc.org/golang.org/x/tools/cmd/eg][golang.org/x/tools/cmd/eg]] tool performs precise example-based refactoring.

	The `before` expression specifies a pattern and the `after` expression its replacement.

	To replace `x.M(y)` with `x.MContext(context.TODO(),`y)`:

	.code gotham-context/eg.go

	* Dealing with interfaces

	We need to update dynamic calls to `x.M(y)`.

	If `M` called via interface `I`, then `I.M` also needs to change. The eg tool can update call sites with receiver type `I`.

	When we change `I`, we need to update all of its implementations.

	Find types assignable to `I` using [[http://godoc.org/golang.org/x/tools/go/types][golang.org/x/tools/go/types]].

	More to do here.

	* What about the standard library?

	The Go 1.0 compatibility guarantee means we will not break existing code.

	Interfaces like `io.Reader` and `io.Writer` are widely used.

	For Google files, used a currying approach:

	f, err := file.Open(ctx, "/gfs/cell/path")
	...
	fio := f.IO(ctx) // returns an io.ReadWriteCloser that passes ctx
	data, err := ioutil.ReadAll(fio)

	For versioned public packages, add `Context` parameters in a new API version and provide `eg` templates to insert `context.TODO()`.

	More to do here.

	* Conclusion

	Cancellation needs a uniform API across package boundaries.

	Retrofitting code is hard, but Go is tool-friendly.

	New code should use `Context`.

	Links:

	- [[http://golang.org/x/net/context][golang.org/x/net/context]] - package
	- [[http://blog.golang.org/context][blog.golang.org/context]] - blog post
	- [[http://golang.org/x/tools/cmd/eg][golang.org/x/tools/cmd/eg]] - eg tool