Testing Techniques
Google I/O 2014

Andrew Gerrand
adg@golang.org

* Video

This talk was presented at golang-syd in July 2014.

.link http://www.youtube.com/watch?v=ndmB0bj7eyw Watch the talk on YouTube


* The basics

* Testing Go code

Go has a built-in testing framework.

It is provided by the `testing` package and the `go` `test` command.

Here is a complete test file that tests the `strings.Index` function:

.code testing/test1/string_test.go


* Table-driven tests

Go's struct literal syntax makes it easy to write table-driven tests:

.code testing/test2/string_test.go /func TestIndex/,/^}/


* T

The `*testing.T` argument is used for error reporting:

	t.Errorf("got bar = %v, want %v", got, want)
	t.Fatalf("Frobnicate(%v) returned error: %v", arg, err)
	t.Logf("iteration %v", i)

And enabling parallel tests:

	t.Parallel()

And controlling whether a test runs at all:

	if runtime.GOARCH == "arm" {
		t.Skip("this doesn't work on ARM")
	}


* Running tests

The `go` `test` command runs tests for the specified package.
(It defaults to the package in the current directory.)

	$ go test
	PASS

	$ go test -v
	=== RUN TestIndex
	--- PASS: TestIndex (0.00 seconds)
	PASS

To run the tests for all my projects:

	$ go test github.com/nf/...

Or for the standard library:

	$ go test std


* Test coverage

The `go` tool can report test coverage statistics.

	$ go test -cover
	PASS
	coverage: 96.4% of statements
	ok  	strings	0.692s

The `go` tool can generate coverage profiles that may be intepreted by the `cover` tool.

	$ go test -coverprofile=cover.out
	$ go tool cover -func=cover.out
	strings/reader.go:    Len             66.7%
	strings/strings.go:   TrimSuffix     100.0%
	... many lines omitted ...
	strings/strings.go:   Replace        100.0%
	strings/strings.go:   EqualFold      100.0%
	total:                (statements)    96.4%

* Coverage visualization

	$ go tool cover -html=cover.out

.image testing/cover.png


* Advanced techniques

* An example program

*outyet* is a web server that announces whether or not a particular Go version has been tagged.

	go get github.com/golang/example/outyet

.image testing/go1.1.png


* Testing HTTP clients and servers

The `net/http/httptest` package provides helpers for testing code that makes or serves HTTP requests.


* httptest.Server

An `httptest.Server` listens on a system-chosen port on the local loopback interface, for use in end-to-end HTTP tests.

	type Server struct {
		URL      string // base URL of form http://ipaddr:port with no trailing slash
		Listener net.Listener

		// TLS is the optional TLS configuration, populated with a new config
		// after TLS is started. If set on an unstarted server before StartTLS
		// is called, existing fields are copied into the new config.
		TLS *tls.Config

		// Config may be changed after calling NewUnstartedServer and
		// before Start or StartTLS.
		Config *http.Server
	}

	func NewServer(handler http.Handler) *Server

	func (*Server) Close() error

* httptest.Server in action

This code sets up a temporary HTTP server that serves a simple "Hello" response.

.play testing/httpserver.go /START/,/STOP/


* httptest.ResponseRecorder

`httptest.ResponseRecorder` is an implementation of `http.ResponseWriter` that records its mutations for later inspection in tests.

	type ResponseRecorder struct {
		Code      int           // the HTTP response code from WriteHeader
		HeaderMap http.Header   // the HTTP response headers
		Body      *bytes.Buffer // if non-nil, the bytes.Buffer to append written data to
		Flushed   bool
	}

* httptest.ResponseRecorder in action

By passing a `ResponseRecorder` into an HTTP handler we can inspect the generated response.

.play testing/httprecorder.go /START/,/STOP/


* Race Detection

A data race occurs when two goroutines access the same variable concurrently and at least one of the accesses is a write.

To help diagnose such bugs, Go includes a built-in data race detector.

Pass the `-race` flag to the go tool to enable the race detector:

	$ go test -race mypkg    // to test the package
	$ go run -race mysrc.go  // to run the source file
	$ go build -race mycmd   // to build the command
	$ go install -race mypkg // to install the package


* Testing with concurrency

When testing concurrent code, there's a temptation to use sleep;
it's easy and works most of the time.

But "most of the time" isn't always and flaky tests result.

We can use Go's concurrency primitives to make flaky sleep-driven tests reliable.


* Finding errors with static analysis: vet

The `vet` tool checks code for common programmer mistakes:

- bad printf formats,
- bad build tags,
- bad range loop variable use in closures,
- useless assignments,
- unreachable code,
- bad use of mutexes,
- and more.

Usage:

	go vet [package]


* Testing from the inside

Most tests are compiled as part of the package under test.

This means they can access unexported details, as we have already seen.


* Testing from the outside

Occasionally you want to run your tests from outside the package under test.

(Test files as `package` `foo_test` instead of `package` `foo`.)

This can break dependency cycles. For example:

- The `testing` package uses `fmt`.
- The `fmt` tests must import `testing`.
- So the `fmt` tests are in package `fmt_test` and can import both `testing` and `fmt`.


* Mocks and fakes

Go eschews mocks and fakes in favor of writing code that takes broad interfaces.

For example, if you're writing a file format parser, don't write a function like this:

	func Parse(f *os.File) error

instead, write functions that take the interface you need:

	func Parse(r io.Reader) error

(An `*os.File` implements `io.Reader`, as does `bytes.Buffer` or `strings.Reader`.)


* Subprocess tests

Sometimes you need to test the behavior of a process, not just a function.

.code testing/subprocess/subprocess.go /func Crasher/,/^}/

To test this code, we invoke the test binary itself as a subprocess:

.code testing/subprocess/subprocess_test.go /func TestCrasher/,/^}/


* More information

.link http://golang.org/pkg/testing/

.link http://golang.org/cmd/go/

.link http://golang.org