"Testing Techniques" talk
LGTM=gmlewis, r, campoy
R=dsymonds, crawshaw, gmlewis, r, sameer, bcmills, campoy
CC=golang-codereviews
https://golang.org/cl/108170043
diff --git a/2014/testing.slide b/2014/testing.slide
new file mode 100644
index 0000000..1f290cf
--- /dev/null
+++ b/2014/testing.slide
@@ -0,0 +1,259 @@
+Testing Techniques
+Google I/O 2014
+
+Andrew Gerrand
+adg@golang.org
+
+* Video
+
+This talk was presented at golang-syd in July 2014.
+
+.link http://www.youtube.com/watch?v=ndmB0bj7eyw Watch the talk on YouTube
+
+
+* The basics
+
+* Testing Go code
+
+Go has a built-in testing framework.
+
+It is provided by the `testing` package and the `go` `test` command.
+
+Here is a complete test file that tests the `strings.Index` function:
+
+.code testing/test1/string_test.go
+
+
+* Table-driven tests
+
+Go's struct literal syntax makes it easy to write table-driven tests:
+
+.code testing/test2/string_test.go /func TestIndex/,/^}/
+
+
+* T
+
+The `*testing.T` argument is used for error reporting:
+
+ t.Errorf("got bar = %v, want %v", got, want)
+ t.Fatalf("Frobnicate(%v) returned error: %v", arg, err)
+ t.Logf("iteration %v", i)
+
+And enabling parallel tests:
+
+ t.Parallel()
+
+And controlling whether a test runs at all:
+
+ if runtime.GOARCH == "arm" {
+ t.Skip("this doesn't work on ARM")
+ }
+
+
+* Running tests
+
+The `go` `test` command runs tests for the specified package.
+(It defaults to the package in the current directory.)
+
+ $ go test
+ PASS
+
+ $ go test -v
+ === RUN TestIndex
+ --- PASS: TestIndex (0.00 seconds)
+ PASS
+
+To run the tests for all my projects:
+
+ $ go test github.com/nf/...
+
+Or for the standard library:
+
+ $ go test std
+
+
+* Test coverage
+
+The `go` tool can report test coverage statistics.
+
+ $ go test -cover
+ PASS
+ coverage: 96.4% of statements
+ ok strings 0.692s
+
+The `go` tool can generate coverage profiles that may be intepreted by the `cover` tool.
+
+ $ go test -coverprofile=cover.out
+ $ go tool cover -func=cover.out
+ strings/reader.go: Len 66.7%
+ strings/strings.go: TrimSuffix 100.0%
+ ... many lines omitted ...
+ strings/strings.go: Replace 100.0%
+ strings/strings.go: EqualFold 100.0%
+ total: (statements) 96.4%
+
+* Coverage visualization
+
+ $ go tool cover -html=cover.out
+
+.image testing/cover.png
+
+
+* Advanced techniques
+
+* An example program
+
+*outyet* is a web server that announces whether or not a particular Go version has been tagged.
+
+ go get github.com/golang/example/outyet
+
+.image testing/go1.1.png
+
+
+* Testing HTTP clients and servers
+
+The `net/http/httptest` package provides helpers for testing code that makes or serves HTTP requests.
+
+
+* httptest.Server
+
+An `httptest.Server` listens on a system-chosen port on the local loopback interface, for use in end-to-end HTTP tests.
+
+ type Server struct {
+ URL string // base URL of form http://ipaddr:port with no trailing slash
+ Listener net.Listener
+
+ // TLS is the optional TLS configuration, populated with a new config
+ // after TLS is started. If set on an unstarted server before StartTLS
+ // is called, existing fields are copied into the new config.
+ TLS *tls.Config
+
+ // Config may be changed after calling NewUnstartedServer and
+ // before Start or StartTLS.
+ Config *http.Server
+ }
+
+ func NewServer(handler http.Handler) *Server
+
+ func (*Server) Close() error
+
+* httptest.Server in action
+
+This code sets up a temporary HTTP server that serves a simple "Hello" response.
+
+.play testing/httpserver.go /START/,/STOP/
+
+
+* httptest.ResponseRecorder
+
+`httptest.ResponseRecorder` is an implementation of `http.ResponseWriter` that records its mutations for later inspection in tests.
+
+ type ResponseRecorder struct {
+ Code int // the HTTP response code from WriteHeader
+ HeaderMap http.Header // the HTTP response headers
+ Body *bytes.Buffer // if non-nil, the bytes.Buffer to append written data to
+ Flushed bool
+ }
+
+* httptest.ResponseRecorder in action
+
+By passing a `ResponseRecorder` into an HTTP handler we can inspect the generated response.
+
+.play testing/httprecorder.go /START/,/STOP/
+
+
+* Race Detection
+
+A data race occurs when two goroutines access the same variable concurrently and at least one of the accesses is a write.
+
+To help diagnose such bugs, Go includes a built-in data race detector.
+
+Pass the `-race` flag to the go tool to enable the race detector:
+
+ $ go test -race mypkg // to test the package
+ $ go run -race mysrc.go // to run the source file
+ $ go build -race mycmd // to build the command
+ $ go install -race mypkg // to install the package
+
+
+* Testing with concurrency
+
+When testing concurrent code, there's a temptation to use sleep;
+it's easy and works most of the time.
+
+But "most of the time" isn't always and flaky tests result.
+
+We can use Go's concurrency primitives to make flaky sleep-driven tests reliable.
+
+
+* Finding errors with static analysis: vet
+
+The `vet` tool checks code for common programmer mistakes:
+
+- bad printf formats,
+- bad build tags,
+- bad range loop variable use in closures,
+- useless assignments,
+- unreachable code,
+- bad use of mutexes,
+- and more.
+
+Usage:
+
+ go vet [package]
+
+
+* Testing from the inside
+
+Most tests are compiled as part of the package under test.
+
+This means they can access unexported details, as we have already seen.
+
+
+* Testing from the outside
+
+Occasionally you want to run your tests from outside the package under test.
+
+(Test files as `package` `foo_test` instead of `package` `foo`.)
+
+This can break dependency cycles. For example:
+
+- The `testing` package uses `fmt`.
+- The `fmt` tests must import `testing`.
+- So the `fmt` tests are in package `fmt_test` and can import both `testing` and `fmt`.
+
+
+* Mocks and fakes
+
+Go eschews mocks and fakes in favor of writing code that takes broad interfaces.
+
+For example, if you're writing a file format parser, don't write a function like this:
+
+ func Parse(f *os.File) error
+
+instead, write functions that take the interface you need:
+
+ func Parse(r io.Reader) error
+
+(An `*os.File` implements `io.Reader`, as does `bytes.Buffer` or `strings.Reader`.)
+
+
+* Subprocess tests
+
+Sometimes you need to test the behavior of a process, not just a function.
+
+.code testing/subprocess/subprocess.go /func Crasher/,/^}/
+
+To test this code, we invoke the test binary itself as a subprocess:
+
+.code testing/subprocess/subprocess_test.go /func TestCrasher/,/^}/
+
+
+* More information
+
+.link http://golang.org/pkg/testing/
+
+.link http://golang.org/cmd/go/
+
+.link http://golang.org
+
diff --git a/2014/testing/cover.png b/2014/testing/cover.png
new file mode 100644
index 0000000..57d5d67
--- /dev/null
+++ b/2014/testing/cover.png
Binary files differ
diff --git a/2014/testing/go1.1.png b/2014/testing/go1.1.png
new file mode 100644
index 0000000..beb3325
--- /dev/null
+++ b/2014/testing/go1.1.png
Binary files differ
diff --git a/2014/testing/httprecorder.go b/2014/testing/httprecorder.go
new file mode 100644
index 0000000..95ea793
--- /dev/null
+++ b/2014/testing/httprecorder.go
@@ -0,0 +1,26 @@
+package main
+
+import (
+ "fmt"
+ "log"
+ "net/http"
+ "net/http/httptest"
+)
+
+func main() {
+ // START OMIT
+ handler := func(w http.ResponseWriter, r *http.Request) {
+ http.Error(w, "something failed", http.StatusInternalServerError)
+ }
+
+ req, err := http.NewRequest("GET", "http://example.com/foo", nil)
+ if err != nil {
+ log.Fatal(err)
+ }
+
+ w := httptest.NewRecorder()
+ handler(w, req)
+
+ fmt.Printf("%d - %s", w.Code, w.Body.String())
+ // STOP OMIT
+}
diff --git a/2014/testing/httpserver.go b/2014/testing/httpserver.go
new file mode 100644
index 0000000..5917705
--- /dev/null
+++ b/2014/testing/httpserver.go
@@ -0,0 +1,31 @@
+package main
+
+import (
+ "fmt"
+ "io/ioutil"
+ "log"
+ "net/http"
+ "net/http/httptest"
+)
+
+func main() {
+ // START OMIT
+ ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+ fmt.Fprintln(w, "Hello, client")
+ }))
+ defer ts.Close()
+
+ res, err := http.Get(ts.URL)
+ if err != nil {
+ log.Fatal(err)
+ }
+
+ greeting, err := ioutil.ReadAll(res.Body)
+ res.Body.Close()
+ if err != nil {
+ log.Fatal(err)
+ }
+
+ fmt.Printf("%s", greeting)
+ // STOP OMIT
+}
diff --git a/2014/testing/subprocess/subprocess.go b/2014/testing/subprocess/subprocess.go
new file mode 100644
index 0000000..8fb8fc0
--- /dev/null
+++ b/2014/testing/subprocess/subprocess.go
@@ -0,0 +1,11 @@
+package subprocess
+
+import (
+ "fmt"
+ "os"
+)
+
+func Crasher() {
+ fmt.Println("Going down in flames!")
+ os.Exit(1)
+}
diff --git a/2014/testing/subprocess/subprocess_test.go b/2014/testing/subprocess/subprocess_test.go
new file mode 100644
index 0000000..27b05d0
--- /dev/null
+++ b/2014/testing/subprocess/subprocess_test.go
@@ -0,0 +1,21 @@
+package subprocess
+
+import (
+ "os"
+ "os/exec"
+ "testing"
+)
+
+func TestCrasher(t *testing.T) {
+ if os.Getenv("BE_CRASHER") == "1" {
+ Crasher()
+ return
+ }
+ cmd := exec.Command(os.Args[0], "-test.run=TestCrasher")
+ cmd.Env = append(os.Environ(), "BE_CRASHER=1")
+ err := cmd.Run()
+ if e, ok := err.(*exec.ExitError); ok && !e.Success() {
+ return
+ }
+ t.Fatalf("process ran with err %v, want exit status 1", err)
+}
diff --git a/2014/testing/test1/string_test.go b/2014/testing/test1/string_test.go
new file mode 100644
index 0000000..c458ffe
--- /dev/null
+++ b/2014/testing/test1/string_test.go
@@ -0,0 +1,14 @@
+package strings_test
+
+import (
+ "strings"
+ "testing"
+)
+
+func TestIndex(t *testing.T) {
+ const s, sep, want = "chicken", "ken", 4
+ got := strings.Index(s, sep)
+ if got != want {
+ t.Errorf("Index(%q,%q) = %v; want %v", s, sep, want, got)
+ }
+}
diff --git a/2014/testing/test2/string_test.go b/2014/testing/test2/string_test.go
new file mode 100644
index 0000000..886597a
--- /dev/null
+++ b/2014/testing/test2/string_test.go
@@ -0,0 +1,43 @@
+package strings_test
+
+import (
+ "fmt"
+ "strings"
+ "testing"
+)
+
+func TestIndex(t *testing.T) {
+ var tests = []struct {
+ s string
+ sep string
+ out int
+ }{
+ {"", "", 0},
+ {"", "a", -1},
+ {"fo", "foo", -1},
+ {"foo", "foo", 0},
+ {"oofofoofooo", "f", 2},
+ // etc
+ }
+ for _, test := range tests {
+ actual := strings.Index(test.s, test.sep)
+ if actual != test.out {
+ t.Errorf("Index(%q,%q) = %v; want %v", test.s, test.sep, actual, test.out)
+ }
+ }
+}
+
+func BenchmarkIndex(b *testing.B) {
+ const s = "some_text=some☺value"
+ for i := 0; i < b.N; i++ {
+ strings.Index(s, "v")
+ }
+}
+
+func ExampleIndex() {
+ fmt.Println(strings.Index("chicken", "ken"))
+ fmt.Println(strings.Index("chicken", "dmr"))
+ // Output:
+ // 4
+ // -1
+}
