blob: 04504c0ce8ac8e3870cad81ca51c5bdb79a664ab [file] [log] [blame]
Gophers With Hammers
Josh Bleecher Snyder
PayPal
josharian@gmail.com
@offbymany
* Go was designed with tools in mind. (Rob Pike)
* Designed with tools in mind
Simple, regular syntax
Simple semantics
Batteries included
* Tools everywhere
- go
- gofmt, goimports
- godoc
- go test [-cover] [-race]
- go vet
- gofix, gofmt -r, eg
- oracle
- golint
- godep
and more...
* go command
$ go list -f '{{.Deps}}' bytes
[errors io runtime sync sync/atomic unicode unicode/utf8 unsafe]
* gofmt
from
for{
fmt.Println( "I feel pretty." );
}
to
for {
fmt.Println("I feel pretty.")
}
* godoc
$ godoc strings Repeat
func Repeat(s string, count int) string
Repeat returns a new string consisting of count copies of the string s.
* go vet
Oops
if suffix != ".md" || suffix != ".markdown" {
Flagged
suspect or: suffix != ".md" || suffix != ".markdown"
* go tool cover -mode=set
func Repeat(s string, count int) string {
b := make([]byte, len(s)*count)
bp := 0
for i := 0; i < count; i++ {
bp += copy(b[bp:], s)
}
return string(b)
}
to
func Repeat(s string, count int) string {
GoCover.Count[0] = 1
b := make([]byte, len(s)*count)
bp := 0
for i := 0; i < count; i++ {
GoCover.Count[2] = 1
bp += copy(b[bp:], s)
}
GoCover.Count[1] = 1
return string(b)
}
* go test -cover
$ go test -coverprofile=c.out strings
ok strings 0.455s coverage: 96.9% of statements
$ go tool cover -func=c.out
strings/reader.go: Len 66.7%
strings/reader.go: Read 100.0%
strings/reader.go: ReadAt 100.0%
strings/reader.go: ReadByte 100.0%
strings/reader.go: UnreadByte 100.0%
strings/reader.go: ReadRune 100.0%
strings/reader.go: UnreadRune 100.0%
strings/reader.go: Seek 90.9%
strings/reader.go: WriteTo 83.3%
...
$ go tool cover -html=c.out
# opens a browser window, shows line-by-line coverage
* Tools to make tools
- text/template
- go/build
- go/doc
- go/format
- go/{parser,token,ast,printer}
- go.tools/go/types and friends
and more...
* Hammers are fun!
# Why to write your own tools: Fun, learning, profit
* impl
Generate implementation stubs given an interface.
go get github.com/josharian/impl
Generate
$ impl 'f *File' io.Reader
func (f *File) Read(p []byte) (n int, err error) {
}
from
package io
type Reader interface {
Read(p []byte) (n int, err error)
}
* impl
Generate
$ impl 'f *File' io.ReadWriter
func (f *File) Read(p []byte) (n int, err error) {
}
func (f *File) Write(p []byte) (n int, err error) {
}
from
package io
type ReadWriter interface {
Reader
Writer
}
* impl
Generate
$ impl 'c *Ctx' http.Handler
func (c *Ctx) ServeHTTP(http.ResponseWriter, *http.Request) {
}
from
package http
type Handler interface {
ServeHTTP(ResponseWriter, *Request)
}
* Plan
*Find*import*path*and*interface*name*
http.Handler net/http, Handler
Parse interface
net/http, Handler {{"ServeHTTP", {{"", "http.ResponseWriter"}, {"", "*http.Request"}}, {}}}}
Generate output
{{"ServeHTTP", {{"", "http.ResponseWriter"}, {"", "*http.Request"}}, {}}}} profit!
* goimports ftw
import "golang.org/x/tools/imports"
.play hammers/importpath.go /func main/,/^}/
* Hello, AST
*ast.File {
. Package: 1:1
. Name: *ast.Ident {
. . NamePos: 1:9
. . Name: "hack"
. }
. Decls: []ast.Decl (len = 2) {
. . 0: *ast.GenDecl {
. . . TokPos: 1:15
. . . Tok: import
. . . Lparen: -
. . . Specs: []ast.Spec (len = 1) {
. . . . 0: *ast.ImportSpec {
. . . . . Path: *ast.BasicLit {
. . . . . . ValuePos: 1:22
. . . . . . Kind: STRING
. . . . . . Value: "\"net/http\""
. . . . . }
[truncated]
* Extract the import path
import (
"go/parser"
"go/token"
)
.play hammers/extractpath.go /func main/,/^}/
* Extract the interface name
import "go/ast"
.play hammers/extractiface.go /func main/,/^}/
A `GenDecl` can have many `Specs`
var (
r io.Reader
w io.Writer
)
* Plan
Find import path and interface name
http.Handler net/http, Handler
*Parse*interface*
net/http, Handler {{"ServeHTTP", {{"", "http.ResponseWriter"}, {"", "*http.Request"}}, {}}}}
Generate output
{{"ServeHTTP", {{"", "http.ResponseWriter"}, {"", "*http.Request"}}, {}}}} profit!
* Data structures
Represent
Read(p []byte) (n int, err error)
as
Func{
Name: "Read",
Params: []Param{{Name: "p", Type: "[]byte"}},
Res: []Param{
{Name: "n", Type: "int"},
{Name: "err", Type: "error"},
},
},
* Data structures
.code hammers/types.go /type Func/,/^}/
.code hammers/types.go /type Param/,/^}/
* Find the code
import "go/build"
.play hammers/findthecode.go /func main/,/^}/
* Find the interface declaration
import "go/printer"
.play hammers/findtheifacedecl.go /func main/,/^}/
* Extract function names
No name? It's an embedded interface. Recurse.
type ByteScanner interface {
ByteReader
UnreadByte() error
}
* Extract params and results
No name? Just use `""`.
type ByteWriter interface {
WriteByte(c byte) error
}
* Qualify types
Types can be arbitrarily complicated.
type CrazyGopher interface {
CrazyGoph(int) func(chan<- [32]byte, map[string]int64) ([]rune, error)
}
And we need to rewrite some of them.
int ⇒ int
*Request ⇒ *http.Request
io.Reader ⇒ io.Reader
func(io.Reader, chan map[S][]*T) ⇒ func(io.Reader, chan map[foo.S][]*foo.T))
* Qualify types
.play hammers/fulltype.go /func main/,/end main/
* Plan
Find import path and interface name
http.Handler ⇒ net/http, Handler
Parse interface
net/http, Handler ⇒ {{"ServeHTTP", {{"", "http.ResponseWriter"}, {"", "*http.Request"}}, {}}}}
*Generate*output*
{{"ServeHTTP", {{"", "http.ResponseWriter"}, {"", "*http.Request"}}, {}}}} ⇒ profit!
* Method type
.code hammers/types.go /type Method/,/^}/
.code hammers/types.go /type Func/,/^}/
.code hammers/types.go /type Param/,/^}/
* Use text/template
.play hammers/codegen.go /func main/,/^}/
# Don't generate an AST. It's a lot of work, and Go is its own DSL.
* Ugly is ok
import "go/format"
.play hammers/format.go /func main/,/^}/
* Great success
Full code plus tests at `github.com/josharian/impl`
* Tips
Use `go`get`-d` to download lots of code from `godoc.org/-/index`. (Don't forget to set a temporary `GOPATH`!)
Use (and improve) `github.com/yuroyoro/goast-viewer`.
You don't have to generate all the code. And generating data is even better.
The `go/ast` docs are your friend.
`go.tools/go/types` is powerful.
`go`generate` is coming.
* Nails!
- Break up long strings
- Enums and flags to Stringers
- Dynamic code analysis
- Vet checks
- Reflect ⇒ codegen
- Convention-based http dispatch
- Detect "last line" copy/paste bugs
- AST-aware diff, merge, blame; automated fork analysis
- Machine learning models of ASTs: anomaly detection, bug-prone code detection
- Code fingerprinting
- Examine usage patterns
- Compiler stress tests