Go: 90% Perfect, 100% of the time.
"GoCon Tokyo"
31 May 2014
Tags: go, gocon

Brad Fitzpatrick
@bradfitz
bradfitz@golang.org
http://bradfitz.com
http://camlistore.org

* Hello!

* "60% of the time, it works every time...."

.image gocon-tokyo/60p.jpg

* "That makes no sense"

* Starting a new project, picking a language...

* Disclaimer

- Personal experience only
- My opinions
- Fake, fun numbers (e.g. "90% Perfect, 100% of the time.")
- Maybe it's 91%.
- All languages are both wonderful and terrible
- <3

* 2010

* Idea: Camlistore

- All my life's data from everywhere,
- Import, backup, serve, browse, search, share, index
- Designed to last 100+ years
- Don't trust any company
- In 2010: an idea
- In 2014: busy, fun, active project

* Which language to use for Camlistore?

* Before Go

- Starting a new project from 1994 ~ 2010...
- Perl or C?
- Rarely but occasionally Java.

* Perl: The Good

- Fun
- Expressive
- Get things done, quickly

* Python, Ruby, Javascript: The Good

- Same

* Perl: The Bad

- Not fast like C
- Sometimes need to write in C
- ... or write in XS (Perl-C glue language)
- Perl-C, C-Perl calls are slow
- Single-threaded, single-core, callback hell

* Python, Ruby, Javascript: The Bad

- Same, but even slower :)
- JavaScript is faster now, but not as fast as C

* Summary

- Perlythonubyscript: slow for computer, fast & fun for human
- C: slow for human, fast for computer
- C++: little more fun than C, but more complexity
- Java: tedious (verbose), slow start-up, easy to find programmers
- PHP: if somebody's paying you enough, maybe

* Fun vs. fast

.image gocon-tokyo/funfast-nogo.svg

* Maybe mix of two languages?

- Fun, slow parts in Perlythonubyscript?
- Important, fast parts in C?

* But...

- You'll want to share code.
- Calling from C to X or X to C is slow.
- The C-X glue language is the least fun language
- You'll be lazy, and write in the fun language
- Or be lazy and waste time, and write too much stuff in C++ (like Google)

* Server? Threads or events?

* Threads

- C, C++, Java, ...
- Threads? Heavy. Lots of memory per thread for stacks. Be careful!
  Executor pool = new ScheduledThreadPoolExecutor(/* ???? guess a number */);

* Events

- Perlythonubyscript: events. no choice. Callback hell.
- C, C++: libevent, etc. Callback hell.

.image gocon-tokyo/spaghetti.jpg

- C#: await async await async await async await async noise

* Concurrency and performance

.image gocon-tokyo/concurrency0.svg

* It's really hard to pick a language!

* 2010. Let's try Go!

* Go: 90% Perfect, 100% of the time.

* Go: 90% Perfect, 100% of the time.

- Go is very good at lots of things
- High-level code when you want
- Low-level code when you want
- Start high, profile, go low-level when needed
- Static typing without too much keyboard typing

* Before Go

.image gocon-tokyo/funfast-nogo.svg

* After Go

.image gocon-tokyo/funfast.svg

* Go's Concurrency

- Built-in! Lightweight threads, no callback hell
- Readable, top-down code
- So easy to write servers

  for {
    conn, err := listener.Accept()
    // check err
    go serve(conn)
  }

- Goroutine-per-request can scale

* Concurrency before Go

.image gocon-tokyo/concurrency0.svg

* After Go

.image gocon-tokyo/concurrency.svg

* Go is showing up for all sorts of tasks

* Web frameworks

- Typically a job for scripting languages: Node.js, Python, Ruby, Perl, PHP, etc.
- net/http in the standard library
- Other http frameworks on top: gorilla, martini, Revel etc...
- No need for nginx: standard library's http server scales

* Image processing

- no libpng, imagemagick, etc
- pure Go PNG, JPEG, GIF encoding & decoding in the standard library
- slower, but safer
- getting faster

* Crypto

- no OpenSSL, no GnuTLS
- pure Go AES, TLS (SSL), OpenPGP, etc
- slower, but safer
- getting faster

* Replacing shell scripts

- Instead of little Perl & shell scripts, I now write in Go
- os/exec package & goroutines makes subprocess management easier than shell

* Camlistore

- web handlers
- using third-party web APIs
- image thumbnails: decode, resize, encode
- EXIF, ID3 parsing
- database drivers
- crypto, image decoding/resizing/encoding
- search, indexing: tight memory layout control
- WebSockets, HTTPS
- all pure Go!

* Controlling flying drones

.link http://www.godrone.io
- "Isn’t Go unsuitable for real-time applications like this?"
- "However, for all practical purposes the GC just needs to keep up with the stabilization loop which runs at 200 Hz. This means that GC pauses below 5ms have no impact on performance. Longer pauses will degrade stabilization performance, but the tolerance threshold may be up to a second ..." (small heap)

.image gocon-tokyo/drone.png

* Emulators

.link http://dave.cheney.net/2013/01/09/go-the-language-for-emulators Go: The Language For Emulators

.image gocon-tokyo/trs.png

* Disassembler, linker, compiler.

- llgo compiler (Go -> LLVM IR -> ...)
- Go 1.3's disassembler
- Go 1.4's linker
- Go 1.4 or Go 1.5's compiler

* Mobile

- Go runs on ARM
- minux's iOS port of Go
- Camlistore child process, goandroid

.image gocon-tokyo/goandroid.png

* Audio synthesis

- https://github.com/nf/sigourney

.image gocon-tokyo/sigourney.png

* Cloud infrastructure

.link http://redmonk.com/dberkholz/2014/03/18/go-the-emerging-language-of-cloud-infrastructure/ Go: the emerging language of cloud infrastructure

- Docker, Packer
- CoreOS’s etcd and fleet
- Ubuntu Juju, Mozilla Heka, Apcera’s NATS, gnatsd

.image gocon-tokyo/docker.png

* Load balancers & servers

- YouTube's vitess: MySQL
- net/http/httputil.ReverseProxy
- dl.google.com
- etc

* Raspberry Pi GPIO

.link https://github.com/davecheney/gpio
.link https://github.com/stianeikeland/go-rpio
.link https://github.com/luismesas/goPi

.image gocon-tokyo/gpio.gif

* What other language is used for all these?

- web apps
- scripts
- system administration
- image processing
- load balancers, servers
- crypto
- hardware

Go!

* And even better...

* Go built-in tools

- testing
- benchmarking
- profiling (CPU, memory, blocking)
- huge standard library
- "go get"
- godoc
- gofmt, goimports
- race detector
- static binaries

* So why isn't Go perfect?

* Generics

- No generics
- It sucks sometimes, but rarely
- Use maps, slices, interfaces
- Occasionally a gross interface{}
- No algorithms in stdlib
- No great proposal yet

* No generics (Simon Peyton Jones: "Haskell is useless")

.image gocon-tokyo/generics.svg

* Data races can happen

- shared, mutable state is possible
- use channels
- be careful
- conventions and documentation help
- runtime race detector helps too
  $ go test -race
* Code generation

- The default Go compiler often generates pretty dumb code
- Especially on ARM
- But getting better
- In Go 1.4, 1.5+: Go compiler in Go, refactor, add SSA, more optimizations

* gccgo

- gccgo generates very good code
- ... but lacks escape analysis: kills performance with many small allocs + garbage
- ... GC isn't precise. Bad for 32-bit.

* Compiling to JavaScript isn't yet great

- Would be nice to write frontend & backend in the same language, share code
- go/types + go/ssa enables many things:
.link http://tardisgo.github.io/ TARDIS Go Compiler
.image gocon-tokyo/tardis.png

* Limited Mobile Support

- No built-in Android support (but goandroid, child processes...)
- No built-in iOS support (but unmerged port...)

* Embedding Go in C/C++/Java/etc

- Can't embed Go in other languages, environments.
- e.g. Android, iOS, C/C++ programs
- problems with memory setup, GC, signals, ...
- need embedding API design
- In Go 1.4?

* Shared libraries

- no shared libraries
- can't load Go code at runtime

* Garbage collector

- Pauses: faster each release, good people working on it.
- Precision: heap in 1.2, most stacks in 1.3, more in 1.4.
- 4 versions of memcached: Perl, C, C++, Go

* Hot stack splits

- Almost entirely fixed in Go 1.3's contiguous stacks
- Rest should be in Go 1.4.

.image gocon-tokyo/contig-stack.png

* But...

- Problems are fixable.
- It all keeps getting better with each release.

* Go for everything

- Since mid-2010, I prefer 90%-perfect Go for all my hacking:
.image gocon-tokyo/changestats.png
- Go is flexible, fun, readable
- Go is fast (for computers & humans)
- Go is good for working with others
- Go for everything!