go.talks: add 'go at google'
Keynote from Oct 25, 2012 by Rob Pike at ACM SPLASH conference.
R=adg, dchest
CC=golang-dev
https://golang.org/cl/6777060
diff --git a/2012/splash.slide b/2012/splash.slide
new file mode 100644
index 0000000..4271bf9
--- /dev/null
+++ b/2012/splash.slide
@@ -0,0 +1,832 @@
+Go at Google
+SPLASH, Tucson, Oct 25, 2012
+
+Rob Pike
+Google, Inc.
+@rob_pike
+http://golang.org/s/plusrob
+http://golang.org
+
+* Preamble
+
+* What is Go?
+
+Go is:
+
+- open source
+- concurrent
+- garbage-collected
+- efficient
+- scalable
+- simple
+- fun
+- boring (to some)
+
+.link http://golang.org http://golang.org
+
+* History
+
+Design began in late 2007.
+
+Key players:
+
+- Robert Griesemer, Rob Pike, Ken Thompson
+- Later: Ian Lance Taylor, Russ Cox
+
+Became open source in November 2009.
+
+Developed entirely in the open; very active community.
+Language stable as of Go 1, early 2012.
+
+* Go at Google
+
+Go is a programming language designed by Google to help solve Google's problems.
+
+Google has big problems.
+
+* Big hardware
+
+.image splash/datacenter.jpg
+
+* Big software
+
+- C++ (mostly) for servers, plus lots of Java and Python
+- thousands of engineers
+- gazillions of lines of code
+- distributed build system
+- one tree
+
+And of course:
+
+- zillions of machines, which we treat as a modest number of compute clusters
+
+Development at Google can be slow, often clumsy.
+
+But it _is_ effective.
+
+* The reason for Go
+
+Goals:
+
+- eliminate slowness
+- eliminate clumsiness
+- improve effectiveness
+- maintain (even improve) scale
+
+Go was designed by and for people who write—and read and debug and maintain—large software systems.
+
+Go's purpose is _not_ research into programming language design.
+
+Go's purpose is to make its designers' programming lives better.
+
+* Today's theme
+
+A talk about software engineering more than language design.
+
+To be more accurate:
+
+- Language design in the service of software engineering.
+
+In short:
+
+- How does a language help software engineering?
+
+* Features?
+
+Reaction upon launch:
+My favorite feature isn't in Go! Go Sucks!
+
+This misses the point.
+
+* Pain
+
+What makes large-scale development hard with C++ or Java (at least):
+
+- slow builds
+- uncontrolled dependencies
+- each programmer using a different subset of the language
+- poor program understanding (documentation, etc.)
+- duplication of effort
+- cost of updates
+- version skew
+- difficulty of automation (auto rewriters etc.): tooling
+- cross-language builds
+
+Language _features_ don't usually address these.
+
+* Focus
+
+In the design of Go, we tried to focus on solutions to _these_ problems.
+
+Example: indentation for structure vs. C-like braces
+
+* Dependencies in C and C++
+
+* A personal history of dependencies in C
+
+`#ifdef`
+
+`#ifndef` "guards":
+
+ #ifndef _SYS_STAT_H_
+
+1984: `<sys/stat.h>` times 37
+
+ANSI C and `#ifndef` guards:
+
+- dependencies accumulate
+- throw includes at the program until it compiles
+- no way to know what can be removed
+
+* A personal history of dependencies in Plan 9's C
+
+Plan 9, another take:
+
+- no `#ifdefs` (or `#ifndefs`)
+- documentation and topological sort
+- easy to find out what can be removed
+
+Need to document dependencies, but much faster compilation.
+
+In short:
+
+- _ANSI_C_made_a_costly_mistake_ in requiring `#ifndef` guards.
+
+* A personal history of dependencies in C++
+
+C++ exacerbated the problem:
+
+- one `#include` file per class
+- code (not just declarations) in `#include` files
+- `#ifndef` guards persist
+
+2004: Mike Burrows and Chubby: `<xxx>` times 37,000
+
+1984: Tom Cargill and pi
+
+* A personal history of dependencies at Google
+
+Plan 9 demo: a story
+
+Early Google: one `Makefile`
+
+2003: `Makefile` generated from per-directory `BUILD` files
+
+- explicit dependencies
+- 40% smaller binaries
+
+Dependencies still not checkable!
+
+* Result
+
+To build a large Google binary on a single computer is impractical.
+
+In 2007, instrumented the build of a major Google binary:
+
+- 2000 files
+- 4.2 megabytes
+- 8 gigabytes delivered to compiler
+- 2000 bytes sent to compiler for every C++ source byte
+- it's real work too: `<string>` for example
+- hours to build
+
+* Tools can help
+
+New distributed build system:
+
+- no more `Makefile` (still uses `BUILD` files)
+- many buildbots
+- much caching
+- much complexity (a large program in its own right)
+
+Even with Google's massive distributed build system, a large build still takes minutes.
+(In 2007 that binary took 45 minutes; today, 27 minutes.)
+
+Poor quality of life.
+
+* Enter Go
+
+While that build runs, we have time to think.
+
+Want a language to improve the quality of life.
+
+And dependencies are only one such problem....
+
+* Primary considerations
+
+Must work at scale:
+
+- large programs
+- large teams
+- large number of dependencies
+
+Must be familiar, roughly C-like
+
+* Modernize
+
+The old ways are _old_.
+
+Go should be:
+
+- suitable for multicore machines
+- suitable for networked machines
+- suitable for web stuff
+
+* The design of Go
+
+From a software engineering perspective.
+
+* Dependencies in Go
+
+* Dependencies
+
+Dependencies are defined (syntactically) in the language.
+
+Explicit, clear, computable.
+
+ import "encoding/json"
+
+Unused dependencies cause error at compile time.
+
+Efficient: dependencies traversed once per source file...
+
+* Hoisting dependencies
+
+Consider:
+`A` imports `B` imports `C` but `A` does not directly import `C`.
+
+The object code for `B` includes all the information about `C` needed to import `B`.
+Therefore in `A` the line
+
+ import "B"
+
+does not require the compiler to read `C` when compiling `A`.
+
+Also, the object files are designed so the "export" information comes first; compiler doing import does not need to read whole file.
+
+Exponentially less data read than with `#include` files.
+
+With Go in Google, about 40X fanout (recall C++ was 2000x)
+Plus in C++ it's general code that must be parsed; in Go it's just export data.
+
+* No circular imports
+
+Circular imports are illegal in Go.
+
+The big picture in a nutshell:
+
+- occasional minor pain,
+- but great reduction in annoyance overall
+- structural typing makes it less important than with type hierarchies
+- keeps us honest!
+
+Forces clear demarcation between packages.
+
+Simplifies compilation, linking, initialization.
+
+* API design
+
+Through the design of the standard library, great effort spent on controlling dependencies.
+
+It can be better to copy a little code than to pull in a big library for one function.
+(A test in the system build complains if new core dependencies arise.)
+
+Dependency hygiene trumps code reuse.
+
+Example:
+The (low-level) `net` package has own `itoa` to avoid dependency on the big formatted I/O package.
+
+* Packages
+
+* Packages
+
+Every Go source file, e.g. `"encoding/json/json.go"`, starts
+
+ package json
+
+where `json` is the "package name", an identifier.
+Package names are usually concise.
+
+To use a package, need to identify it by path:
+
+ import "encoding/json"
+
+And then the package name is used to qualify items from package:
+
+ var dec = json.NewDecoder(reader)
+
+Clarity: can always tell if name is local to package from its syntax: `Name` vs. `pkg.Name`.
+(More on this later.)
+
+Package combines properties of library, name space, and module.
+
+* Package paths are unique, not package names
+
+The path is `"encoding/json"` but the package name is `json`.
+The path identifies the package and must be unique.
+Project or company name at root of name space.
+
+ import "google/base/go/log"
+
+Package name might not be unique; can be overridden. These are both `package`log`:
+
+ import "log" // Standard package
+ import googlelog "google/base/go/log" // Google-specific package
+
+Every company might have its own `log` package; no need to make the package name unique.
+
+Another: there are many `server` packages in Google's code base.
+
+* Remote packages
+
+Package path syntax works with remote repositories.
+The import path is just a string.
+
+Can be a file, can be a URL:
+
+ go get github.com/4ad/doozer // Command to fetch package
+
+ import "github.com/4ad/doozer" // Doozer client's import statement
+
+ var client doozer.Conn // Client's use of package
+
+* Go's Syntax
+
+* Syntax
+
+Syntax is not important...
+
+- unless you're programming
+- or writing tools
+
+Tooling is essential, so Go has a clean syntax.
+Not super small, just clean:
+
+- regular (mostly)
+- only 25 keywords
+- straightforward to parse (no type-specific context required)
+- easy to predict, reason about
+
+* Declarations
+
+Uses Pascal/Modula-style syntax: name before type, more type keywords.
+
+ var fn func([]int) int
+ type T struct { a, b int }
+
+not
+
+ int (*fn)(int[]);
+ struct T { int a, b; }
+
+Easier to parse—no symbol table needed. Tools become easier to write.
+
+One nice effect: can drop `var` and derive type of variable from expression:
+
+ var buf *bytes.Buffer = bytes.NewBuffer(x) // explicit
+ buf := bytes.NewBuffer(x) // derived
+
+For more information:
+
+.link http://golang.org/s/decl-syntax
+
+* Function syntax
+
+Function on type `T`:
+
+ func Abs(t T) float64
+
+Method of type `T`:
+
+ func (t T) Abs() float64
+
+Variable (closure) of type `T`:
+
+ negAbs := func(t T) float64 { return -Abs(t) }
+
+In Go, functions can return multiple values. Common case: `error`.
+
+ func ReadByte() (c byte, err error)
+
+ c, err := ReadByte()
+ if err != nil { ... }
+
+More about errors later.
+
+* No default arguments
+
+Go does not support default function arguments.
+
+Why not?
+
+- too easy to throw in defaulted args to fix design problems
+- encourages too many args
+- too hard to understand the effect of the fn for different combinations of args
+
+Extra verbosity may happen but that encourages extra thought about names.
+
+Related: Go has easy-to-use, type-safe support for variadic functions.
+
+* Naming
+
+* Export syntax
+
+Simple rule:
+
+- upper case initial letter: `Name` is visible to clients of package
+- otherwise: `name` (or `_Name`) is not visible to clients of package
+
+Applies to variables, types, functions, methods, constants, fields....
+
+That Is It.
+
+Not an easy decision.
+One of the most important things about the language.
+
+Can see the visibility of an identifier without discovering the declaration.
+
+Clarity.
+
+* Scope
+
+Go has very simple scope hierarchy:
+
+- universe
+- package
+- file (for imports only)
+- function
+- block
+
+* Locality of naming
+
+Nuances:
+
+- no implicit `this` in methods (receiver is explicit); always see `rcvr.Field`
+- package qualifier always present for imported names
+- (first component of) every name is always declared in current package
+
+No surprises when importing:
+
+- adding an exported name to my package cannot break your package!
+
+Names do not leak across boundaries.
+
+In C, C++, Java the name `y` could refer to anything
+In Go, `y` (or even `Y`) is always defined within the package.
+In Go, `x.Y` is clear: find `x` locally, `Y` belongs to it.
+
+* Function and method lookup
+
+Method lookup by name only, not type.
+A type cannot have two methods with the same name, ever.
+Easy to identify which function/method is referred to.
+Simple implementation, simpler program, fewer surprises.
+
+Given a method `x.M`, there's only ever one `M` associated with `x`.
+
+* Semantics
+
+* Basics
+
+Generally C-like:
+
+- statically typed
+- procedural
+- compiled
+- pointers etc.
+
+Should feel familiar to programmers from the C family.
+
+* But...
+
+Many small changes in the aid of robustness:
+
+- no pointer arithmetic
+- no implicit numeric conversions
+- array bounds checking
+- no type aliases
+- `++` and `--` as statements not expressions
+- assignment not an expression
+- legal (encouraged even) to take address of stack variable
+- many more
+
+Plus some big ones...
+
+* Bigger things
+
+Some elements of Go step farther from C, even C++ and Java:
+
+- concurrency
+- garbage collection
+- interface types
+- reflection
+- type switches
+
+* Concurrency
+
+* Concurrency
+
+Important to modern computing environment.
+Not well served by C++ or even Java.
+
+Go embodies a variant of CSP with first-class channels.
+
+Why CSP?
+
+- The rest of the language can be ordinary and familiar.
+
+Must be able to couple concurrency with computation.
+
+Example: concurrency and cryptography.
+
+* CSP is practical
+
+For a web server, the canonical Go program, the model is a great fit.
+
+Go _enables_ simple, safe concurrent programming.
+It doesn't _forbid_ bad programming.
+
+Focus on _composition_ of regular code.
+
+Caveat: not purely memory safe; sharing is legal.
+Passing a pointer over a channel is idiomatic.
+
+Experience shows this is a practical design.
+
+* Garbage collection
+
+* The need for garbage collection
+
+Too much programming in C and C++ is about memory allocation.
+But also the design revolves too much about memory management.
+Leaky abstractions, leaky dependencies.
+
+Go has garbage collection, only.
+
+Needed for concurrency: tracking ownership too hard otherwise.
+Important for abstraction: separate behavior from resource management.
+A key part of scalability: APIs remain local.
+
+Use of the language is much simpler because of GC.
+Adds run-time cost, latency, complexity to the implementation.
+
+Day 1 design decision.
+
+* Garbage collection in Go
+
+A garbage-collected systems language is heresy!
+Experience with Java: Uncontrollable cost, too much tuning.
+
+But Go is different.
+Go lets you limit allocation by controlling memory layout.
+
+Example:
+
+ type X struct {
+ a, b, c int
+ buf [256]byte
+ }
+
+Example: Custom arena allocator with free list.
+
+* Interior pointers
+
+Early decision: allow interior pointers such as `X.buf` from previous slide.
+
+Tradeoff: Affects which GC algorithms that can be used, but in return reduces pressure on the collector.
+
+Gives the _programmer_ tools to control GC overhead.
+
+Experience, compared to Java, shows it has significant effect on memory pressure.
+
+GC remains an active subject.
+Current design: parallel mark-and-sweep.
+With care to use memory wisely, works well in production.
+
+* Interfaces
+
+Composition not inheritance
+
+* Object orientation and big software
+
+Go is object-oriented.
+Go does not have classes or subtype inheritance.
+
+What does this mean?
+
+* No type hierarchy
+
+O-O is important because it provides uniformity of interface.
+Outrageous example: the Plan 9 kernel.
+
+Problem: subtype inheritance encourages _non-uniform_ interfaces.
+
+* O-O and program evolution
+
+Design by type inheritance oversold.
+Generates brittle code.
+Early decisions hard to change, often poorly informed.
+Makes every programmer an interface designer.
+(Plan 9 was built around a single interface everything needed to satisfy.)
+
+Therefore encourages overdesign early on: predict every eventuality.
+Exacerbates the problem, complicates designs.
+
+* Go: interface composition
+
+In Go an interface is _just_ a set of methods:
+
+ type Hash interface {
+ Write(p []byte) (n int, err error)
+ Sum(b []byte) []byte
+ Reset()
+ Size() int
+ BlockSize() int
+ }
+
+No `implements` declaration.
+All hash implementations satisfy this implicitly. (Statically checked.)
+
+* Interfaces in practice: composition
+
+Tend to be small: one or two methods are common.
+
+Composition falls out trivially. Easy example, from package `io`:
+
+ type Reader interface {
+ Read(p []byte) (n int, err error)
+ }
+
+`Reader` (plus the complementary `Writer`) makes it easy to chain:
+
+- files, buffers, networks, encryptors, compressors, GIF, JPEG, PNG, ...
+
+Dependency structure is not a hierarchy; these also implement other interfaces.
+
+Growth through composition is _natural_, does not need to be pre-declared.
+
+And that growth can be _ad_hoc_ and linear.
+
+* Compose with functions, not methods
+
+Hard to overstate the effect that Go's interfaces have on program design.
+
+One big effect: functions with interface arguments.
+
+ func ReadAll(r io.Reader) ([]byte, error)
+
+Wrappers:
+
+ func LoggingReader(r io.Reader) io.Reader
+ func LimitingReader(r io.Reader, n int64) io.Reader
+ func ErrorInjector(r io.Reader) io.Reader
+
+The designs are nothing like hierarchical, subtype-inherited methods.
+Much looser, organic, decoupled, independent.
+
+* Errors
+
+* Error handling
+
+Multiple function return values inform the design for handling errors.
+
+Go has no `try-catch` control structures for exceptions.
+Return `error` instead: built-in interface type that can "stringify" itself:
+
+ type error interface { Error() string }
+
+Clear and simple.
+
+Philosophy:
+
+Forces you think about errors—and deal with them—when they arise.
+Errors are _normal_. Errors are _not_exceptional_.
+Use the existing language to compute based on them.
+Don't need a sublanguage that treats them as exceptional.
+
+Result is better code (if more verbose).
+
+* (OK, not all errors are normal. But most are.)
+
+.image splash/fire.jpg
+
+* Tools
+
+* Tools
+
+Software engineering requires tools.
+
+Go's syntax, package design, naming, etc. make tools easy to write.
+
+Standard library includes lexer and parser; type checker nearly done.
+
+* Gofmt
+
+Always intended to do automatic code formatting.
+Eliminates an entire class of argument.
+Runs as a "presubmit" to the code repositories.
+
+Training:
+
+- The community has always seen `gofmt` output.
+
+Sharing:
+
+- Uniformity of presentation simplifies sharing.
+
+Scaling:
+
+- Less time spent on formatting, more on content.
+
+Often cited as one of Go's best features.
+
+* Gofmt and other tools
+
+Surprise: The existence of `gofmt` enabled _semantic_ tools:
+Can rewrite the tree; `gofix` will clean up output.
+
+Examples:
+
+- `gofmt`-r`'a[b:len(a)]`->`a[b:]'`
+- `gofix`
+
+And good front-end libraries enable ancillary tools:
+
+- `godoc`
+- `go`get`, `go`build`, etc.
+- `api`
+
+* Gofix
+
+The `gofix` tool allowed us to make sweeping changes to APIs and language features leading up to the release of Go 1.
+
+- change to map deletion syntax
+- new time API
+- many more
+
+Also allows us to _update_ code even if the old code still works.
+
+Recent example:
+
+Changed Go's protocol buffer implementation to use getter functions; updated _all_ Google Go code to use them with `gofix`.
+
+* Conclusion
+
+* Go at Google
+
+Go's use is growing inside Google.
+
+Several big services use it:
+
+- golang.org
+- youtube.com
+- dl.google.com
+
+Many small ones do, many using Google App Engine.
+
+* Go outside Google
+
+Many outside companies use it, including:
+
+- BBC Worldwide
+- Canonical
+- Heroku
+- Nokia
+- SoundCloud
+
+* What's wrong?
+
+Not enough experience yet to know if Go is truly successful.
+Not enough big programs.
+
+Some minor details wrong. Examples:
+
+- declarations still too fussy
+- `nil` is overloaded
+- lots of library details
+
+`Gofix` and `gofmt` gave us the opportunity to fix many problems, ranging from eliminating semicolons to redesigning the `time` package.
+But we're still learning (and the language is frozen for now).
+
+The implementation still needs work, the run-time system in particular.
+
+But all indicators are positive.
+
+* Summary
+
+Software engineering guided the design.
+But a productive, fun language resulted because that design enabled productivity.
+
+Clear dependencies
+Clear syntax
+Clear semantics
+Composition not inheritance
+Simplicity of model (GC, concurrency)
+Easy tooling (the `go` tool, `gofmt`, `godoc`, `gofix`)
+
+* Try it!
+
+.link http://golang.org http://golang.org
+
+.image splash/appenginegophercolor.jpg
+
diff --git a/2012/splash/appenginegophercolor.jpg b/2012/splash/appenginegophercolor.jpg
new file mode 100644
index 0000000..43e84fd
--- /dev/null
+++ b/2012/splash/appenginegophercolor.jpg
Binary files differ
diff --git a/2012/splash/datacenter.jpg b/2012/splash/datacenter.jpg
new file mode 100644
index 0000000..c2de0f3
--- /dev/null
+++ b/2012/splash/datacenter.jpg
Binary files differ
diff --git a/2012/splash/fire.jpg b/2012/splash/fire.jpg
new file mode 100644
index 0000000..4cdbd6b
--- /dev/null
+++ b/2012/splash/fire.jpg
Binary files differ
