2015/gophercon-goevolution.slide

The Evolution of Go
GopherCon 2015 Keynote - July 9, 2015

Robert Griesemer
Google, Inc.
gri@golang.org


* Video

A video of this talk was recorded at GopherCon in Denver.

.link https://www.youtube.com/watch?v=0ReKdcpNyQg Watch the talk on YouTube


* 

.image gophercon-goevolution/GopherEvolution.svg


* Personal background

- Personal interest in programming languages.

- After grad school, programming in industry felt like a huge step backwards.

- Over the years, spent a lot of time thinking about programming languages.

- Good language design is *hard*.

- After 15 years of using C++, only two ways forward: *sell*out*or*bail*out*!

- Lucky to be on board with Rob and Ken.


* Starting points

- Clear goal in mind: We needed a better language for what we do at Google.

- Personal motivation: Wanted a clean, small, compiled language with modern features.

- Clear about what was wrong: Complexity, missing concurrency support, lack of scalability, insane build times.

- Good ideas about how to address issues.

- Unpolished thoughts about the rest.

- Three experienced people's insights on how not to do it.

- Lure: Designing a new language is easy.

What could possibly go wrong?


* Guiding principles

- Simplicity, safety, and readability are paramount.

- Striving for orthogonality in design.

- Minimal: One way to write a piece of code.

- It's about *expressing*algorithms*, not the type system.

- Collective unconscious history of programming languages.

Things of interest should be easy; even if that means not everything is possible.


* Literature on good language design is sparse

.image gophercon-goevolution/HoaresPLHints.JPG 400 600

- "Hints on Programming Language Design" (C.A.R. Hoare, 1973)
- "Everything you always wanted to know about programming languages but were afraid to ask" (C.A.R. Hoare, 1978)


* First design notes

	Date: Sun, 23 Sep 2007 23:33:41 -0700
	From: "Robert Griesemer" <gri@google.com>
	To: "Rob 'Commander' Pike" <r@google.com>, ken@google.com
	Subject: prog lang discussion
	...
	*** General:
	Starting point: C, fix some obvious flaws, remove crud, add a few missing features
	  - no includes, instead: import
	  - no macros (do we need something instead?)
	  - ideally only one file instead of a .h and .c file, module interface
	should be extracted automatically
	  - statements: like in C, though should fix 'switch' statement
	  - expressions: like in C, though with caveats (do we need ',' expressions?)
	  - essentially strongly typed, but probably w/ support for runtime types
	  - want arrays with bounds checking on always (except perhaps in 'unsafe mode'-see section on GC)
	  - mechanism to hook up GC (I think that most code can live w/ GC, but for a true systems
	    programming language there should be mode w/ full control over memory allocation)
	  - support for interfaces (differentiate between concrete, or implementation types, and abstract,
	    or interface types)
	  - support for nested and anonymous functions/closures (don't pay if not used)
	  - a simple compiler should be able to generate decent code
	  - the various language mechanisms should result in predictable code
	...


* Looking back

Many Day One ideas made it into Go:

- Syntax: Leading keyword notation (const, var, type, etc), many cleanups.
- Expressions: 5 binary precedence levels maximum.
- Explicitly sized basic types, rune type, no implicit conversions.

- Packages and imports.
- Methods with explicit receiver parameter.
- Interfaces.
- Understanding that we would somehow add concurrency support based on Rob's previous work.

Many concepts were missing, and even more ideas didn't make it.

However, we were off to a good start. This was not at all obvious at the time!


* Most ideas come from previous ideas.
(Alan Kay)

Or, as some critics would say: There's nothing new in Go!


They are missing the point:

The task of the programming language designer *"*is*consolidation*not*innovation*"*.
(Hoare, 1973).


* The Algol family


* Algol60


(John Backus, John McCarthy, Alan Perlis, et al, 1958-1960)

=> Block structure, nested and recursive functions and procedures, type declarations and static typing, "for" statement, "return" statement, semicolon separated statements, "begin"-"end" blocks, "call by name", etc.

"Here is a language [Algol60] so far ahead of its time, that it was not only an improvement on its predecessors, but also on nearly all its successors." (C.A.R. Hoare)

Coincidentally, a few years before:
- Backus Naur Normal Form (John Backus, Peter Naur, 1958)


* Algol successors

- *Pascal* (N. Wirth, ETH Zürich, 1968-1970)
=> BEGIN/END for blocks, semicolons as separators, left-to-right declarations,
principled structured data types, notion of predeclared ("standard") functions, designed for teaching.

- *C* (Dennis Ritchie, Bell Labs, 1969-1973)
=> Curly braces for blocks, semicolons as terminators, declarations mimic use,
duality between arrays and pointers, static typing but weak enforcement, designed to write Unix kernel.


* Pascal successors

- *Modula*, *Modula-2* (N. Wirth, 1978, 1980)
=> Modules separate compilation and encapsulation, coroutines and monitors, support for low-level programming.

- *Oberon* (N. Wirth, 1986)
=> Simplified modules, dynamic type extension and type tests, streamlined syntax.

Philosophy: "Make it as simple as possible, but not simpler." (A. Einstein)

- *Object*Oberon* (J. Templ, H.P. Moessenboeck, 1989)
=> Experimental Oberon dialect with classes and methods.

- *Oberon-2* (J. Templ, H.P Moessenboeck, N. Wirth, 1991)
=> Oberon with methods on records (== structs), replaces Object Oberon.


* Tree node lookup in Oberon-2


	MODULE Trees;

	IMPORT Texts, Oberon;

	TYPE
		Tree* = POINTER TO Node;  (* star denotes export, not pointer! *)
		Node* = RECORD
			name-: POINTER TO ARRAY OF CHAR;  (* minus denotes read-only export *)
			left, right: Tree
		END;

	PROCEDURE (t: Tree) Lookup* (name: ARRAY OF CHAR): Tree;
		VAR p: Tree;
	BEGIN p := t;
		WHILE (p # NIL) & (name # p.name^) DO
			IF name < p.name^ THEN p := p.left ELSE p := p.right END
		END;
		RETURN p
	END Lookup;

	...


* Analogous code in Go


	package trees

	import ( "fmt"; "runtime" )

	type (
		Tree *Node
		Node struct {
			name        string
			left, right Tree
		}
	)

	func (t *Node) Lookup(name string) Tree {
		var p Tree
		p = t
		for p != nil && name != p.name {
			if name < p.name { p = p.left } else { p = p.right }
		}
		return p
	}

	...


* Observations

- Syntax details are different but structure is the same.
=> C tokens, Oberon structure.

- Same concepts (packages, imports, types, functions/methods, etc).
=> Go concepts further distilled (e.g.; just one loop construct).

Go's heritage is at least as much Oberon as it is C!
(packages, imports, strict memory safety, garbage collection, dynamic type checks, etc.)


* Object orientation and generics

Around 1990: OO and type-system "craze" taking its toll on programming languages.
- C++, Java, others
- complex OO type systems
- complex generic type systems

Proliferation of dynamically typed interpreted languages:
- Erlang, Perl, Python, Lua, Javascript, Ruby, etc.

1990s, 2000s: Backlash.
- Complex OO code is modern analog to unstructured "spaghetti code" of 1970.
- Realization that large programs in dynamically typed languages become unmaintainable.
- Cluttered notation: “Public Static Void” (Rob Pike, OSCON 2010).


* Object orientation in Go: Interfaces

Inspiration: Smalltalk (Alan Kay, Dan Ingalls, Adele Goldberg, 1972-1980)
- Everything is an object.
- Any message can be sent to any object.

Want: Similar power in (mostly) statically typed language without the type-system fuss.
- Notion of interfaces for static typing.
- Usually objects carry type information => restricts object types to "classes".

*Crucial*insight*: Can attach methods to any type if interfaces carry type info rather than objects.

Methods and interfaces are the only additional mechanisms needed for object-oriented programming.


* Concurrency

- Good concurrency support was considered essential from day one.

- Rob Pike’s work on NewSqueak turned out to fit really well into Go.

Origins:

- “Newsqueak: A Language for Communicating with Mice”, Rob Pike, 1994.

- “Communicating Sequential Processes”, CACM, C.A.R. Hoare, 1978.


* Generics

- Single biggest language feature (what exactly is it?) absent in Go.

- Often missed by newcomers to Go.

- Type-system mechanism; unclear if essential language feature.

- Incredibly complex in both semantics and implementation.

- Significant trade-offs: Larger binary, slower binary, or larger source code.

- Not-orthogonal: Affects many other language features as well as how library is written.

- For now: Hold off and keep thinking about it.


* Putting it all together

Luxury to spend two years to hammer out basics (thanks, Google!).

Crucial: *Added*one*feature*at*a*time.*

Initially: Team of three very different people.
- Intensive discussions, emotional.
- Humbling experience.

Having multiple people illuminating each new feature from different angles
made language much stronger.

Later:
- Russ Cox's razor cutting through the crud, making it work well.
- Ian Lance Taylor providing a 2nd implementation (validation of design).
- go/types (now in 1.5!) provides a 3rd frontend (validation of compilers and spec).

Having 3 frontends proved tremendously useful.


* Evolving Go

Original design went through many (syntactic and semantic) transitions:

- Parallel library development ensured we didn't design "into the blue".

- gofmt (for language changes) and gofix (for API changes) for existing code.

Features that came in much later:

- Optional semicolons, optional types for composite literals, optional bounds in slice expressions, recover, etc.

- Semantic clarifications (maps, channel ops, etc.).

- Small backward-compatible adjustments still happening at very low rate.


* The future of Go


* What makes a programming language successful?

- Clear target

- Solid implementation: *language*, *libraries*, and *tools*!

- Market readiness

- Technological breakthrough

- Language features without competitors

- Rarely: Marketing


* How about Go?

- Clear target behind design
- Multi-paradigm (imperative, functional, object-oriented)
- Syntactically light-weight
- Language features without competition: goroutines, interfaces, defer
- Tools without competition: fast compiler, gofmt, go build
- Strong standard libraries
- Solid implementation
- Excellent documentation, online tools (playground, tour)
- But: No corporate marketing to speak of


* Will Go become mainstream?

- Need to cross the chasm from early adopters to early mainstream. Are we there yet?
- Go community must remain unified behind this goal.
- Don't make too many mistakes going forward.

It takes about 10 years for a programming language to become "established".


* Pitfalls

The language is frozen, but these are a form of "language design":

- +build tags and other specialized comments
- special interpretation of import paths and canonical import path comments
- internal packages
- vendoring descriptions

These mechanisms are not part of the language spec and thus may diverge
over time or have different semantics on different platforms.

Need to be watchful of this development.


* Closing thoughts

- In 1960, language experts from America and Europe teamed up to create Algol 60.
- In 1970, the Algol tree split into the C and the Pascal branch.
- ~40 years later, the two branches join again in Go.
- Let's see if Go can enjoy an equally long run as its predecessors!