The Path to Go 1

Rob Pike
Google
http://golang.org/s/plusrob
@rob_pike
http://golang.org

Andrew Gerrand
Google
http://andrewgerrand.com
@go_nuts
http://golang.org


* Introduction

Go is a concurrent open source programming language developed at Google.

Combines native compilation and static types with a lightweight dynamic feel.

Fast, fun, and productive.


* What is Go?

Features:

- Native code generation (compiled)
- Statically typed
- Composition via interfaces
- Memory safe
- Garbage collected
- Native concurrency support
- Excellent standard library
- Great tools


* History


* History

Began as a Google 20% project in late 2007.

Released as an open source project in November 2009.

Go 1 released in March 2012.


* The Go project

Planned from the start as an open source project.

Publicly released under a BSD license.

To date: 28 committers from Google and elsewhere.

More than 200 other people have contributed to the project.

9735 changes committed since release.


* Development process

Mercurial version control system with plugin for code review.

No branches; linear history.

Code review central to the project. All changes reviewed on golang-dev list using Rietveld (codereview.appspot.com).

Custom continuous build system tests across all supported platforms.

Contributions accepted on a "Discuss first, code later" basis.


* Contributions over time

The project moves fast:

.image go1/changes.png

Windows, FreeBSD, OpenBSD, NetBSD, and Plan 9 ports are community-driven.


* Managing the project

* Development cycle

Changes are made continuously.

Things break. Things get fixed.
Some days are better than others.

Continuous builders help, but don't reveal all issues.

We needed to provide some stability for our users.


* Weekly snapshots

Attempt to keep everyone in sync.

- Apply a Mercurial tag to a specific, stable revision.
- Announce to user mailing list with detailed changelog.

Great for early adopters and core developers.


* Problems with weeklies

Contributors work at tip; users sync to weeklies.

Burden on users:

- annoying to update weekly,
- painful to update less often.

Version skew results because users are at different weeklies.

Skew fragments the community and slows adoption.


* Formal release process

March 2011: introduced releases every 1-2 months.

- Pick the most stable of the past few snapshots and tag it.
- Announce with abridged "must read" release notes.

Keeps the community more in sync. Reduces churn.

Popular with users.


* Problem with releases

Easy to make a few small changes once a week...

\...but hard to make many small changes once a month.

Skew still prevalent: adventurers and core devs still use weeklies (or tip!).


* Introducing Gofix

A tool to mechanically update code to accommodate language and library changes.

  gofix prog.go

Announced in May 2011.
Gofix automates updates for backward-incompatible changes.
Eases the burden of staying current.

Release notes now mostly say "run gofix."

Not a sed script. Works on the AST.


* Gofix

Gofix enables sweeping changes without fear of breaking the code base.

Gofix gave us the freedom to make widespread changes that would have been too daunting otherwise.

Can even update foreign code:

"Yesterday I gofixed some third-party packages without even reading their code and without waiting for the authors to update them." - Dmitry Chestnykh


* Versioning issues persist

Gofix is no panacea.

As the root of the dependency graph, a programming language can suffer acutely from version skew.

The fundamental issue remains:
Code you write today may not compile tomorrow.

Some companies unwilling to bet on Go as they saw it as unstable.


* A need for stability

Gofix makes changes very easy, and also makes it easy to experiment.
But it can't do everything.

Priorities: If change is easy, what change is important?

Wanted to make major changes to the language and libraries but, even with gofix, some things are too disruptive without proper planning.

Decision: design and implement a stable version of Go, its libraries, and its tools.


* Go 1

* What is Go 1?

A specification of the language and libraries that will be supported for years.

Available as downloadable binary packages.

An opportunity to:

- fix minor language irritations,
- fix inconsistencies in the standard library,
- focus on bug fixing and cleaning up TODOs,
- design and build a strong build tool set (get rid of make),
- bring Windows support up to par.

Polish and refine, not redesign.


* Planning Go 1

Wrote a detailed proposal document.

Implemented (but not committed) many of the proposed changes.

Core team met for a week to discuss and refine the document (October 2011).

Presented the document to the community for discussion.

Community feedback essential in refining the proposal.


* Preparing Go 1


Create many new issues on the tracker.

Categorize new and existing issues as either "Go 1" or "after Go 1".

Contributors nominate themselves to address specific issues.

Stop developing new features; prioritize stability.


* Rolling it out

Daily number of lines changed in the months leading up to Go 1:

.image go1/go1lines.png


* Gofix and Go 1

The largest Go 1 edits were performed by gofix.

Gofix made it easy to try out a change and refine it incrementally.

Some significant changes were tried and abandoned.


* Gofix work flow

Scripted work flow allowed us to avoid branches and merging nightmares.
#Total automation of edits makes it easy to do large changes without branching.

Process:

  while !satisfied {
    in tree 1:
      refine gofix module in tree 1, build binary
    in tree 2:
      revert to tip (note: tip)
      apply gofix binary from tree 2
      build and test
  }
  commit tree 1
  commit tree 2

Even as other changes are happening, this leads to no branching or skew.

* Sample change

.image go1/errordiff1.png
.image go1/errordiff2.png

* Go 1 release process

Releases paused from r60 (August 2011).
Weeklies continued as normal.

Issued release candidates in the weeks leading up to launch.

Release candidates included binary distributions for the supported operating systems (FreeBSD, Linux, Mac OS X, and Windows).


* What is in Go 1?

* Go 1

Specification of the language.

Specification of the libraries.

Promise of long term compatibility.

Windows as a first class citizen.

New tool chain centered around the `go` tool.

"Phase change" in the way the project runs.


* Language changes

A new `rune` type to represent a Unicode code point.
(Important step in making `int` either 32 or 64 bits; currently just 32.)

A new built-in `error` type to replace `os.Error`.
This affected almost all Go code in existence.

Equality defined on structs.

Cleaned up some clumsy operations.


* API changes

A re-designed `time` package with a clean, simple interface.

Regularization of `strconv`, breaking away from the old C-style API.

Widespread package re-organization. Put things in more appropriate places.

Dozens of lesser changes.


* Time

Old time package was based on the Unix epoch. Limited range, poor features,
no type safety (just integer nanoseconds).

Substantial redesign creates separate Time and Duration types.
Time can represent huge range of times with nanosecond precision.

Duration specifies intervals. Example:

  time.Sleep(2)             // Old API, unsafe: How long is this?
  time.Sleep(2*time.Second) // New API: type-safe, readable.

  fmt.Println(time.Now().Add(1e6*time.Hour))

Also a new flag type!

  $ command -timeout 1m30s

* Re-organization

Rearranged the organically constructed tree to group related things together.
Examples:

  Old         New

  "asn1"      "encoding/asn1"
  "csv"       "encoding/csv"
  "gob"       "encoding/gob"
  "json"      "encoding/json"
  "xml"       "encoding/xml"
  
  "unicode"   "unicode"
  "utf8"      "unicode/utf8"
  "utf16"     "unicode/utf16"

All updated by gofix, of course.

* Demoting immature packages and commands

Parts of the tree deemed unready should not be part of Go 1.

Working but immature packages were moved to sub-repositories of the main Go repository. (They remain installable.)

Unfinished and old packages and tools were left out.


* How to build

Before Go 1, Go programs were built with make, but Makefiles are annoying to write and, for Go, redundant.

From the beginning, a goal of Go was good dependency management.

By design, Go source code contains all the information necessary to build.

Go 1 includes a new "go tool" that eliminates the need for make.

Given a Go tree (including remote dependencies), can build and install directly:

  $ go build file.go

* The go tool

A complete build, test, and install tool for Go programs.

Some realistic examples:

  $ go run hello.go       # Compile-and-go. (Ha!).
  $ go build package      # Build everything in directory (and deps).
  $ go install            # Install everything in dir and (and deps).
  $ go test archive/zip   # Compile and run unit tests for package.

The go tool also wraps `gofmt`, `gofix`, etc.:

  $ go fmt                # Run gofmt on package in current dir.
  $ go fix                # Run gofix on package in current dir.

* The go tool and remote repositories

The go tool automates installation of remote packages.

Packages are addressed by import strings.
Import strings are just source repository URLs.
Go tool downloads and installs all dependencies, transitively.

  $ go get code.google.com/p/myrepo/mypackage

Installs my package, plus any remote dependencies it may have.
And to use the package in Go source:

	import "code.google.com/p/myrepo/mypackage"

The tool can even run gofix as it installs:

  $ go get -fix code.google.com/p/myrepo/mypackage


* Documentation

Complete reworking:

- updated,
- unified,
- added new documents,
- added new (executable) examples,
- redesigned web site.


* API compatibility tool

Scans the entire standard library and checks it against a master list (`go1.txt`).

Helps guarantee compatibility as development continues.

Part of our build process in the lead up to Go 1 (and continues today).


* Today and tomorrow

* What are we working on?

The goal for Go 1 was a stable, productive environment.

Now that Go 1 is out, we are shifting our focus to using Go more than just developing it.

Only through using Go extensively can we learn what might be needed in a future version, say Go 2.

* There is still active development

The design is locked down but work continues.

Stability: bug fixes.

Efficiency:

- code generation,
- garbage collection,
- scheduling,
- hot spots in standard libraries.

Portability: NetBSD, OpenBSD, and Plan 9 ports in progress.

New libraries: HTML parsing and Unicode collation packages, for example.

* Releases after Go 1

Two minor point releases (`go1.0.1` and `go1.0.2`) have been issued to fix bugs.

The next major point release (`go1.1`) is planned for the end of 2012.
It will include:

- code generation improvements to the `gc` compiler,
- performance and accuracy improvements to the garbage collector,
- (possibly) an improved scheduler,
- a slew of bug fixes.

Go 2 is likely years away.


* Learn more

The Go web site has a huge amount of documentation:

.link http://golang.org

Learn Go from a web browser:

.link http://tour.golang.org

"Meet the Go team" panel from Google I/O 2012:

.link http://golang.org/s/meet-the-go-team

Google Code project:

.link http://code.google.com/p/go