| Inside the Go Playground |
| 12 Dec 2013 |
| Tags: playground |
| |
| Andrew Gerrand |
| |
| * Introduction |
| |
| In September 2010 we [[https://blog.golang.org/introducing-go-playground][introduced the Go Playground]], |
| a web service that compiles and executes arbitrary Go code and returns the |
| program output. |
| |
| If you're a Go programmer then you have probably already used the playground |
| by using the [[https://play.golang.org][Go Playground]] directly, |
| taking the [[https://tour.golang.org][Go Tour]], |
| or running [[https://golang.org/pkg/strings/#pkg-examples][executable examples]] |
| from the Go documentation. |
| |
| You may also have used it by clicking one of the "Run" buttons in a slide |
| deck on [[https://talks.golang.org/][talks.golang.org]] or a post on this |
| very blog |
| (such as the [[https://blog.golang.org/strings][recent article on Strings]]). |
| |
| In this article we will take a look at how the playground is implemented |
| and integrated with these services. |
| The implementation involves a variant operating system environment and runtime |
| and our description here assumes you have some familiarity with systems |
| programming using Go. |
| |
| * Overview |
| |
| .image playground/overview.png |
| |
| The playground service has three parts: |
| |
| - A back end that runs on Google's servers. |
| It receives RPC requests, compiles the user program using the gc tool chain, |
| executes the user program, and returns the program output (or compilation |
| errors) as the RPC response. |
| - A front end that runs on [[https://cloud.google.com/appengine/docs/go/][Google App Engine]]. |
| It receives HTTP requests from the client and makes corresponding RPC requests to the back end. |
| It also does some caching. |
| - A JavaScript client that implements the user interface and makes HTTP requests to the front end. |
| |
| * The back end |
| |
| The back end program itself is trivial, so we won't discuss its implementation |
| here. The interesting part is how we safely execute arbitrary user code in a |
| secure environment while still providing core functionality such as time, the |
| network, and the file system. |
| |
| To isolate user programs from Google's infrastructure, the back end runs |
| them under [[https://developers.google.com/native-client/][Native Client]] |
| (or "NaCl"), a technology developed by Google to permit the safe execution of |
| x86 programs inside web browsers. The back end uses a special version of the gc |
| tool chain that generates NaCl executables. |
| |
| (This special tool chain was merged into Go 1.3. |
| To learn more, read the [[https://golang.org/s/go13nacl][design document]].) |
| |
| NaCl limits the amount of CPU and RAM a program may consume, and it prevents |
| programs from accessing the network or file system. |
| This presents a problem, however. |
| Go's concurrency and networking support are among its key strengths, |
| and access to the file system is vital for many programs. |
| To demonstrate concurrency effectively we need time, and to demonstrate |
| networking and the file system we obviously need a network and a file system. |
| |
| Although all these things are supported today, the first version of the |
| playground, launched in 2010, had none of them. |
| The current time was fixed at 10 November 2009, `time.Sleep` had no effect, |
| and most functions of the `os` and `net` packages were stubbed out to |
| return an `EINVALID` error. |
| |
| A year ago we |
| [[https://groups.google.com/d/msg/golang-nuts/JBsCrDEVyVE/30MaQsiQcWoJ][implemented fake time]] |
| in the playground, so that programs that sleep would behave correctly. |
| A more recent update to the playground introduced a fake network stack and a |
| fake file system, making the playground's tool chain similar to a normal |
| Go tool chain. |
| These facilities are described in the following sections. |
| |
| ** Faking time |
| |
| Playground programs are limited in the amount of CPU time and memory they can |
| use, but they are also restricted in how much real time they can use. |
| This is because each running program consumes resources on the back end |
| and any stateful infrastructure between it and the client. |
| Limiting the run time of each playground program makes our service more |
| predictable and defends us against denial of service attacks. |
| |
| But these restrictions become stifling when running code that uses time. |
| The [[https://talks.golang.org/2012/concurrency.slide][Go Concurrency Patterns]] |
| talk demonstrates concurrency with examples that use timing functions like |
| [[https://golang.org/pkg/time/#Sleep][`time.Sleep`]] and |
| [[https://golang.org/pkg/time/#After][`time.After`]]. |
| When run under early versions of the playground, these programs' sleeps would |
| have no effect and their behavior would be strange (and sometimes wrong). |
| |
| By using a clever trick we can make a Go program _think_ that it is sleeping, |
| when really the sleeps take no time at all. |
| To explain the trick we first need to understand how the scheduler manages |
| sleeping goroutines. |
| |
| When a goroutine calls `time.Sleep` (or similar) the scheduler adds a timer to |
| a heap of pending timers and puts the goroutine to sleep. |
| Meanwhile, a special timer goroutine manages that heap. |
| When the timer goroutine starts it tells the scheduler to wake |
| it when the next pending timer is ready to fire and then sleeps. |
| When it wakes up it checks which timers have expired, wakes the appropriate |
| goroutines, and goes back to sleep. |
| |
| The trick is to change the condition that wakes the timer goroutine. |
| Instead of waking it after a specific time period, we modify the scheduler to |
| wait for a deadlock; the state where all goroutines are blocked. |
| |
| The playground version of the runtime maintains its own internal clock. When |
| the modified scheduler detects a deadlock it checks whether any timers are |
| pending. If so, it advances the internal clock to the trigger time of the |
| earliest timer and then wakes the timer goroutine. Execution continues and the |
| program believes that time has passed, when in fact the sleep was nearly |
| instantaneous. |
| |
| These changes to the scheduler can be found in [[https://golang.org/cl/73110043][`proc.c`]] |
| and [[https://golang.org/cl/73110043][`time.goc`]]. |
| |
| Fake time fixes the issue of resource exhaustion on the back end, but what |
| about the program output? It would be odd to see a program that sleeps run to |
| completion correctly without taking any time. |
| |
| The following program prints the current time each second and then exits after |
| three seconds. Try running it. |
| |
| .play -edit playground/time.go /^func main/,$ |
| |
| How does this work? It is a collaboration between the back end, front end, and client. |
| |
| We capture the timing of each write to standard output and standard error and |
| provide it to the client. Then the client can "play back" the writes with the |
| correct timing, so that the output appears just as if the program were running |
| locally. |
| |
| The playground's `runtime` package provides a special |
| [[https://github.com/golang/go/blob/go1.3/src/pkg/runtime/sys_nacl_amd64p32.s#L54][`write` function]] |
| that includes a small "playback header" before each write. |
| The playback header comprises a magic string, the current time, and the |
| length of the write data. A write with a playback header has this structure: |
| |
| 0 0 P B <8-byte time> <4-byte data length> <data> |
| |
| The raw output of the program above looks like this: |
| |
| \x00\x00PB\x11\x74\xef\xed\xe6\xb3\x2a\x00\x00\x00\x00\x1e2009-11-10 23:00:01 +0000 UTC |
| \x00\x00PB\x11\x74\xef\xee\x22\x4d\xf4\x00\x00\x00\x00\x1e2009-11-10 23:00:02 +0000 UTC |
| \x00\x00PB\x11\x74\xef\xee\x5d\xe8\xbe\x00\x00\x00\x00\x1e2009-11-10 23:00:03 +0000 UTC |
| |
| The front end parses this output as a series of events |
| and returns a list of events to the client as a JSON object: |
| |
| { |
| "Errors": "", |
| "Events": [ |
| { |
| "Delay": 1000000000, |
| "Message": "2009-11-10 23:00:01 +0000 UTC\n" |
| }, |
| { |
| "Delay": 1000000000, |
| "Message": "2009-11-10 23:00:02 +0000 UTC\n" |
| }, |
| { |
| "Delay": 1000000000, |
| "Message": "2009-11-10 23:00:03 +0000 UTC\n" |
| } |
| ] |
| } |
| |
| The JavaScript client (running in the user's web browser) then plays back the |
| events using the provided delay intervals. |
| To the user it appears that the program is running in real time. |
| |
| ** Faking the file system |
| |
| Programs built with the Go's NaCl tool chain cannot access the local machine's |
| file system. Instead, the `syscall` package's file-related functions |
| (`Open`, `Read`, `Write`, and so on) operate on an in-memory file system |
| that is implemented by the `syscall` package itself. |
| Since package `syscall` is the interface between the Go code and the operating |
| system kernel, user programs see the file system exactly the same way as they |
| would a real one. |
| |
| The following example program writes data to a file, and then copies |
| its contents to standard output. Try running it. (You can edit it, too!) |
| |
| .play -edit playground/os.go /^func main/,$ |
| |
| When a process starts, the file system is populated with some devices under |
| `/dev` and an empty `/tmp` directory. The program can manipulate the file |
| system as usual, but when the process exits any changes to the file system are |
| lost. |
| |
| There is also a provision to load a zip file into the file system at init time |
| (see [[https://github.com/golang/go/blob/go1.3/src/pkg/syscall/unzip_nacl.go][`unzip_nacl.go`]]). |
| So far we have only used the unzip facility to provide the data files required |
| to run the standard library tests, but we intend to provide playground programs |
| with a set of files that can be used in documentation examples, blog posts, and |
| the Go Tour. |
| |
| The implementation can be found in the |
| [[https://github.com/golang/go/blob/master/src/syscall/fs_nacl.go][`fs_nacl.go`]] and |
| [[https://github.com/golang/go/blob/master/src/syscall/fd_nacl.go][`fd_nacl.go`]] files |
| (which, by virtue of their `_nacl` suffix, are built into package `syscall` only |
| when `GOOS` is set to `nacl`). |
| |
| The file system itself is represented by the |
| [[https://github.com/golang/go/blob/master/src/syscall/fs_nacl.go#L26][`fsys` struct]], |
| of which a global instance (named `fs`) is created during init time. |
| The various file-related functions then operate on `fs` instead of making the |
| actual system call. |
| For instance, here is the [[https://github.com/golang/go/blob/master/src/syscall/fs_nacl.go#L473][`syscall.Open`]] function: |
| |
| func Open(path string, openmode int, perm uint32) (fd int, err error) { |
| fs.mu.Lock() |
| defer fs.mu.Unlock() |
| f, err := fs.open(path, openmode, perm&0777|S_IFREG) |
| if err != nil { |
| return -1, err |
| } |
| return newFD(f), nil |
| } |
| |
| File descriptors are tracked by a global slice named |
| [[https://github.com/golang/go/blob/master/src/syscall/fd_nacl.go#L17][`files`]]. |
| Each file descriptor corresponds to a [[https://github.com/golang/go/blob/master/src/syscall/fd_nacl.go#L23][`file`]] |
| and each `file` provides a value that implements the [[https://github.com/golang/go/blob/master/src/syscall/fd_nacl.go#L30][`fileImpl`]] interface. |
| There are several implementations of the interface: |
| |
| - regular files and devices (such as `/dev/random`) are represented by [[https://github.com/golang/go/blob/master/src/syscall/fs_nacl.go#L58][`fsysFile`]], |
| - standard input, output, and error are instances of [[https://github.com/golang/go/blob/master/src/syscall/fd_nacl.go#L216][`naclFile`]], |
| which uses system calls to interact with the actual files (these are a playground |
| program's only way to interact with the outside world), |
| - network sockets have their own implementation, discussed in the next section. |
| |
| ** Faking the network |
| |
| Like the file system, the playground's network stack is an in-process fake |
| implemented by the `syscall` package. It permits playground projects to use |
| the loopback interface (`127.0.0.1`). Requests to other hosts will fail. |
| |
| For an executable example, run the following program. It listens on a TCP port, |
| waits for an incoming connection, copies the data from that connection to |
| standard output, and exits. In another goroutine, it makes a connection to the |
| listening port, writes a string to the connection, and closes it. |
| |
| .play -edit playground/net.go /^func main/,$ |
| |
| The interface to the network is more complex than the one for files, so the |
| implementation of the fake network is larger and more complex than the fake |
| file system. It must simulate read and write timeouts, different address types |
| and protocols, and so on. |
| |
| The implementation can be found in [[https://github.com/golang/go/blob/master/src/syscall/net_nacl.go][`net_nacl.go`]]. |
| A good place to start reading is [[https://github.com/golang/go/blob/master/src/syscall/net_nacl.go#L461][`netFile`]], |
| the network socket implementation of the `fileImpl` interface. |
| |
| * The front end |
| |
| The playground front end is another simple program (shorter than 100 lines). |
| It receives HTTP requests from the client, makes RPC requests to the back end, |
| and does some caching. |
| |
| The front end serves an HTTP handler at `https://golang.org/compile`. |
| The handler expects a POST request with a `body` field |
| (the Go program to run) and an optional `version` field |
| (for most clients this should be `"2"`). |
| |
| When the front end receives a compilation request it first checks |
| [[https://developers.google.com/appengine/docs/memcache/][memcache]] |
| to see if it has cached the results of a previous compilation of that source. |
| If found, it returns the cached response. |
| The cache prevents popular programs such as those on the |
| [[https://golang.org/][Go home page]] from overloading the back ends. |
| If there is no cached response, the front end makes an RPC request to the back |
| end, stores the response in memcache, parses the playback events, and returns |
| a JSON object to the client as the HTTP response (as described above). |
| |
| * The client |
| |
| The various sites that use the playground each share some common JavaScript |
| code for setting up the user interface (the code and output boxes, the run |
| button, and so on) and communicating with the playground front end. |
| |
| This implementation is in the file |
| [[https://github.com/golang/tools/blob/master/godoc/static/playground.js][`playground.js`]] |
| in the `go.tools` repository, which can be imported from the |
| [[https://godoc.org/golang.org/x/tools/godoc/static][`golang.org/x/tools/godoc/static`]] package. |
| Some of it is clean and some is a bit crufty, as it is the result of |
| consolidating several divergent implementations of the client code. |
| |
| The [[https://github.com/golang/tools/blob/master/godoc/static/playground.js#L227][`playground`]] |
| function takes some HTML elements and turns them into an interactive |
| playground widget. You should use this function if you want to put the |
| playground on your own site (see 'Other clients' below). |
| |
| The [[https://github.com/golang/tools/blob/master/godoc/static/playground.js#L6][`Transport`]] |
| interface (not formally defined, this being JavaScript) |
| abstracts the user interface from the means of talking to the web front end. |
| [[https://github.com/golang/tools/blob/master/godoc/static/playground.js#L43][`HTTPTransport`]] |
| is an implementation of `Transport` that speaks the HTTP-based protocol |
| described earlier. |
| [[https://github.com/golang/tools/blob/master/godoc/static/playground.js#L115][`SocketTransport`]] |
| is another implementation that speaks WebSocket (see 'Playing offline' below). |
| |
| To comply with the [[https://en.wikipedia.org/wiki/Same-origin_policy][same-origin policy]], |
| the various web servers (godoc, for instance) proxy requests to |
| `/compile` through to the playground service at `https://golang.org/compile`. |
| The common [[https://godoc.org/golang.org/x/tools/playground][`golang.org/x/tools/playground`]] |
| package does this proxying. |
| |
| * Playing offline |
| |
| Both the [[https://tour.golang.org][Go Tour]] and the |
| [[https://godoc.org/golang.org/x/tools/present][Present Tool]] can be |
| run offline. This is great for people with limited internet connectivity |
| or presenters at conferences who cannot (and _should_ not) rely on a working |
| internet connection. |
| |
| To run offline, the tools run their own version of the playground back end on |
| the local machine. The back end uses a regular Go tool chain with none of the |
| aforementioned modifications and uses a WebSocket to communicate with the |
| client. |
| |
| The WebSocket back end implementation can be found in the |
| [[https://godoc.org/golang.org/x/tools/playground/socket][`golang.org/x/tools/playground/socket`]] package. |
| The [[https://talks.golang.org/2012/insidepresent.slide#1][Inside Present]] talk discusses this code in detail. |
| |
| * Other clients |
| |
| The playground service is used by more than just the official Go project |
| ([[https://gobyexample.com/][Go by Example]] is one other instance) |
| and we are happy for you to use it on your own site. All we ask is that |
| you [[mailto:golang-dev@googlegroups.com][contact us first]], |
| use a unique user agent in your requests (so we can identify you), and that |
| your service is of benefit to the Go community. |
| |
| * Conclusion |
| |
| From godoc to the tour to this very blog, the playground has become an |
| essential part of our Go documentation story. With the recent additions |
| of the fake file system and network stack we are excited to expand |
| our learning materials to cover those areas. |
| |
| But, ultimately, the playground is just the tip of the iceberg. |
| With Native Client support scheduled for Go 1.3, |
| we look forward to seeing what the community can do with it. |
| |
| _This_article_is_part_12_of_the_ |
| [[https://blog.gopheracademy.com/go-advent-2013][Go Advent Calendar]], |
| _a_series_of_daily_blog_posts_throughout_December_._ |