doc/articles/laws_of_reflection.tmpl

<!--{
	"Title": "The Laws of Reflection"
}-->
{{donotedit}}

<p>
Reflection in computing is the
ability of a program to examine its own structure, particularly
through types; it's a form of metaprogramming. It's also a great
source of confusion.
</p>

<p>
In this article we attempt to clarify things by explaining how
reflection works in Go. Each language's reflection model is
different (and many languages don't support it at all), but
this article is about Go, so for the rest of this article the word
"reflection" should be taken to mean "reflection in Go".
</p>

<p><b>Types and interfaces</b></p>

<p>
Because reflection builds on the type system, let's start with a
refresher about types in Go.
</p>

<p>
Go is statically typed. Every variable has a static type, that is,
exactly one type known and fixed at compile time: <code>int</code>,
<code>float32</code>, <code>*MyType</code>, <code>[]byte</code>,
and so on. If we declare
</p>

{{code "progs/interface.go" `/type MyInt/` `/STOP/`}}

<p>
then <code>i</code> has type <code>int</code> and <code>j</code>
has type <code>MyInt</code>. The variables <code>i</code> and
<code>j</code> have distinct static types and, although they have
the same underlying type, they cannot be assigned to one another
without a conversion.
</p>

<p>
One important category of type is interface types, which represent
fixed sets of methods. An interface variable can store any concrete
(non-interface) value as long as that value implements the
interface's methods. A well-known pair of examples is
<code>io.Reader</code> and <code>io.Writer</code>, the types
<code>Reader</code> and <code>Writer</code> from the <a href=
"http://golang.org/pkg/io/">io package</a>:
</p>

{{code "progs/interface.go" `/// Reader/` `/STOP/`}}

<p>
Any type that implements a <code>Read</code> (or
<code>Write</code>) method with this signature is said to implement
<code>io.Reader</code> (or <code>io.Writer</code>). For the
purposes of this discussion, that means that a variable of type
<code>io.Reader</code> can hold any value whose type has a
<code>Read</code> method:
</p>

{{code "progs/interface.go" `/func readers/` `/STOP/`}}

<p>
It's important to be clear that whatever concrete value
<code>r</code> may hold, <code>r</code>'s type is always
<code>io.Reader</code>: Go is statically typed and the static type
of <code>r</code> is <code>io.Reader</code>.</p>

<p>
An extremely important example of an interface type is the empty
interface:
</p>

<pre>
interface{}
</pre>

<p>
It represents the empty set of methods and is satisfied by any
value at all, since any value has zero or more methods.
</p>

<p>
Some people say that Go's interfaces are dynamically typed, but
that is misleading. They are statically typed: a variable of
interface type always has the same static type, and even though at
run time the value stored in the interface variable may change
type, that value will always satisfy the interface.
</p>

<p>
We need to be precise about all this because reflection and
interfaces are closely related.
</p>

<p><b>The representation of an interface</b></p>

<p>
Russ Cox has written a <a href=
"http://research.swtch.com/2009/12/go-data-structures-interfaces.html">
detailed blog post</a> about the representation of interface values
in Go. It's not necessary to repeat the full story here, but a
simplified summary is in order.
</p>

<p>
A variable of interface type stores a pair: the concrete value
assigned to the variable, and that value's type descriptor.
To be more precise, the value is the underlying concrete data item
that implements the interface and the type describes the full type
of that item. For instance, after
</p>

{{code "progs/interface.go" `/func typeAssertions/` `/STOP/`}}

<p>
<code>r</code> contains, schematically, the (value, type) pair,
(<code>tty</code>, <code>*os.File</code>). Notice that the type
<code>*os.File</code> implements methods other than
<code>Read</code>; even though the interface value provides access
only to the <code>Read</code> method, the value inside carries all
the type information about that value. That's why we can do things
like this:
</p>

{{code "progs/interface.go" `/var w io.Writer/` `/STOP/`}}

<p>
The expression in this assignment is a type assertion; what it
asserts is that the item inside <code>r</code> also implements
<code>io.Writer</code>, and so we can assign it to <code>w</code>.
After the assignment, <code>w</code> will contain the pair
(<code>tty</code>, <code>*os.File</code>). That's the same pair as
was held in <code>r</code>. The static type of the interface
determines what methods may be invoked with an interface variable,
even though the concrete value inside may have a larger set of
methods.
</p>

<p>
Continuing, we can do this:
</p>

{{code "progs/interface.go" `/var empty interface{}/` `/STOP/`}}

<p>
and our empty interface value <code>e</code> will again contain
that same pair, (<code>tty</code>, <code>*os.File</code>). That's
handy: an empty interface can hold any value and contains all the
information we could ever need about that value.
</p>

<p>
(We don't need a type assertion here because it's known statically
that <code>w</code> satisfies the empty interface. In the example
where we moved a value from a <code>Reader</code> to a
<code>Writer</code>, we needed to be explicit and use a type
assertion because <code>Writer</code>'s methods are not a
subset of <code>Reader</code>'s.)
</p>

<p>
One important detail is that the pair inside an interface always
has the form (value, concrete type) and cannot have the form
(value, interface type). Interfaces do not hold interface
values.
</p>

<p>
Now we're ready to reflect.
</p>

<p><b>The first law of reflection</b></p>

<p><b>1. Reflection goes from interface value to reflection object.</b></p>

<p>
At the basic level, reflection is just a mechanism to examine the
type and value pair stored inside an interface variable. To get
started, there are two types we need to know about in
<a href="http://golang.org/pkg/reflect">package reflect</a>:
<a href="http://golang.org/pkg/reflect/#Type">Type</a>and
<a href="http://golang.org/pkg/reflect/#Value">Value</a>. Those two types
give access to the contents of an interface variable, and two
simple functions, called <code>reflect.TypeOf</code> and
<code>reflect.ValueOf</code>, retrieve <code>reflect.Type</code>
and <code>reflect.Value</code> pieces out of an interface value.
(Also, from the <code>reflect.Value</code> it's easy to get
to the <code>reflect.Type</code>, but let's keep the
<code>Value</code> and <code>Type</code> concepts separate for
now.)
</p>

<p>
Let's start with <code>TypeOf</code>:
</p>

{{code "progs/interface2.go" `/package main/` `/STOP main/`}}

<p>
This program prints
</p>

<pre>
type: float64
</pre>

<p>
You might be wondering where the interface is here, since the
program looks like it's passing the <code>float64</code>
variable <code>x</code>, not an interface value, to
<code>reflect.TypeOf</code>. But it's there; as <a href=
"http://golang.org/pkg/reflect/#Type.TypeOf">godoc reports</a>, the
signature of <code>reflect.TypeOf</code> includes an empty
interface:
</p>

<pre>
// TypeOf returns the reflection Type of the value in the interface{}.
func TypeOf(i interface{}) Type
</pre>

<p>
When we call <code>reflect.TypeOf(x)</code>, <code>x</code> is
first stored in an empty interface, which is then passed as the
argument; <code>reflect.TypeOf</code> unpacks that empty interface
to recover the type information.
</p>

<p>
The <code>reflect.ValueOf</code> function, of course, recovers the
value (from here on we'll elide the boilerplate and focus just on
the executable code):
</p>

{{code "progs/interface2.go" `/var x/` `/STOP/`}}

<p>
prints
</p>

<pre>
value: &lt;float64 Value&gt;
</pre>

<p>
Both <code>reflect.Type</code> and <code>reflect.Value</code> have
lots of methods to let us examine and manipulate them. One
important example is that <code>Value</code> has a
<code>Type</code> method that returns the <code>Type</code> of a
<code>reflect.Value</code>. Another is that both <code>Type</code>
and <code>Value</code> have a <code>Kind</code> method that returns
a constant indicating what sort of item is stored:
<code>Uint</code>, <code>Float64</code>, <code>Slice</code>, and so
on. Also methods on <code>Value</code> with names like
<code>Int</code> and <code>Float</code> let us grab values (as
<code>int64</code> and <code>float64</code>) stored inside:
</p>

{{code "progs/interface2.go" `/START f1/` `/STOP/`}}

<p>
prints
</p>

<pre>
type: float64
kind is float64: true
value: 3.4
</pre>

<p>
There are also methods like <code>SetInt</code> and
<code>SetFloat</code> but to use them we need to understand
settability, the subject of the third law of reflection, discussed
below.
</p>

<p>
The reflection library has a couple of properties worth singling
out. First, to keep the API simple, the "getter" and "setter"
methods of <code>Value</code> operate on the largest type that can
hold the value: <code>int64</code> for all the signed integers, for
instance. That is, the <code>Int</code> method of
<code>Value</code> returns an <code>int64</code> and the
<code>SetInt</code> value takes an <code>int64</code>; it may be
necessary to convert to the actual type involved:
</p>

{{code "progs/interface2.go" `/START f2/` `/STOP/`}}

<p>
The second property is that the <code>Kind</code> of a reflection
object describes the underlying type, not the static type. If a
reflection object contains a value of a user-defined integer type,
as in
</p>

{{code "progs/interface2.go" `/START f3/` `/START/`}}

<p>
the <code>Kind</code> of <code>v</code> is still
<code>reflect.Int</code>, even though the static type of
<code>x</code> is <code>MyInt</code>, not <code>int</code>. In
other words, the <code>Kind</code> cannot discriminate an int from
a <code>MyInt</code> even though the <code>Type</code> can.
</p>

<p><b>The second law of reflection</b></p>

<p><b>2. Reflection goes from reflection object to interface
value.</b></p>

<p>
Like physical reflection, reflection in Go generates its own
inverse.
</p>

<p>
Given a <code>reflect.Value</code> we can recover an interface
value using the <code>Interface</code> method; in effect the method
packs the type and value information back into an interface
representation and returns the result:
</p>

<pre>
// Interface returns v's value as an interface{}.
func (v Value) Interface() interface{}
</pre>

<p>
As a consequence we can say
</p>

{{code "progs/interface2.go" `/START f3b/` `/START/`}}

<p>
to print the <code>float64</code> value represented by the
reflection object <code>v</code>.
</p>

<p>
We can do even better, though. The arguments to
<code>fmt.Println</code>, <code>fmt.Printf</code> and so on are all
passed as empty interface values, which are then unpacked by the
<code>fmt</code> package internally just as we have been doing in
the previous examples. Therefore all it takes to print the contents
of a <code>reflect.Value</code> correctly is to pass the result of
the <code>Interface</code> method to the formatted print
routine:
</p>

{{code "progs/interface2.go" `/START f3c/` `/START/`}}

<p>
(Why not <code>fmt.Println(v)</code>? Because <code>v</code> is a
<code>reflect.Value</code>; we want the concrete value it holds.)
Since our value is a <code>float64</code>, we can even use a
floating-point format if we want:
</p>

{{code "progs/interface2.go" `/START f3d/` `/STOP/`}}

<p>
and get in this case
</p>

<pre>
3.4e+00
</pre>

<p>
Again, there's no need to type-assert the result of
<code>v.Interface()</code> to <code>float64</code>; the empty
interface value has the concrete value's type information inside
and <code>Printf</code> will recover it.
</p>

<p>
In short, the <code>Interface</code> method is the inverse of the
<code>ValueOf</code> function, except that its result is always of
static type <code>interface{}</code>.
</p>

<p>
Reiterating: Reflection goes from interface values to reflection
objects and back again.
</p>

<p><b>The third law of reflection</b></p>

<p><b>3. To modify a reflection object, the value must be settable.</b></p>

<p>
The third law is the most subtle and confusing, but it's easy
enough to understand if we start from first principles.
</p>

<p>
Here is some code that does not work, but is worth studying.
</p>

{{code "progs/interface2.go" `/START f4/` `/STOP/`}}

<p>
If you run this code, it will panic with the cryptic message
</p>

<pre>
panic: reflect.Value.SetFloat using unaddressable value
</pre>

<p>
The problem is not that the value <code>7.1</code> is not
addressable; it's that <code>v</code> is not settable. Settability
is a property of a reflection <code>Value</code>, and not all
reflection <code>Values</code> have it.
</p>

<p>
The <code>CanSet</code> method of <code>Value</code> reports the
settability of a <code>Value</code>; in our case,
</p>

{{code "progs/interface2.go" `/START f5/` `/STOP/`}}

<p>
prints
</p>

<pre>
settability of v: false
</pre>

<p>
It is an error to call a <code>Set</code> method on an non-settable
<code>Value</code>. But what is settability?
</p>

<p>
Settability is a bit like addressability, but stricter. It's the
property that a reflection object can modify the actual storage
that was used to create the reflection object. Settability is
determined by whether the reflection object holds the original
item. When we say
</p>

{{code "progs/interface2.go" `/START f6/` `/START/`}}

<p>
we pass a <em>copy</em> of <code>x</code> to
<code>reflect.ValueOf</code>, so the interface value created as the
argument to <code>reflect.ValueOf</code> is a <em>copy</em> of
<code>x</code>, not <code>x</code> itself. Thus, if the
statement
</p>

{{code "progs/interface2.go" `/START f6b/` `/STOP/`}}

<p>
were allowed to succeed, it would not update <code>x</code>, even
though <code>v</code> looks like it was created from
<code>x</code>. Instead, it would update the copy of <code>x</code>
stored inside the reflection value and <code>x</code> itself would
be unaffected. That would be confusing and useless, so it is
illegal, and settability is the property used to avoid this
issue.
</p>

<p>
If this seems bizarre, it's not. It's actually a familiar situation
in unusual garb. Think of passing <code>x</code> to a
function:
</p>

<pre>
f(x)
</pre>

<p>
We would not expect <code>f</code> to be able to modify
<code>x</code> because we passed a copy of <code>x</code>'s value,
not <code>x</code> itself. If we want <code>f</code> to modify
<code>x</code> directly we must pass our function the address of
<code>x</code> (that is, a pointer to <code>x</code>):</p>

<p>
<code>f(&amp;x)</code>
</p>

<p>
This is straightforward and familiar, and reflection works the same
way. If we want to modify <code>x</code> by reflection, we must
give the reflection library a pointer to the value we want to
modify.
</p>

<p>
Let's do that. First we initialize <code>x</code> as usual
and then create a reflection value that points to it, called
<code>p</code>.
</p>

{{code "progs/interface2.go" `/START f7/` `/START/`}}

<p>
The output so far is
</p>

<pre>
type of p: *float64
settability of p: false
</pre>

<p>
The reflection object <code>p</code> isn't settable, but it's not
<code>p</code> we want to set, it's (in effect) <code>*p</code>. To
get to what <code>p</code> points to, we call the <code>Elem</code>
method of <code>Value</code>, which indirects through the pointer,
and save the result in a reflection <code>Value</code> called
<code>v</code>:
</p>

{{code "progs/interface2.go" `/START f7b/` `/START/`}}

<p>
Now <code>v</code> is a settable reflection object, as the output
demonstrates,
</p>

<pre>
settability of v: true
</pre>

<p>
and since it represents <code>x</code>, we are finally able to use
<code>v.SetFloat</code> to modify the value of
<code>x</code>:
</p>

{{code "progs/interface2.go" `/START f7c/` `/STOP/`}}

<p>
The output, as expected, is
</p>

<pre>
7.1
7.1
</pre>

<p>
Reflection can be hard to understand but it's doing exactly what
the language does, albeit through reflection <code>Types</code> and
<code>Values</code> that can disguise what's going on. Just keep in
mind that reflection Values need the address of something in order
to modify what they represent.
</p>

<p><b>Structs</b></p>

<p>
In our previous example <code>v</code> wasn't a pointer itself, it
was just derived from one. A common way for this situation to arise
is when using reflection to modify the fields of a structure. As
long as we have the address of the structure, we can modify its
fields.
</p>

<p>
Here's a simple example that analyzes a struct value,
<code>t</code>. We create the reflection object with the address of
the struct because we'll want to modify it later. Then we set
<code>typeOfT</code> to its type and iterate over the fields using
straightforward method calls (see 
<a href="http://golang.org/pkg/reflect/">package reflect</a> for details).
Note that we extract the names of the fields from the struct type,
but the fields themselves are regular <code>reflect.Value</code>
objects.
</p>

{{code "progs/interface2.go" `/START f8/` `/STOP/`}}

<p>
The output of this program is
</p>

<pre>
0: A int = 23
1: B string = skidoo
</pre>

<p>
There's one more point about settability introduced in
passing here: the field names of <code>T</code> are upper case
(exported) because only exported fields of a struct are
settable.
</p>

<p>
Because <code>s</code> contains a settable reflection object, we
can modify the fields of the structure.
</p>

{{code "progs/interface2.go" `/START f8b/` `/STOP/`}}

<p>
And here's the result:
</p>

<pre>
t is now {77 Sunset Strip}
</pre>

<p>
If we modified the program so that <code>s</code> was created from
<code>t</code>, not <code>&amp;t</code>, the calls to
<code>SetInt</code> and <code>SetString</code> would fail as the
fields of <code>t</code> would not be settable.
</p>

<p><b>Conclusion</b></p>

<p>
Here again are the laws of reflection:
</p>

<ol>
<li>Reflection goes from interface value to reflection
object.</li>
<li>Reflection goes from reflection object to interface
value.</li>
<li>To modify a reflection object, the value must be settable.</li>
</ol>

<p>
Once you understand these laws reflection in Go becomes much easier
to use, although it remains subtle. It's a powerful tool that
should be used with care and avoided unless strictly
necessary.
</p>

<p>
There's plenty more to reflection that we haven't covered &mdash;
sending and receiving on channels, allocating memory, using slices
and maps, calling methods and functions &mdash; but this post is
long enough. We'll cover some of those topics in a later
article.
</p>