This is a very incomplete and probably out-of-date guide to programming in the Go runtime and how it differs from writing normal Go.
In general, the runtime tries to use regular heap allocation. However, in some cases the runtime must allocate objects outside of the garbage collected heap, in unmanaged memory. This is necessary if the objects are part of the memory manager itself or if they must be allocated in situations where the caller may not have a P.
There are three mechanisms for allocating unmanaged memory:
sysAlloc obtains memory directly from the OS. This comes in whole multiples of the system page size, but it can be freed with sysFree.
persistentalloc combines multiple smaller allocations into a single sysAlloc to avoid fragmentation. However, there is no way to free persistentalloced objects (hence the name).
fixalloc is a SLAB-style allocator that allocates objects of a fixed size. fixalloced objects can be freed, but this memory can only be reused by the same fixalloc pool, so it can only be reused for objects of the same type.
In general, types that are allocated using any of these should be marked //go:notinheap (see below).
Objects that are allocated in unmanaged memory must not contain heap pointers unless the following rules are also obeyed:
Any pointers from unmanaged memory to the heap must be added as explicit garbage collection roots in runtime.markroot.
If the memory is reused, the heap pointers must be zero-initialized before they become visible as GC roots. Otherwise, the GC may observe stale heap pointers. See “Zero-initialization versus zeroing”.
There are two types of zeroing in the runtime, depending on whether the memory is already initialized to a type-safe state.
If memory is not in a type-safe state, meaning it potentially contains “garbage” because it was just allocated and it is being initialized for first use, then it must be zero-initialized using memclrNoHeapPointers or non-pointer writes. This does not perform write barriers.
If memory is already in a type-safe state and is simply being set to the zero value, this must be done using regular writes, typedmemclr, or memclrHasPointers. This performs write barriers.
In addition to the “//go:” directives documented in “go doc compile”, the compiler supports additional directives only in the runtime.
go:systemstack indicates that a function must run on the system stack. This is checked dynamically by a special function prologue.
go:nowritebarrier directs the compiler to emit an error if the following function contains any write barriers. (It does not suppress the generation of write barriers; it is simply an assertion.)
Usually you want go:nowritebarrierrec. go:nowritebarrier is primarily useful in situations where it's “nice” not to have write barriers, but not required for correctness.
go:nowritebarrierrec directs the compiler to emit an error if the following function or any function it calls recursively, up to a go:yeswritebarrierrec, contains a write barrier.
Logically, the compiler floods the call graph starting from each go:nowritebarrierrec function and produces an error if it encounters a function containing a write barrier. This flood stops at go:yeswritebarrierrec functions.
go:nowritebarrierrec is used in the implementation of the write barrier to prevent infinite loops.
Both directives are used in the scheduler. The write barrier requires an active P (getg().m.p != nil) and scheduler code often runs without an active P. In this case, go:nowritebarrierrec is used on functions that release the P or may run without a P and go:yeswritebarrierrec is used when code re-acquires an active P. Since these are function-level annotations, code that releases or acquires a P may need to be split across two functions.
go:notinheap applies to type declarations. It indicates that a type must never be heap allocated. Specifically, pointers to this type must always fail the runtime.inheap check. The type may be used for global variables, for stack variables, or for objects in unmanaged memory (e.g., allocated with sysAlloc, persistentalloc, or fixalloc). Specifically:
new(T), make([]T), append([]T, ...) and implicit heap allocation of T are disallowed. (Though implicit allocations are disallowed in the runtime anyway.)
A pointer to a regular type (other than unsafe.Pointer) cannot be converted to a pointer to a go:notinheap type, even if they have the same underlying type.
Any type that contains a go:notinheap type is itself go:notinheap. Structs and arrays are go:notinheap if their elements are. Maps and channels of go:notinheap types are disallowed. To keep things explicit, any type declaration where the type is implicitly go:notinheap must be explicitly marked go:notinheap as well.
Write barriers on pointers to go:notinheap types can be omitted.
The last point is the real benefit of go:notinheap. The runtime uses it for low-level internal structures to avoid memory barriers in the scheduler and the memory allocator where they are illegal or simply inefficient. This mechanism is reasonably safe and does not compromise the readability of the runtime.