src/encoding/json/v2/doc.go

// Copyright 2020 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.

//go:build goexperiment.jsonv2

// Package json implements semantic processing of JSON as specified in RFC 8259.
// JSON is a simple data interchange format that can represent
// primitive data types such as booleans, strings, and numbers,
// in addition to structured data types such as objects and arrays.
//
// See the [Working with JSON] tutorial for an introduction to this package.
//
// [Marshal] and [Unmarshal] encode and decode Go values
// to/from JSON text contained within a []byte.
// [MarshalWrite] and [UnmarshalRead] operate on JSON text
// by writing to or reading from an [io.Writer] or [io.Reader].
// [MarshalEncode] and [UnmarshalDecode] operate on JSON text
// by encoding to or decoding from a [jsontext.Encoder] or [jsontext.Decoder].
// [Options] may be passed to each of the marshal or unmarshal functions
// to configure the semantic behavior of marshaling and unmarshaling
// (i.e., alter how JSON data is understood as Go data and vice versa).
// [jsontext.Options] may also be passed to the marshal or unmarshal functions
// to configure the syntactic behavior of encoding or decoding.
//
// The data types of JSON are mapped to/from the data types of Go based on
// the closest logical equivalent between the two type systems. For example,
// a JSON boolean corresponds with a Go bool,
// a JSON string corresponds with a Go string,
// a JSON number corresponds with a Go int, uint or float,
// a JSON array corresponds with a Go slice or array, and
// a JSON object corresponds with a Go struct or map.
// See the documentation on [Marshal] and [Unmarshal] for a comprehensive list
// of how the JSON and Go type systems correspond.
//
// Arbitrary Go types can customize their JSON representation by implementing
// [Marshaler], [MarshalerTo], [Unmarshaler], or [UnmarshalerFrom].
// This provides authors of Go types with control over how their types are
// serialized as JSON. Alternatively, users can implement functions that match
// [MarshalFunc], [MarshalToFunc], [UnmarshalFunc], or [UnmarshalFromFunc]
// to specify the JSON representation for arbitrary types.
// This provides callers of JSON functionality with control over
// how any arbitrary type is serialized as JSON.
//
// # JSON Representation of Go structs
//
// A Go struct is naturally represented as a JSON object,
// where each Go struct field corresponds with a JSON object member.
// When marshaling, all Go struct fields are recursively encoded in depth-first
// order as JSON object members except those that are ignored or omitted.
// When unmarshaling, JSON object members are recursively decoded
// into the corresponding Go struct fields.
// Object members that do not match any struct fields,
// also known as “unknown members”, are ignored by default or rejected
// if [RejectUnknownMembers] is specified.
//
// The representation of each struct field can be customized in the
// "json" struct field tag, where the tag is a comma-separated list of options.
// As a special case, if the entire tag is `json:"-"`,
// then the field is ignored with regard to its JSON representation.
// Some options also have equivalent behavior controlled by a caller-specified [Options].
// Field-specified options take precedence over caller-specified options.
//
// The first option is the JSON object name override for the Go struct field.
// If the name is not specified, then the Go struct field name
// is used as the JSON object name.
// By default, unmarshaling uses case-sensitive matching to identify
// the Go struct field associated with a JSON object name.
//
// After the name, the following tag options are supported:
//
//   - omitzero: When marshaling, the "omitzero" option specifies that
//     the struct field should be omitted if the field value is zero
//     as determined by the "IsZero() bool" method if present,
//     otherwise based on whether the field is the zero Go value.
//     This option has no effect when unmarshaling.
//
//   - omitempty: When marshaling, the "omitempty" option specifies that
//     the struct field should be omitted if the field value would have been
//     encoded as a JSON null, empty string, empty object, or empty array.
//     This option has no effect when unmarshaling.
//
//   - string: The "string" option specifies that [StringifyNumbers] be set
//     when marshaling or unmarshaling a struct field value.
//     This causes types that would normally be encoded as a JSON number
//     to instead be encoded as a JSON number quoted within a JSON string,
//     and to be decoded from a JSON string containing the JSON number
//     without any surrounding whitespace.
//     The "string" option only applies to the top-level of the Go struct field
//     value. It is an error to apply this option to any type that does not
//     encode as a JSON number.
//     Note that composite types such as arrays, slices, structs, and maps do
//     not encode as a JSON number, so applying this option will cause an error
//     rather than affecting JSON numbers within such types.
//     This extra level of encoding is often necessary since many JSON parsers
//     cannot precisely represent 64-bit integers.
//
//   - case: When unmarshaling, the "case" option specifies how
//     JSON object names are matched with the JSON name for Go struct fields.
//     The option is a key-value pair specified as "case:value" where
//     the value must either be 'ignore' or 'strict'.
//     The 'ignore' value specifies that matching is case-insensitive,
//     and also ignores dashes and underscores. If multiple fields match,
//     then the field with an exact name match is selected, otherwise an error
//     is reported because the choice of field to unmarshal into is ambiguous.
//     The 'strict' value specifies that matching is case-sensitive.
//     This takes precedence over the [MatchCaseInsensitiveNames] option.
//
//   - embed: The "embed" option specifies that
//     the JSON representable content of this field type is to be promoted
//     as if it were specified in the parent struct.
//     It is the JSON equivalent of Go struct embedding.
//     A Go embedded field is implicitly JSON embedded unless
//     an explicit JSON name is specified. The embedded field must be a Go struct
//     (that does not implement any JSON methods), [jsontext.Value],
//     map[~string]T, or an unnamed pointer to such types. When marshaling,
//     embedded fields from a pointer type are omitted if it is nil.
//     Embedded fields of type [jsontext.Value] and map[~string]T are called
//     “embedded fallbacks” as they can represent all possible
//     JSON object members not directly handled by the parent struct.
//     Only one embedded fallback field may be specified in a struct,
//     while many non-fallback fields may be specified. This option
//     must not be specified with any other option (including the JSON name).
//
// The "omitzero" and "omitempty" options behave similarly.
// The former is defined in terms of the Go type system,
// while the latter in terms of the JSON type system.
// Consequently they behave differently in some circumstances.
// For example, only a nil slice or map is omitted under "omitzero", while
// an empty slice or map is omitted under "omitempty" regardless of nilness.
// The "omitzero" option is useful for types with a well-defined zero value
// (e.g., [net/netip.Addr]) or have an IsZero method (e.g., [time.Time.IsZero]).
//
// Every Go struct corresponds to a list of JSON-representable fields
// which is constructed by performing a breadth-first search over
// all struct fields (excluding unexported or ignored fields),
// where the search recursively descends into embedded structs.
// The set of non-embedded fields in a struct must have unique JSON names.
// If multiple fields all have the same JSON name, then the one
// at shallowest depth takes precedence and the other fields at deeper depths
// are excluded from the list of JSON-representable fields.
// If multiple fields at the shallowest depth have the same JSON name,
// but exactly one is explicitly tagged with a JSON name,
// then that field takes precedence and all others are excluded from the list.
// This is analogous to Go visibility rules for struct field selection
// with embedded struct types.
//
// Marshaling or unmarshaling a non-empty struct
// without any JSON-representable fields results in a [SemanticError],
// unless the Go struct has a field with an explicit `json` tag,
// which signals that the type has a valid JSON representation (even if empty).
// Unexported fields must not have any `json` tags except for `json:"-"`.
//
// # Security Considerations
//
// JSON is frequently used as a data interchange format to communicate
// between different systems, possibly implemented in different languages.
// For interoperability and security reasons, it is important that
// all implementations agree upon the semantic meaning of the data.
//
// [For example, suppose we have two micro-services.]
// The first service is responsible for authenticating a JSON request,
// while the second service is responsible for executing the request,
// assuming that it was authenticated.
// If an attacker were able to maliciously craft a JSON request such that
// both services believe that the same request is from different users,
// it could bypass the authenticator with valid credentials for one user,
// but maliciously perform an action on behalf of a different user.
//
// According to RFC 8259, there unfortunately exist many JSON texts
// that are syntactically valid but semantically ambiguous.
// For example, the standard does not define how to interpret duplicate
// names within an object.
//
// The v1 [encoding/json] and [encoding/json/v2] packages
// interpret some inputs in different ways. In particular:
//
//   - The standard specifies that JSON must be encoded using UTF-8.
//     By default, v1 replaces invalid bytes of UTF-8 in JSON strings
//     with the Unicode replacement character,
//     while v2 rejects inputs with invalid UTF-8.
//     To change the default, specify the [jsontext.AllowInvalidUTF8] option.
//     The replacement of invalid UTF-8 is a form of data corruption
//     that alters the precise meaning of strings.
//
//   - The standard does not specify a particular behavior when
//     duplicate names are encountered within a JSON object,
//     which means that different implementations may behave differently.
//     By default, v1 allows for the presence of duplicate names,
//     while v2 rejects duplicate names.
//     To change the default, specify the [jsontext.AllowDuplicateNames] option.
//     If allowed, object members are processed in the order they are observed,
//     meaning that later values will replace or be merged into prior values,
//     depending on the Go value type.
//
//   - The standard defines a JSON object as an unordered collection of name/value pairs.
//     While ordering can be observed through the underlying [jsontext] API,
//     both v1 and v2 generally avoid exposing the ordering.
//     No application should semantically depend on the order of object members.
//     Allowing duplicate names is a vector through which ordering of members
//     can accidentally be observed and depended upon.
//
//   - The standard suggests that JSON object names are typically compared
//     based on equality of the sequence of Unicode code points,
//     which implies that comparing names is often case-sensitive.
//     When unmarshaling a JSON object into a Go struct,
//     by default, v1 uses a (loose) case-insensitive match on the name,
//     while v2 uses a (strict) case-sensitive match on the name.
//     To change the default, specify the [MatchCaseInsensitiveNames] option.
//     The use of case-insensitive matching provides another vector through
//     which duplicate names can occur. Allowing case-insensitive matching
//     means that v1 or v2 might interpret JSON objects differently from most
//     other JSON implementations (which typically use a case-sensitive match).
//
//   - The standard does not specify a particular behavior when
//     an unknown name in a JSON object is encountered.
//     When unmarshaling a JSON object into a Go struct, by default
//     both v1 and v2 ignore unknown names and their corresponding values.
//     To change the default, specify the [RejectUnknownMembers] option.
//
//   - The standard suggests that implementations may use a float64
//     to represent a JSON number. Consequently, large JSON integers
//     may lose precision when stored as a floating-point type.
//     Both v1 and v2 correctly preserve precision when marshaling and
//     unmarshaling a concrete integer type. However, even if v1 and v2
//     preserve precision for concrete types, other JSON implementations
//     may not be able to preserve precision for outputs produced by v1 or v2.
//     The `string` tag option can be used to specify that an integer type
//     is to be quoted within a JSON string to avoid loss of precision.
//     Furthermore, v1 and v2 may still lose precision when unmarshaling
//     into an any interface value, where unmarshal uses a float64
//     by default to represent a JSON number.
//     To change the default, specify the [WithUnmarshalers] option
//     with a custom unmarshaler that pre-populates the interface value
//     with a concrete Go type that can preserve precision.
//
// RFC 8785 specifies a canonical form for any JSON text,
// which explicitly defines specific behaviors that RFC 8259 leaves undefined.
// In theory, if a text can successfully [jsontext.Value.Canonicalize]
// without changing the semantic meaning of the data, then it provides a
// greater degree of confidence that the data is more secure and interoperable.
//
// The v2 API generally chooses more secure defaults than v1,
// but care should still be taken with large integers or unknown members.
//
// [Working with JSON]: https://go.dev/doc/tutorial/json
// [For example, suppose we have two micro-services.]: https://www.youtube.com/watch?v=avilmOcHKHE&t=1057s
package json

// requireKeyedLiterals can be embedded in a struct to require keyed literals.
type requireKeyedLiterals struct{}

// nonComparable can be embedded in a struct to prevent comparability.
type nonComparable [0]func()