| // Copyright 2020 The Go Authors. All rights reserved. |
| // Use of this source code is governed by a BSD-style |
| // license that can be found in the LICENSE file. |
| |
| //go:build goexperiment.jsonv2 |
| |
| // Package json implements semantic processing of JSON as specified in RFC 8259. |
| // JSON is a simple data interchange format that can represent |
| // primitive data types such as booleans, strings, and numbers, |
| // in addition to structured data types such as objects and arrays. |
| // |
| // This package (encoding/json/v2) is experimental, |
| // and not subject to the Go 1 compatibility promise. |
| // It only exists when building with the GOEXPERIMENT=jsonv2 environment variable set. |
| // Most users should use [encoding/json]. |
| // |
| // [Marshal] and [Unmarshal] encode and decode Go values |
| // to/from JSON text contained within a []byte. |
| // [MarshalWrite] and [UnmarshalRead] operate on JSON text |
| // by writing to or reading from an [io.Writer] or [io.Reader]. |
| // [MarshalEncode] and [UnmarshalDecode] operate on JSON text |
| // by encoding to or decoding from a [jsontext.Encoder] or [jsontext.Decoder]. |
| // [Options] may be passed to each of the marshal or unmarshal functions |
| // to configure the semantic behavior of marshaling and unmarshaling |
| // (i.e., alter how JSON data is understood as Go data and vice versa). |
| // [jsontext.Options] may also be passed to the marshal or unmarshal functions |
| // to configure the syntactic behavior of encoding or decoding. |
| // |
| // The data types of JSON are mapped to/from the data types of Go based on |
| // the closest logical equivalent between the two type systems. For example, |
| // a JSON boolean corresponds with a Go bool, |
| // a JSON string corresponds with a Go string, |
| // a JSON number corresponds with a Go int, uint or float, |
| // a JSON array corresponds with a Go slice or array, and |
| // a JSON object corresponds with a Go struct or map. |
| // See the documentation on [Marshal] and [Unmarshal] for a comprehensive list |
| // of how the JSON and Go type systems correspond. |
| // |
| // Arbitrary Go types can customize their JSON representation by implementing |
| // [Marshaler], [MarshalerTo], [Unmarshaler], or [UnmarshalerFrom]. |
| // This provides authors of Go types with control over how their types are |
| // serialized as JSON. Alternatively, users can implement functions that match |
| // [MarshalFunc], [MarshalToFunc], [UnmarshalFunc], or [UnmarshalFromFunc] |
| // to specify the JSON representation for arbitrary types. |
| // This provides callers of JSON functionality with control over |
| // how any arbitrary type is serialized as JSON. |
| // |
| // # JSON Representation of Go structs |
| // |
| // A Go struct is naturally represented as a JSON object, |
| // where each Go struct field corresponds with a JSON object member. |
| // When marshaling, all Go struct fields are recursively encoded in depth-first |
| // order as JSON object members except those that are ignored or omitted. |
| // When unmarshaling, JSON object members are recursively decoded |
| // into the corresponding Go struct fields. |
| // Object members that do not match any struct fields, |
| // also known as “unknown members”, are ignored by default or rejected |
| // if [RejectUnknownMembers] is specified. |
| // |
| // The representation of each struct field can be customized in the |
| // "json" struct field tag, where the tag is a comma separated list of options. |
| // As a special case, if the entire tag is `json:"-"`, |
| // then the field is ignored with regard to its JSON representation. |
| // Some options also have equivalent behavior controlled by a caller-specified [Options]. |
| // Field-specified options take precedence over caller-specified options. |
| // |
| // The first option is the JSON object name override for the Go struct field. |
| // If the name is not specified, then the Go struct field name |
| // is used as the JSON object name. JSON names containing commas or quotes, |
| // or names identical to "" or "-", can be specified using |
| // a single-quoted string literal, where the syntax is identical to |
| // the Go grammar for a double-quoted string literal, |
| // but instead uses single quotes as the delimiters. |
| // By default, unmarshaling uses case-sensitive matching to identify |
| // the Go struct field associated with a JSON object name. |
| // |
| // After the name, the following tag options are supported: |
| // |
| // - omitzero: When marshaling, the "omitzero" option specifies that |
| // the struct field should be omitted if the field value is zero |
| // as determined by the "IsZero() bool" method if present, |
| // otherwise based on whether the field is the zero Go value. |
| // This option has no effect when unmarshaling. |
| // |
| // - omitempty: When marshaling, the "omitempty" option specifies that |
| // the struct field should be omitted if the field value would have been |
| // encoded as a JSON null, empty string, empty object, or empty array. |
| // This option has no effect when unmarshaling. |
| // |
| // - string: The "string" option specifies that [StringifyNumbers] |
| // be set when marshaling or unmarshaling a struct field value. |
| // This causes numeric types to be encoded as a JSON number |
| // within a JSON string, and to be decoded from a JSON string |
| // containing the JSON number without any surrounding whitespace. |
| // This extra level of encoding is often necessary since |
| // many JSON parsers cannot precisely represent 64-bit integers. |
| // |
| // - case: When unmarshaling, the "case" option specifies how |
| // JSON object names are matched with the JSON name for Go struct fields. |
| // The option is a key-value pair specified as "case:value" where |
| // the value must either be 'ignore' or 'strict'. |
| // The 'ignore' value specifies that matching is case-insensitive |
| // where dashes and underscores are also ignored. If multiple fields match, |
| // the first declared field in breadth-first order takes precedence. |
| // The 'strict' value specifies that matching is case-sensitive. |
| // This takes precedence over the [MatchCaseInsensitiveNames] option. |
| // |
| // - inline: The "inline" option specifies that |
| // the JSON representable content of this field type is to be promoted |
| // as if they were specified in the parent struct. |
| // It is the JSON equivalent of Go struct embedding. |
| // A Go embedded field is implicitly inlined unless an explicit JSON name |
| // is specified. The inlined field must be a Go struct |
| // (that does not implement any JSON methods), [jsontext.Value], |
| // map[~string]T, or an unnamed pointer to such types. When marshaling, |
| // inlined fields from a pointer type are omitted if it is nil. |
| // Inlined fields of type [jsontext.Value] and map[~string]T are called |
| // “inlined fallbacks” as they can represent all possible |
| // JSON object members not directly handled by the parent struct. |
| // Only one inlined fallback field may be specified in a struct, |
| // while many non-fallback fields may be specified. This option |
| // must not be specified with any other option (including the JSON name). |
| // |
| // - unknown: The "unknown" option is a specialized variant |
| // of the inlined fallback to indicate that this Go struct field |
| // contains any number of unknown JSON object members. The field type must |
| // be a [jsontext.Value], map[~string]T, or an unnamed pointer to such types. |
| // If [DiscardUnknownMembers] is specified when marshaling, |
| // the contents of this field are ignored. |
| // If [RejectUnknownMembers] is specified when unmarshaling, |
| // any unknown object members are rejected regardless of whether |
| // an inlined fallback with the "unknown" option exists. This option |
| // must not be specified with any other option (including the JSON name). |
| // |
| // - format: The "format" option specifies a format flag |
| // used to specialize the formatting of the field value. |
| // The option is a key-value pair specified as "format:value" where |
| // the value must be either a literal consisting of letters and numbers |
| // (e.g., "format:RFC3339") or a single-quoted string literal |
| // (e.g., "format:'2006-01-02'"). The interpretation of the format flag |
| // is determined by the struct field type. |
| // |
| // The "omitzero" and "omitempty" options are mostly semantically identical. |
| // The former is defined in terms of the Go type system, |
| // while the latter in terms of the JSON type system. |
| // Consequently they behave differently in some circumstances. |
| // For example, only a nil slice or map is omitted under "omitzero", while |
| // an empty slice or map is omitted under "omitempty" regardless of nilness. |
| // The "omitzero" option is useful for types with a well-defined zero value |
| // (e.g., [net/netip.Addr]) or have an IsZero method (e.g., [time.Time.IsZero]). |
| // |
| // Every Go struct corresponds to a list of JSON representable fields |
| // which is constructed by performing a breadth-first search over |
| // all struct fields (excluding unexported or ignored fields), |
| // where the search recursively descends into inlined structs. |
| // The set of non-inlined fields in a struct must have unique JSON names. |
| // If multiple fields all have the same JSON name, then the one |
| // at shallowest depth takes precedence and the other fields at deeper depths |
| // are excluded from the list of JSON representable fields. |
| // If multiple fields at the shallowest depth have the same JSON name, |
| // but exactly one is explicitly tagged with a JSON name, |
| // then that field takes precedence and all others are excluded from the list. |
| // This is analogous to Go visibility rules for struct field selection |
| // with embedded struct types. |
| // |
| // Marshaling or unmarshaling a non-empty struct |
| // without any JSON representable fields results in a [SemanticError]. |
| // Unexported fields must not have any `json` tags except for `json:"-"`. |
| // |
| // # Security Considerations |
| // |
| // JSON is frequently used as a data interchange format to communicate |
| // between different systems, possibly implemented in different languages. |
| // For interoperability and security reasons, it is important that |
| // all implementations agree upon the semantic meaning of the data. |
| // |
| // [For example, suppose we have two micro-services.] |
| // The first service is responsible for authenticating a JSON request, |
| // while the second service is responsible for executing the request |
| // (having assumed that the prior service authenticated the request). |
| // If an attacker were able to maliciously craft a JSON request such that |
| // both services believe that the same request is from different users, |
| // it could bypass the authenticator with valid credentials for one user, |
| // but maliciously perform an action on behalf of a different user. |
| // |
| // According to RFC 8259, there unfortunately exist many JSON texts |
| // that are syntactically valid but semantically ambiguous. |
| // For example, the standard does not define how to interpret duplicate |
| // names within an object. |
| // |
| // The v1 [encoding/json] and [encoding/json/v2] packages |
| // interpret some inputs in different ways. In particular: |
| // |
| // - The standard specifies that JSON must be encoded using UTF-8. |
| // By default, v1 replaces invalid bytes of UTF-8 in JSON strings |
| // with the Unicode replacement character, |
| // while v2 rejects inputs with invalid UTF-8. |
| // To change the default, specify the [jsontext.AllowInvalidUTF8] option. |
| // The replacement of invalid UTF-8 is a form of data corruption |
| // that alters the precise meaning of strings. |
| // |
| // - The standard does not specify a particular behavior when |
| // duplicate names are encountered within a JSON object, |
| // which means that different implementations may behave differently. |
| // By default, v1 allows for the presence of duplicate names, |
| // while v2 rejects duplicate names. |
| // To change the default, specify the [jsontext.AllowDuplicateNames] option. |
| // If allowed, object members are processed in the order they are observed, |
| // meaning that later values will replace or be merged into prior values, |
| // depending on the Go value type. |
| // |
| // - The standard defines a JSON object as an unordered collection of name/value pairs. |
| // While ordering can be observed through the underlying [jsontext] API, |
| // both v1 and v2 generally avoid exposing the ordering. |
| // No application should semantically depend on the order of object members. |
| // Allowing duplicate names is a vector through which ordering of members |
| // can accidentally be observed and depended upon. |
| // |
| // - The standard suggests that JSON object names are typically compared |
| // based on equality of the sequence of Unicode code points, |
| // which implies that comparing names is often case-sensitive. |
| // When unmarshaling a JSON object into a Go struct, |
| // by default, v1 uses a (loose) case-insensitive match on the name, |
| // while v2 uses a (strict) case-sensitive match on the name. |
| // To change the default, specify the [MatchCaseInsensitiveNames] option. |
| // The use of case-insensitive matching provides another vector through |
| // which duplicate names can occur. Allowing case-insensitive matching |
| // means that v1 or v2 might interpret JSON objects differently from most |
| // other JSON implementations (which typically use a case-sensitive match). |
| // |
| // - The standard does not specify a particular behavior when |
| // an unknown name in a JSON object is encountered. |
| // When unmarshaling a JSON object into a Go struct, by default |
| // both v1 and v2 ignore unknown names and their corresponding values. |
| // To change the default, specify the [RejectUnknownMembers] option. |
| // |
| // - The standard suggests that implementations may use a float64 |
| // to represent a JSON number. Consequently, large JSON integers |
| // may lose precision when stored as a floating-point type. |
| // Both v1 and v2 correctly preserve precision when marshaling and |
| // unmarshaling a concrete integer type. However, even if v1 and v2 |
| // preserve precision for concrete types, other JSON implementations |
| // may not be able to preserve precision for outputs produced by v1 or v2. |
| // The `string` tag option can be used to specify that an integer type |
| // is to be quoted within a JSON string to avoid loss of precision. |
| // Furthermore, v1 and v2 may still lose precision when unmarshaling |
| // into an any interface value, where unmarshal uses a float64 |
| // by default to represent a JSON number. |
| // To change the default, specify the [WithUnmarshalers] option |
| // with a custom unmarshaler that pre-populates the interface value |
| // with a concrete Go type that can preserve precision. |
| // |
| // RFC 8785 specifies a canonical form for any JSON text, |
| // which explicitly defines specific behaviors that RFC 8259 leaves undefined. |
| // In theory, if a text can successfully [jsontext.Value.Canonicalize] |
| // without changing the semantic meaning of the data, then it provides a |
| // greater degree of confidence that the data is more secure and interoperable. |
| // |
| // The v2 API generally chooses more secure defaults than v1, |
| // but care should still be taken with large integers or unknown members. |
| // |
| // [For example, suppose we have two micro-services.]: https://www.youtube.com/watch?v=avilmOcHKHE&t=1057s |
| package json |
| |
| // requireKeyedLiterals can be embedded in a struct to require keyed literals. |
| type requireKeyedLiterals struct{} |
| |
| // nonComparable can be embedded in a struct to prevent comparability. |
| type nonComparable [0]func() |