encoding/internal/identifier/identifier.go

// Copyright 2015 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.

//go:generate go run gen.go

// Package identifier defines the contract between implementations of Encoding
// and Index by defining identifiers that uniquely identify standardized coded
// character sets (CCS) and character encoding schemes (CES), which we will
// together refer to as encodings, for which Encoding implementations provide
// converters to and from UTF-8. This package is typically only of concern to
// implementers of Indexes and Encodings.
//
// One part of the identifier is the MIB code, which is defined by IANA and
// uniquely identifies a CCS or CES. Each code is associated with data that
// references authorities, official documentation as well as aliases and MIME
// names.
//
// Not all CESs are covered by the IANA registry. The "other" string that is
// returned by ID can be used to identify other character sets or versions of
// existing ones.
//
// It is recommended that each package that provides a set of Encodings provide
// the All and Common variables to reference all supported encodings and
// commonly used subset. This allows Index implementations to include all
// available encodings without explicitly referencing or knowing about them.
package identifier

// Note: this package is internal, but could be made public if there is a need
// for writing third-party Indexes and Encodings.

// References:
// - http://source.icu-project.org/repos/icu/icu/trunk/source/data/mappings/convrtrs.txt
// - http://www.iana.org/assignments/character-sets/character-sets.xhtml
// - http://www.iana.org/assignments/ianacharset-mib/ianacharset-mib
// - http://www.ietf.org/rfc/rfc2978.txt
// - http://www.unicode.org/reports/tr22/
// - http://www.w3.org/TR/encoding/
// - http://www.w3.org/TR/encoding/indexes/encodings.json
// - https://encoding.spec.whatwg.org/
// - https://tools.ietf.org/html/rfc6657#section-5

// Interface can be implemented by Encodings to define the CCS or CES for which
// it implements conversions.
type Interface interface {
	// ID returns an encoding identifier. Exactly one of the mib and other
	// values should be non-zero.
	//
	// In the usual case it is only necessary to indicate the MIB code. The
	// other string can be used to specify encodings for which there is no MIB,
	// such as "x-mac-dingbat".
	//
	// The other string may only contain the characters a-z, A-Z, 0-9, - and _.
	ID() (mib MIB, other string)

	// NOTE: the restrictions on the encoding are to allow extending the syntax
	// with additional information such as versions, vendors and other variants.
}

// A MIB identifies an encoding. It is derived from the IANA MIB codes and adds
// some identifiers for some encodings that are not covered by the IANA
// standard.
//
// See http://www.iana.org/assignments/ianacharset-mib.
type MIB uint16

// These additional MIB types are not defined in IANA. They are added because
// they are common and defined within the text repo.
const (
	// Unofficial marks the start of encodings not registered by IANA.
	Unofficial MIB = 10000 + iota

	// TODO: add Replacement?

	// XUserDefined is the code for x-user-defined.
	XUserDefined

	// MacintoshCyrillic is the code for x-mac-cyrillic.
	MacintoshCyrillic
)