| commit | 3a7a2557e7386e7e39d8b31290c3e8962c39e0fc | [log] [tgz] |
|---|---|---|
| author | Marcel van Lohuizen <mpvl@golang.org> | Wed Jun 07 13:35:29 2023 +0200 |
| committer | Russ Cox <rsc@golang.org> | Mon Jun 12 16:55:37 2023 +0000 |
| tree | c3ecd44a815ee5bf7a43ae70b910f1c8214c1dd6 | |
| parent | d61dd50441c6e2d3595be435b162ca96c58848f9 [diff] |
internal/export/idna: make more space for mapping index This prepares for an upcoming Unicode upgrade. Beyond Unicode 13 the size of mappings will grow beyond what can be represented in the allocated 13 bits. Instread of doubling the size of info, we introduce a table of indices into the mappings data. This also allows us to remove the length byte, reducing the overhead of introducing this new table. This change allows for about a 5x growth of the number of mappings, which should suffice for the foreseeable future. Change-Id: Id475dc2473145a1f36bd83b983fa4aa170df6206 Reviewed-on: https://go-review.googlesource.com/c/text/+/501515 Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: David Chase <drchase@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
This repository holds supplementary Go libraries for text processing, many involving Unicode.
It is important that the Unicode version used in x/text matches the one used by your Go compiler. The x/text repository supports multiple versions of Unicode and will match the version of Unicode to that of the Go compiler. At the moment this is supported for Go compilers from version 1.7.
The easiest way to install is to run go get -u golang.org/x/text. You can also manually git clone the repository to $GOPATH/src/golang.org/x/text.
To submit changes to this repository, see http://golang.org/doc/contribute.html.
To generate the tables in this repository (except for the encoding tables), run go generate from this directory. By default tables are generated for the Unicode version in core and the CLDR version defined in golang.org/x/text/unicode/cldr.
Running go generate will as a side effect create a DATA subdirectory in this directory, which holds all files that are used as a source for generating the tables. This directory will also serve as a cache.
Run
go test ./...
from this directory to run all tests. Add the “-tags icu” flag to also run ICU conformance tests (if available). This requires that you have the correct ICU version installed on your system.
TODO:
To generate the tables in this repository (except for the encoding tables), run go generate from this directory. By default tables are generated for the Unicode version in core and the CLDR version defined in golang.org/x/text/unicode/cldr.
Running go generate will as a side effect create a DATA subdirectory in this directory which holds all files that are used as a source for generating the tables. This directory will also serve as a cache.
To update a Unicode version run
UNICODE_VERSION=x.x.x go generate
where x.x.x must correspond to a directory in https://www.unicode.org/Public/. If this version is newer than the version in core it will also update the relevant packages there. The idna package in x/net will always be updated.
To update a CLDR version run
CLDR_VERSION=version go generate
where version must correspond to a directory in https://www.unicode.org/Public/cldr/.
Note that the code gets adapted over time to changes in the data and that backwards compatibility is not maintained. So updating to a different version may not work.
The files in DATA/{iana|icu|w3|whatwg} are currently not versioned.
This repository uses Gerrit for code changes. To learn how to submit changes to this repository, see https://golang.org/doc/contribute.html.
The main issue tracker for the image repository is located at https://github.com/golang/go/issues. Prefix your issue with “x/text:” in the subject line, so it is easy to find.