secure/precis: update to RFCs 8264, 8265, 8266

See:

https://www.rfc-editor.org/info/rfc8264 (Obsoletes RFC 7564)
https://www.rfc-editor.org/info/rfc8265 (Obsoletes RFC 7613)
https://www.rfc-editor.org/info/rfc8266 (Obsoletes RFC 7700)

benchmark                                                    old ns/op     new ns/op     delta
BenchmarkString/FreeForm/ASCII-4                             91.4          94.4          +3.28%
BenchmarkString/FreeForm/NotNormalized-4                     642           648           +0.93%
BenchmarkString/FreeForm/Arabic-4                            187           191           +2.14%
BenchmarkString/FreeForm/Hangul-4                            453           453           +0.00%
BenchmarkString/Nickname/ASCII-4                             363           370           +1.93%
BenchmarkString/Nickname/NotNormalized-4                     938           2095          +123.35%
BenchmarkString/Nickname/Arabic-4                            371           1319          +255.53%
BenchmarkString/Nickname/Hangul-4                            704           2564          +264.20%
BenchmarkString/OpaqueString/ASCII-4                         188           195           +3.72%
BenchmarkString/OpaqueString/NotNormalized-4                 775           775           +0.00%
BenchmarkString/OpaqueString/Arabic-4                        260           265           +1.92%
BenchmarkString/OpaqueString/Hangul-4                        577           577           +0.00%
BenchmarkString/UsernameCaseMapped/ASCII-4                   95.2          96.5          +1.37%
BenchmarkString/UsernameCaseMapped/NotNormalized-4           853           857           +0.47%
BenchmarkString/UsernameCaseMapped/Arabic-4                  339           345           +1.77%
BenchmarkString/UsernameCaseMapped/Hangul-4                  748           756           +1.07%
BenchmarkString/UsernameCasePreserved/ASCII-4                94.1          95.4          +1.38%
BenchmarkString/UsernameCasePreserved/NotNormalized-4        728           738           +1.37%
BenchmarkString/UsernameCasePreserved/Arabic-4               260           262           +0.77%
BenchmarkString/UsernameCasePreserved/Hangul-4               587           592           +0.85%
BenchmarkBytes/FreeForm/ASCII-4                              81.3          81.4          +0.12%
BenchmarkBytes/FreeForm/NotNormalized-4                      590           585           -0.85%
BenchmarkBytes/FreeForm/Arabic-4                             178           176           -1.12%
BenchmarkBytes/FreeForm/Hangul-4                             420           428           +1.90%
BenchmarkBytes/Nickname/ASCII-4                              321           320           -0.31%
BenchmarkBytes/Nickname/NotNormalized-4                      885           2038          +130.28%
BenchmarkBytes/Nickname/Arabic-4                             338           1278          +278.11%
BenchmarkBytes/Nickname/Hangul-4                             655           2540          +287.79%
BenchmarkBytes/OpaqueString/ASCII-4                          187           181           -3.21%
BenchmarkBytes/OpaqueString/NotNormalized-4                  717           713           -0.56%
BenchmarkBytes/OpaqueString/Arabic-4                         250           249           -0.40%
BenchmarkBytes/OpaqueString/Hangul-4                         542           546           +0.74%
BenchmarkBytes/UsernameCaseMapped/ASCII-4                    82.4          82.0          -0.49%
BenchmarkBytes/UsernameCaseMapped/NotNormalized-4            797           800           +0.38%
BenchmarkBytes/UsernameCaseMapped/Arabic-4                   325           329           +1.23%
BenchmarkBytes/UsernameCaseMapped/Hangul-4                   720           724           +0.56%
BenchmarkBytes/UsernameCasePreserved/ASCII-4                 82.1          82.1          +0.00%
BenchmarkBytes/UsernameCasePreserved/NotNormalized-4         672           674           +0.30%
BenchmarkBytes/UsernameCasePreserved/Arabic-4                244           250           +2.46%
BenchmarkBytes/UsernameCasePreserved/Hangul-4                550           553           +0.55%
BenchmarkAppend/FreeForm/ASCII-4                             67.6          65.5          -3.11%
BenchmarkAppend/FreeForm/NotNormalized-4                     590           590           +0.00%
BenchmarkAppend/FreeForm/Arabic-4                            155           155           +0.00%
BenchmarkAppend/FreeForm/Hangul-4                            409           399           -2.44%
BenchmarkAppend/Nickname/ASCII-4                             321           320           -0.31%
BenchmarkAppend/Nickname/NotNormalized-4                     886           2051          +131.49%
BenchmarkAppend/Nickname/Arabic-4                            340           1270          +273.53%
BenchmarkAppend/Nickname/Hangul-4                            652           2537          +289.11%
BenchmarkAppend/OpaqueString/ASCII-4                         154           153           -0.65%
BenchmarkAppend/OpaqueString/NotNormalized-4                 720           723           +0.42%
BenchmarkAppend/OpaqueString/Arabic-4                        219           220           +0.46%
BenchmarkAppend/OpaqueString/Hangul-4                        517           520           +0.58%
BenchmarkAppend/UsernameCaseMapped/ASCII-4                   68.1          66.0          -3.08%
BenchmarkAppend/UsernameCaseMapped/NotNormalized-4           795           801           +0.75%
BenchmarkAppend/UsernameCaseMapped/Arabic-4                  287           299           +4.18%
BenchmarkAppend/UsernameCaseMapped/Hangul-4                  683           700           +2.49%
BenchmarkAppend/UsernameCasePreserved/ASCII-4                65.9          67.1          +1.82%
BenchmarkAppend/UsernameCasePreserved/NotNormalized-4        668           679           +1.65%
BenchmarkAppend/UsernameCasePreserved/Arabic-4               217           211           -2.76%
BenchmarkAppend/UsernameCasePreserved/Hangul-4               536           517           -3.54%
BenchmarkTransform/FreeForm/ASCII-4                          103           101           -1.94%
BenchmarkTransform/FreeForm/NotNormalized-4                  488           494           +1.23%
BenchmarkTransform/FreeForm/Arabic-4                         171           168           -1.75%
BenchmarkTransform/FreeForm/Hangul-4                         412           410           -0.49%
BenchmarkTransform/Nickname/ASCII-4                          237           919           +287.76%
BenchmarkTransform/Nickname/NotNormalized-4                  660           1554          +135.45%
BenchmarkTransform/Nickname/Arabic-4                         229           919           +301.31%
BenchmarkTransform/Nickname/Hangul-4                         551           2107          +282.40%
BenchmarkTransform/OpaqueString/ASCII-4                      195           188           -3.59%
BenchmarkTransform/OpaqueString/NotNormalized-4              614           616           +0.33%
BenchmarkTransform/OpaqueString/Arabic-4                     234           229           -2.14%
BenchmarkTransform/OpaqueString/Hangul-4                     583           536           -8.06%
BenchmarkTransform/UsernameCaseMapped/ASCII-4                345           334           -3.19%
BenchmarkTransform/UsernameCaseMapped/NotNormalized-4        816           820           +0.49%
BenchmarkTransform/UsernameCaseMapped/Arabic-4               389           377           -3.08%
BenchmarkTransform/UsernameCaseMapped/Hangul-4               848           840           -0.94%
BenchmarkTransform/UsernameCasePreserved/ASCII-4             175           173           -1.14%
BenchmarkTransform/UsernameCasePreserved/NotNormalized-4     621           620           -0.16%
BenchmarkTransform/UsernameCasePreserved/Arabic-4            268           278           +3.73%
BenchmarkTransform/UsernameCasePreserved/Hangul-4            599           593           -1.00%

Change-Id: I98ee577c3c33fce547d839505c4f04477ab74e83
Reviewed-on: https://go-review.googlesource.com/68790
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Marcel van Lohuizen <mpvl@golang.org>
6 files changed
tree: 13580ca8cce523dc3e4fded1f1064e0ddc9f483a
  1. cases/
  2. cmd/
  3. collate/
  4. currency/
  5. encoding/
  6. feature/
  7. internal/
  8. language/
  9. message/
  10. number/
  11. runes/
  12. search/
  13. secure/
  14. transform/
  15. unicode/
  16. width/
  17. .gitattributes
  18. .gitignore
  19. AUTHORS
  20. codereview.cfg
  21. CONTRIBUTING.md
  22. CONTRIBUTORS
  23. doc.go
  24. gen.go
  25. LICENSE
  26. PATENTS
  27. README.md
README.md

Go Text

This repository holds supplementary Go libraries for text processing, many involving Unicode.

Semantic Versioning

This repo uses Semantic versioning (http://semver.org/), so

  1. MAJOR version when you make incompatible API changes,
  2. MINOR version when you add functionality in a backwards-compatible manner, and
  3. PATCH version when you make backwards-compatible bug fixes.

A Unicode major and minor version bump is mapped to a major version bump in x/text. A path version bump in Unicode is mapped to a minor version bump in x/text. Note that, consistent with the definitions in semver, until version 1.0.0 of x/text is reached, the minor version is considered a major version. So going from 0.1.0 to 0.2.0 is considered to be a major version bump.

A major new CLDR version is mapped to a minor version increase in x/text. Any other new CLDR version is mapped to a patch version increase in x/text.

Download/Install

The easiest way to install is to run go get -u golang.org/x/text. You can also manually git clone the repository to $GOPATH/src/golang.org/x/text.

Contribute

To submit changes to this repository, see http://golang.org/doc/contribute.html.

To generate the tables in this repository (except for the encoding tables), run go generate from this directory. By default tables are generated for the Unicode version in core and the CLDR version defined in golang.org/x/text/unicode/cldr.

Running go generate will as a side effect create a DATA subdirectory in this directory, which holds all files that are used as a source for generating the tables. This directory will also serve as a cache.

Testing

Run

go test ./...

from this directory to run all tests. Add the “-tags icu” flag to also run ICU conformance tests (if available). This requires that you have the correct ICU version installed on your system.

TODO:

  • updating unversioned source files.

Generating Tables

To generate the tables in this repository (except for the encoding tables), run go generate from this directory. By default tables are generated for the Unicode version in core and the CLDR version defined in golang.org/x/text/unicode/cldr.

Running go generate will as a side effect create a DATA subdirectory in this directory which holds all files that are used as a source for generating the tables. This directory will also serve as a cache.

Versions

To update a Unicode version run

UNICODE_VERSION=x.x.x go generate

where x.x.x must correspond to a directory in http://www.unicode.org/Public/. If this version is newer than the version in core it will also update the relevant packages there. The idna package in x/net will always be updated.

To update a CLDR version run

CLDR_VERSION=version go generate

where version must correspond to a directory in http://www.unicode.org/Public/cldr/.

Note that the code gets adapted over time to changes in the data and that backwards compatibility is not maintained. So updating to a different version may not work.

The files in DATA/{iana|icu|w3|whatwg} are currently not versioned.

Report Issues / Send Patches

This repository uses Gerrit for code changes. To learn how to submit changes to this repository, see https://golang.org/doc/contribute.html.

The main issue tracker for the image repository is located at https://github.com/golang/go/issues. Prefix your issue with “x/image:” in the subject line, so it is easy to find.