all: support enforce_utf8 override

In 2014, when proto3 was being developed, there were a number of early
adopters of the new syntax. Before the finalization of proto3 when
it was released in open-source in July 2016, a decision was made to
strictly validate strings in proto3. However, some of the early adopters
were already using invalid UTF-8 with string fields.
The google.protobuf.FieldOptions.enforce_utf8 option only exists to support
those grandfathered users where they can opt-out of the validation logic.
Practical use of that option in open source is impossible even if a user
specifies the proto1_legacy build tag since it requires a hacked
variant of descriptor.proto that is not externally available.

This CL supports enforce_utf8 by modifiyng internal/filedesc to
expose the flag if it detects it in the raw descriptor.
We add an strs.EnforceUTF8 function as a centralized place to determine
whether to perform validation. Validation opt-out is supported
only in builds with legacy support.

We implement support for validating UTF-8 in all proto3 string fields,
even if they are backed by a Go []byte.

Change-Id: I9c0628b84909bc7181125f09db730c80d490e485
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/186002
Reviewed-by: Damien Neil <dneil@google.com>
13 files changed
tree: 83eb3482279281b6a76819e0c2907fed59ac4e31
  1. benchmarks/
  2. cmd/
  3. compiler/
  4. encoding/
  5. internal/
  6. proto/
  7. reflect/
  8. runtime/
  9. testing/
  10. types/
  11. .gitignore
  12. .travis.yml
  13. AUTHORS
  14. CONTRIBUTING.md
  15. CONTRIBUTORS
  16. go.mod
  17. go.sum
  18. integration_test.go
  19. LICENSE
  20. PATENTS
  21. README.md
  22. regenerate.bash
  23. test.bash
README.md

Next Generation Go Protocol Buffers

WARNING: This repository is in active development. There are no guarantees about API stability. Breaking changes will occur until a stable release is made and announced.

This repository is for the development of the next major Go implementation of protocol buffers. This library makes breaking API changes relative to the existing Go protobuf library. Of particular note, this API aims to make protobuf reflection a first-class feature of the API and implements the protobuf ecosystem in terms of reflection.

Design Documents

List of relevant design documents:

Contributing

We appreciate community contributions. See CONTRIBUTING.md.

Reporting Issues

Issues regarding the new API can be filed at github.com/golang/protobuf. Please use a APIv2: prefix in the title to make it clear that the issue is regarding the new API work.