modfile: scan ASCII brackets and commas as separate tokens

The characters '( ) [ ] { } ,' are now scanned as separate tokens,
even when they are preceded by non-whitespace "identifier"
characters. Previously, '( )' were scanned like this when preceded by
whitespace, but they could be in the middle of an identifier
token. None of these characters are allowed in module paths or
versions. They are allowed within file paths, so file paths containing
them will need to be quoted in the future. Using these characters
should not break ParseLax, since replace directives (the only directive
that allows files paths) are ignored by ParseLax.

Additionally, '(' is only treated as the beginning of a block if it
appears at the end of the line or is immediately followed by ')' at the
end of the line. ')' is treated as the end of a block if it
appears within a block at the beginning of a line.

Fixes golang/go#38167
Updates golang/go#38144
Updates golang/go#24031

Change-Id: I5a7fb522163802c3723d289cf0fbc5856ca075ec
Reviewed-on: https://go-review.googlesource.com/c/mod/+/226639
Reviewed-by: Bryan C. Mills <bcmills@google.com>
8 files changed