commit 89ce4c7ba8043b0e8a8c03f1a266fb0e56637314
author Jay Conrod <jayconrod@google.com> Tue Jun 30 12:20:13 2020 -0400
committer Jay Conrod <jayconrod@google.com> Mon Jul 06 16:06:32 2020 +0000
tree 85e82b9b46d8b6f7ee2e4043e268385027e1f7fe
parent 0b26df4a2231f23352410b323188a0e4002e26a3

modfile: remove trailing newline from comment tokens

In v0.2.0, the go.mod lexer removed trailing LF bytes from comment
tokens. This regressed in v0.3.0. Documentation on Comment.Token says
the trailing newline should not be included.

This CL fixes the lexer to strip trailing newlines again. It will now
strip both LF and CRLF newlines. It also includes a test to ensure
comments are attached at the right place in the syntax tree with the
right content.

Fixes golang/go#39913

Change-Id: I7fba0ed3c85f0a3c23fefc6b7fecfe6df7777aea
Reviewed-on: https://go-review.googlesource.com/c/mod/+/240557
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>