commit 9c0375277ca88df0065fe442242a110201029793
author Evan Phoenix <evan@phx.io> Sat Mar 21 23:42:47 2015 -0700
committer Robert Griesemer <gri@golang.org> Wed Apr 01 03:39:22 2015 +0000
tree 6d2bcf45f5c21b81a7423659c76ad616bb179ca3
parent 15e66f9d01da995d63683245a429e64d71ac698e

text/scanner: Fix EOF reporting on strange Readers

Currently, scanner uses -1 to represent 2 different states:

1. I haven't yet scanned anything, call it "Beginning of File"
2. I've reached the end of the input, ie EOF

The result of this behavior is that calling Peek() when next()
has detected the end of the input and set s.ch to scanner.EOF,
is that Peek() things "oh, s.ch is < 0, which to me means that
I haven't scanned any next yet, let me try and clear the BOM
marker."

When this behavior is run on a typical IO, next() will issue
a Read and get (0, io.EOF) back for the second time without
blocking and Peek() will return scanner.EOF.

The bug comes into play when, inside a terminal, hitting Control-D.
This causes the terminal to return a EOF condition to the reader
but it does not actually close the fd.

So, combining these 2 situations, we arrive at the bug:

What is expected: hitting Control-D in a terminal will make Peek()
return scanner.EOF instantly.

What actually happens:

0. Code waiting in Next()
1. User hits Control-D
2. fd returns EOF condition
3. EOF bubbles it's way out to line 249 in scanner.go
4. next() returns scanner.EOF
5. Next() saves the scanner.EOF to s.ch and returns the previous value
6. Peek() runs, sees s.ch < 0, mistakenly thinks it hasn't run yet and
   tries to read the BOM marker.
7. next() sees the buffer is empty and tries to fill it again, blocking
   on line 249.

The fix is simple: use a different code to indicate that no data
has been scanned.

Change-Id: Iee8f4da5881682c4d4c36b93b9bf397ac5798179
Reviewed-on: https://go-review.googlesource.com/7913
Reviewed-by: Robert Griesemer <gri@golang.org>