go/analysis: add frame pointer check for vet

Our calling convention requires BP to be preserved (it is
callee-save).  But that only happened with the introduction of
framepointers in 1.5. Furthermore, nothing checks it, so there is
assembly from before and since which violates the calling
convention. The frame pointer is not used by Go, but only by kernel
traceback code (for sampled profiling), so no one noticed.  Also, the
frame pointer tends to "fix itself", since after it is clobbered by an
assembly function f, the calling function fixes the frame pointer up
when that function returns.

I've fixed the stdlib, CLs 248260, 248261, 248262.

This CL is a simple check, intended to be used in vet, to catch
assembly that violates the calling convention by overwriting the
frame pointer without saving/restoring it. It is not intended to
catch all violations, just ones that are easy to find.

We look for a write to the frame pointer before any use of the frame
pointer or any control flow instruction. In such a situation, there's
no way the code could restore the value of BP before returning. This
analysis is very conservative; it gives up in lots of cases. False
positive should be very rare, though.

Possible false positives:

 // looks like a write to BP, but isn't.
CMPQ  BP, BP

// BP actually isn't clobbered, just swapped with AX.
XORQ  AX, BP
XORQ  BP, AX
XORQ  AX, BP

The first is unlikely, as it is using the contents of an incoming
register, which is junk. The second is a general problem of op=
assembly operations that are indistiguishable in syntax from =
operations.  But anything other than the swap above also depends on
the incoming junk value in BP. The swap itself is pointless (XCHQ
works better).

Change-Id: Ie9d91ab3396409486f7022380ad46ac76c3fbed4
Reviewed-on: https://go-review.googlesource.com/c/tools/+/248686
Trust: Keith Randall <khr@golang.org>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
Run-TryBot: Keith Randall <khr@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
diff --git a/go/analysis/passes/framepointer/framepointer_test.go b/go/analysis/passes/framepointer/framepointer_test.go
new file mode 100644
index 0000000..e849308
--- /dev/null
+++ b/go/analysis/passes/framepointer/framepointer_test.go
@@ -0,0 +1,26 @@
+// Copyright 2020 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package framepointer_test
+
+import (
+	"go/build"
+	"testing"
+
+	"golang.org/x/tools/go/analysis/analysistest"
+	"golang.org/x/tools/go/analysis/passes/framepointer"
+)
+
+func Test(t *testing.T) {
+	if build.Default.GOOS != "linux" && build.Default.GOOS != "darwin" {
+		// The test has an os-generic assembly file, testdata/a/asm_amd64.s.
+		// It should produce errors on linux or darwin, but not on other archs.
+		// Unfortunately, there's no way to say that in the "want" comments
+		// in that file. So we skip testing on other GOOSes. The framepointer
+		// analyzer should not report any errors on those GOOSes, so it's not
+		// really a hard test on those platforms.
+		t.Skipf("test for GOOS=%s is not implemented", build.Default.GOOS)
+	}
+	analysistest.Run(t, analysistest.TestData(), framepointer.Analyzer, "a")
+}