tree dcbafd2e1cd0e26c71932e0a422582c3b03057d0
parent 24e798e2876f05d628f1e9a32ce8c7f4a3ed3268
author Robert Griesemer <gri@golang.org> 1634184785 -0700
committer Robert Griesemer <gri@golang.org> 1634224615 +0000

cmd/compile/internal/types2: avoid infinite expansion for invalid recursive generic types

The algorithm for detecting invalid recursive types that
expand indefinitely suffered from the exact problem is was
intended to detect: if the indefinite expansion is happening
through type parameters, the algorithm ended up in an infinite
sequence of instantiations. (This is only a problem for generic
types).

Changed the algorithm to always only consider the "original"
uninstantiated types. This avoids the problem but it will also
not detect some invalid recursive generic types anymore. That
requires a more sophisticated type flow analysis.
Opened #48962 to track.

Addressed with help from @findleyr.

For #48951.

Change-Id: Ie29cea8f810dae55153dbb1b17c9390cd823c2d9
Reviewed-on: https://go-review.googlesource.com/c/go/+/355732
Trust: Robert Griesemer <gri@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>