ssh: fix infinite loop on large channel writes due to integer overflow

The internal 'min' helper function in channel.go incorrectly cast the
input data length (int) to uint32 before comparing it with the
maximum packet size. On 64-bit systems, if the data length is a
multiple of 2^32 (approx. 4GB), this cast results in 0.

Consequently, the function returns 0, causing the WriteExtended loop
to spin indefinitely because it attempts to reserve 0 bytes while
the remaining data length is still positive.

This change renames the helper to 'minPayloadSize' to avoid confusion
with the Go 1.21 built-in 'min' and updates the logic to use int64
for comparisons, preventing truncation and the resulting infinite loop.

This issue was found during a security audit by NCC Group Cryptography
Services, sponsored by Teleport.

Fixes golang/go#79567
Fixes CVE-2026-39834

Change-Id: Id5bf81d9f06c7042452acffe1c76580ff878665e
Reviewed-on: https://go-review.googlesource.com/c/crypto/+/781663
Reviewed-by: Neal Patel <nealpatel@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: golang-scoped@luci-project-accounts.iam.gserviceaccount.com <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>