cmd/5g: fix bit mask for div/mod routines clobbering R12

This patch is based only on reading the code.  I have not
tried to construct a test case.

Fixes #9077.

LGTM=minux
R=minux
CC=golang-codereviews
https://golang.org/cl/172110043