tree 6db27ace272bd4d11f2f1dc48292bd934ccbcf61
parent 79383073a4c77d928258e9586350ea77a05964b6
author Dmitri Shuralyov <dmitshur@golang.org> 1656177179 -0400
committer Gopher Robot <gobot@golang.org> 1656538777 +0000

maintner: retry network operations that may fail in getNewSegments

On the internet, servers generally don't go out of their way to ensure
graceful stream shutdowns, preferring to instead rely on the clients to
receive a network error and decide to repeat the same request.

netMutSource implements the MutationSource interface, whose GetMutations
method should return a final MutationStreamEvent event with a non-nil
Err only when the mutation source encounters a fatal error. Most callers
correctly handle such fatal errors with a log.Fatalln or equivalent.

In practice, the netMutSource.getNewSegments method does a lot of work
to ensure it doesn't give up unnecessarily soon, so most errors are
really fatal. However, it was attempting to rely on HTTP requests to
a remote server to always succeed on the first try. That is a mistake.

This change augments the getServerSegments and syncSeg methods to
annotate errors that they believe might be due to transient network
problems, and adds non-zero number of retries to getNewSegments.
That is a good layer to retry since it allows throwing away the least
amount of successful progress made thus far-much less than completely
restarting a maintner-based program and having it reload the corpus.

While here, start relying on the maintnerd server having support for
long-polling since it was added in 2017 (CL 42871), which simplifies
code by allowing the top-level loop and its nesting to be deleted.

Updates golang/go#52048.

Change-Id: I8148109f97365697beab5575358069adfa095f0e
Reviewed-on: https://go-review.googlesource.com/c/build/+/414174
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Alex Rakoczy <jenny@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>