On a build network where clients cannot reach sum.golang.org at all, the
proxy should keep answering for the database during an outage, since a 404
on supported would only send the go command somewhere it cannot go. With
the switch set, a cached tree of any age is enough; without it, the tree
has to be fresh, as before.
That does not make every cached module verify. The go command checks a
lookup against tiles of the tree that came with it, and sum.golang.org
attaches a different tree to almost every lookup, so it can ask for a tile
nobody fetched through the proxy before. configuration.md says so.
While the database is known to be unreachable, offline mode also serves
cached lookups and tiles straight away. Otherwise each of them would sit
through the upstream timeout before the fetch falls back to the cache,
and a build would take minutes for as long as the outage lasts. A lookup
or tile that falls back to the cache renews what the proxy knows about
the database, unless a probe has reached it since the request started.
The go command asks supported only once per process, so a long build would
otherwise be back to waiting a minute into the outage. recordTimeout is
now recordFailure, since lookups use it too.
The option needs cache_metadata, since without it there is no cached tree
to fall back on, and the config refuses the combination.
Also the cases around a silent or slow database: a fresh cached tree still
gets a yes on supported, a stale one does not, a late answer from the
database clears the remembered failure, a fallback to the cached tree is
not mistaken for an answer, and an answer still counts when the cache
write fails afterwards. The damaged tile test guards the size check on the
cached copy of a complete tile: it shortens the stored blob and expects
the tile to be fetched again, also past the TTL from a database that sends
an ETag.
A block page in place of a lookup must not replace the cached record, not
even one that starts like a record. The lookup fixtures now look like real
records, since lookups are checked too.
Complete tiles are never fetched again once cached, so a bad response would
stick around until someone deletes it by hand. That could be an error page
from a proxy in between that comes back with 200.
A hash tile is always W*32 bytes, so anything else is refused now and not
stored, and the same check runs when a cached tile is served. Data tiles
have no fixed size and only must not be empty.
A cached tile that fails the check is fetched again without asking the
database whether it changed. Past the TTL the cache would ask, and a
database that sends an ETag would answer 304 and hand back the damaged
copy every time.
The fake database in the tests now guards its maps with the mutex, since the
tests change them between requests.
Once a proxy answers 200 on supported, the go command sends all checksum
requests there and never goes to the database directly. A proxy that cannot
reach sum.golang.org would therefore break every build that needs a lookup.
supported now fetches latest first, through the cache, and answers 404 if
that fails and nothing is cached, so clients fall back like before. A failed
check is remembered for a minute so an unreachable database doesn't make
each go command wait for an upstream timeout.
Everything under /go/sumdb/ was passed on to the module upstream. But
proxy.golang.org doesn't serve the checksum database, so those requests
always came back 404 and every client had to talk to sum.golang.org itself.
The Go handler now serves /sumdb/{name}/ for the databases in
upstream.go_sumdbs, which defaults to sum.golang.org. Lookups, latest and
tiles come from the database through the metadata cache. Complete tiles
never change, so once they're cached we don't ask again. Unknown names and
paths that aren't part of the protocol get a 404 without going upstream.
If the database answers 404 we pass on up to 4 KiB of its text, because the
go command prints it and it tells you why a version is missing. For that the
not-found error from the metadata fetch now carries the start of the body.
Other handlers still just see ErrUpstreamNotFound.
