metadata_max_size bounds what is read into memory, but a packument
streamed from upstream into storage had no limit, and with cooldown was
stored twice. It is now capped by metadata_stream_max_size, 1 GB by
default. A document over the cap fails as an upstream failure: nothing is
stored, and the stored copy, if any, is served instead.
The metadata settings move into configureMetadata to keep serve under
the maintidx limit.
The change to ReadMetadataSized and fetchUpstreamMetadata, with its tests,
is now #649. upstreamBodyReader stays for storeUpstreamMetadata, in a file
of its own that #649 adds with the same content.
A body that ended before its Content-Length was read as complete in
memory: ReadMetadataSized took the transport's io.ErrUnexpectedEOF for
the end of a body shorter than its size hint. The half document was
served and cached, or failed the filter with a 502, instead of the stored
copy being served. Upstream bodies are now read through a reader that
marks their read errors, and such a failure is an upstream failure on
both paths: streamed, it no longer counts as a failure to store, which
had sent the request to the bounded in-memory path.
A stored packument whose bytes no longer match its row's digest is now
fetched again like a missing one.
Tests also cover concurrent requests sharing one fetch, client
revalidation and upstream failures with nothing cached. The stream tests
build cooldown through cooldown.NewPolicy as the server does, since a
bare cooldown.Config parses itself on first use without locking.
