A cached rewrite made with cooldown on stays right until a version it held
back leaves cooldown, so it now expires then instead of after a fixed
time. Handlers ask cooldown through Proxy.cooldown, which reports each
withheld version to the rewrites running at that moment. A rewrite that
held nothing back is kept for the shortest cooldown, at most a day.
With cooldown on, every npm, Composer and Chef metadata request rewrote
the whole document, because a cached rewrite could keep hiding a
version after its cooldown ended. For a 24.5 MB packument that is about
45 ms and 31 MB of allocations per request, and the result was served
uncompressed and without Last-Modified, so Composer, which revalidates
p2 files with If-Modified-Since only, never got a 304.
The rewrite cache now keeps cooldown rewrites with an expiry of one
minute, so a version leaving cooldown shows up at most that late.
Concurrent requests share one rewrite, clients get the gzip copy, and
the rewrite is dated through metadataDate like any other. Its ETag is
the hash of its output, so replicas agree on it.
The cache is keyed by the digest the metadata cache already recorded,
instead of hashing the document again, which also lets storedRewrite
serve cooldown rewrites without reading the stored document.
Without the rewrite cache, cooldown rewrites behave as before.
* max_size now counts cached metadata as well as artifacts, and eviction drops whichever was used least recently, so metadata no longer piles up for good (#461). Metadata rows record when they were last served, an artifact not yet served counts from its download, and Arch stores a database blob again if eviction removed it under a refreshed row.
* Shorter comments on the metadata last-access migration.
* Show evicted metadata on the analytics page next to evicted artifacts, and cover the new counter in the metric surface test.
* Evicting metadata now drops the row and queues its path in one transaction, so a crash before the blob is deleted leaves it to reclaim instead of orphaning it.
* The last-access backfill now runs in the background in batches of 1000 instead of inside the migration, resuming after a restart, and until a row is backfilled eviction orders it by its write time.
---------
Co-authored-by: Andrew Nesbitt <andrewnez@gmail.com>
