- Keep a gzip copy of each rewrite cache entry when it is smaller, made once
per rewrite and counted against metadata_rewrite_cache_size
- Serve it with Content-Encoding: gzip to clients that accept gzip, with an
ETag of its own ("<sha256>-gzip") and Vary: Accept-Encoding
- Rewrites made outside the cache (cache disabled, cooldown on) stay
uncompressed
- rewriteCache.get and rewrite return rewrittenMetadata
- Rewritten metadata went out with only Content-Type, so pnpm, npm and
Composer could never revalidate and downloaded every document in full
- ETag is the SHA-256 of the rewritten body, computed once per rewrite and
kept with the rewrite cache entry; exact under cooldown as well
- Last-Modified is the upstream Last-Modified, never earlier than the process
start, since a restart can change the rewrite; omitted with cooldown on
- Answer If-None-Match, and If-Modified-Since when there is no If-None-Match,
with 304 on the rewrite fast path and on the fetch path
Rewriting a packument decoded the whole document into map[string]any and
encoded it again. For a full packument of tens of megabytes that costs many
times the document size per request, and with cooldown enabled (which skips
the rewrite cache and forces full packuments) it happens on every request.
- Rewrite npm metadata in place: walk the raw JSON for member offsets, decode
only the version names, time and dist-tags for the existing filters, and
assemble the response from slices of the original bytes with only tarball
URLs (and filtered time/dist-tags) written fresh. Untouched values now reach
clients byte for byte instead of round-tripping through encoding/json.
- Look up versions[v].dist.tarball and time[v] directly on the tarball
download path instead of decoding every version.
- Record the stored metadata's SHA-256 in metadata_cache.content_digest and
key the rewrite cache by it, so a repeat npm or Composer request inside the
TTL is served from the rewrite cache without reading or hashing the stored
document.
On a synthetic 25 MB packument with 5000 versions, rewriteMetadata drops from
~150-174 MB / 706k allocations per call to ~31 MB / 90k, and runs in about
two thirds of the time. npmVersionTarball drops from ~1 MB / 10k allocations
to one allocation.
