Watch
1
0
Fork
You've already forked pkg-proxy
1
mirror of https://github.com/git-pkgs/proxy.git synced 2026-11-09 15:49:16 -05:00
Commit graph pkg-proxy/internal/handler/arch.go
Author SHA1 Message Date
Abhinav Gautam
a97b690858
Separate artifact and metadata cache metrics on /metrics and the analytics page (#463)
* Separate artifact and metadata cache metrics on /metrics and the analytics page

* Serve metadata cache stats from a shared snapshot and split Grafana cache panels by kind
2026-10-20 11:25:33 +01:00
Christian Heim
d56d4367c9
Count cached metadata against max_size and evict it LRU (#507)
* max_size now counts cached metadata as well as artifacts, and eviction drops whichever was used least recently, so metadata no longer piles up for good (#461). Metadata rows record when they were last served, an artifact not yet served counts from its download, and Arch stores a database blob again if eviction removed it under a refreshed row.

* Shorter comments on the metadata last-access migration.

* Show evicted metadata on the analytics page next to evicted artifacts, and cover the new counter in the metric surface test.

* Evicting metadata now drops the row and queues its path in one transaction, so a crash before the blob is deleted leaves it to reclaim instead of orphaning it.

* The last-access backfill now runs in the background in batches of 1000 instead of inside the migration, resuming after a restart, and until a row is backfilled eviction orders it by its write time.

---------

Co-authored-by: Andrew Nesbitt <andrewnez@gmail.com>
2026-10-20 10:22:55 +01:00
Giles Westwood
b2955ef458 Keep holds through late successes and away from the circuit breaker
A response arriving during a hold was sent before it began, so a late
success no longer releases the host; only one after the hold has ended
does.

Artifact fetches check RateLimitGate.Held before calling the fetcher, so
held cache misses fail with the HeldError without counting as failures
in the host's circuit breaker.

Every handler that answers an upstream fetch error with 502 now answers
a held request with 503 and Retry-After instead.

OnHold and RecordUpstreamRateLimitHold drop the unused duration.
2026-10-15 13:35:12 +01:00
Christian Heim
854182fa28 Concurrent clients could still end up with a database and a signature that don't belong together. Databases are now stored under their digest like signatures, so a cache row and its bytes always agree, also with several replicas on shared storage. Writes for one database are serialized, and the signature recovery shares the refresh with database requests, so a slow fetch can no longer store an older database over a newer one. When the cache write fails, the proxy serves the database the cache still describes. Unchanged databases are no longer rewritten, and a remembered missing signature is asked for again after the metadata TTL. The new tests force these interleavings with storage and upstream hooks. 2026-10-12 20:06:59 +02:00
Christian Heim
59f0203b00 When a signature has to be fetched on request because its fetch failed during the database refresh, it is now only served if it belongs to the database the proxy had cached at that point. If the upstream has published a new database in the meantime, the request fails with 502 instead of handing out a signature that cannot verify the database the client already has. The next sync gets the new pair. 2026-10-12 17:08:17 +02:00
Christian Heim
086abebe6e A database refresh now fetches the signature as well and stores it for that exact database content, keyed by its digest. A signature requested later, after the upstream has moved on, is still the one for the database the client got, and a refresh that returns the same bytes keeps the cached signature, so it is served offline too. The database is fetched once more after its signature to catch a mirror that swapped both in between. A missing signature is remembered per database, so official repos answer 404 without going upstream. 2026-10-12 10:59:22 +02:00
Christian Heim
5ce445f5d3 A database signature that belongs to an older database is no longer served. When the database was refreshed and the signature refetch fails, the proxy answers 404 or 502 instead of falling back to the cached signature, which pacman would reject. Paths with a decoded ? or # are refused, since they would fetch one file and cache it under another name. The README now says that pacman -Sy only works offline with cache_metadata enabled. 2026-10-12 00:02:02 +02:00
Christian Heim
111b271273 Arch Linux pacman repositories are now proxied at /arch/{name}/, with geo.mirror.pkgbuild.com as the built-in archlinux repository and more mirrors configurable under upstream.arch. Databases and file lists go through the metadata cache byte for byte, and a database signature is refreshed together with its database so the two always match. Packages and package signatures go to the artifact cache and stay available offline. The path is passed through unchanged, so Arch Linux ARM's $arch/$repo layout works too. README, config example, configuration docs and the dashboard have pacman instructions. 2026-10-12 00:02:02 +02:00