mirrors/pkg-proxy
Watch
1
0
Fork
You've already forked pkg-proxy
 1
mirror of https://github.com/git-pkgs/proxy.git synced 2026-11-13 01:29:17 -05:00
Code Issues Releases Packages Wiki Activity

Commit graph

  • 9532c6a94e Add upstream.go_sumdbs_offline for networks where only the proxy has egress Christian Heim 2026-10-10 17:55:04 +02:00
  • c7f8e2fc64 README: say when the go command still goes to the database itself Christian Heim 2026-10-10 11:31:01 +02:00
  • 1b20307e81 Test the probe timeout, the block pages and a damaged cached tile Christian Heim 2026-10-10 11:31:01 +02:00
  • c1a597d2d8 Don't let the supported check hang, and don't take a block page for an answer Christian Heim 2026-10-10 11:31:01 +02:00
  • f47dd2f91c Say that the 4 KiB limit on 404 text only applies with the metadata cache Christian Heim 2026-10-10 10:19:46 +02:00
  • 9a93ce05b4 Check tile sizes before caching them Christian Heim 2026-10-10 10:19:39 +02:00
  • e6aadb297c Only claim checksum database support when the database is reachable Christian Heim 2026-10-10 10:18:40 +02:00
  • 9eb88e499f Document upstream.go_sumdbs Christian Heim 2026-10-10 09:57:43 +02:00
  • 1c6cfae813 Serve the Go checksum database from its own URL Christian Heim 2026-10-10 09:57:43 +02:00
  • 0ce007fdea
         Merge pull request #436 from pinguinfuss/issue-263-arch-pacman Andrew Nesbitt 2026-10-13 13:30:55 +01:00
  • 00e812421b
         Merge pull request #439 from simonchrz/feature/failure-details Andrew Nesbitt 2026-10-13 13:28:41 +01:00
  • 9a6b467412
         Merge pull request #446 from pinguinfuss/issue-306-retention-nuget Andrew Nesbitt 2026-10-13 13:22:28 +01:00
  • ff2c9cf37f
         Merge pull request #450 from pinguinfuss/issue-306-retention-gem Andrew Nesbitt 2026-10-13 13:21:32 +01:00
  • c96b4a955b
         Merge pull request #452 from pinguinfuss/issue-306-retention-hex Andrew Nesbitt 2026-10-13 13:19:53 +01:00
  • 131ebc4b68
         Merge pull request #454 from pinguinfuss/issue-306-retention-conda Andrew Nesbitt 2026-10-13 13:19:19 +01:00
  • a23ced2ef0
         Merge pull request #459 from gilesw/feat/generic-builtin-artifacts Andrew Nesbitt 2026-10-13 13:18:27 +01:00
  • 73d82bd78a
         Separate artifact and metadata cache metrics on /metrics and the analytics page abhinavgautam01 2026-10-13 14:33:12 +05:30
  • 9d0175b1e2 Try configured generic_artifacts before built-in patterns Giles Westwood 2026-10-12 19:22:42 +01:00
  • 854182fa28 Concurrent clients could still end up with a database and a signature that don't belong together. Databases are now stored under their digest like signatures, so a cache row and its bytes always agree, also with several replicas on shared storage. Writes for one database are serialized, and the signature recovery shares the refresh with database requests, so a slow fetch can no longer store an older database over a newer one. When the cache write fails, the proxy serves the database the cache still describes. Unchanged databases are no longer rewritten, and a remembered missing signature is asked for again after the metadata TTL. The new tests force these interleavings with storage and upstream hooks. Christian Heim 2026-10-12 20:06:59 +02:00
  • c58cdaece9 Refuse conda rules with more than a channel and a name Christian Heim 2026-10-12 19:59:35 +02:00
  • c3c120401c Fix hex tarballs whose version contains a hyphen Christian Heim 2026-10-12 19:57:57 +02:00
  • bffba28262 Enable storage retention for conda Christian Heim 2026-10-10 23:37:32 +02:00
  • 743fbe5026 Enable storage retention for hex Christian Heim 2026-10-10 23:28:13 +02:00
  • 5d7855ffc8 Store mirrored NuGet ids lowercased Christian Heim 2026-10-12 19:56:38 +02:00
  • 680a2daba1 Fix gem filenames whose platform ends in a number Christian Heim 2026-10-12 19:55:10 +02:00
  • 093ae8295e
         Merge pull request #445 from pinguinfuss/issue-306-retention-swift Andrew Nesbitt 2026-10-12 17:44:37 +01:00
  • b131db394d Isolate the recent failure history in tests Simon Chrzanowski 2026-10-12 18:44:12 +02:00
  • 7cb05b4cf6
         Merge pull request #455 from pinguinfuss/issue-306-retention-conan Andrew Nesbitt 2026-10-12 16:37:08 +01:00
  • 4273a308f7
         Merge pull request #420 from simonchrz/metadata-validators Andrew Nesbitt 2026-10-12 16:36:57 +01:00
  • f8ca190274
         Enable storage retention for swift Christian Heim 2026-10-11 01:34:46 +02:00
  • 9f75e05e8c
         Merge pull request #447 from pinguinfuss/issue-306-retention-julia Andrew Nesbitt 2026-10-12 16:32:53 +01:00
  • 4958f22e46
         Merge pull request #448 from pinguinfuss/issue-306-retention-rpm Andrew Nesbitt 2026-10-12 16:31:35 +01:00
  • ff4d5ca10d
         Enable storage retention for conan Christian Heim 2026-10-11 18:01:15 +02:00
  • 4256aac92c
         Merge pull request #449 from pinguinfuss/issue-306-retention-helm Andrew Nesbitt 2026-10-12 16:29:24 +01:00
  • f8c234fe1a
         Merge pull request #451 from pinguinfuss/issue-306-retention-pub Andrew Nesbitt 2026-10-12 16:29:18 +01:00
  • 3c1c118fd9
         Merge pull request #453 from pinguinfuss/issue-306-retention-cran Andrew Nesbitt 2026-10-12 16:29:11 +01:00
  • e20aff6649
         Merge pull request #456 from pinguinfuss/issue-306-retention-composer Andrew Nesbitt 2026-10-12 16:28:42 +01:00
  • 9c75e59ef4
         Enable storage retention for composer Christian Heim 2026-10-11 00:25:33 +02:00
  • c0f710f267
         Merge pull request #457 from pinguinfuss/issue-306-retention-cargo Andrew Nesbitt 2026-10-12 16:22:40 +01:00
  • 70ea9215c6
         Enable storage retention for cargo Christian Heim 2026-10-10 22:10:31 +02:00
  • adf844cb26
         Merge pull request #458 from pinguinfuss/issue-306-retention-alpine Andrew Nesbitt 2026-10-12 16:14:14 +01:00
  • 59f0203b00 When a signature has to be fetched on request because its fetch failed during the database refresh, it is now only served if it belongs to the database the proxy had cached at that point. If the upstream has published a new database in the meantime, the request fails with 502 instead of handing out a signature that cannot verify the database the client already has. The next sync gets the new pair. Christian Heim 2026-10-12 17:08:17 +02:00
  • fdd0c814eb Build in version-pinned paths for well-known download hosts Giles Westwood 2026-10-12 15:39:23 +01:00
  • 9f42425646
         Test upgrading cached metadata through the metadata_dates migration simon chrzanowski 2026-10-12 16:11:49 +02:00
  • dcfb9a820d Enable storage retention for nuget Christian Heim 2026-10-10 22:55:09 +02:00
  • 42f459e961 Enable storage retention for rpm Christian Heim 2026-10-11 01:21:03 +02:00
  • 48edbb8ba0 Enable storage retention for pub Christian Heim 2026-10-10 23:15:44 +02:00
  • 20855cae21 Enable storage retention for gem Christian Heim 2026-10-10 23:51:55 +02:00
  • 8f30c6d478 Enable storage retention for julia Christian Heim 2026-10-11 01:38:14 +02:00
  • f3505ad557 Enable storage retention for helm Christian Heim 2026-10-11 01:55:16 +02:00
  • 054c45c5a7
         Share the dates of rewritten metadata across replicas Simon Chrzanowski 2026-10-12 12:52:02 +02:00
  • 599b2e1473
         Add a metadata_dates table for rewritten metadata dates Simon Chrzanowski 2026-10-12 12:52:02 +02:00
  • cff4b5188d
         Date rewritten metadata by when its package started serving it Simon Chrzanowski 2026-10-12 07:07:03 +02:00
  • 8903a8c9c9
         Date rewritten metadata by when the rewrite was made Simon Chrzanowski 2026-10-10 13:47:39 +02:00
  • bf7d17d39a
         Derive cooldown metadata ETags from the rewrite's inputs simon chrzanowski 2026-10-09 08:10:41 +02:00
  • 4f680ddbd0
         Send ETag and Last-Modified with rewritten npm and Composer metadata simon chrzanowski 2026-10-08 13:26:55 +02:00
  • 6e334b4c42
         npm: version lookup returns 502 (#419) Ondrej Kokes 2026-10-12 14:45:34 +02:00
  • 1a70c962be gocognit Ondrej Kokes 2026-10-12 14:40:08 +02:00
  • ca0a442497 package not found -> version not found Ondrej Kokes 2026-10-12 14:24:10 +02:00
  • 4b8678c2c5
         Enable storage retention for npm (#440) Christian Heim 2026-10-12 14:21:38 +02:00
  • 4f86688fa7
         Merge main and resolve retention documentation conflict Andrew Nesbitt 2026-10-12 13:14:16 +01:00
  • 9357abf9f4 Merge branch 'main' into kokes/npm-version-lookup-502 Ondrej Kokes 2026-10-12 14:11:01 +02:00
  • 35a720e470 honor cooldown in version lookups Ondrej Kokes 2026-10-12 14:08:26 +02:00
  • 18f410d021
         Enable storage retention for pypi (#441) Christian Heim 2026-10-12 14:06:37 +02:00
  • c61cb51a0e
         Enable storage retention for golang (#442) Christian Heim 2026-10-12 14:04:16 +02:00
  • b40a3f067a
         Enable storage retention for oci (#443) Christian Heim 2026-10-12 14:00:57 +02:00
  • 90989eb030
         Enable storage retention for maven (#444) Christian Heim 2026-10-12 13:58:09 +02:00
  • 496f272fed
         List recent failures under each failure count on /ui/analytics Simon Chrzanowski 2026-10-12 09:41:42 +02:00
  • 3b262f101b
         Record the detail behind each failure counter Simon Chrzanowski 2026-10-12 09:41:42 +02:00
  • 086abebe6e A database refresh now fetches the signature as well and stores it for that exact database content, keyed by its digest. A signature requested later, after the upstream has moved on, is still the one for the database the client got, and a refresh that returns the same bytes keeps the cached signature, so it is served offline too. The database is fetched once more after its signature to catch a mirror that swapped both in between. A missing signature is remembered per database, so official repos answer 404 without going upstream. Christian Heim 2026-10-12 10:59:22 +02:00
  • 67fd1ad6c1 The mirror now resolves a tag without caching it, mirrors the image it points to, and stores the tag only once that image is complete, so a failed or interrupted mirror keeps the previously cached tag instead of pointing it at an image with missing layers. Christian Heim 2026-10-12 10:36:51 +02:00
  • 5b9a5fca6b When the shared tag entry holds a representation the client does not accept, the lookup now goes on to the client's own per-Accept entry instead of giving up, so a Docker-only client still gets its cached manifest list after an OCI index was cached for the same tag. Christian Heim 2026-10-12 10:29:12 +02:00
  • 546550587b Mention repeated separators in the pypi rule note Christian Heim 2026-10-10 18:00:14 +02:00
  • 7694e023a5 Enable storage retention for golang Christian Heim 2026-10-10 21:03:23 +02:00
  • ccb0935f9a Enable storage retention for oci Christian Heim 2026-10-11 00:45:58 +02:00
  • 72e986d912 Enable storage retention for maven Christian Heim 2026-10-10 18:02:49 +02:00
  • 42b4303b71 Enable storage retention for cran Christian Heim 2026-10-11 01:50:16 +02:00
  • 1d88be53ab Enable storage retention for alpine Christian Heim 2026-10-11 01:07:07 +02:00
  • 7ecb00a4ed Tighten npm retention tests and docs Christian Heim 2026-10-10 13:24:40 +02:00
  • aa27a36c20 Correct the pypi package rule note Christian Heim 2026-10-10 17:53:00 +02:00
  • b002885fcf Enable storage retention for npm Christian Heim 2026-10-10 11:47:55 +02:00
  • bb85f015e3 Enable storage retention for pypi Christian Heim 2026-10-10 13:32:04 +02:00
  • d2a069f6f2
         Cache configured version-pinned paths of generic upstreams (#424) Giles Westwood 2026-10-12 09:12:16 +01:00
  • 0c495fd906
         Proxy the OCI referrers API in the container handler (#427) Christian Heim 2026-10-12 10:08:01 +02:00
  • b664530f88
         Add age-based retention for cached artifacts (#428) Christian Heim 2026-10-12 10:06:01 +02:00
  • dcf90cfec3
         Close the Composer download gaps left after #430 (#433) Christian Heim 2026-10-12 09:56:57 +02:00
  • 16ebeaa975
         On Postgres, metadata fetched_at is now stored in UTC, so metadata_ttl no longer expires hours late when the proxy runs east of UTC. Entries written before this expire late one last time and are stored in UTC on their next refresh. (#435) Christian Heim 2026-10-12 09:53:53 +02:00
  • 729138d778
         Reject package requests with ? or # in the decoded path (#437) Christian Heim 2026-10-12 09:42:38 +02:00
  • 7b6c64af0a proxy mirror and the mirror API now pre-fetch container images from pkg:docker and pkg:oci PURLs: the mirror walks index, platform manifests (with their attestations) and blobs through the container handler, so a later pull is served from cache even when the upstream is down. A --platform filter (platforms in the API) limits multi-arch images, repository_url only resolves to configured registries, and names, tags and digests are validated before anything is fetched. Christian Heim 2026-10-12 00:20:10 +02:00
  • 411b436dd3 Let Composer archive downloads through the '?'/'#' guard: they resolve the upstream URL from metadata, and with #433 a branch like dev-fix/#42 is requested as dev-fix%2F%2342. Adversarial review finding F1. Christian Heim 2026-10-12 00:14:41 +02:00
  • 9fca85d70c Reject package requests whose decoded path contains '?' or '#', in one middleware in front of every ecosystem mount. Handlers build the upstream URL from the decoded path, so busybox.apk%3F/foo-9.9-r0.apk fetched busybox but cached it as foo; PyPI could be poisoned the same way, and the other ecosystems stored files under mangled names past the denylist and cooldown. Christian Heim 2026-10-12 00:08:20 +02:00
  • cc18c889ad proxy mirror now builds its upstream clients the same way as serve, so it follows OCI Bearer challenges, uses configured upstream credentials and the SSRF guard, and artifact downloads have no overall time limit. Before this the CLI used a plain client and Docker Hub answered every request with 401. Christian Heim 2026-10-12 00:02:31 +02:00
  • 5ce445f5d3 A database signature that belongs to an older database is no longer served. When the database was refreshed and the signature refetch fails, the proxy answers 404 or 502 instead of falling back to the cached signature, which pacman would reject. Paths with a decoded ? or # are refused, since they would fetch one file and cache it under another name. The README now says that pacman -Sy only works offline with cache_metadata enabled. Christian Heim 2026-10-11 23:51:43 +02:00
  • 094a962697 At most 64 fetches keep running after their client hangs up; beyond that a fetch is tied to its client again, so nobody can start an unbounded number of large downloads and leave. A server test now checks that client requests actually get the detach marker. Christian Heim 2026-10-11 23:51:43 +02:00
  • 111b271273 Arch Linux pacman repositories are now proxied at /arch/{name}/, with geo.mirror.pkgbuild.com as the built-in archlinux repository and more mirrors configurable under upstream.arch. Databases and file lists go through the metadata cache byte for byte, and a database signature is refreshed together with its database so the two always match. Packages and package signatures go to the artifact cache and stay available offline. The path is passed through unchanged, so Arch Linux ARM's $arch/$repo layout works too. README, config example, configuration docs and the dashboard have pacman instructions. Christian Heim 2026-10-11 23:41:16 +02:00
  • 055540a540 Manifests are now cached under a key that ignores the client's Accept header: a digest names one document, and a tag is shared by all clients that accept an index type, so Docker, containerd and Podman read the same entry and a pre-fetched image is found offline. Clients without index support keep their per-Accept entry, and the old keys are still written and read as fallback. Christian Heim 2026-10-11 23:59:29 +02:00
  • 2ed0327946 Keep fetching an artifact after the client that started the fetch disconnects. pacman gives up after 10 seconds without data, and since the proxy only starts sending once the whole file is stored, a large package on a slow upstream was cancelled on every try and never got cached. Client requests now mark their context so the shared fetch runs on context.WithoutCancel, still bounded by the artifact client timeout. Mirror jobs don't set the marker, so cancelling a job still aborts its fetches. Christian Heim 2026-10-11 23:29:52 +02:00
  • 140b4b3b5f Serve generic release assets cached under the unhashed key Giles Westwood 2026-10-11 22:16:52 +01:00
  • 7e76bc3d69 Store artifact times in UTC t 2026-10-11 21:43:48 +02:00
  • 0804b15441 On Postgres, metadata fetched_at is now stored in UTC, so metadata_ttl no longer expires hours late when the proxy runs east of UTC. Entries written before this expire late one last time and are stored in UTC on their next refresh. Christian Heim 2026-10-11 21:22:42 +02:00