Cooldown filtering only ran when rewriting metadata, so a version could
be missing from the npm packument and the PyPI simple index while its
tarball stayed reachable. Lockfiles record artifact URLs verbatim, so
npm ci and pinned pip requirements reach handleDownload without ever
requesting metadata.
The shared artifact path has no publish time to check against, since
updateCacheDB upserts versions without PublishedAt and the column is
only set by enrichment. Each handler now resolves the publish time from
metadata it already fetches and returns 404 while a version is inside
the window. Versions with no usable publish time are still served, as
they are when filtering metadata.
When cooldown is disabled, send:
Accept: application/vnd.npm.install-v1+json;q=1.0, application/json;q=0.8
This allows upstreams like JFrog Artifactory that return 406 for the
abbreviated packument type to fall back to full JSON metadata, while
letting the public npm registry continue to serve the smaller
abbreviated format it prefers.
When cooldown is enabled, keep sending only application/json because
the abbreviated format omits the "time" map required for version age
filtering.
Fixes#228
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
NormalizedPackages runs config keys through purl.Parse().String(), which
applies per-type rules like lowercasing pypi names. The handler side was
building lookup keys with MakePURLString, which does not, so a config
entry for pkg:pypi/Django would be normalized to pkg:pypi/django and
never match the runtime key pkg:pypi/Django.
Route both sides through the same canonical form: a new
canonicalPackagePURL helper calls Normalize() on the constructed PURL
before stringifying, and all cooldown IsAllowed call sites use it.
Add a `proxy mirror` CLI command and `/api/mirror` API endpoints that
pre-populate the cache from various input sources: individual PURLs,
SBOM files (CycloneDX and SPDX), or full registry enumeration.
The mirror reuses the existing handler.Proxy.GetOrFetchArtifact()
pipeline so cached artifacts are identical to those fetched on demand.
A bounded worker pool controls download parallelism.
Metadata caching is opt-in via `cache_metadata: true` in config (or
PROXY_CACHE_METADATA=true). The mirror command always enables it. When
enabled, upstream metadata responses are stored for offline fallback
with ETag-based conditional revalidation.
New internal/mirror package with Source interface, PURLSource,
SBOMSource, RegistrySource, and async JobStore. New metadata_cache
database table for offline metadata serving.
Request application/vnd.npm.install-v1+json from the npm registry
when cooldown filtering is not enabled. This format strips READMEs
and other bulk data, reducing drizzle-orm metadata from 92MB to 4MB.
Fall back to full metadata when cooldown is enabled since the
abbreviated format lacks the time map needed for publish-date filtering.
* Fix all golangci-lint issues across the codebase
Resolve 77 lint issues reported by golangci-lint with gocritic, gocognit,
gocyclo, maintidx, dupl, mnd, unparam, ireturn, goconst, and errcheck
enabled. Net reduction of ~175 lines through shared helpers and
deduplication.
* Suppress staticcheck SA1019 for intentional deprecated field usage
The Storage.Path field is deprecated but still read for backwards
compatibility with existing configs that haven't migrated to the URL field.
All handler metadata and proxy requests were using http.DefaultClient directly,
bypassing any timeout or transport configuration. Added an HTTPClient field to
the Proxy struct with a 30-second default timeout, and updated every handler
to use it for upstream HTTP requests.
POST endpoints (/api/outdated, /api/bulk) now reject bodies over 1 MB
using http.MaxBytesReader. Upstream metadata reads (npm, pypi, composer,
nuget, pub) now use io.LimitReader capped at 50 MB to prevent OOM from
unexpectedly large responses.
Hides package versions published too recently from metadata responses,
giving the community time to spot malicious releases. Configurable
per-ecosystem and per-package with duration overrides. Supported for
npm, PyPI, pub.dev, and Composer.
