Cooldown filtering only ran when rewriting metadata, so a version could
be missing from the npm packument and the PyPI simple index while its
tarball stayed reachable. Lockfiles record artifact URLs verbatim, so
npm ci and pinned pip requirements reach handleDownload without ever
requesting metadata.
The shared artifact path has no publish time to check against, since
updateCacheDB upserts versions without PublishedAt and the column is
only set by enrichment. Each handler now resolves the publish time from
metadata it already fetches and returns 404 while a version is inside
the window. Versions with no usable publish time are still served, as
they are when filtering metadata.
* resolve name and version for PEP 658 metadata sidecars
* fix(pypi): parse Windows installer and egg filenames separately
The bdist_wininst and bdist_msi layout joins the platform to the version
with a '.' rather than a '-', so treating .exe/.msi like a wheel folded
the platform into the version: foo-1.0.win32-py2.0.exe resolved to
version "1.0.win32". Eggs shared the problem, as setuptools' hyphen
escaping is not universal: aws-sdk-1.0.0-py3.11.egg resolved to name
"aws", version "sdk".
Give each format its own parser. Wheels keep the PEP 427
spec-guaranteed field positions, eggs locate the version relative to the
py{X.Y} interpreter field, and Windows installers strip the platform and
interpreter fields before splitting name from version.
A PEP 658 sidecar resolves to the same name and version as the
distribution it describes, so it is cached under that version. Browse and
compare took the first cached artifact without checking its extension,
handing openArchive plain text: a version pip had only fetched metadata
for reported hasCached and then 500'd.
Add firstBrowsableArtifact, replacing five duplicated selection loops,
and export PyPIMetadataSuffix so the suffix has a single definition.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
NormalizedPackages runs config keys through purl.Parse().String(), which
applies per-type rules like lowercasing pypi names. The handler side was
building lookup keys with MakePURLString, which does not, so a config
entry for pkg:pypi/Django would be normalized to pkg:pypi/django and
never match the runtime key pkg:pypi/Django.
Route both sides through the same canonical form: a new
canonicalPackagePURL helper calls Normalize() on the constructed PURL
before stringifying, and all cooldown IsAllowed call sites use it.
- ProxyCached now stores upstream Last-Modified in the cache and uses it
(along with ETag) for conditional request handling, returning 304 when
client validators match. Adds Content-Length to cached responses.
- Handlers calling FetchOrCacheMetadata (pypi, composer, pub, nuget) now
check for ErrUpstreamNotFound and return 404 instead of 502, matching
the existing npm and cargo behavior.
- Mirror jobs report live progress via a periodic callback while running,
so API polls return real counts instead of zeroed progress.
- Registry mirroring removed from CLI flags, API acceptance, README, and
docs since every enumerator was a stub returning "not yet implemented".
- Added tests for the conditional metadata path (ETag/If-None-Match,
Last-Modified/If-Modified-Since, 304 responses, header omission).
Add a `proxy mirror` CLI command and `/api/mirror` API endpoints that
pre-populate the cache from various input sources: individual PURLs,
SBOM files (CycloneDX and SPDX), or full registry enumeration.
The mirror reuses the existing handler.Proxy.GetOrFetchArtifact()
pipeline so cached artifacts are identical to those fetched on demand.
A bounded worker pool controls download parallelism.
Metadata caching is opt-in via `cache_metadata: true` in config (or
PROXY_CACHE_METADATA=true). The mirror command always enables it. When
enabled, upstream metadata responses are stored for offline fallback
with ETag-based conditional revalidation.
New internal/mirror package with Source interface, PURLSource,
SBOMSource, RegistrySource, and async JobStore. New metadata_cache
database table for offline metadata serving.
* Fix all golangci-lint issues across the codebase
Resolve 77 lint issues reported by golangci-lint with gocritic, gocognit,
gocyclo, maintidx, dupl, mnd, unparam, ireturn, goconst, and errcheck
enabled. Net reduction of ~175 lines through shared helpers and
deduplication.
* Suppress staticcheck SA1019 for intentional deprecated field usage
The Storage.Path field is deprecated but still read for backwards
compatibility with existing configs that haven't migrated to the URL field.
All handler metadata and proxy requests were using http.DefaultClient directly,
bypassing any timeout or transport configuration. Added an HTTPClient field to
the Proxy struct with a 30-second default timeout, and updated every handler
to use it for upstream HTTP requests.
POST endpoints (/api/outdated, /api/bulk) now reject bodies over 1 MB
using http.MaxBytesReader. Upstream metadata reads (npm, pypi, composer,
nuget, pub) now use io.LimitReader capped at 50 MB to prevent OOM from
unexpectedly large responses.
Hides package versions published too recently from metadata responses,
giving the community time to spot malicious releases. Configurable
per-ecosystem and per-package with duration overrides. Supported for
npm, PyPI, pub.dev, and Composer.
