Every npm and Composer download ran GetLocalArtifact, a full artifact
row lookup, even with hosted packages disabled. localVersions already
returns early in that case, so metadata does not list pushes, and
/generic/hosted/ is only served while hosting is enabled.
GetLocalArtifact now returns nil without a lookup when hosting is off.
Downloads then take the normal proxy path. An archive pushed earlier is
still answered from storage by that path and is never replaced by the
upstream copy; only cooldown now applies to it, as metadata no longer
lists it either. The hosted docs say so.
- Record a fetched artifact with its package and version in one
transaction that refuses to replace a pushed artifact, so a fetch that
finishes after a push keeps both the pushed file and its version's
integrity. Pushes and cache fills share this write, including the
compare-and-swap on the storage path that queues every replaced object.
- Don't open the stored file for ecosystems without a manifest.
- Derive multi-file versions from the file being named in the path.
- Validate DELETE paths like PUT paths, answering 400 for malformed ones.
- Parse ?sha256= with integrity.ParseHex.
- Record the package, version and artifact of a push in one transaction
that only inserts the artifact if no pushed copy exists. Of two
concurrent pushes of the same file exactly one wins, and the loser's
version integrity is rolled back with it.
- Check for an existing file before reading the request body, and send
Expect: 100-continue from proxy hosted push, so a refused re-push isn't
uploaded first.
- Leave an existing version record alone for multi-file ecosystems, so a
generic push doesn't reset a version cached from upstream.generic.
- Reserve the upstream.generic name "hosted" only while hosting is
enabled, and only intercept /generic/hosted/ then.
- Accept a file argument in proxy hosted delete for generic only.
- Mark multi-file ecosystems explicitly, and give parse a min/max range.
Hosted ecosystems are now entries in one table: each defines its package
names, how the stored filename is chosen, how the manifest is read and which
namespaces are served from pushes alone. npm and Composer move there
unchanged, and generic is added as a third entry.
Generic files are pushed to /api/hosted/generic/{name}/{version}/{file}
and served without a token from /generic/hosted/{name}/{version}/{file},
with the sha256 as ETag. They have no manifest, a version can hold several
files, and they never come from an upstream. "hosted" is reserved as an
upstream.generic name. DELETE with a file removes only that file.
For all hosted ecosystems:
- A second push of the same file is refused with 409. Clients pin pushed
files by version and often by hash, so a published file is no longer
silently replaced. Delete it first to push it again.
- PUT takes an optional ?sha256= and refuses bytes that don't match.
proxy hosted push always sends it.
Versions of generic files carry no version-level integrity, since it could
only describe one of their files; each file is checked against its own
sha256.
Refs #133, #134, #535
The manifest lookup took the first composer.json at the root or inside a top directory, in zip order. Dist archives keep composer.json files for tests and examples, so tests/composer.json stored before the root file refused the push with a name mismatch. The root composer.json now always wins. Without one, the manifest inside a top directory is used only when there is exactly one; more than one refuses the archive as ambiguous.
Refs #134
checkCache answered nil both when a stored object couldn't be opened and after an integrity failure had cleared the record. Download handlers then fetched the upstream archive for the same version and filename, and recording it turned the pushed row into an upstream one without its manifest and queued the pushed object for deletion. One storage error while downloading a patched version served the unpatched upstream archive and dropped the push.
checkCache now looks at the record on a miss. For a pushed artifact it returns ErrHostedArtifactUnavailable when the object is recorded but couldn't be served, and ErrHostedArtifactMissing when no object is recorded any more. Every path that would fetch upstream goes through it, so none of them can replace the pushed artifact. Downloads answer 503 and 404 for the two.
Refs #133
Resolvers take a version's dependencies from the registry metadata, not from the archive, so listing a pushed version needs its manifest. A push now reads package.json from the top directory of an npm tarball, or composer.json from the root or the single top directory of a Composer zip, and keeps it in the new artifacts.manifest column (migration 013_add_artifact_manifest).
The manifest has to name the pushed package and version. A Composer manifest may leave the version out, and its name is compared case-insensitively. An archive that can't be read or whose manifest names something else is refused with 400 and removed from storage.
The version's integrity is now the archive's sha512, which npm clients expect in dist.integrity.
Refs #134
With hosted.enabled, PUT /api/hosted/{ecosystem}/{name}/{version} stores the request body as the archive for that version, marked local. It replaces any cached upstream copy and is served from the cache without contacting the upstream. GET /api/hosted/{ecosystem}/{name} lists what was pushed and DELETE removes a version again, after which clients get the upstream archive if there is one. Pushing and deleting need the bearer token; listing is open like the rest of the API.
Download handlers look archives up by the filename clients request, so a push is stored under the name its ecosystem derives from name and version: {name}-{version}.tgz for npm, without the scope, and {package}-{version}.zip for Composer. Other ecosystems are refused for now.
The Composer download handler resolves the upstream metadata before it looks at the cache and answers 404 for versions the upstream doesn't list. It now serves a pushed archive first. Only artifacts marked local take that path, so cached upstream archives still go through the metadata and commit checks.
The version's integrity is set to the pushed bytes, so the cache's integrity check verifies them rather than an upstream digest. Names and versions with path separators, dot segments or control characters are refused before anything is stored.
Refs #133
