Watch
1
0
Fork
You've already forked pkg-proxy
1
mirror of https://github.com/git-pkgs/proxy.git synced 2026-11-09 07:39:19 -05:00
pkg-proxy/internal/retention/retention_test.go

244 lines
8.1 KiB
Go
Raw Permalink Normal View History

Add age-based retention for cached artifacts (#428) * Add age-based retention for cached artifacts Cached artifacts stayed forever unless storage.max_size pushed them out. storage.retention adds a default, per-ecosystem and per-package duration after which an artifact nobody downloaded gets evicted. Ecosystems opt in one at a time: a handler registers with retention.Default, and until it does, configuring that ecosystem (or one of its packages) fails validation, and the default doesn't touch it. This commit registers none, so nothing changes yet. Startup logs which ecosystems the default covers and which it doesn't. An artifact counts as expired when both its fetch time and its last access are older than the cutoff. Checking only the last access would evict a refetched artifact again right away, since clearing a record keeps its old access time and a cache miss doesn't record a hit. The sweep scans artifacts in id windows, so no single query holds the SQLite connection for long, and checks each row against its own rule. Expired records are cleared and their files go through pending_deletes instead of being deleted directly, because a buffered hit can make an artifact that's being served right now look expired. Buffered hits are flushed before each sweep, and one sweep clears at most 10,000 artifacts. Since a big first sweep can queue a lot, reclaim now keeps working through due entries for up to 30 seconds per tick instead of stopping after 100, and pending_deletes gets an index on queued_at. Also: - proxy_artifacts_evicted_total{reason, ecosystem} counts LRU and retention evictions. - gradle.build_cache.max_age accepts "7d", as its comment always said. Refs #306 * Document storage retention Describes the new storage.retention block, the order rules apply in, that the default only covers ecosystems that support retention yet, and what to expect from the first sweep over an old cache. Also lists the new eviction counter and notes that gradle max_age takes days. Refs #306 * Fix retention timing on Postgres and tighten the sweep On Postgres, fetched_at and last_accessed_at are TIMESTAMP columns without a zone. They hold the local time the proxy wrote, but lib/pq reads them back labelled UTC. East of UTC that makes every artifact look hours younger than it is, so the sweep evicted late: two hours in Berlin, nine in Tokyo. The candidates' times are now read back as local time, and the database tests run on both SQLite and Postgres, with one that sets the local zone to Asia/Tokyo. A few smaller things: - When a delete fails, reclaim stops for the rest of the minute instead of trying every queued path. Otherwise a storage outage turns into thirty seconds of failed deletes and warnings every minute. - If the buffered hits can't be written, the sweep skips that round. Running on stale access times could clear something that is being downloaded right now. - DefaultCanonical only accepts a package key when rebuilding it gives the same PURL. For alpine and deb it would otherwise produce keys that never match. - sweep_interval has to be at least a minute. - New tests cover the eviction counter and an artifact that gets a hit between the scan and the clear. - The docs no longer say nothing changes without retention, and they point out that the example's ecosystem and package rules only pass validation once those ecosystems support retention. Refs #306 * Reject qualifiers and subpaths in retention package keys A rule names a whole package. Qualifiers like ?type=pom were dropped on the way to the stored PURL and the rule quietly covered every file of the package. Now such a key fails validation. Refs #306 * Say when a package key doesn't match how packages are stored A package rule for an ecosystem that supports retention, written in a form its packages aren't stored under (pkg:npm/babel/core without the @), failed with "retention is not supported for npm packages". Once the ecosystem is enabled that is no longer true. Such a key now fails with "does not match how npm packages are stored" and points to the package rule forms in the docs. Keys for ecosystems without retention support keep the old message. Refs #306 * Note that re-running the mirror keeps packages cached A mirror run over a package that is still cached goes through the normal cache lookup and records a download, so the package's age starts over. The docs only said that mirrored packages age from the time they were mirrored. Refs #306 * Say when an ecosystem takes no package rules Some ecosystems will support retention without package rules, because nothing in their stored records names a package the way a PURL would (conan, helm). A package key for one of them used to fail with "does not match how conan packages are stored; see the package rule forms", which sends people looking for a form that doesn't exist. It now says package rules aren't supported there and points to the ecosystem rule. Refs #306 * Use helm as the example of an ecosystem without package rules Conan gets package rules once its handler stores archives under the recipe name, so the config test that shows the refusal now uses helm, whose cache records name the repository, not the chart. Refs #306 * Say that each cached file ages on its own The retention docs said each artifact, each version of a package, ages on its own. A version can have several cached files (a jar and its pom, several wheels, Conan sources and binaries), and each of them ages separately. Refs #306 * Clear an expired artifact and queue its file in one transaction The sweep cleared the record and queued the file for deletion as two separate writes. If queueing failed, or the process stopped in between, the file stayed in storage with no record pointing at it and no queue entry, and no later sweep would find it again. Both writes now share a transaction, so a failed queue rolls the clear back and the next sweep retries. Refs #306 * Show artifact evictions on the analytics page proxy_artifacts_evicted_total had no place on /ui/analytics. The Runtime card now lists evicted artifacts by reason and ecosystem, without the failure tint, and the coverage test records an eviction so it doesn't depend on test order. Refs #306 * Store artifact times in UTC fetched_at and last_accessed_at are now written in UTC, and the retention cutoffs are compared in UTC, the same convention #434 uses for the metadata cache. lib/pq sends a time with its offset and a Postgres TIMESTAMP keeps only the wall clock, so UTC on write is what reads back correctly. That replaces reinterpreting the times as local on read. Rows written before this still hold local time. East of UTC they look younger by the offset until they are rewritten, so retention evicts them a few hours late once; west of UTC that much early. Refs #306 --------- Co-authored-by: t <t@t>
2026-10-12 10:06:01 +02:00
package retention
import (
"slices"
"testing"
"time"
"github.com/git-pkgs/purl"
)
func TestSpecDBEcosystems(t *testing.T) {
tests := []struct {
key string
want []string
}{
{"npm", []string{"npm"}},
{"gem", []string{"gem", "rubygems"}},
{"composer", []string{"composer", "packagist"}},
{"alpine", []string{"alpine", "apk"}},
{"golang", []string{"golang"}},
{"deb", []string{"deb"}},
}
for _, tt := range tests {
t.Run(tt.key, func(t *testing.T) {
got := Spec{Key: tt.key}.dbEcosystems()
if !slices.Equal(got, tt.want) {
t.Errorf("dbEcosystems() = %v, want %v", got, tt.want)
}
})
}
}
func TestRegistryRegister(t *testing.T) {
reg := NewRegistry()
reg.Register(Spec{Key: "gem"})
reg.Register(Spec{Key: "npm"})
if got := reg.Keys(); !slices.Equal(got, []string{"gem", "npm"}) {
t.Errorf("Keys() = %v, want [gem npm]", got)
}
for _, value := range []string{"gem", "rubygems"} {
if key, ok := reg.KeyForDBEcosystem(value); !ok || key != "gem" {
t.Errorf("KeyForDBEcosystem(%q) = %q, %v, want gem, true", value, key, ok)
}
}
if _, ok := reg.KeyForDBEcosystem("pypi"); ok {
t.Error("KeyForDBEcosystem(pypi) found a key for an unregistered ecosystem")
}
if _, ok := reg.Lookup("pypi"); ok {
t.Error("Lookup(pypi) found an unregistered ecosystem")
}
}
func TestRegistryRegisterPanics(t *testing.T) {
tests := []struct {
name string
specs []Spec
}{
{"unknown key", []Spec{{Key: "not-an-ecosystem"}}},
Add age-based retention for cached artifacts (#428) * Add age-based retention for cached artifacts Cached artifacts stayed forever unless storage.max_size pushed them out. storage.retention adds a default, per-ecosystem and per-package duration after which an artifact nobody downloaded gets evicted. Ecosystems opt in one at a time: a handler registers with retention.Default, and until it does, configuring that ecosystem (or one of its packages) fails validation, and the default doesn't touch it. This commit registers none, so nothing changes yet. Startup logs which ecosystems the default covers and which it doesn't. An artifact counts as expired when both its fetch time and its last access are older than the cutoff. Checking only the last access would evict a refetched artifact again right away, since clearing a record keeps its old access time and a cache miss doesn't record a hit. The sweep scans artifacts in id windows, so no single query holds the SQLite connection for long, and checks each row against its own rule. Expired records are cleared and their files go through pending_deletes instead of being deleted directly, because a buffered hit can make an artifact that's being served right now look expired. Buffered hits are flushed before each sweep, and one sweep clears at most 10,000 artifacts. Since a big first sweep can queue a lot, reclaim now keeps working through due entries for up to 30 seconds per tick instead of stopping after 100, and pending_deletes gets an index on queued_at. Also: - proxy_artifacts_evicted_total{reason, ecosystem} counts LRU and retention evictions. - gradle.build_cache.max_age accepts "7d", as its comment always said. Refs #306 * Document storage retention Describes the new storage.retention block, the order rules apply in, that the default only covers ecosystems that support retention yet, and what to expect from the first sweep over an old cache. Also lists the new eviction counter and notes that gradle max_age takes days. Refs #306 * Fix retention timing on Postgres and tighten the sweep On Postgres, fetched_at and last_accessed_at are TIMESTAMP columns without a zone. They hold the local time the proxy wrote, but lib/pq reads them back labelled UTC. East of UTC that makes every artifact look hours younger than it is, so the sweep evicted late: two hours in Berlin, nine in Tokyo. The candidates' times are now read back as local time, and the database tests run on both SQLite and Postgres, with one that sets the local zone to Asia/Tokyo. A few smaller things: - When a delete fails, reclaim stops for the rest of the minute instead of trying every queued path. Otherwise a storage outage turns into thirty seconds of failed deletes and warnings every minute. - If the buffered hits can't be written, the sweep skips that round. Running on stale access times could clear something that is being downloaded right now. - DefaultCanonical only accepts a package key when rebuilding it gives the same PURL. For alpine and deb it would otherwise produce keys that never match. - sweep_interval has to be at least a minute. - New tests cover the eviction counter and an artifact that gets a hit between the scan and the clear. - The docs no longer say nothing changes without retention, and they point out that the example's ecosystem and package rules only pass validation once those ecosystems support retention. Refs #306 * Reject qualifiers and subpaths in retention package keys A rule names a whole package. Qualifiers like ?type=pom were dropped on the way to the stored PURL and the rule quietly covered every file of the package. Now such a key fails validation. Refs #306 * Say when a package key doesn't match how packages are stored A package rule for an ecosystem that supports retention, written in a form its packages aren't stored under (pkg:npm/babel/core without the @), failed with "retention is not supported for npm packages". Once the ecosystem is enabled that is no longer true. Such a key now fails with "does not match how npm packages are stored" and points to the package rule forms in the docs. Keys for ecosystems without retention support keep the old message. Refs #306 * Note that re-running the mirror keeps packages cached A mirror run over a package that is still cached goes through the normal cache lookup and records a download, so the package's age starts over. The docs only said that mirrored packages age from the time they were mirrored. Refs #306 * Say when an ecosystem takes no package rules Some ecosystems will support retention without package rules, because nothing in their stored records names a package the way a PURL would (conan, helm). A package key for one of them used to fail with "does not match how conan packages are stored; see the package rule forms", which sends people looking for a form that doesn't exist. It now says package rules aren't supported there and points to the ecosystem rule. Refs #306 * Use helm as the example of an ecosystem without package rules Conan gets package rules once its handler stores archives under the recipe name, so the config test that shows the refusal now uses helm, whose cache records name the repository, not the chart. Refs #306 * Say that each cached file ages on its own The retention docs said each artifact, each version of a package, ages on its own. A version can have several cached files (a jar and its pom, several wheels, Conan sources and binaries), and each of them ages separately. Refs #306 * Clear an expired artifact and queue its file in one transaction The sweep cleared the record and queued the file for deletion as two separate writes. If queueing failed, or the process stopped in between, the file stayed in storage with no record pointing at it and no queue entry, and no later sweep would find it again. Both writes now share a transaction, so a failed queue rolls the clear back and the next sweep retries. Refs #306 * Show artifact evictions on the analytics page proxy_artifacts_evicted_total had no place on /ui/analytics. The Runtime card now lists evicted artifacts by reason and ecosystem, without the failure tint, and the coverage test records an eviction so it doesn't depend on test order. Refs #306 * Store artifact times in UTC fetched_at and last_accessed_at are now written in UTC, and the retention cutoffs are compared in UTC, the same convention #434 uses for the metadata cache. lib/pq sends a time with its offset and a Postgres TIMESTAMP keeps only the wall clock, so UTC on write is what reads back correctly. That replaces reinterpreting the times as local on read. Rows written before this still hold local time. East of UTC they look younger by the offset until they are rewritten, so retention evicts them a few hours late once; west of UTC that much early. Refs #306 --------- Co-authored-by: t <t@t>
2026-10-12 10:06:01 +02:00
{"duplicate key", []Spec{{Key: "npm"}, {Key: "npm"}}},
{"overlapping ecosystem value", []Spec{{Key: "gem"}, {Key: "npm", ExtraDBEcosystems: []string{"rubygems"}}}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
reg := NewRegistry()
defer func() {
if recover() == nil {
t.Error("Register did not panic")
}
}()
for _, s := range tt.specs {
reg.Register(s)
}
})
}
}
func mustParse(t *testing.T, s string) *purl.PURL {
t.Helper()
p, err := purl.Parse(s)
if err != nil {
t.Fatalf("parsing %q: %v", s, err)
}
return p
}
func TestDefaultCanonical(t *testing.T) {
canonical := DefaultCanonical("npm")
got, ok := canonical(mustParse(t, "pkg:npm/%40babel/core"))
if !ok || got != "pkg:npm/%40babel/core" {
t.Errorf("scoped package = %q, %v, want pkg:npm/%%40babel/core, true", got, ok)
}
if got, ok := canonical(mustParse(t, "pkg:pypi/requests")); ok {
t.Errorf("other type accepted as %q", got)
}
if _, ok := canonical(nil); ok {
t.Error("nil PURL accepted")
}
// alpine stores pkg:apk/alpine/{name} and deb stores pkg:deb/{name}, so
// building their keys back from the full name gives a different package.
// The default refuses them rather than produce keys that never match.
for _, tt := range []struct{ key, purl string }{
{"alpine", "pkg:apk/alpine/curl"},
{"alpine", "pkg:apk/curl"},
{"deb", "pkg:deb/debian/curl"},
} {
if got, ok := DefaultCanonical(tt.key)(mustParse(t, tt.purl)); ok {
t.Errorf("DefaultCanonical(%q) accepted %s as %q", tt.key, tt.purl, got)
}
}
}
func TestRegistryCanonical(t *testing.T) {
reg := NewRegistry()
reg.Register(Spec{Key: "julia"})
reg.Register(Spec{Key: "npm", CanonicalPackage: DefaultCanonical("npm")})
if got, ok := reg.Canonical(mustParse(t, "pkg:npm/lodash")); !ok || got != "pkg:npm/lodash" {
t.Errorf("Canonical(npm) = %q, %v, want pkg:npm/lodash, true", got, ok)
}
if got, ok := reg.Canonical(mustParse(t, "pkg:cargo/serde")); ok {
t.Errorf("Canonical(cargo) = %q for an unregistered ecosystem", got)
}
}
func TestRulesFor(t *testing.T) {
rules := Rules{
Default: 30 * 24 * time.Hour,
Ecosystems: map[string]time.Duration{"npm": 14 * 24 * time.Hour, "maven": 0},
Packages: map[string]time.Duration{"pkg:npm/lodash": 0, "pkg:maven/a/b": 90 * 24 * time.Hour},
}
tests := []struct {
key, pkg string
want time.Duration
}{
{"npm", "pkg:npm/lodash", 0},
{"npm", "pkg:npm/left-pad", 14 * 24 * time.Hour},
{"maven", "pkg:maven/a/b", 90 * 24 * time.Hour},
{"maven", "pkg:maven/c/d", 0},
{"cargo", "pkg:cargo/serde", 30 * 24 * time.Hour},
}
for _, tt := range tests {
if got := rules.For(tt.key, tt.pkg); got != tt.want {
t.Errorf("For(%q, %q) = %v, want %v", tt.key, tt.pkg, got, tt.want)
}
}
}
func TestGenericRulesRequireExplicitConfiguration(t *testing.T) {
day := 24 * time.Hour
const pkg = "pkg:generic/jqlang%2Fjq"
tests := []struct {
name string
ecosystems map[string]time.Duration
packages map[string]time.Duration
want time.Duration
}{
{"global default ignored", nil, nil, 0},
{"ecosystem rule", map[string]time.Duration{"generic": 7 * day}, nil, 7 * day},
{"ecosystem disabled", map[string]time.Duration{"generic": 0}, nil, 0},
{"package only", nil, map[string]time.Duration{pkg: 14 * day}, 14 * day},
{"other package ignored", nil, map[string]time.Duration{"pkg:generic/other%2Frepo": day}, 0},
{"package overrides ecosystem", map[string]time.Duration{"generic": 7 * day}, map[string]time.Duration{pkg: 14 * day}, 14 * day},
{"package disables eviction", map[string]time.Duration{"generic": 7 * day}, map[string]time.Duration{pkg: 0}, 0},
{"package enables eviction", map[string]time.Duration{"generic": 0}, map[string]time.Duration{pkg: day}, day},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
rules := Rules{Default: 30 * day, Ecosystems: tt.ecosystems, Packages: tt.packages}
if got := rules.For("generic", pkg); got != tt.want {
t.Errorf("generic retention = %v, want %v", got, tt.want)
}
if got := rules.For("npm", "pkg:npm/lodash"); got != 30*day {
t.Errorf("npm default retention = %v, want 30d", got)
}
})
}
}
Add age-based retention for cached artifacts (#428) * Add age-based retention for cached artifacts Cached artifacts stayed forever unless storage.max_size pushed them out. storage.retention adds a default, per-ecosystem and per-package duration after which an artifact nobody downloaded gets evicted. Ecosystems opt in one at a time: a handler registers with retention.Default, and until it does, configuring that ecosystem (or one of its packages) fails validation, and the default doesn't touch it. This commit registers none, so nothing changes yet. Startup logs which ecosystems the default covers and which it doesn't. An artifact counts as expired when both its fetch time and its last access are older than the cutoff. Checking only the last access would evict a refetched artifact again right away, since clearing a record keeps its old access time and a cache miss doesn't record a hit. The sweep scans artifacts in id windows, so no single query holds the SQLite connection for long, and checks each row against its own rule. Expired records are cleared and their files go through pending_deletes instead of being deleted directly, because a buffered hit can make an artifact that's being served right now look expired. Buffered hits are flushed before each sweep, and one sweep clears at most 10,000 artifacts. Since a big first sweep can queue a lot, reclaim now keeps working through due entries for up to 30 seconds per tick instead of stopping after 100, and pending_deletes gets an index on queued_at. Also: - proxy_artifacts_evicted_total{reason, ecosystem} counts LRU and retention evictions. - gradle.build_cache.max_age accepts "7d", as its comment always said. Refs #306 * Document storage retention Describes the new storage.retention block, the order rules apply in, that the default only covers ecosystems that support retention yet, and what to expect from the first sweep over an old cache. Also lists the new eviction counter and notes that gradle max_age takes days. Refs #306 * Fix retention timing on Postgres and tighten the sweep On Postgres, fetched_at and last_accessed_at are TIMESTAMP columns without a zone. They hold the local time the proxy wrote, but lib/pq reads them back labelled UTC. East of UTC that makes every artifact look hours younger than it is, so the sweep evicted late: two hours in Berlin, nine in Tokyo. The candidates' times are now read back as local time, and the database tests run on both SQLite and Postgres, with one that sets the local zone to Asia/Tokyo. A few smaller things: - When a delete fails, reclaim stops for the rest of the minute instead of trying every queued path. Otherwise a storage outage turns into thirty seconds of failed deletes and warnings every minute. - If the buffered hits can't be written, the sweep skips that round. Running on stale access times could clear something that is being downloaded right now. - DefaultCanonical only accepts a package key when rebuilding it gives the same PURL. For alpine and deb it would otherwise produce keys that never match. - sweep_interval has to be at least a minute. - New tests cover the eviction counter and an artifact that gets a hit between the scan and the clear. - The docs no longer say nothing changes without retention, and they point out that the example's ecosystem and package rules only pass validation once those ecosystems support retention. Refs #306 * Reject qualifiers and subpaths in retention package keys A rule names a whole package. Qualifiers like ?type=pom were dropped on the way to the stored PURL and the rule quietly covered every file of the package. Now such a key fails validation. Refs #306 * Say when a package key doesn't match how packages are stored A package rule for an ecosystem that supports retention, written in a form its packages aren't stored under (pkg:npm/babel/core without the @), failed with "retention is not supported for npm packages". Once the ecosystem is enabled that is no longer true. Such a key now fails with "does not match how npm packages are stored" and points to the package rule forms in the docs. Keys for ecosystems without retention support keep the old message. Refs #306 * Note that re-running the mirror keeps packages cached A mirror run over a package that is still cached goes through the normal cache lookup and records a download, so the package's age starts over. The docs only said that mirrored packages age from the time they were mirrored. Refs #306 * Say when an ecosystem takes no package rules Some ecosystems will support retention without package rules, because nothing in their stored records names a package the way a PURL would (conan, helm). A package key for one of them used to fail with "does not match how conan packages are stored; see the package rule forms", which sends people looking for a form that doesn't exist. It now says package rules aren't supported there and points to the ecosystem rule. Refs #306 * Use helm as the example of an ecosystem without package rules Conan gets package rules once its handler stores archives under the recipe name, so the config test that shows the refusal now uses helm, whose cache records name the repository, not the chart. Refs #306 * Say that each cached file ages on its own The retention docs said each artifact, each version of a package, ages on its own. A version can have several cached files (a jar and its pom, several wheels, Conan sources and binaries), and each of them ages separately. Refs #306 * Clear an expired artifact and queue its file in one transaction The sweep cleared the record and queued the file for deletion as two separate writes. If queueing failed, or the process stopped in between, the file stayed in storage with no record pointing at it and no queue entry, and no later sweep would find it again. Both writes now share a transaction, so a failed queue rolls the clear back and the next sweep retries. Refs #306 * Show artifact evictions on the analytics page proxy_artifacts_evicted_total had no place on /ui/analytics. The Runtime card now lists evicted artifacts by reason and ecosystem, without the failure tint, and the coverage test records an eviction so it doesn't depend on test order. Refs #306 * Store artifact times in UTC fetched_at and last_accessed_at are now written in UTC, and the retention cutoffs are compared in UTC, the same convention #434 uses for the metadata cache. lib/pq sends a time with its offset and a Postgres TIMESTAMP keeps only the wall clock, so UTC on write is what reads back correctly. That replaces reinterpreting the times as local on read. Rows written before this still hold local time. East of UTC they look younger by the offset until they are rewritten, so retention evicts them a few hours late once; west of UTC that much early. Refs #306 --------- Co-authored-by: t <t@t>
2026-10-12 10:06:01 +02:00
func TestRulesMinPositive(t *testing.T) {
if got := (Rules{}).MinPositive(); got != 0 {
t.Errorf("empty rules MinPositive() = %v, want 0", got)
}
rules := Rules{
Default: 0,
Ecosystems: map[string]time.Duration{"npm": 14 * time.Hour, "maven": 0},
Packages: map[string]time.Duration{"pkg:npm/lodash": 2 * time.Hour},
}
if got := rules.MinPositive(); got != 2*time.Hour {
t.Errorf("MinPositive() = %v, want 2h", got)
}
}
Accept Swift source repository PURLs in the denylist and retention rules (#641) * Accept Swift source repository PURLs in the denylist and retention rules Closes #613. pkg:swift/github.com/apple/swift-nio, as SBOM tools and GHSA write Swift packages, is resolved to its registry identity through the upstream's /identifiers: - Denylist entries are resolved at startup, and the proxy refuses to start when one can't be, since a skipped entry would let the version through. - Retention keys are accepted at validation and kept pending. The retention loop resolves them before each sweep, and Swift isn't swept while any is pending, so a rule that keeps a package is never skipped. The mirror's source URL helper moves to the handler as SwiftSourceURL. * Resolve Swift source denylist entries in proxy mirror too serve resolved source-form Swift denylist entries through the upstream, but proxy mirror built its denylist directly, so such an entry never matched a mirror download. Both now use server.ConfigureDenylist, and neither runs when an entry can't be resolved. * Apply Swift source policies to every listed identity A Swift registry may list several identities for one source repository, but a denylist entry or retention rule naming the repository only covered the first. Denying pkg:swift/github.com/acme/widget@1.0.0 let acme.widget through when the registry listed other.widget first, and a never-evict rule cleared the cached apple.swift-nio archive when other.swift-nio came first. Both now cover every identity the registry lists. A retention rule that conflicts with another rule for any of them stays pending, with none applied. Mirroring a source PURL still downloads the first identity.
2026-10-31 12:34:05 +01:00
func TestRulesResolveSources(t *testing.T) {
rules := Rules{
Packages: map[string]time.Duration{"pkg:x/kept": 0},
Sources: map[string]map[string]time.Duration{"swift": {
"pkg:swift/github.com/a/one": 0,
"pkg:swift/github.com/a/two": time.Hour,
}},
}
if got := rules.MinPositive(); got != time.Hour {
t.Errorf("MinPositive = %v, want the pending rule's hour", got)
}
if err := rules.Resolve("swift", "pkg:swift/github.com/a/two", []string{"pkg:x/kept"}); err == nil {
t.Error("a rule resolving to a package another rule sets differently was accepted")
}
if !rules.Pending("swift") {
t.Fatal("conflicting rule no longer pending")
}
if err := rules.Resolve("swift", "pkg:swift/github.com/a/one", []string{"pkg:x/kept"}); err != nil {
t.Errorf("same duration for the same package: %v", err)
}
if err := rules.Resolve("swift", "pkg:swift/github.com/a/two", []string{"pkg:x/two"}); err != nil {
t.Fatal(err)
}
if rules.Pending("swift") || rules.Packages["pkg:x/two"] != time.Hour {
t.Errorf("after resolving: sources %v, packages %v", rules.Sources, rules.Packages)
}
}
// A source that resolves to several stored packages sets the rule for all
// of them, or, on a conflict with any, for none.
func TestRulesResolveSeveralPackages(t *testing.T) {
rules := Rules{
Packages: map[string]time.Duration{"pkg:x/b": time.Hour},
Sources: map[string]map[string]time.Duration{"swift": {"pkg:swift/github.com/a/one": 0}},
}
if err := rules.Resolve("swift", "pkg:swift/github.com/a/one", []string{"pkg:x/a", "pkg:x/b"}); err == nil {
t.Error("a rule conflicting for one of its packages was accepted")
}
if _, set := rules.Packages["pkg:x/a"]; set || !rules.Pending("swift") {
t.Fatalf("conflicting rule partly applied: sources %v, packages %v", rules.Sources, rules.Packages)
}
delete(rules.Packages, "pkg:x/b")
if err := rules.Resolve("swift", "pkg:swift/github.com/a/one", []string{"pkg:x/a", "pkg:x/b"}); err != nil {
t.Fatal(err)
}
if rules.Pending("swift") || len(rules.Packages) != 2 {
t.Errorf("after resolving: sources %v, packages %v", rules.Sources, rules.Packages)
}
}