Watch
1
0
Fork
You've already forked pkg-proxy
1
mirror of https://github.com/git-pkgs/proxy.git synced 2026-11-09 15:49:16 -05:00
pkg-proxy/internal/handler/container_ns_test.go

801 lines
33 KiB
Go
Raw Permalink Normal View History

Route containerd ns mirror requests to configured registries (#405) * Route containerd ns mirror requests to configured registries containerd's hosts.toml mirrors append ?ns=<registry-host> to every request. The container handler ignored it, so a single _default mirror entry could not serve more than one registry. ns is a closed-world lookup key and is never dialed. Docker Hub aliases and the host of upstream.oci_default select the default route, hosts of upstream.oci entries select their named upstream. Unknown hosts return NAME_UNKNOWN so containerd falls back to its next host. Registry URLs with a path are not indexed, because ns names the registry at the root of a host. Host collisions log a warning and resolve deterministically. With ns the path is the verbatim upstream repository, the reserved upstream/ prefix is rejected, and repository prefix routes such as Homebrew's do not apply. Cache names match the unprefixed and upstream/{name}/ routes, so all routes share cached blobs, manifests and tag lists. ns is no longer forwarded on tag-list requests, and the pagination Link is rewritten per request instead of at store time so clients on different routes get links for their own route. Refs #303 * Document containerd ns mirroring Refs #303 * Version the tag-list cache identity for the raw-link format Tag-list rows now store the upstream Link verbatim and rewrite it per request. Rows written by earlier versions hold a Link already rewritten for one route. Under the unchanged key an older binary running next to this one (rolling update on shared Postgres, or a rollback) would serve the raw Link unrewritten, sending clients to the wrong route for the next page. A format marker in the key keeps the two apart at the cost of one cache miss per tag list after the upgrade. Adversarial review finding F1: tag-list cache rows hold raw upstream Links under unchanged keys, so an older binary serves them unrewritten. * Qualify the README claim about ns mirror coverage Registries whose URL has a path and hosts shared by two entries are not reachable through ns; point to the configuration guide for both rules. Adversarial review finding F3: README says ns covers every upstream.oci registry. * Redact registry URLs in ns index warnings Configured registry URLs may carry userinfo credentials, which the new startup warnings wrote to the log verbatim. Log them with the password masked, and say precisely what a path-prefixed default registry means: its host is not indexed, while the Docker Hub aliases still select it. Adversarial review finding F2: startup warnings print registry URLs unredacted and the default-registry message is misleading. * Keep non-default ports apart in the ns index Dropping 80 and 443 regardless of scheme turned https://host:80 and https://host into the same key, so a request for one of them could land on the other registry. Only the default port of the URL's scheme is optional now. A configured host is indexed with and without that port, because image references spell it either way; any other port has to match exactly, and the ns value is used as containerd sends it, apart from case and IPv6 brackets. Since a URL now yields two keys that can belong to different routes, the collision warning lists the owner per key. The handler test runs two registries on one host that differ only in the port. Tests can't bind 80 or 443, so a dialer maps those addresses to the fake servers. * Allow upstream/{name}/ together with ns when they agree containerd sends ns on every request to a mirror host, override_path or not. Per-registry mirrors pointing at /v2/upstream/{name} therefore arrive as upstream/{name}/...?ns=<registry>, and refusing every prefixed name under ns broke them, including the only way to mirror an upstream whose URL has a path. Such requests are now handled like the prefix route without ns, as long as ns names that upstream's own host under the same port rules as the index. Any other ns on a prefixed name is still NAME_UNKNOWN, so ns can't be used to create cache entries under another registry's name. * README: containerd config_path and how to check it A _default/hosts.toml does nothing while CRI has no hosts directory configured, so add the config.toml snippet for containerd 2.x and 1.x. To check the setup, look at containerd config dump and pull with crictl, then find the request in the proxy log; ctr --hosts-dir reads the directory on its own and would succeed even without config_path. Also mention that existing per-registry override_path entries keep working and that k3s generates the directory itself. * Let the prefix route take ns for aliases and mirrored registries With upstream.oci.hub pointing at registry-1.docker.io and a docker.io hosts.toml mirror on /v2/upstream/hub, containerd sends ns=docker.io, and the prefix check only accepted the upstream's own host. The same happens whenever the upstream is a mirror of the registry the nodes pull from, say an Artifactory remote for ghcr.io: ns carries ghcr.io, the upstream host is something else, and the pull got a 404. The prefix already decides where the content comes from and which cache entries are used, so ns can't redirect anything there. The check now only refuses an ns that contradicts the prefix, meaning a host the proxy knows that belongs to another route. The Docker Hub aliases count as one host, and a host the proxy doesn't know at all is accepted, since a client can only arrive at the prefix through its own mirror entry. * Take docker.io on every prefix route A Docker Hub mirror configured as a named upstream (mirror.gcr.io, an Artifactory remote) with a docker.io hosts.toml pointing at /v2/upstream/hub got a 404 again: docker.io is always in the index for the default route, so the check saw a known host of another route. That setup worked before this branch. Docker Hub repository names have exactly two path components, so docker.io/upstream/... can never be a real image, and accepting the aliases on every prefix route shadows nothing. While here: the collision warning claimed ns requests go to the other route, which is only true without the prefix, and the README said pulls for unknown registries always get a 404, which isn't the case for paths under upstream/{name}/. * Only take a foreign ns on a prefix route when it is configured A containerd _default mirror sends the same request for a pull of unconfigured.example/upstream/mirror/owner/app as a ghcr.io hosts.toml with override_path does for a mirror upstream. Accepting any host the proxy didn't know therefore let such a pull be answered with owner/app from the mirror upstream, a different image, instead of a 404 that lets containerd go to the registry the image actually names. The prefix route now accepts ns only for the upstream's own host, for Docker Hub, and for hosts listed in the new upstream.oci_mirrors setting. That setting is how an Artifactory remote of ghcr.io keeps working: upstream: oci_mirrors: ghcr: ["ghcr.io"] The listed hosts also select the upstream for unprefixed ns requests, which makes a mirror with a path in its URL reachable through the _default entry as well. Hosts from configured URLs still win over mirror entries; the proxy warns about such overlaps at startup. * Fix the docs sentence on which ns a prefix route takes configuration.md still said a prefix request with ns is only refused when ns names another route's host. Since the last commit it is the other way round: only the upstream's own host, its oci_mirrors entries and Docker Hub get through. * Note what a host accepted on a prefix route gives up Listing ghcr.io for an upstream (or the upstream's own host) still has the ambiguity from the wildcard case: ghcr.io/upstream/ghcr/owner/app lands on that upstream. That is the price of an explicit entry and fine in practice, but it should be written down. * Test that oci_mirrors entries stay with their own upstream Two gaps in the coverage of the last change: nothing failed if a host listed for one upstream was also accepted on another upstream's prefix, and nothing checked that the server actually hands upstream.oci_mirrors to the container handler. The handler test now has a second upstream that must refuse ghcr.io, and a small server test pulls through a mirror entry on both the prefix and the unprefixed route. * Log when a prefix route refuses an ns Setups with a ghcr.io hosts.toml pointing at a mirror upstream worked before this branch and now need an oci_mirrors entry. Without one, containerd gets a 404, quietly pulls from ghcr.io directly, and nobody notices the proxy is being skipped. The refusal is now logged as a warning that names the upstream and the ns and points at upstream.oci_mirrors. * Drop a check in validateOCIMirrors that could never fire user@ghcr.io parses with ghcr.io as its host, so the parsed.Host != host comparison already rejects it and the extra parsed.User test was dead. A test case for a host with credentials keeps that path covered. * Reject an oci_mirrors host that is listed for two upstreams With ghcr.io under both art and nexus, both prefix routes took ns=ghcr.io while unprefixed requests went to whichever name sorted first, with nothing but a startup warning. Config validation now refuses the second entry, comparing hosts without case and without the default :443. * Log each refused ns once per upstream and refuse overlong values The refusal warning fired on every request, and a node with a missing oci_mirrors entry asks for the manifest and each layer, so the same line piled up. Each upstream/ns pair is now logged once, for up to 1024 pairs; after that every refusal is logged again. ns comes straight from the client and can be as long as the request line allows, so remembering it as is would let anyone pin a lot of memory with a handful of requests. No registry host is longer than 259 characters (a 253-character name plus a port), so anything longer is refused right away, before it is logged or remembered. * Compare oci_mirrors hosts the way the handler does The duplicate check lowercased the host and trimmed :443, while the handler splits host and port, brackets IPv6 literals and drops an empty port. So ghcr.io and ghcr.io:, or [::1] and [::1]:, passed as different hosts in two upstreams and then collided in the handler. The check now builds the same key the handler uses. * Rewrap the oci_mirrors paragraphs in the configuration docs Two sentences had been dropped into the middle of wrapped lines, and the prefix-route paragraph named Docker Hub twice in a row. Same content, shorter lines, one mention. * Stop logging ns refusals once the set is full Past 1024 different upstream/ns pairs every refusal was logged again, so anyone could fill the set with made-up hosts and then flood the log at warning level. When the set is full the proxy now says so once and keeps quiet after that. A real misconfiguration shows up long before that, with the first pull. * Test the limit on remembered ns refusals Nothing covered what happens once 1024 pairs are remembered. The new test fills the set, then sends a known pair and two new ones: the proxy must log the limit notice exactly once, log none of the refusals and keep the set at its size.
2026-10-10 08:35:50 +02:00
package handler
import (
"bytes"
"context"
"crypto/sha256"
"encoding/hex"
"fmt"
"io"
"log/slog"
"net"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"sync"
"testing"
"time"
"github.com/git-pkgs/registries/fetch"
)
const (
nsTestBlob = "layer bytes"
nsTestProxyURL = "http://proxy.example.test"
)
// nsTestRegistry is a fake OCI registry serving one repository below an
// optional path prefix. It records every request URI it receives.
type nsTestRegistry struct {
*httptest.Server
repository string
mu sync.Mutex
requests []string
}
func newNSTestRegistry(t *testing.T, repository, pathPrefix string) *nsTestRegistry {
t.Helper()
registry := &nsTestRegistry{repository: repository}
manifest := nsTestManifest()
blobDigest := "sha256:" + sha256Hex(nsTestBlob)
manifestDigest := "sha256:" + sha256Hex(manifest)
base := pathPrefix + "/v2/" + repository
registry.Server = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
registry.mu.Lock()
registry.requests = append(registry.requests, r.URL.RequestURI())
registry.mu.Unlock()
switch r.URL.Path {
case base + "/manifests/latest", base + "/manifests/" + manifestDigest:
w.Header().Set("Content-Type", "application/vnd.oci.image.manifest.v1+json")
w.Header().Set("Docker-Content-Digest", manifestDigest)
_, _ = io.WriteString(w, manifest)
case base + "/blobs/" + blobDigest:
w.Header().Set("Content-Type", "application/octet-stream")
_, _ = io.WriteString(w, nsTestBlob)
case base + "/tags/list":
w.Header().Set("Content-Type", "application/json")
if r.URL.Query().Get("last") == "" {
w.Header().Set("Link", `<`+base+`/tags/list?last=1.0&n=1>; rel="next"`)
_, _ = io.WriteString(w, `{"name":"`+repository+`","tags":["1.0"]}`)
return
}
_, _ = io.WriteString(w, `{"name":"`+repository+`","tags":["2.0"]}`)
default:
http.NotFound(w, r)
}
}))
t.Cleanup(registry.Close)
return registry
}
func nsTestManifest() string {
return `{"schemaVersion":2,"layers":[{"digest":"sha256:` + sha256Hex(nsTestBlob) + `"}]}`
}
func (r *nsTestRegistry) host() string {
return strings.TrimPrefix(r.URL, "http://")
}
func (r *nsTestRegistry) requestCount() int {
r.mu.Lock()
defer r.mu.Unlock()
return len(r.requests)
}
func (r *nsTestRegistry) lastRequest() string {
r.mu.Lock()
defer r.mu.Unlock()
if len(r.requests) == 0 {
return ""
}
return r.requests[len(r.requests)-1]
}
// newNSTestHandler builds a container handler the way the server does, with a
// real fetcher so blob downloads reach the fake registries. Warnings logged
// while building the handler are written to the returned buffer.
func newNSTestHandler(t *testing.T, defaultURL string, named map[string]string) (http.Handler, *ContainerHandler, *bytes.Buffer) {
t.Helper()
proxy, _, _, _ := setupTestProxy(t)
logs := &bytes.Buffer{}
proxy.Logger = slog.New(slog.NewTextHandler(logs, nil))
client := &http.Client{}
proxy.HTTPClient = client
proxy.MetadataTTL = time.Hour
fetcher := fetch.NewFetcher(fetch.WithHTTPClient(client), fetch.WithMaxRetries(0))
proxy.Fetcher = fetcher
t.Cleanup(func() { _ = fetcher.Close() })
h := NewContainerHandlerWithRegistry(proxy, nsTestProxyURL, defaultURL, named)
return http.StripPrefix("/v2", h.Routes()), h, logs
}
func serveNS(routes http.Handler, target string) *httptest.ResponseRecorder {
response := httptest.NewRecorder()
routes.ServeHTTP(response, httptest.NewRequest(http.MethodGet, target, nil))
return response
}
func assertNameUnknown(t *testing.T, response *httptest.ResponseRecorder) {
t.Helper()
if response.Code != http.StatusNotFound {
t.Fatalf("status = %d, want 404: %s", response.Code, response.Body.String())
}
if !strings.Contains(response.Body.String(), `"NAME_UNKNOWN"`) {
t.Errorf("body = %s, want NAME_UNKNOWN error", response.Body.String())
}
}
func TestContainerHandler_NamespaceSelectsDefaultRegistry(t *testing.T) {
registry := newNSTestRegistry(t, "library/nginx", "")
// The default registry is a custom oci_default, so its host is only
// resolvable when the index is built from the final registry URL.
routes, _, _ := newNSTestHandler(t, registry.URL, nil)
for _, namespace := range []string{"docker.io", "index.docker.io", "registry-1.docker.io", registry.host()} {
t.Run(namespace, func(t *testing.T) {
response := serveNS(routes, "/v2/library/nginx/manifests/latest?ns="+namespace)
if response.Code != http.StatusOK {
t.Fatalf("status = %d, want 200: %s", response.Code, response.Body.String())
}
if got, want := registry.lastRequest(), "/v2/library/nginx/manifests/latest"; got != want {
t.Errorf("upstream request = %q, want %q", got, want)
}
})
}
}
func TestContainerHandler_NamespaceSelectsNamedRegistry(t *testing.T) {
fallback := newNSTestRegistry(t, "owner/app", "")
named := newNSTestRegistry(t, "owner/app", "")
routes, _, _ := newNSTestHandler(t, fallback.URL, map[string]string{"ghcr": named.URL})
digest := "sha256:" + sha256Hex(nsTestBlob)
manifest := serveNS(routes, "/v2/owner/app/manifests/latest?ns="+named.host())
if manifest.Code != http.StatusOK {
t.Fatalf("manifest status = %d, want 200: %s", manifest.Code, manifest.Body.String())
}
blob := serveNS(routes, "/v2/owner/app/blobs/"+digest+"?ns="+named.host())
if blob.Code != http.StatusOK {
t.Fatalf("blob status = %d, want 200: %s", blob.Code, blob.Body.String())
}
if got := blob.Body.String(); got != nsTestBlob {
t.Errorf("blob body = %q, want %q", got, nsTestBlob)
}
if got := named.requestCount(); got != 2 {
t.Errorf("named registry requests = %d, want 2", got)
}
if got := fallback.requestCount(); got != 0 {
t.Errorf("default registry requests = %d, want 0", got)
}
}
func TestContainerHandler_NamespaceRejectsUnresolvableRequests(t *testing.T) {
registry := newNSTestRegistry(t, "owner/app", "")
other := newNSTestRegistry(t, "owner/app", "")
routes, _, _ := newNSTestHandler(t, registry.URL, map[string]string{"ghcr": registry.URL, "quay": other.URL})
digest := "sha256:" + sha256Hex(nsTestBlob)
tests := map[string]string{
"unknown host manifest": "/v2/owner/app/manifests/latest?ns=quay.io",
"unknown host blob": "/v2/owner/app/blobs/" + digest + "?ns=quay.io",
"unknown host tags": "/v2/owner/app/tags/list?ns=quay.io",
"multiple ns values": "/v2/owner/app/manifests/latest?ns=docker.io&ns=" + registry.host(),
"prefix route with default host": "/v2/upstream/quay/owner/app/manifests/latest?ns=" + registry.host(),
"prefix route with other upstream": "/v2/upstream/ghcr/owner/app/blobs/" + digest + "?ns=" + other.host(),
"prefix route with unknown upstream": "/v2/upstream/nope/owner/app/manifests/latest?ns=" + registry.host(),
"prefix route without repository": "/v2/upstream/ghcr/manifests/latest?ns=" + registry.host(),
}
for name, target := range tests {
t.Run(name, func(t *testing.T) {
assertNameUnknown(t, serveNS(routes, target))
})
}
if got := registry.requestCount() + other.requestCount(); got != 0 {
t.Errorf("upstream requests = %d, want 0", got)
}
}
func TestContainerHandler_NamespaceAcceptsPrefixRouteForOwnHost(t *testing.T) {
// Per-registry containerd mirrors with override_path address the
// upstream/{name}/ prefix and still send ns. They must keep working and
// share the prefix route's cache entries.
registry := newNSTestRegistry(t, "owner/app", "")
routes, _, _ := newNSTestHandler(t, "", map[string]string{"ghcr": registry.URL})
digest := "sha256:" + sha256Hex(nsTestBlob)
for _, target := range []string{
"/v2/upstream/ghcr/owner/app/manifests/latest?ns=" + registry.host(),
"/v2/upstream/ghcr/owner/app/blobs/" + digest + "?ns=" + registry.host(),
} {
if response := serveNS(routes, target); response.Code != http.StatusOK {
t.Fatalf("%s status = %d: %s", target, response.Code, response.Body.String())
}
}
if got, want := registry.lastRequest(), "/v2/owner/app/blobs/"+digest; got != want {
t.Errorf("upstream request = %q, want %q", got, want)
}
warmed := registry.requestCount()
for _, target := range []string{
"/v2/upstream/ghcr/owner/app/manifests/latest",
"/v2/upstream/ghcr/owner/app/blobs/" + digest,
} {
if response := serveNS(routes, target); response.Code != http.StatusOK {
t.Fatalf("%s status = %d: %s", target, response.Code, response.Body.String())
}
}
if got := registry.requestCount(); got != warmed {
t.Errorf("upstream requests after prefix pulls without ns = %d, want %d (cache hits)", got, warmed)
}
tags := serveNS(routes, "/v2/upstream/ghcr/owner/app/tags/list?n=1&ns="+registry.host())
if tags.Code != http.StatusOK {
t.Fatalf("tags status = %d: %s", tags.Code, tags.Body.String())
}
wantLink := `<` + nsTestProxyURL + `/v2/upstream/ghcr/owner/app/tags/list?last=1.0&n=1&ns=` + url.QueryEscape(registry.host()) + `>; rel="next"`
if got := tags.Header().Get("Link"); got != wantLink {
t.Errorf("Link = %q, want %q", got, wantLink)
}
if got, want := registry.lastRequest(), "/v2/owner/app/tags/list?n=1"; got != want {
t.Errorf("upstream tags request = %q, want %q (ns must not be forwarded)", got, want)
}
}
func TestContainerHandler_NamespacePrefixRouteAcceptsDockerHubAliases(t *testing.T) {
// upstream.oci.hub points at Docker Hub and a docker.io hosts.toml mirror
// with override_path addresses /v2/upstream/hub, so containerd sends
// ns=docker.io. The dialer stands in for registry-1.docker.io.
hub := newNSTestRegistry(t, "library/nginx", "")
other := newNSTestRegistry(t, "library/nginx", "")
routes, h, _ := newNSTestHandler(t, "", map[string]string{
"hub": "http://registry-1.docker.io",
"quay": other.URL,
})
transport := &http.Transport{DialContext: func(ctx context.Context, network, addr string) (net.Conn, error) {
if addr != "registry-1.docker.io:80" {
return nil, fmt.Errorf("unexpected dial to %s", addr)
}
return (&net.Dialer{}).DialContext(ctx, network, hub.Listener.Addr().String())
}}
t.Cleanup(transport.CloseIdleConnections)
h.proxy.HTTPClient = &http.Client{Transport: transport}
for _, query := range []string{"?ns=docker.io", "?ns=index.docker.io", "?ns=registry-1.docker.io", "?ns=docker.io:443", ""} {
response := serveNS(routes, "/v2/upstream/hub/library/nginx/manifests/latest"+query)
if response.Code != http.StatusOK {
t.Fatalf("%q status = %d: %s", query, response.Code, response.Body.String())
}
}
if got := hub.requestCount(); got != 1 {
t.Errorf("hub requests = %d, want 1 (the rest are cache hits)", got)
}
// A known host of another route still contradicts the prefix.
assertNameUnknown(t, serveNS(routes, "/v2/upstream/hub/library/nginx/manifests/latest?ns="+other.host()))
if got := other.requestCount(); got != 0 {
t.Errorf("other registry requests = %d, want 0", got)
}
t.Run("Docker Hub mirror as named upstream", func(t *testing.T) {
// The upstream is a Docker Hub mirror on some other host (mirror.gcr.io,
// an Artifactory remote); nodes still pull docker.io/... through
// /v2/upstream/hub, so ns says docker.io.
mirror := newNSTestRegistry(t, "library/nginx", "")
routes, _, _ := newNSTestHandler(t, "", map[string]string{"hub": mirror.URL})
response := serveNS(routes, "/v2/upstream/hub/library/nginx/manifests/latest?ns=docker.io")
if response.Code != http.StatusOK {
t.Fatalf("status = %d: %s", response.Code, response.Body.String())
}
if got, want := mirror.lastRequest(), "/v2/library/nginx/manifests/latest"; got != want {
t.Errorf("upstream request = %q, want %q", got, want)
}
})
}
func TestContainerHandler_NamespacePrefixRouteRejectsUnknownHosts(t *testing.T) {
// A _default wildcard mirror turns a pull of
// unconfigured.example/upstream/mirror/owner/app into exactly this
// request. Serving it would hand out owner/app from the mirror upstream
// instead of letting containerd fall back to unconfigured.example.
mirror := newNSTestRegistry(t, "owner/app", "")
routes, h, logs := newNSTestHandler(t, "", map[string]string{"mirror": mirror.URL})
for _, namespace := range []string{"unconfigured.example", "ghcr.io"} {
assertNameUnknown(t, serveNS(routes, "/v2/upstream/mirror/owner/app/manifests/latest?ns="+namespace))
}
if got := mirror.requestCount(); got != 0 {
t.Errorf("upstream requests = %d, want 0", got)
}
// containerd swallows the 404, so the refusal has to show up in the log.
for _, want := range []string{"upstream.oci_mirrors", "upstream=mirror", "ns=ghcr.io"} {
if !strings.Contains(logs.String(), want) {
t.Errorf("logs missing %q:\n%s", want, logs.String())
}
}
// A node repeats the refused request for every pull; one line per
// upstream/ns pair is enough. An ns longer than any registry host is
// refused before it is logged or remembered.
assertNameUnknown(t, serveNS(routes, "/v2/upstream/mirror/owner/app/manifests/latest?ns=GHCR.io"))
long := strings.Repeat("a", 300) + ".example"
assertNameUnknown(t, serveNS(routes, "/v2/upstream/mirror/owner/app/manifests/latest?ns="+long))
if got := strings.Count(logs.String(), "upstream.oci_mirrors"); got != 2 {
t.Errorf("refusal warnings = %d, want 2 (one per upstream/ns pair):\n%s", got, logs.String())
}
if strings.Contains(logs.String(), "aaaa") {
t.Errorf("logs mention the overlong ns, want it refused silently:\n%s", logs.String())
}
if got := len(h.refusals); got != 2 {
t.Errorf("remembered refusals = %d, want 2", got)
}
}
func TestContainerHandler_NamespaceRefusalLogIsBounded(t *testing.T) {
mirror := newNSTestRegistry(t, "owner/app", "")
routes, h, logs := newNSTestHandler(t, "", map[string]string{"mirror": mirror.URL})
h.refusals = make(map[string]struct{}, maxReportedNamespaceRefusals)
for i := range maxReportedNamespaceRefusals {
h.refusals[fmt.Sprintf("mirror\x00filler-%d.example", i)] = struct{}{}
}
for _, namespace := range []string{"filler-0.example", "first.example", "second.example"} {
assertNameUnknown(t, serveNS(routes, "/v2/upstream/mirror/owner/app/manifests/latest?ns="+namespace))
}
if got := strings.Count(logs.String(), "further refusals are not logged"); got != 1 {
t.Errorf("limit notices = %d, want 1:\n%s", got, logs.String())
}
if strings.Contains(logs.String(), "refusing ns") {
t.Errorf("refusal logged past the limit:\n%s", logs.String())
}
if got := len(h.refusals); got != maxReportedNamespaceRefusals {
t.Errorf("remembered refusals = %d, want %d", got, maxReportedNamespaceRefusals)
}
}
func TestContainerHandler_NamespaceMirroredRegistries(t *testing.T) {
// The upstream mirrors ghcr.io, and upstream.oci_mirrors says so. Nodes
// keep pulling ghcr.io/..., either through a ghcr.io hosts.toml pointing
// at /v2/upstream/ghcr or through the _default wildcard mirror.
tests := map[string]string{
"plain mirror host": "",
"artifactory remote": "/artifactory/api/docker/ghcr-remote",
}
for name, pathPrefix := range tests {
t.Run(name, func(t *testing.T) {
mirror := newNSTestRegistry(t, "owner/app", pathPrefix)
routes, h, _ := newNSTestHandler(t, "", map[string]string{"ghcr": mirror.URL + pathPrefix})
h.SetMirroredRegistries(map[string][]string{"ghcr": {"ghcr.io"}})
for _, target := range []string{
"/v2/upstream/ghcr/owner/app/manifests/latest?ns=ghcr.io",
"/v2/upstream/ghcr/owner/app/manifests/latest?ns=GHCR.io:443",
"/v2/owner/app/manifests/latest?ns=ghcr.io",
} {
before := mirror.requestCount()
response := serveNS(routes, target)
if response.Code != http.StatusOK {
t.Fatalf("GET %s status = %d: %s", target, response.Code, response.Body.String())
}
// The manifest is cached after the first request, so only
// the first one may reach the upstream.
if before == 0 {
if got, want := mirror.lastRequest(), pathPrefix+"/v2/owner/app/manifests/latest"; got != want {
t.Errorf("upstream request = %q, want %q", got, want)
}
}
}
if got := mirror.requestCount(); got != 1 {
t.Errorf("upstream requests = %d, want 1 (all routes share the cache)", got)
}
assertNameUnknown(t, serveNS(routes, "/v2/upstream/ghcr/owner/app/manifests/latest?ns=quay.io"))
})
}
}
func TestContainerHandler_NamespaceMirrorHostsRankBelowConfiguredURLs(t *testing.T) {
proxy, _, _, _ := setupTestProxy(t)
logs := &bytes.Buffer{}
proxy.Logger = slog.New(slog.NewTextHandler(logs, nil))
h := NewContainerHandlerWithRegistry(proxy, nsTestProxyURL, "", map[string]string{
"ghcr": "https://ghcr.io",
"art": "https://artifactory.example/api/docker/ghcr-remote",
"other": "https://other.example",
})
h.SetMirroredRegistries(map[string][]string{"art": {"ghcr.io"}, "missing": {"quay.io"}})
if got := h.namespaces["ghcr.io"]; got != "ghcr" {
t.Errorf("index[ghcr.io] = %q, want ghcr (configured URL wins over a mirror entry)", got)
}
if !h.namespaceNamesPrefixUpstream("ghcr.io", "upstream/art/owner/app") {
t.Error("ns ghcr.io refused on upstream/art/, want accepted as its mirrored registry")
}
if h.namespaceNamesPrefixUpstream("ghcr.io", "upstream/other/owner/app") {
t.Error("ns ghcr.io accepted on upstream/other/, want refused: only art lists it")
}
if _, ok := h.namespaces["quay.io"]; ok {
t.Error("index has quay.io from a mirror entry without upstream")
}
for _, want := range []string{"ghcr.io=ghcr", "upstream=missing"} {
if !strings.Contains(logs.String(), want) {
t.Errorf("logs missing %q:\n%s", want, logs.String())
}
}
}
func TestContainerHandler_NamespaceSharesCacheWithOtherRoutes(t *testing.T) {
digest := "sha256:" + sha256Hex(nsTestBlob)
manifestDigest := "sha256:" + sha256Hex(nsTestManifest())
t.Run("named registry", func(t *testing.T) {
registry := newNSTestRegistry(t, "owner/app", "")
routes, _, _ := newNSTestHandler(t, "", map[string]string{"ghcr": registry.URL})
for _, target := range []string{
"/v2/upstream/ghcr/owner/app/blobs/" + digest,
"/v2/upstream/ghcr/owner/app/manifests/" + manifestDigest,
} {
if response := serveNS(routes, target); response.Code != http.StatusOK {
t.Fatalf("warm %s status = %d: %s", target, response.Code, response.Body.String())
}
}
warmed := registry.requestCount()
for _, target := range []string{
"/v2/owner/app/blobs/" + digest + "?ns=" + registry.host(),
"/v2/owner/app/manifests/" + manifestDigest + "?ns=" + registry.host(),
} {
if response := serveNS(routes, target); response.Code != http.StatusOK {
t.Fatalf("ns %s status = %d: %s", target, response.Code, response.Body.String())
}
}
if got := registry.requestCount(); got != warmed {
t.Errorf("upstream requests after ns pulls = %d, want %d (cache hits)", got, warmed)
}
})
t.Run("default registry", func(t *testing.T) {
registry := newNSTestRegistry(t, "library/nginx", "")
routes, _, _ := newNSTestHandler(t, registry.URL, nil)
for _, target := range []string{
"/v2/library/nginx/blobs/" + digest + "?ns=docker.io",
"/v2/library/nginx/manifests/" + manifestDigest + "?ns=docker.io",
} {
if response := serveNS(routes, target); response.Code != http.StatusOK {
t.Fatalf("warm %s status = %d: %s", target, response.Code, response.Body.String())
}
}
warmed := registry.requestCount()
for _, target := range []string{
"/v2/library/nginx/blobs/" + digest,
"/v2/library/nginx/manifests/" + manifestDigest,
} {
if response := serveNS(routes, target); response.Code != http.StatusOK {
t.Fatalf("unprefixed %s status = %d: %s", target, response.Code, response.Body.String())
}
}
if got := registry.requestCount(); got != warmed {
t.Errorf("upstream requests after unprefixed pulls = %d, want %d (cache hits)", got, warmed)
}
})
}
func TestContainerHandler_NamespaceTagsList(t *testing.T) {
registry := newNSTestRegistry(t, "owner/app", "")
routes, _, _ := newNSTestHandler(t, "", map[string]string{"ghcr": registry.URL})
// The prefix route fills the shared cache entry first.
prefixed := serveNS(routes, "/v2/upstream/ghcr/owner/app/tags/list?n=1")
if prefixed.Code != http.StatusOK {
t.Fatalf("prefixed status = %d: %s", prefixed.Code, prefixed.Body.String())
}
const wantPrefixedLink = `<` + nsTestProxyURL + `/v2/upstream/ghcr/owner/app/tags/list?last=1.0&n=1>; rel="next"`
if got := prefixed.Header().Get("Link"); got != wantPrefixedLink {
t.Errorf("prefixed Link = %q, want %q", got, wantPrefixedLink)
}
namespaced := serveNS(routes, "/v2/owner/app/tags/list?n=1&ns="+registry.host())
if namespaced.Code != http.StatusOK {
t.Fatalf("ns status = %d: %s", namespaced.Code, namespaced.Body.String())
}
if got := registry.requestCount(); got != 1 {
t.Errorf("upstream requests = %d, want 1 (ns request served from the shared cache)", got)
}
wantNSLink := `<` + nsTestProxyURL + `/v2/owner/app/tags/list?last=1.0&n=1&ns=` + url.QueryEscape(registry.host()) + `>; rel="next"`
link := namespaced.Header().Get("Link")
if link != wantNSLink {
t.Fatalf("ns Link = %q, want %q", link, wantNSLink)
}
next := serveNS(routes, strings.TrimPrefix(strings.SplitN(link, ">", 2)[0], "<"))
if next.Code != http.StatusOK {
t.Fatalf("next page status = %d: %s", next.Code, next.Body.String())
}
if got, want := next.Body.String(), `{"name":"owner/app","tags":["2.0"]}`; got != want {
t.Errorf("next page body = %q, want %q", got, want)
}
if got, want := registry.lastRequest(), "/v2/owner/app/tags/list?last=1.0&n=1"; got != want {
t.Errorf("upstream request = %q, want %q (ns must not be forwarded)", got, want)
}
}
func TestContainerHandler_NamespaceDefaultBypassesRepositoryPrefixRoutes(t *testing.T) {
hub := newNSTestRegistry(t, "homebrew/core/jq", "")
brew := newNSTestRegistry(t, "homebrew/core/jq", "")
routes, h, _ := newNSTestHandler(t, hub.URL, nil)
RegisterHomebrewArtifacts(h, brew.URL)
if response := serveNS(routes, "/v2/homebrew/core/jq/manifests/latest?ns=docker.io"); response.Code != http.StatusOK {
t.Fatalf("ns status = %d: %s", response.Code, response.Body.String())
}
if hub.requestCount() != 1 || brew.requestCount() != 0 {
t.Errorf("ns request reached hub=%d brew=%d, want hub=1 brew=0", hub.requestCount(), brew.requestCount())
}
if response := serveNS(routes, "/v2/homebrew/core/jq/manifests/latest"); response.Code != http.StatusOK {
t.Fatalf("unprefixed status = %d: %s", response.Code, response.Body.String())
}
if hub.requestCount() != 1 || brew.requestCount() != 1 {
t.Errorf("unprefixed request reached hub=%d brew=%d, want hub=1 brew=1", hub.requestCount(), brew.requestCount())
}
}
func TestContainerHandler_NamespaceMatchesConfiguredHosts(t *testing.T) {
// Each pair is a configured registry URL and an ns value containerd sends
// for an image reference on that registry. The scheme-default port may be
// spelled out or left out on either side; any other port must match.
matches := []struct {
configured string
namespace string
}{
{"https://GHCR.io", "ghcr.io"},
{"https://ghcr.io/", "GHCR.IO"},
{"http://[fd00::1]:5000", "[fd00::1]:5000"},
{"https://[fd00::1]", "[fd00::1]"},
{"https://[fd00::1]", "[fd00::1]:443"},
{"https://reg.example:80", "reg.example:80"},
{"https://reg.example:443", "reg.example"},
{"https://reg.example", "reg.example:443"},
{"http://reg.example:80", "reg.example"},
{"http://reg.example", "reg.example:80"},
{"http://reg.example:5000", "reg.example:5000"},
}
for _, tt := range matches {
t.Run(tt.configured+" "+tt.namespace, func(t *testing.T) {
h := NewContainerHandlerWithRegistry(nil, nsTestProxyURL, "", map[string]string{"lab": tt.configured})
registryURL, upstreamName, cacheName, ok := h.registryForNamespace(tt.namespace, "owner/app")
if !ok {
t.Fatalf("ns %q did not resolve for upstream %q", tt.namespace, tt.configured)
}
if registryURL != strings.TrimSuffix(tt.configured, "/") || upstreamName != "owner/app" || cacheName != "upstream/lab/owner/app" {
t.Errorf("route = (%q, %q, %q), want (%q, owner/app, upstream/lab/owner/app)",
registryURL, upstreamName, cacheName, tt.configured)
}
})
}
mismatches := []struct {
configured string
namespace string
}{
{"http://reg.example:5000", "reg.example:5001"},
{"https://reg.example:80", "reg.example"},
{"https://reg.example", "reg.example:80"},
{"http://reg.example:443", "reg.example"},
}
for _, tt := range mismatches {
t.Run("mismatch "+tt.configured+" "+tt.namespace, func(t *testing.T) {
h := NewContainerHandlerWithRegistry(nil, nsTestProxyURL, "", map[string]string{"lab": tt.configured})
if _, _, _, ok := h.registryForNamespace(tt.namespace, "owner/app"); ok {
t.Errorf("ns %q resolved for upstream %q, want no match", tt.namespace, tt.configured)
}
})
}
}
func TestContainerHandler_NamespaceKeepsNonDefaultPorts(t *testing.T) {
// Two registries on one host that differ only in the port, one of them on
// the scheme-default port. Tests cannot listen on ports 80 or 443, so the
// dialer maps those addresses to the fake registries.
onDefaultPort := newNSTestRegistry(t, "owner/app", "")
onOtherPort := newNSTestRegistry(t, "owner/app", "")
routes, h, _ := newNSTestHandler(t, "", map[string]string{
"std": "http://registry.example",
"alt": "http://registry.example:443",
})
endpoints := map[string]string{
"registry.example:80": onDefaultPort.Listener.Addr().String(),
"registry.example:443": onOtherPort.Listener.Addr().String(),
}
transport := &http.Transport{DialContext: func(ctx context.Context, network, addr string) (net.Conn, error) {
target, ok := endpoints[addr]
if !ok {
return nil, fmt.Errorf("unexpected dial to %s", addr)
}
return (&net.Dialer{}).DialContext(ctx, network, target)
}}
t.Cleanup(transport.CloseIdleConnections)
h.proxy.HTTPClient = &http.Client{Transport: transport}
// Every request below misses the cache: a different registry or a
// different endpoint than the request before it.
tests := []struct {
target string
wantDefault int
wantOther int
}{
{"/v2/owner/app/manifests/latest?ns=registry.example:80", 1, 0},
{"/v2/owner/app/manifests/latest?ns=registry.example:443", 1, 1},
{"/v2/owner/app/tags/list?ns=registry.example", 2, 1},
}
for _, tt := range tests {
response := serveNS(routes, tt.target)
if response.Code != http.StatusOK {
t.Fatalf("%s status = %d: %s", tt.target, response.Code, response.Body.String())
}
if onDefaultPort.requestCount() != tt.wantDefault || onOtherPort.requestCount() != tt.wantOther {
t.Errorf("after %s: requests default-port=%d other-port=%d, want %d/%d",
tt.target, onDefaultPort.requestCount(), onOtherPort.requestCount(), tt.wantDefault, tt.wantOther)
}
}
}
func TestContainerHandler_NamespaceHostCollisions(t *testing.T) {
proxy, _, _, _ := setupTestProxy(t)
logs := &bytes.Buffer{}
proxy.Logger = slog.New(slog.NewTextHandler(logs, nil))
h := NewContainerHandlerWithRegistry(proxy, nsTestProxyURL, "https://mirror.example", map[string]string{
"zeta": "https://shared.example",
"alpha": "https://Shared.example:443",
"mirror": "https://mirror.example",
"hub": "https://registry-1.docker.io",
})
tests := map[string]string{
"shared.example": "https://Shared.example:443",
"mirror.example": "https://mirror.example",
"registry-1.docker.io": "https://mirror.example",
}
for namespace, want := range tests {
registryURL, _, cacheName, ok := h.registryForNamespace(namespace, "owner/app")
if !ok || registryURL != want {
t.Errorf("ns %q resolved to (%q, %v), want %q", namespace, registryURL, ok, want)
}
if namespace == "shared.example" && cacheName != "upstream/alpha/owner/app" {
t.Errorf("ns %q cache name = %q, want upstream/alpha/owner/app", namespace, cacheName)
}
}
for _, upstream := range []string{"upstream=zeta", "upstream=mirror", "upstream=hub"} {
if !strings.Contains(logs.String(), upstream) {
t.Errorf("missing collision warning for %s in logs:\n%s", upstream, logs.String())
}
}
}
func TestContainerHandler_NamespaceCollisionWarningNamesEachOwner(t *testing.T) {
// r's two keys end up with two different owners; the warning has to
// name both, or an admin fixing one entry misses the other.
proxy, _, _, _ := setupTestProxy(t)
logs := &bytes.Buffer{}
proxy.Logger = slog.New(slog.NewTextHandler(logs, nil))
h := NewContainerHandlerWithRegistry(proxy, nsTestProxyURL, "", map[string]string{
"p": "http://h.example",
"q": "http://h.example:443",
"r": "https://h.example",
})
for namespace, want := range map[string]string{"h.example": "p", "h.example:80": "p", "h.example:443": "q"} {
if got := h.namespaces[namespace]; got != want {
t.Errorf("index[%q] = %q, want %q", namespace, got, want)
}
}
for _, want := range []string{"upstream=r", "h.example=p", "h.example:443=q"} {
if !strings.Contains(logs.String(), want) {
t.Errorf("logs missing %q:\n%s", want, logs.String())
}
}
}
func TestContainerHandler_NamespaceSkipsRegistryURLsWithPath(t *testing.T) {
registry := newNSTestRegistry(t, "owner/app", "/artifactory/api/docker/remote")
routes, _, logs := newNSTestHandler(t, "", map[string]string{
"art": registry.URL + "/artifactory/api/docker/remote",
})
assertNameUnknown(t, serveNS(routes, "/v2/owner/app/manifests/latest?ns="+registry.host()))
if got := registry.requestCount(); got != 0 {
t.Errorf("upstream requests via ns = %d, want 0", got)
}
if !strings.Contains(logs.String(), "upstream=art") {
t.Errorf("missing warning for path-prefixed upstream in logs:\n%s", logs.String())
}
// Per-registry mirrors with override_path reach it with ns attached.
if response := serveNS(routes, "/v2/upstream/art/owner/app/manifests/latest?ns="+registry.host()); response.Code != http.StatusOK {
t.Fatalf("prefix route with ns status = %d, want 200: %s", response.Code, response.Body.String())
}
if got := registry.requestCount(); got != 1 {
t.Errorf("upstream requests via prefix route with ns = %d, want 1", got)
}
if response := serveNS(routes, "/v2/upstream/art/owner/app/manifests/latest"); response.Code != http.StatusOK {
t.Fatalf("prefix route status = %d, want 200: %s", response.Code, response.Body.String())
}
t.Run("default registry", func(t *testing.T) {
mirror := newNSTestRegistry(t, "library/nginx", "/hub")
routes, _, logs := newNSTestHandler(t, mirror.URL+"/hub", nil)
assertNameUnknown(t, serveNS(routes, "/v2/library/nginx/manifests/latest?ns="+mirror.host()))
if got := mirror.requestCount(); got != 0 {
t.Errorf("upstream requests via ns host = %d, want 0", got)
}
if response := serveNS(routes, "/v2/library/nginx/manifests/latest?ns=docker.io"); response.Code != http.StatusOK {
t.Fatalf("ns=docker.io status = %d, want 200: %s", response.Code, response.Body.String())
}
if !strings.Contains(logs.String(), "default OCI registry") {
t.Errorf("missing warning for path-prefixed default registry in logs:\n%s", logs.String())
}
})
}
func TestContainerHandler_TagsListIgnoresLegacyCacheRows(t *testing.T) {
registry := newNSTestRegistry(t, "owner/app", "")
routes, h, _ := newNSTestHandler(t, "", map[string]string{"ghcr": registry.URL})
// Rows written before the raw-link format hold a Link already rewritten
// for the route that filled them. Seed one under the legacy identity.
query := url.Values{"n": {"1"}}
legacySum := sha256.Sum256([]byte(registry.URL + "\x00owner/app\x00" + query.Encode()))
legacyKey := hex.EncodeToString(legacySum[:])
if legacyKey == h.containerTagsCacheKey(registry.URL, "owner/app", query) {
t.Fatal("legacy and current tag-list cache keys are equal")
}
legacy := &cachedContainerTags{
body: []byte(`{"name":"owner/app","tags":["legacy"]}`),
contentType: contentTypeJSON,
link: `<` + nsTestProxyURL + `/v2/upstream/ghcr/owner/app/tags/list?last=legacy&n=1>; rel="next"`,
fetchedAt: time.Now(),
}
if err := h.storeContainerTags(context.Background(), legacyKey, legacy); err != nil {
t.Fatalf("store legacy tag list: %v", err)
}
response := serveNS(routes, "/v2/owner/app/tags/list?n=1&ns="+registry.host())
if response.Code != http.StatusOK {
t.Fatalf("status = %d: %s", response.Code, response.Body.String())
}
if got, want := response.Body.String(), `{"name":"owner/app","tags":["1.0"]}`; got != want {
t.Errorf("body = %q, want %q (legacy row must not be served)", got, want)
}
if got := registry.requestCount(); got != 1 {
t.Errorf("upstream requests = %d, want 1 (legacy row must not be served)", got)
}
}
func TestContainerHandler_NamespaceWarningsRedactCredentials(t *testing.T) {
proxy, _, _, _ := setupTestProxy(t)
logs := &bytes.Buffer{}
proxy.Logger = slog.New(slog.NewTextHandler(logs, nil))
NewContainerHandlerWithRegistry(proxy, nsTestProxyURL, "https://svc:s3cret@mirror.example/hub", map[string]string{
"art": "https://bot:hunter2@art.example/artifactory/api/docker/remote",
})
for _, secret := range []string{"s3cret", "hunter2"} {
if strings.Contains(logs.String(), secret) {
t.Errorf("logs contain credential %q:\n%s", secret, logs.String())
}
}
for _, want := range []string{
"svc:xxxxx@mirror.example/hub",
"bot:xxxxx@art.example/artifactory",
"Docker Hub aliases still select it",
} {
if !strings.Contains(logs.String(), want) {
t.Errorf("logs missing %q:\n%s", want, logs.String())
}
}
}