mirrors/pkg-proxy
Watch
1
0
Fork
You've already forked pkg-proxy
 1
mirror of https://github.com/git-pkgs/proxy.git synced 2026-11-09 07:39:19 -05:00
Code Issues Releases Packages Wiki Activity
pkg-proxy/internal/handler/container_ns_test.go
pinguinfuss 1559be63e2
 Route containerd ns mirror requests to configured registries (#405) 
* Route containerd ns mirror requests to configured registries

containerd's hosts.toml mirrors append ?ns=<registry-host> to every
request. The container handler ignored it, so a single _default mirror
entry could not serve more than one registry.

ns is a closed-world lookup key and is never dialed. Docker Hub aliases
and the host of upstream.oci_default select the default route, hosts of
upstream.oci entries select their named upstream. Unknown hosts return
NAME_UNKNOWN so containerd falls back to its next host. Registry URLs
with a path are not indexed, because ns names the registry at the root
of a host. Host collisions log a warning and resolve deterministically.

With ns the path is the verbatim upstream repository, the reserved
upstream/ prefix is rejected, and repository prefix routes such as
Homebrew's do not apply. Cache names match the unprefixed and
upstream/{name}/ routes, so all routes share cached blobs, manifests and
tag lists. ns is no longer forwarded on tag-list requests, and the
pagination Link is rewritten per request instead of at store time so
clients on different routes get links for their own route.

Refs #303

* Document containerd ns mirroring

Refs #303

* Version the tag-list cache identity for the raw-link format

Tag-list rows now store the upstream Link verbatim and rewrite it per
request. Rows written by earlier versions hold a Link already rewritten
for one route. Under the unchanged key an older binary running next to
this one (rolling update on shared Postgres, or a rollback) would serve
the raw Link unrewritten, sending clients to the wrong route for the
next page. A format marker in the key keeps the two apart at the cost of
one cache miss per tag list after the upgrade.

Adversarial review finding F1: tag-list cache rows hold raw upstream
Links under unchanged keys, so an older binary serves them unrewritten.

* Qualify the README claim about ns mirror coverage

Registries whose URL has a path and hosts shared by two entries are not
reachable through ns; point to the configuration guide for both rules.

Adversarial review finding F3: README says ns covers every upstream.oci
registry.

* Redact registry URLs in ns index warnings

Configured registry URLs may carry userinfo credentials, which the new
startup warnings wrote to the log verbatim. Log them with the password
masked, and say precisely what a path-prefixed default registry means:
its host is not indexed, while the Docker Hub aliases still select it.

Adversarial review finding F2: startup warnings print registry URLs
unredacted and the default-registry message is misleading.

* Keep non-default ports apart in the ns index

Dropping 80 and 443 regardless of scheme turned https://host:80 and
https://host into the same key, so a request for one of them could land
on the other registry. Only the default port of the URL's scheme is
optional now. A configured host is indexed with and without that port,
because image references spell it either way; any other port has to
match exactly, and the ns value is used as containerd sends it, apart
from case and IPv6 brackets.

Since a URL now yields two keys that can belong to different routes,
the collision warning lists the owner per key.

The handler test runs two registries on one host that differ only in
the port. Tests can't bind 80 or 443, so a dialer maps those addresses
to the fake servers.

* Allow upstream/{name}/ together with ns when they agree

containerd sends ns on every request to a mirror host, override_path or
not. Per-registry mirrors pointing at /v2/upstream/{name} therefore
arrive as upstream/{name}/...?ns=<registry>, and refusing every prefixed
name under ns broke them, including the only way to mirror an upstream
whose URL has a path.

Such requests are now handled like the prefix route without ns, as long
as ns names that upstream's own host under the same port rules as the
index. Any other ns on a prefixed name is still NAME_UNKNOWN, so ns
can't be used to create cache entries under another registry's name.

* README: containerd config_path and how to check it

A _default/hosts.toml does nothing while CRI has no hosts directory
configured, so add the config.toml snippet for containerd 2.x and 1.x.
To check the setup, look at containerd config dump and pull with
crictl, then find the request in the proxy log; ctr --hosts-dir reads
the directory on its own and would succeed even without config_path.
Also mention that existing per-registry override_path entries keep
working and that k3s generates the directory itself.

* Let the prefix route take ns for aliases and mirrored registries

With upstream.oci.hub pointing at registry-1.docker.io and a docker.io
hosts.toml mirror on /v2/upstream/hub, containerd sends ns=docker.io,
and the prefix check only accepted the upstream's own host. The same
happens whenever the upstream is a mirror of the registry the nodes
pull from, say an Artifactory remote for ghcr.io: ns carries ghcr.io,
the upstream host is something else, and the pull got a 404.

The prefix already decides where the content comes from and which
cache entries are used, so ns can't redirect anything there. The check
now only refuses an ns that contradicts the prefix, meaning a host the
proxy knows that belongs to another route. The Docker Hub aliases count
as one host, and a host the proxy doesn't know at all is accepted,
since a client can only arrive at the prefix through its own mirror
entry.

* Take docker.io on every prefix route

A Docker Hub mirror configured as a named upstream (mirror.gcr.io, an
Artifactory remote) with a docker.io hosts.toml pointing at
/v2/upstream/hub got a 404 again: docker.io is always in the index for
the default route, so the check saw a known host of another route. That
setup worked before this branch. Docker Hub repository names have
exactly two path components, so docker.io/upstream/... can never be a
real image, and accepting the aliases on every prefix route shadows
nothing.

While here: the collision warning claimed ns requests go to the other
route, which is only true without the prefix, and the README said pulls
for unknown registries always get a 404, which isn't the case for paths
under upstream/{name}/.

* Only take a foreign ns on a prefix route when it is configured

A containerd _default mirror sends the same request for a pull of
unconfigured.example/upstream/mirror/owner/app as a ghcr.io hosts.toml
with override_path does for a mirror upstream. Accepting any host the
proxy didn't know therefore let such a pull be answered with owner/app
from the mirror upstream, a different image, instead of a 404 that lets
containerd go to the registry the image actually names.

The prefix route now accepts ns only for the upstream's own host, for
Docker Hub, and for hosts listed in the new upstream.oci_mirrors setting.
That setting is how an Artifactory remote of ghcr.io keeps working:

    upstream:
      oci_mirrors:
        ghcr: ["ghcr.io"]

The listed hosts also select the upstream for unprefixed ns requests,
which makes a mirror with a path in its URL reachable through the
_default entry as well. Hosts from configured URLs still win over
mirror entries; the proxy warns about such overlaps at startup.

* Fix the docs sentence on which ns a prefix route takes

configuration.md still said a prefix request with ns is only refused
when ns names another route's host. Since the last commit it is the
other way round: only the upstream's own host, its oci_mirrors entries
and Docker Hub get through.

* Note what a host accepted on a prefix route gives up

Listing ghcr.io for an upstream (or the upstream's own host) still has
the ambiguity from the wildcard case: ghcr.io/upstream/ghcr/owner/app
lands on that upstream. That is the price of an explicit entry and
fine in practice, but it should be written down.

* Test that oci_mirrors entries stay with their own upstream

Two gaps in the coverage of the last change: nothing failed if a host
listed for one upstream was also accepted on another upstream's prefix,
and nothing checked that the server actually hands upstream.oci_mirrors
to the container handler. The handler test now has a second upstream
that must refuse ghcr.io, and a small server test pulls through a
mirror entry on both the prefix and the unprefixed route.

* Log when a prefix route refuses an ns

Setups with a ghcr.io hosts.toml pointing at a mirror upstream worked
before this branch and now need an oci_mirrors entry. Without one,
containerd gets a 404, quietly pulls from ghcr.io directly, and nobody
notices the proxy is being skipped. The refusal is now logged as a
warning that names the upstream and the ns and points at
upstream.oci_mirrors.

* Drop a check in validateOCIMirrors that could never fire

user@ghcr.io parses with ghcr.io as its host, so the parsed.Host != host
comparison already rejects it and the extra parsed.User test was dead.
A test case for a host with credentials keeps that path covered.

* Reject an oci_mirrors host that is listed for two upstreams

With ghcr.io under both art and nexus, both prefix routes took ns=ghcr.io
while unprefixed requests went to whichever name sorted first, with
nothing but a startup warning. Config validation now refuses the second
entry, comparing hosts without case and without the default :443.

* Log each refused ns once per upstream and refuse overlong values

The refusal warning fired on every request, and a node with a missing
oci_mirrors entry asks for the manifest and each layer, so the same line
piled up. Each upstream/ns pair is now logged once, for up to 1024
pairs; after that every refusal is logged again.

ns comes straight from the client and can be as long as the request
line allows, so remembering it as is would let anyone pin a lot of
memory with a handful of requests. No registry host is longer than 259
characters (a 253-character name plus a port), so anything longer is
refused right away, before it is logged or remembered.

* Compare oci_mirrors hosts the way the handler does

The duplicate check lowercased the host and trimmed :443, while the
handler splits host and port, brackets IPv6 literals and drops an empty
port. So ghcr.io and ghcr.io:, or [::1] and [::1]:, passed as different
hosts in two upstreams and then collided in the handler. The check now
builds the same key the handler uses.

* Rewrap the oci_mirrors paragraphs in the configuration docs

Two sentences had been dropped into the middle of wrapped lines, and the
prefix-route paragraph named Docker Hub twice in a row. Same content,
shorter lines, one mention.

* Stop logging ns refusals once the set is full

Past 1024 different upstream/ns pairs every refusal was logged again,
so anyone could fill the set with made-up hosts and then flood the log
at warning level. When the set is full the proxy now says so once and
keeps quiet after that. A real misconfiguration shows up long before
that, with the first pull.

* Test the limit on remembered ns refusals

Nothing covered what happens once 1024 pairs are remembered. The new
test fills the set, then sends a known pair and two new ones: the proxy
must log the limit notice exactly once, log none of the refusals and
keep the set at its size.
2026-10-10 07:35:50 +01:00

801 lines
33 KiB
Go
Raw Permalink Blame History

package handler
import (
"bytes"
"context"
"crypto/sha256"
"encoding/hex"
"fmt"
"io"
"log/slog"
"net"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"sync"
"testing"
"time"
"github.com/git-pkgs/registries/fetch"
)
const (
nsTestBlob = "layer bytes"
nsTestProxyURL = "http://proxy.example.test"
)
// nsTestRegistry is a fake OCI registry serving one repository below an
// optional path prefix. It records every request URI it receives.
type nsTestRegistry struct {
*httptest.Server
repository string
mu sync.Mutex
requests []string
}
func newNSTestRegistry(t *testing.T, repository, pathPrefix string) *nsTestRegistry {
t.Helper()
registry := &nsTestRegistry{repository: repository}
manifest := nsTestManifest()
blobDigest := "sha256:" + sha256Hex(nsTestBlob)
manifestDigest := "sha256:" + sha256Hex(manifest)
base := pathPrefix + "/v2/" + repository
registry.Server = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
registry.mu.Lock()
registry.requests = append(registry.requests, r.URL.RequestURI())
registry.mu.Unlock()
switch r.URL.Path {
case base + "/manifests/latest", base + "/manifests/" + manifestDigest:
w.Header().Set("Content-Type", "application/vnd.oci.image.manifest.v1+json")
w.Header().Set("Docker-Content-Digest", manifestDigest)
_, _ = io.WriteString(w, manifest)
case base + "/blobs/" + blobDigest:
w.Header().Set("Content-Type", "application/octet-stream")
_, _ = io.WriteString(w, nsTestBlob)
case base + "/tags/list":
w.Header().Set("Content-Type", "application/json")
if r.URL.Query().Get("last") == "" {
w.Header().Set("Link", `<`+base+`/tags/list?last=1.0&n=1>; rel="next"`)
_, _ = io.WriteString(w, `{"name":"`+repository+`","tags":["1.0"]}`)
return
}
_, _ = io.WriteString(w, `{"name":"`+repository+`","tags":["2.0"]}`)
default:
http.NotFound(w, r)
}
}))
t.Cleanup(registry.Close)
return registry
}
func nsTestManifest() string {
return `{"schemaVersion":2,"layers":[{"digest":"sha256:` + sha256Hex(nsTestBlob) + `"}]}`
}
func (r *nsTestRegistry) host() string {
return strings.TrimPrefix(r.URL, "http://")
}
func (r *nsTestRegistry) requestCount() int {
r.mu.Lock()
defer r.mu.Unlock()
return len(r.requests)
}
func (r *nsTestRegistry) lastRequest() string {
r.mu.Lock()
defer r.mu.Unlock()
if len(r.requests) == 0 {
return ""
}
return r.requests[len(r.requests)-1]
}
// newNSTestHandler builds a container handler the way the server does, with a
// real fetcher so blob downloads reach the fake registries. Warnings logged
// while building the handler are written to the returned buffer.
func newNSTestHandler(t *testing.T, defaultURL string, named map[string]string) (http.Handler, *ContainerHandler, *bytes.Buffer) {
t.Helper()
proxy, _, _, _ := setupTestProxy(t)
logs := &bytes.Buffer{}
proxy.Logger = slog.New(slog.NewTextHandler(logs, nil))
client := &http.Client{}
proxy.HTTPClient = client
proxy.MetadataTTL = time.Hour
fetcher := fetch.NewFetcher(fetch.WithHTTPClient(client), fetch.WithMaxRetries(0))
proxy.Fetcher = fetcher
t.Cleanup(func() { _ = fetcher.Close() })
h := NewContainerHandlerWithRegistry(proxy, nsTestProxyURL, defaultURL, named)
return http.StripPrefix("/v2", h.Routes()), h, logs
}
func serveNS(routes http.Handler, target string) *httptest.ResponseRecorder {
response := httptest.NewRecorder()
routes.ServeHTTP(response, httptest.NewRequest(http.MethodGet, target, nil))
return response
}
func assertNameUnknown(t *testing.T, response *httptest.ResponseRecorder) {
t.Helper()
if response.Code != http.StatusNotFound {
t.Fatalf("status = %d, want 404: %s", response.Code, response.Body.String())
}
if !strings.Contains(response.Body.String(), `"NAME_UNKNOWN"`) {
t.Errorf("body = %s, want NAME_UNKNOWN error", response.Body.String())
}
}
func TestContainerHandler_NamespaceSelectsDefaultRegistry(t *testing.T) {
registry := newNSTestRegistry(t, "library/nginx", "")
// The default registry is a custom oci_default, so its host is only
// resolvable when the index is built from the final registry URL.
routes, _, _ := newNSTestHandler(t, registry.URL, nil)
for _, namespace := range []string{"docker.io", "index.docker.io", "registry-1.docker.io", registry.host()} {
t.Run(namespace, func(t *testing.T) {
response := serveNS(routes, "/v2/library/nginx/manifests/latest?ns="+namespace)
if response.Code != http.StatusOK {
t.Fatalf("status = %d, want 200: %s", response.Code, response.Body.String())
}
if got, want := registry.lastRequest(), "/v2/library/nginx/manifests/latest"; got != want {
t.Errorf("upstream request = %q, want %q", got, want)
}
})
}
}
func TestContainerHandler_NamespaceSelectsNamedRegistry(t *testing.T) {
fallback := newNSTestRegistry(t, "owner/app", "")
named := newNSTestRegistry(t, "owner/app", "")
routes, _, _ := newNSTestHandler(t, fallback.URL, map[string]string{"ghcr": named.URL})
digest := "sha256:" + sha256Hex(nsTestBlob)
manifest := serveNS(routes, "/v2/owner/app/manifests/latest?ns="+named.host())
if manifest.Code != http.StatusOK {
t.Fatalf("manifest status = %d, want 200: %s", manifest.Code, manifest.Body.String())
}
blob := serveNS(routes, "/v2/owner/app/blobs/"+digest+"?ns="+named.host())
if blob.Code != http.StatusOK {
t.Fatalf("blob status = %d, want 200: %s", blob.Code, blob.Body.String())
}
if got := blob.Body.String(); got != nsTestBlob {
t.Errorf("blob body = %q, want %q", got, nsTestBlob)
}
if got := named.requestCount(); got != 2 {
t.Errorf("named registry requests = %d, want 2", got)
}
if got := fallback.requestCount(); got != 0 {
t.Errorf("default registry requests = %d, want 0", got)
}
}
func TestContainerHandler_NamespaceRejectsUnresolvableRequests(t *testing.T) {
registry := newNSTestRegistry(t, "owner/app", "")
other := newNSTestRegistry(t, "owner/app", "")
routes, _, _ := newNSTestHandler(t, registry.URL, map[string]string{"ghcr": registry.URL, "quay": other.URL})
digest := "sha256:" + sha256Hex(nsTestBlob)
tests := map[string]string{
"unknown host manifest": "/v2/owner/app/manifests/latest?ns=quay.io",
"unknown host blob": "/v2/owner/app/blobs/" + digest + "?ns=quay.io",
"unknown host tags": "/v2/owner/app/tags/list?ns=quay.io",
"multiple ns values": "/v2/owner/app/manifests/latest?ns=docker.io&ns=" + registry.host(),
"prefix route with default host": "/v2/upstream/quay/owner/app/manifests/latest?ns=" + registry.host(),
"prefix route with other upstream": "/v2/upstream/ghcr/owner/app/blobs/" + digest + "?ns=" + other.host(),
"prefix route with unknown upstream": "/v2/upstream/nope/owner/app/manifests/latest?ns=" + registry.host(),
"prefix route without repository": "/v2/upstream/ghcr/manifests/latest?ns=" + registry.host(),
}
for name, target := range tests {
t.Run(name, func(t *testing.T) {
assertNameUnknown(t, serveNS(routes, target))
})
}
if got := registry.requestCount() + other.requestCount(); got != 0 {
t.Errorf("upstream requests = %d, want 0", got)
}
}
func TestContainerHandler_NamespaceAcceptsPrefixRouteForOwnHost(t *testing.T) {
// Per-registry containerd mirrors with override_path address the
// upstream/{name}/ prefix and still send ns. They must keep working and
// share the prefix route's cache entries.
registry := newNSTestRegistry(t, "owner/app", "")
routes, _, _ := newNSTestHandler(t, "", map[string]string{"ghcr": registry.URL})
digest := "sha256:" + sha256Hex(nsTestBlob)
for _, target := range []string{
"/v2/upstream/ghcr/owner/app/manifests/latest?ns=" + registry.host(),
"/v2/upstream/ghcr/owner/app/blobs/" + digest + "?ns=" + registry.host(),
} {
if response := serveNS(routes, target); response.Code != http.StatusOK {
t.Fatalf("%s status = %d: %s", target, response.Code, response.Body.String())
}
}
if got, want := registry.lastRequest(), "/v2/owner/app/blobs/"+digest; got != want {
t.Errorf("upstream request = %q, want %q", got, want)
}
warmed := registry.requestCount()
for _, target := range []string{
"/v2/upstream/ghcr/owner/app/manifests/latest",
"/v2/upstream/ghcr/owner/app/blobs/" + digest,
} {
if response := serveNS(routes, target); response.Code != http.StatusOK {
t.Fatalf("%s status = %d: %s", target, response.Code, response.Body.String())
}
}
if got := registry.requestCount(); got != warmed {
t.Errorf("upstream requests after prefix pulls without ns = %d, want %d (cache hits)", got, warmed)
}
tags := serveNS(routes, "/v2/upstream/ghcr/owner/app/tags/list?n=1&ns="+registry.host())
if tags.Code != http.StatusOK {
t.Fatalf("tags status = %d: %s", tags.Code, tags.Body.String())
}
wantLink := `<` + nsTestProxyURL + `/v2/upstream/ghcr/owner/app/tags/list?last=1.0&n=1&ns=` + url.QueryEscape(registry.host()) + `>; rel="next"`
if got := tags.Header().Get("Link"); got != wantLink {
t.Errorf("Link = %q, want %q", got, wantLink)
}
if got, want := registry.lastRequest(), "/v2/owner/app/tags/list?n=1"; got != want {
t.Errorf("upstream tags request = %q, want %q (ns must not be forwarded)", got, want)
}
}
func TestContainerHandler_NamespacePrefixRouteAcceptsDockerHubAliases(t *testing.T) {
// upstream.oci.hub points at Docker Hub and a docker.io hosts.toml mirror
// with override_path addresses /v2/upstream/hub, so containerd sends
// ns=docker.io. The dialer stands in for registry-1.docker.io.
hub := newNSTestRegistry(t, "library/nginx", "")
other := newNSTestRegistry(t, "library/nginx", "")
routes, h, _ := newNSTestHandler(t, "", map[string]string{
"hub": "http://registry-1.docker.io",
"quay": other.URL,
})
transport := &http.Transport{DialContext: func(ctx context.Context, network, addr string) (net.Conn, error) {
if addr != "registry-1.docker.io:80" {
return nil, fmt.Errorf("unexpected dial to %s", addr)
}
return (&net.Dialer{}).DialContext(ctx, network, hub.Listener.Addr().String())
}}
t.Cleanup(transport.CloseIdleConnections)
h.proxy.HTTPClient = &http.Client{Transport: transport}
for _, query := range []string{"?ns=docker.io", "?ns=index.docker.io", "?ns=registry-1.docker.io", "?ns=docker.io:443", ""} {
response := serveNS(routes, "/v2/upstream/hub/library/nginx/manifests/latest"+query)
if response.Code != http.StatusOK {
t.Fatalf("%q status = %d: %s", query, response.Code, response.Body.String())
}
}
if got := hub.requestCount(); got != 1 {
t.Errorf("hub requests = %d, want 1 (the rest are cache hits)", got)
}
// A known host of another route still contradicts the prefix.
assertNameUnknown(t, serveNS(routes, "/v2/upstream/hub/library/nginx/manifests/latest?ns="+other.host()))
if got := other.requestCount(); got != 0 {
t.Errorf("other registry requests = %d, want 0", got)
}
t.Run("Docker Hub mirror as named upstream", func(t *testing.T) {
// The upstream is a Docker Hub mirror on some other host (mirror.gcr.io,
// an Artifactory remote); nodes still pull docker.io/... through
// /v2/upstream/hub, so ns says docker.io.
mirror := newNSTestRegistry(t, "library/nginx", "")
routes, _, _ := newNSTestHandler(t, "", map[string]string{"hub": mirror.URL})
response := serveNS(routes, "/v2/upstream/hub/library/nginx/manifests/latest?ns=docker.io")
if response.Code != http.StatusOK {
t.Fatalf("status = %d: %s", response.Code, response.Body.String())
}
if got, want := mirror.lastRequest(), "/v2/library/nginx/manifests/latest"; got != want {
t.Errorf("upstream request = %q, want %q", got, want)
}
})
}
func TestContainerHandler_NamespacePrefixRouteRejectsUnknownHosts(t *testing.T) {
// A _default wildcard mirror turns a pull of
// unconfigured.example/upstream/mirror/owner/app into exactly this
// request. Serving it would hand out owner/app from the mirror upstream
// instead of letting containerd fall back to unconfigured.example.
mirror := newNSTestRegistry(t, "owner/app", "")
routes, h, logs := newNSTestHandler(t, "", map[string]string{"mirror": mirror.URL})
for _, namespace := range []string{"unconfigured.example", "ghcr.io"} {
assertNameUnknown(t, serveNS(routes, "/v2/upstream/mirror/owner/app/manifests/latest?ns="+namespace))
}
if got := mirror.requestCount(); got != 0 {
t.Errorf("upstream requests = %d, want 0", got)
}
// containerd swallows the 404, so the refusal has to show up in the log.
for _, want := range []string{"upstream.oci_mirrors", "upstream=mirror", "ns=ghcr.io"} {
if !strings.Contains(logs.String(), want) {
t.Errorf("logs missing %q:\n%s", want, logs.String())
}
}
// A node repeats the refused request for every pull; one line per
// upstream/ns pair is enough. An ns longer than any registry host is
// refused before it is logged or remembered.
assertNameUnknown(t, serveNS(routes, "/v2/upstream/mirror/owner/app/manifests/latest?ns=GHCR.io"))
long := strings.Repeat("a", 300) + ".example"
assertNameUnknown(t, serveNS(routes, "/v2/upstream/mirror/owner/app/manifests/latest?ns="+long))
if got := strings.Count(logs.String(), "upstream.oci_mirrors"); got != 2 {
t.Errorf("refusal warnings = %d, want 2 (one per upstream/ns pair):\n%s", got, logs.String())
}
if strings.Contains(logs.String(), "aaaa") {
t.Errorf("logs mention the overlong ns, want it refused silently:\n%s", logs.String())
}
if got := len(h.refusals); got != 2 {
t.Errorf("remembered refusals = %d, want 2", got)
}
}
func TestContainerHandler_NamespaceRefusalLogIsBounded(t *testing.T) {
mirror := newNSTestRegistry(t, "owner/app", "")
routes, h, logs := newNSTestHandler(t, "", map[string]string{"mirror": mirror.URL})
h.refusals = make(map[string]struct{}, maxReportedNamespaceRefusals)
for i := range maxReportedNamespaceRefusals {
h.refusals[fmt.Sprintf("mirror\x00filler-%d.example", i)] = struct{}{}
}
for _, namespace := range []string{"filler-0.example", "first.example", "second.example"} {
assertNameUnknown(t, serveNS(routes, "/v2/upstream/mirror/owner/app/manifests/latest?ns="+namespace))
}
if got := strings.Count(logs.String(), "further refusals are not logged"); got != 1 {
t.Errorf("limit notices = %d, want 1:\n%s", got, logs.String())
}
if strings.Contains(logs.String(), "refusing ns") {
t.Errorf("refusal logged past the limit:\n%s", logs.String())
}
if got := len(h.refusals); got != maxReportedNamespaceRefusals {
t.Errorf("remembered refusals = %d, want %d", got, maxReportedNamespaceRefusals)
}
}
func TestContainerHandler_NamespaceMirroredRegistries(t *testing.T) {
// The upstream mirrors ghcr.io, and upstream.oci_mirrors says so. Nodes
// keep pulling ghcr.io/..., either through a ghcr.io hosts.toml pointing
// at /v2/upstream/ghcr or through the _default wildcard mirror.
tests := map[string]string{
"plain mirror host": "",
"artifactory remote": "/artifactory/api/docker/ghcr-remote",
}
for name, pathPrefix := range tests {
t.Run(name, func(t *testing.T) {
mirror := newNSTestRegistry(t, "owner/app", pathPrefix)
routes, h, _ := newNSTestHandler(t, "", map[string]string{"ghcr": mirror.URL + pathPrefix})
h.SetMirroredRegistries(map[string][]string{"ghcr": {"ghcr.io"}})
for _, target := range []string{
"/v2/upstream/ghcr/owner/app/manifests/latest?ns=ghcr.io",
"/v2/upstream/ghcr/owner/app/manifests/latest?ns=GHCR.io:443",
"/v2/owner/app/manifests/latest?ns=ghcr.io",
} {
before := mirror.requestCount()
response := serveNS(routes, target)
if response.Code != http.StatusOK {
t.Fatalf("GET %s status = %d: %s", target, response.Code, response.Body.String())
}
// The manifest is cached after the first request, so only
// the first one may reach the upstream.
if before == 0 {
if got, want := mirror.lastRequest(), pathPrefix+"/v2/owner/app/manifests/latest"; got != want {
t.Errorf("upstream request = %q, want %q", got, want)
}
}
}
if got := mirror.requestCount(); got != 1 {
t.Errorf("upstream requests = %d, want 1 (all routes share the cache)", got)
}
assertNameUnknown(t, serveNS(routes, "/v2/upstream/ghcr/owner/app/manifests/latest?ns=quay.io"))
})
}
}
func TestContainerHandler_NamespaceMirrorHostsRankBelowConfiguredURLs(t *testing.T) {
proxy, _, _, _ := setupTestProxy(t)
logs := &bytes.Buffer{}
proxy.Logger = slog.New(slog.NewTextHandler(logs, nil))
h := NewContainerHandlerWithRegistry(proxy, nsTestProxyURL, "", map[string]string{
"ghcr": "https://ghcr.io",
"art": "https://artifactory.example/api/docker/ghcr-remote",
"other": "https://other.example",
})
h.SetMirroredRegistries(map[string][]string{"art": {"ghcr.io"}, "missing": {"quay.io"}})
if got := h.namespaces["ghcr.io"]; got != "ghcr" {
t.Errorf("index[ghcr.io] = %q, want ghcr (configured URL wins over a mirror entry)", got)
}
if !h.namespaceNamesPrefixUpstream("ghcr.io", "upstream/art/owner/app") {
t.Error("ns ghcr.io refused on upstream/art/, want accepted as its mirrored registry")
}
if h.namespaceNamesPrefixUpstream("ghcr.io", "upstream/other/owner/app") {
t.Error("ns ghcr.io accepted on upstream/other/, want refused: only art lists it")
}
if _, ok := h.namespaces["quay.io"]; ok {
t.Error("index has quay.io from a mirror entry without upstream")
}
for _, want := range []string{"ghcr.io=ghcr", "upstream=missing"} {
if !strings.Contains(logs.String(), want) {
t.Errorf("logs missing %q:\n%s", want, logs.String())
}
}
}
func TestContainerHandler_NamespaceSharesCacheWithOtherRoutes(t *testing.T) {
digest := "sha256:" + sha256Hex(nsTestBlob)
manifestDigest := "sha256:" + sha256Hex(nsTestManifest())
t.Run("named registry", func(t *testing.T) {
registry := newNSTestRegistry(t, "owner/app", "")
routes, _, _ := newNSTestHandler(t, "", map[string]string{"ghcr": registry.URL})
for _, target := range []string{
"/v2/upstream/ghcr/owner/app/blobs/" + digest,
"/v2/upstream/ghcr/owner/app/manifests/" + manifestDigest,
} {
if response := serveNS(routes, target); response.Code != http.StatusOK {
t.Fatalf("warm %s status = %d: %s", target, response.Code, response.Body.String())
}
}
warmed := registry.requestCount()
for _, target := range []string{
"/v2/owner/app/blobs/" + digest + "?ns=" + registry.host(),
"/v2/owner/app/manifests/" + manifestDigest + "?ns=" + registry.host(),
} {
if response := serveNS(routes, target); response.Code != http.StatusOK {
t.Fatalf("ns %s status = %d: %s", target, response.Code, response.Body.String())
}
}
if got := registry.requestCount(); got != warmed {
t.Errorf("upstream requests after ns pulls = %d, want %d (cache hits)", got, warmed)
}
})
t.Run("default registry", func(t *testing.T) {
registry := newNSTestRegistry(t, "library/nginx", "")
routes, _, _ := newNSTestHandler(t, registry.URL, nil)
for _, target := range []string{
"/v2/library/nginx/blobs/" + digest + "?ns=docker.io",
"/v2/library/nginx/manifests/" + manifestDigest + "?ns=docker.io",
} {
if response := serveNS(routes, target); response.Code != http.StatusOK {
t.Fatalf("warm %s status = %d: %s", target, response.Code, response.Body.String())
}
}
warmed := registry.requestCount()
for _, target := range []string{
"/v2/library/nginx/blobs/" + digest,
"/v2/library/nginx/manifests/" + manifestDigest,
} {
if response := serveNS(routes, target); response.Code != http.StatusOK {
t.Fatalf("unprefixed %s status = %d: %s", target, response.Code, response.Body.String())
}
}
if got := registry.requestCount(); got != warmed {
t.Errorf("upstream requests after unprefixed pulls = %d, want %d (cache hits)", got, warmed)
}
})
}
func TestContainerHandler_NamespaceTagsList(t *testing.T) {
registry := newNSTestRegistry(t, "owner/app", "")
routes, _, _ := newNSTestHandler(t, "", map[string]string{"ghcr": registry.URL})
// The prefix route fills the shared cache entry first.
prefixed := serveNS(routes, "/v2/upstream/ghcr/owner/app/tags/list?n=1")
if prefixed.Code != http.StatusOK {
t.Fatalf("prefixed status = %d: %s", prefixed.Code, prefixed.Body.String())
}
const wantPrefixedLink = `<` + nsTestProxyURL + `/v2/upstream/ghcr/owner/app/tags/list?last=1.0&n=1>; rel="next"`
if got := prefixed.Header().Get("Link"); got != wantPrefixedLink {
t.Errorf("prefixed Link = %q, want %q", got, wantPrefixedLink)
}
namespaced := serveNS(routes, "/v2/owner/app/tags/list?n=1&ns="+registry.host())
if namespaced.Code != http.StatusOK {
t.Fatalf("ns status = %d: %s", namespaced.Code, namespaced.Body.String())
}
if got := registry.requestCount(); got != 1 {
t.Errorf("upstream requests = %d, want 1 (ns request served from the shared cache)", got)
}
wantNSLink := `<` + nsTestProxyURL + `/v2/owner/app/tags/list?last=1.0&n=1&ns=` + url.QueryEscape(registry.host()) + `>; rel="next"`
link := namespaced.Header().Get("Link")
if link != wantNSLink {
t.Fatalf("ns Link = %q, want %q", link, wantNSLink)
}
next := serveNS(routes, strings.TrimPrefix(strings.SplitN(link, ">", 2)[0], "<"))
if next.Code != http.StatusOK {
t.Fatalf("next page status = %d: %s", next.Code, next.Body.String())
}
if got, want := next.Body.String(), `{"name":"owner/app","tags":["2.0"]}`; got != want {
t.Errorf("next page body = %q, want %q", got, want)
}
if got, want := registry.lastRequest(), "/v2/owner/app/tags/list?last=1.0&n=1"; got != want {
t.Errorf("upstream request = %q, want %q (ns must not be forwarded)", got, want)
}
}
func TestContainerHandler_NamespaceDefaultBypassesRepositoryPrefixRoutes(t *testing.T) {
hub := newNSTestRegistry(t, "homebrew/core/jq", "")
brew := newNSTestRegistry(t, "homebrew/core/jq", "")
routes, h, _ := newNSTestHandler(t, hub.URL, nil)
RegisterHomebrewArtifacts(h, brew.URL)
if response := serveNS(routes, "/v2/homebrew/core/jq/manifests/latest?ns=docker.io"); response.Code != http.StatusOK {
t.Fatalf("ns status = %d: %s", response.Code, response.Body.String())
}
if hub.requestCount() != 1 || brew.requestCount() != 0 {
t.Errorf("ns request reached hub=%d brew=%d, want hub=1 brew=0", hub.requestCount(), brew.requestCount())
}
if response := serveNS(routes, "/v2/homebrew/core/jq/manifests/latest"); response.Code != http.StatusOK {
t.Fatalf("unprefixed status = %d: %s", response.Code, response.Body.String())
}
if hub.requestCount() != 1 || brew.requestCount() != 1 {
t.Errorf("unprefixed request reached hub=%d brew=%d, want hub=1 brew=1", hub.requestCount(), brew.requestCount())
}
}
func TestContainerHandler_NamespaceMatchesConfiguredHosts(t *testing.T) {
// Each pair is a configured registry URL and an ns value containerd sends
// for an image reference on that registry. The scheme-default port may be
// spelled out or left out on either side; any other port must match.
matches := []struct {
configured string
namespace string
}{
{"https://GHCR.io", "ghcr.io"},
{"https://ghcr.io/", "GHCR.IO"},
{"http://[fd00::1]:5000", "[fd00::1]:5000"},
{"https://[fd00::1]", "[fd00::1]"},
{"https://[fd00::1]", "[fd00::1]:443"},
{"https://reg.example:80", "reg.example:80"},
{"https://reg.example:443", "reg.example"},
{"https://reg.example", "reg.example:443"},
{"http://reg.example:80", "reg.example"},
{"http://reg.example", "reg.example:80"},
{"http://reg.example:5000", "reg.example:5000"},
}
for _, tt := range matches {
t.Run(tt.configured+" "+tt.namespace, func(t *testing.T) {
h := NewContainerHandlerWithRegistry(nil, nsTestProxyURL, "", map[string]string{"lab": tt.configured})
registryURL, upstreamName, cacheName, ok := h.registryForNamespace(tt.namespace, "owner/app")
if !ok {
t.Fatalf("ns %q did not resolve for upstream %q", tt.namespace, tt.configured)
}
if registryURL != strings.TrimSuffix(tt.configured, "/") || upstreamName != "owner/app" || cacheName != "upstream/lab/owner/app" {
t.Errorf("route = (%q, %q, %q), want (%q, owner/app, upstream/lab/owner/app)",
registryURL, upstreamName, cacheName, tt.configured)
}
})
}
mismatches := []struct {
configured string
namespace string
}{
{"http://reg.example:5000", "reg.example:5001"},
{"https://reg.example:80", "reg.example"},
{"https://reg.example", "reg.example:80"},
{"http://reg.example:443", "reg.example"},
}
for _, tt := range mismatches {
t.Run("mismatch "+tt.configured+" "+tt.namespace, func(t *testing.T) {
h := NewContainerHandlerWithRegistry(nil, nsTestProxyURL, "", map[string]string{"lab": tt.configured})
if _, _, _, ok := h.registryForNamespace(tt.namespace, "owner/app"); ok {
t.Errorf("ns %q resolved for upstream %q, want no match", tt.namespace, tt.configured)
}
})
}
}
func TestContainerHandler_NamespaceKeepsNonDefaultPorts(t *testing.T) {
// Two registries on one host that differ only in the port, one of them on
// the scheme-default port. Tests cannot listen on ports 80 or 443, so the
// dialer maps those addresses to the fake registries.
onDefaultPort := newNSTestRegistry(t, "owner/app", "")
onOtherPort := newNSTestRegistry(t, "owner/app", "")
routes, h, _ := newNSTestHandler(t, "", map[string]string{
"std": "http://registry.example",
"alt": "http://registry.example:443",
})
endpoints := map[string]string{
"registry.example:80": onDefaultPort.Listener.Addr().String(),
"registry.example:443": onOtherPort.Listener.Addr().String(),
}
transport := &http.Transport{DialContext: func(ctx context.Context, network, addr string) (net.Conn, error) {
target, ok := endpoints[addr]
if !ok {
return nil, fmt.Errorf("unexpected dial to %s", addr)
}
return (&net.Dialer{}).DialContext(ctx, network, target)
}}
t.Cleanup(transport.CloseIdleConnections)
h.proxy.HTTPClient = &http.Client{Transport: transport}
// Every request below misses the cache: a different registry or a
// different endpoint than the request before it.
tests := []struct {
target string
wantDefault int
wantOther int
}{
{"/v2/owner/app/manifests/latest?ns=registry.example:80", 1, 0},
{"/v2/owner/app/manifests/latest?ns=registry.example:443", 1, 1},
{"/v2/owner/app/tags/list?ns=registry.example", 2, 1},
}
for _, tt := range tests {
response := serveNS(routes, tt.target)
if response.Code != http.StatusOK {
t.Fatalf("%s status = %d: %s", tt.target, response.Code, response.Body.String())
}
if onDefaultPort.requestCount() != tt.wantDefault || onOtherPort.requestCount() != tt.wantOther {
t.Errorf("after %s: requests default-port=%d other-port=%d, want %d/%d",
tt.target, onDefaultPort.requestCount(), onOtherPort.requestCount(), tt.wantDefault, tt.wantOther)
}
}
}
func TestContainerHandler_NamespaceHostCollisions(t *testing.T) {
proxy, _, _, _ := setupTestProxy(t)
logs := &bytes.Buffer{}
proxy.Logger = slog.New(slog.NewTextHandler(logs, nil))
h := NewContainerHandlerWithRegistry(proxy, nsTestProxyURL, "https://mirror.example", map[string]string{
"zeta": "https://shared.example",
"alpha": "https://Shared.example:443",
"mirror": "https://mirror.example",
"hub": "https://registry-1.docker.io",
})
tests := map[string]string{
"shared.example": "https://Shared.example:443",
"mirror.example": "https://mirror.example",
"registry-1.docker.io": "https://mirror.example",
}
for namespace, want := range tests {
registryURL, _, cacheName, ok := h.registryForNamespace(namespace, "owner/app")
if !ok || registryURL != want {
t.Errorf("ns %q resolved to (%q, %v), want %q", namespace, registryURL, ok, want)
}
if namespace == "shared.example" && cacheName != "upstream/alpha/owner/app" {
t.Errorf("ns %q cache name = %q, want upstream/alpha/owner/app", namespace, cacheName)
}
}
for _, upstream := range []string{"upstream=zeta", "upstream=mirror", "upstream=hub"} {
if !strings.Contains(logs.String(), upstream) {
t.Errorf("missing collision warning for %s in logs:\n%s", upstream, logs.String())
}
}
}
func TestContainerHandler_NamespaceCollisionWarningNamesEachOwner(t *testing.T) {
// r's two keys end up with two different owners; the warning has to
// name both, or an admin fixing one entry misses the other.
proxy, _, _, _ := setupTestProxy(t)
logs := &bytes.Buffer{}
proxy.Logger = slog.New(slog.NewTextHandler(logs, nil))
h := NewContainerHandlerWithRegistry(proxy, nsTestProxyURL, "", map[string]string{
"p": "http://h.example",
"q": "http://h.example:443",
"r": "https://h.example",
})
for namespace, want := range map[string]string{"h.example": "p", "h.example:80": "p", "h.example:443": "q"} {
if got := h.namespaces[namespace]; got != want {
t.Errorf("index[%q] = %q, want %q", namespace, got, want)
}
}
for _, want := range []string{"upstream=r", "h.example=p", "h.example:443=q"} {
if !strings.Contains(logs.String(), want) {
t.Errorf("logs missing %q:\n%s", want, logs.String())
}
}
}
func TestContainerHandler_NamespaceSkipsRegistryURLsWithPath(t *testing.T) {
registry := newNSTestRegistry(t, "owner/app", "/artifactory/api/docker/remote")
routes, _, logs := newNSTestHandler(t, "", map[string]string{
"art": registry.URL + "/artifactory/api/docker/remote",
})
assertNameUnknown(t, serveNS(routes, "/v2/owner/app/manifests/latest?ns="+registry.host()))
if got := registry.requestCount(); got != 0 {
t.Errorf("upstream requests via ns = %d, want 0", got)
}
if !strings.Contains(logs.String(), "upstream=art") {
t.Errorf("missing warning for path-prefixed upstream in logs:\n%s", logs.String())
}
// Per-registry mirrors with override_path reach it with ns attached.
if response := serveNS(routes, "/v2/upstream/art/owner/app/manifests/latest?ns="+registry.host()); response.Code != http.StatusOK {
t.Fatalf("prefix route with ns status = %d, want 200: %s", response.Code, response.Body.String())
}
if got := registry.requestCount(); got != 1 {
t.Errorf("upstream requests via prefix route with ns = %d, want 1", got)
}
if response := serveNS(routes, "/v2/upstream/art/owner/app/manifests/latest"); response.Code != http.StatusOK {
t.Fatalf("prefix route status = %d, want 200: %s", response.Code, response.Body.String())
}
t.Run("default registry", func(t *testing.T) {
mirror := newNSTestRegistry(t, "library/nginx", "/hub")
routes, _, logs := newNSTestHandler(t, mirror.URL+"/hub", nil)
assertNameUnknown(t, serveNS(routes, "/v2/library/nginx/manifests/latest?ns="+mirror.host()))
if got := mirror.requestCount(); got != 0 {
t.Errorf("upstream requests via ns host = %d, want 0", got)
}
if response := serveNS(routes, "/v2/library/nginx/manifests/latest?ns=docker.io"); response.Code != http.StatusOK {
t.Fatalf("ns=docker.io status = %d, want 200: %s", response.Code, response.Body.String())
}
if !strings.Contains(logs.String(), "default OCI registry") {
t.Errorf("missing warning for path-prefixed default registry in logs:\n%s", logs.String())
}
})
}
func TestContainerHandler_TagsListIgnoresLegacyCacheRows(t *testing.T) {
registry := newNSTestRegistry(t, "owner/app", "")
routes, h, _ := newNSTestHandler(t, "", map[string]string{"ghcr": registry.URL})
// Rows written before the raw-link format hold a Link already rewritten
// for the route that filled them. Seed one under the legacy identity.
query := url.Values{"n": {"1"}}
legacySum := sha256.Sum256([]byte(registry.URL + "\x00owner/app\x00" + query.Encode()))
legacyKey := hex.EncodeToString(legacySum[:])
if legacyKey == h.containerTagsCacheKey(registry.URL, "owner/app", query) {
t.Fatal("legacy and current tag-list cache keys are equal")
}
legacy := &cachedContainerTags{
body: []byte(`{"name":"owner/app","tags":["legacy"]}`),
contentType: contentTypeJSON,
link: `<` + nsTestProxyURL + `/v2/upstream/ghcr/owner/app/tags/list?last=legacy&n=1>; rel="next"`,
fetchedAt: time.Now(),
}
if err := h.storeContainerTags(context.Background(), legacyKey, legacy); err != nil {
t.Fatalf("store legacy tag list: %v", err)
}
response := serveNS(routes, "/v2/owner/app/tags/list?n=1&ns="+registry.host())
if response.Code != http.StatusOK {
t.Fatalf("status = %d: %s", response.Code, response.Body.String())
}
if got, want := response.Body.String(), `{"name":"owner/app","tags":["1.0"]}`; got != want {
t.Errorf("body = %q, want %q (legacy row must not be served)", got, want)
}
if got := registry.requestCount(); got != 1 {
t.Errorf("upstream requests = %d, want 1 (legacy row must not be served)", got)
}
}
func TestContainerHandler_NamespaceWarningsRedactCredentials(t *testing.T) {
proxy, _, _, _ := setupTestProxy(t)
logs := &bytes.Buffer{}
proxy.Logger = slog.New(slog.NewTextHandler(logs, nil))
NewContainerHandlerWithRegistry(proxy, nsTestProxyURL, "https://svc:s3cret@mirror.example/hub", map[string]string{
"art": "https://bot:hunter2@art.example/artifactory/api/docker/remote",
})
for _, secret := range []string{"s3cret", "hunter2"} {
if strings.Contains(logs.String(), secret) {
t.Errorf("logs contain credential %q:\n%s", secret, logs.String())
}
}
for _, want := range []string{
"svc:xxxxx@mirror.example/hub",
"bot:xxxxx@art.example/artifactory",
"Docker Hub aliases still select it",
} {
if !strings.Contains(logs.String(), want) {
t.Errorf("logs missing %q:\n%s", want, logs.String())
}
}
}
Reference in a new issue View git blame Copy permalink