mirrors/pkg-proxy
Watch
1
0
Fork
You've already forked pkg-proxy
 1
mirror of https://github.com/git-pkgs/proxy.git synced 2026-11-09 15:49:16 -05:00
Code Issues Releases Packages Wiki Activity
pkg-proxy/internal/handler/container_referrers_test.go

552 lines
23 KiB
Go
Raw Permalink Normal View History

Proxy the OCI referrers API in the container handler (#427) * Proxy the OCI referrers API in the container handler GET /v2/{name}/referrers/{digest} used to fall through to a plain 404, so clients only ever saw referrers stored under the sha256-<hex> tag schema. Registries that keep SBOMs and signatures behind the referrers API looked empty through the proxy. The index is now fetched from the upstream that the repository routes to and cached like the tag list: fresh within the metadata TTL, revalidated with If-None-Match, and served stale when the upstream is down. artifactType is not forwarded, so one cached row covers every filter and clients filter the full list themselves, which they do when OCI-Filters-Applied is absent. An upstream 404 is passed through untouched, so clients still fall back to the tag schema for registries like GHCR. When the upstream cannot answer and nothing is cached, the proxy keeps answering 404 as before rather than failing the request. Closes #144 * Cover absolute upstream links in the referrers link test The second page now answers with an absolute Link pointing back at the upstream host, and the test checks that it comes back rewritten to the proxy. That was missing so far, and it also gives the upstream variable a reason to be declared before the handler closure, which staticcheck complained about. * Read cached referrers through a small shared loader loadContainerReferrers was a line-for-line copy of loadContainerTags with the types renamed, which dupl flags in CI. The part that opens the cache row and reads the body now lives in loadContainerMetadata next to storeContainerMetadata, and the referrers loader only maps the row onto its struct. The tag and manifest loaders are left as they are. * Make the manifest routing test notice which handler answered For foo/manifests/referrers/<digest> the manifest handler and the referrers handler send the very same upstream path, so the test passed even with the referrers case moved in front of the manifest case. The fake upstream now rejects requests that only accept an image index, which is what the referrers handler sends. * Check that a cached referrers row links back to whoever reads it A repository can be reached as upstream/<name>/<repo> and, when the default registry is the same host, as plain <repo>. Both end up in the same cache row, so the pagination Link has to be rewritten when the row is served, not when it is stored. The link test now reads the row through both paths; with a store-time rewrite the second path would get the first one's link. * Make the referrers ETag test prove the revalidation Without If-None-Match the fake upstream just sends the full index again, so the old assertions held either way. The test now counts the 304s the upstream sent and expects exactly one. * Reword which tools use the referrers API in the README The cosign mention was too broad. cosign reaches the referrers API for its newer signature bundles, while the old .sig tags keep going through the normal manifest path. Also rewraps the paragraph, one line had run past the others. * Check that a 304 restarts the referrers cache TTL The ETag test now backdates the cached row instead of running with a zero TTL. After the upstream answers 304, a third request has to come from the cache. Before, the test would also have passed if the 304 branch forgot to bump fetchedAt and every request kept revalidating. * Test the referrers Content-Type fallback None of the fake upstreams left out Content-Type, so the default to the image index type was never exercised. oras-go rejects anything else, so it is worth pinning, both for the first fetch and for the cached copy. * Test referrers digests with extra characters around them The invalid-request table only had digests that were too short or used an unknown algorithm, so it would not notice if the digest pattern lost its anchors. Two cases now put junk before and after an otherwise valid sha256 digest. * Reuse the manifest cache key for referrers The referrers handler had its own key function, but it did exactly what containerManifestCacheKey already does with four strings. Referrers rows sit in their own oci-referrers ecosystem anyway, so they can't collide with manifests. * Drop the separate struct for cached referrers A cached referrers response carries the same fields as a cached tag list, and the freshness check was a straight copy too. So the referrers code just uses cachedContainerTags and containerTagsFresh now. Loading, storing and writing stay on their own, since the ecosystem and the Link handling are different. * Route referrers requests by ns like the other OCI endpoints Now that containerd ns support is on main, referrers requests have to go through registryForRequest too, otherwise a _default mirror would look every subject up on Docker Hub. ns only picks the registry, so it is dropped from the forwarded query and the cache key, and the next-page Link keeps it, the same way tag lists do. This also follows the new rewriteContainerTagsLink signature, which takes the namespace. * Without a cached index, referrers now pass upstream errors through and answer 502 when the upstream can't be reached or sends garbage, so clients no longer read an outage as a registry without the referrers API. Stale cache is still served when there is one. * The tag list and manifest routes now only match paths that end in their endpoint, so a repository with tags/list or manifests in its name can still reach the referrers endpoint.
2026-10-12 10:08:01 +02:00
package handler
import (
"io"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"testing"
"time"
)
const testReferrersIndex = `{"schemaVersion":2,"mediaType":"application/vnd.oci.image.index.v1+json","manifests":[` +
`{"mediaType":"application/vnd.oci.image.manifest.v1+json","digest":"sha256:1111111111111111111111111111111111111111111111111111111111111111","size":10,"artifactType":"application/spdx+json"},` +
`{"mediaType":"application/vnd.oci.image.manifest.v1+json","digest":"sha256:2222222222222222222222222222222222222222222222222222222222222222","size":10,"artifactType":"application/vnd.cncf.notary.signature"}]}`
var testReferrersSubject = "sha256:" + sha256Hex("subject manifest")
func newReferrersTestHandler(t *testing.T, upstream *httptest.Server) (*ContainerHandler, *Proxy) {
t.Helper()
proxy, _, _, _ := setupTestProxy(t)
proxy.HTTPClient = upstream.Client()
return &ContainerHandler{proxy: proxy, registryURL: upstream.URL, proxyURL: "http://proxy.example.test"}, proxy
}
func serveReferrersRequest(h *ContainerHandler, method, target string) *httptest.ResponseRecorder {
recorder := httptest.NewRecorder()
h.Routes().ServeHTTP(recorder, httptest.NewRequest(method, target, nil))
return recorder
}
func TestContainerHandler_ReferrersCachesIndexForEveryFilter(t *testing.T) {
upstreamAvailable := true
upstreamRequests := 0
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
upstreamRequests++
if r.URL.Path != "/v2/library/nginx/referrers/"+testReferrersSubject {
http.NotFound(w, r)
return
}
if r.URL.RawQuery != "" {
t.Errorf("upstream query = %q, want artifactType stripped", r.URL.RawQuery)
}
if got := r.Header.Get("Accept"); got != containerReferrersMediaType {
t.Errorf("upstream Accept = %q, want %q", got, containerReferrersMediaType)
}
if !upstreamAvailable {
http.Error(w, "upstream unavailable", http.StatusServiceUnavailable)
return
}
w.Header().Set("Content-Type", containerReferrersMediaType)
w.Header().Set("OCI-Filters-Applied", "artifactType")
_, _ = io.WriteString(w, testReferrersIndex)
}))
defer upstream.Close()
h, proxy := newReferrersTestHandler(t, upstream)
proxy.MetadataTTL = time.Hour
base := "/library/nginx/referrers/" + testReferrersSubject
first := serveReferrersRequest(h, http.MethodGet, base+"?artifactType=application/spdx%2Bjson")
if first.Code != http.StatusOK {
t.Fatalf("first status = %d, want 200: %s", first.Code, first.Body.String())
}
if first.Body.String() != testReferrersIndex {
t.Errorf("first body = %q, want upstream index", first.Body.String())
}
if got := first.Header().Get("Content-Type"); got != containerReferrersMediaType {
t.Errorf("first Content-Type = %q, want %q", got, containerReferrersMediaType)
}
if got := first.Header().Get("OCI-Filters-Applied"); got != "" {
t.Errorf("OCI-Filters-Applied = %q, want absent so clients filter", got)
}
other := serveReferrersRequest(h, http.MethodGet, base+"?artifactType=application/vnd.cncf.notary.signature")
if other.Code != http.StatusOK || other.Body.String() != testReferrersIndex {
t.Fatalf("other filter = %d %q, want cached full index", other.Code, other.Body.String())
}
if upstreamRequests != 1 {
t.Fatalf("upstream requests after fresh hit = %d, want 1", upstreamRequests)
}
proxy.MetadataTTL = 0
upstreamAvailable = false
stale := serveReferrersRequest(h, http.MethodGet, base)
if stale.Code != http.StatusOK || stale.Body.String() != testReferrersIndex {
t.Fatalf("stale = %d %q, want cached index", stale.Code, stale.Body.String())
}
if got := stale.Header().Get("Content-Type"); got != containerReferrersMediaType {
t.Errorf("stale Content-Type = %q, want %q", got, containerReferrersMediaType)
}
if got := stale.Header().Get("Warning"); got != containerStaleWarning {
t.Errorf("stale Warning = %q, want stale warning", got)
}
if upstreamRequests != 2 {
t.Errorf("upstream requests after stale fallback = %d, want 2", upstreamRequests)
}
}
func TestContainerHandler_ReferrersRevalidatesWithETag(t *testing.T) {
upstreamRequests := 0
notModified := 0
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
upstreamRequests++
if r.Header.Get("If-None-Match") == `"referrers-etag"` {
notModified++
w.WriteHeader(http.StatusNotModified)
return
}
w.Header().Set("Content-Type", containerReferrersMediaType)
w.Header().Set("ETag", `"referrers-etag"`)
_, _ = io.WriteString(w, testReferrersIndex)
}))
defer upstream.Close()
h, proxy := newReferrersTestHandler(t, upstream)
proxy.MetadataTTL = time.Hour
target := "/library/nginx/referrers/" + testReferrersSubject
if first := serveReferrersRequest(h, http.MethodGet, target); first.Code != http.StatusOK {
t.Fatalf("first status = %d, want 200", first.Code)
}
// Age the row past the TTL so the next request has to revalidate.
cacheKey := h.containerManifestCacheKey(upstream.URL, "library/nginx", testReferrersSubject, "")
entry, err := proxy.DB.GetMetadataCache(containerReferrersCacheEcosystem, cacheKey)
if err != nil || entry == nil {
t.Fatalf("cached row = %v, %v, want one", entry, err)
}
entry.FetchedAt.Time = time.Now().Add(-2 * time.Hour)
if err := proxy.DB.UpsertMetadataCache(entry); err != nil {
t.Fatalf("ageing cached row: %v", err)
}
second := serveReferrersRequest(h, http.MethodGet, target)
if second.Code != http.StatusOK || second.Body.String() != testReferrersIndex {
t.Fatalf("revalidated = %d %q, want cached index", second.Code, second.Body.String())
}
if got := second.Header().Get("Warning"); got != "" {
t.Errorf("Warning = %q, want none after 304", got)
}
// The 304 starts the TTL again, so this one stays in the cache.
if third := serveReferrersRequest(h, http.MethodGet, target); third.Code != http.StatusOK {
t.Fatalf("third status = %d, want 200", third.Code)
}
if upstreamRequests != 2 || notModified != 1 {
t.Errorf("upstream requests = %d with %d answered 304, want 2 with 1", upstreamRequests, notModified)
}
}
func TestContainerHandler_ReferrersDefaultsMissingContentType(t *testing.T) {
upstreamRequests := 0
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
upstreamRequests++
// A nil value stops net/http from sniffing a Content-Type.
w.Header()["Content-Type"] = nil
_, _ = io.WriteString(w, testReferrersIndex)
}))
defer upstream.Close()
h, proxy := newReferrersTestHandler(t, upstream)
proxy.MetadataTTL = time.Hour
target := "/library/nginx/referrers/" + testReferrersSubject
// oras-go only accepts the exact image index type, both fresh and cached.
for _, label := range []string{"fetched", "cached"} {
got := serveReferrersRequest(h, http.MethodGet, target)
if got.Code != http.StatusOK {
t.Fatalf("%s status = %d, want 200", label, got.Code)
}
if ct := got.Header().Get("Content-Type"); ct != containerReferrersMediaType {
t.Errorf("%s Content-Type = %q, want %q", label, ct, containerReferrersMediaType)
}
}
if upstreamRequests != 1 {
t.Errorf("upstream requests = %d, want 1", upstreamRequests)
}
}
func TestContainerHandler_ReferrersRelaysUpstreamErrorsWithoutCaching(t *testing.T) {
tests := []struct {
name string
status int
header string
value string
wantError string
}{
{name: "no referrers API", status: http.StatusNotFound, header: "Content-Type", value: "application/json", wantError: "NOT_FOUND"},
{name: "repository unknown", status: http.StatusNotFound, header: "Content-Type", value: "application/json", wantError: "NAME_UNKNOWN"},
{name: "auth challenge", status: http.StatusUnauthorized, header: "WWW-Authenticate", value: `Bearer realm="https://auth.example.test/token"`, wantError: "UNAUTHORIZED"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
upstreamRequests := 0
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
upstreamRequests++
w.Header().Set(tt.header, tt.value)
w.WriteHeader(tt.status)
_, _ = io.WriteString(w, `{"errors":[{"code":"`+tt.wantError+`"}]}`)
}))
defer upstream.Close()
h, proxy := newReferrersTestHandler(t, upstream)
proxy.MetadataTTL = time.Hour
target := "/library/nginx/referrers/" + testReferrersSubject
for range 2 {
got := serveReferrersRequest(h, http.MethodGet, target)
if got.Code != tt.status {
t.Fatalf("status = %d, want %d", got.Code, tt.status)
}
if !strings.Contains(got.Body.String(), tt.wantError) {
t.Errorf("body = %q, want upstream error %s", got.Body.String(), tt.wantError)
}
if got.Header().Get(tt.header) != tt.value {
t.Errorf("%s = %q, want %q", tt.header, got.Header().Get(tt.header), tt.value)
}
}
if upstreamRequests != 2 {
t.Errorf("upstream requests = %d, want 2 (error must not be cached)", upstreamRequests)
}
})
}
}
func TestContainerHandler_ReferrersReportsUpstreamFailureWithoutCache(t *testing.T) {
tests := []struct {
name string
handler http.HandlerFunc
wantStatus int
wantRetryAfter string
}{
{name: "upstream 503", wantStatus: http.StatusServiceUnavailable, wantRetryAfter: "120", handler: func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Retry-After", "120")
http.Error(w, "unavailable", http.StatusServiceUnavailable)
}},
{name: "upstream 429", wantStatus: http.StatusTooManyRequests, wantRetryAfter: "60", handler: func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Retry-After", "60")
http.Error(w, "slow down", http.StatusTooManyRequests)
}},
{name: "upstream not JSON", wantStatus: http.StatusBadGateway, handler: func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "text/html")
_, _ = io.WriteString(w, "<html>captive portal</html>")
}},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
upstreamRequests := 0
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
upstreamRequests++
tt.handler(w, r)
}))
defer upstream.Close()
h, proxy := newReferrersTestHandler(t, upstream)
proxy.MetadataTTL = time.Hour
target := "/library/nginx/referrers/" + testReferrersSubject
for range 2 {
got := serveReferrersRequest(h, http.MethodGet, target)
if got.Code != tt.wantStatus {
t.Fatalf("status = %d, want %d: %s", got.Code, tt.wantStatus, got.Body.String())
}
if got.Header().Get("Retry-After") != tt.wantRetryAfter {
t.Errorf("Retry-After = %q, want %q", got.Header().Get("Retry-After"), tt.wantRetryAfter)
}
}
if upstreamRequests != 2 {
t.Errorf("upstream requests = %d, want 2 (failure must not be cached)", upstreamRequests)
}
})
}
t.Run("upstream unreachable", func(t *testing.T) {
upstream := httptest.NewServer(http.NotFoundHandler())
h, _ := newReferrersTestHandler(t, upstream)
upstream.Close()
got := serveReferrersRequest(h, http.MethodGet, "/library/nginx/referrers/"+testReferrersSubject)
if got.Code != http.StatusBadGateway {
t.Fatalf("status = %d, want 502 so clients keep using the referrers API: %s", got.Code, got.Body.String())
}
})
}
func TestContainerHandler_ReferrersNamedRegistryRewritesLink(t *testing.T) {
upstreamRequests := 0
var upstream *httptest.Server
upstream = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
upstreamRequests++
if r.URL.Path != "/v2/owner/img/referrers/"+testReferrersSubject {
http.NotFound(w, r)
return
}
w.Header().Set("Content-Type", containerReferrersMediaType)
switch r.URL.Query().Get("last") {
case "":
w.Header().Add("Link", `</v2/owner/img/referrers/`+testReferrersSubject+`?last=abc>; rel="next"`)
w.Header().Add("Link", `<https://elsewhere.example.test/docs>; rel="help"`)
case "abc":
w.Header().Set("Link", `<`+upstream.URL+`/v2/owner/img/referrers/`+testReferrersSubject+`?last=def>; rel="next"`)
}
_, _ = io.WriteString(w, testReferrersIndex)
}))
defer upstream.Close()
proxy, _, _, _ := setupTestProxy(t)
proxy.HTTPClient = upstream.Client()
proxy.MetadataTTL = time.Hour
// The default registry is the same upstream, so upstream/test/owner/img
// and owner/img share one cache row.
h := NewContainerHandlerWithRegistry(proxy, "http://proxy.example.test", upstream.URL, map[string]string{"test": upstream.URL})
routes := http.StripPrefix("/v2", h.Routes())
first := httptest.NewRecorder()
routes.ServeHTTP(first, httptest.NewRequest(http.MethodGet,
"/v2/upstream/test/owner/img/referrers/"+testReferrersSubject+"?artifactType=application/spdx%2Bjson", nil))
if first.Code != http.StatusOK {
t.Fatalf("first page status = %d, want 200: %s", first.Code, first.Body.String())
}
wantLink := `<http://proxy.example.test/v2/upstream/test/owner/img/referrers/` + testReferrersSubject +
`?last=abc>; rel="next", <https://elsewhere.example.test/docs>; rel="help"`
if got := first.Header().Get("Link"); got != wantLink {
t.Fatalf("Link = %q, want %q", got, wantLink)
}
shared := httptest.NewRecorder()
routes.ServeHTTP(shared, httptest.NewRequest(http.MethodGet, "/v2/owner/img/referrers/"+testReferrersSubject, nil))
if shared.Code != http.StatusOK {
t.Fatalf("shared row status = %d, want 200: %s", shared.Code, shared.Body.String())
}
wantSharedLink := `<http://proxy.example.test/v2/owner/img/referrers/` + testReferrersSubject +
`?last=abc>; rel="next", <https://elsewhere.example.test/docs>; rel="help"`
if got := shared.Header().Get("Link"); got != wantSharedLink {
t.Errorf("shared row Link = %q, want it rewritten for the second path %q", got, wantSharedLink)
}
if upstreamRequests != 1 {
t.Fatalf("upstream requests after shared row hit = %d, want 1", upstreamRequests)
}
nextURL := strings.TrimPrefix(strings.SplitN(first.Header().Get("Link"), ">", 2)[0], "<")
next := httptest.NewRecorder()
routes.ServeHTTP(next, httptest.NewRequest(http.MethodGet, nextURL, nil))
if next.Code != http.StatusOK {
t.Fatalf("next page status = %d, want 200: %s", next.Code, next.Body.String())
}
wantNextLink := `<http://proxy.example.test/v2/upstream/test/owner/img/referrers/` + testReferrersSubject + `?last=def>; rel="next"`
if got := next.Header().Get("Link"); got != wantNextLink {
t.Errorf("next page Link = %q, want absolute upstream link rewritten to %q", got, wantNextLink)
}
if upstreamRequests != 2 {
t.Errorf("upstream requests = %d, want 2 (pages are separate rows)", upstreamRequests)
}
}
func TestContainerHandler_ReferrersRejectsInvalidRequests(t *testing.T) {
upstreamRequests := 0
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
upstreamRequests++
w.WriteHeader(http.StatusInternalServerError)
}))
defer upstream.Close()
proxy, _, _, _ := setupTestProxy(t)
proxy.HTTPClient = upstream.Client()
h := NewContainerHandlerWithRegistry(proxy, "http://proxy.example.test", upstream.URL)
tests := []struct {
name string
method string
path string
wantStatus int
wantCode string
}{
{name: "post", method: http.MethodPost, path: "/library/nginx/referrers/" + testReferrersSubject, wantStatus: http.StatusMethodNotAllowed},
{name: "short digest", method: http.MethodGet, path: "/library/nginx/referrers/sha256:abc", wantStatus: http.StatusBadRequest, wantCode: "DIGEST_INVALID"},
{name: "trailing characters", method: http.MethodGet, path: "/library/nginx/referrers/" + testReferrersSubject + "x", wantStatus: http.StatusBadRequest, wantCode: "DIGEST_INVALID"},
{name: "leading characters", method: http.MethodGet, path: "/library/nginx/referrers/x" + testReferrersSubject, wantStatus: http.StatusBadRequest, wantCode: "DIGEST_INVALID"},
{name: "unknown algorithm", method: http.MethodGet, path: "/library/nginx/referrers/md5:" + strings.Repeat("a", 32), wantStatus: http.StatusBadRequest, wantCode: "DIGEST_INVALID"},
{name: "unknown named upstream", method: http.MethodGet, path: "/upstream/missing/owner/img/referrers/" + testReferrersSubject, wantStatus: http.StatusNotFound, wantCode: "NAME_UNKNOWN"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := serveReferrersRequest(h, tt.method, tt.path)
if got.Code != tt.wantStatus {
t.Fatalf("status = %d, want %d: %s", got.Code, tt.wantStatus, got.Body.String())
}
if tt.wantCode != "" && !strings.Contains(got.Body.String(), tt.wantCode) {
t.Errorf("body = %q, want %s", got.Body.String(), tt.wantCode)
}
})
}
if upstreamRequests != 0 {
t.Errorf("upstream requests = %d, want 0", upstreamRequests)
}
}
func TestContainerHandler_ReferrersRouteKeepsManifestPaths(t *testing.T) {
tagSchema := "sha256-" + strings.TrimPrefix(testReferrersSubject, "sha256:")
var upstreamPaths []string
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
upstreamPaths = append(upstreamPaths, r.URL.Path)
// Both handlers build the same upstream path here, so tell them apart
// by Accept: only the referrers handler asks for the index type alone.
if r.Header.Get("Accept") == containerReferrersMediaType {
http.Error(w, "reached the referrers handler", http.StatusBadRequest)
return
}
w.Header().Set("Content-Type", containerReferrersMediaType)
_, _ = io.WriteString(w, testReferrersIndex)
}))
defer upstream.Close()
h, _ := newReferrersTestHandler(t, upstream)
for _, path := range []string{
// A repository may contain a "referrers" component; manifest paths
// must keep reaching the manifest handler.
"/foo/referrers/app/manifests/latest",
// Clients without a referrers API read the tag schema as a manifest.
"/library/nginx/manifests/" + tagSchema,
} {
if got := serveReferrersRequest(h, http.MethodGet, path); got.Code != http.StatusOK {
t.Fatalf("GET %s status = %d, want 200: %s", path, got.Code, got.Body.String())
}
}
want := []string{
"/v2/foo/referrers/app/manifests/latest",
"/v2/library/nginx/manifests/" + tagSchema,
}
if strings.Join(upstreamPaths, "\n") != strings.Join(want, "\n") {
t.Errorf("upstream paths = %q, want %q", upstreamPaths, want)
}
}
func TestContainerHandler_ReferrersRouteAllowsEndpointNamesInRepository(t *testing.T) {
var upstreamPaths []string
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
upstreamPaths = append(upstreamPaths, r.URL.Path)
// The manifest handler would forward the same path, so only answer
// the referrers handler, which asks for the index type alone.
if r.Header.Get("Accept") != containerReferrersMediaType {
http.Error(w, "reached the manifest handler", http.StatusBadRequest)
return
}
w.Header().Set("Content-Type", containerReferrersMediaType)
_, _ = io.WriteString(w, testReferrersIndex)
}))
defer upstream.Close()
h, _ := newReferrersTestHandler(t, upstream)
for _, name := range []string{"owner/tags/list", "owner/manifests/app"} {
upstreamPaths = nil
got := serveReferrersRequest(h, http.MethodGet, "/"+name+"/referrers/"+testReferrersSubject)
if got.Code != http.StatusOK {
t.Fatalf("%s: status = %d, want 200: %s", name, got.Code, got.Body.String())
}
want := "/v2/" + name + "/referrers/" + testReferrersSubject
if len(upstreamPaths) != 1 || upstreamPaths[0] != want {
t.Errorf("%s: upstream paths = %q, want [%q]", name, upstreamPaths, want)
}
}
}
func TestContainerHandler_ReferrersFollowsNamespace(t *testing.T) {
var upstreamQueries []string
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/v2/owner/img/referrers/"+testReferrersSubject {
http.NotFound(w, r)
return
}
upstreamQueries = append(upstreamQueries, r.URL.RawQuery)
w.Header().Set("Content-Type", containerReferrersMediaType)
w.Header().Set("Link", `</v2/owner/img/referrers/`+testReferrersSubject+`?last=abc>; rel="next"`)
_, _ = io.WriteString(w, testReferrersIndex)
}))
defer upstream.Close()
proxy, _, _, _ := setupTestProxy(t)
proxy.HTTPClient = upstream.Client()
proxy.MetadataTTL = time.Hour
Update x/net and remove unused production helpers
2026-10-19 08:35:46 +01:00
h := NewContainerHandlerWithRegistry(proxy, "http://proxy.example.test", "", map[string]string{"test": upstream.URL})
Proxy the OCI referrers API in the container handler (#427) * Proxy the OCI referrers API in the container handler GET /v2/{name}/referrers/{digest} used to fall through to a plain 404, so clients only ever saw referrers stored under the sha256-<hex> tag schema. Registries that keep SBOMs and signatures behind the referrers API looked empty through the proxy. The index is now fetched from the upstream that the repository routes to and cached like the tag list: fresh within the metadata TTL, revalidated with If-None-Match, and served stale when the upstream is down. artifactType is not forwarded, so one cached row covers every filter and clients filter the full list themselves, which they do when OCI-Filters-Applied is absent. An upstream 404 is passed through untouched, so clients still fall back to the tag schema for registries like GHCR. When the upstream cannot answer and nothing is cached, the proxy keeps answering 404 as before rather than failing the request. Closes #144 * Cover absolute upstream links in the referrers link test The second page now answers with an absolute Link pointing back at the upstream host, and the test checks that it comes back rewritten to the proxy. That was missing so far, and it also gives the upstream variable a reason to be declared before the handler closure, which staticcheck complained about. * Read cached referrers through a small shared loader loadContainerReferrers was a line-for-line copy of loadContainerTags with the types renamed, which dupl flags in CI. The part that opens the cache row and reads the body now lives in loadContainerMetadata next to storeContainerMetadata, and the referrers loader only maps the row onto its struct. The tag and manifest loaders are left as they are. * Make the manifest routing test notice which handler answered For foo/manifests/referrers/<digest> the manifest handler and the referrers handler send the very same upstream path, so the test passed even with the referrers case moved in front of the manifest case. The fake upstream now rejects requests that only accept an image index, which is what the referrers handler sends. * Check that a cached referrers row links back to whoever reads it A repository can be reached as upstream/<name>/<repo> and, when the default registry is the same host, as plain <repo>. Both end up in the same cache row, so the pagination Link has to be rewritten when the row is served, not when it is stored. The link test now reads the row through both paths; with a store-time rewrite the second path would get the first one's link. * Make the referrers ETag test prove the revalidation Without If-None-Match the fake upstream just sends the full index again, so the old assertions held either way. The test now counts the 304s the upstream sent and expects exactly one. * Reword which tools use the referrers API in the README The cosign mention was too broad. cosign reaches the referrers API for its newer signature bundles, while the old .sig tags keep going through the normal manifest path. Also rewraps the paragraph, one line had run past the others. * Check that a 304 restarts the referrers cache TTL The ETag test now backdates the cached row instead of running with a zero TTL. After the upstream answers 304, a third request has to come from the cache. Before, the test would also have passed if the 304 branch forgot to bump fetchedAt and every request kept revalidating. * Test the referrers Content-Type fallback None of the fake upstreams left out Content-Type, so the default to the image index type was never exercised. oras-go rejects anything else, so it is worth pinning, both for the first fetch and for the cached copy. * Test referrers digests with extra characters around them The invalid-request table only had digests that were too short or used an unknown algorithm, so it would not notice if the digest pattern lost its anchors. Two cases now put junk before and after an otherwise valid sha256 digest. * Reuse the manifest cache key for referrers The referrers handler had its own key function, but it did exactly what containerManifestCacheKey already does with four strings. Referrers rows sit in their own oci-referrers ecosystem anyway, so they can't collide with manifests. * Drop the separate struct for cached referrers A cached referrers response carries the same fields as a cached tag list, and the freshness check was a straight copy too. So the referrers code just uses cachedContainerTags and containerTagsFresh now. Loading, storing and writing stay on their own, since the ecosystem and the Link handling are different. * Route referrers requests by ns like the other OCI endpoints Now that containerd ns support is on main, referrers requests have to go through registryForRequest too, otherwise a _default mirror would look every subject up on Docker Hub. ns only picks the registry, so it is dropped from the forwarded query and the cache key, and the next-page Link keeps it, the same way tag lists do. This also follows the new rewriteContainerTagsLink signature, which takes the namespace. * Without a cached index, referrers now pass upstream errors through and answer 502 when the upstream can't be reached or sends garbage, so clients no longer read an outage as a registry without the referrers API. Stale cache is still served when there is one. * The tag list and manifest routes now only match paths that end in their endpoint, so a repository with tags/list or manifests in its name can still reach the referrers endpoint.
2026-10-12 10:08:01 +02:00
routes := http.StripPrefix("/v2", h.Routes())
host := strings.TrimPrefix(upstream.URL, "http://")
target := "/v2/owner/img/referrers/" + testReferrersSubject
withNS := httptest.NewRecorder()
routes.ServeHTTP(withNS, httptest.NewRequest(http.MethodGet, target+"?ns="+url.QueryEscape(host), nil))
if withNS.Code != http.StatusOK {
t.Fatalf("ns request status = %d, want 200: %s", withNS.Code, withNS.Body.String())
}
wantLink := `<http://proxy.example.test` + target + `?last=abc&ns=` + url.QueryEscape(host) + `>; rel="next"`
if got := withNS.Header().Get("Link"); got != wantLink {
t.Errorf("Link = %q, want next page kept on the ns route %q", got, wantLink)
}
// The prefix route reaches the same registry and shares the row.
prefixed := httptest.NewRecorder()
routes.ServeHTTP(prefixed, httptest.NewRequest(http.MethodGet, "/v2/upstream/test/owner/img/referrers/"+testReferrersSubject, nil))
if prefixed.Code != http.StatusOK {
t.Fatalf("prefix request status = %d, want 200: %s", prefixed.Code, prefixed.Body.String())
}
if len(upstreamQueries) != 1 || upstreamQueries[0] != "" {
t.Errorf("upstream queries = %q, want one request without ns", upstreamQueries)
}
}
Answer uncached referrers tags with 404 when the upstream is down Docker falls back to a sha256-<digest> tag when the referrers API fails and aborts the pull on anything but a manifest or 404.
2026-10-14 12:50:03 +01:00
// Docker looks up a sha256-<digest> referrers tag on every pull. With the
// upstream down and nothing cached, answering it as unknown lets a pull of a
// cached image go ahead; other tags still report the failure.
func TestContainerHandler_ReferrersTagIsUnknownWhenUpstreamFails(t *testing.T) {
referrersTag := "sha256-" + strings.Repeat("ab", 32)
tests := []struct {
name string
reference string
upstream func(t *testing.T) string
want int
}{
{"referrers tag, upstream 503", referrersTag, upstreamAnswering(http.StatusServiceUnavailable), http.StatusNotFound},
{"referrers tag, upstream unreachable", referrersTag, unreachableUpstream, http.StatusNotFound},
{"referrers tag, upstream 404", referrersTag, upstreamAnswering(http.StatusNotFound), http.StatusNotFound},
{"other tag, upstream 503", "latest", upstreamAnswering(http.StatusServiceUnavailable), http.StatusServiceUnavailable},
{"other tag, upstream unreachable", "latest", unreachableUpstream, http.StatusBadGateway},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
proxy, _, _, _ := setupTestProxy(t)
proxy.HTTPClient = &http.Client{}
h := &ContainerHandler{proxy: proxy, registryURL: tt.upstream(t), proxyURL: "http://localhost:8080"}
for _, method := range []string{http.MethodGet, http.MethodHead} {
w := httptest.NewRecorder()
h.Routes().ServeHTTP(w, httptest.NewRequest(method, "/library/nginx/manifests/"+tt.reference, nil))
if w.Code != tt.want {
t.Errorf("%s status = %d, want %d; body: %s", method, w.Code, tt.want, w.Body.String())
}
}
})
}
}
func upstreamAnswering(status int) func(t *testing.T) string {
return func(t *testing.T) string {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
http.Error(w, http.StatusText(status), status)
}))
t.Cleanup(srv.Close)
return srv.URL
}
}
func unreachableUpstream(t *testing.T) string {
srv := httptest.NewServer(http.NotFoundHandler())
srv.Close()
return srv.URL
}
Reference in a new issue Copy permalink