quay.io tag list rework

[mjnagel]

Tell us more.

This is related to some previous discussions:

• Quay.io authentication problem #29682
• Quay authentication issue #10188

In digging into these previous discussions it seems like:

• There are/were limitations on the v2 api with Quay that made it inefficient to perform tag lookups
• The api/v1 from Quay is being used instead in Renovate for Quay images as it can return the latest tags more efficiently
• The Quay v1 API requires an OAuth token that is different and separate from a normal pull token for the v2 endpoints
• As a result of the separate auth you need to setup 2 host rules with different authentication to allow Renovate to properly check (private) quay dependencies
• Potentially there are still issues (see here) with log output, even if both auth formats are provided

As a consumer of some private images from quay.io, I have a robot account that can be used to pull those private images. Unfortunately, I do not have an OAuth token for the api/v1 endpoints, meaning that I'm currently unable to use Renovate to monitor/update these images. After digging into quay, a few other tools, and renovate code I think there may be a few alternative paths to make the quay lookup/auth process easier.

If trying to use the v2 API only the first step is to use basic auth to acquire a token from the /v2/auth endpoint:

# Example with public image quay.io/prometheus/prometheus
curl -s -X GET https://quay.io/v2/auth\?scope\=repository%3Aprometheus%2Fprometheus%3Apull\&service\=quay.io
# Example with "private image" quay.io/private/image
curl -s -X GET -u "username:password" https://quay.io/v2/auth?scope=repository%3Aprivate%2Fimages%3Apull&service=quay.io

This API request returns JSON with a token (I think at this point this is similar to other registries). That token can then be used in tag list calls:

# Example with quay.io/prometheus/prometheus, using the token from the auth response
curl -s -X GET -H "Authorization: Bearer $TOKEN" https://quay.io/v2/prometheus/prometheus/tags/list

This is where this approach can be problematic. As noted in previous discussions the results returned from this list (1) start with the oldest tags and (2) are limited to 100 results. Previously it seems like results were limited to 50 results so this is a slight improvement, but still inefficient. Each request returns a link header as well that can be followed to get the "next page" of results. As noted in previous discussions this could be 100+ pages of results for some images with a large amount of tags.

I do think however that it could be usable in some cases, IF using the last query parameter in the request. As an example of what this might look like:

# Example with quay.io/prometheus/prometheus, using 3.3.0 as the last tag
❯ curl -s -X GET -H "Authorization: Bearer $TOKEN" "https://quay.io/v2/prometheus/prometheus/tags/list?n=100&last=v3.3.0"
{"name":"prometheus/prometheus","tags":["v3.3.0-rc.0","v3.3.0-rc.1","v3.3.1","v3.4.0","v3.4.0-rc.0","v3.4.1"]}

In theory renovate could use the currentVersion for this query parameter, limiting the results to only tags newer than the tag currently being used. To note two potential edge cases here:

• Currently using a nonexistent tag returns all tags from the start, inefficient but will ensure that the latest can still be grabbed

❯ curl -s -X GET -H "Authorization: Bearer $TOKEN" "https://quay.io/v2/prometheus/prometheus/tags/list?n=100&last=doesnotexist"
{"name":"prometheus/prometheus","tags":["latest","main","master","v1.0.0","v1.0.0-rc.0",...

• Currently using the latest tag returns an empty list of tags, seems acceptable

❯ curl -s -X GET -H "Authorization: Bearer $TOKEN" "https://quay.io/v2/prometheus/prometheus/tags/list?n=100&last=v3.4.1"
{"name":"prometheus/prometheus","tags":[]}

What I would propose to rework the quay.io auth logic (and be willing to contribute changes towards this):

• (potentially) Keep the current logic to use api/v1 first/by default (this would continue to be used for all public images in other words)
• if api/v1 returns 401 errors, fallback to v2 (this would likely be the case only for private images)
• When using the v2 api with quay, append the last query param with the current version to limit the number of pages Renovate has to query

I wanted to open this as a discussion to start and would be willing to contribute changes if we can find an acceptable path forward. Also happy to provide additional logs/requests that might help with identifying a different path forward.

[mjnagel]

I pushed up a commit on my fork that I think would implement the proposal mentioned: 95134f6

I can get this opened as a PR if there is interest - it should behave the same for most users since public images will still hit the api/v1. For private images, if folks had auth working, that would also behave the same. Then for private images where there is no auth to the v1 api (401 response):

• It will fallback to v2, hopefully working rather than failing dep lookups
• It may have a slight efficiency hit because of the quay limitations
• Efficiency hit should be partially mitigated by the last query param

The main thing I'm not sure on with is some of the nuances around currentValue and whether we may need to use extractVersion to get the "raw tag" if we want to use it with the quay query.