Running codegen against server-provided swagger spec

[Arnavion]

The API server exposes its spec at /openapi/v2

Is it worth allowing a user to run k8s-openapi-codegen against this endpoint to generate a custom API bindings crate?

---

Edit: Starting with v1.15, CRDs are also included in the spec:

beta: CustomResourceDefinition OpenAPI Publishing

OpenAPI specs for native types have been served at /openapi/v2 by kube-apiserver for a long time, and they are consumed by a number of components, notably kubectl client-side validation, kubectl explain and OpenAPI based client generators.

OpenAPI publishing for CRDs will be available with Kubernetes 1.15 as beta, yet again only for structural schemas.

[ragne]

Tried to tackle with that recently; set spec_url to http://127.0.0.1:8001/openapi/v2, codegen failed with:

[v1_16] INFO src\main.rs:110 Parsing spec file at http://127.0.0.1:8001/openapi/v2 ...
[v1_16] INFO src\main.rs:119 Applying fixups...
[v1_16] operation patchAcmeCertManagerIoV1alpha2NamespacedChallenge is a patch operation but doesn't have a force parameter

I've tried to modify fixups.rs to skip that for now, replacing returning an Err with continue, then codegen failed with:

[v1_16] Nested anonymous types not supported

Any ideas how to implement support for anonymous types?

[Arnavion]

Depends. Share your AcmeCertManagerIo CRD.

[Arnavion]

Also, if the only reason you're attempting this is to get CRUD API for your CRD, you may want to just use k8s-openapi-derive instead. (Read the docs from the source because docs.rs is still broken for proc-macro crates.)

[ragne]

Unfortunately, that's not mine CRD. Those came from cert-manager: https://github.com/jetstack/cert-manager/blob/v0.11.0/deploy/manifests/00-crds.yaml#L29
I guess that Nested anonymous types codegen refers to is actually a listMeta with corresponding openapi schema: https://gist.github.com/ragne/ecebdbb09a29658c03624c5f03247683
Basically every CRD that allows listing resources having that one, but some refers to it as:

                "metadata": {
                    "description": "Standard list metadata. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds",
                    "$ref": "#/definitions/io.k8s.apimachinery.pkg.apis.meta.v1.ListMeta_v2"
                }

while others put all def into metadata directly.

I think one can try to construct the appropriate ListMeta struct dynamically or find already defined ListMeta or replace the literal definition with $ref.

[Arnavion]

I guess that Nested anonymous types codegen refers to is actually a listMeta with corresponding openapi schema: https://gist.github.com/ragne/ecebdbb09a29658c03624c5f03247683

I'm surprised that Kubernetes is inlining the defintion like that instead of using a $ref like it does for built-in *List types. The list type is synthesized by Kubernetes after all, so it should treat it just like a PodList etc.

It's worth testing if it does that for all CRDs, eg for the one in this repo's test.

I think one can try to construct the appropriate ListMeta struct dynamically or find already defined ListMeta or replace the literal definition with $ref.

Replacing it with a $ref to ListMeta isn't the hard part. The hard part is making the decision to replace it, ie determining that ChallengeList is a list and thus its metadata ought to be a $ref to ListMeta.

[ragne]

Might be wrong, looks like when custom resource is described in code and uses kubebuilder to generate yamls it can also be annotated to generate openapi spec (haven’t yet looked how the generated openapi spec gets into k8s) via +k8s:openapi-gen=true.
The generator looks for inline jsonTag to perform inlining

[Arnavion]

I think one can try to construct the appropriate ListMeta struct dynamically or find already defined ListMeta or replace the literal definition with $ref.

Replacing it with a $ref to ListMeta isn't the hard part. The hard part is making the decision to replace it, ie determining that ChallengeList is a list and thus its metadata ought to be a $ref to ListMeta.

So in 4c55b81 I've implemented emitting lists as aliases of a single k8s_openapi::List<T> type instead of their raw spec, which means the code generator already has to guess what is a list and what isn't.

Currently it believes anything with an items property and a metadata property that's a ref to a ListMeta is a List. It should be fine to extend that to also consider anything with a metadata property that's instead a SchemaKind::Properties with the same structure as spec.definitions.get("...ListMeta").

[wdv4758h]

I use k8s-openapi-codegen for Traefik CRDs recently. I use some hack directly in k8s-openapi-codegen codebase, and tweak the Swagger manually. Making a CLI tool to accept arbitrary CRD URL or Swagger file seems valuable.

[ibotty]

@wdv4758h Can you share how you did that?

[DD5HT]

I think it would be super valuable to generate custom API bindings.

[wdv4758h]

@wdv4758h Can you share how you did that?

@ibotty It was years ago, I don't have the source code of the hack version anymore. And I forgot the real steps.

IIRC, what I did was:

• build the vanilla k8s-openapi-codegen, just to make sure I can run it
• edit URL in k8s-openapi-codegen/src/supported_version.rs which will be requested in k8s-openapi-codegen/src/main.rs (I launched a miniserve with pre-downloaded spec I need)
• fix some Swagger format issue (I think some ref stuffs was not implemented for k8s-openapi-codegen, so need a more simple and plain version)
• maybe comment out some special fixups k8s-openapi-codegen (not sure)
• compile and run, get the generated binding for your CRDs.

[Arnavion]

k8s-openapi-codegen has been able to generate bindings from arbitrary file path or URL for quite some time:

k8s-openapi/k8s-openapi-codegen/src/main.rs

Lines 125 to 143 in 5a079ad

	struct Options {
	/// This parameter specifies the versions of Kubernetes that the API bindings should be generated for.
	///
	/// `--generate=1.20` means "generate bindings for Kubernetes v1.20,
	/// using that version's OpenAPI spec from the https://github.com/kubernetes/kubernetes repository".
	///
	/// `--generate=1.20:https://example.org/swagger.json` means "generate bindings for v1.20
	/// using the OpenAPI spec at the URL https://example.org/swagger.json".
	///
	/// `--generate=1.20:file:///path/to/swagger.json` means "generate binding for v1.20,
	/// using the OpenAPI spec in the file /path/to/swagger.json".
	///
	/// This parameter can be specified multiple times to generate bindings for multiple versions.
	///
	/// If this parameter isn't specified, bindings will be generated for all supported versions,
	/// using their respective OpenAPI specs from the https://github.com/kubernetes/kubernetes repository.
	#[clap(long = "generate", value_name = "VERSION")]
	versions: Vec<RequestedVersion>,
	}

It's not meant for general purpose use outside the k8s-openapi crate, but you can try pointing it to your /openapi/v2 endpoint or to a pre-downloaded spec file using an arbitrary version string for the parameter name and see how usable the output is.