[cluster-autoscaler] Add build support for ARM64

[hakman]

I tried to keep existing target names to be as close as possible to existing ones and to build only AMD64 by default.
Similar targets with the -arch-% suffix were added and allow specifying the architecture, for example build-arch-amd64 and build-arch-arm64.

A new Dockerfile had to be added for ARM64 support and also a target named push-manifest for generating the multi-arch manifest after both AMD64 and ARM64 images are pushed.
Releasing both AMD64 and ARM64 is a bit awkward due to the About to push image/manifest. Are you sure? [y/N] input in push_image.sh.

The only major change is that both the binaries and the images are now suffixed with the arch name.

Fixes: #3419

/cc @MaciekPytel @gjtempleton

[mwielgus]

@gjtempleton Are there any implications of this pr to Helm charts/releases?

[detiber]

Couldn't this make use of an additional ARG for the architecture to avoid having to have two separate docker files?

[hakman]

The ARG would work here, but not further when copying the binary. Not sure if there's a good way to not add both ARG and ENV var. Separate Dockerfile seemed cleaner.

[detiber]

Unless things have changed recently, this would require that the arch specific images already be pushed, in which case I think it might make sense to handle the manifest creation/pushing in make rather than here.

[hakman]

The push process is a bit strange, as it asks for confirmation that you want to push the image. Not sure if still needed considering that after promoting images from k8s.io they cannot be changed anymore.
I think you are right and the manifest doesn't need these checks as it just uses the already pushed images.

[gjtempleton]

@gjtempleton Are there any implications of this pr to Helm charts/releases?

Not at this point, no. The next release bumping the default image after this being merged would need to change the image default to the newly published amd64 image, but nothing blocking this going ahead from the helm side.

[hakman]

Not at this point, no. The next release bumping the default image after this being merged would need to change the image default to the newly published amd64 image, but nothing blocking this going ahead from the helm side.

@gjtempleton any special reason to default to the -amd64 image instead of leaving it as is, pointing to the manifest?

[hakman]

Thanks for the review @detiber. I added most of the suggested changes, but there are a few things that would need to be discussed, especially the way the push image script works and if it needs to change.

[mwielgus]

Can you squash commits to just 1?

[hakman]

Can you squash commits to just 1?

Done. :)

[mwielgus]

@detiber are you OK with the PR in the current shape?

[detiber]

/lgtm

[mwielgus]

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hakman, mwielgus

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• cluster-autoscaler/OWNERS [mwielgus]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[hakman]

Thanks everyone for helping with this.
I don't know the cherry-picking policies for cluster-autoscaler, but would like to know if there's any chance to cherry-pick this to 1.19. It should not change in any way the current functionality.