[Fix] Remove remote client of insecurely setup cluster

[mszadkow]

What type of PR is this?

/kind bug

What this PR does / why we need it:

Even though we validate and prevent the use of invalid/insecure kubeconfigs, the remoteClient of the Cluster that kuebconfig was updated with invalid kubeconfig still remains on the list of valid clusters.
Because of that it is still possible to admit a workload to this cluster.
To give a stronger message to the user that Kubeconfig was improperly updated remoteClient will be removed from the clusterReconciler remoteClients list.

Which issue(s) this PR fixes:

Relates to this PR: #7483

Special notes for your reviewer:

Does this PR introduce a user-facing change?

MultiKueue: Remove remoteClient from clusterReconciler when kubeconfig is detected as invalid or insecure, preventing workloads from being admitted to misconfigured clusters.

[netlify]

✅ Deploy Preview for kubernetes-sigs-kueue canceled.

Name	Link
🔨 Latest commit	a65a2f7
🔍 Latest deploy log	https://app.netlify.com/projects/kubernetes-sigs-kueue/deploys/6908af1c0926940008edccf5

[mimowo]

LGTM, I will merge after rebased

[mimowo]

/lgtm
/approve
/cherrypick release-0.14
/cherrypick release-0.13

[k8s-infra-cherrypick-robot]

@mimowo: once the present PR merges, I will cherry-pick it on top of release-0.13, release-0.14 in new PRs and assign them to you.

In response to this:

/lgtm
/approve
/cherrypick release-0.14
/cherrypick release-0.13

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[k8s-ci-robot]

LGTM label has been added.

Git tree hash: e375806e3444d60a3cb21983ba96d08686918dc0

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mimowo, mszadkow

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [mimowo]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[mimowo]

https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_kueue/7486/pull-kueue-test-integration-baseline-main/1985355810229719040

is a known flake

[mimowo]

/test pull-kueue-test-integration-baseline-main

[k8s-infra-cherrypick-robot]

@mimowo: #7486 failed to apply on top of branch "release-0.14":

Applying: Remove remote client of insecurely setup cluster
Using index info to reconstruct a base tree...
M	pkg/controller/admissionchecks/multikueue/multikueuecluster.go
M	pkg/controller/admissionchecks/multikueue/multikueuecluster_test.go
M	test/integration/multikueue/setup_test.go
Falling back to patching base and 3-way merge...
Auto-merging test/integration/multikueue/setup_test.go
CONFLICT (content): Merge conflict in test/integration/multikueue/setup_test.go
Auto-merging pkg/controller/admissionchecks/multikueue/multikueuecluster_test.go
Auto-merging pkg/controller/admissionchecks/multikueue/multikueuecluster.go
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config advice.mergeConflict false"
Patch failed at 0001 Remove remote client of insecurely setup cluster

In response to this:

/lgtm
/approve
/cherrypick release-0.14
/cherrypick release-0.13

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[k8s-infra-cherrypick-robot]

@mimowo: #7486 failed to apply on top of branch "release-0.13":

Applying: Remove remote client of insecurely setup cluster
Using index info to reconstruct a base tree...
M	pkg/controller/admissionchecks/multikueue/multikueuecluster.go
M	pkg/controller/admissionchecks/multikueue/multikueuecluster_test.go
M	test/integration/multikueue/setup_test.go
Falling back to patching base and 3-way merge...
Auto-merging test/integration/multikueue/setup_test.go
CONFLICT (content): Merge conflict in test/integration/multikueue/setup_test.go
Auto-merging pkg/controller/admissionchecks/multikueue/multikueuecluster_test.go
Auto-merging pkg/controller/admissionchecks/multikueue/multikueuecluster.go
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config advice.mergeConflict false"
Patch failed at 0001 Remove remote client of insecurely setup cluster

In response to this:

/lgtm
/approve
/cherrypick release-0.14
/cherrypick release-0.13

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.