[OLM install method] [Trivial fix] Verb "list" for "(mutating|validating)webhookconfigurations" resources in API group "admissionregistration.k8s.io" missing in clusterPermissions: section of argocd-operator.<tag>.clusterserviceversion.yaml CSV file?

[iankko]

Describe the bug
Hello, firstly, thank you for the great operator!

Secondly, given the default OLM cluster role is overly strong, leading to associated security concern, in order to install the argocd-operator CSV (and thus subsequently the operator itself), we are parsing the clusterPermissions: section, listed in particular version of the ArgoCD operator CSV (since using v0.13.0 ATM, here's direct link to ArgoCD operator CSV of that version):

• https://github.com/argoproj-labs/argocd-operator/blob/master/deploy/olm-catalog/argocd-operator/0.13.0/argocd-operator.v0.13.0.clusterserviceversion.yaml#L1679-L1873

to determine the minimal set of K8s API groups, resources, and associated verbs, that need to be present in the OLM ClusterRole the ArgoCD operator to install correctly. This works pretty well (all checked dependants requirements are reported as Satisfied):

    Dependents:
      Group:    rbac.authorization.k8s.io
      Kind:     PolicyRule
      Message:  cluster rule:{"verbs":["create","delete","get","list","patch","update","watch"],"apiGroups":[""],"resources":["configmaps"]}
      Status:   Satisfied
      Version:  v1
      Group:    rbac.authorization.k8s.io
      Kind:     PolicyRule
      Message:  cluster rule:{"verbs":["*"],"apiGroups":[""],"resources":["configmaps","endpoints","events","namespaces","persistentvolumeclaims","pods","secrets","serviceaccounts","services","services/finalizers"]}
      Status:   Satisfied
      Version:  v1
      Group:    rbac.authorization.k8s.io
      Kind:     PolicyRule
      Message:  cluster rule:{"verbs":["get"],"apiGroups":[""],"resources":["pods","pods/log"]}
      Status:   Satisfied
      Version:  v1
      Group:    rbac.authorization.k8s.io
      Kind:     PolicyRule
      Message:  cluster rule:{"verbs":["get","list"],"apiGroups":["apiregistration.k8s.io"],"resources":["apiservices"]}
      Status:   Satisfied
      Version:  v1
      Group:    rbac.authorization.k8s.io
      Kind:     PolicyRule
      Message:  cluster rule:{"verbs":["*"],"apiGroups":["apps"],"resources":["daemonsets","deployments","replicasets","statefulsets"]}
      Status:   Satisfied
      Version:  v1
      Group:    rbac.authorization.k8s.io
      Kind:     PolicyRule
      Message:  cluster rule:{"verbs":["update"],"apiGroups":["apps"],"resources":["deployments/finalizers"],"resourceNames":["argocd-operator"]}
      Status:   Satisfied
      Version:  v1
      Group:    rbac.authorization.k8s.io
      Kind:     PolicyRule
      Message:  cluster rule:{"verbs":["*"],"apiGroups":["apps.openshift.io"],"resources":["deploymentconfigs"]}
      Status:   Satisfied
      Version:  v1
      Group:    rbac.authorization.k8s.io
      Kind:     PolicyRule
      Message:  cluster rule:{"verbs":["*"],"apiGroups":["argoproj.io"],"resources":["applications","appprojects"]}
      Status:   Satisfied
      Version:  v1
      Group:    rbac.authorization.k8s.io
      Kind:     PolicyRule
      Message:  cluster rule:{"verbs":["*"],"apiGroups":["argoproj.io"],"resources":["argocdexports","argocdexports/finalizers","argocdexports/status"]}
      Status:   Satisfied
      Version:  v1
      Group:    rbac.authorization.k8s.io
      Kind:     PolicyRule
      Message:  cluster rule:{"verbs":["*"],"apiGroups":["argoproj.io"],"resources":["argocds","argocds/finalizers","argocds/status"]}
      Status:   Satisfied
      Version:  v1
      Group:    rbac.authorization.k8s.io
      Kind:     PolicyRule
      Message:  cluster rule:{"verbs":["*"],"apiGroups":["argoproj.io"],"resources":["notificationsconfigurations","notificationsconfigurations/finalizers"]}
      Status:   Satisfied
      Version:  v1
      Group:    rbac.authorization.k8s.io
      Kind:     PolicyRule
      Message:  cluster rule:{"verbs":["*"],"apiGroups":["autoscaling"],"resources":["horizontalpodautoscalers"]}
      Status:   Satisfied
      Version:  v1
      Group:    rbac.authorization.k8s.io
      Kind:     PolicyRule
      Message:  cluster rule:{"verbs":["*"],"apiGroups":["batch"],"resources":["cronjobs","jobs"]}
      Status:   Satisfied
      Version:  v1
      Group:    rbac.authorization.k8s.io
      Kind:     PolicyRule
      Message:  cluster rule:{"verbs":["get","list","watch"],"apiGroups":["config.openshift.io"],"resources":["clusterversions"]}
      Status:   Satisfied
      Version:  v1
      Group:    rbac.authorization.k8s.io
      Kind:     PolicyRule
      Message:  cluster rule:{"verbs":["*"],"apiGroups":["monitoring.coreos.com"],"resources":["prometheuses","prometheusrules","servicemonitors"]}
      Status:   Satisfied
      Version:  v1
      Group:    rbac.authorization.k8s.io
      Kind:     PolicyRule
      Message:  cluster rule:{"verbs":["*"],"apiGroups":["networking.k8s.io"],"resources":["ingresses"]}
      Status:   Satisfied
      Version:  v1
      Group:    rbac.authorization.k8s.io
      Kind:     PolicyRule
      Message:  cluster rule:{"verbs":["create","delete","get","list","patch","update","watch"],"apiGroups":["networking.k8s.io"],"resources":["networkpolicies"]}
      Status:   Satisfied
      Version:  v1
      Group:    rbac.authorization.k8s.io
      Kind:     PolicyRule
      Message:  cluster rule:{"verbs":["create","delete","get","list","patch","update","watch"],"apiGroups":["oauth.openshift.io"],"resources":["oauthclients"]}
      Status:   Satisfied
      Version:  v1
      Group:    rbac.authorization.k8s.io
      Kind:     PolicyRule
      Message:  cluster rule:{"verbs":["*"],"apiGroups":["rbac.authorization.k8s.io"],"resources":["*"]}
      Status:   Satisfied
      Version:  v1
      Group:    rbac.authorization.k8s.io
      Kind:     PolicyRule
      Message:  cluster rule:{"verbs":["*"],"apiGroups":["rbac.authorization.k8s.io"],"resources":["clusterrolebindings","clusterroles"]}
      Status:   Satisfied
      Version:  v1
      Group:    rbac.authorization.k8s.io
      Kind:     PolicyRule
      Message:  cluster rule:{"verbs":["*"],"apiGroups":["route.openshift.io"],"resources":["routes","routes/custom-host"]}
      Status:   Satisfied
      Version:  v1
      Group:    rbac.authorization.k8s.io
      Kind:     PolicyRule
      Message:  cluster rule:{"verbs":["*"],"apiGroups":["template.openshift.io"],"resources":["templateconfigs","templateinstances","templates"]}
      Status:   Satisfied
      Version:  v1
      Group:    rbac.authorization.k8s.io
      Kind:     PolicyRule
      Message:  cluster rule:{"verbs":["create"],"apiGroups":["authentication.k8s.io"],"resources":["tokenreviews"]}
      Status:   Satisfied
      Version:  v1
      Group:    rbac.authorization.k8s.io
      Kind:     PolicyRule
      Message:  cluster rule:{"verbs":["create"],"apiGroups":["authorization.k8s.io"],"resources":["subjectaccessreviews"]}
      Status:   Satisfied
      Version:  v1

But except a single failure of ArgoCD operator failing to install the necessary webhooks correctly.

More info from $ kubectl -n <namespace> describe csv/argocd-operator.v0.13.0 Status section (the actual namespace in SA below was intentionally obfuscated with instead):

    ...
    Phase:                 InstallReady
    Reason:                InstallWaiting
    Last Transition Time:  2025-03-11T09:16:18Z
    Last Update Time:      2025-03-11T09:16:18Z
    Message:               waiting for install components to report healthy
    Phase:                 Installing
    Reason:                InstallSucceeded
    Last Transition Time:  2025-03-11T09:16:21Z
    Last Update Time:      2025-03-11T09:16:21Z
    Message:               Webhook install failed: validatingwebhookconfigurations.admissionregistration.k8s.io is forbidden: User "system:serviceaccount:<namespace>:olm-operator-serviceaccount" cannot list resource "validatingwebhookconfigurations" in API group "admissionregistration.k8s.io" at the cluster scope
    ...

The same error message is then reported couple of times in the ArgoCD operator CSV status.

Clearly, this can be fixed by adding the following rule* into the ArgoCD operator CSV clusterPermissions: sections yet (AKA patch is trivial):

- apiGroups:
  - "admissionregistration.k8s.io"
  resources:
  - "mutatingwebhookconfigurations"
  - "validatingwebhookconfigurations"
  verbs:
  - list

to the underlying OLM ClusterRole.

Therefore checking, should the clusterPermissions: section of ArgoCD operator CSV (e.g. v.0.13.0), contain also listing of validatingwebhookconfigurations within the required cluster permissions?

To Reproduce
Steps to reproduce the behavior:
Install OLM, replacing the default service account ClusterRole system:controller:operator-lifecycle-manager with the form of it, as follows:

apiVersion: {{ .Values.rbacApiVersion }}/v1
kind: ClusterRole
metadata:
  name: system:controller:operator-lifecycle-manager
rules:
# Basic rules for olm-operator & catalog-operator pods
- apiGroups:
  - "operators.coreos.com"
  resources:
  - "catalogsources"
  - "catalogsources/status"
  - "clusterserviceversions"
  - "clusterserviceversions/status"
  - "installplans"
  - "installplans/status"
  - "olmconfigs"
  - "olmconfigs/status"
  - "operatorconditions"
  - "operatorconditions/status"
  - "operatorgroups"
  - "operatorgroups/status"
  - "operators"
  - "operators/status"
  - "subscriptions"
  - "subscriptions/status"
  verbs: ["watch", "list", "get", "create", "update", "patch", "delete", "deletecollection", "escalate", "bind"]
- apiGroups:
  - "apiextensions.k8s.io"
  resources:
  - "customresourcedefinitions"
  verbs: ["watch", "list", "get", "create", "update", "patch", "delete"]
# Rules for argocd-operator CSV
{{ include "argocd-operator.csv.clusterpermissions" . }}
---

where argocd-operator.csv.clusterpermissions is taken from upstream CSV version (for clarity it's also inlined in its full form in next comment):

• https://github.com/argoproj-labs/argocd-operator/blob/master/deploy/olm-catalog/argocd-operator/0.13.0/argocd-operator.v0.13.0.clusterserviceversion.yaml#L1679-L1873

after dropping the leading eight space characters.

Expected behavior
ArgoCD operator CSV reports Succeeded, e.g.:

# kubectl -n <namespace> get csv
NAME                      DISPLAY          VERSION                        REPLACES                  PHASE
argocd-operator.v0.13.0   Argo CD          0.13.0                         argocd-operator.v0.12.0   Succeeded

Current behavior
Describe ArgoCD operator CSV instead reports InstallReady due failing to list validatingwebhookconfigurations and mutatingwebhookconfigurations from "admissionregistration.k8s.io" API group.

# # kubectl -n <namespace> get csv/argocd-operator.v0.13.0
NAME                      DISPLAY   VERSION   REPLACES                  PHASE
argocd-operator.v0.13.0   Argo CD   0.13.0    argocd-operator.v0.12.0   InstallReady

[iankko]

Yet, content of the promised _argocd-operator.csv.clusterpermissions.yaml (but it's truly just copy of upstream rules: defined in https://github.com/argoproj-labs/argocd-operator/blob/master/deploy/olm-catalog/argocd-operator/0.13.0/argocd-operator.v0.13.0.clusterserviceversion.yaml#L1680)

$ cat _argocd-operator.csv.clusterpermissions.yaml
# Taken from (after removing the leading eight space characters):
# https://github.com/argoproj-labs/argocd-operator/blob/master/deploy/olm-catalog/argocd-operator/0.13.0/argocd-operator.v0.13.0.clusterserviceversion.yaml#L1681-L1873
{{- define "argocd-operator.csv.clusterpermissions" -}}
- apiGroups:
  - ""
  resources:
  - configmaps
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - configmaps
  - endpoints
  - events
  - namespaces
  - persistentvolumeclaims
  - pods
  - secrets
  - serviceaccounts
  - services
  - services/finalizers
  verbs:
  - '*'
- apiGroups:
  - ""
  resources:
  - pods
  - pods/log
  verbs:
  - get
- apiGroups:
  - apiregistration.k8s.io
  resources:
  - apiservices
  verbs:
  - get
  - list
- apiGroups:
  - apps
  resources:
  - daemonsets
  - deployments
  - replicasets
  - statefulsets
  verbs:
  - '*'
- apiGroups:
  - apps
  resourceNames:
  - argocd-operator
  resources:
  - deployments/finalizers
  verbs:
  - update
- apiGroups:
  - apps.openshift.io
  resources:
  - deploymentconfigs
  verbs:
  - '*'
- apiGroups:
  - argoproj.io
  resources:
  - applications
  - appprojects
  verbs:
  - '*'
- apiGroups:
  - argoproj.io
  resources:
  - argocdexports
  - argocdexports/finalizers
  - argocdexports/status
  verbs:
  - '*'
- apiGroups:
  - argoproj.io
  resources:
  - argocds
  - argocds/finalizers
  - argocds/status
  verbs:
  - '*'
- apiGroups:
  - argoproj.io
  resources:
  - notificationsconfigurations
  - notificationsconfigurations/finalizers
  verbs:
  - '*'
- apiGroups:
  - autoscaling
  resources:
  - horizontalpodautoscalers
  verbs:
  - '*'
- apiGroups:
  - batch
  resources:
  - cronjobs
  - jobs
  verbs:
  - '*'
- apiGroups:
  - config.openshift.io
  resources:
  - clusterversions
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - monitoring.coreos.com
  resources:
  - prometheuses
  - prometheusrules
  - servicemonitors
  verbs:
  - '*'
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses
  verbs:
  - '*'
- apiGroups:
  - networking.k8s.io
  resources:
  - networkpolicies
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - oauth.openshift.io
  resources:
  - oauthclients
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - rbac.authorization.k8s.io
  resources:
  - '*'
  verbs:
  - '*'
- apiGroups:
  - rbac.authorization.k8s.io
  resources:
  - clusterrolebindings
  - clusterroles
  verbs:
  - '*'
- apiGroups:
  - route.openshift.io
  resources:
  - routes
  - routes/custom-host
  verbs:
  - '*'
- apiGroups:
  - template.openshift.io
  resources:
  - templateconfigs
  - templateinstances
  - templates
  verbs:
  - '*'
- apiGroups:
  - authentication.k8s.io
  resources:
  - tokenreviews
  verbs:
  - create
- apiGroups:
  - authorization.k8s.io
  resources:
  - subjectaccessreviews
  verbs:
  - create
{{- end -}}

[svghadi]

Thanks @iankko for raising this issue. It was very informative to read through the OLM issue which you linked. I like the idea of least privileges. However, I believe csv.spec.install.spec.clusterPermissions is not the right place to add mutatingwebhookconfigurations & validatingwebhookconfigurations permissions.

My understanding is that the clusterPermissions field in CSV lists all the rules required by operator's service account i.e argocd-operator-controller-manager to function properly. While the Argo CD Operator uses a conversion webhook, it relies on OLM to register it with K8s. Therefore the OLM's operator-lifecycle-manager service account needs to have the required permissions. If it is added into clusterPermissions in CSV, it will end up in Argo CD Operator's argocd-operator-controller-manager service account which is unnecessary as the operator itself doesn't require these permissions to function.

I believe the correct place to add these webhook permissions is in ClusterRole: system:controller:operator-lifecycle-manager, which you create for OLM

apiVersion: {{ .Values.rbacApiVersion }}/v1
kind: ClusterRole
metadata:
  name: system:controller:operator-lifecycle-manager
rules:
# Basic rules for olm-operator & catalog-operator pods
.......
.......
.......
# Rules for webhook
- apiGroups:
  - "admissionregistration.k8s.io"
  resources:
  - "mutatingwebhookconfigurations"
  - "validatingwebhookconfigurations"
  verbs:
  - list
# Rules for argocd-operator CSV
{{ include "argocd-operator.csv.clusterpermissions" . }}
---

Let me know your thoughts. Happy to discuss further! 😄