Skip to content

feat: add support for watching pods across multiple namespaces #106

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Stream Kubernetes Pod events to the Coder startup logs.
- Easily determine the reason for a pod provision failure, or why a pod is stuck in a pending state.
- Visibility into when pods are OOMKilled, or when they are evicted.
- Filter by namespace, field selector, and label selector to reduce Kubernetes API load.
- Support for watching pods across multiple namespaces or all namespaces.

![Log Stream](./scripts/demo.png)

Expand All @@ -24,6 +25,27 @@ helm install coder-logstream-kube coder-logstream-kube/coder-logstream-kube \
--set url=<your-coder-url-including-http-or-https>
```

### Multi-Namespace Support

By default, `coder-logstream-kube` will watch pods in all namespaces. This is useful for deployments where workspaces are spread across multiple namespaces (e.g., per-user namespaces).

To watch all namespaces (default behavior):
```console
helm install coder-logstream-kube coder-logstream-kube/coder-logstream-kube \
--namespace coder \
--set url=<your-coder-url>
```

To watch a specific namespace only:
```console
helm install coder-logstream-kube coder-logstream-kube/coder-logstream-kube \
--namespace coder \
--set url=<your-coder-url> \
--set namespace=<target-namespace>
```

**Important**: When watching all namespaces, the Helm chart will create a `ClusterRole` and `ClusterRoleBinding` to provide the necessary cluster-wide permissions. When watching a specific namespace, it will create a `Role` and `RoleBinding` scoped to that namespace.

> **Note**
> For additional customization (such as customizing the image, pull secrets, annotations, etc.), you can use the
> [values.yaml](helm/values.yaml) file directly.
Expand All @@ -46,6 +68,8 @@ Kubernetes provides an [informers](https://pkg.go.dev/k8s.io/client-go/informers

`coder-logstream-kube` listens for pod creation events with containers that have the `CODER_AGENT_TOKEN` environment variable set. All pod events are streamed as logs to the Coder API using the agent token for authentication.

When no namespace is specified (or the `CODER_NAMESPACE` environment variable is empty), the informers will watch all namespaces in the cluster. When a specific namespace is provided, the informers are scoped to that namespace only.

## Custom Certificates

- [`SSL_CERT_FILE`](https://go.dev/src/crypto/x509/root_unix.go#L19): Specifies the path to an SSL certificate.
Expand Down
52 changes: 52 additions & 0 deletions README.md.backup
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# coder-logstream-kube

[![discord](https://img.shields.io/discord/747933592273027093?label=discord)](https://discord.gg/coder)
[![release](https://img.shields.io/github/v/tag/coder/coder-logstream-kube)](https://github.com/coder/envbuilder/pkgs/container/coder-logstream-kube)
[![godoc](https://pkg.go.dev/badge/github.com/coder/coder-logstream-kube.svg)](https://pkg.go.dev/github.com/coder/coder-logstream-kube)
[![license](https://img.shields.io/github/license/coder/coder-logstream-kube)](./LICENSE)

Stream Kubernetes Pod events to the Coder startup logs.

- Easily determine the reason for a pod provision failure, or why a pod is stuck in a pending state.
- Visibility into when pods are OOMKilled, or when they are evicted.
- Filter by namespace, field selector, and label selector to reduce Kubernetes API load.

![Log Stream](./scripts/demo.png)

## Usage

Apply the Helm chart to start streaming logs into your Coder instance:

```console
helm repo add coder-logstream-kube https://helm.coder.com/logstream-kube
helm install coder-logstream-kube coder-logstream-kube/coder-logstream-kube \
--namespace coder \
--set url=<your-coder-url-including-http-or-https>
```

> **Note**
> For additional customization (such as customizing the image, pull secrets, annotations, etc.), you can use the
> [values.yaml](helm/values.yaml) file directly.

Your Coder template should be using a `kubernetes_deployment` resource with `wait_for_rollout` set to `false`.

```hcl
resource "kubernetes_deployment" "hello_world" {
count = data.coder_workspace.me.start_count
wait_for_rollout = false
...
}
```

This ensures all pod events will be sent during initialization and startup.

## How?

Kubernetes provides an [informers](https://pkg.go.dev/k8s.io/client-go/informers) API that streams pod and event data from the API server.

`coder-logstream-kube` listens for pod creation events with containers that have the `CODER_AGENT_TOKEN` environment variable set. All pod events are streamed as logs to the Coder API using the agent token for authentication.

## Custom Certificates

- [`SSL_CERT_FILE`](https://go.dev/src/crypto/x509/root_unix.go#L19): Specifies the path to an SSL certificate.
- [`SSL_CERT_DIR`](https://go.dev/src/crypto/x509/root_unix.go#L25): Identifies which directory to check for SSL certificate files.
46 changes: 38 additions & 8 deletions helm/templates/service.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
{{- if .Values.namespace }}
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
Expand All @@ -10,13 +11,6 @@ rules:
resources: ["replicasets", "events"]
verbs: ["get", "watch", "list"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ .Values.serviceAccount.name | quote }}
annotations: {{ toYaml .Values.serviceAccount.annotations | nindent 4 }}
labels: {{ toYaml .Values.serviceAccount.labels | nindent 4 }}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
Expand All @@ -28,6 +22,39 @@ roleRef:
subjects:
- kind: ServiceAccount
name: {{ .Values.serviceAccount.name | quote }}
{{- else }}
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: coder-logstream-kube-clusterrole
rules:
- apiGroups: [""]
resources: ["pods", "events"]
verbs: ["get", "watch", "list"]
- apiGroups: ["apps"]
resources: ["replicasets", "events"]
verbs: ["get", "watch", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: coder-logstream-kube-clusterrolebinding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: coder-logstream-kube-clusterrole
subjects:
- kind: ServiceAccount
name: {{ .Values.serviceAccount.name | quote }}
namespace: {{ .Release.Namespace }}
{{- end }}
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ .Values.serviceAccount.name | quote }}
annotations: {{ toYaml .Values.serviceAccount.annotations | nindent 4 }}
labels: {{ toYaml .Values.serviceAccount.labels | nindent 4 }}
---
apiVersion: apps/v1
kind: Deployment
Expand Down Expand Up @@ -75,8 +102,10 @@ spec:
env:
- name: CODER_URL
value: {{ .Values.url }}
{{- if .Values.namespace }}
- name: CODER_NAMESPACE
value: {{ .Values.namespace | default .Release.Namespace }}
value: {{ .Values.namespace }}
{{- end }}
{{- if .Values.image.sslCertFile }}
- name: SSL_CERT_FILE
value: {{ .Values.image.sslCertFile }}
Expand All @@ -95,3 +124,4 @@ spec:
{{- if .Values.volumes }}
volumes: {{- toYaml .Values.volumes | nindent 8 }}
{{- end }}

97 changes: 97 additions & 0 deletions helm/templates/service.yaml.backup
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: coder-logstream-kube-role
rules:
- apiGroups: [""]
resources: ["pods", "events"]
verbs: ["get", "watch", "list"]
- apiGroups: ["apps"]
resources: ["replicasets", "events"]
verbs: ["get", "watch", "list"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ .Values.serviceAccount.name | quote }}
annotations: {{ toYaml .Values.serviceAccount.annotations | nindent 4 }}
labels: {{ toYaml .Values.serviceAccount.labels | nindent 4 }}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: coder-logstream-kube-rolebinding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: coder-logstream-kube-role
subjects:
- kind: ServiceAccount
name: {{ .Values.serviceAccount.name | quote }}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: coder-logstream-kube
spec:
# This must remain at 1 otherwise duplicate logs can occur!
replicas: 1
selector:
matchLabels:
app.kubernetes.io/instance: {{ .Release.Name }}
template:
metadata:
labels:
app.kubernetes.io/instance: {{ .Release.Name }}
{{- with .Values.labels }}
{{- toYaml . | nindent 8 }}
{{- end }}
spec:
serviceAccountName: {{ .Values.serviceAccount.name | quote }}
restartPolicy: Always
{{- with .Values.image.pullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
containers:
- name: coder-logstream-kube
image: "{{ .Values.image.repo }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
command:
- /coder-logstream-kube
resources: {{ toYaml .Values.resources | nindent 12 }}
env:
- name: CODER_URL
value: {{ .Values.url }}
- name: CODER_NAMESPACE
value: {{ .Values.namespace | default .Release.Namespace }}
{{- if .Values.image.sslCertFile }}
- name: SSL_CERT_FILE
value: {{ .Values.image.sslCertFile }}
{{- end }}
{{- if .Values.image.sslCertDir }}
- name: SSL_CERT_DIR
value: {{ .Values.image.sslCertDir }}
{{- end }}
{{- with .Values.securityContext }}
securityContext:
{{- toYaml . | nindent 12 }}
{{- end }}
{{- if .Values.volumeMounts }}
volumeMounts: {{- toYaml .Values.volumeMounts | nindent 12 }}
{{- end }}
{{- if .Values.volumes }}
volumes: {{- toYaml .Values.volumes | nindent 8 }}
{{- end }}
10 changes: 7 additions & 3 deletions helm/values.yaml
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
# url -- The URL of your Coder deployment. Must prefix with http or https
url: ""

# namespace -- The namespace to searching for Pods within.
# If unspecified, this defaults to the Helm namespace.
# namespace -- The namespace to search for Pods within.
# If unspecified or empty, coder-logstream-kube will watch pods in all namespaces.
# When watching all namespaces, ClusterRole and ClusterRoleBinding will be created
# instead of Role and RoleBinding to provide the necessary permissions.
# If specified, only pods in that namespace will be watched and Role/RoleBinding
# will be used for namespace-scoped permissions.
namespace: ""

# volumes -- A list of extra volumes to add to the coder-logstream pod.
Expand Down Expand Up @@ -101,3 +104,4 @@ securityContext: {}
# runAsNonRoot: true
# seccompProfile:
# type: RuntimeDefault

103 changes: 103 additions & 0 deletions helm/values.yaml.backup
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# url -- The URL of your Coder deployment. Must prefix with http or https
url: ""

# namespace -- The namespace to searching for Pods within.
# If unspecified, this defaults to the Helm namespace.
namespace: ""

# volumes -- A list of extra volumes to add to the coder-logstream pod.
volumes:
# emptyDir: {}
# - name: "my-volume"

# volumeMounts -- A list of extra volume mounts to add to the coder-logstream pod.
volumeMounts:
# - name: "my-volume"
# mountPath: "/mnt/my-volume"

# image -- The image to use.
image:
# image.repo -- The repository of the image.
repo: "ghcr.io/coder/coder-logstream-kube"
# image.tag -- The tag of the image, defaults to {{.Chart.AppVersion}}
# if not set. If you're using the chart directly from git, the default
# app version will not work and you'll need to set this value. The helm
# chart helpfully fails quickly in this case.
tag: ""
# image.pullPolicy -- The pull policy to use for the image. See:
# https://kubernetes.io/docs/concepts/containers/images/#image-pull-policy
pullPolicy: IfNotPresent
# image.pullSecrets -- The secrets used for pulling the Coder image from
# a private registry.
pullSecrets: []
# - name: "pull-secret"
# image.sslCertFile -- Location of the SSL certificate file. Sets the $SSL_CERT_FILE
# variable inside of the container.
sslCertFile: ""
# image.sslCertDir -- Directory to check for SSL certificate files. Sets the $SSL_CERT_DIR
# variable inside of the container.
sslCertDir: ""

serviceAccount:
# serviceAccount.annotations -- The service account annotations.
annotations: {}
# serviceAccount.labels -- The service account labels.
labels: {}
# coder.serviceAccount.name -- The service account name
name: coder-logstream-kube

# resources -- The resources to request for the Deployment. These are optional
# and are not set by default.
resources:
{}
# limits:
# cpu: 500m
# memory: 500Mi
# requests:
# cpu: 2000m
# memory: 2000Mi

# nodeSelector -- Node labels for constraining the coder-logstream pod to specific nodes.
nodeSelector: {}

# affinity -- Allows specifying an affinity rule for the Deployment.
# The default rule prefers to schedule coder pods on different
# nodes, which is only applicable if coder.replicaCount is greater than 1.
affinity:
{}
# podAntiAffinity:
# preferredDuringSchedulingIgnoredDuringExecution:
# - podAffinityTerm:
# labelSelector:
# matchExpressions:
# - key: app.kubernetes.io/instance: coder-logstream-kube
# operator: In
# values:
# - "true"
# topologyKey: kubernetes.io/hostname
# weight: 1

# tolerations -- Tolerations for tainted nodes.
# See: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
tolerations:
{}
# - key: "key"
# operator: "Equal"
# value: "value"
# effect: "NoSchedule"

# labels -- The pod labels for coder-logstream-kube. See:
# https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
labels: {}

# securityContext -- Container-level security context
# See: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/
securityContext: {}
# allowPrivilegeEscalation: false
# capabilities:
# drop:
# - ALL
# readOnlyRootFilesystem: true
# runAsNonRoot: true
# seccompProfile:
# type: RuntimeDefault
Loading
Loading