[META] Define semantic conventions for k8s metrics

[ChrsMark]

Area(s)

area:k8s

Is your change request related to a problem? Please describe.

At the moment there are not Semantic Conventions for k8s metrics.

Describe the solution you'd like

Even if we cannot consider the k8s metrics as stable we can start considering adding metrics that are not controversial to get some progress here. This issue aims to collect the existing k8s metrics that exist in the Collector and keep track of any related work.
Bellow I'm providing an initial list with metrics coming from the kubeletstats and k8scluster receivers. Note that these are matter to change with time being so we should get back to the Collector to verify the current state.

cc: @open-telemetry/semconv-k8s-approvers

Describe alternatives you've considered

No response

Additional context

Below there are some metrics from namespaces other than k8s.* as well. I leave them in there intentionally in order to take them into account accordingly.

kubeletstats metrics

cpu metrics: #1489
memory metrics: #1490
filesystem metrics: #1488
network metrics: #1487 ✅
uptime metrics: #1486 ✅
volume metrics: #1485

k8scluster metrics

deployment metrics: #1636 ✅

cronjob metrics: #1660 ✅

k8s.cronjob.active_jobs

daemonset metrics: #1649 ✅

k8s.daemonset.current_scheduled_nodes
k8s.daemonset.desired_scheduled_nodes
k8s.daemonset.misscheduled_nodes
k8s.daemonset.ready_nodes

hpa metrics: #1644 ✅

k8s.hpa.max_replicas
k8s.hpa.min_replicas
k8s.hpa.current_replicas
k8s.hpa.desired_replicas

job metrics: #1660 ✅

k8s.job.active_pods
k8s.job.desired_successful_pods
k8s.job.failed_pods
k8s.job.max_parallel_pods
k8s.job.successful_pods

namespace metrics: #1668 ✅

k8s.namespace.phase

replicaset metrics: #1636 ✅

k8s.replicaset.desired
k8s.replicaset.available

replication_controller metrics #1636 ✅

k8s.replication_controller.desired
k8s.replication_controller.available

statefulset metrics: #1637 ✅

k8s.statefulset.desired_pods
k8s.statefulset.ready_pods
k8s.statefulset.current_pods
k8s.statefulset.updated_pods

container metrics #2074 ✅

k8s.container.cpu_request
k8s.container.cpu_limit
k8s.container.memory_request
k8s.container.memory_limit
k8s.container.storage_request
k8s.container.storage_limit
k8s.container.ephemeralstorage_request
k8s.container.ephemeralstorage_limit
k8s.container.restarts
k8s.container.ready

pod metrics #2075

k8s.pod.phase
k8s.pod.status_reason

resource_quota metrics #2076

k8s.resource_quota.hard_limit
k8s.resource_quota.used

node metrics #2077

k8s.node.condition

related issue: open-telemetry/opentelemetry-collector-contrib#33760

Openshift metrics #2078

openshift.clusterquota.limit
openshift.clusterquota.used
openshift.appliedclusterquota.limit
openshift.appliedclusterquota.used

Related issues

TBA

[TylerHelmuth]

I love the idea of moving forward with this work. According to the collector end-user survey k8s and the collector are a big part of our end-user's stack, so moving the related semconvs forwards is a great idea.

[sirianni]

In general, my team has been happy with the metrics collected by kubeletstatsreceiver and how they are modeled. They are struggling significantly with the "state" metrics that come from k8sclusterreceiver. We are coming from a Datadog background.

[ChrsMark]

I have updated the description to group metrics together in a meaningful way.
For kubeletstats the grouping is per resource type (cpu, memory, network etc).
For k8sclusterrecevier metrics the grouping is per K8s Resource type (pod, deployment etc).

I hope this makes the list less overwhelming and people willing to help on this could pick up a group all together and work on it. Maybe we could create standalone issues per group if that helps, link them here to simplify the list in this issue's description and use this issue as a meta issue.

[mx-psi]

I removed this from the system semantic conventions WG since this WG does not handle Kubernetes-related semantic conventions

[dmitryax]

My confusion was mainly because of the k8s.container.restarts being a Resource Attribute in SemConv while also a metric in the Collector. (maybe I generalized wrongly here, apologies for that)

That resource attribute being used to identify a particular container instance in a pod when we scrape log from. Container logs are written in files with the following pattern: <namespace>_<pod_name>_<pod_uid>/<container_name>/<restart_count>.log. The last part, restart_count, identifies a particular container instance. It can be called a container run counter or something like that. This is a valid attribute.