container_spec_cpu_quota not exported for main container when pod uses shareProcessNamespace: true

[einhorneyal]

Summary

When a Kubernetes pod is configured with shareProcessNamespace: true,
container_spec_cpu_quota is silently not exported for the main application
container in Prometheus. Only sidecar containers that are not the PID namespace
owner retain the metric. This causes all CPU utilization percentage calculations
that rely on this metric to be either completely absent or severely inflated (~60×).

---

Environment

• Kubernetes: GKE, control plane v1.34.8-gke.1000000, nodes v1.34.6-gke.1307000 (confirmed); likely any Kubernetes cluster using containerd
• Container runtime: containerd v2.1.5
• Node OS: Container-Optimized OS from Google, kernel 6.12.68+
• cAdvisor: bundled with kubelet on nodes — confirmed affected across multiple GKE minor versions up to v0.57.0 (latest at time of writing)
• cgroup version: v2 (COS default on GKE since kernel 5.x)

---

Minimal Reproduction

Deploy a pod with shareProcessNamespace: true and a sidecar:

apiVersion: v1
kind: Pod
metadata:
  name: test-shared-pid
spec:
  shareProcessNamespace: true
  containers:
    - name: main-app
      image: nginx
      resources:
        requests:
          cpu: "1"
        limits:
          cpu: "1"
    - name: sidecar
      image: busybox
      command: ["sleep", "infinity"]
      resources:
        requests:
          cpu: 100m
        limits:
          cpu: 100m

Then query Prometheus:

container_spec_cpu_quota{pod="test-shared-pid"}

---

Expected Behavior

Two series present — one per container:

container_spec_cpu_quota{container="main-app", pod="test-shared-pid"} = 100000
container_spec_cpu_quota{container="sidecar",  pod="test-shared-pid"} = 10000

---

Actual Behavior

Only the sidecar's quota is exported:

container_spec_cpu_quota{container="sidecar", pod="test-shared-pid"} = 10000

container_spec_cpu_quota for main-app is completely absent. The main container
has a CPU limit set and the limit IS enforced by the kernel — the cgroup
cpu.cfs_quota_us file contains the correct value — but cAdvisor does not export it.

---

What Is and Isn't Affected

Metric	Affected?
container_spec_cpu_quota	✅ Yes — absent for main container
container_spec_cpu_period	✅ Yes — absent for main container
container_cpu_usage_seconds_total	❌ No — correctly exported for all containers
container_memory_working_set_bytes	❌ No — correctly exported for all containers
container_spec_memory_limit_bytes	❌ No — correctly exported for all containers

Only OCI-spec-derived metrics are affected. cAdvisor reads CPU usage from
cgroup stat counters (unaffected), but reads CPU quota via a path that is
disrupted by shareProcessNamespace.

Removing shareProcessNamespace: true from the pod spec immediately restores
container_spec_cpu_quota for the main container.

---

Impact

Adding a sidecar that requires shareProcessNamespace: true to read /proc entries
of sibling containers causes a widely-used PromQL CPU utilization pattern to
produce severely inflated values (~60×):

sum(rate(container_cpu_usage_seconds_total{pod=~"$pod"}[5m])) by (pod)
/
sum(container_spec_cpu_quota{pod=~"$pod"} / container_spec_cpu_period{pod=~"$pod"}) by (pod)

Before (no sidecar, no shareProcessNamespace):

• denominator = 600,000 µs = 6 cores ✅

After (sidecar added, shareProcessNamespace: true):

• container_spec_cpu_quota{container="main-app"} → absent
• container_spec_cpu_quota{container="sidecar"} → 10,000 µs = 100m
• denominator collapses to 10,000 µs = 0.1 cores → CPU % inflated ~60×

Alert rules that include an explicit container= label filter in the denominator
go silently dark (no data → no alert, even at genuine high CPU), which is
arguably more dangerous than the inflation.

---

Root Cause Analysis

How container_spec_cpu_quota is built

The metric is populated in container/common/helpers.go::getSpecInternal() by
reading directly from the cgroup filesystem:

// cgroup v1
quota := readString(cpuRoot, "cpu.cfs_quota_us")
if quota != "" && quota != "-1" {
    val, err := strconv.ParseUint(quota, 10, 64)
    if err == nil {
        spec.Cpu.Quota = val
    }
}

// cgroup v2
max := readString(cpuRoot, "cpu.max")
if splits[0] != "max" {
    spec.Cpu.Quota = parseUint64String(splits[0])
}

Quota stays 0 when the cgroup reports -1 (unlimited). The containerd handler
in container/containerd/handler.go::GetSpec() then calls:

spec, err := common.GetSpec(h.cgroupPaths, h.machineInfoFactory, hasNet, hasFilesystem)

And metrics/prometheus.go only exports the metric when Quota != 0:

if cont.Spec.Cpu.Quota != 0 {
    ch <- prometheus.MustNewConstMetric("container_spec_cpu_quota", ...)
}

There is no explicit suppression — the metric simply never gets set because
the wrong cgroup is being read, which reports -1.

Why the wrong cgroup is read

When shareProcessNamespace: true is set, containerd designates the
infra/pause container as the PID namespace owner. The main application container
is created with a shared PID namespace reference in its OCI spec:

"linux": {
  "namespaces": [
    { "type": "pid", "path": "/proc/<pause-pid>/ns/pid" }
  ],
  "cgroupsPath": "/kubepods/guaranteed/pod<uid>/<main-container-id>"
}

The cgroupsPath field correctly points to the container's own cgroup. However,
cAdvisor's cgroupPaths initialization in newContainerdContainerHandler() — when
it encounters the shared PID namespace entry — appears to resolve the container's
cgroup to the pause/infra container's cgroup rather than the container's own.

The pause container has no CPU limit → its cgroup reports cpu.cfs_quota_us = -1
→ getSpecInternal() skips setting Quota → Quota stays 0 → metric not
exported.

The sidecar is also in the shared PID namespace but is not the namespace owner,
so it is processed differently and its own cgroup (with the correct limit) is read.

---

Suggested Fix

Where to look

container/containerd/handler.go — newContainerdContainerHandler()

Inspect how h.cgroupPaths is populated. Verify the cgroup path is always derived
from the container's own spec.Linux.CgroupsPath OCI field, and never
influenced by the PID namespace owner reference at
spec.Linux.Namespaces[type=pid].path.

Proposed fix direction

Ensure cgroupPaths is always derived from the container's own linux.cgroupsPath
OCI spec field, regardless of namespace sharing:

// Always use the container's own cgroup — do NOT follow the PID namespace
// owner reference to resolve the cgroup path
if spec.Linux != nil && spec.Linux.CgroupsPath != "" {
    cgroupPath = spec.Linux.CgroupsPath
}

The fix cannot be in getSpecInternal() — it only receives the resolved cgroup path
and has no knowledge of namespace topology. The fix must be upstream at cgroup path
resolution time in the containerd handler.

Verification

After the fix, querying with shareProcessNamespace: true should yield both series:

container_spec_cpu_quota{container="main-app"} = <limit × 100000>
container_spec_cpu_quota{container="sidecar"}  = <sidecar-limit × 100000>

---

Workaround

Replace container_spec_cpu_quota / container_spec_cpu_period with
kube_pod_container_resource_limits{resource="cpu"} from kube-state-metrics,
which reads from the Kubernetes API and is completely unaffected by this bug:

# Before (broken when shareProcessNamespace: true)
rate(container_cpu_usage_seconds_total{container="main-app"}[5m])
/ (container_spec_cpu_quota{container="main-app"} / container_spec_cpu_period{container="main-app"})

# After (correct regardless of namespace configuration)
rate(container_cpu_usage_seconds_total{container="main-app"}[5m])
/ kube_pod_container_resource_limits{container="main-app", resource="cpu"}

---

Additional Notes

• The bug is 100% reproducible by adding/removing shareProcessNamespace: true
  with no other changes
• container_cpu_usage_seconds_total is not affected — cAdvisor reads CPU
  usage from cgroup stat counters, not OCI spec paths; only container_spec_*
  metrics are impacted
• No fix found in any cAdvisor release up to v0.57.0
• Related but distinct prior issues: container_spec_cpu_quota is missing #3154, Unable to obtain "container_spec_cpu_quota" metrics #3601, container_spec_cpu_quota not showing up #2220 (missing quota metrics,
  not in the context of shared PID namespaces)

[danielrozenblum]

We're seeing a related variant without shareProcessNamespace. Pods with Guaranteed QoS (cgroup under kubepods-guaranteed.slice) also lose container_spec_cpu_quota on containerd v2.1.5 + cgroup v2. Burstable pods in the same namespace report correctly. The fix in container/containerd/handler.go (always derive cgroupPaths from spec.Linux.CgroupsPath) should cover both cases — please verify the fix applies to the Guaranteed QoS path as well.