When designing Kubernetes manifests, it is common to encounter a mixed configuration: a single Pod containing one container with strict CPU/memory limits and requests, alongside another container (like a sidecar or a utility script) with no resource constraints defined.
While this configuration is valid syntax, it creates a disconnect between how Kubernetes classifies the Pod and how the underlying Linux kernel manages the hardware.
Here is a breakdown of how Kubernetes and Linux handle this configuration across three operational scenarios.
Requests, Limits, and the OOM Reality
To understand why a mixed-resource Pod behaves unpredictably, we must first establish how Kubernetes handles Requests and Limits, and why applications still experience Out-Of-Memory (OOM) failures despite internal memory tuning.
Resource Allocation Spectrum
When configuring container resources, you are configuring two distinct guardrails:
Requests: The minimum allocation a container requires. The Kubernetes scheduler uses this number to find a worker node with sufficient unallocated capacity.
Limits: The absolute maximum allocation a container is allowed to consume on the host.
The underlying Linux kernel handles breaches of these ceilings differently depending on the resource type:
CPU: Exceeding a limit results in throttling. The kernel slows down execution threads via Completely Fair Scheduler (CFS) quotas, but the application remains running.
Memory: Memory cannot be throttled. Exceeding a limit results in immediateOOM terminationby the kernel.
The "RAM-Aware" Mismatch
A common misconception is that sophisticated applications—such as a database with strict buffer pools or a JVM configured with an explicit max heap (-Xmx)—are safe from OOM kills because they manage their own memory.
In production, a visibility gap exists between the runtime and the host operating system:
The Application Tracks: Internal heap utilization and managed caching pools.
The Linux Kernel Tracks: Total Resident Set Size (RSS). This includes the application heap plus native memory overhead, thread stacks, off-heap allocations, and glibc memory fragmentation.
If your runtime configurations are tuned too close to your container limits, this unmanaged native overhead will eventually cross the cgroup threshold and trigger an OOM kill. In a mixed-resource Pod, this risk is compounded: your primary application may be perfectly optimized, but an unconstrained neighbor container can leak memory and exhaust the host - either forcing the Linux kernel to OOM-kill your container processes, or triggering a kubelet eviction of the entire Pod.
Pod QoS vs. Container Cgroups
The root of the tricky behavior lies in a layer mismatch:
Kubernetes evaluates Quality of Service (QoS) at the Pod level. To qualify for the Guaranteed tier, every container in the Pod must have identical requests and limits. If even one container lacks these definitions, the entire Pod drops to the Burstable tier.
Linux manages resources at the container level. The kernel uses control groups (cgroups) to enforce limits on individual processes. It does not natively understand Kubernetes Pod boundaries.
Because of this, the container with explicit limits still gets dedicated cgroup constraints, while the unconstrained container gets the host defaults.
Scenario 1: Idle Node Conditions
When the worker node has excess, unallocated CPU and memory, both containers operate without immediate friction.
The limited container is restricted by its cpu.max (CFS quota) settings and cannot exceed its defined limit, even if the node is entirely idle. The unlimited container faces no such restrictions; it can scale up its consumption to use any spare CPU cycles available on the host.
Scenario 1: Idle Node (Uncapped Surplus)
Behavior: Container B consumes available node resources dynamically. Container A remains strictly capped by its cgroup quota despite available host capacity.
Scenario 2: Node CPU Contention
When the node experiences a high CPU load and threads begin competing for execution time, the Linux kernel relies on cpu.shares to distribute cycles proportionally.
Container A: Kubernetes translates its CPU request into a standard cpu.shares value. During contention, the kernel guarantees Container A its requested share of cycles.
Container B: Because it has no request defined, it receives the minimum possible priority weight (typically 2 shares).
When CPU utilization hits 100%, Linux prioritizes Container A. Container B is heavily throttled and starved of CPU cycles, ensuring the limited container performs as intended.
Scenario 2: CPU Contention (Cgroup Enforcement)
Behavior: CPU shares protect Container A. Container B is allocated only the remaining residual cycles, minimizing its performance impact on the host.
Scenario 3: Memory Exhaustion and OOM Risks
While CPU contention is handled via throttling, memory exhaustion is handled via termination. This is where the mixed-resource configuration introduces operational risk.
If Container B consumes unexpected amounts of memory and pushes the node into an Out-Of-Memory (OOM) state, the kernel must select a process to terminate using the oom_kill_score.
The score is calculated based on two factors: the percentage of memory the process uses, and the Kubernetes-assigned oom_score_adj.
Because the Pod is classified as Burstable, Kubernetes assigns it a higher (less protected) base adjustment value than a Guaranteed Pod. While the OOM killer will initially target Container B because of its higher raw usage, a severe node-level memory crunch can lead to the eviction or disruption of the entire Pod. This means Container A can be terminated as collateral damage due to Container B's resource consumption.
Scenario 3: Node Memory Pressure
Behavior: The kernel targets Container B for termination first. However, because the whole Pod carries a Burstable OOM adjustment score, Container A shares the elevated eviction risk.
Cluster-Level Eviction Priorities
While Scenario 3 illustrates how memory pressure forces the Linux kernel to make an immediate termination decision via the OOM killer, Kubernetes also possesses its own proactive mechanism for handling resource scarcity: Node-Pressure Eviction.
When a node's available memory or disk space drops below defined thresholds, the kubelet does not wait for the Linux kernel to panic. Instead, it actively selects pods to evict to reclaim stability.
If our Franken-Pod avoids an immediate OOM kill, it still faces an elevated risk during a node-pressure eviction cycle because of its Burstable classification. The kubelet evaluates and ranks pods for eviction based on a strict hierarchy.
The Eviction Tier List
The kubelet groups pods into two primary risk categories based on their QoS class and current consumption:
Tier 1: Over-Consumers
BestEffort Pods: Because these pods have no requests defined (Request = 0), any resource consumption is technically considered "exceeding requests." They are always the first to be evicted.
Burstable Pods (Over-consuming): If a Burstable pod is currently utilizing more memory than its defined request block, it is grouped into this high-risk tier.
Tier 2: Within-Request Pods
Burstable Pods (Under-consuming): If a Burstable pod's usage is below or equal to its requested allocation, it is protected during the initial eviction passes.
Guaranteed Pods: Because Guaranteed pods always have matching requests and limits, they naturally fall into this protected tier.
Within these tiers, the kubelet breaks ties using the pod's explicit PriorityClass. If priorities are identical, the pod consuming the highest percentage of resources relative to its request is evicted first.
Because our mixed-resource Franken-Pod is classified as Burstable, any spike in Container B’s usage shifts the entire Pod into Tier 1, making it a primary target for eviction.
Kubelet Eviction Queue Hierarchy
Behavior: When resource thresholds are breached, the kubelet processes Tier 1 sequentially before ever touching Tier 2. A single unconstrained container pushes the whole Pod's queue position to the left.
Summary
Configuring a Pod with asymmetric resource definitions creates a hidden architectural trade-off:
CPU management remains robust because the Linux kernel enforces container-level boundaries via cpu.shares. The limited container will get its performance slice during a crunch.
Memory management and eviction risk are degraded to the lowest common denominator. Because Kubernetes applies QoS tags and oom_score_adj values at the Pod level, the stability of your bounded, critical container is directly tied to the behavior of its unbounded roommate.
To protect critical workloads from premature eviction or OOM termination, resource constraints must be treated as a binary configuration: either every container within the Pod boundary is strictly declared, or the components must be separated into independent deployments.