Container-Aware Resource Management in Go: The Problem, Go 1.25’s Fix, and What’s Still Missing

작성자

카테고리:

← 피드로
DEV Community · Prasad Ekke · 2026-07-22 개발(SW)

If you’ve deployed a Go service to Kubernetes and set careful CPU and memory limits on your pod, there’s a good chance your Go runtime was ignoring them entirely — spawning too many threads, running GC on the wrong schedule, and getting throttled or OOM-killed as a result.

Go 1.25 fixed half of this problem natively. This post explains both halves: what was broken, what’s fixed, what’s still not fixed, and what to do if you can’t upgrade yet.

The problem: Go doesn’t know it’s in a container

When a Go program starts, the runtime reads the host machine’s CPU count and sets GOMAXPROCS — the number of OS threads that can run Go code in parallel — to that number. On a 64-core Kubernetes node, your Go service thinks it has 64 CPUs available, even if your pod’s resource limit is cpu: "2".

The same problem exists for memory. Go’s garbage collector uses a target heap growth ratio (controlled by GOGC) to decide when to trigger collection. By default, it allows the heap to double before collecting. If your container has a 512 MB memory limit and the Go runtime doesn’t know about it, the GC may allow heap growth that pushes your process past the cgroup limit before it triggers a collection. The result: an OOM kill.

These are two distinct problems with different severities.

Problem 1: GOMAXPROCS ignores cgroup CPU limits

What actually happens

Container runtimes use Linux cgroups to enforce CPU limits. A Kubernetes cpu: "2" limit translates into a cgroup CPU bandwidth quota: the container gets 2 CPU-seconds of time per second, spread across all threads.

If GOMAXPROCS is set to 64 (the node’s core count), your Go service creates up to 64 OS threads competing for 2 CPUs worth of time. The kernel enforces the quota by throttling — when the cgroup exhausts its CPU time budget for the current period (typically 100ms), the kernel pauses all threads in the container until the next period begins.

Throttling is brutal. It’s not gentle scheduling slowdown. It’s a hard pause of your entire process. Under this scenario:

  • A p99 latency that should be 10ms can spike to hundreds of milliseconds
  • The Go GC, which runs its own goroutines, can exhaust the CPU budget and cause application threads to stall
  • Context switching overhead multiplies — 64 threads competing for 2 CPUs thrash the scheduler
Node: 64 cores
Pod CPU limit: 2 cores
GOMAXPROCS (before 1.25): 64   ← runtime sees the host
Effective CPU available: 2     ← kernel enforces this

Result: 64 threads, 2 CPUs, aggressive throttling

Enter fullscreen mode Exit fullscreen mode

Real-world benchmarks have shown CPU wait times reaching tens of seconds under this misconfiguration on high-core-count nodes.

The Go 1.25 fix

Go 1.25 makes GOMAXPROCS container-aware by default on Linux. At startup, the runtime reads the cgroup CPU bandwidth limit and sets GOMAXPROCS to the lower of the usual non-container default and the container’s CPU limit. Fractional CPU quotas are rounded up because GOMAXPROCS is an integer.

Node: 64 cores
Pod CPU limit: 2 cores
GOMAXPROCS (Go 1.25): 2   ← runtime reads the cgroup

Enter fullscreen mode Exit fullscreen mode

Two additional behaviors from the official release notes:

Dynamic updates: GOMAXPROCS is now periodically re-evaluated at runtime. If Kubernetes adjusts your pod’s CPU limit on the fly (via VPA or manual edit), the Go runtime will pick up the change automatically — no restart needed.

Opt-out: both behaviors can be disabled via GODEBUG:

# Disable cgroup CPU awareness at startup
GODEBUG=containermaxprocs=0

# Disable periodic GOMAXPROCS updates
GODEBUG=updatemaxprocs=0

Enter fullscreen mode Exit fullscreen mode

Manual override still works: if you set GOMAXPROCS via the environment variable or a runtime.GOMAXPROCS() call, Go 1.25’s automatic behavior is completely disabled. Your explicit value takes precedence.

One edge case worth noting: Go 1.25 respects cpu: limits in Kubernetes but not cpu: requests. The cgroup CPU quota corresponds to the limit, not the request. If you set requests but not limits (a common pattern), Go 1.25’s container awareness has nothing to read and defaults to the host core count.

resources:
  requests:
    cpu: "0.5"    # Go 1.25 ignores this
  limits:
    cpu: "2"      # Go 1.25 reads this ✅

Enter fullscreen mode Exit fullscreen mode

Problem 2: GOMEMLIMIT still doesn’t read cgroup memory limits (not fixed in 1.25)

This is the important accuracy note. Go 1.25 only fixed CPU (GOMAXPROCS). Memory awareness via GOMEMLIMIT is a separate proposal (golang/go issue #75164) that has not yet shipped. As of Go 1.25, GOMEMLIMIT still defaults to “unlimited” — the Go runtime has no built-in mechanism to read the container’s memory limit from cgroups and apply it automatically.

GOMEMLIMIT (introduced in Go 1.19) is a soft memory limit for the Go runtime. When heap usage approaches the limit, the GC becomes more aggressive to stay within it. Without it set to your container’s memory limit, the GC may allow heap growth that triggers the kernel OOM killer before the GC intervenes.

Container memory limit: 512 MB
GOMEMLIMIT default: math.MaxInt64 (unlimited)

Go heap target (GOGC=100): doubles before collecting
If live heap grows past 512 MB: OOM kill before GC triggers

With GOMEMLIMIT=450MB:
GC triggers more aggressively as heap approaches 450 MB
OOM kills become rare

Enter fullscreen mode Exit fullscreen mode

Workaround 1: Set GOMEMLIMIT as an environment variable

The simplest approach. Set it in your Kubernetes deployment to slightly below your memory limit — leaving headroom for non-heap memory (stack space, runtime overhead, cgo memory):

# Kubernetes pod spec
env:
  - name: GOMEMLIMIT
    value: "450MiB"   # 512MB limit minus ~12% headroom
resources:
  limits:
    memory: "512Mi"

Enter fullscreen mode Exit fullscreen mode

Or use the Kubernetes Downward API to derive it from the actual limit:

env:
  - name: GOMEMLIMIT
    valueFrom:
      resourceFieldRef:
        containerName: my-service
        resource: limits.memory
        divisor: "1"   # bytes

Enter fullscreen mode Exit fullscreen mode

Caveat: the Downward API injects the raw memory limit, giving the runtime 0% headroom. For most services this is fine — GOMEMLIMIT is a soft limit; the runtime won’t OOM itself trying to hit it. But memory-mapped files, cgo allocations, and runtime overhead sit outside the Go heap and don’t count against GOMEMLIMIT. Leave 10–15% headroom if your service uses any of these.

Workaround 2: Set GOMEMLIMIT in code at startup

import "runtime/debug"

func main() {
    // Set to 90% of container memory limit
    // Replace with your actual limit or read from env
    const memLimitBytes = 450 * 1024 * 1024 // 450 MB
    debug.SetMemoryLimit(memLimitBytes)

    // ... rest of main
}

Enter fullscreen mode Exit fullscreen mode

This works but hardcodes the value, which is brittle across environments.

Third-party libraries: pre-1.25 CPU fix and memory fix

If you can’t upgrade to Go 1.25, or if you need memory awareness that 1.25 doesn’t provide, two libraries solve these problems cleanly.

uber-go/automaxprocs — CPU (pre-1.25)

The library that Go 1.25 made redundant for most users. A single blank import sets GOMAXPROCS to the container’s CPU limit at startup by reading cgroup v1 and v2 files.

import _ "go.uber.org/automaxprocs"

Enter fullscreen mode Exit fullscreen mode

That’s the entire integration. The init() function runs at startup, reads the cgroup CPU quota, calculates the effective CPU count, and calls runtime.GOMAXPROCS(). If no cgroup limit is found, it leaves GOMAXPROCS at the default.

When to still use it: if you’re on Go 1.24 or earlier. Go 1.25 reads both cgroup v1 and v2, so on Go 1.25+ you’re covered natively regardless of cgroup version.

Migration path: upgrade to Go 1.25 and remove the import. Several major projects have done exactly this.

KimMachineGun/automemlimit — Memory (still relevant on all versions)

Since GOMEMLIMIT container awareness isn’t in Go 1.25, this library fills the gap. It reads the cgroup memory limit and sets GOMEMLIMIT automatically at startup, with configurable headroom.

import _ "github.com/KimMachineGun/automemlimit"

Enter fullscreen mode Exit fullscreen mode

Default behavior: sets GOMEMLIMIT to 90% of the cgroup memory limit, leaving 10% headroom for non-heap allocations. The ratio is configurable:

import "github.com/KimMachineGun/automemlimit/memlimit"

func main() {
    // Set to 85% of container memory limit
    memlimit.SetGoMemLimitWithOpts(
        memlimit.WithRatio(0.85),
        memlimit.WithProvider(memlimit.FromCgroup),
    )
    // ...
}

Enter fullscreen mode Exit fullscreen mode

Supports cgroup v1 and v2. Falls back gracefully if no cgroup limit is found (leaves GOMEMLIMIT unchanged).

tprasadtp/go-autotune — Both CPU and memory, cgroup v2 + Windows

A newer library that handles both GOMAXPROCS and GOMEMLIMIT in a single import, with support for Windows via the Job Objects API in addition to Linux cgroups:

import _ "github.com/tprasadtp/go-autotune"

Enter fullscreen mode Exit fullscreen mode

Constraint: cgroup v2 only on Linux. If you’re on older infrastructure still running cgroup v1 (pre-Kubernetes 1.25, RHEL 8 and below), use automaxprocs + automemlimit instead.

What to use and when

Go 1.25+, Linux, Kubernetes 1.25+ — CPU: built-in. Memory: use automemlimit or set GOMEMLIMIT manually.

Go 1.24 or earlier — CPU: automaxprocs. Memory: automemlimit or the GOMEMLIMIT env var.

Go 1.25+, cgroup v2, need memory automation — CPU: built-in. Memory: automemlimit or go-autotune.

Go 1.25+, want explicit control — CPU: GODEBUG=containermaxprocs=0 plus a manual value. Memory: GOMEMLIMIT env var.

Non-Linux (Windows containers) — CPU: go-autotune. Memory: go-autotune.

Practical Kubernetes setup for a Go 1.25 service

apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      containers:
        - name: my-service
          image: my-service:latest
          resources:
            requests:
              cpu: "500m"
              memory: "256Mi"
            limits:
              cpu: "2"       # Go 1.25 reads this for GOMAXPROCS
              memory: "512Mi"
          env:
            # GOMAXPROCS: handled automatically by Go 1.25
            # GOMEMLIMIT: still needs to leave headroom for non-heap memory
            - name: GOMEMLIMIT
              value: "450MiB"  # 512Mi limit minus roughly 12% headroom

Enter fullscreen mode Exit fullscreen mode

Alternatively, in main.go, use automemlimit instead of setting GOMEMLIMIT manually:

package main

import (
    _ "github.com/KimMachineGun/automemlimit" // sets GOMEMLIMIT to 90% of cgroup limit
)

func main() {
    // GOMAXPROCS handled by Go 1.25 runtime automatically.
    // GOMEMLIMIT handled by automemlimit with its default 10% headroom.
    // ...
}

Enter fullscreen mode Exit fullscreen mode

Summary

The container resource problem in Go has two parts: CPU throttling from a mismatched GOMAXPROCS, and OOM kills from a GC that doesn’t know its memory budget. Go 1.25 solves the CPU side natively and dynamically. The memory side remains a manual step — either via the GOMEMLIMIT environment variable, the Kubernetes Downward API, or automemlimit.

If you’re running Go 1.25 on Kubernetes with CPU limits set, you can remove automaxprocs and get the same behavior for free. For memory, automemlimit remains the cleanest solution until the Go team ships the cgroup memory proposal.

This post is part of a series on Go in production. Previous posts cover goroutine leaks, context propagation, and profiling with pprof.
Next: Async Rust vs Go goroutines — the mental model shift.

원문에서 계속 ↗

코멘트

답글 남기기

이메일 주소는 공개되지 않습니다. 필수 필드는 *로 표시됩니다