Observability · Profiling

Diagnose CPU, memory, and deadlocks with Go pprof

Cosmo Router exposes the full set of Go pprof endpoints: heap, CPU, goroutine, block, and thread. Enable with one environment variable. Visualize with the tooling you already know.

Off by default. No overhead until enabled. Standard Go tooling.

PPROF_ADDR=:6060

Cosmo Router:6060/debug/pprofProfile types/heap/profile (CPU · 30s)/goroutine/threadcreate/blockgo tool pprofVisualize

Available onFreeProEnterprise

The problem

Symptom-level metrics do not fix performance

Dashboards tell you something is wrong. A profile tells you what is wrong.

Metrics show symptoms, not causes

CPU at 80% says you have a problem. It does not say which function. Without a profile, optimization becomes a search problem.

Memory leaks hide in production

Heap usage grows over days. Restart restores normal, for a while. Without a profile, the leak survives the restart and so does the on-call.

Deadlocks reveal themselves slowly

The router stops responding but does not crash. Logs are clean. The behavior is obvious in a goroutine dump and invisible without one.

Our solution

Native Go pprof, one environment variable away

Cosmo Router integrates Go's standard pprof package. Set PPROF_ADDR and the full set of profiling endpoints becomes available: heap, CPU, goroutine, threadcreate, and block.

From symptom to root cause

  1. Set PPROF_ADDR=:6060 to start the pprof HTTP server inside the router.

  2. Capture profiles via curl: /debug/pprof/heap for memory, /debug/pprof/profile for CPU, /debug/pprof/goroutine for goroutines.

  3. /debug/pprof/threadcreate and /debug/pprof/block cover thread creation and blocking operations.

  4. CPU profiles run for 30 seconds by default; the duration is configurable via the seconds query parameter.

  5. Run go tool pprof on the captured profile for flame graphs, top tables, and call trees in your browser.

  6. When the endpoint is unset, there is no overhead; pprof is off entirely.

No instrumentation library. No SDK. Standard Go.

Profiling

Before & After

Before CosmoWith Cosmo
Metrics show memory growth but not which code path allocatesHeap profiles pinpoint hot allocation sites in the router
Production memory leaks discovered only after OOMPPROF_ADDR enables targeted captures without redeploying subgraphs
Blocked goroutines surface slowly through logs alonegoroutine and block profiles show contention and stalls
Always-on profiling tax on every requestpprof is off until PPROF_ADDR is set: capture on demand only

Operational safety

Off by default

The pprof endpoint is disabled until PPROF_ADDR is set. Enable only when you need it. Bind to a private interface and never expose the pprof port to the public internet.

How profiling works in Cosmo Router

01
One env var. Off by default.

Enable

Set PPROF_ADDR to the address you want the pprof HTTP server on (for example :6060). The endpoint is off by default; you only pay for it when you turn it on.

02
curl one endpoint per profile type.

Capture

Pull a heap, CPU, goroutine, threadcreate, or block profile with curl. CPU profiles capture 30 seconds of activity by default.

03
Standard go tool pprof, any viewer.

Visualize

go tool pprof renders the profile as flame graphs, top tables, and source views. Any compatible Go profiling visualizer reads the same output.

04
Hot function, allocation site, blocker.

Diagnose

Find the hot function, the growing allocation site, or the blocked goroutine. Ship the fix with data, not guesswork.

Telemetry controls

Profile types and safety

Heap, CPU, goroutine, block, and thread profiles, on demand with no overhead when disabled.

CPU profile

Pull a 30-second CPU profile during peak load. Use the seconds query parameter to set a different duration.

Heap profile

Capture heap snapshots at intervals. go tool pprof diff mode shows what is growing between samples.

Goroutine dump

Use /debug/pprof/goroutine?debug=2 for stack traces of every goroutine. The view that finds deadlocks and lock contention.

Visualization

go tool pprof -http 127.0.0.1:8079 profile.out starts a local web UI with flame graphs, top tables, and source views.

Security note

The pprof endpoint is disabled by default. When you enable it, bind to a private interface (for example a Kubernetes internal service) and gate access at the network level. Never expose the pprof port to the public internet.

Profile the router when you need to

Set PPROF_ADDR, capture heap or CPU profiles with curl, and visualize with go tool pprof.

FAQ

Go pprof profiling on Cosmo Router

Deep dive in the profiling documentation.