Updated

Linux load average is the number of tasks either running on a CPU, waiting for a free CPU, or blocked in uninterruptible sleep, averaged over the last 1, 5 and 15 minutes. It is not a percentage, it has no ceiling, and the number on its own is meaningless until you know how many logical cores the machine has. Load 8 on an 8 core box is one task per core and perfectly healthy. Load 8 on a 2 core box is four times more work than the machine can run at once.

Key takeaways

  • Load average counts runnable tasks plus tasks in uninterruptible sleep (D state), so Linux load can sit at 40 while CPU utilization reads 5%.
  • Divide load by the logical core count from nproc. Above 1.00 per core, work is queueing.
  • The three figures are exponentially damped moving averages built from a run queue sample taken every 5 seconds, so the 1 minute value still carries older history.
  • High load with low CPU and a large wa figure in top points at disk or NFS, not at the processor.
  • Load average never tells you which resource is saturated. Read it next to iowait, disk queue depth, and a count of D state processes.

What the three numbers actually measure

Run uptime on any Linux host and the three figures land at the end of the line.

uptime
 15:42:11 up 37 days,  4:19,  2 users,  load average: 3.42, 2.91, 2.14

Those three values are the load average, covering roughly the last 1, 5 and 15 minutes. They are not simple means over fixed windows. The kernel samples the run queue every 5 seconds and folds each sample into three exponentially damped moving averages, which is why the 1 minute value still contains a fading contribution from several minutes back. The arithmetic lives in kernel/sched/loadavg.c.

Reading the three in order gives you direction for free. 3.42, 2.91, 2.14 is a machine getting busier. 2.14, 2.91, 3.42 is a machine recovering from a spike that already passed. When all three sit within about 10% of each other, you have a steady state, and that is the shape worth alerting on because it will not resolve on its own.

Plotting the three together makes that direction obvious without any mental arithmetic.

Load average chart for the host prod-web-01 with the 1m, 5m and 15m series plotted together over a one hour window

Why Linux load average is not a CPU queue metric

On most other Unix systems, load average counts only tasks that want CPU. Linux also counts tasks in TASK_UNINTERRUPTIBLE, shown as D in ps output. Those are threads parked inside a syscall waiting on a block device, on a network filesystem such as NFS, or on a kernel lock they cannot be signalled out of.

Brendan Gregg went through the kernel mailing list archives in 2017 and traced the behaviour to a 1993 patch, on the reasoning that a machine wedged on I/O is just as unavailable as a machine wedged on CPU. The consequence is that Linux load average measures demand on the whole system, not demand on the processor.

That single design decision explains the most common confusion in server monitoring. Thirty processes blocked on one hung NFS mount produce a load average of 30 on a box whose CPUs are completely idle.

The only rule that matters: divide by core count

Load average counts tasks, not cores, so the raw value needs a denominator before it means anything.

nproc
8

nproc reports logical cores, hyperthreads included, and that is the right denominator because the scheduler treats every logical CPU as a slot where a task can run. On older systems without nproc, getconf _NPROCESSORS_ONLN returns the same figure.

This one line does the division for you:

awk -v c="$(nproc)" '{printf "1m %.2f  5m %.2f  15m %.2f  per core\n", $1/c, $2/c, $3/c}' /proc/loadavg
1m 0.43  5m 0.36  15m 0.27  per core

Load per core: what each range means

Load per core Reading What I do about it
Below 0.70 Spare capacity on every core Nothing
0.70 to 1.00 Busy, approaching fully committed Open a capacity planning ticket
1.00 to 2.00 Tasks are queueing, tail latency rising Find the saturated resource today
2.00 to 4.00 Sustained oversubscription, response times degrade visibly Page someone if it holds 15 minutes
Above 4.00 Severe, even SSH logins feel slow Treat it as an outage

The 0.70 threshold is a convention, not a kernel constant. It is simply the point where a queue starts forming often enough to be visible in p95 latency.

The same load number on three different machines

Logical cores Load 2 Load 8 Load 32
2 1.00 per core, fully committed 4.00, severe 16.00, emergency
8 0.25, healthy 1.00, fully committed 4.00, severe
32 0.06, idle 0.25, healthy 1.00, fully committed

One number, three different verdicts. This is why a fleet-wide rule such as "alert when load exceeds 10" fires constantly on the small nodes and stays silent while the large ones are drowning. Scale the threshold by core count, or accept that you have built a false positive generator.

High load with low CPU usage means I/O, not CPU

When load climbs and CPU utilization stays low, the D state tasks are doing it. Start with top and look at the wa field on the %Cpu(s) line, which is the percentage of wall clock time CPUs spent idle with at least one outstanding I/O request.

top - 15:42:11 up 37 days,  4:19,  2 users,  load average: 3.42, 2.91, 2.14
Tasks: 214 total,   2 running, 212 sleeping,   0 stopped,   0 zombie
%Cpu(s):  4.2 us,  1.1 sy,  0.0 ni, 21.3 id, 73.1 wa,  0.0 hi,  0.3 si,  0.0 st

4.2% user time against 73.1% iowait is a storage problem wearing a CPU problem's clothes. Press 1 inside top to break the line out per CPU, which matters when a single core is pinned by an interrupt handler.

vmstat gives you the same picture in a form you can log. The r column counts runnable tasks and the b column counts tasks blocked in uninterruptible sleep, so r plus b is roughly the instantaneous value the load average is smoothing.

vmstat 1 5
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 1 14      0 198432  91244 3820116    0    0  1024 41236 2311 4102  4  2 21 73  0

One runnable task and fourteen blocked ones. The load average of 3.42 is almost entirely disk wait.

Name the culprits directly:

ps -eo state,pid,comm | awk '$1 == "D"'
D    9021 rsync
D    9044 rsync
D   14872 postgres

Then confirm at the device level with iostat. On Debian and Ubuntu it comes from sudo apt install sysstat, on RHEL, Rocky and Fedora from sudo dnf install sysstat.

iostat -x 1 3
Device     r/s   rkB/s  r_await    w/s     wkB/s  w_await  aqu-sz  %util
nvme0n1   12.0   384.0     0.42  980.0   41236.0    28.40   27.90   99.60

Columns are trimmed for width here, and exact column names vary between sysstat 11 and 12. %util at 99.60 with aqu-sz near 28 means the device is saturated and requests are stacking up. That is your load average. For a longer explanation of the underlying counters, see disk I/O.

If wa is near zero and the load is still high, look for lock contention in the kernel, a saturated network filesystem, or a driver stuck on a failing device. dmesg -T | tail -50 usually shows the hardware side of that story.

Where to read load average: uptime, /proc/loadavg, top

/proc/loadavg is the raw source, and it carries two fields that uptime throws away.

cat /proc/loadavg
3.42 2.91 2.14 2/487 30219
Field Value Meaning
1 to 3 3.42 2.91 2.14 1, 5 and 15 minute load averages
4 2/487 Currently runnable scheduling entities over total entities
5 30219 PID of the most recently created process

The fourth field is the useful one. A load of 3.42 with 2/487 in that slot tells you only two tasks are actually runnable, so the rest of the load is blocked I/O. That is a diagnosis in a single cat.

For history rather than the current instant, sar -q replays what the sysstat collector recorded. On Debian and Ubuntu, collection is off until you set ENABLED="true" in /etc/default/sysstat.

sar -q
15:40:01     runq-sz  plist-sz   ldavg-1   ldavg-5  ldavg-15   blocked
15:50:01           1       487      3.42      2.91      2.14        14

The blocked column is the D state count over time, which is the fastest way to prove that a past incident was storage and not compute.

What load average will not tell you

It has no unit and no target. It cannot separate CPU pressure from I/O pressure. It is damped, so a 20 second stall barely moves the 1 minute figure. And it is a whole-host number, which trips up anyone reading it inside a container: /proc/loadavg is not namespaced, so a process in a container sees the load of the entire host, including every neighbour on that box.

On kernels 4.20 and newer, pressure stall information gives you the signal load average was reaching for, split by resource and expressed as a percentage of time work was delayed.

cat /proc/pressure/io
some avg10=68.31 avg60=61.02 avg300=44.17 total=1841203119
full avg10=51.44 avg60=47.90 avg300=33.88 total=1402881766

some is the share of time at least one task was stalled on I/O, full is the share where every runnable task was stalled. If the files are missing, your kernel needs CONFIG_PSI and, on some distributions, the psi=1 boot parameter.

Tracking load average across a fleet with Hyperping

Reading load on one box during an incident is the easy part. The hard part is having 1, 5 and 15 minute values already recorded for every host, next to the core count, when someone asks what happened at 03:12.

The Hyperping agent installs in one line and starts reporting on a 30 second cadence:

curl -fsSL https://hyperping.com/install.sh | sh -s HP_INSTALL_xxxxx

It collects the three load averages, the logical core count, per-state CPU time including iowait, memory, filesystem usage per mount point, disk read and write bytes per device, and network bytes per interface, along with the usual host metrics such as kernel version, CPU model and boot time. The full list is in metrics collected. Because the core count arrives with every sample, load per core is a division you never have to do by hand.

Three things I would set up on day one:

  1. A load-per-core threshold rather than a raw load threshold, so the same rule works on a 2 core VM and a 64 core database host.
  2. An iowait alert alongside it, so a load spike arrives already labelled as storage or compute.
  3. Agent liveness alerting. When metrics stop, Hyperping marks the host Stale and then Offline, opens an outage and runs your escalation policy, which catches the case where the machine got so loaded that the agent itself stopped reporting.

Agent-based server monitoring covers the inside of the host. Pair it with external uptime checks so you also know whether users could reach the service while the load was climbing. If you are running containerized workloads, the Kubernetes monitoring guide covers the cluster layer, and the roundup of server performance monitoring tools compares the agents themselves.

The short version

Never read a load average without nproc next to it. Divide, and 0.70 per core is where you start paying attention, 1.00 is fully committed, above 2.00 you are shipping latency to users. Then check the wa figure and the fourth field of /proc/loadavg before you assume you need more CPU, because most of the alarming load averages I have chased turned out to be a disk, an NFS server, or a database flushing harder than the storage could take.