Painting every kernel stack at allocation and scanning it on each trap puts the
high-water mark at **3736 of 4096 bytes** -- 91%, in netd, inside lwIP's input
path. 360 bytes of headroom on a kernel that takes interrupts on the current
stack is not headroom: a timer landing at that depth pushes a trapframe and
runs trap() straight off the bottom of the page and into whatever kalloc handed
out next, which is very often another process's kernel stack.
The canary at kstack[0] did not catch this and could not have. It is checked on
trap entry and exit, and a frame allocated in one `sub $N, %rsp` steps clean
over the bottom eight bytes and writes below them without ever touching it. It
catches a slow walk past the end, not one deep call.
8192 now, and the stacks are static -- one per ptable slot, NPROC * KSTACKSIZE
= 1 MB of BSS. Static rather than a bigger kalloc because kalloc() hands out
exactly one page and there is no contiguous multi-page allocator to ask. A
process's slot is fixed for its lifetime, so the stack belonging to that slot is
the natural answer: no allocation to fail, no free to forget, and the kalloc
failure path in allocproc() goes away with it. The page was being paid for out
of kalloc before, one at a time, and undercounted.
**And the AP scheduler stack was already wrong.** startothers() did
`stack = kalloc()` for one page and then handed the AP `stack + KSTACKSIZE` as
its stack top. Exactly right for as long as KSTACKSIZE happened to equal
PGSIZE, and a whole page past the allocation the moment it did not -- which
this commit would have made true. That is not a throwaway boot stack either:
the AP runs mpmain() and then scheduler() on it for the rest of the machine's
life, and takes interrupts on it whenever no process is current. Static too,
NCPU of them.
**This does not fix the intermittent zsh crash**, and the measurement above was
taken while looking for it. It is worth saying plainly: 1-3 shells per boot
still die, so the hypothesis was wrong and the stack was a separate latent bug
that the search turned up.
What the search did establish is worth more than the guess was:
- **It is SMP-only.** Three boots at -smp 1: zero faults. Three at -smp 2:
one, three, one.
- **It is not only zsh.** schinitd took one too.
- **The faults name their cause if you print enough.** `rip 0x0` is a process
handed a zero and following it, not a process that branched anywhere. And
schinitd's `addr 0xffffffffffffffea` is -22, which is -EINVAL: a call
returned an error and the caller used it as a pointer.
So trap.c dumps registers, segments, and the top of the user stack for a fault
at rip below a page or a faulting address near zero, the way it already did for
a GP fault. Neither of the two facts above was visible from the one-line
report, and both came out of the first dump that included them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
d58b1cda17