smp: every stack on every AP was misaligned by eight
`jmp mpenter`.
x86-64 requires rsp to be 16-byte aligned immediately before a `call`, so a
function starts with rsp ending in 8 -- the return address -- and its
`push %rbp; mov %rsp,%rbp` prologue leaves %rbp 16-byte aligned. Compilers
build on that: an aligned vector store to a frame slot is `movaps`, and
`movaps` to a misaligned address is #GP.
entry64mp loaded the AP's stack pointer, which is page-aligned, and *jumped*.
Nothing pushed a return address, so %rbp came out eight bytes off -- on every
frame, on every application processor, for the life of the machine.
It cost nothing for as long as no function an AP ran needed an aligned vector
store. Then the console's glyph path started returning a small struct by value
(the cell-as-a-surface change), clang spilled it with movaps, and CPU 1 took a
#GP the instant it announced itself on a verbose boot:
unexpected trap 13 from cpu 1 rip ffff800000120a3b <draw_glyph_px+0x1cb>
120a37: 0f 10 45 90 movups -0x70(%rbp), %xmm0
120a3b: 0f 29 45 b0 movaps %xmm0, -0x50(%rbp) <- #GP
movups on the line above loads fine; movaps stores and faults. The
displacement is a multiple of 16, so what is misaligned is %rbp itself, which
is a fact about how the function was entered.
`call mpenter`, not `jmp`. Eight bytes for an ABI the whole toolchain assumes.
**It was hiding behind a second bug.** The first report was not a #GP at all:
Stop code ACQUIRE
#2 cprintf+0xae
#3 trap+0x2c4
trap() reported the fault with cprintf(), cprintf() takes the console lock,
and the faulting code was already holding it -- on a verbose boot every
character is drawn under that lock. acquire() panicked about a lock its own
call chain owned, and the trap number, the faulting address and the registers
were all lost. A kernel-mode trap ends in panic() a dozen lines later anyway,
so the console goes into panic mode *before* the report rather than during it.
**And a third, in the same boot.** The console reported "JetBrains Mono, 8x16
cells" -- a name from one font and metrics from another. The built-in bitmap
became a provider like any other in the previous commit, so draw_glyph_px()
installs it lazily on the first glyph; on a verbose boot that happens before
gfx_font_init() runs, which then returned early because a font was already
installed. The console spent the whole boot in the fallback while claiming not
to be. "A font is installed" and "a typeface was chosen" had been the same test
and quietly stopped being; they are two flags now.
Verified: verbose boot reaches a shell twice running with no traps and no
panics, and reports 10x21 cells -- the real typeface -- where it reported 8x16
before. Both supported configurations quiet: one boot each, no traps, no
panics, socktest and memfdtest green.
Written up as section 22 of docs/history/kernel.md, because the interesting
part is not the missing eight bytes but that three separate faults had to be
peeled off in order before the first one could be seen.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
35ea398334
commit
26e5d89c73
4 changed files with 120 additions and 3 deletions
|
|
@ -1125,3 +1125,73 @@ and startup is 0.14 s. mksh keeps `/bin/sh`, so schinit's no-jobs fallback is a
|
|||
different program from the one that might have failed.
|
||||
|
||||
Completion is the piece that waits on the filesystem.
|
||||
|
||||
## 22. Every stack on every application processor was misaligned by eight
|
||||
|
||||
`jmp mpenter`.
|
||||
|
||||
x86-64 requires `rsp` to be 16-byte aligned immediately before a `call`, so a
|
||||
function begins with `rsp` ending in 8 — the return address — and its
|
||||
`push %rbp; mov %rsp,%rbp` prologue leaves `%rbp` 16-byte aligned. Compilers
|
||||
build on that: an aligned vector store to a frame slot is `movaps`, and
|
||||
`movaps` to a misaligned address is `#GP`.
|
||||
|
||||
`entry64mp` loaded the AP's stack pointer, which is page-aligned, and *jumped*.
|
||||
No return address was pushed, so `%rbp` came out eight bytes off — on every
|
||||
frame, on every application processor, for the life of the machine.
|
||||
|
||||
It cost nothing for as long as no function an AP ran needed an aligned vector
|
||||
store. Then the console's glyph path started returning a small struct by value,
|
||||
clang spilled it with `movaps`, and CPU 1 took a `#GP` the instant it announced
|
||||
itself:
|
||||
|
||||
```
|
||||
unexpected trap 13 from cpu 1 rip ffff800000120a3b <draw_glyph_px+0x1cb>
|
||||
```
|
||||
|
||||
```
|
||||
ffff800000120a37: 0f 10 45 90 movups -0x70(%rbp), %xmm0
|
||||
ffff800000120a3b: 0f 29 45 b0 movaps %xmm0, -0x50(%rbp) <- #GP
|
||||
```
|
||||
|
||||
`movups` — *unaligned* — loads fine on the line above; `movaps` stores and
|
||||
faults. The displacement is a multiple of 16, so the misalignment is `%rbp`
|
||||
itself, which is a fact about how the function was entered rather than
|
||||
anything it did.
|
||||
|
||||
`call mpenter` instead of `jmp`. Eight bytes for an ABI the whole toolchain
|
||||
assumes.
|
||||
|
||||
### It hid behind a second bug
|
||||
|
||||
The first report of this was not a `#GP`, it was:
|
||||
|
||||
```
|
||||
Stop code ACQUIRE
|
||||
#2 cprintf+0xae
|
||||
#3 trap+0x2c4
|
||||
```
|
||||
|
||||
`trap()` reported the fault with `cprintf()`, `cprintf()` takes the console
|
||||
lock, and the faulting code was already holding it — every character drawn on
|
||||
a verbose boot is drawn under that lock. So `acquire()` panicked about a lock
|
||||
its own call chain owned, and the trap number, the faulting address and the
|
||||
register state were all lost.
|
||||
|
||||
A kernel-mode trap ends in `panic()` a dozen lines later regardless, so the
|
||||
console is put into panic mode *before* the report rather than during it. The
|
||||
same fault then prints what it actually was, which is the entire purpose of
|
||||
the report.
|
||||
|
||||
### And a third, in the same boot
|
||||
|
||||
The console reported `JetBrains Mono, 8x16 cells` — a name from one font and
|
||||
metrics from another. The built-in bitmap had become a provider like any other,
|
||||
so `draw_glyph_px()` installed it lazily on the first glyph; on a verbose boot
|
||||
that happens before `gfx_font_init()` runs, and `gfx_font_init()` returned early
|
||||
because a font was already installed. The console spent the whole boot in the
|
||||
fallback while claiming not to be.
|
||||
|
||||
"A font is installed" and "a typeface was chosen" had been the same test
|
||||
(`cfont == 0`) and quietly stopped being. They are two flags now, and
|
||||
`gfx_font_ready()` answers the second one.
|
||||
|
|
|
|||
Loading…
Reference in a new issue