SchOS/docs/history
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Nexoniarz 26e5d89c73 smp: every stack on every AP was misaligned by eight
`jmp mpenter`.

x86-64 requires rsp to be 16-byte aligned immediately before a `call`, so a
function starts with rsp ending in 8 -- the return address -- and its
`push %rbp; mov %rsp,%rbp` prologue leaves %rbp 16-byte aligned. Compilers
build on that: an aligned vector store to a frame slot is `movaps`, and
`movaps` to a misaligned address is #GP.

entry64mp loaded the AP's stack pointer, which is page-aligned, and *jumped*.
Nothing pushed a return address, so %rbp came out eight bytes off -- on every
frame, on every application processor, for the life of the machine.

It cost nothing for as long as no function an AP ran needed an aligned vector
store. Then the console's glyph path started returning a small struct by value
(the cell-as-a-surface change), clang spilled it with movaps, and CPU 1 took a
#GP the instant it announced itself on a verbose boot:

    unexpected trap 13 from cpu 1 rip ffff800000120a3b <draw_glyph_px+0x1cb>

    120a37: 0f 10 45 90    movups -0x70(%rbp), %xmm0
    120a3b: 0f 29 45 b0    movaps %xmm0, -0x50(%rbp)   <- #GP

movups on the line above loads fine; movaps stores and faults. The
displacement is a multiple of 16, so what is misaligned is %rbp itself, which
is a fact about how the function was entered.

`call mpenter`, not `jmp`. Eight bytes for an ABI the whole toolchain assumes.

**It was hiding behind a second bug.** The first report was not a #GP at all:

    Stop code     ACQUIRE
          #2  cprintf+0xae
          #3  trap+0x2c4

trap() reported the fault with cprintf(), cprintf() takes the console lock,
and the faulting code was already holding it -- on a verbose boot every
character is drawn under that lock. acquire() panicked about a lock its own
call chain owned, and the trap number, the faulting address and the registers
were all lost. A kernel-mode trap ends in panic() a dozen lines later anyway,
so the console goes into panic mode *before* the report rather than during it.

**And a third, in the same boot.** The console reported "JetBrains Mono, 8x16
cells" -- a name from one font and metrics from another. The built-in bitmap
became a provider like any other in the previous commit, so draw_glyph_px()
installs it lazily on the first glyph; on a verbose boot that happens before
gfx_font_init() runs, which then returned early because a font was already
installed. The console spent the whole boot in the fallback while claiming not
to be. "A font is installed" and "a typeface was chosen" had been the same test
and quietly stopped being; they are two flags now.

Verified: verbose boot reaches a shell twice running with no traps and no
panics, and reports 10x21 cells -- the real typeface -- where it reported 8x16
before. Both supported configurations quiet: one boot each, no traps, no
panics, socktest and memfdtest green.

Written up as section 22 of docs/history/kernel.md, because the interesting
part is not the missing eight bytes but that three separate faults had to be
peeled off in order before the first one could be seen.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-23 19:37:35 +02:00
..
kernel.md smp: every stack on every AP was misaligned by eight 2026-09-23 19:37:35 +02:00
moving-to-linux.md docs: four directories, and a rule for when a thing leaves the roadmap 2026-09-12 17:13:35 +02:00
README.md docs: four directories, and a rule for when a thing leaves the roadmap 2026-09-12 17:13:35 +02:00
schfs.md docs: four directories, and a rule for when a thing leaves the roadmap 2026-09-12 17:13:35 +02:00

History

Why SchOS is the way it is. Bugs whose cause was nowhere near their symptom, and the few write-ups that have nowhere better to live.

kernel.md The expensive ones. Boot, SMP, signals, ABI assumptions, /proc, input.
schfs.md How the filesystem was built, and what each phase cost.
moving-to-linux.md The Windows-to-Fedora move.

What belongs in here

A bug whose cause is not visible from the code that fixed it.

That is the whole test. If reading the fix explains the problem, the fix is the documentation and nothing needs writing down. If the fix looks arbitrary — a constant moved by eight, a field reordered, a check inverted — then the strangeness is load-bearing and the next person to touch it deserves to know why before they change it back.

Every entry here is a thing that looked like a different problem than it was.

What does not belong

  • A changelog. git log is the changelog.
  • A bug that was found and fixed in an hour. Cost is the filter.
  • How something works. That is ../architecture/, written in the present tense for somebody who does not care what it used to be.
  • Anything still open. An unfinished investigation goes in ../plan/ or in the code as a comment. This directory is past tense.

How to write one

Symptom first, then diagnosis, then the fix, then the transferable part.

The transferable part is the reason the entry exists. __thread being wrong for per-CPU data is worth more than the specific null dereference it caused; "a layout constant that several artifacts must agree on wants a check that stops the build, not a comment asking people to remember" is worth more than the bootloader offset that taught it.

Say how it was found, when the technique was the hard part. A hardware watchpoint through QEMU's gdb stub named an instruction on the first run after three rounds of cprintf had narrowed it to nothing — that is reusable, and the bug is not.

Do not clean up the embarrassing part. The near-miss where a change passed every assertion green without having been applied at all is one of the most useful entries in this directory, and it only works because it admits that a grep returning three hits was read as confirmation without counting what they were.