boot: the loader jumped into the framebuffer, because rbx was not clobbered

Boot died straight after "exiting boot services" with a KVM internal error,
RIP=0x801a5dc5 and RBX=0x80000000 -- the framebuffer base. The bytes at RIP were
0x0C0E13 and 0xD1D5DC: SCH_BG and a grey from the splash's metal gradient. The
machine was executing the picture of itself starting up.

The hand-off to the kernel writes rbx and did not say so:

    "mov %1, %%rbx\n\t"      <- writes rbx
    "jmp *%2\n\t"
    : : "r"(PML4_ADDR), "r"(fb), "r"(entry)
    : "memory");            <- rbx not declared clobbered

With rbx absent from the clobber list the compiler may allocate an input there,
and clang 21 allocated `entry`:

    movq %rax, %cr3
    movq %rcx, %rbx         <- the framebuffer, into rbx
    jmpq *%rbx              <- %2 was rbx, and now holds the framebuffer

Declaring "rbx" forbids the allocation. Verified in the generated assembly
rather than by reasoning: jmpq *%rbx became jmpq *%r14, and the machine now
boots through to schinit: ready.

This had been latent since the loader was written. It worked only because every
compiler that had built it happened to choose a different register, and it
surfaced the first time the binary was relinked with a different clang -- which
only happened because the previous commit fixed the Makefile dependency that had
been leaving the loader stale. A stale artifact hides the bug the fresh one
reveals.

It is also the third time this project has been broken by a register the code
depends on but never asks for: docs/history/kernel.md section 4 (the BIOS
loader passing the framebuffer in EAX by luck) and section 8 (%fs:0 used as
scratch when the ABI reserves it). Written up as section 7b, with the one-line
recipe that settles all three -- read the generated assembly.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
This commit is contained in:
Nexoniarz 2026-09-12 18:44:24 +02:00
commit 1e52cb591b
2 changed files with 81 additions and 1 deletions

View file

@ -1164,13 +1164,29 @@ EfiMain(EFI_HANDLE image, EFI_SYSTEM_TABLE *systab)
// RBX carries the framebuffer address into kmain, and must never be 1 --
// that value marks an application processor in entry64high. A physical
// MMIO address or zero, so it cannot collide.
//
// "rbx" is in the clobber list because this writes it, and leaving it out
// is not a tidiness problem -- it is a jump to the wrong address. Without
// it the compiler may allocate an *input* into rbx, and it did: clang 21
// put `entry` there, so the second instruction overwrote the jump target
// with the framebuffer address and the third jumped into video memory. The
// machine then executed pixels until KVM gave up.
//
// movq %rax, %cr3 <- %0, the page tables
// movq %rcx, %rbx <- %1, the framebuffer, into rbx
// jmpq *%rbx <- %2 was rbx, and now holds the framebuffer
//
// It had worked for as long as the compiler happened to choose differently.
// That is the same failure as the BIOS loader's EAX hand-off in
// docs/history/kernel.md section 4: a register the code depends on, that
// nothing in the source actually asks for.
__asm__ volatile(
"mov %0, %%cr3\n\t"
"mov %1, %%rbx\n\t"
"jmp *%2\n\t"
:
: "r"((UINT64)PML4_ADDR), "r"((UINT64)fb), "r"(entry)
: "memory");
: "rbx", "memory");
for (;;)
__asm__ volatile("hlt");