SchOS/docs/architecture/boot.md

# Boot

Firmware to the first shell prompt. This is the shortest path through the whole
system, which makes it the best thing to read first.

The loader's side of the hand-off is specified in
[`boot/README.md`](../../boot/README.md) — that file is the contract, and this
one is the narrative.

---

## 1. Firmware

**UEFI only.** There was a BIOS path and it was removed: everything since
roughly 2012 boots UEFI, and CSM — the firmware-side BIOS emulation that let
legacy loaders run — has been going away since 2020. The cost of keeping it was
not the code but the tax on everything it touched, and the asymmetry is total. A
BIOS loader lives in 16-bit real mode and calls the firmware through software
interrupts; UEFI hands a loader long mode with paging already on and a table of
function pointers.

The loader is `boot/uefi/`, a 64-bit PE application built by the same clang
install as everything else (`--target=x86_64-unknown-windows`). It imports
neither gnu-efi nor EDK II — the handful of structures it uses were written from
the specification, which is both a smaller dependency and no licence question.

## 2. The loader

Six steps, in this order, and the order is load-bearing:

1. Find the kernel on the volume this image was loaded from — `\schos\kernel`,
   or `\kernel`.
2. Load its ELF `PT_LOAD` segments to their physical addresses.
3. Record the framebuffer geometry from the Graphics Output Protocol.
4. Translate `GetMemoryMap()` into the 24-byte E820-shaped entries
   `kernel/arch/x86_64/e820.c` already parses — not a second format for the
   kernel to learn.
5. `ExitBootServices`.
6. Build the page tables the kernel's 64-bit entry expects, and jump.

**Nothing allocates after step 5 and nothing calls a firmware service after it,
because there are none.** Everything the jump needs is computed before.

### The hand-off block

One page of low memory at `0x30000`, defined once in
[`include/schboot.h`](../../include/schboot.h) and included by both the loader
and `entry.S` so the two cannot disagree.

| Offset | What |
|---|---|
| `+0x000` | VBE-shaped mode info: LFB address, stride, width, height, depth |
| `+0x100` | Verbose-boot flag |
| `+0x101` | Boot medium (`1` = CD; `0` = not stated) |
| `+0x200` | E820 entry count, then up to 40 × 24-byte entries |
| `+0x600` | ACPI RSDP physical address, or 0 |

Two of these exist because of specific failures worth knowing about.

**The RSDP is passed explicitly** because the kernel's own search scans the EBDA
and `0xE0000`–`0x100000`, which is a BIOS convention. UEFI publishes the RSDP in
its configuration table and is under no obligation to leave a copy anywhere;
OVMF does not. Without it the kernel finds no MADT, so no IOAPIC, so no
interrupt routing — and the machine boots to a prompt on polled serial output
and then ignores every keystroke, which looks like a broken console rather than
a missing table.

**The E820 cap is 40 entries, not 32.** Real UEFI firmware produces far more
descriptors than QEMU's, and the space was already there.

### The entry point

The kernel publishes a `sch64_header` — magic `SCH64BOO`, 8-byte aligned within
the first 16 KB, followed by a 64-bit physical entry address — found by
scanning, exactly as a multiboot loader scans for its own magic.

It exists because the firmware has *already* put the machine in long mode.
Dropping back to protected mode to re-enter through the 32-bit multiboot path
is a mode switch whose failure mode is a triple fault with no console. The
32-bit entry is still present in `entry.S` and now has no caller.

## 3. `kmain()`

`kernel/core/main.c`. The ordering here is not arbitrary and several steps are
where they are because of a bug:

```
  e820_init()          memory map first — PHYSTOP is a variable, not a constant
  kinit1()             the allocator, given everything below the DMA arena
  dma_init()           the arena, carved off the TOP of physical memory
  kvmalloc()           the kernel page tables
  acpi_init()          CPU enumeration from the MADT; mpinit() only as fallback
  lapicinit() tvinit() seginit() ioapicinit()
  irq_init()           before any driver that registers a handler
  inputinit()          before the keyboard and mouse start producing events
  consoleinit() uartinit() mouseinit()
  platform_register_devices(fb)   the fixed furniture of a PC, plus the framebuffer
  pci_init()           enumerate, and bind drivers
  pinit() binit() fileinit() sockinit()
  time_init()          after the timer runs, before userspace
  ideinit() ramdiskinit()
  startothers()        the other processors
  kinit2() sinit()     the rest of memory, then PID 1
  netinit()            after sinit — it starts a kernel thread
  mpmain()             enter the scheduler; never returns
```

**The DMA arena is taken before `kalloc` gets the rest.** A device needs
physically contiguous memory and `kalloc`'s free list makes no such promise —
two pages from it are adjacent only by chance. Taking it first is the only way
to be sure the region exists.

**ACPI is asked before the legacy MP table**, and getting this backwards is
invisible under QEMU. Modern firmware publishes a complete MADT *and* an MP
table containing one processor, so asking the older interface first means never
seeing more than one CPU. VMware published the real count, which is why the
resulting triple fault only ever appeared there.

**`main()` names no display driver.** Whichever driver owns the framebuffer the
loader left running publishes it with `gfx_attach_scanout()`, and the console
and crash screen draw on that. See [graphics.md](graphics.md).

### Memory sizing

`PHYSTOP` is a variable read from the firmware's map, not a constant. It used to
be `0xE000000` — 224 MB — inherited from xv6, and it was not a limit on what the
kernel used but an **assertion that exactly that much RAM existed**. Less and the
allocator walked off the end of memory; more and it was invisible.

Three details survive that fix:

- It takes the end of the usable region **containing the kernel**, not the
  highest usable address anywhere. A memory map has reserved holes, and freeing
  a single range from the kernel's end to a global maximum puts firmware tables
  on the free list — failing later, elsewhere, looking like anything but a
  memory map problem.
- The usable regions are *also* kept as a list, and `kinit1()` frees each one.
  One range ending at a top address is what a BIOS map looks like and what UEFI
  never does: OVMF drops an ACPI NVS region at `0x800000`, so the run containing
  the kernel ends at 8 MB and everything above the hole was invisible.
- The ceiling is 3 GB, because `kvmalloc()` maps 0–4 GB as four 1 GB pages and
  marks the top one uncached for PCI BARs. The floor is 24 MB, and below it the
  kernel prints why and stops rather than triple faulting in silence.

## 4. SMP

`startothers()` wakes each application processor through the local APIC, pointing
it at a trampoline copied to `0x7000`. Each AP runs `mpenter()` → `mpmain()`,
reports `cpu%d: starting`, and enters the scheduler.

This was broken from the project's beginning and is fixed. Five distinct bugs,
each hidden behind the one before it — the details are in
[../history/kernel.md](../history/kernel.md), and the one worth carrying
forward is in [kernel.md](kernel.md#per-cpu-state): **`__thread` is wrong for
per-CPU kernel data.**

APs are counted on the boot processor *after* each one reports in, so the count
cannot include a processor that started and died.

## 5. The first process

`sinit()` builds PID 1 by hand: a page directory, `initcode` mapped at 0, 32
pages of stack, a trapframe pointing at user mode, and session and process group
both set to 1.

It does **not** mount the root filesystem, and cannot: mounting reads blocks,
`bread()` sleeps, and sleeping needs a process context. So the mount happens in
`forkret()`, the first time any process is scheduled — guarded by a `static int
first`.

That is also where PID 1's working directory is re-pointed at the real root,
and the reason is worth keeping:

> `sinit()` sets `cwd = namei("/")` before any filesystem is mounted. The old
> v1 root survived that because `iget(ROOTDEV, ROOTINO)` is two compile-time
> constants and touches no disk. SchFS has no constant to offer — the root
> object id comes off the uberblock — so that early `namei("/")` builds an inode
> from an unmounted volume. Nothing complains, because absolute paths never
> consult cwd: an absolute `exec` works fine. It is the first *relative* lookup
> that fails, and in init that is `open("console")`. No console means no fd
> 0/1/2, so every shell it spawns exits immediately and init respawns it
> forever. The symptom is a silent boot that reads like a hang and is not one.

`initcode` then execs **`/bin/schinit`**, and [init.md](init.md) takes over.
There is no fallback path and deliberately no `/bin/init` any more: the older
hardcoded fork-a-shell PID 1 was unreachable from the day that string changed,
and an unreachable fallback rots rather than saves.

## 6. What the screen shows

**The kernel log is on screen from the first line that can draw.** `kmain`
attaches the loader's framebuffer and clears `cons_serial_only` before it does
anything else, so `pci: 7 devices` and `schfs: mounted` are visible on the
machine itself.

This is the reverse of what the flag was added for, and the reversal was earned.
The original argument was that a boot which prints a developer's log to the
screen looks like a hobby project, and it is a good argument — right up until a
Chromebox with **no serial port** froze at boot showing nothing at all. The whole
diagnosis had to be made from a photograph of a screen that had been told to stay
quiet. A tidy boot that cannot be debugged on the hardware it fails on is not a
trade worth making.

`cons_serial_only` still exists for the case it is now the right answer to: no
framebuffer from the loader. Then everything goes to the UART and
`console_takeover()` clears the flag when userspace starts.

---

## 7. What the boot says

A working boot prints one line:

```
SchOS UEFI loader
Press F8 for boot options... booting

Welcome to SchOS!  (F8 at boot for the full log, or dmesg once you are in)

root@schos / »
```

It used to print about thirty — the memory map, the local APIC's calibration,
every PCI device, each driver announcing itself, each job starting. All of that
is genuinely useful, and all of it is useful *once*. Printed on every boot it
trains the person in front of the machine to stop reading their own console,
which is expensive on the day one of those lines changes.

**Nothing was deleted; it was routed.** `consputc()` writes to the klog ring
before it writes anywhere else, so a line can be kept in full while being held
back from the two places a person is looking. `kinfo()` is `cprintf()` with
that routing, and the whole boot is still there afterwards:

```
# dmesg | head -4
--- boot 1 ---
[     0.00] e820: 1017 MB usable in 8 regions, top 1006 MB (22 entries)
[     0.00] acpi: 2 cpus, lapic at ffff8000fee00000, ioapic id 0
[     0.00] pci: 7 devices
```

The rule for `kinfo` against `cprintf` is not how interesting a line is —
everything was interesting to somebody once. It is: **would you want to see
this on a machine that is working?** `pci: 7 devices` is yes the first time and
no every time after. `e1000: no DMA memory for the rings` is yes for ever, and
stays a `cprintf`.

Three levels, then, and the middle one is new:

| | |
|---|---|
| quiet (default) | the greeting, plus anything that went wrong |
| verbose (F8) | every line above, on the console and the serial line |
| the klog | all of it, always, whichever of the two you booted |

Userspace follows the same rule through `/proc/bootmode`, which holds one word.
`schinit`, `schinitd` and `vgpud` read it and hold back their progress lines —
but never their failures, because a supervisor that goes quiet when a job dies
is worse than one that chatters.

**`schinit` reads it after `bootstrap()` and not before**, which is the sort of
detail that only shows up when tested: `/proc` is what `bootstrap()` mounts, so
reading it earlier returned "quiet" every time and made the verbose boot quiet
in the one process that runs first.

## What it does not do

- **No BIOS path.** A machine without UEFI does not boot SchOS.
- **The quiet boot hides a slow one.** With nothing printing between the
  loader and the greeting, a boot that stalls for ten seconds looks the same as
  one that does not. F8 is the answer and it has to be pressed *before* the
  stall, which is the wrong order for finding out.
- **`/proc/bootmode` is read once, at start.** A program that wants to be
  quieter or louder later has nowhere to say so, and nothing re-reads it.
- **The root device is compiled in.** Booting from a USB disk needs the root
  device to be selectable, which it is not; the USB mass-storage driver works
  and a superblock reads back through `bread()`, but nothing can mount it.
- **`make ramroot` is the only way to boot with no data disk.** It links a
  filesystem image into the kernel binary, which costs boot time and carries a
  licensing constraint — see [../licensing.md](../licensing.md) §4.
- **The 32-bit multiboot entry is dead code.** It is left in place deliberately:
  deleting kernel entry code is its own piece of work with its own way of
  failing silently.
- **`make qemu-uefi` under OVMF reports less memory than the machine has** in
  some configurations. The loader's map is correct; the fix went in with the
  region list above, and any remaining case is a fragmentation pattern not yet
  seen.