docs: four directories, and a rule for when a thing leaves the roadmap
docs/ was ten files where a plan, a description, a guide and a bug report all
sat next to each other. It is now architecture/ (what SchOS is), plan/ (what it
will be), guides/ (how to work on it) and history/ (why it is like this), with
an index that states the one rule holding it together: when something is
finished it moves from plan/ to architecture/ and is rewritten as a description
of the thing rather than a record of building it.
A ticked checkbox says the work happened. It does not say what the work
*produced*, and six months later the checkbox is all that is left. The network
stack proved the point: finished, verified end to end, and for a month the only
description of it anywhere was a checkbox four levels of indentation deep.
So roadmap.md went from 534 lines to 181 -- nothing deleted, the done work
graduated. Seven subsystems that were finished and undescribed now have
descriptions: boot, kernel, syscalls, filesystem, networking, graphics, usb,
init. Every one ends with "What it does not do", which is the section that pays;
a document listing only strengths is marketing, and the reader who needs it most
is the one about to trip over a limitation nobody wrote down.
schfs.md still opened with "phases 1 through 3 1/2 are done, phase 4 is next"
long after all five had landed. It is a description now, with the phase record
and the hard-won bugs moved to history/.
Three corrections found on the way, all load-bearing:
* projects/schcom/README.md claimed SCM_RIGHTS was done. There is no
sendmsg/recvmsg in the kernel at all, and an entire GUI plan rested on it.
* The root README said "No TCP/IP" months after the stack landed.
* history/kernel.md sec.7 was marked "in progress" on a problem that process
groups solved.
Also adds plan/1.0.md, which draws a finish line so the open list stops being
infinite, and marks schwm/ and schde/ superseded: in Wayland the compositor is
the window manager, so that split dissolves rather than being abandoned.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
This commit is contained in:
Nexoniarz
parent
817d666068
commit
57aff40228
59 changed files with 3981 additions and 1124 deletions
215
docs/architecture/boot.md
Normal file
215
docs/architecture/boot.md
Normal file
|
|
@ -0,0 +1,215 @@
|
|||
# Boot
|
||||
|
||||
Firmware to the first shell prompt. This is the shortest path through the whole
|
||||
system, which makes it the best thing to read first.
|
||||
|
||||
The loader's side of the hand-off is specified in
|
||||
[`boot/README.md`](../../boot/README.md) — that file is the contract, and this
|
||||
one is the narrative.
|
||||
|
||||
---
|
||||
|
||||
## 1. Firmware
|
||||
|
||||
**UEFI only.** There was a BIOS path and it was removed: everything since
|
||||
roughly 2012 boots UEFI, and CSM — the firmware-side BIOS emulation that let
|
||||
legacy loaders run — has been going away since 2020. The cost of keeping it was
|
||||
not the code but the tax on everything it touched, and the asymmetry is total. A
|
||||
BIOS loader lives in 16-bit real mode and calls the firmware through software
|
||||
interrupts; UEFI hands a loader long mode with paging already on and a table of
|
||||
function pointers.
|
||||
|
||||
The loader is `boot/uefi/`, a 64-bit PE application built by the same clang
|
||||
install as everything else (`--target=x86_64-unknown-windows`). It imports
|
||||
neither gnu-efi nor EDK II — the handful of structures it uses were written from
|
||||
the specification, which is both a smaller dependency and no licence question.
|
||||
|
||||
## 2. The loader
|
||||
|
||||
Six steps, in this order, and the order is load-bearing:
|
||||
|
||||
1. Find the kernel on the volume this image was loaded from — `\schos\kernel`,
|
||||
or `\kernel`.
|
||||
2. Load its ELF `PT_LOAD` segments to their physical addresses.
|
||||
3. Record the framebuffer geometry from the Graphics Output Protocol.
|
||||
4. Translate `GetMemoryMap()` into the 24-byte E820-shaped entries
|
||||
`kernel/arch/x86_64/e820.c` already parses — not a second format for the
|
||||
kernel to learn.
|
||||
5. `ExitBootServices`.
|
||||
6. Build the page tables the kernel's 64-bit entry expects, and jump.
|
||||
|
||||
**Nothing allocates after step 5 and nothing calls a firmware service after it,
|
||||
because there are none.** Everything the jump needs is computed before.
|
||||
|
||||
### The hand-off block
|
||||
|
||||
One page of low memory at `0x30000`, defined once in
|
||||
[`include/schboot.h`](../../include/schboot.h) and included by both the loader
|
||||
and `entry.S` so the two cannot disagree.
|
||||
|
||||
| Offset | What |
|
||||
|---|---|
|
||||
| `+0x000` | VBE-shaped mode info: LFB address, stride, width, height, depth |
|
||||
| `+0x100` | Verbose-boot flag |
|
||||
| `+0x101` | Boot medium (`1` = CD; `0` = not stated) |
|
||||
| `+0x200` | E820 entry count, then up to 40 × 24-byte entries |
|
||||
| `+0x600` | ACPI RSDP physical address, or 0 |
|
||||
|
||||
Two of these exist because of specific failures worth knowing about.
|
||||
|
||||
**The RSDP is passed explicitly** because the kernel's own search scans the EBDA
|
||||
and `0xE0000`–`0x100000`, which is a BIOS convention. UEFI publishes the RSDP in
|
||||
its configuration table and is under no obligation to leave a copy anywhere;
|
||||
OVMF does not. Without it the kernel finds no MADT, so no IOAPIC, so no
|
||||
interrupt routing — and the machine boots to a prompt on polled serial output
|
||||
and then ignores every keystroke, which looks like a broken console rather than
|
||||
a missing table.
|
||||
|
||||
**The E820 cap is 40 entries, not 32.** Real UEFI firmware produces far more
|
||||
descriptors than QEMU's, and the space was already there.
|
||||
|
||||
### The entry point
|
||||
|
||||
The kernel publishes a `sch64_header` — magic `SCH64BOO`, 8-byte aligned within
|
||||
the first 16 KB, followed by a 64-bit physical entry address — found by
|
||||
scanning, exactly as a multiboot loader scans for its own magic.
|
||||
|
||||
It exists because the firmware has *already* put the machine in long mode.
|
||||
Dropping back to protected mode to re-enter through the 32-bit multiboot path
|
||||
is a mode switch whose failure mode is a triple fault with no console. The
|
||||
32-bit entry is still present in `entry.S` and now has no caller.
|
||||
|
||||
## 3. `kmain()`
|
||||
|
||||
`kernel/core/main.c`. The ordering here is not arbitrary and several steps are
|
||||
where they are because of a bug:
|
||||
|
||||
```
|
||||
e820_init() memory map first — PHYSTOP is a variable, not a constant
|
||||
kinit1() the allocator, given everything below the DMA arena
|
||||
dma_init() the arena, carved off the TOP of physical memory
|
||||
kvmalloc() the kernel page tables
|
||||
acpi_init() CPU enumeration from the MADT; mpinit() only as fallback
|
||||
lapicinit() tvinit() seginit() ioapicinit()
|
||||
irq_init() before any driver that registers a handler
|
||||
inputinit() before the keyboard and mouse start producing events
|
||||
consoleinit() uartinit() mouseinit()
|
||||
platform_register_devices(fb) the fixed furniture of a PC, plus the framebuffer
|
||||
pci_init() enumerate, and bind drivers
|
||||
pinit() binit() fileinit() sockinit()
|
||||
time_init() after the timer runs, before userspace
|
||||
ideinit() ramdiskinit()
|
||||
startothers() the other processors
|
||||
kinit2() sinit() the rest of memory, then PID 1
|
||||
netinit() after sinit — it starts a kernel thread
|
||||
mpmain() enter the scheduler; never returns
|
||||
```
|
||||
|
||||
**The DMA arena is taken before `kalloc` gets the rest.** A device needs
|
||||
physically contiguous memory and `kalloc`'s free list makes no such promise —
|
||||
two pages from it are adjacent only by chance. Taking it first is the only way
|
||||
to be sure the region exists.
|
||||
|
||||
**ACPI is asked before the legacy MP table**, and getting this backwards is
|
||||
invisible under QEMU. Modern firmware publishes a complete MADT *and* an MP
|
||||
table containing one processor, so asking the older interface first means never
|
||||
seeing more than one CPU. VMware published the real count, which is why the
|
||||
resulting triple fault only ever appeared there.
|
||||
|
||||
**`main()` names no display driver.** Whichever driver owns the framebuffer the
|
||||
loader left running publishes it with `gfx_attach_scanout()`, and the console
|
||||
and crash screen draw on that. See [graphics.md](graphics.md).
|
||||
|
||||
### Memory sizing
|
||||
|
||||
`PHYSTOP` is a variable read from the firmware's map, not a constant. It used to
|
||||
be `0xE000000` — 224 MB — inherited from xv6, and it was not a limit on what the
|
||||
kernel used but an **assertion that exactly that much RAM existed**. Less and the
|
||||
allocator walked off the end of memory; more and it was invisible.
|
||||
|
||||
Three details survive that fix:
|
||||
|
||||
- It takes the end of the usable region **containing the kernel**, not the
|
||||
highest usable address anywhere. A memory map has reserved holes, and freeing
|
||||
a single range from the kernel's end to a global maximum puts firmware tables
|
||||
on the free list — failing later, elsewhere, looking like anything but a
|
||||
memory map problem.
|
||||
- The usable regions are *also* kept as a list, and `kinit1()` frees each one.
|
||||
One range ending at a top address is what a BIOS map looks like and what UEFI
|
||||
never does: OVMF drops an ACPI NVS region at `0x800000`, so the run containing
|
||||
the kernel ends at 8 MB and everything above the hole was invisible.
|
||||
- The ceiling is 3 GB, because `kvmalloc()` maps 0–4 GB as four 1 GB pages and
|
||||
marks the top one uncached for PCI BARs. The floor is 24 MB, and below it the
|
||||
kernel prints why and stops rather than triple faulting in silence.
|
||||
|
||||
## 4. SMP
|
||||
|
||||
`startothers()` wakes each application processor through the local APIC, pointing
|
||||
it at a trampoline copied to `0x7000`. Each AP runs `mpenter()` → `mpmain()`,
|
||||
reports `cpu%d: starting`, and enters the scheduler.
|
||||
|
||||
This was broken from the project's beginning and is fixed. Five distinct bugs,
|
||||
each hidden behind the one before it — the details are in
|
||||
[../history/kernel.md](../history/kernel.md), and the one worth carrying
|
||||
forward is in [kernel.md](kernel.md#per-cpu-state): **`__thread` is wrong for
|
||||
per-CPU kernel data.**
|
||||
|
||||
APs are counted on the boot processor *after* each one reports in, so the count
|
||||
cannot include a processor that started and died.
|
||||
|
||||
## 5. The first process
|
||||
|
||||
`sinit()` builds PID 1 by hand: a page directory, `initcode` mapped at 0, 32
|
||||
pages of stack, a trapframe pointing at user mode, and session and process group
|
||||
both set to 1.
|
||||
|
||||
It does **not** mount the root filesystem, and cannot: mounting reads blocks,
|
||||
`bread()` sleeps, and sleeping needs a process context. So the mount happens in
|
||||
`forkret()`, the first time any process is scheduled — guarded by a `static int
|
||||
first`.
|
||||
|
||||
That is also where PID 1's working directory is re-pointed at the real root,
|
||||
and the reason is worth keeping:
|
||||
|
||||
> `sinit()` sets `cwd = namei("/")` before any filesystem is mounted. The old
|
||||
> v1 root survived that because `iget(ROOTDEV, ROOTINO)` is two compile-time
|
||||
> constants and touches no disk. SchFS has no constant to offer — the root
|
||||
> object id comes off the uberblock — so that early `namei("/")` builds an inode
|
||||
> from an unmounted volume. Nothing complains, because absolute paths never
|
||||
> consult cwd: `exec("/bin/init")` works fine. It is the first *relative* lookup
|
||||
> that fails, and in init that is `open("console")`. No console means no fd
|
||||
> 0/1/2, so every shell it spawns exits immediately and init respawns it
|
||||
> forever. The symptom is a silent boot that reads like a hang and is not one.
|
||||
|
||||
`initcode` then execs `/bin/init`, and [init.md](init.md) takes over.
|
||||
|
||||
## 6. What the screen shows
|
||||
|
||||
**The display is not the kernel log.** `cons_serial_only` starts *set*: every
|
||||
boot message goes out the UART and nothing goes to the framebuffer. The loader's
|
||||
splash therefore survives the whole of boot rather than being wiped a few
|
||||
hundred milliseconds in, and `console_takeover()` clears the flag on the first
|
||||
`exec()`, when userspace genuinely starts.
|
||||
|
||||
Nothing is lost — the serial line is where `bootcheck` reads from and where you
|
||||
want the messages when something is wrong — and the F8 verbose boot never sets
|
||||
the flag, so "show me everything" still works.
|
||||
|
||||
---
|
||||
|
||||
## What it does not do
|
||||
|
||||
- **No BIOS path.** A machine without UEFI does not boot SchOS.
|
||||
- **The root device is compiled in.** Booting from a USB disk needs the root
|
||||
device to be selectable, which it is not; the USB mass-storage driver works
|
||||
and a superblock reads back through `bread()`, but nothing can mount it.
|
||||
- **`make ramroot` is the only way to boot with no data disk.** It links a
|
||||
filesystem image into the kernel binary, which costs boot time and carries a
|
||||
licensing constraint — see [../licensing.md](../licensing.md) §4.
|
||||
- **The 32-bit multiboot entry is dead code.** It is left in place deliberately:
|
||||
deleting kernel entry code is its own piece of work with its own way of
|
||||
failing silently.
|
||||
- **`make qemu-uefi` under OVMF reports less memory than the machine has** in
|
||||
some configurations. The loader's map is correct; the fix went in with the
|
||||
region list above, and any remaining case is a fragmentation pattern not yet
|
||||
seen.
|
||||
Loading…
Reference in a new issue