SchOS/docs/architecture/kernel.md

# SchIFT — the kernel

Long mode, 4-level paging, preemptive multitasking, and more than one
processor. What follows is the parts of the kernel that are not a filesystem, a
driver or a protocol stack: memory, processes, scheduling, traps and signals.

---

## Address space

One page table per process; the kernel is mapped into all of them at
`KERNBASE = 0xFFFF800000000000`, so a trap does not switch `CR3`.

`kvmalloc()` maps as many 1 GB pages of physical memory as the machine has RAM
(up to `PHYSTOP_MAX`, 256 GB), always at least the low four. The 3–4 GB slot stays
uncached because that is where the PCI hole is; everything above 4 GB is RAM, so
those are cached, and a device BAR up there still gets its own uncached window
from `ioremap()`, which takes the entries the direct map did not. On a CPU
without 1 GB pages (Intel's desktop parts before Haswell), each gigabyte is a
page directory of 2 MB pages instead — the same reach, one extra page of table
per gigabyte. A direct map with a fixed offset makes `P2V`/`V2P` arithmetic
rather than a lookup.

How much RAM there is comes from the firmware's E820/UEFI map, as a *list* of
usable regions, not one run ending at a top address (OVMF drops an ACPI NVS
region at 8 MB, which used to hide everything above it). The kernel itself is
linked at 16 MB, clear of what firmware keeps low. `kalloc` holds never-used
memory as ranges, so a large machine costs nothing until its memory is used.

`kalloc()` hands out one 4 KB page at a time from a free list. Objects smaller
than a page come from the slab caches below; objects larger than one do not have
a home yet, so the biggest tables (`NPROC` processes, `NSOCK` sockets, `NDEVICE`
devices, `NSHM` segments) are still fixed-size arrays, which is why their limits
are compile-time constants rather than a design decision about each one.

The free list carries a canary, checked at the top of both `kalloc()` and
`kfree()`, so a wild write into it panics at the point it is touched rather than
handing back a bad pointer that fails somewhere else entirely.

### DMA

`kernel/core/dma.c` owns an arena carved off the top of physical memory before
`kalloc` is given the rest. Devices need physically contiguous memory; `kalloc`
cannot promise it, so the region has to be reserved before there is any
competition for it.

## Per-CPU state

The per-CPU block's base is in **both** `FS_BASE` and `GS_BASE`, set by
`seginit()`:

```
  +0   self-pointer     required by the x86-64 TLS ABI (%fs:0)
  +8   syscall scratch  trapasm.S parks %rsp here across the stack switch
  +16  struct cpu *
  +24  struct proc *
  +32  syscall rax      the three registers wrmsr destroys, parked across
  +40  syscall rcx      the FS_BASE swap at the top of syscall_entry
  +48  syscall rdx
```

**These offsets are shared with assembly and must not be reordered.** Offset 8
is not free: `syscall_entry` uses `%fs:(8)`, and because FS and GS base point at
the same address, that is the same memory as `%gs:8`. A `_Static_assert` in
`vm.c` pins all seven, because assembly spells them as literals and a slot that
moves leaves a register quietly holding somebody else's value.

Reached through `mycpu()` and `myproc()`, which compile to a single
`movq %gs:16` / `%gs:24` with no cached base.

### The three words at +32, and what they cost before they were here

`syscall_entry` has to swap `FS_BASE` from the user's TLS to the kernel's
before it can touch any `%fs:` data, and `wrmsr` destroys `rax`, `rcx` and
`rdx`. Those three have to be parked somewhere first — not on the user stack,
which is untrusted and has not been switched away from yet.

They were parked in **three globals**, `_sc_rax`, `_sc_rcx` and `_sc_rdx`,
under a comment reading:

> Single-CPU scratch for syscall_entry FS swap (safe while IF cleared by
> SYSCALL/SFMASK)

`IF` being clear answers for an interrupt arriving on *this* CPU. It says
nothing about a second CPU executing the same three stores at the same instant,
and with more than one processor entering syscalls that is not a rare
interleaving — it is the ordinary one.

**`%rcx` is the user return address.** `SYSCALL` puts the user's `rip` there and
`SYSRET` takes it back from there. So the CPU that lost the race returned its
process into *another process's* instruction stream, and `%rax` — the syscall
number on the way in, the return value on the way out — was swapped with it.

The symptom was one to four processes per boot dying at `rip 0x0` (an
instruction fetch at zero, which a process does not branch to; it is handed a
zero and follows it) or dereferencing `0xffffffffffffffea`, which is `-EINVAL`:
a syscall return value used as a pointer. It hit shells and `schinitd` alike,
and it had been written off as a mysterious "zsh sometimes jumps into .data".

Measured, at four CPUs, with the same pristine disk image each run:

| | boots | faults |
|---|---|---|
| three globals | 10 | **10**, in five of the ten boots |
| three per-CPU words | 18 | **0** |

This is the *second* bug of exactly this shape in this file. The first was
`kernel_fs_base`, a single global that every CPU wrote during its own
`seginit()`, so the last writer won and every CPU restored one CPU's block.
`RESTORE_KERNEL_FS` reads `%gs:0` now for that reason. The lesson generalises
and is worth stating plainly: **on the entry path, a global is a bug.** There is
one per-CPU block and everything that changes per CPU belongs in it.

> **`__thread` is wrong for per-CPU kernel data**, and this is the single most
> expensive lesson in the tree. `__thread` means per *thread*, and the compiler
> is entitled to read the thread pointer once and keep it in a register across
> calls. But `%fs` here is per *CPU*, and a kernel thread changes CPU at every
> `sched()`. So the assumption is false for exactly the variables that most need
> to be right, and the failure is a null dereference deep in the scheduler on any
> machine with more than one core. `include/core/proc.h` carries the disassembly that
> proves it.

## Memory

`kalloc()` hands out one 4096-byte page and `kfree()` takes it back. That is the
whole page allocator: a free list threaded through the free pages themselves,
no buddy system, no zones, no page-frame database.

### Small objects

`kernel/core/kmem.c`, `include/core/kmem.h`. A slab is one page from `kalloc()` with
a header at the front and objects packed after it; free objects hold the free
list in their own first eight bytes.

That last detail is the whole design. **An object needs no header and freeing
needs no search**: the page base of any object *is* its slab, and the slab
names its cache. It is also why an object cannot exceed a page.

| | |
|---|---|
| `kmem_cache_create/alloc/free` | a named cache of one object size, for a subsystem with one kind of thing |
| `kmalloc` / `kmfree` | power-of-two classes from 16 to 2048, for code with no single type to build a cache around |

Each cache keeps `partial` and `full` lists; a slab that empties completely
goes straight back to `kalloc()` rather than being kept, so `/proc/slab` never
overstates what the kernel is holding.

`make kmem-test` asserts the bookkeeping on the build machine — the same
arrangement as `schfs-objtest`, and for the same reason: slab lists, free-list
threading and whether pages actually come back are all things that can be wrong
without a kernel to be wrong in.

### Shared memory objects

`shmget`/`shmat`/`shmdt` are System V shared memory: segments of `kalloc`'d
pages, reference counted, mapped through a VMA with `no_free` set so that
`freevm()` does not hand another process's pages to `kfree`.

`memfd_create(2)` is the same machinery reached by **descriptor** instead of by
key. That difference is the whole point and the reason it exists on Linux at
all: two processes share memory by passing the fd over a socket, so neither has
to agree on a path or race to invent a key. It is how a display server and its client
share a buffer, and it is why descriptor passing (`SCM_RIGHTS`) had to be built first.

| | |
|---|---|
| `memfd_create(name, flags)` | a new segment of size 0, and a descriptor owning one reference |
| `ftruncate(fd, n)` | gives it a size — growing allocates zeroed pages, shrinking frees from the end |
| `mmap(…, MAP_SHARED, fd, off)` | maps the segment's existing pages, so both mappers get the same physical memory |
| `read`/`write`/`lseek`/`fstat` | work, so it is a file and not a token only `mmap` understands |

Three things are refused rather than faked, and each refusal is the honest
version of something this kernel cannot do:

- **`MAP_PRIVATE` on a memfd.** Every caller is sharing on purpose; a private
  mapping of an anonymous object is an expensive `MAP_ANONYMOUS`.
- **Mapping past the end.** Linux lets it through and faults the pages in.
  There is no fault handler here that could, so the range is rejected.
- **Shrinking while mapped.** Linux truncates the pages out from under the
  mapper. Here a freed page would simply stay mapped to memory somebody else
  now owns.

Writable `MAP_SHARED` of a **real file** is still refused. That needs a page
cache, it is a separate project, and nothing that runs today asks for it — an
anonymous object has no such contract to break, which is exactly why `memfd` is
the narrow answer.

### Demand paging

Anonymous memory can be recorded without being backed: the page appears,
zeroed, the first time it is touched — by the program, through `trap()`, or by
the kernel on its behalf, through `uva2ka()`. `mmap` does this for mappings of
1 MB or more, for `MAP_NORESERVE`, and for `PROT_NONE`. The reason is WebKit,
which reserves 4 GB of address space at start for its JIT and heaps and touches
a sliver of it; backing all of it at `mmap` time failed with `ENOMEM` and the
browser aborted. Go, Java and Firefox reserve the same way.

`vm_fault()` takes a spinlock and nothing that sleeps, so `uva2ka()` may call it
from anywhere; the lock keeps two threads of one process from both mapping the
same page. `fork` and `munmap` skip the holes, so a page nobody touched is one
they never see, and a child that touches it reads zeroes — what it would have
read in the parent.

File-backed mappings are **not** lazy: `mmap` of a file still reads the range in
at map time.

### TLB shootdowns

Unmapping from an address space that threads on other processors are running in
interrupts those processors and waits until each has reloaded `CR3`
(`tlb_shootdown()` in `vm.c`; the vector is one of the MSI lines, which `trap()`
already dispatches with an EOI). Without it, when one thread unmapped a range and
`mmap` handed the same addresses straight back, a thread on another CPU went on
reading and writing the *old* pages through its TLB: stores vanished and loads
were stale. musl's allocator noticed first — a group's metadata no longer
pointing back at it — and Epiphany, forty threads on four processors, died at a
different address on every run.

Only one shootdown runs at a time, and it is taken without disabling interrupts:
two processors each waiting with interrupts off for the other to answer would
wait for ever. On a single processor, or if no vector could be allocated, it
does nothing (and says so at boot).

### What the fixed tables actually cost

The standing claim was that every fixed-size table in the kernel exists to
avoid asking `kalloc` for something smaller than a page. Measured, that is only
partly true:

| | size | table | |
|---|---|---|---|
| `struct proc` | 1984 | `NPROC` 128 | 254 KB in all; the address space, descriptors and signal handlers live in their own objects now, shared by threads |
| `struct mm` | 8232 | one per process, 128 | 1 MB in all, almost all of it the 256-entry VMA table |
| `struct socket` | about 4900 | `NSOCK` 128 | larger than a page |
| `struct device` | 120 | `NDEVICE` 128 | 15 KB, never hit |
| `struct buf` | 200 | `NBUF` 256 to start | grows from a slab, data from `kalloc`, up to RAM/32 |
| `struct inode` | about 460 | none | grows from a slab; `NINODE` 200 is only how many are made up front |
| `struct file` | about 88 | none | grows from a slab; `NFILE` 512 is only the static pool |

(Sizes from `nm -S` on a build of the tree, not from a header comment.)

`struct proc` shrank by a factor of six when threads arrived, because the page
tables, VMAs, descriptor table and signal dispositions moved out of it into
objects a thread group shares. `struct socket` is still too large for a
page-backed slab; freeing it needs a contiguous multi-page allocator, which is a
separate thing and is not written. The buffer header shrank to 200 bytes when its
4 KB of data moved out to a page of its own, which is what let the cache grow at
all.

## Processes

`struct proc` in [`include/core/proc.h`](../../include/core/proc.h). A fixed table of
`NPROC` entries, allocated by linear scan. A thread is a `struct proc` too.

**The field order at the top of the struct is fixed by assembly.** `trapasm.S`
hard-codes `fs_base` at offset `0x18`, and there is a `_Static_assert` in
`proc.c` that fails the build if it moves. Anything new goes at the end.

Each process carries: a kernel stack, cwd inode *and* cwd path string, signal
masks, POSIX identity (pgid, sid, and all six of real/effective/saved uid and
gid), the Linux-ABI syscall map, and its x87/SSE registers (saved on a switch;
the kernel itself stays off them). Three things it *points to* rather than
owns, because a thread group shares them:

| | |
|---|---|
| `struct mm` | the address space: page tables, heap top, the VMA table, which file each span came from, and a lock so one thread at a time changes the mappings |
| `struct files` | the descriptor table and its `FD_CLOEXEC` flags |
| `struct sighand` | signal dispositions — what `sigaction()` sets |

Each is reference counted. `clone()` shares what its flags say and `fork()`
copies all three.

### Threads

`clone()` takes `pthread_create`'s flags (`CLONE_VM`, `FILES`, `SIGHAND`,
`THREAD`, `SETTLS`, `PARENT_SETTID`, `CHILD_CLEARTID`) and `posix_spawn`'s
(`VM`, `VFORK`, a stack). What follows from that:

- **futex** — `WAIT`, `WAIT_BITSET` with timeouts, `WAKE`, `REQUEUE`,
  `CMP_REQUEUE`, keyed by the word's *kernel* address so processes that map the
  same page meet. `set_tid_address` keeps the word that exit zeroes and wakes,
  which is what `pthread_join` waits on.
- **Exit.** `exit` ends the calling thread; `exit_group`, a fatal signal, or the
  leader exiting ends them all. `reapd` frees exited threads, and the leader is
  reaped by its parent once the rest are gone. `getpid` is the group, `gettid`
  the thread.
- **Signals.** Stops and resumes reach every thread; other process-directed
  signals go to the leader.
- **`exec`** kills the other threads and unshares the dispositions.
- **`/proc`** lists processes, `task/` lists threads; `comm`, `Threads:` and
  `Tgid` are there.

The first program to need all of this was GTK's, then WebKit's.

### Slot reuse

The table recycles slots, and a recycled slot that inherits the previous
occupant's state is a whole class of bug that has bitten here twice.
`allocproc()` therefore explicitly clears the signal fields, the exit status,
the kill signal, and the identity fields. The failure mode when it did not:
`deliver_signals()` jumping to a totally unrelated earlier process's handler
address on this process's very first syscall return.

### Kernel threads

`kthread_create(fn, name)` makes a process that runs `fn` in kernel mode and
never returns to userland. It gets a page directory with only the kernel mapped,
no files, no cwd, no parent — it is not in the process tree, cannot be waited
for, and must never exit.

The point is that it is a *process* and not a timer callback: it shares the
scheduler's ordinary rotation, so it can hold sleeplocks and block on the disk
exactly as a syscall would. The network thread is the current user.

## Scheduling

Fair share by virtual runtime, one scheduler per CPU, each with its own context
— the rule, the guards and `nice` are in [scheduler.md](scheduler.md).
`swtch()` saves callee-saved registers and switches stacks; `sched()` returns to
the scheduler and asserts four invariants on the way — holding `ptable.lock`,
`ncli == 1`, not `RUNNING`, interrupts off.

When nothing is runnable the CPU goes idle rather than spinning: into ACPI's
`_CST` C2 state (CC6 on Zen), by port read or `MWAIT`, where the APIC timer keeps
running through it, and otherwise a plain `hlt`.

**Locking is a single `ptable.lock` over the whole table.** Not per-process: a
scheduling operation touches two processes (the one leaving and the one
arriving) and two-object locking needs an ordering rule that every path obeys.
The same reasoning produced a single lock over the socket table.

### sleep / wakeup

xv6's discipline, unchanged because it is correct: `sleep(chan, lk)` atomically
releases `lk` and blocks; `wakeup(chan)` makes every sleeper on that channel
runnable. The atomicity is what matters — a sleeper that tests its condition
under one lock while the waker sets it under another has no interlock and sleeps
through the wakeup meant for it.

## Traps and syscalls

Interrupts and exceptions enter through `vectors.S` → `trapasm.S` → `trap()`.
Syscalls enter through `SYSCALL`/`SYSRET` and `syscall_entry`, which is a
separate and faster path.

Every entry from user mode checks a **canary at the bottom of the kernel stack
page**. A deep or nested trap that overflows the 4 KB stack clobbers that word
first, so the failure is reported as `kstack overflow` at the point it happened
rather than as arbitrary corruption later.

One configuration detail with an expensive history: `MSR_STAR`'s `SYSRET`
selector field must hold the **compat 32-bit** code selector, because `SYSRETQ`
computes the returned `cs`/`ss` as `STAR[63:48]+16`/`+8`. Putting the real
64-bit user selector there makes every `sysretq` load an invalid pair — which
QEMU's TCG does not validate, so it stayed hidden until a hardware interrupt
forced a strictly-validated `IRETQ`.

## Signals

Real delivery, not stubs. `sys_rt_sigaction()` records the handler;
`deliver_signals()` runs in the tail of `syscall()`, just before returning to
user mode, and if a pending unblocked signal has a handler it rewrites the
trapframe to enter the handler instead of resuming — stashing the original in
`proc->saved_sigtf`. `sys_rt_sigreturn()` puts it back.

The return address is **not** `sa_restorer`. `SA_RESTORER` is not defined in
this musl build, so `sigaction()` never fills one in, and pushing 0 as a return
address crashes with `rip = 0` the moment the handler returns. Instead a 7-byte
trampoline (`mov eax,15; syscall`) is injected onto the user's own stack below
`rsp`, clear of the SysV red zone — the standard technique when the C library
supplies no restorer.

Signals without a handler fall back to a default action: termination for the
fatal set, dropped otherwise.

### PID 1 does not die from a signal it never asked to handle

`init_ignores()` discards any signal whose disposition is the default action
when the target is PID 1 — **including `SIGKILL`**. This is what POSIX requires
and what Linux does in `sig_task_ignored()`, and without it the system has a
single keystroke that kills it: Ctrl-C signals the foreground process group,
init is pgid 1, and any moment where the terminal's foreground group is still 1
lands SIGINT on init. It has no handler, so `exit()` runs, and `exit()` panics
outright on initproc.

Spamming Ctrl-C at a frozen program is exactly the input that finds that window.

## Process groups and sessions

`setsid`, `setpgid`, `getpgid`, `getsid`, and signal delivery to a whole group.
Job control is defined entirely in terms of these — a shell puts each pipeline
in its own process group and signals the group, never individual pids — and
mksh's job control could not make progress until they existed.

## What userspace gets

`fork`, `clone` (threads), `exec`, `wait`/`waitpid`/`wait4` with `WNOHANG` and
real pid selection, pipes, signals, `mmap` (anonymous and file-backed), POSIX
shared memory, `memfd`, Unix domain sockets, `poll`/`select`/`epoll`, `eventfd`,
`timerfd`, `signalfd`, `inotify`, `futex`.

### Dynamic linking

`exec` handles `ET_EXEC` and `ET_DYN`. A dynamically linked program names an
interpreter in `PT_INTERP` — on musl, `/lib/ld-musl-x86_64.so.1`, which is libc
under another name — and the kernel's whole part is to load that as well and
start it instead of the program. The interpreter finds the program through the
auxiliary vector (`AT_PHDR`, `AT_ENTRY`), maps libc and whatever else it needs
with `mmap()`, relocates, and jumps to `AT_ENTRY` itself. A position-independent
program goes where a fixed one would (`0x400000`); the interpreter goes at the
start of the `mmap` area (`0x40000000`) and `mmap` carries on above it, so the
two never meet. This is what lets Alpine's binaries run unmodified, and
`dlopen` works, so Xorg loads its drivers as modules.

`mmap` uses a bump allocator (`proc->mmap_next`, starting at `0x40000000`) so
every mapping gets its own disjoint, exactly-sized range. It previously assigned
a fixed 8 MB window per VMA slot regardless of the requested length, so a single
allocation over 8 MB mapped straight into a neighbouring live mapping.

---

## What it does not do

- **`ru_stime` is effectively always zero.** CPU time is sampled at the timer
  tick and split on the interrupted privilege level, but `MSR_SFMASK` includes
  `FL_IF`, so `SYSCALL` clears interrupts and no tick can land while a process
  is inside a syscall. User time is real and discriminates properly — a busy
  loop reports 97%, `sleep 2` reports 0% — but kernel time needs TSC deltas
  closed and reopened around every `sched()`, which is scheduler accounting
  rather than sampling, and is not written.
- **`struct socket` is too big for a slab**, and every subsystem's fixed-size
  table (`NPROC`, `NSOCK`, `NDEVICE`, `NSHM`) is still a compile-time constant.
  Slabs exist for the objects that fit in a page.
- **No page cache.** `mmap` of a file *reads the range in* at map time, and
  writable `MAP_SHARED` of a file is refused outright rather than faked. Demand
  paging covers anonymous memory only.
- **Root is `euid == 0` and nothing finer.** No capabilities; file access, `kill`, `exec`
  and the privileged calls are enforced as [users.md](users.md) says.
- **`mprotect` is accepted and not enforced** (`sys_mprotect` returns 0). RELRO is not read-only,
  a `PROT_NONE` guard does not fault, and nothing has `NX` (`EFER.NXE` is never set), so every
  mapping is executable.
- **`poll`, `select` and `epoll` wait a tick at a time** (10 ms at `HZ=100`) unless something
  that can make a descriptor ready calls `poll_kick()` — which the pipe, socket and event paths
  do. busybox's `wget` polls before every read and downloaded at 85 KB/s, a segment per tick,
  until the network path started kicking.
- **No swap**, and no memory pressure handling of any kind. `kalloc` failing is
  reported up as `-ENOMEM` and that is the whole policy.
- **Nested signal delivery is not supported.** `in_signal` guards against
  re-entering a handler while one is running.
- **One global run queue.** Fair share and `nice` exist; per-CPU run queues, work
  stealing and CPU affinity do not, and `sched_setaffinity` is accepted and
  ignored.
- **No IOMMU**, so DMA-capable devices (and the driver processes that program
  them) are trusted.
- **256 GB is the most RAM** the kernel uses; the rest is ignored, with a line
  saying so.