# SchIFT — the kernel Long mode, 4-level paging, preemptive multitasking, and more than one processor. What follows is the parts of the kernel that are not a filesystem, a driver or a protocol stack: memory, processes, scheduling, traps and signals. --- ## Address space One page table per process; the kernel is mapped into all of them at `KERNBASE = 0xFFFF800000000000`, so a trap does not switch `CR3`. `kvmalloc()` maps physical 0–4 GB as four 1 GB pages, with the top one marked uncached — that is where PCI BARs and the framebuffer live. This is why 3 GB is the most memory the kernel can treat as RAM, and it is a deliberate simplicity: a direct map with a fixed offset makes `P2V`/`V2P` arithmetic rather than a lookup. `kalloc()` hands out one 4 KB page at a time from a free list. **There is no slab or pool allocator**, and the consequence is visible throughout the tree: every subsystem that needs many small objects has invented its own fixed-size table — `NPROC` processes, `NSOCK` sockets, `NDEVICE` devices, `NSHM` segments. That is why the limits are compile-time constants rather than a design decision about each one. The free list carries a canary, checked at the top of both `kalloc()` and `kfree()`, so a wild write into it panics at the point it is touched rather than handing back a bad pointer that fails somewhere else entirely. ### DMA `kernel/core/dma.c` owns an arena carved off the top of physical memory before `kalloc` is given the rest. Devices need physically contiguous memory; `kalloc` cannot promise it, so the region has to be reserved before there is any competition for it. ## Per-CPU state The per-CPU block's base is in **both** `FS_BASE` and `GS_BASE`, set by `seginit()`: ``` +0 self-pointer required by the x86-64 TLS ABI (%fs:0) +8 syscall scratch trapasm.S parks %rsp here across the stack switch +16 struct cpu * +24 struct proc * ``` **These offsets are shared with assembly and must not be reordered.** Offset 8 is not free: `syscall_entry` uses `%fs:(8)`, and because FS and GS base point at the same address, that is the same memory as `%gs:8`. Reached through `mycpu()` and `myproc()`, which compile to a single `movq %gs:16` / `%gs:24` with no cached base. > **`__thread` is wrong for per-CPU kernel data**, and this is the single most > expensive lesson in the tree. `__thread` means per *thread*, and the compiler > is entitled to read the thread pointer once and keep it in a register across > calls. But `%fs` here is per *CPU*, and a kernel thread changes CPU at every > `sched()`. So the assumption is false for exactly the variables that most need > to be right, and the failure is a null dereference deep in the scheduler on any > machine with more than one core. `include/proc.h` carries the disassembly that > proves it. ## Processes `struct proc` in [`include/proc.h`](../../include/proc.h). A fixed table of `NPROC` entries, allocated by linear scan. **The field order at the top of the struct is fixed by assembly.** `trapasm.S` hard-codes `fs_base` at offset `0x18`, and there is a `_Static_assert` in `proc.c` that fails the build if it moves. Anything new goes at the end. Each process carries: page directory and size, kernel stack, open file table, cwd inode *and* cwd path string, signal handlers and masks, VMA table for `mmap`, POSIX identity (pgid, sid, and all six of real/effective/saved uid and gid), and the Linux-ABI syscall map. ### Slot reuse The table recycles slots, and a recycled slot that inherits the previous occupant's state is a whole class of bug that has bitten here twice. `allocproc()` therefore explicitly clears the signal fields, the exit status, the kill signal, and the identity fields. The failure mode when it did not: `deliver_signals()` jumping to a totally unrelated earlier process's handler address on this process's very first syscall return. ### Kernel threads `kthread_create(fn, name)` makes a process that runs `fn` in kernel mode and never returns to userland. It gets a page directory with only the kernel mapped, no files, no cwd, no parent — it is not in the process tree, cannot be waited for, and must never exit. The point is that it is a *process* and not a timer callback: it shares the scheduler's ordinary rotation, so it can hold sleeplocks and block on the disk exactly as a syscall would. The network thread is the current user. ## Scheduling Round-robin over the process table, one scheduler per CPU, each with its own context. `swtch()` saves callee-saved registers and switches stacks; `sched()` returns to the scheduler and asserts four invariants on the way — holding `ptable.lock`, `ncli == 1`, not `RUNNING`, interrupts off. When a full pass finds nothing runnable the CPU halts rather than spinning. **Locking is a single `ptable.lock` over the whole table.** Not per-process: a scheduling operation touches two processes (the one leaving and the one arriving) and two-object locking needs an ordering rule that every path obeys. The same reasoning produced a single lock over the socket table. ### sleep / wakeup xv6's discipline, unchanged because it is correct: `sleep(chan, lk)` atomically releases `lk` and blocks; `wakeup(chan)` makes every sleeper on that channel runnable. The atomicity is what matters — a sleeper that tests its condition under one lock while the waker sets it under another has no interlock and sleeps through the wakeup meant for it. ## Traps and syscalls Interrupts and exceptions enter through `vectors.S` → `trapasm.S` → `trap()`. Syscalls enter through `SYSCALL`/`SYSRET` and `syscall_entry`, which is a separate and faster path. Every entry from user mode checks a **canary at the bottom of the kernel stack page**. A deep or nested trap that overflows the 4 KB stack clobbers that word first, so the failure is reported as `kstack overflow` at the point it happened rather than as arbitrary corruption later. One configuration detail with an expensive history: `MSR_STAR`'s `SYSRET` selector field must hold the **compat 32-bit** code selector, because `SYSRETQ` computes the returned `cs`/`ss` as `STAR[63:48]+16`/`+8`. Putting the real 64-bit user selector there makes every `sysretq` load an invalid pair — which QEMU's TCG does not validate, so it stayed hidden until a hardware interrupt forced a strictly-validated `IRETQ`. ## Signals Real delivery, not stubs. `sys_rt_sigaction()` records the handler; `deliver_signals()` runs in the tail of `syscall()`, just before returning to user mode, and if a pending unblocked signal has a handler it rewrites the trapframe to enter the handler instead of resuming — stashing the original in `proc->saved_sigtf`. `sys_rt_sigreturn()` puts it back. The return address is **not** `sa_restorer`. `SA_RESTORER` is not defined in this musl build, so `sigaction()` never fills one in, and pushing 0 as a return address crashes with `rip = 0` the moment the handler returns. Instead a 7-byte trampoline (`mov eax,15; syscall`) is injected onto the user's own stack below `rsp`, clear of the SysV red zone — the standard technique when the C library supplies no restorer. Signals without a handler fall back to a default action: termination for the fatal set, dropped otherwise. ### PID 1 does not die from a signal it never asked to handle `init_ignores()` discards any signal whose disposition is the default action when the target is PID 1 — **including `SIGKILL`**. This is what POSIX requires and what Linux does in `sig_task_ignored()`, and without it the system has a single keystroke that kills it: Ctrl-C signals the foreground process group, init is pgid 1, and any moment where the terminal's foreground group is still 1 lands SIGINT on init. It has no handler, so `exit()` runs, and `exit()` panics outright on initproc. Spamming Ctrl-C at a frozen program is exactly the input that finds that window. ## Process groups and sessions `setsid`, `setpgid`, `getpgid`, `getsid`, and signal delivery to a whole group. Job control is defined entirely in terms of these — a shell puts each pipeline in its own process group and signals the group, never individual pids — and mksh's job control could not make progress until they existed. ## What userspace gets `fork`, `exec`, `wait`/`waitpid`/`wait4` with `WNOHANG` and real pid selection, pipes, signals, `mmap` (anonymous and file-backed), POSIX shared memory, Unix domain sockets, `poll`/`select`/`epoll`, `eventfd`, `timerfd`, `signalfd`. `mmap` uses a bump allocator (`proc->mmap_next`, starting at `0x40000000`) so every mapping gets its own disjoint, exactly-sized range. It previously assigned a fixed 8 MB window per VMA slot regardless of the requested length, so a single allocation over 8 MB mapped straight into a neighbouring live mapping. --- ## What it does not do - **No slab allocator.** One page at a time, and every subsystem's fixed-size table is a consequence. This is the single change that would most improve the rest of the kernel. - **No demand paging and no page cache.** `mmap` of a file *reads the whole thing in* at map time. Writable `MAP_SHARED` of a file is therefore refused outright rather than faked — see [../plan/foundations.md](../plan/foundations.md). - **No `sendmsg`/`recvmsg`, so no `SCM_RIGHTS`.** File descriptors cannot be passed between processes. - **No dynamic linking.** `exec` handles `ET_EXEC` only: no `PT_INTERP`, no `ET_DYN`, no shared libraries, no `dlopen`. - **No threads.** `clone` exists but does not share an address space; `gettid` returns the pid. - **Permissions are tracked but not enforced.** uid/gid are maintained honestly so `getuid()`/`setuid()` stop lying, and nothing consults them on access. - **No swap**, and no memory pressure handling of any kind. `kalloc` failing is reported up as `-ENOMEM` and that is the whole policy. - **Nested signal delivery is not supported.** `in_signal` guards against re-entering a handler while one is running. - **Scheduling is round-robin with no priorities**, no fairness accounting, and no CPU affinity despite `sched_setaffinity` existing.