SchOS/docs/architecture/kernel.md

SchIFT — the kernel

Long mode, 4-level paging, preemptive multitasking, and more than one processor. What follows is the parts of the kernel that are not a filesystem, a driver or a protocol stack: memory, processes, scheduling, traps and signals.

---

Address space

One page table per process; the kernel is mapped into all of them at KERNBASE = 0xFFFF800000000000, so a trap does not switch CR3.

kvmalloc() maps as many 1 GB pages of physical memory as the machine has RAM (up to PHYSTOP_MAX, 256 GB), always at least the low four. The 3–4 GB slot stays uncached because that is where the PCI hole is; everything above 4 GB is RAM, so those are cached, and a device BAR up there still gets its own uncached window from ioremap(), which takes the entries the direct map did not. On a CPU without 1 GB pages (Intel's desktop parts before Haswell), each gigabyte is a page directory of 2 MB pages instead — the same reach, one extra page of table per gigabyte. A direct map with a fixed offset makes P2V/V2P arithmetic rather than a lookup.

How much RAM there is comes from the firmware's E820/UEFI map, as a list of usable regions, not one run ending at a top address (OVMF drops an ACPI NVS region at 8 MB, which used to hide everything above it). The kernel itself is linked at 16 MB, clear of what firmware keeps low. kalloc holds never-used memory as ranges, so a large machine costs nothing until its memory is used.

kalloc() hands out one 4 KB page at a time from a free list. Objects smaller than a page come from the slab caches below; objects larger than one do not have a home yet, so the biggest tables (NPROC processes, NSOCK sockets, NDEVICE devices, NSHM segments) are still fixed-size arrays, which is why their limits are compile-time constants rather than a design decision about each one.

The free list carries a canary, checked at the top of both kalloc() and kfree(), so a wild write into it panics at the point it is touched rather than handing back a bad pointer that fails somewhere else entirely.

DMA

kernel/core/dma.c owns an arena carved off the top of physical memory before kalloc is given the rest. Devices need physically contiguous memory; kalloc cannot promise it, so the region has to be reserved before there is any competition for it.

Per-CPU state

The per-CPU block's base is in both FS_BASE and GS_BASE, set by seginit():

  +0   self-pointer     required by the x86-64 TLS ABI (%fs:0)
  +8   syscall scratch  trapasm.S parks %rsp here across the stack switch
  +16  struct cpu *
  +24  struct proc *
  +32  syscall rax      the three registers wrmsr destroys, parked across
  +40  syscall rcx      the FS_BASE swap at the top of syscall_entry
  +48  syscall rdx

These offsets are shared with assembly and must not be reordered. Offset 8 is not free: syscall_entry uses %fs:(8), and because FS and GS base point at the same address, that is the same memory as %gs:8. A _Static_assert in vm.c pins all seven, because assembly spells them as literals and a slot that moves leaves a register quietly holding somebody else's value.

Reached through mycpu() and myproc(), which compile to a single movq %gs:16 / %gs:24 with no cached base.

The three words at +32, and what they cost before they were here

syscall_entry has to swap FS_BASE from the user's TLS to the kernel's before it can touch any %fs: data, and wrmsr destroys rax, rcx and rdx. Those three have to be parked somewhere first — not on the user stack, which is untrusted and has not been switched away from yet.

They were parked in three globals, _sc_rax, _sc_rcx and _sc_rdx, under a comment reading:

Single-CPU scratch for syscall_entry FS swap (safe while IF cleared by SYSCALL/SFMASK)

IF being clear answers for an interrupt arriving on this CPU. It says nothing about a second CPU executing the same three stores at the same instant, and with more than one processor entering syscalls that is not a rare interleaving — it is the ordinary one.

%rcx is the user return address. SYSCALL puts the user's rip there and SYSRET takes it back from there. So the CPU that lost the race returned its process into another process's instruction stream, and %rax — the syscall number on the way in, the return value on the way out — was swapped with it.

The symptom was one to four processes per boot dying at rip 0x0 (an instruction fetch at zero, which a process does not branch to; it is handed a zero and follows it) or dereferencing 0xffffffffffffffea, which is -EINVAL: a syscall return value used as a pointer. It hit shells and schinitd alike, and it had been written off as a mysterious "zsh sometimes jumps into .data".

Measured, at four CPUs, with the same pristine disk image each run:

	boots	faults
three globals	10	10, in five of the ten boots
three per-CPU words	18	0

This is the second bug of exactly this shape in this file. The first was kernel_fs_base, a single global that every CPU wrote during its own seginit(), so the last writer won and every CPU restored one CPU's block. RESTORE_KERNEL_FS reads %gs:0 now for that reason. The lesson generalises and is worth stating plainly: on the entry path, a global is a bug. There is one per-CPU block and everything that changes per CPU belongs in it.

__thread is wrong for per-CPU kernel data, and this is the single most expensive lesson in the tree. __thread means per thread, and the compiler is entitled to read the thread pointer once and keep it in a register across calls. But %fs here is per CPU, and a kernel thread changes CPU at every sched(). So the assumption is false for exactly the variables that most need to be right, and the failure is a null dereference deep in the scheduler on any machine with more than one core. include/core/proc.h carries the disassembly that proves it.

Memory

kalloc() hands out one 4096-byte page and kfree() takes it back. That is the whole page allocator: a free list threaded through the free pages themselves, no buddy system, no zones, no page-frame database.

Small objects

kernel/core/kmem.c, include/core/kmem.h. A slab is one page from kalloc() with a header at the front and objects packed after it; free objects hold the free list in their own first eight bytes.

That last detail is the whole design. An object needs no header and freeing needs no search: the page base of any object is its slab, and the slab names its cache. It is also why an object cannot exceed a page.

kmem_cache_create/alloc/free	a named cache of one object size, for a subsystem with one kind of thing
kmalloc / kmfree	power-of-two classes from 16 to 2048, for code with no single type to build a cache around

Each cache keeps partial and full lists; a slab that empties completely goes straight back to kalloc() rather than being kept, so /proc/slab never overstates what the kernel is holding.

make kmem-test asserts the bookkeeping on the build machine — the same arrangement as schfs-objtest, and for the same reason: slab lists, free-list threading and whether pages actually come back are all things that can be wrong without a kernel to be wrong in.

Shared memory objects

shmget/shmat/shmdt are System V shared memory: segments of kalloc'd pages, reference counted, mapped through a VMA with no_free set so that freevm() does not hand another process's pages to kfree.

memfd_create(2) is the same machinery reached by descriptor instead of by key. That difference is the whole point and the reason it exists on Linux at all: two processes share memory by passing the fd over a socket, so neither has to agree on a path or race to invent a key. It is how a display server and its client share a buffer, and it is why descriptor passing (SCM_RIGHTS) had to be built first.

memfd_create(name, flags)	a new segment of size 0, and a descriptor owning one reference
ftruncate(fd, n)	gives it a size — growing allocates zeroed pages, shrinking frees from the end
mmap(…, MAP_SHARED, fd, off)	maps the segment's existing pages, so both mappers get the same physical memory
read/write/lseek/fstat	work, so it is a file and not a token only mmap understands

Three things are refused rather than faked, and each refusal is the honest version of something this kernel cannot do:

• MAP_PRIVATE on a memfd. Every caller is sharing on purpose; a private mapping of an anonymous object is an expensive MAP_ANONYMOUS.
• Mapping past the end. Linux lets it through and faults the pages in. There is no fault handler here that could, so the range is rejected.
• Shrinking while mapped. Linux truncates the pages out from under the mapper. Here a freed page would simply stay mapped to memory somebody else now owns.

Writable MAP_SHARED of a real file is still refused. That needs a page cache, it is a separate project, and nothing that runs today asks for it — an anonymous object has no such contract to break, which is exactly why memfd is the narrow answer.

Demand paging

Anonymous memory can be recorded without being backed: the page appears, zeroed, the first time it is touched — by the program, through trap(), or by the kernel on its behalf, through uva2ka(). mmap does this for mappings of 1 MB or more, for MAP_NORESERVE, and for PROT_NONE. The reason is WebKit, which reserves 4 GB of address space at start for its JIT and heaps and touches a sliver of it; backing all of it at mmap time failed with ENOMEM and the browser aborted. Go, Java and Firefox reserve the same way.

vm_fault() takes a spinlock and nothing that sleeps, so uva2ka() may call it from anywhere; the lock keeps two threads of one process from both mapping the same page. fork and munmap skip the holes, so a page nobody touched is one they never see, and a child that touches it reads zeroes — what it would have read in the parent.

File-backed mappings are not lazy: mmap of a file still reads the range in at map time.

TLB shootdowns

Unmapping from an address space that threads on other processors are running in interrupts those processors and waits until each has reloaded CR3 (tlb_shootdown() in vm.c; the vector is one of the MSI lines, which trap() already dispatches with an EOI). Without it, when one thread unmapped a range and mmap handed the same addresses straight back, a thread on another CPU went on reading and writing the old pages through its TLB: stores vanished and loads were stale. musl's allocator noticed first — a group's metadata no longer pointing back at it — and Epiphany, forty threads on four processors, died at a different address on every run.

Only one shootdown runs at a time, and it is taken without disabling interrupts: two processors each waiting with interrupts off for the other to answer would wait for ever. On a single processor, or if no vector could be allocated, it does nothing (and says so at boot).

What the fixed tables actually cost

The standing claim was that every fixed-size table in the kernel exists to avoid asking kalloc for something smaller than a page. Measured, that is only partly true:

	size	table
struct proc	1984	NPROC 128	254 KB in all; the address space, descriptors and signal handlers live in their own objects now, shared by threads
struct mm	8232	one per process, 128	1 MB in all, almost all of it the 256-entry VMA table
struct socket	about 4900	NSOCK 128	larger than a page
struct device	120	NDEVICE 128	15 KB, never hit
struct buf	200	NBUF 256 to start	grows from a slab, data from kalloc, up to RAM/32
struct inode	about 460	none	grows from a slab; NINODE 200 is only how many are made up front
struct file	about 88	none	grows from a slab; NFILE 512 is only the static pool

(Sizes from nm -S on a build of the tree, not from a header comment.)

struct proc shrank by a factor of six when threads arrived, because the page tables, VMAs, descriptor table and signal dispositions moved out of it into objects a thread group shares. struct socket is still too large for a page-backed slab; freeing it needs a contiguous multi-page allocator, which is a separate thing and is not written. The buffer header shrank to 200 bytes when its 4 KB of data moved out to a page of its own, which is what let the cache grow at all.

Processes

struct proc in include/core/proc.h. A fixed table of NPROC entries, allocated by linear scan. A thread is a struct proc too.

The field order at the top of the struct is fixed by assembly. trapasm.S hard-codes fs_base at offset 0x18, and there is a _Static_assert in proc.c that fails the build if it moves. Anything new goes at the end.

Each process carries: a kernel stack, cwd inode and cwd path string, signal masks, POSIX identity (pgid, sid, and all six of real/effective/saved uid and gid), the Linux-ABI syscall map, and its x87/SSE registers (saved on a switch; the kernel itself stays off them). Three things it points to rather than owns, because a thread group shares them:

struct mm	the address space: page tables, heap top, the VMA table, which file each span came from, and a lock so one thread at a time changes the mappings
struct files	the descriptor table and its FD_CLOEXEC flags
struct sighand	signal dispositions — what sigaction() sets

Each is reference counted. clone() shares what its flags say and fork() copies all three.

Threads

clone() takes pthread_create's flags (CLONE_VM, FILES, SIGHAND, THREAD, SETTLS, PARENT_SETTID, CHILD_CLEARTID) and posix_spawn's (VM, VFORK, a stack). What follows from that:

• futex — WAIT, WAIT_BITSET with timeouts, WAKE, REQUEUE, CMP_REQUEUE, keyed by the word's kernel address so processes that map the same page meet. set_tid_address keeps the word that exit zeroes and wakes, which is what pthread_join waits on.
• Exit. exit ends the calling thread; exit_group, a fatal signal, or the leader exiting ends them all. reapd frees exited threads, and the leader is reaped by its parent once the rest are gone. getpid is the group, gettid the thread.
• Signals. Stops and resumes reach every thread; other process-directed signals go to the leader.
• exec kills the other threads and unshares the dispositions.
• /proc lists processes, task/ lists threads; comm, Threads: and Tgid are there.

The first program to need all of this was GTK's, then WebKit's.

Slot reuse

The table recycles slots, and a recycled slot that inherits the previous occupant's state is a whole class of bug that has bitten here twice. allocproc() therefore explicitly clears the signal fields, the exit status, the kill signal, and the identity fields. The failure mode when it did not: deliver_signals() jumping to a totally unrelated earlier process's handler address on this process's very first syscall return.

Kernel threads

kthread_create(fn, name) makes a process that runs fn in kernel mode and never returns to userland. It gets a page directory with only the kernel mapped, no files, no cwd, no parent — it is not in the process tree, cannot be waited for, and must never exit.

The point is that it is a process and not a timer callback: it shares the scheduler's ordinary rotation, so it can hold sleeplocks and block on the disk exactly as a syscall would. The network thread is the current user.

Scheduling

Fair share by virtual runtime, one scheduler per CPU, each with its own context — the rule, the guards and nice are in scheduler.md. swtch() saves callee-saved registers and switches stacks; sched() returns to the scheduler and asserts four invariants on the way — holding ptable.lock, ncli == 1, not RUNNING, interrupts off.

When nothing is runnable the CPU goes idle rather than spinning: into ACPI's _CST C2 state (CC6 on Zen), by port read or MWAIT, where the APIC timer keeps running through it, and otherwise a plain hlt.

Locking is a single ptable.lock over the whole table. Not per-process: a scheduling operation touches two processes (the one leaving and the one arriving) and two-object locking needs an ordering rule that every path obeys. The same reasoning produced a single lock over the socket table.

sleep / wakeup

xv6's discipline, unchanged because it is correct: sleep(chan, lk) atomically releases lk and blocks; wakeup(chan) makes every sleeper on that channel runnable. The atomicity is what matters — a sleeper that tests its condition under one lock while the waker sets it under another has no interlock and sleeps through the wakeup meant for it.

Traps and syscalls

Interrupts and exceptions enter through vectors.S → trapasm.S → trap(). Syscalls enter through SYSCALL/SYSRET and syscall_entry, which is a separate and faster path.

Every entry from user mode checks a canary at the bottom of the kernel stack page. A deep or nested trap that overflows the 4 KB stack clobbers that word first, so the failure is reported as kstack overflow at the point it happened rather than as arbitrary corruption later.

One configuration detail with an expensive history: MSR_STAR's SYSRET selector field must hold the compat 32-bit code selector, because SYSRETQ computes the returned cs/ss as STAR[63:48]+16/+8. Putting the real 64-bit user selector there makes every sysretq load an invalid pair — which QEMU's TCG does not validate, so it stayed hidden until a hardware interrupt forced a strictly-validated IRETQ.

Signals

Real delivery, not stubs. sys_rt_sigaction() records the handler; deliver_signals() runs in the tail of syscall(), just before returning to user mode, and if a pending unblocked signal has a handler it rewrites the trapframe to enter the handler instead of resuming — stashing the original in proc->saved_sigtf. sys_rt_sigreturn() puts it back.

The return address is not sa_restorer. SA_RESTORER is not defined in this musl build, so sigaction() never fills one in, and pushing 0 as a return address crashes with rip = 0 the moment the handler returns. Instead a 7-byte trampoline (mov eax,15; syscall) is injected onto the user's own stack below rsp, clear of the SysV red zone — the standard technique when the C library supplies no restorer.

Signals without a handler fall back to a default action: termination for the fatal set, dropped otherwise.

PID 1 does not die from a signal it never asked to handle

init_ignores() discards any signal whose disposition is the default action when the target is PID 1 — including SIGKILL. This is what POSIX requires and what Linux does in sig_task_ignored(), and without it the system has a single keystroke that kills it: Ctrl-C signals the foreground process group, init is pgid 1, and any moment where the terminal's foreground group is still 1 lands SIGINT on init. It has no handler, so exit() runs, and exit() panics outright on initproc.

Spamming Ctrl-C at a frozen program is exactly the input that finds that window.

Process groups and sessions

setsid, setpgid, getpgid, getsid, and signal delivery to a whole group. Job control is defined entirely in terms of these — a shell puts each pipeline in its own process group and signals the group, never individual pids — and mksh's job control could not make progress until they existed.

What userspace gets

fork, clone (threads), exec, wait/waitpid/wait4 with WNOHANG and real pid selection, pipes, signals, mmap (anonymous and file-backed), POSIX shared memory, memfd, Unix domain sockets, poll/select/epoll, eventfd, timerfd, signalfd, inotify, futex.

Dynamic linking

exec handles ET_EXEC and ET_DYN. A dynamically linked program names an interpreter in PT_INTERP — on musl, /lib/ld-musl-x86_64.so.1, which is libc under another name — and the kernel's whole part is to load that as well and start it instead of the program. The interpreter finds the program through the auxiliary vector (AT_PHDR, AT_ENTRY), maps libc and whatever else it needs with mmap(), relocates, and jumps to AT_ENTRY itself. A position-independent program goes where a fixed one would (0x400000); the interpreter goes at the start of the mmap area (0x40000000) and mmap carries on above it, so the two never meet. This is what lets Alpine's binaries run unmodified, and dlopen works, so Xorg loads its drivers as modules.

mmap uses a bump allocator (proc->mmap_next, starting at 0x40000000) so every mapping gets its own disjoint, exactly-sized range. It previously assigned a fixed 8 MB window per VMA slot regardless of the requested length, so a single allocation over 8 MB mapped straight into a neighbouring live mapping.

---

What it does not do

• ru_stime is effectively always zero. CPU time is sampled at the timer tick and split on the interrupted privilege level, but MSR_SFMASK includes FL_IF, so SYSCALL clears interrupts and no tick can land while a process is inside a syscall. User time is real and discriminates properly — a busy loop reports 97%, sleep 2 reports 0% — but kernel time needs TSC deltas closed and reopened around every sched(), which is scheduler accounting rather than sampling, and is not written.
• struct socket is too big for a slab, and every subsystem's fixed-size table (NPROC, NSOCK, NDEVICE, NSHM) is still a compile-time constant. Slabs exist for the objects that fit in a page.
• No page cache. mmap of a file reads the range in at map time, and writable MAP_SHARED of a file is refused outright rather than faked. Demand paging covers anonymous memory only.
• Root is euid == 0 and nothing finer. No capabilities; file access, kill, exec and the privileged calls are enforced as users.md says.
• mprotect is accepted and not enforced (sys_mprotect returns 0). RELRO is not read-only, a PROT_NONE guard does not fault, and nothing has NX (EFER.NXE is never set), so every mapping is executable.
• poll, select and epoll wait a tick at a time (10 ms at HZ=100) unless something that can make a descriptor ready calls poll_kick() — which the pipe, socket and event paths do. busybox's wget polls before every read and downloaded at 85 KB/s, a segment per tick, until the network path started kicking.
• No swap, and no memory pressure handling of any kind. kalloc failing is reported up as -ENOMEM and that is the whole policy.
• Nested signal delivery is not supported. in_signal guards against re-entering a handler while one is running.
• One global run queue. Fair share and nice exist; per-CPU run queues, work stealing and CPU affinity do not, and sched_setaffinity is accepted and ignored.
• No IOMMU, so DMA-capable devices (and the driver processes that program them) are trusted.
• 256 GB is the most RAM the kernel uses; the rest is ignored, with a line saying so.