nexoniarz/SchOS
nexoniarz/SchOS
Watch
1
0
Fork
You've already forked SchOS
 0
Code Issues Pull requests Projects Releases 1 Packages Wiki Activity Actions

docs: architecture describes what is built now

Browse source 
kernel (threads, demand paging, TLB shootdowns, dynamic linking, RAM direct map), syscalls, filesystem (/proc, /sys,
/dev), terminals (12, VT handshake, ptys), networking (r8169 through LDLS), LDLS (module sets; Linux code may ship),
devices (what stays in the kernel; hybrid.md's lasting content), boot (the loader's placement of the kernel),
graphics (fbdev, GMMOS, QR 1-40), audio (ALSA over OSS), debugging, usb. New: packages.md (the apk root) and
desktop.md (Xorg on the framebuffer).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EVRhxi9zpdW2uDUN4FVcao
This commit is contained in:
Nexoniarz 2026-10-26 07:57:25 +01:00
commit 2556a955b3
15 changed files with 921 additions and 341 deletions
Download patch file Download diff file Expand all files Collapse all files

209
docs/architecture/kernel.md
View file

@ -11,18 +11,27 @@ driver or a protocol stack: memory, processes, scheduling, traps and signals.
One page table per process; the kernel is mapped into all of them at
`KERNBASE = 0xFFFF800000000000`, so a trap does not switch `CR3`.
`kvmalloc()` maps physical 0–4 GB as four 1 GB pages, with the top one marked
uncached — that is where PCI BARs and the framebuffer live. This is why 3 GB is
the most memory the kernel can treat as RAM, and it is a deliberate simplicity:
a direct map with a fixed offset makes `P2V`/`V2P` arithmetic rather than a
lookup.
`kvmalloc()` maps as many 1 GB pages of physical memory as the machine has RAM
(up to `PHYSTOP_MAX`, 256 GB), always at least the low four. The 3–4 GB slot stays
uncached because that is where the PCI hole is; everything above 4 GB is RAM, so
those are cached, and a device BAR up there still gets its own uncached window
from `ioremap()`, which takes the entries the direct map did not. On a CPU
without 1 GB pages (Intel's desktop parts before Haswell), each gigabyte is a
page directory of 2 MB pages instead — the same reach, one extra page of table
per gigabyte. A direct map with a fixed offset makes `P2V`/`V2P` arithmetic
rather than a lookup.
`kalloc()` hands out one 4 KB page at a time from a free list. **There is no
slab or pool allocator**, and the consequence is visible throughout the tree:
every subsystem that needs many small objects has invented its own fixed-size
table — `NPROC` processes, `NSOCK` sockets, `NDEVICE` devices, `NSHM` segments.
That is why the limits are compile-time constants rather than a design decision
about each one.
How much RAM there is comes from the firmware's E820/UEFI map, as a *list* of
usable regions, not one run ending at a top address (OVMF drops an ACPI NVS
region at 8 MB, which used to hide everything above it). The kernel itself is
linked at 16 MB, clear of what firmware keeps low. `kalloc` holds never-used
memory as ranges, so a large machine costs nothing until its memory is used.
`kalloc()` hands out one 4 KB page at a time from a free list. Objects smaller
than a page come from the slab caches below; objects larger than one do not have
a home yet, so the biggest tables (`NPROC` processes, `NSOCK` sockets, `NDEVICE`
devices, `NSHM` segments) are still fixed-size arrays, which is why their limits
are compile-time constants rather than a design decision about each one.
The free list carries a canary, checked at the top of both `kalloc()` and
`kfree()`, so a wild write into it panics at the point it is touched rather than
@ -150,8 +159,8 @@ pages, reference counted, mapped through a VMA with `no_free` set so that
`memfd_create(2)` is the same machinery reached by **descriptor** instead of by
key. That difference is the whole point and the reason it exists on Linux at
all: two processes share memory by passing the fd over a socket, so neither has
to agree on a path or race to invent a key. It is what a Wayland buffer crosses
on, and it is why descriptor passing (`SCM_RIGHTS`) had to be built first.
to agree on a path or race to invent a key. It is how a display server and its client
share a buffer, and it is why descriptor passing (`SCM_RIGHTS`) had to be built first.
| | |
|---|---|
@ -171,10 +180,46 @@ version of something this kernel cannot do:
mapper. Here a freed page would simply stay mapped to memory somebody else
now owns.
Writable `MAP_SHARED` of a **real file** is still `ENODEV`. That needs a page
cache, it is a separate project, and nothing in 1.0 asks for it — an anonymous
object has no such contract to break, which is exactly why `memfd` is the
narrow answer.
Writable `MAP_SHARED` of a **real file** is still refused. That needs a page
cache, it is a separate project, and nothing that runs today asks for it — an
anonymous object has no such contract to break, which is exactly why `memfd` is
the narrow answer.
### Demand paging
Anonymous memory can be recorded without being backed: the page appears,
zeroed, the first time it is touched — by the program, through `trap()`, or by
the kernel on its behalf, through `uva2ka()`. `mmap` does this for mappings of
1 MB or more, for `MAP_NORESERVE`, and for `PROT_NONE`. The reason is WebKit,
which reserves 4 GB of address space at start for its JIT and heaps and touches
a sliver of it; backing all of it at `mmap` time failed with `ENOMEM` and the
browser aborted. Go, Java and Firefox reserve the same way.
`vm_fault()` takes a spinlock and nothing that sleeps, so `uva2ka()` may call it
from anywhere; the lock keeps two threads of one process from both mapping the
same page. `fork` and `munmap` skip the holes, so a page nobody touched is one
they never see, and a child that touches it reads zeroes — what it would have
read in the parent.
File-backed mappings are **not** lazy: `mmap` of a file still reads the range in
at map time.
### TLB shootdowns
Unmapping from an address space that threads on other processors are running in
interrupts those processors and waits until each has reloaded `CR3`
(`tlb_shootdown()` in `vm.c`; the vector is one of the MSI lines, which `trap()`
already dispatches with an EOI). Without it, when one thread unmapped a range and
`mmap` handed the same addresses straight back, a thread on another CPU went on
reading and writing the *old* pages through its TLB: stores vanished and loads
were stale. musl's allocator noticed first — a group's metadata no longer
pointing back at it — and Epiphany, forty threads on four processors, died at a
different address on every run.
Only one shootdown runs at a time, and it is taken without disabling interrupts:
two processors each waiting with interrupts off for the other to answer would
wait for ever. On a single processor, or if no vector could be allocated, it
does nothing (and says so at boot).
### What the fixed tables actually cost
@ -184,31 +229,69 @@ partly true:
| | size | table | |
|---|---|---|---|
| `struct proc` | 11744 | `NPROC` 128 | larger than a page; 1 KB of it is FPU state |
| `struct socket` | 4936 | `NSOCK` 128 | larger than a page |
| `struct proc` | 1984 | `NPROC` 128 | 254 KB in all; the address space, descriptors and signal handlers live in their own objects now, shared by threads |
| `struct mm` | 8232 | one per process, 128 | 1 MB in all, almost all of it the 256-entry VMA table |
| `struct socket` | about 4900 | `NSOCK` 128 | larger than a page |
| `struct device` | 120 | `NDEVICE` 128 | 15 KB, never hit |
| `struct buf` | 200 | `NBUF` 256 to start | grows from a slab, data from `kalloc`, up to RAM/32 |
| `struct inode` | 296 | none | grows from a slab; `NINODE` 200 is only how many are made up front |
| `struct file` | 64 | none | grows from a slab; `NFILE` 512 is only the static pool |
| `struct inode` | about 460 | none | grows from a slab; `NINODE` 200 is only how many are made up front |
| `struct file` | about 88 | none | grows from a slab; `NFILE` 512 is only the static pool |
The two largest cannot go in a page-backed slab at all. Freeing them needs a
contiguous multi-page allocator, which is a separate thing and is not written.
The buffer header shrank to 200 bytes when its 4 KB of data moved out to a page
of its own, which is what let the cache grow at all.
(Sizes from `nm -S` on a build of the tree, not from a header comment.)
`struct proc` shrank by a factor of six when threads arrived, because the page
tables, VMAs, descriptor table and signal dispositions moved out of it into
objects a thread group shares. `struct socket` is still too large for a
page-backed slab; freeing it needs a contiguous multi-page allocator, which is a
separate thing and is not written. The buffer header shrank to 200 bytes when its
4 KB of data moved out to a page of its own, which is what let the cache grow at
all.
## Processes
`struct proc` in [`include/core/proc.h`](../../include/proc.h). A fixed table of
`NPROC` entries, allocated by linear scan.
`struct proc` in [`include/core/proc.h`](../../include/core/proc.h). A fixed table of
`NPROC` entries, allocated by linear scan. A thread is a `struct proc` too.
**The field order at the top of the struct is fixed by assembly.** `trapasm.S`
hard-codes `fs_base` at offset `0x18`, and there is a `_Static_assert` in
`proc.c` that fails the build if it moves. Anything new goes at the end.
Each process carries: page directory and size, kernel stack, open file table,
cwd inode *and* cwd path string, signal handlers and masks, VMA table for
`mmap`, POSIX identity (pgid, sid, and all six of real/effective/saved uid and
gid), and the Linux-ABI syscall map.
Each process carries: a kernel stack, cwd inode *and* cwd path string, signal
masks, POSIX identity (pgid, sid, and all six of real/effective/saved uid and
gid), the Linux-ABI syscall map, and its x87/SSE registers (saved on a switch;
the kernel itself stays off them). Three things it *points to* rather than
owns, because a thread group shares them:
| | |
|---|---|
| `struct mm` | the address space: page tables, heap top, the VMA table, which file each span came from, and a lock so one thread at a time changes the mappings |
| `struct files` | the descriptor table and its `FD_CLOEXEC` flags |
| `struct sighand` | signal dispositions — what `sigaction()` sets |
Each is reference counted. `clone()` shares what its flags say and `fork()`
copies all three.
### Threads
`clone()` takes `pthread_create`'s flags (`CLONE_VM`, `FILES`, `SIGHAND`,
`THREAD`, `SETTLS`, `PARENT_SETTID`, `CHILD_CLEARTID`) and `posix_spawn`'s
(`VM`, `VFORK`, a stack). What follows from that:
- **futex** — `WAIT`, `WAIT_BITSET` with timeouts, `WAKE`, `REQUEUE`,
`CMP_REQUEUE`, keyed by the word's *kernel* address so processes that map the
same page meet. `set_tid_address` keeps the word that exit zeroes and wakes,
which is what `pthread_join` waits on.
- **Exit.** `exit` ends the calling thread; `exit_group`, a fatal signal, or the
leader exiting ends them all. `reapd` frees exited threads, and the leader is
reaped by its parent once the rest are gone. `getpid` is the group, `gettid`
the thread.
- **Signals.** Stops and resumes reach every thread; other process-directed
signals go to the leader.
- **`exec`** kills the other threads and unshares the dispositions.
- **`/proc`** lists processes, `task/` lists threads; `comm`, `Threads:` and
`Tgid` are there.
The first program to need all of this was GTK's, then WebKit's.
### Slot reuse
@ -232,12 +315,15 @@ exactly as a syscall would. The network thread is the current user.
## Scheduling
Round-robin over the process table, one scheduler per CPU, each with its own
context. `swtch()` saves callee-saved registers and switches stacks; `sched()`
returns to the scheduler and asserts four invariants on the way — holding
`ptable.lock`, `ncli == 1`, not `RUNNING`, interrupts off.
Fair share by virtual runtime, one scheduler per CPU, each with its own context
— the rule, the guards and `nice` are in [scheduler.md](scheduler.md).
`swtch()` saves callee-saved registers and switches stacks; `sched()` returns to
the scheduler and asserts four invariants on the way — holding `ptable.lock`,
`ncli == 1`, not `RUNNING`, interrupts off.
When a full pass finds nothing runnable the CPU halts rather than spinning.
When nothing is runnable the CPU goes idle rather than spinning: into ACPI's
`_CST` C2 state (CC6 on Zen), by port read or `MWAIT`, where the APIC timer keeps
running through it, and otherwise a plain `hlt`.
**Locking is a single `ptable.lock` over the whole table.** Not per-process: a
scheduling operation touches two processes (the one leaving and the one
@ -309,9 +395,23 @@ mksh's job control could not make progress until they existed.
## What userspace gets
`fork`, `exec`, `wait`/`waitpid`/`wait4` with `WNOHANG` and real pid selection,
pipes, signals, `mmap` (anonymous and file-backed), POSIX shared memory, Unix
domain sockets, `poll`/`select`/`epoll`, `eventfd`, `timerfd`, `signalfd`.
`fork`, `clone` (threads), `exec`, `wait`/`waitpid`/`wait4` with `WNOHANG` and
real pid selection, pipes, signals, `mmap` (anonymous and file-backed), POSIX
shared memory, `memfd`, Unix domain sockets, `poll`/`select`/`epoll`, `eventfd`,
`timerfd`, `signalfd`, `inotify`, `futex`.
### Dynamic linking
`exec` handles `ET_EXEC` and `ET_DYN`. A dynamically linked program names an
interpreter in `PT_INTERP` — on musl, `/lib/ld-musl-x86_64.so.1`, which is libc
under another name — and the kernel's whole part is to load that as well and
start it instead of the program. The interpreter finds the program through the
auxiliary vector (`AT_PHDR`, `AT_ENTRY`), maps libc and whatever else it needs
with `mmap()`, relocates, and jumps to `AT_ENTRY` itself. A position-independent
program goes where a fixed one would (`0x400000`); the interpreter goes at the
start of the `mmap` area (`0x40000000`) and `mmap` carries on above it, so the
two never meet. This is what lets Alpine's binaries run unmodified, and
`dlopen` works, so Xorg loads its drivers as modules.
`mmap` uses a bump allocator (`proc->mmap_next`, starting at `0x40000000`) so
every mapping gets its own disjoint, exactly-sized range. It previously assigned
@ -329,24 +429,23 @@ allocation over 8 MB mapped straight into a neighbouring live mapping.
loop reports 97%, `sleep 2` reports 0% — but kernel time needs TSC deltas
closed and reopened around every `sched()`, which is scheduler accounting
rather than sampling, and is not written.
- **No slab allocator.** One page at a time, and every subsystem's fixed-size
table is a consequence. This is the single change that would most improve the
rest of the kernel.
- **No demand paging and no page cache.** `mmap` of a file *reads the whole
thing in* at map time. Writable `MAP_SHARED` of a file is therefore refused
outright rather than faked — see [../plan/foundations.md](../plan/foundations.md).
- **No dynamic linking.** `exec` handles `ET_EXEC` only: no `PT_INTERP`, no
`ET_DYN`, no shared libraries, no `dlopen`.
- **No threads.** `clone` exists but does not share an address space;
`gettid` returns the pid.
- **`struct socket` is too big for a slab**, and every subsystem's fixed-size
table (`NPROC`, `NSOCK`, `NDEVICE`, `NSHM`) is still a compile-time constant.
Slabs exist for the objects that fit in a page.
- **No page cache.** `mmap` of a file *reads the range in* at map time, and
writable `MAP_SHARED` of a file is refused outright rather than faked. Demand
paging covers anonymous memory only.
- **Permissions are tracked but not enforced.** uid/gid are maintained honestly
so `getuid()`/`setuid()` stop lying, and nothing consults them on access.
so `getuid()`/`setuid()` stop lying, and file access does not consult them;
`setpriority` is the one place the uids are checked.
- **No swap**, and no memory pressure handling of any kind. `kalloc` failing is
reported up as `-ENOMEM` and that is the whole policy.
- **Nested signal delivery is not supported.** `in_signal` guards against
re-entering a handler while one is running.
- **Scheduling is fair-share on one global run queue.** Priorities and
fairness accounting exist now ([scheduler.md](scheduler.md)); per-CPU run
queues, work stealing and CPU affinity do not, despite `sched_setaffinity`
existing.
- **One global run queue.** Fair share and `nice` exist; per-CPU run queues, work
stealing and CPU affinity do not, and `sched_setaffinity` is accepted and
ignored.
- **No IOMMU**, so DMA-capable devices (and the driver processes that program
them) are trusted.
- **256 GB is the most RAM** the kernel uses; the rest is ignored, with a line
saying so.