nexoniarz/SchOS
nexoniarz/SchOS
Watch
1
0
Fork
You've already forked SchOS
 0
Code Issues Pull requests Projects Releases 1 Packages Wiki Activity Actions

docs: architecture describes what is built now

Browse source 
kernel (threads, demand paging, TLB shootdowns, dynamic linking, RAM direct map), syscalls, filesystem (/proc, /sys,
/dev), terminals (12, VT handshake, ptys), networking (r8169 through LDLS), LDLS (module sets; Linux code may ship),
devices (what stays in the kernel; hybrid.md's lasting content), boot (the loader's placement of the kernel),
graphics (fbdev, GMMOS, QR 1-40), audio (ALSA over OSS), debugging, usb. New: packages.md (the apk root) and
desktop.md (Xorg on the framebuffer).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EVRhxi9zpdW2uDUN4FVcao
This commit is contained in:
Nexoniarz 2026-10-26 07:57:25 +01:00
commit 2556a955b3
15 changed files with 921 additions and 341 deletions
Download patch file Download diff file Expand all files Collapse all files

21
docs/architecture/audio.md
View file

@ -40,7 +40,7 @@ until the speaker has caught up*.
**It is one stream.** Mixing several programs into one device is a real feature
with a real design behind it, and building it before anything had played a
single sound would be exactly the mistake [../plan/1.0.md](../plan/1.0.md)
keeps a register for: work that exists and is connected to nothing. The second
keeps a register against: work that exists and is connected to nothing. The second
program that wants to play at the same time is what should cause it to be
written.
@ -84,6 +84,25 @@ Both register files are **I/O ports**, not MMIO — which is why nothing in the
driver maps anything. `bar[n].is_io` is what says so, and a driver that
`ioremap`ed these would be mapping a port number as though it were an address.
### Through ALSA, for programs that expect it
SDL 3 has no OSS backend and SchOS has no ALSA *kernel* side, so sound from SDL games
and anything else that speaks ALSA goes **SDL → alsa-lib → alsa-plugins' OSS plugin →
`/dev/dsp`**. `alsa-lib` and `alsa-plugins` are in the Alpine `world`, and
`/etc/asound.conf` (from `schos-base`) makes the default device a `plug` onto the OSS
device at 16-bit stereo 48 kHz, which is what every SchOS audio driver plays.
Running that path found two bugs underneath it:
- **AC'97:** resetting the PCM-out box also clears the descriptor-list address and nothing
set it again, so every stream after the first stop played a "list" at physical address 0
— silence — and ALSA resets before each stream. If the engine halts at the end of the
queue, the run bit is set again.
- **`/dev/dsp` was missing the ioctls the plugin's ring-and-pointer model uses:**
`SETFRAGMENT`, `GETFMTS`, `GETCAPS`, `SETTRIGGER`, `GETOPTR`, `GETODELAY`, and `poll()`,
writable once a whole fragment fits (ALSA spun at 76% CPU otherwise). `speaker-test` now
plays in real time at 0% CPU, verified in a recording of QEMU's AC'97.
## How it is tested
QEMU's `wav` audio backend writes what the guest plays to a file, so the test

48
docs/architecture/boot.md
View file

@ -131,6 +131,42 @@ when the early console is set up. So a low framebuffer is attached exactly as
before, and a high one is attached a few dozen instructions later, immediately
after `kvmalloc()` — still thousands of lines before anything else happens.
### Where the kernel goes, and how it gets there
The kernel is linked at physical **16 MB** (`KERNPHYS` in `memlayout.h`, repeated in
`kernel.ld`, which must agree) and is about 34 MB with the ramdisk inside it. From the
old 1 MB it ran across 8 MB, where OVMF keeps ACPI NVS memory that outlives the firmware,
so it was overwritten on every boot, after `ExitBootServices`. Firmware leaves its
leftovers low; above 16 MB is where a machine is most reliably empty. (The 15 MB below
the kernel is not given to the page allocator yet.)
**Every free page of that range is claimed first.** A segment lying wholly on claimed
pages is read straight home; one that is not is read into a buffer elsewhere and
**copied home after `ExitBootServices`**, when boot-services memory is ours. An HP
all-in-one from 2012 used to freeze at the start of the loading bar because the loader
said "could not reserve" and read the kernel there anyway, over the firmware's own memory
with the firmware still running on it. Memory under the kernel that outlives the firmware
(ACPI NVS, runtime services) is named, and so is the loader or its stack sitting in the
range. Segments are read a megabyte at a time, so the bar moves through the ramdisk and
shows where a machine stopped.
The copy and the jump themselves run from **one page below the kernel's lowest address**,
allocated as loader code before the firmware exits: a short position-independent routine,
the copy list and a stack, switching to the loader's page tables and its own stack before
it copies anything. Doing the copy from code the firmware had placed wherever it liked put
some of it inside the kernel's range on that machine: a triple fault and a reset. The
routine is `sysv_abi`, because UEFI code is the Microsoft convention and the first try loaded
the framebuffer address into `rsp`.
**No 1 GB pages, no problem.** Intel's desktop CPUs before Haswell have none, and there the
PS bit in a PDPT entry is reserved: the first access after the switch is a page fault that
cannot be handled either. Both the loader and the kernel check CPUID `0x80000001` EDX bit 26;
without it the low 4 GB is four page directories of 2 MB pages (placed below the kernel, with
the staged segments kept below 4 GB), and the kernel gives each gigabyte of its direct map
and of `ioremap()` a page directory of its own. Neither KVM nor QEMU enforces the reserved
bit, so the fault could not be reproduced here; with the fallback forced on, the kernel
boots at 2 and 8 GB and both CPUs start.
### The loader jumped out from under itself
A machine with a lot of RAM froze at "Starting the kernel" — the splash still
@ -610,7 +646,9 @@ in the one process that runs first.
## What it does not do
- **No BIOS path.** A machine without UEFI does not boot SchOS.
- **No BIOS path.** A machine without UEFI does not boot SchOS. The ISO carries a real EFI
system partition (MBR type `0xEF`, FAT16 by cluster count, the type old firmware reads best),
so a stick written with `dd` or `write.sh` boots on strict firmware as well as on lenient.
- **The quiet boot hides a slow one.** With nothing printing between the
loader and the greeting, a boot that stalls for ten seconds looks the same as
one that does not. F8 is the answer and it has to be pressed *before* the
@ -621,9 +659,11 @@ in the one process that runs first.
with its partitions, but nothing mounts it. See [usb.md](usb.md).
- **NVMe and AHCI poll for completion**, one command at a time per disk. That
is enough to boot and run from them, not to be fast.
- **`make ramroot` is the only way to boot with no data disk.** It links a
filesystem image into the kernel binary, which costs boot time and carries a
licensing constraint — see [../licensing.md](../licensing.md) §4.
- **The live ISO is a ramdisk root.** `make iso` builds the root as a SchFS image,
compresses it ([scff.md](scff.md)) and links it into the kernel; nothing is written to
the machine's disks, and everything written to the root is gone at reboot. There is
no installer, and no kernel that finds a root on a disk by SchFS's GPT type as its
*default*. `make ramroot` is the same without compression.
- **The 32-bit multiboot entry is dead code.** It is left in place deliberately:
deleting kernel entry code is its own piece of work with its own way of
failing silently.

18
docs/architecture/debugging.md
View file

@ -179,6 +179,10 @@ part of the boot is blank for the part that is hardest to debug any other way.
---
A panic is not the end of the log. GMMOS ([graphics.md](graphics.md)) keeps running
under the report, can show this log on screen or as QR codes a phone can read, and
has a small terminal for looking around a machine whose kernel is dead.
## Asking a question
`/dev/dbg` is a conversation: write a line, read the answer.
@ -493,10 +497,16 @@ something that reports nonsense.
- **One conversation at a time.** The answer buffer is global, so two programs
asking at once get one answer each in some order — never half of the other's,
but not two independent sessions either.
- **No userspace symbolization.** The kernel names kernel symbols. A fault in a
process still reports a bare address, and resolving it is a host-side job.
- **The stack walk stops where the frame pointer stops.** `getcallerpcs()` walks
`rbp` chains, so a trap frame in the middle of a chain ends the walk.
- **No userspace symbolization.** The kernel names kernel symbols. A process
killed by a fault is told which *file and offset* it died in (from
`/proc/<pid>/maps`' records), which is enough to resolve with the file's own
symbols on the host; a bare address is all there is for anonymous memory.
- **The stack walk stops where the frame pointer stops, and it will not follow a
bad one.** `getstackpcs()` walks `rbp` chains, and each link must go up the same
stack by less than `KSTACKSIZE`: a chain can end at the syscall entry with a *user*
`rbp` that a program uses as an ordinary register (`-1024` in one game), and
dereferencing that under the syscall tracer once panicked the kernel inside
`acquire()`. A trap frame in the middle of a chain ends the walk.
- **`sysrq g` costs you log history.** Its output goes through `cprintf`, so
sixteen kilobytes of views push the same amount out of a 128 KB ring.
- **No tracing.** No static tracepoints, no event ring, no counters. "What

74
docs/architecture/desktop.md Normal file
View file

@ -0,0 +1,74 @@
# The desktop: Xorg on the framebuffer
The desktop is X11, on the framebuffer: Alpine's `xorg-server` with the `fbdev` video
driver and the `evdev` input driver, Window Maker as the window manager, and `xterm`.
`startx` runs `wmaker`. Nothing here is written by SchOS except the kernel interfaces it
stands on and four small files in `tools/posix/mkaroot.sh`.
## Why X and not Wayland
Every Wayland compositor worth running — labwc, and everything built on wlroots — needs
DRM/KMS, libinput, udev and libseat. DRM/KMS is the largest of them by far: a kernel graphics
subsystem, and in practice a great deal taken from Linux. Xorg runs on a plain framebuffer
through `fbdev` and reads input through `evdev`, both interfaces SchOS already had. That is
the shorter road to a desktop by a long way, and it brings the window managers with it.
DRM/KMS is not planned; libinput, udev and libseat are worth having for their own sake and
may come later.
## What it stands on
| | |
|---|---|
| **fbdev** | `/dev/fb0` speaks Linux's fbdev ([graphics.md](graphics.md#devfb0-and-how-xorg-draws)) |
| **evdev** | `/dev/input/event0` (every keyboard) and `event1` (every mouse and touchpad) with Linux's queries, LED writes and key/button/relative-axis bits |
| **VTs** | `/dev/tty0`, `VT_SETMODE(VT_PROCESS)`, `KD_GRAPHICS`, the release/acquire handshake ([terminals.md](terminals.md)) |
| **dlopen** | Xorg loads its drivers as modules, so `exec` does dynamic linking ([kernel.md](kernel.md)) |
| **sysfs** | `/sys/bus/pci/devices/*/{boot_vga,enable,graphics/fb0}`, which libpciaccess reads |
| **ptys** | `xterm` runs its shell on a pseudo-terminal |
| **Unix sockets** | the X11 socket, abstract names, `SO_PEERCRED`, `flock` |
| **headers** | `<linux/*.h>` in the build sysroot, as `ports/linux-headers` |
`cmd/xready` checks every one of the kernel calls Xorg makes before it draws anything — the
evdev queries, the fbdev calls, the console calls, and the `/sys` files — one at a time, on a
terminal of its own, and puts everything back. The calls were written against what the
programs are documented to do; each step past it found one that was not, which is what running
the real program is for.
## The four files
- **`/root/.xserverrc`** starts `X` on the VT `startx` was started from (startx works that
out only when `uname -s` says Linux, and SchOS's says SchIFT) with **`-dumbSched`**. Xorg
normally reads input on a thread of its own; when SchOS had no threads the server crashed
the first time a client disconnected, in `pthread_join` on a thread that was never made, and
`-dumbSched` is Xorg's own switch for input on the main loop. Threads exist now; the switch
is still on, and nobody has run the server without it since.
- **`/root/.xinitrc`** is `exec wmaker`. Without it xinit falls back to twm and an xterm, which
are not in the image.
- **`/etc/X11/xorg.conf.d/10-schos.conf`** names the input devices. Alpine's Xorg asks udev for
them, SchOS has no udev, and X came up with no keyboard and no mouse: the pointer did not
move and nothing could end the session. SchOS has exactly two input nodes whatever the
machine, so a fixed list is the whole answer (`AutoAddDevices` off, Ctrl+Alt+Backspace
enabled).
- **`/etc/asound.conf`** is for sound, not X ([audio.md](audio.md)).
## Using it
Twelve virtual terminals, switched with Ctrl+Alt+F1–F12 — on the console and under X
alike, because X owns the keyboard while it runs and switches only with that chord. `startx` on
a terminal runs the desktop there; Ctrl+Alt+Backspace ends the session and gives the terminal
back. Window Maker, `xterm`, `xeyes`, `xclock`, `xcalc`, DejaVu fonts; SDL 2 games
(`lbreakouthd`) and NetSurf, a GTK 3 browser, run in it.
Touchpads that sit on an I2C bus (most laptops) are driven by `kernel/acpi/i2c_hid.c`, found
through ACPI, and reach X as a mouse: pointer, click and tap.
## What it does not do
- **No acceleration.** Software rendering to the framebuffer: no DRM, no glamor, no DRI, no
OpenGL beyond what software gives. GPU drivers are a plan ([../plan/gpu.md](../plan/gpu.md)).
- **No udev.** Input devices are named in a config file, not found; `/sys` has no `devices`
tree and no uevent socket.
- **Touchpads are mice**: no two-finger scroll, no gestures, no palm rejection.
- **No resolution change from inside X.** The mode is the console's; `schmode` changes the
`vgpud` head only, and the loader's menu is where a real panel's mode is chosen.
- **Not a security boundary.** X runs as root on a system that enforces no file permissions.

142
docs/architecture/devices.md
View file

@ -4,7 +4,50 @@ How hardware gets found, claimed and used, and what the rest of the kernel sees
of it.
Writing a driver is [../guides/adding-a-driver.md](../guides/adding-a-driver.md).
Moving drivers out of the kernel entirely is [../plan/hybrid.md](../plan/hybrid.md).
Drivers that live in a process rather than in the kernel are described below
(*Drivers that are not in the kernel*), and the foreign ones in [ldls.md](ldls.md).
---
## What stays in the kernel, and what may leave
"Hybrid" is used to mean everything from "a monolith with a nice device model" to
"a microkernel that gave up", so the line is worth drawing exactly.
**Stays in, permanently:** scheduling, address spaces, the trap and syscall path;
IPC and shared memory; the VFS and the **root** filesystem; IRQ demultiplexing; and
the console and PSOGM. That last one is not a size compromise but an invariant: the
crash screen must work when everything else is dead, so it cannot depend on anything
that could *be* dead. A crash screen served by a userspace process is absent in
exactly the case it exists for.
**May leave, and should:** GPU drivers, non-root filesystems, NICs after the first,
and anything written by someone else.
> **Anything needed to reach a shell stays in. Anything discovered at runtime,
> replaceable, or plausibly written by someone else goes out.**
>
> **The kernel keeps a minimal driver for everything on the path to a shell, and
> userspace owns the good one.** Break that and the compositor cannot start because
> the display driver is a process that needs a compositor.
### USB stays in the kernel, and why
It was on the list to leave, and the premise was wrong. USB is not a peripheral; it
is the bus peripherals arrive on, structurally the same kind of thing as PCI, and
nobody has proposed evicting PCI. The keyboard and the root disk that arrive over it
are on the path to a shell, so the stack that finds them is part of that path. Even
with a minimal in-kernel fallback written, the good driver and the minimal one would
be the same driver, and an eviction that leaves an identical copy behind has moved
nothing. A second, quieter difference: `/dev/pci` hands over one PCI *function*, one
owner and nothing shared, while a USB device sits behind a controller other devices
also sit behind — "the xHCI controller" includes the keyboard. What the eviction was
for, third-party drivers living outside the kernel, survives and belongs to
[ldls.md](ldls.md).
The root filesystem stays in the kernel for the same reason (SchFS cannot be
restarted, and a bug in it is fatal), and so does the first NIC, because the network
thread and lwIP are in the kernel.
---
@ -184,8 +227,47 @@ client — and that is worth a sentence here because it is the thing the device
model makes possible rather than the thing it does. A modeset is five commands
to the device. In the kernel those five commands are also a reason to be
nervous, because a wrong one takes the machine; in a process the worst case is
that `schinit` starts another one. See
[../plan/hybrid.md](../plan/hybrid.md), "A modeset, from a process".
that `schinit` starts another one. That is why the in-kernel driver never grew
one — not difficulty, exposure.
```
# schmode
1280x800 current
1280x800 maximum, framebuffer 4000 KB
# schmode 1024 768
1024x768 ok
```
The scanout really resizes, and nothing in the kernel changed for it: `schmode`
writes a line to a socket and `vgpud` sends five commands to the device. The
framebuffer is allocated **once, at the largest mode**, and every smaller mode
reuses it, because `PCIIOC_DMA` has no free — a driver's blocks come back when it
exits and not before — so a modeset that allocated would spend the arena and the
failure would look like a GPU bug rather than like counting. A larger mode is
refused, and says why. The control socket speaks lines of text and has no buffer
submission, damage or vblank, which a compositor would need and nothing has asked
for.
### A panic has to reach the screen
Handing a device away costs something that was not on the list. The kernel panics,
paints the report into the console framebuffer, and halts — and the display it gave
to a process keeps showing whatever that process last put there. On a machine with
two heads, one says the kernel is dead and the other shows a shell prompt. There is
no scheduling after a panic, so the process holding the device is never going to run
again, and the kernel let go of the device deliberately.
So the driver says in advance where it is scanning out, with `PCIIOC_PANICFB`
(`include/dev/pciuser.h`), and the kernel: (1) paints the report into that memory;
(2) asks the *kernel* driver the device was taken from to put it on screen —
`panic_present` in `struct pci_driver`; (3) which resets the device and rebuilds one
queue from **static** memory — no allocation, no locks, no sleeping, because a
corrupted heap is one of the ways a machine gets here. The registration is checked
like the promise it is: the range must lie inside a block that process got from
`PCIIOC_DMA` on that device, or it would be a way to nominate any physical address
to be overwritten at panic. It costs 20 KB of static memory in the virtio-gpu driver,
paid whether or not the machine ever panics; allocating it on demand would be
cheaper and would not work.
### Taking it back
@ -222,8 +304,9 @@ enforces it.
Coming back is a job, not a kernel feature. `etc/schinit/vgpud.job` carries
`KeepAlive = yes`, so killing the driver is the entire test: `schinitd` starts
another one, it claims the device, and the pid of whichever process drew the
frame is legible in the frame itself. See
[../plan/hybrid.md](../plan/hybrid.md), "The driver came back".
frame is legible in the frame itself: read off the scanout across four generations of
the driver it said 4, 8, 10, 12, and the DMA arena handed back the same addresses
every time.
### What this is not
@ -231,8 +314,7 @@ frame is legible in the frame itself. See
buffer can point the device at any physical address. This isolates a driver
against its own *bugs* and not against its own *intent* — a crashed driver that
restarts instead of taking the machine down is the entire benefit, and it is
worth having on those terms. [../plan/hybrid.md](../plan/hybrid.md) says so at
length.
worth having on those terms. VT-d/AMD-Vi is its own project.
**One device per process.** A driver drives one thing; if you need two, run two.
@ -240,29 +322,24 @@ length.
## Linux drivers
**Decided 2026-08-08.** Linux drivers are not shipped with SchOS and never
will be. The user may install one on their own machine, through a device
manager or equivalent, and SchOS will load it.
[ldls.md](ldls.md) is how SchOS uses them: a Linux `.ko` runs in a process and
reaches the hardware through `/dev/pci`. The licensing position, stated exactly
because it is easy to get wrong in both directions:
The licensing reasoning matters and is worth stating exactly, because it is
easy to get wrong in both directions:
- **Distribution is what triggers the GPL.** SchOS is GPL-2.0-or-later, so shipping a
GPL-2.0-only driver in an image is *lawful* — see [../licensing.md](../licensing.md) §6 —
and the owner has decided it is allowed (code and firmware blobs). The repository does
not carry one all the same: `.gitignore` refuses `*.ko`, and the modules are staged into
a *local* image from the build machine's own Linux kernel (`make ldls-stage`,
`make ldls-r8169`), where the matching BTF is.
- **Implementing a loader is not copying.** Writing support for a module format and
a driver ABI from published documentation produces our own work; the layouts come
from the kernel's BTF at load time on the user's machine, so the source holds none
of Linux's.
- **Distribution is what triggers the GPL.** SchOS is GPL-2.0-or-later, so
shipping a GPL-2.0-only driver in an image is now *possible* — see
[../licensing.md](../licensing.md) §6 — but LDLS still does not, by policy. A
user obtaining a driver themselves and loading it on their own machine is
their act, not ours, and is not a distribution by us.
- **Implementing a loader is not copying.** Writing support for a module format
and a driver ABI from published documentation produces our own work. It is a
clean-room implementation, and it stays one only if it is written from
documentation rather than transcribed from Linux source. Reading Linux to
understand *hardware* is fine; translating its code is not, whatever the
identifiers say.
So the rule for this subsystem is the same as everywhere else in the tree: the
loader is ours and GPL-2.0-or-later, what it loads is the user's business, and
no third-party module enters an image we build. `RAMDISK_EXCLUDE` and `NOTPORTS` are the
existing mechanisms for keeping that true mechanically rather than by intention.
The loader is ours and GPL-2.0-or-later; what it loads is not in the tree.
`RAMDISK_EXCLUDE` and `NOTPORTS` are the mechanisms that keep that true mechanically
rather than by intention.
---
@ -371,12 +448,13 @@ the console's minors to the console and the rest to their own functions. But
- **No hot-plug.** Devices are registered at boot and never removed; the
`in_use` flag is the hook a hot-unplug path would need, and nothing sets it
for that purpose.
- **`NDEVICE` is 32**, compile-time, like every other table in the kernel.
- **`NDEVICE` is 128**, compile-time, like every other table in the kernel.
- **In-kernel drivers are fixed at link time.** There is no module loader, and
nothing can be added to the kernel after the build. A driver *outside* the
kernel is an ordinary program and can be started whenever — but it reaches its
device through `/dev/pci` and publishes nothing to the class vtables, so the
console cannot be handed to one yet.
kernel is an ordinary program and can be started whenever — it reaches its
device through `/dev/pci`, and a NIC driver can publish itself to the network
class through `/dev/netdev` (LDLS does); a display driver publishes nothing to
`display_ops`, so the console cannot be handed to one.
- **The kernel does not rebind its own driver.** `PCIIOC_RELEASE`, or the process
exiting, leaves the interrupt line masked and the in-kernel driver unbound —
the console stays wherever the release put it. A *supervisor* can start a

53
docs/architecture/filesystem.md
View file

@ -199,11 +199,14 @@ provides — reads and writes never reach it, because `fileread`/`filewrite` see
It is read-only: `dirlink` and `ialloc` answer `EROFS` rather than pretending.
A node exists because the kernel has the device, or it does not exist.
### What is there
### What is there in `/dev`
`console`, `null`, `keyboard`, `zero`, `full`, `random`, `urandom`, `tty`,
`stdin`, `stdout`, `stderr`, `fb0`, `mouse`, `hda`, `hdb`, and `input/event0`
and `input/event1` in one subdirectory.
`console`, `null`, `zero`, `full`, `random`, `urandom`, `tty`, `tty0` and
`tty1`..`tty12`, `stdin`, `stdout`, `stderr`, `keyboard`, `mouse`, `fb0`, `dsp`,
`ptmx` and a `pts/` directory of pseudo-terminal slaves, `hda`, `hdb`,
`input/event0`.., and the SchOS-specific `pci` (a device's registers, DMA and
interrupts, for a driver in a process), `netdev` (a network interface for a
process), `dbg` and `dbgro` (the kernel debugger's views).
Every disk and partition the block layer registered is there too, under the
name it was given (`sata0`, `sata0p2`, `nvme0p1`, `usb0p1`): major 8, minor its
@ -218,11 +221,18 @@ nothing behind it fails with `ENXIO`, which is a better answer than a missing
path — and with no node at all, `echo x > /dev/hdb` silently creates an ordinary
file of that name and appears to work.
### What is there
### What is there in `/proc`
Per-process directories, plus the global files: `meminfo`, `uptime`, `cpuinfo`,
`version`, `stat`, `loadavg`, `mounts`, `filesystems`, `schfs`, `pci`,
`interrupts`, `devices`, `netif`, and `sysrq`.
`/proc/self`, and one directory per process holding `stat`, `status`, `cmdline`,
`statm`, `comm`, `schedstat`, `maps`, `exe` (a symlink), `fd/` (a symlink per open
descriptor) and `task/` (one directory per thread). `maps` lists the file
mappings, and a process killed by a fault is told which file and offset it died
in.
The global files: `meminfo`, `uptime`, `cpuinfo`, `version`, `stat`, `loadavg`,
`mounts`, `filesystems`, `partitions`, `schfs`, `pci`, `interrupts`, `devices`,
`netif`, `resolv.conf`, `slab`, `fsstat`, `kmsg`, `strace`, `bootmode`,
`bootdebug`, `quiet`, `verbose`, `schinit` and `sysrq`.
**/proc is read-only except `sysrq`**, which is the one deliberate write target,
and the code says so in one place. Marking every procfs inode read-only once
@ -249,6 +259,8 @@ mounted by init with `mount("sysfs", "/sys", "sysfs", 0, 0)`.
| `/sys/firmware/memmap/N/` | `start end type`, the firmware's map as the loader handed it over |
| `/sys/firmware/efi/` | present when UEFI booted the kernel; empty |
| `/sys/class/dmi/id/` | `sys_vendor product_name product_version board_name bios_version`, and the rest Linux has, except serials and the UUID |
| `/sys/class/power_supply/` | `BAT0` (`capacity`, `status`, `energy_*`/`charge_*`, `voltage_now`, `cycle_count`, ...) and `AC` (`online`), from ACPI — the battery and the adapter are read through the laptop's embedded controller |
| `/sys/bus/pci/devices/…/` | also `boot_vga`, `enable` and `graphics/fb0`, which libpciaccess (and so Xorg) reads |
**Linux's layout because everything that reads hardware reads it from there.**
toybox's `lspci` and `lsusb` walk `/sys/bus/*/devices` and parse each `uevent`
@ -265,7 +277,8 @@ tables, `config`) report their real length, since tools copy them by it.
What is missing, compared with Linux: `/sys/devices` and its symlinks (the
`bus/*/devices` entries are directories rather than links into it),
`/sys/class` beyond `dmi`, `/sys/block`, and anything writable. PCI `config` is
`/sys/class` beyond `dmi` and `power_supply`, `/sys/block`, a uevent socket (so
no udev), and anything writable. PCI `config` is
256 bytes, because config space is read through port `0xCF8`, which reaches
no further.
@ -376,8 +389,6 @@ through.
- **No page cache.** `mmap` of a file reads the whole thing in at map time, and
writable `MAP_SHARED` of a file is refused rather than faked. This is the
single largest gap in the layer.
- **No symlinks in procfs**, which is why `/proc/<pid>/exe` is a regular file
and `ps -o comm` prints `[init]`.
- **`rename()` does not move directories**, and will not replace one. `EISDIR`
either way.
- **No named pipes.** `mkfifo()` fails with `EPERM`, Linux's answer for a
@ -385,19 +396,13 @@ through.
device, because `mknod()` read a Linux program's `(mode, dev)` as the native
`(major, minor)`: `ls -l` showed `crw-rw-rw- 438, 0` where a `p` belongs. The
zsh port was configured with `zsh_cv_sys_fifo=yes` on the strength of that.
- **No `/dev/fd`, and no `/proc/self`.** A process cannot reach its own open
descriptors by path. `<(cmd)` needs one or the other — zsh now knows it has
neither and says so.
- **No `AF_UNIX` socket inodes.** Socket names live in a table in
`kernel/net/unix.c`, so `bind()` creates nothing that can be `stat()`ed or
`unlink()`ed.
- **No `/sys/devices` and no uevents**, so nothing can be told that a device
appeared; Xorg is given its input devices by a config file instead.
- **No unified inode cache across filesystems** beyond the single `itable`.
- **A known, un-root-caused hang:** heavy `ls -lR` use across many commands in
one long shell session can eventually make a *later*, unrelated directory
hang; the identical command succeeds on a fresh boot. It is not inode or file
table exhaustion — both fail cleanly and neither fired. It smells like a lock
or refcount leak in the `*at()` paths or in toybox's own fd handling. The next
step is instrumenting `itable`/`ftable` occupancy across a long session to see
what is climbing.
- **`sys_delete`/`sys_show`/`sys_showstat`** are orphaned entries from the old
v1 filesystem's recycle bin. Nothing reaches them and they die with `fs.c`.
- **One long-standing hang was never found.** Heavy `ls -lR` use across many
commands in one long shell session once made a *later*, unrelated directory
hang; the identical command succeeded on a fresh boot. It was instrumented and
hunted and could not be reproduced, which is recorded in
[../history/kernel.md](../history/kernel.md) rather than forgotten.

127
docs/architecture/graphics.md
View file

@ -1,8 +1,8 @@
# Graphics, text and the console
Four things share this area and they are deliberately separate: the kernel's own
drawing surface, the compositor's display interface, the text-rendering layer,
and the crash screen.
Five things share this area and they are deliberately separate: the kernel's own
drawing surface, the `display` class a display server could use, the text-rendering
layer, the crash screen — and `/dev/fb0`, which is how Xorg reaches the screen.
---
@ -13,11 +13,11 @@ first thing here.
| | [`gfx.h`](../../include/gfx.h) | [`display.h`](../../include/display.h) |
|---|---|---|
| **What** | The kernel's own drawing surface | The compositor's interface |
| **What** | The kernel's own drawing surface | A frame interface for a display server |
| **About** | Pixels and text | *Frames* |
| **Operations** | `draw_pixel`, `fill_rect`, `putc`, `text_at` | `acquire`, `present`, `wait_vblank` |
| **Buffering** | None. Direct writes to the scanout. | Double-buffered |
| **Users** | The console and PSOGM | Schcom |
| **Users** | The console, PSOGM, GMMOS | None today — Xorg draws through `/dev/fb0` |
| **Must work when** | **Nothing else in the system does** | Userspace is healthy |
That last row is the reason they are not one interface. The console and the
@ -67,7 +67,7 @@ and `vgpud` mirroring it see what is on the screen.
`struct display_ops`, found with `device_find_class(DEVCLASS_DISPLAY)`.
`acquire()` allocates the back buffer **on first call, never at boot** — it is
around two megabytes at 800×600, and a machine whose compositor never starts
around two megabytes at 800×600, and a machine whose display server never starts
should not pay that. A driver that cannot double-buffer returns 0, and the
caller draws directly and accepts tearing rather than not drawing at all.
`wait_vblank` is optional for the same reason.
@ -133,7 +133,7 @@ A machine with both a stdvga and a virtio-gpu has two displays, and `vgpud` —
the out-of-kernel driver, `cmd/vgpud/main.c` — owns the second one. For a long
time it drew a test card there: colour bands, gradients, a checkerboard, and
its own pid in binary along the bottom. That was scaffolding for
[../plan/hybrid.md](../plan/hybrid.md)'s H2 and H3, where a screenshot had to
the out-of-kernel driver work ([devices.md](devices.md)), where a screenshot had to
answer *did a userspace process put this here* and *did the driver come back*.
Both of those are closed, so the test card became the Shift mark. Which was
@ -223,9 +223,8 @@ shows the top-left corner, which is where a terminal puts everything worth
seeing. Scaling is the alternative and it is the wrong one: text resampled to a
non-integer ratio is text you cannot read.
**This is not the endpoint.** A compositor (Gate 3) will want this head for
itself, and when there is one, the mirror goes away. Until then the honest
thing for a working display to show is the system running on it.
A display server that wants the head for itself can have it; until one does, the
honest thing for a working display to show is the system running on it.
### The F8 resolution is the head's resolution
@ -263,6 +262,23 @@ not apply: two `vgpud`s cannot coexist, because the first holds the PCI claim
and the second is refused the device long before it reaches `bind()`. The claim
is the mutual exclusion; the socket node never was.
## /dev/fb0, and how Xorg draws
Xorg runs on the framebuffer through its `fbdev` driver, so `/dev/fb0` speaks Linux's
fbdev: `FBIOGET_FSCREENINFO`, `FBIOGET_VSCREENINFO`, `FBIOPUT_VSCREENINFO` (accepted
when it asks for the mode already in force), `FBIOPAN_DISPLAY`, `FBIOBLANK`,
`FBIOPUTCMAP`, and `mmap` of the physical framebuffer. `FBIOGET_VSCREENINFO` reports
nominal timings, as Linux's `efifb` does, instead of zeros — a zero pixel clock made
fbdev compute a 0 Hz RandR mode. Pixel layout comes from the same table the kernel
draws with, so a colour a program builds is the colour the kernel would.
Once a program maps the framebuffer the kernel has no way to know when it draws, so
`fb_mapped()` marks the screen as no longer the text console's and `fbclose()` puts it
back. A terminal switched to `KD_GRAPHICS` is not repainted ([terminals.md](terminals.md)),
and the `VT_PROCESS` handshake tells the server before it loses the screen. `cmd/xready`
checks every one of these calls before Xorg is started; the picture itself is the proof
beyond that.
## Text
[`include/gfx/text.h`](../../include/text.h) + `kernel/dev/text.c`.
@ -537,20 +553,59 @@ channel that works when the machine will not come back at all.
machine whose network stack is dead**, and a phone camera is an out-of-band
channel needing no driver on our side.
It encodes the **report itself**, not a URL — it reaches nothing. Deliberately
small in three ways: byte mode only, EC level L only (the screen is a clean,
evenly lit, high-contrast source; the damage QR's stronger levels protect
against does not apply), and versions 1–6 only, stopping short of version 7's
extra version-information blocks and version 10's wider count field.
It encodes the **report itself**, not a URL — it reaches nothing. Byte mode only
and EC level L only (the screen is a clean, evenly lit, high-contrast source; the
damage QR's stronger levels protect against does not apply), but **every version,
1 to 40**: version 40 at level L holds 2,953 bytes, which is what lets GMMOS put the
kernel log on screen as a series of codes. Checked against the `segno` library module
for module at every version.
134 bytes is a large crash report. If a payload ever needs more, the honest fix
is to shorten the report.
It draws on the report unconditionally rather than on a keypress, which is better:
it costs nothing, and requiring input would mean the one case where input is least
trustworthy is also the case where the report is hardest to retrieve. Verified by
`make crashshot && make crashqr`, which decodes the symbol out of the captured
framebuffer.
It draws unconditionally rather than on a keypress, which is better: it costs
nothing, and requiring input would mean the one case where input is least
trustworthy is also the case where the report is hardest to retrieve. Verified
by `make crashshot && make crashqr`, which decodes the symbol out of the
captured framebuffer.
### Every processor stops
A panic used to stop only the processor it happened on; the rest carried on with a
dead kernel, drew over the report, and kept the fans loud until somebody held the power
button. The panicking processor now sends every other one an **NMI**, which arrives even
with interrupts off, and `trap()` halts a processor that takes one while `panic_stop` is
set — first thing, before the canary check, because that stack may be what broke. NMI has
its own per-CPU stack (IST 1): for a few instructions at the top of `syscall_entry` the
live stack is the user's, at ring 0, and a fault pushing the frame there would be a double
fault on the same stack and a reset instead of a report. A second processor that panics
while one is reporting halts.
## GMMOS — the Guru Meditation Micro OS
The report used to be the last thing the kernel did: a picture, blinking keyboard lights,
and a machine that needed its power button. GMMOS keeps the report and goes on living
under it (`kernel/gmmos/`, about 2,000 lines). The real kernel stays dead — nothing of it is
called again that takes a lock, sleeps, allocates or waits for an interrupt. GMMOS has its
own: the framebuffer, a clock read from the TSC, and keyboards it polls itself through
**micro drivers**, cut-down copies of the real ones that do only what a keyboard needs.
| Key | |
|---|---|
| (nothing) | a status line counts down **three minutes** to power-off; any key cancels it |
| **Space** | the last 300 lines of this boot's log as **QR codes**, a page per symbol, numbered so a phone can take them in order; `+` / `-` trade module size for capacity |
| **L** | the log on screen, scrolled with the arrows |
| **T** | a terminal: `why`, `log`, `grep`, `ps`, `cpus`, `pci`, `mem` (page tables walked first, so a bad address cannot fault), `uptime`, `restart`, `poweroff` |
| **R R** / **P P** | restart / power off — twice, so a stray key does neither |
The micro drivers are **PS/2**, **I2C-HID** (the laptop's own keyboard, over a DesignWare
controller — the live driver hands over what it found at boot, because AML cannot run on a
dead kernel) and **xHCI** (USB keyboards: every controller is halted, reset and enumerated
from nothing, root ports and USB 2 hubs up to two deep, a boot-protocol keyboard, no
interrupts, a static 256 KB pool). GMMOS probes all of them and polls every one that answers.
Every buffer is static. A panic inside GMMOS, or a second one in the report, is reported
and then only blinks; GMMOS does not re-enter itself. Checked under QEMU: every screen, two
QR pages read off a screenshot with `zbar` and decoded to the log text exactly, restart
reboots, and left alone it powers off at three minutes. What has been seen on real hardware
is the report and the keyboards the laptops have; the xHCI path is the least exercised.
## One palette
@ -571,16 +626,20 @@ makes it mean something.
## What it does not do
- **The console still has its cell grid.** Cells are now a Unicode codepoint
drawn by the TrueType rasteriser rather than a CP437 byte drawn from a bitmap,
and the cell size follows the font's measured advance — but it is still a
fixed grid, and the text layer's proportional path is unused by it.
- **No mode setting.** Resolution is whatever the loader asked for. There is no
`/dev/dri`-style device and no way to change it at runtime.
- **No double buffering in practice.** `display_ops.acquire` is specified and
the compositor needs it; virtio-gpu binds but exposes no modeset API, and
- **The console still has its cell grid.** Cells are a Unicode codepoint drawn by the
TrueType rasteriser, and the cell size follows the font's measured advance (the font
scales to about 45 rows, 16 to 40 px per em, on whatever mode the firmware booted
in) — but it is a fixed grid, and the text layer's proportional path is unused by it.
- **Mode setting is the loader's and `vgpud`'s.** On real hardware resolution is whatever
the loader chose from its menu — which now asks "Keep this resolution?" and falls back
by itself, because AMD's GOP "sets" smaller modes without leaving the panel's native
timing and the screen showed three or four squashed copies side by side. There is no
`/dev/dri`, no KMS, and Xorg runs at the mode the console is in.
- **No double buffering in practice.** `display_ops.acquire` is specified and nothing
uses it; virtio-gpu exposes a modeset API on `vgpud`'s socket and nothing more, and
`wait_vblank` is unimplemented everywhere.
- **No hardware acceleration of any kind**, and none is planned before 2D blit.
- **No hardware acceleration of any kind**: Xorg draws in software to the framebuffer
(no glamor, no DRI), and GPU drivers are a plan, not a feature ([../plan/gpu.md](../plan/gpu.md)).
- **No CFF/Type 2 charstrings**, so OTF fonts do not render — that is a separate
`font_ops` provider and a bytecode interpreter, not an extension of the `glyf`
path. TrueType hinting is deliberately out of scope.
@ -593,6 +652,6 @@ makes it mean something.
- **No composite point-matching.** Components positioned by matching a point in
the base rather than by an offset are placed at the origin. Fonts that use it
are rare and none in the tree do.
- **PSOGM renders and stops.** The interactive crash mode described in
[../plan/roadmap.md](../plan/roadmap.md) — walk the stack, read another
process's memory, dump page tables — is designed and not built.
- **GMMOS cannot look at a live process by name.** The "interactive crash mode" is built as far
as a dead machine allows — log, a terminal, `ps`, `cpus`, `pci`, memory by physical walk — but
the scheduler is gone, so there is no running a debugger against a process.

209
docs/architecture/kernel.md
View file

@ -11,18 +11,27 @@ driver or a protocol stack: memory, processes, scheduling, traps and signals.
One page table per process; the kernel is mapped into all of them at
`KERNBASE = 0xFFFF800000000000`, so a trap does not switch `CR3`.
`kvmalloc()` maps physical 0–4 GB as four 1 GB pages, with the top one marked
uncached — that is where PCI BARs and the framebuffer live. This is why 3 GB is
the most memory the kernel can treat as RAM, and it is a deliberate simplicity:
a direct map with a fixed offset makes `P2V`/`V2P` arithmetic rather than a
lookup.
`kvmalloc()` maps as many 1 GB pages of physical memory as the machine has RAM
(up to `PHYSTOP_MAX`, 256 GB), always at least the low four. The 3–4 GB slot stays
uncached because that is where the PCI hole is; everything above 4 GB is RAM, so
those are cached, and a device BAR up there still gets its own uncached window
from `ioremap()`, which takes the entries the direct map did not. On a CPU
without 1 GB pages (Intel's desktop parts before Haswell), each gigabyte is a
page directory of 2 MB pages instead — the same reach, one extra page of table
per gigabyte. A direct map with a fixed offset makes `P2V`/`V2P` arithmetic
rather than a lookup.
`kalloc()` hands out one 4 KB page at a time from a free list. **There is no
slab or pool allocator**, and the consequence is visible throughout the tree:
every subsystem that needs many small objects has invented its own fixed-size
table — `NPROC` processes, `NSOCK` sockets, `NDEVICE` devices, `NSHM` segments.
That is why the limits are compile-time constants rather than a design decision
about each one.
How much RAM there is comes from the firmware's E820/UEFI map, as a *list* of
usable regions, not one run ending at a top address (OVMF drops an ACPI NVS
region at 8 MB, which used to hide everything above it). The kernel itself is
linked at 16 MB, clear of what firmware keeps low. `kalloc` holds never-used
memory as ranges, so a large machine costs nothing until its memory is used.
`kalloc()` hands out one 4 KB page at a time from a free list. Objects smaller
than a page come from the slab caches below; objects larger than one do not have
a home yet, so the biggest tables (`NPROC` processes, `NSOCK` sockets, `NDEVICE`
devices, `NSHM` segments) are still fixed-size arrays, which is why their limits
are compile-time constants rather than a design decision about each one.
The free list carries a canary, checked at the top of both `kalloc()` and
`kfree()`, so a wild write into it panics at the point it is touched rather than
@ -150,8 +159,8 @@ pages, reference counted, mapped through a VMA with `no_free` set so that
`memfd_create(2)` is the same machinery reached by **descriptor** instead of by
key. That difference is the whole point and the reason it exists on Linux at
all: two processes share memory by passing the fd over a socket, so neither has
to agree on a path or race to invent a key. It is what a Wayland buffer crosses
on, and it is why descriptor passing (`SCM_RIGHTS`) had to be built first.
to agree on a path or race to invent a key. It is how a display server and its client
share a buffer, and it is why descriptor passing (`SCM_RIGHTS`) had to be built first.
| | |
|---|---|
@ -171,10 +180,46 @@ version of something this kernel cannot do:
mapper. Here a freed page would simply stay mapped to memory somebody else
now owns.
Writable `MAP_SHARED` of a **real file** is still `ENODEV`. That needs a page
cache, it is a separate project, and nothing in 1.0 asks for it — an anonymous
object has no such contract to break, which is exactly why `memfd` is the
narrow answer.
Writable `MAP_SHARED` of a **real file** is still refused. That needs a page
cache, it is a separate project, and nothing that runs today asks for it — an
anonymous object has no such contract to break, which is exactly why `memfd` is
the narrow answer.
### Demand paging
Anonymous memory can be recorded without being backed: the page appears,
zeroed, the first time it is touched — by the program, through `trap()`, or by
the kernel on its behalf, through `uva2ka()`. `mmap` does this for mappings of
1 MB or more, for `MAP_NORESERVE`, and for `PROT_NONE`. The reason is WebKit,
which reserves 4 GB of address space at start for its JIT and heaps and touches
a sliver of it; backing all of it at `mmap` time failed with `ENOMEM` and the
browser aborted. Go, Java and Firefox reserve the same way.
`vm_fault()` takes a spinlock and nothing that sleeps, so `uva2ka()` may call it
from anywhere; the lock keeps two threads of one process from both mapping the
same page. `fork` and `munmap` skip the holes, so a page nobody touched is one
they never see, and a child that touches it reads zeroes — what it would have
read in the parent.
File-backed mappings are **not** lazy: `mmap` of a file still reads the range in
at map time.
### TLB shootdowns
Unmapping from an address space that threads on other processors are running in
interrupts those processors and waits until each has reloaded `CR3`
(`tlb_shootdown()` in `vm.c`; the vector is one of the MSI lines, which `trap()`
already dispatches with an EOI). Without it, when one thread unmapped a range and
`mmap` handed the same addresses straight back, a thread on another CPU went on
reading and writing the *old* pages through its TLB: stores vanished and loads
were stale. musl's allocator noticed first — a group's metadata no longer
pointing back at it — and Epiphany, forty threads on four processors, died at a
different address on every run.
Only one shootdown runs at a time, and it is taken without disabling interrupts:
two processors each waiting with interrupts off for the other to answer would
wait for ever. On a single processor, or if no vector could be allocated, it
does nothing (and says so at boot).
### What the fixed tables actually cost
@ -184,31 +229,69 @@ partly true:
| | size | table | |
|---|---|---|---|
| `struct proc` | 11744 | `NPROC` 128 | larger than a page; 1 KB of it is FPU state |
| `struct socket` | 4936 | `NSOCK` 128 | larger than a page |
| `struct proc` | 1984 | `NPROC` 128 | 254 KB in all; the address space, descriptors and signal handlers live in their own objects now, shared by threads |
| `struct mm` | 8232 | one per process, 128 | 1 MB in all, almost all of it the 256-entry VMA table |
| `struct socket` | about 4900 | `NSOCK` 128 | larger than a page |
| `struct device` | 120 | `NDEVICE` 128 | 15 KB, never hit |
| `struct buf` | 200 | `NBUF` 256 to start | grows from a slab, data from `kalloc`, up to RAM/32 |
| `struct inode` | 296 | none | grows from a slab; `NINODE` 200 is only how many are made up front |
| `struct file` | 64 | none | grows from a slab; `NFILE` 512 is only the static pool |
| `struct inode` | about 460 | none | grows from a slab; `NINODE` 200 is only how many are made up front |
| `struct file` | about 88 | none | grows from a slab; `NFILE` 512 is only the static pool |
The two largest cannot go in a page-backed slab at all. Freeing them needs a
contiguous multi-page allocator, which is a separate thing and is not written.
The buffer header shrank to 200 bytes when its 4 KB of data moved out to a page
of its own, which is what let the cache grow at all.
(Sizes from `nm -S` on a build of the tree, not from a header comment.)
`struct proc` shrank by a factor of six when threads arrived, because the page
tables, VMAs, descriptor table and signal dispositions moved out of it into
objects a thread group shares. `struct socket` is still too large for a
page-backed slab; freeing it needs a contiguous multi-page allocator, which is a
separate thing and is not written. The buffer header shrank to 200 bytes when its
4 KB of data moved out to a page of its own, which is what let the cache grow at
all.
## Processes
`struct proc` in [`include/core/proc.h`](../../include/proc.h). A fixed table of
`NPROC` entries, allocated by linear scan.
`struct proc` in [`include/core/proc.h`](../../include/core/proc.h). A fixed table of
`NPROC` entries, allocated by linear scan. A thread is a `struct proc` too.
**The field order at the top of the struct is fixed by assembly.** `trapasm.S`
hard-codes `fs_base` at offset `0x18`, and there is a `_Static_assert` in
`proc.c` that fails the build if it moves. Anything new goes at the end.
Each process carries: page directory and size, kernel stack, open file table,
cwd inode *and* cwd path string, signal handlers and masks, VMA table for
`mmap`, POSIX identity (pgid, sid, and all six of real/effective/saved uid and
gid), and the Linux-ABI syscall map.
Each process carries: a kernel stack, cwd inode *and* cwd path string, signal
masks, POSIX identity (pgid, sid, and all six of real/effective/saved uid and
gid), the Linux-ABI syscall map, and its x87/SSE registers (saved on a switch;
the kernel itself stays off them). Three things it *points to* rather than
owns, because a thread group shares them:
| | |
|---|---|
| `struct mm` | the address space: page tables, heap top, the VMA table, which file each span came from, and a lock so one thread at a time changes the mappings |
| `struct files` | the descriptor table and its `FD_CLOEXEC` flags |
| `struct sighand` | signal dispositions — what `sigaction()` sets |
Each is reference counted. `clone()` shares what its flags say and `fork()`
copies all three.
### Threads
`clone()` takes `pthread_create`'s flags (`CLONE_VM`, `FILES`, `SIGHAND`,
`THREAD`, `SETTLS`, `PARENT_SETTID`, `CHILD_CLEARTID`) and `posix_spawn`'s
(`VM`, `VFORK`, a stack). What follows from that:
- **futex** — `WAIT`, `WAIT_BITSET` with timeouts, `WAKE`, `REQUEUE`,
`CMP_REQUEUE`, keyed by the word's *kernel* address so processes that map the
same page meet. `set_tid_address` keeps the word that exit zeroes and wakes,
which is what `pthread_join` waits on.
- **Exit.** `exit` ends the calling thread; `exit_group`, a fatal signal, or the
leader exiting ends them all. `reapd` frees exited threads, and the leader is
reaped by its parent once the rest are gone. `getpid` is the group, `gettid`
the thread.
- **Signals.** Stops and resumes reach every thread; other process-directed
signals go to the leader.
- **`exec`** kills the other threads and unshares the dispositions.
- **`/proc`** lists processes, `task/` lists threads; `comm`, `Threads:` and
`Tgid` are there.
The first program to need all of this was GTK's, then WebKit's.
### Slot reuse
@ -232,12 +315,15 @@ exactly as a syscall would. The network thread is the current user.
## Scheduling
Round-robin over the process table, one scheduler per CPU, each with its own
context. `swtch()` saves callee-saved registers and switches stacks; `sched()`
returns to the scheduler and asserts four invariants on the way — holding
`ptable.lock`, `ncli == 1`, not `RUNNING`, interrupts off.
Fair share by virtual runtime, one scheduler per CPU, each with its own context
— the rule, the guards and `nice` are in [scheduler.md](scheduler.md).
`swtch()` saves callee-saved registers and switches stacks; `sched()` returns to
the scheduler and asserts four invariants on the way — holding `ptable.lock`,
`ncli == 1`, not `RUNNING`, interrupts off.
When a full pass finds nothing runnable the CPU halts rather than spinning.
When nothing is runnable the CPU goes idle rather than spinning: into ACPI's
`_CST` C2 state (CC6 on Zen), by port read or `MWAIT`, where the APIC timer keeps
running through it, and otherwise a plain `hlt`.
**Locking is a single `ptable.lock` over the whole table.** Not per-process: a
scheduling operation touches two processes (the one leaving and the one
@ -309,9 +395,23 @@ mksh's job control could not make progress until they existed.
## What userspace gets
`fork`, `exec`, `wait`/`waitpid`/`wait4` with `WNOHANG` and real pid selection,
pipes, signals, `mmap` (anonymous and file-backed), POSIX shared memory, Unix
domain sockets, `poll`/`select`/`epoll`, `eventfd`, `timerfd`, `signalfd`.
`fork`, `clone` (threads), `exec`, `wait`/`waitpid`/`wait4` with `WNOHANG` and
real pid selection, pipes, signals, `mmap` (anonymous and file-backed), POSIX
shared memory, `memfd`, Unix domain sockets, `poll`/`select`/`epoll`, `eventfd`,
`timerfd`, `signalfd`, `inotify`, `futex`.
### Dynamic linking
`exec` handles `ET_EXEC` and `ET_DYN`. A dynamically linked program names an
interpreter in `PT_INTERP` — on musl, `/lib/ld-musl-x86_64.so.1`, which is libc
under another name — and the kernel's whole part is to load that as well and
start it instead of the program. The interpreter finds the program through the
auxiliary vector (`AT_PHDR`, `AT_ENTRY`), maps libc and whatever else it needs
with `mmap()`, relocates, and jumps to `AT_ENTRY` itself. A position-independent
program goes where a fixed one would (`0x400000`); the interpreter goes at the
start of the `mmap` area (`0x40000000`) and `mmap` carries on above it, so the
two never meet. This is what lets Alpine's binaries run unmodified, and
`dlopen` works, so Xorg loads its drivers as modules.
`mmap` uses a bump allocator (`proc->mmap_next`, starting at `0x40000000`) so
every mapping gets its own disjoint, exactly-sized range. It previously assigned
@ -329,24 +429,23 @@ allocation over 8 MB mapped straight into a neighbouring live mapping.
loop reports 97%, `sleep 2` reports 0% — but kernel time needs TSC deltas
closed and reopened around every `sched()`, which is scheduler accounting
rather than sampling, and is not written.
- **No slab allocator.** One page at a time, and every subsystem's fixed-size
table is a consequence. This is the single change that would most improve the
rest of the kernel.
- **No demand paging and no page cache.** `mmap` of a file *reads the whole
thing in* at map time. Writable `MAP_SHARED` of a file is therefore refused
outright rather than faked — see [../plan/foundations.md](../plan/foundations.md).
- **No dynamic linking.** `exec` handles `ET_EXEC` only: no `PT_INTERP`, no
`ET_DYN`, no shared libraries, no `dlopen`.
- **No threads.** `clone` exists but does not share an address space;
`gettid` returns the pid.
- **`struct socket` is too big for a slab**, and every subsystem's fixed-size
table (`NPROC`, `NSOCK`, `NDEVICE`, `NSHM`) is still a compile-time constant.
Slabs exist for the objects that fit in a page.
- **No page cache.** `mmap` of a file *reads the range in* at map time, and
writable `MAP_SHARED` of a file is refused outright rather than faked. Demand
paging covers anonymous memory only.
- **Permissions are tracked but not enforced.** uid/gid are maintained honestly
so `getuid()`/`setuid()` stop lying, and nothing consults them on access.
so `getuid()`/`setuid()` stop lying, and file access does not consult them;
`setpriority` is the one place the uids are checked.
- **No swap**, and no memory pressure handling of any kind. `kalloc` failing is
reported up as `-ENOMEM` and that is the whole policy.
- **Nested signal delivery is not supported.** `in_signal` guards against
re-entering a handler while one is running.
- **Scheduling is fair-share on one global run queue.** Priorities and
fairness accounting exist now ([scheduler.md](scheduler.md)); per-CPU run
queues, work stealing and CPU affinity do not, despite `sched_setaffinity`
existing.
- **One global run queue.** Fair share and `nice` exist; per-CPU run queues, work
stealing and CPU affinity do not, and `sched_setaffinity` is accepted and
ignored.
- **No IOMMU**, so DMA-capable devices (and the driver processes that program
them) are trusted.
- **256 GB is the most RAM** the kernel uses; the rest is ignored, with a line
saying so.

125
docs/architecture/ldls.md
View file

@ -6,7 +6,11 @@ drivers, and LDLS is what does that.
It works, and on more than one driver. `8139too` and `e1000` — unmodified GPL
kernel modules — load into a SchOS process, drive their hardware through
`/dev/pci`, and carry packets:
`/dev/pci`, and carry packets (both under QEMU). A third, Linux's `r8169`, is
the NIC driver on the ASUS TUF A15 laptop, and it is the first that needs a
*set* of modules and not one; it is described under *A set of modules*. It has
registered its interface and read its PHY on that laptop; whether it carries
traffic there has not been confirmed yet.
```
# ldlsd /lib/modules/8139too.ko /lib/modules/btf &
@ -46,18 +50,20 @@ Every one was a call that succeeded, returned zero, and did nothing.
Five, and everything else is an implementation of them.
### 1. Nothing GPL ever enters an image we build
### 1. Linux code may ship; so far the tree holds none
The driver is **the user's**: obtained by them, loaded on their machine. SchOS
ships the layer and never the thing it loads. `.gitignore` refuses `*.ko`,
`make ldls-stage` is the one command that puts a module into a local image and
is deliberately not part of `all`, and `make ldls-unstage` takes it out again.
This rule used to say nothing GPL would ever enter an image we build. It was a policy,
never a licence constraint — SchOS is GPL-2.0-or-later, and GPL-2.0-only code combines
with it — and the owner has overridden it: **Linux code and firmware blobs may be
shipped.** What survives of the rule is mechanical, not legal. The repository carries no
`*.ko` (`.gitignore` refuses them), and `make ldls-stage KO=…` / `make ldls-r8169` stage
modules into a *local* image from the Linux kernel on the build machine, because a module,
its dependencies and the BTF that describes them have to come from the same Linux build
and that kernel is the easiest place to get a matching set.
[../licensing.md](../licensing.md) §6 is the whole argument. The short form is
that the combination happens in a user process on the user's machine, so no
combined work is distributed by anyone. Since SchOS moved to GPL-2.0-or-later
that is no longer the only way it *could* be done — it is the way it is done,
because a module we never ship is a module we never have to answer for.
Shipping one is therefore a decision about the release, not about the code: an ISO that
contains `r8169.ko` is a distribution of GPL-2.0-only code and carries the usual
obligation — its source goes with it — see [../licensing.md](../licensing.md) §6.
### 2. It runs in a process, never in the kernel
@ -186,6 +192,55 @@ the object has to live where a signed 32-bit absolute address reaches. A
process's mmap area starts at `0x40000000`, which is inside that window; on the
host harness `MAP_32BIT` reaches it from the other end.
### A set of modules — what r8169 needed
Linux's `r8169` is not one module. It needs `mdio-bus`, `libphy`, `mdio_devres`,
the Realtek PHY driver, the NIC itself, the RTL PHY firmware, and the BTF of the
kernel they came from. `make ldls-r8169` stages all of that from the running
kernel; `etc/schinit/ldls-r8169.job` starts `ldlsd` on it at boot, with its output
in `/var/log/ldls-r8169.log`. The job has `KeepAlive = no`: a driver that crashes
on real hardware would otherwise be restarted into the same crash every few seconds.
What that added to `ldlsd`:
- **A comma-separated module set**, loaded in dependency order, each linking
against the ones before it, with the BTF read before any `init_module` runs.
A library module (`libphy`) just exports symbols to the next; the PCI `probe`
runs once, at the end, for whichever module registered a `pci_driver`.
- **A driver core**: bus match, driver probe, so the PHY driver binds to the PHY
the way it does in Linux; **devres**; a kernel `printf`; real `udelay`/`msleep`/
`ktime`; and **1000 Hz `jiffies`**, because the module's `HZ` is compiled in and
phylib re-arms its link poll `HZ` jiffies out.
- **Firmware** read from `/lib/firmware`.
- **phylib's data**, the `link_mode_params` table and its bitmaps.
- **The symbols whose stub-zero was a lie**: list validity, `try_module_get`,
`dev_err_probe`, `eth_platform_get_mac_address`, `napi_consume_skb` (as a stub it
leaked one buffer per transmitted frame until the DMA heap ran out), and the
rest of the table at the end of this document.
- `pdev->bus` is a real `pci_bus`; `net_device.priv_len` is set from BTF (the
driver's `memset` of its ring bookkeeping is bounds-checked against it, and
`__fortify_panic` fired on a length of zero); and `cmovcc` from per-CPU memory is
rewritten like the other `%gs` forms.
- **`__preempt_count` starts at `PREEMPT_ENABLED`.** mdio-bus keeps the register
it has just read in `r11` across `preempt_enable()`, which on x86 is
`decl %gs:__preempt_count; je`, to a call that preserves every register. With the
count at zero the call was made, into a logging stub whose syscall left `RFLAGS`
(0x246) in `r11` — so every MDIO read returned `0x02460246`. `preempt_schedule` is
now a bare `ret`.
- `__fortify_panic` names the module and function (`r8169: rtl_open.cold+0x…`) so
a photograph of the screen is enough, and exits instead of spinning on a core and
holding `/dev/netdev`.
The kernel did two things for it. It turns **ASPM off on Realtek NIC links** at
boot, as `r8169` does through `pci_disable_link_state`, which a process cannot;
and a network interface that **appears after boot starts the stack** — with the
native driver out of the registry the laptop boots with no interface at all, and
`ldls0` arrives later. `ldlsd` says which refusal it got when attaching: `EBUSY` is
another process, anything else is the kernel.
The native `drivers/net/r8169/` stays in the tree, with Linux's 8168H setup, as
the fallback; it is out of the registry.
### The KPI — `ldls/kpi/`
The Linux symbols, implemented. Register access to a mapped BAR; the PCI calls
@ -232,8 +287,8 @@ and setting both to zero makes the whole thing reversible with no lookup table.
### The back end — `/dev/pci`
Already built, for `vgpud`, before any of this existed.
[hybrid.md](../plan/hybrid.md) insisted the interface be about *PCI devices*
Already built, for `vgpud`, before any of this existed. The design insisted the
interface be about *PCI devices*
rather than about a driver model, on the argument that an interface whose first
consumers are native drivers is a general mechanism rather than a hole shaped
like something foreign. A Linux driver's `pci_iomap`, `dma_alloc_coherent` and
@ -241,10 +296,9 @@ like something foreign. A Linux driver's `pci_iomap`, `dma_alloc_coherent` and
### Giving a device up — `release`
A driver in the kernel now has a `release` hook, and `/dev/pci` calls it before
handing its device to a process. `docs/plan/hybrid.md` had said the honest
answer to "may I have the one you are using" is *no* until a driver could be
told to let go; e1000 is the first that can.
A driver in the kernel has a `release` hook, and `/dev/pci` calls it before
handing its device to a process. The honest answer to "may I have the one you are
using" was *no* until a driver could be told to let go; e1000 is the first that can.
Letting go is three things, and the third was missed at first:
@ -336,17 +390,17 @@ point of having built it against our own:
## What it does not do
- **Two drivers, one kernel version, one machine.** `8139too` and `e1000` on
6.18.46 against QEMU. The second driver was worth more than the first: it
found the per-CPU problem, the retpoline problem, the port-I/O problem, the
interrupt storm and a seven-year-old hole in the TSS. Nothing here has been
tried on a second kernel or on real silicon, and "LDLS runs Linux drivers" is
still a claim about a category from a sample of two.
- **Three drivers, one kernel version.** `8139too` and `e1000` on 6.18.46
against QEMU, and `r8169` on that version against QEMU (where it finds no
device) and on one laptop (where it gets as far as the PHY). The second driver
was worth more than the first: it found the per-CPU problem, the retpoline
problem, the port-I/O problem, the interrupt storm and a seven-year-old hole in
the TSS. Nothing here has been tried on a second kernel, and "LDLS runs Linux
drivers" is still a claim about a category from a sample of three.
- **No interrupts**, per the polling note above.
- **`jiffies` is not `HZ`.** It comes from a 100 Hz `uptime()`, which is a
plausible `HZ` and not necessarily the one the module was built for. A driver
measuring a timeout against it gets a longer timeout, which is the safe
direction and is still wrong.
- **`jiffies` assumes `HZ` is 1000**, which is what the kernels LDLS loads from
are built with (`ldls_load.hz` can say otherwise). It comes from the
monotonic clock in milliseconds.
- **One work item at a time, on the main loop.** A driver whose deferred work
blocks stops everything, including the transmit path it is probably waiting
for.
@ -363,14 +417,21 @@ point of having built it against our own:
process, which is a leak in the strict sense and the right trade for a
program that will be killed to restart it.
- **BTF has to come from somewhere.** SchOS has no `/sys/kernel/btf/vmlinux`,
so the user brings it with the module, from the kernel the module was built
for. A kernel built without `CONFIG_DEBUG_INFO_BTF` has neither.
so `make ldls-stage` / `ldls-r8169` copy it from the Linux kernel the module
came from. A kernel built without `CONFIG_DEBUG_INFO_BTF` has neither.
- **There is address-space groundwork nothing uses.** `kernel/arch/x86_64/vm.c`
maps the kernel image a second time at `LDLS_KALIAS`, in the top 2 GB where a
`-mcmodel=kernel` module can reach, with a `vmemmap` and a module area beside it
— at every boot — and `include/ldls/ldls.h` describes a `/dev/ldls` control
channel to load modules into the kernel. That was the start of an in-kernel
loader, contradicting rule 2. Nothing calls either, and it should be finished
as a decision or deleted.
- **No config-space writes.** `/dev/pci` has none, deliberately — a write is
how a BAR gets moved out from under the kernel — so a driver that needs one
is refused loudly rather than quietly succeeded.
- **No module-to-module dependencies, no firmware loading, no sysfs, no
unload.** `request_firmware` fails, `module_exit` is never called, and the
way to stop a driver is to kill the process.
- **No sysfs and no unload.** Modules depend on each other and firmware is read
from `/lib/firmware`, but `module_exit` is never called, and the way to stop a
driver is to kill the process.
- **Some symbols are still logging stubs.** Deliberate, per rule 5 — and the
single most expensive thing in this layer. **A stub that returns zero is not
safe when zero is a meaningful answer**, and the list of ways that bit is

195
docs/architecture/networking.md
View file

@ -1,7 +1,9 @@
# Networking
ARP, IPv4, ICMP, UDP, TCP and DHCP, over a driver-agnostic interface class,
reached through the same socket syscalls as Unix domain sockets.
reached through the same socket syscalls as Unix domain sockets — plus DNS
(resolved in the process, by musl), HTTPS (`wget`, Alpine's TLS libraries) and a
small netlink responder for `getifaddrs`.
Verified end to end rather than by inspection: `bootcheck -Net` starts a
listener and a UDP responder on the host, tells the guest to talk to them, and
@ -25,7 +27,9 @@ that completes a handshake and drops every byte.
└────────────────┬───────────────────┘
│ struct netdev_ops
┌────────────────┴───────────────────┐
│ e1000, and any future NIC driver │
│ e1000 rtl8139 r8169 — or a │ in the kernel
│ Linux driver in a process, which │ (LDLS, via /dev/netdev)
│ is a netdev like any other │
└────────────────────────────────────┘
```
@ -86,50 +90,55 @@ covers two decades of silicon — and why the driver prints the chip's XID on
probe, since the PCI device id tells you almost nothing about which part
arrived.
### Why it is native, when the plan said LDLS
### Native first, then Linux's own
The first plan was to run Linux's `r8169.ko` under [ldls.md](ldls.md). Measuring
it is what changed the answer: the module has **183 undefined symbols, 130 of
them not implemented**, and 35 of those 130 are **phylib** —
`phy_connect_direct`, `phy_start`, `mdiobus_read`, `genphy_soft_reset`.
The first plan was to run Linux's `r8169.ko` under [ldls.md](ldls.md), and
measuring it changed that: the module has **183 undefined symbols, 130 of them not
implemented**, and 35 of those 130 are **phylib** — `phy_connect_direct`,
`phy_start`, `mdiobus_read`, `genphy_soft_reset`. phylib is a subsystem with a link
state machine, not a set of leaf calls, and LDLS's usual answer for an
unimplemented symbol — a stub that reports itself and returns zero — is precisely
wrong for it: `phy_start` returning zero means the link never comes up and nothing
says why. So a **native driver** came first: it reaches the PHY through the
chip's own MII window (`PHYAR`, one register that is a whole MII bus) and skips
the subsystem, in the shape of the e1000 beside it.
phylib is a subsystem with a link state machine, not a set of leaf calls, and
LDLS's usual answer for an unimplemented symbol — a stub that reports itself
and returns zero — is precisely wrong for it. `phy_start` returning zero means
the link never comes up and there is nothing on screen to say why. The 8139too
that ldls.md uses as its model has no phylib dependency, which is why that one
was tractable.
It has since been done the other way as well. LDLS grew a driver core, devres,
firmware loading and phylib's tables ([ldls.md](ldls.md), *A set of modules*),
and the ASUS TUF A15's RTL8168H is now driven by **Linux's r8169, unmodified, in
a process**: `make ldls-r8169` stages it with libphy, mdio-bus and the PHY driver,
and `etc/schinit/ldls-r8169.job` starts it at boot. The native driver is **out of
the registry** — it would claim the device first — and stays in the tree, with
Linux's 8168H setup, as a fallback.
A native driver reaches the PHY through the chip's own MII window (`PHYAR`,
one register that is a whole MII bus) and skips the subsystem entirely. It is
less work than the port, it carries no GPL entanglement, it needs no firmware
blob, and it is not coupled to one Linux version.
The register layout is a *fact about the hardware*, taken the way
[ldls.md](ldls.md) §3 says facts may be taken. The expression is this tree's:
a descriptor-ring driver in the same shape as the e1000 beside it.
On that laptop the interface is `ldls0`, and it appears *after* boot, which is why
`net_attach_late()` starts the stack when the first interface arrives instead of
turning it away.
### The Chromebox has one of these
The Asus Chromebox CN62 "Guado" turns out to carry an RTL8111 rather than the
Intel part its spec sheet suggests, so this driver covers it as well as the
laptop that prompted it — which is the argument for the family match in one
line rather than a device id per board.
The Asus Chromebox CN62 "Guado" carries an RTL8111 rather than the Intel part its
spec sheet suggests, so the family match (`10EC:8168`) covers it as well as the
laptop — which is the argument for matching a family in one line rather than a
device id per board.
### It has not run on hardware
### It has not been shown to work on hardware
**Stated plainly because it cannot be tested here.** QEMU emulates `rtl8139`
and nothing else from Realtek, and the 8139 is a different programming model —
a circular receive buffer rather than descriptors — so it is deliberately *not*
in the match table. Claiming it would break a NIC that works.
**Stated plainly because it cannot be tested here.** QEMU emulates `rtl8139` and
nothing else from Realtek, and the 8139 is a different programming model — a
circular receive buffer rather than descriptors — so it is deliberately *not* in
the match table. Claiming it would break a NIC that works.
What has been verified: the driver builds, its presence changes nothing on a
machine without the device, and with an `rtl8139` on the bus it correctly
leaves it unclaimed (`10ec:8139 network controller [-]`).
What has been verified: both drivers build, their presence changes nothing on a
machine without the device, and Linux's r8169 under LDLS loads, links and
initialises in QEMU, registering and finding no device. On the laptop it reached
probe and read the PHY (after a bug in LDLS that made every MDIO read return
`0x02460246` was fixed); the native 8168H bring-up — the PLL, the receive gate, the
PHY through the OCP window — has not run on a part.
What has not: every register write, on a real part.
What has not: traffic on real hardware, from either.
So the probe says everything a first bug report would need, with `cprintf`
The native probe says everything a first bug report would need, with `cprintf`
rather than `kinfo` so it reaches the screen on a machine with no serial port:
```
@ -141,6 +150,9 @@ register 1, where bit 2 is link and bit 5 is autonegotiation complete. Those
four facts separate "the chip is not there", "the chip is there and the PHY
sees nothing", and "everything is up and the rings are wrong".
The kernel turns **ASPM off** on Realtek NIC links at boot, as Linux's driver
does through `pci_disable_link_state` — which a process under LDLS cannot do.
### Two constants that were wrong first
Recorded because both would have worked, quietly:
@ -262,6 +274,12 @@ to. Measured across three pings: `rx_frames` and `wake_rx` both went 2 → 6.
table, and it returns immediately: everything happens in that thread, which
cannot run until the scheduler is entered.
**The thread follows each device's link into lwIP.** The link used to be read once
at attach, 0.02 s into boot; real autonegotiation takes seconds, so on hardware
the interface stayed down for good. It also starts the stack when an interface
appears after boot (a driver in a process, a USB NIC), and attaches every device
there is as its first act; `net_eth_attach()` refuses one it already has.
DHCP obtains an address at boot. `/proc/netif` reports each interface's address,
netmask, gateway, MAC and frame counts, taken at the lwIP seam.
@ -326,19 +344,20 @@ The creation lives in `sys_bind()` rather than here, because that is the layer
with both: `kernel/net/unix.c` knows about sockets and has no business calling
`create()`.
**`flock()` is still missing.** libwayland locks `wayland-0.lock` beside the
socket so that two compositors cannot claim one display. The lock file can be
created and opened now that paths are real; the advisory lock itself is a
separate mechanism with no other caller yet.
`flock()` and `fcntl` record locks exist (`kernel/fs/lock.c`), so the lock files
next to a socket work the way they do on Linux. Names in
the **abstract namespace** (a `sun_path` starting with a NUL) name no file; they
work, and `getsockname` reports them unnamed. `SO_PASSCRED` is honoured, and
`SO_PEERCRED` answers (tmux asks).
### Descriptor passing
`sendmsg`/`recvmsg` with `SCM_RIGHTS`, which is the one thing a socket can carry
that a pipe cannot: a reference to an object the kernel already knows, rather
than a copy of its contents. Wayland's `wl_shm_create_pool` passes a descriptor
and has no integer-shaped alternative, and
[hybrid.md](../plan/hybrid.md)'s mechanism 1 is how a driver process is handed
the device it is allowed to touch. Both wanted the same primitive.
and has no integer-shaped alternative, and the same mechanism is how a driver
process is handed the device it is allowed to touch ([devices.md](devices.md)).
Both wanted the same primitive.
**A batch of descriptors is pinned to the stream position of the first byte of
the message that carried it.** That single rule is the whole delivery model, and
@ -374,9 +393,9 @@ descriptors to a position that never arrives.
**There is no garbage collector.** An AF_UNIX socket descriptor, sent over that
same socket and never received, holds a reference to the socket that holds the
reference to it, and neither close can reach zero. Linux carries several hundred
lines to break these cycles. Nothing on either plan passes a socket over a
socket — Wayland passes memory, the driver work passes devices — so this is a
known and bounded leak (thirty-two sockets) rather than a silent one.
lines to break these cycles. Nothing passes a socket over a
socket — display servers pass memory, the driver work passes devices — so this is
a known and bounded leak (thirty-two sockets) rather than a silent one.
### AF_INET
@ -593,59 +612,47 @@ information was there and the code did not ask.
## What it does not do
- **`ping` is a first-party program**, `cmd/ping`, not toybox's. `recvmsg` now
exists, so half of what stopped toybox's is gone; what remains is
`IP_RECVTTL`, an `IPPROTO_IP` control message that would have to be generated
on the receive path rather than merely carried. Ours has no `-I`, no flood
mode and no TTL display.
- **`SCM_RIGHTS` is the only control message.** `SCM_CREDENTIALS` is refused
with `EINVAL` rather than ignored: a caller told its credentials were passed
would have the receiver read a uid out of uninitialised memory and act on it.
- **`ping` is a first-party program**, `cmd/ping`, not toybox's or busybox's. Ours
has no `-I`, no flood mode and no TTL display (`IP_RECVTTL` is an `IPPROTO_IP`
control message that would have to be generated on the receive path), and takes a
dotted quad or `localhost` — it does not link musl's resolver. Busybox's `ping`,
from the Alpine root, works over a `SOCK_RAW` ICMP socket.
- **Only `SCM_RIGHTS` and `SCM_CREDENTIALS` travel in a control message.** Anything
else is refused rather than ignored.
- **A message may carry eight descriptors, and a socket eight such messages.**
Past that `sendmsg` answers `ENOBUFS`. The cap exists because the kernel holds
a reference to each one until it is received; a sender eight unreceived
messages ahead of its receiver is not rescued by a ninth slot.
- **`ping` still takes a dotted quad**, even though names now work. It is a
first-party program that does not link musl's resolver, and `localhost` is
special-cased in it rather than resolved. Every *ported* program resolves
names properly; ours is the one that does not.
- **Nothing re-runs `resolvconf`.** The nameserver is written once at boot. A
lease that changes the DNS server, or a `/dev/pci` handover that brings up a
new interface, leaves a stale `/etc/resolv.conf` until somebody runs it again.
- **lwIP's own DNS client is still unused.** It is compiled in and only
`dns_getserver()` is ever called; resolution happens entirely in the process.
Nothing in the kernel can resolve a name, which matters the day something in
the kernel wants to.
- **`httpd` does not serve HTTP/1.1.** Every response is HTTP/1.0 with
`Connection: close`, because the directory listing is generated straight into
the socket and so has no `Content-Length` to give. One request per connection
is the cost.
- **Nothing re-runs `resolvconf`.** The nameserver is written once at boot
(`etc/schinit/resolvconf.job`). A lease that changes the DNS server, or a
`/dev/pci` handover that brings up a new interface, leaves a stale
`/etc/resolv.conf` until somebody runs it again.
- **lwIP's own DNS client is unused.** Resolution happens entirely in the process
(musl); nothing in the kernel can resolve a name.
- **`httpd` serves HTTP/1.0** with `Connection: close`, one request per connection,
and has no timeouts: a client that connects and says nothing ties up a child
process until it goes away, and `SO_RCVTIMEO` does not exist here.
- **A queued-but-never-accepted connection is closed without clearing its
callbacks.** `inet_close()` clears the argument and closes, which is enough
because the pcb is freed — but it is a narrower guarantee than the one the
main path makes, and it relies on `tcp_close()` succeeding.
- **`httpd` has no timeouts.** A client that connects and says nothing ties up
a child process until it goes away; `SO_RCVTIMEO` does not exist here (see
below), so there is nothing to set.
- **No routing table**, and no `/proc/net/route` to report one. With two
interfaces up, the first is the default route and the second is reachable only
by an address on its own subnet. That is a rule rather than an accident, and it
is not routing. The missing *file* is the more immediately visible half: it is
how a program asks which interface carries the default route, and without it
anything that filters on "the default one" sees nothing.
- **Netlink serves two message types.** No `RTM_GETROUTE`, no `RTM_NEWROUTE`, no
event stream, no multicast groups. A program that expects to *subscribe* to
link changes gets a socket that answers dumps and never speaks again.
- **No `getsockname`/`getpeername`**, and `accept()` does not report the peer's
address for an AF_INET socket. A server that wants to log who connected cannot.
- **`setsockopt` covers six options**, and refuses the rest with `ENOPROTOOPT`
rather than pretending: `SO_REUSEADDR`, `SO_KEEPALIVE`, `SO_BROADCAST` and
`TCP_NODELAY` reach lwIP's pcb; `SO_SNDBUF`/`SO_RCVBUF` are accepted and
report the real size on read. Notably absent: `SO_RCVTIMEO`/`SO_SNDTIMEO`.
because the pcb is freed, but it is a narrower guarantee than the main path
makes and relies on `tcp_close()` succeeding.
- **No routing table**, and no `/proc/net/route`. With two interfaces up, the first
is the default route and the second is reachable only by an address on its own
subnet. That is a rule rather than an accident, and it is not routing.
- **Netlink is a responder**: `RTM_GETLINK` and `RTM_GETADDR` dumps (what
`getifaddrs` needs), `MSG_PEEK|MSG_TRUNC` sizing (GLib's network monitor measures
each datagram that way and spun at 100% when it answered 0), and nothing else. No
`RTM_GETROUTE`, no event stream, no multicast groups; a program that expects to
*subscribe* to link changes gets a socket that answers dumps and never speaks
again.
- **`setsockopt` applies what it understands** (`SO_REUSEADDR`, `SO_KEEPALIVE`,
`SO_BROADCAST`, `TCP_NODELAY` reach lwIP's pcb; `SO_SNDBUF`/`SO_RCVBUF` report the
real size; `SO_PASSCRED` on Unix sockets) and refuses the rest with `ENOPROTOOPT`.
Notably absent: `SO_RCVTIMEO`/`SO_SNDTIMEO`.
- **`SOCK_DGRAM` over AF_UNIX does not exist** — stream only.
- **AF_UNIX names are not filesystem paths**, so nothing can `stat()` or
`unlink()` a socket.
- **One NIC driver exists.** The class is proven with two e1000 cards, not with
two different drivers.
- **No IPv6**, no DNS resolver in the kernel (lwIP's is compiled but unused by
anything), and no firewall or packet filter of any kind.
- **No IPv6**, no firewall or packet filter of any kind, and a TCP stack that has
been used for downloads at tens of megabits and no more than that.
- **Three drivers in the kernel, one Linux driver outside it**, and the class is
proven with two e1000 cards and an LDLS handover, not on a wide range of
hardware. The Realtek parts have never carried a packet on real silicon from this
tree.

102
docs/architecture/packages.md Normal file
View file

@ -0,0 +1,102 @@
# Packages: an apk root
Alpine is musl with a different init, and SchOS is musl with a different kernel.
Alpine's binaries run on SchOS unmodified — its dynamic linker, its busybox, and
`apk` itself, fetching over TLS through SchOS's network stack. So instead of
porting programs one at a time, SchOS installs them.
## The model
The root filesystem is an apk-managed root:
| Part | Where it comes from |
|---|---|
| musl, busybox, apk-tools, alpine-keys, CA certificates | Alpine's minirootfs (`ports/alpine`) |
| what `ports/alpine/world` lists (zsh, Xorg, Window Maker, htop, NetSurf …) | Alpine's packages, pinned in `world.lock` |
| schinit, its jobs, SchOS's tools, toybox, mksh, configuration | **schos-base**, an apk package the build makes from `sysroot/` |
`apk add` installs anything from Alpine's main and community repositories.
`apk upgrade` updates Alpine's packages, and schos-base from the SchOS repository
at `/var/lib/schos/repo`.
PID 1 is still schinit. Alpine's OpenRC is not installed, and packages that ship
OpenRC services install them where nothing reads them.
## Who owns what
Where Alpine and SchOS both have a program, Alpine's wins, and SchOS keeps only
what is its own (`tools/posix/mkaroot.sh` says so in its header):
- `/bin/sh`, `cat`, `sed`, `find` and the rest are busybox. Alpine's package scripts
are written against busybox.
- zsh is Alpine's. SchOS keeps its `zshrc` and `zshenv` (`replaces=zsh`).
- `/etc/os-release` and `/etc/motd` are SchOS's (`replaces=`).
- `passwd`, `group` and `profile` are Alpine's. schos-base's post-install sets root's
shell to zsh, and SchOS's profile is `/etc/profile.d/schos.sh`.
- `pci.ids` and `usb.ids` are Alpine's (`hwdata-pci`, `hwdata-usb`): Xorg depends on
`hwdata-pci`, and two owners of one file is an error apk will not install past.
- mksh is `/bin/mksh`. schinit's recovery shell looks for it first because it is static:
a broken musl cannot take recovery down with it.
## Building
make qemu boots build/schfs.img, the root as a 1 GB SchFS disk
make iso the same root as the live system's compressed ramdisk
make alpine-lock re-resolve world against Alpine (needs the network)
It is the only root; `sysroot/` is what schos-base is packaged from. Only
`alpine-lock` asks Alpine what is current. A normal build installs exactly what
`world.lock` names — files in `dist/alpine/`, fetched once and checked against the
lock's sha256, and against Alpine's signatures by apk — and never asks what is newer.
`world.lock` resolves the handful of names in `world` into a few hundred files, every
dependency included, so every image after the first is reproducible.
schos-base and its repository index are signed with a key made on the first build
(`build/apk-keys/`, never committed). The root trusts it beside Alpine's keys. The
install runs as an ordinary user with `--no-scripts`, because apk's scripts want a
chroot; `mkaroot.sh` does by hand the one change schos-base's post-install would
have made.
## What it took from the kernel
All of it found by running apk, and none of it specific to apk:
- **TCP receive window:** it reopened before the reader drained, so lwIP refused data
and the sender waited on retransmission timers (5 KB/s). It reopens on read now, with a
32-segment window; downloads run at tens of megabits.
- **`mmap` address reuse:** freed ranges were never reused, so musl's allocator used up the
256 mapping records with 780 MB of RAM free.
- **`unlinkat` / `renameat`** returned a bare `-1`, read as `EPERM`, where apk expects
`ENOENT`; `renameat` could not replace its target.
- **`#!` scripts in `execve`**, and `ENOEXEC`: apk runs triggers directly.
- **`chroot`**, and mounting devtmpfs and proc more than once.
- Later, as bigger programs arrived: dynamic linking, threads and `futex`, demand paging,
pseudo-terminals, `inotify`, `flock`, Unix-socket credentials. Each is in
[kernel.md](kernel.md) or [syscalls.md](syscalls.md) because a package asked for it.
## The live system
The live ISO runs the apk root from RAM. The ramdisk is the compressed image
([scff.md](scff.md)); at boot the volume grows to half of the free memory (4 GB of root on
an 8 GB machine), backed by pages as they are written. Xorg and its dependencies are in the
image already; anything else installs into that space in about a minute, over the network,
and **is gone at reboot**.
## What it does not do
- **No installer.** Nothing is written to the machine's disks, so packages do not survive
a reboot. Keeping them needs an installer that writes the root to a SATA or NVMe disk,
and a kernel that finds its root there by the SchFS GPT type rather than as IDE disk 1.
- **Not every Alpine package runs.** The ones in `world` were chosen *because* they run:
terminal programs that draw and poll (htop, btop, tmux, mc), X clients, SDL 2 games,
NetSurf on GTK 3. Others hit whatever the kernel does not do yet; Epiphany (WebKit) is
one — its web-process child dies at start-up, and it has not been run to ground.
A package is one `apk add` away, and finding what it needs is how most of the kernel's
Linux surface got written.
- **Fortified libc is strict, and Alpine's packages are not always correct.** `lbreakout2`
hands `vsnprintf` a size larger than its buffer and traps on Alpine too; `world` carries
`lbreakouthd` instead.
- **apk's scripts and triggers do not run at build time** (`--no-scripts`), so a package
whose script matters has to have it done by hand in `mkaroot.sh`.
- **OpenRC services do nothing.** Anything that expects an init script to start it has to
be given a schinit job ([init.md](init.md)).

41
docs/architecture/syscalls.md
View file

@ -74,19 +74,19 @@ machine off, and Linux number 48 is the socket call. Two syscalls cannot share a
## What is implemented
Roughly 190 native numbers covering:
About 215 native numbers (193 of them with a Linux number) covering:
| Area | |
|---|---|
| Process | fork, exec, exit, wait/waitpid/wait4/waitid, clone, kill, sessions, process groups |
| Process | fork, exec, exit, exit_group, wait/waitpid/wait4/waitid, clone (processes and threads), futex, kill, tgkill, sessions, process groups |
| Identity | the full real/effective/saved set for both uid and gid, groups, umask |
| Memory | mmap, munmap, mprotect, brk/sbrk, madvise, System V shared memory |
| Memory | mmap, munmap, mremap, mprotect, brk/sbrk, madvise, System V shared memory, memfd |
| Filesystem | the ordinary set, the whole `*at()` family, statfs, xattr, mount/umount2 |
| I/O | read/write, readv/writev, pread/pwrite, sendfile, poll/select/ppoll/pselect6, epoll (create, create1, ctl, wait, pwait), ioctl, fcntl |
| Sockets | socket, bind, listen, accept, connect, sendto, recvfrom, socketpair, setsockopt, getsockopt |
| Sockets | socket, bind, listen, accept/accept4, connect, sendto/recvfrom, sendmsg/recvmsg, socketpair, getsockname/getpeername, setsockopt/getsockopt, over AF_UNIX, AF_INET and AF_NETLINK |
| Signals | rt_sigaction, rt_sigreturn, rt_sigsuspend, rt_sigpending, rt_sigtimedwait, sigaltstack |
| Time | gettimeofday, clock_gettime, clock_getres, nanosleep, clock_nanosleep, alarm, timers |
| Notification | eventfd, eventfd2, timerfd_create/settime/gettime, signalfd, signalfd4 -- see `kernel/sys/sys_event.c` |
| Notification | eventfd, timerfd, signalfd, inotify -- see `kernel/sys/sys_event.c` |
### Argument fetching
@ -113,20 +113,19 @@ field by field, each through its own translation, or use `copyin`/`copyout`.
## What it does not do
- **`sendmsg`/`recvmsg` carry `SCM_RIGHTS` and nothing else.**
`SCM_CREDENTIALS` is refused rather than ignored. See
[networking.md](networking.md#descriptor-passing).
- **No `memfd_create`.**
- **No `getsockname` or `getpeername`**, and `accept()` does not report an
AF_INET peer's address.
- **`setsockopt` handles six options** and answers `ENOPROTOOPT` for everything
else. `SO_RCVTIMEO`/`SO_SNDTIMEO` are among the refused.
- **`clone` does not share an address space**, so it is `fork` with extra steps.
- **`sendmsg`/`recvmsg` carry `SCM_RIGHTS` and `SCM_CREDENTIALS`**, and nothing
else. Credentials are delivered only to a socket that asked with `SO_PASSCRED`
(D-Bus and WebKit do); other control messages are refused rather than ignored.
See [networking.md](networking.md#descriptor-passing).
- **`setsockopt` applies the options it understands** — they reach lwIP's pcb or
the Unix socket — and answers `ENOPROTOOPT` for everything else, never a
success that did nothing. The options that take a `struct` (`SO_RCVTIMEO`,
`SO_SNDTIMEO`) are among the refused.
- **`sched_setaffinity` is accepted and ignored.**
- **`getrusage` does not distinguish `RUSAGE_CHILDREN`** from `RUSAGE_SELF`, and
fills in only the calling process's own tick count.
- **The ABI is chosen by link address.** It works, and it is a heuristic rather
than a fact about the binary.
- **`/proc/<pid>/exe` is a regular file, not a symlink**, so `readlink()` on it
fails and `ps -o comm` prints `[init]` rather than `init`. Default `ps` and
`top` output is unaffected.
- **The ABI is chosen by link address.** Anything linked at `0x100000` or above
— musl's programs, and every dynamically linked one, whose interpreter is
placed at `0x40000000` — gets the Linux map; a first-party program linked at
`0x4000` gets the native one. It works, and it is a heuristic rather than a
fact about the binary.
- **`ptrace` and `seccomp`** are not implemented; a program that needs them
gets `ENOSYS`.

62
docs/architecture/terminals.md
View file

@ -95,7 +95,9 @@ nothing to choose between. It records `proc->ctty` now, which is what
|---|---|---|
| `/dev/console` | 1 | the **active** terminal, so kernel messages follow a switch |
| `/dev/tty` | 10 | the caller's **controlling** terminal, via `proc->ctty` |
| `/dev/tty1`..`tty6` | 11-16 | that terminal, whether or not it is on screen |
| `/dev/tty0` | | the active terminal, which is what Xorg and `openvt` ask for |
| `/dev/tty1`..`tty12` | 11-22 | that terminal, whether or not it is on screen |
| `/dev/ptmx`, `/dev/pts/N` | | pseudo-terminals, below |
`/dev/tty` was an alias for `/dev/console` with the same major and minor. They
are different questions and now have different answers.
@ -190,13 +192,13 @@ every candidate.
|---|---|
| **The host takes it, and the guest never sees it** | |
| `Alt+F4`, `Alt+F2` | window manager |
| `Ctrl+Alt+F1..F6` | display manager — and it matches **loosely**, so adding Shift does not get past it |
| `Ctrl+Alt+F1..F12` | display manager — and it matches **loosely**, so adding Shift does not get past it |
| `Ctrl+Alt+`digit | QEMU's own console tabs |
| **Applications want it, and lose it permanently if the kernel eats it** | |
| `Alt+`anything | Meta. zsh binds Meta-digit as `digit-argument`; so does every readline editor |
| `Ctrl+`letter | control characters |
| `Ctrl+Shift+`digit | types a character — `Ctrl+Shift+2` is `^@` |
| `F1..F6` alone | help, and whatever a full-screen program wants |
| `F1..F12` alone | help, and whatever a full-screen program wants |
| **The keyboard layout wants it** | |
| `AltGr+`letter | Polish diacritics — ą ć ę ł ń ó ś ź ż — and on some setups AltGr arrives as Ctrl+Alt |
@ -207,7 +209,7 @@ chvt # report the active terminal and the current chord
chvt -c ctrl+alt+F # Ctrl+Alt+F1..F12 (the default)
chvt -c altgr+F # AltGr+F1..F12 (the default until 2026-10)
chvt -c ctrl+shift+F # Ctrl+Shift+F1..F12
chvt -c alt+1 # Alt+1..6
chvt -c alt+1 # Alt+1..9
chvt -c off # no chord; chvt N is then the only way
```
@ -368,34 +370,56 @@ and erase characters — were added at the same time, because `TERM=linux`
declares them and a terminal that drops them silently is the same class of bug
waiting for a different editor.
## Handing a terminal to a program
A display server takes a terminal with `VT_SETMODE(VT_PROCESS)` and `KDSETMODE(KD_GRAPHICS)`,
and from then on switching away from it is a **request**: the kernel sends the owner
`relsig` and waits for `VT_RELDISP`, so the server stops drawing and only then says
yes; switching back sends `acqsig` and the server redraws. A terminal in graphics mode
is not repainted by the kernel. If the owner dies, the terminal goes back to the kernel
whole. This is how Xorg runs on a terminal of its own, with Ctrl+Alt+F-key switching
between it and the text consoles and Ctrl+Alt+Backspace ending the session.
## Pseudo-terminals
`/dev/ptmx` makes a pair on every open and hands back the master; the slave is
`/dev/pts/N`, which the program on the other side gets as its stdin, stdout and
controlling terminal. Between them is a line discipline, the console's in miniature:
what the master writes is the slave's input (canonical editing, echo, `^C` as `SIGINT`
to the foreground group) and what the slave writes is read back by the master with NL
turned into CR NL. xterm, tmux, `script` and `sshd` all need exactly this. The pairs
share one lock; `kernel/dev/pty.c`.
## The text cursor
A steady inverted block — the cell with its colours swapped relative to what it would
show, so a cursor on reverse-video text is the plain text and every cursor on screen
is visible against its cell. `ESC[?25l` hides it, `ESC[?25h` shows it. It is drawn into
the grid's repaint and does not blink.
## What it does not do
- **No scrollback.** The grid is exactly the visible screen. Shift+PageUp does
nothing because there is nothing above the top row to show.
- **No reflow on resize.** The geometry is read once at `vt_init()`. Nothing
resizes a framebuffer console today; a compositor will, and that is when this
has to grow.
- **No `VT_RELDISP` handshake.** `VT_PROCESS` stops the kernel switching, but
the kernel does not signal the owning process and wait for an answer. A
display server that needs to release the GPU *before* the screen changes
hands will need that, and `struct vt_mode` already has the fields.
- **No terminal past 6.** Linux allows 63. Six is what the chords reach and
what the grids are sized for.
- **No reflow on resize.** The geometry is read once at `vt_init()`; the console's
font is sized to the screen at that point, and a resolution change from `schmode`
repaints it, but text is not re-wrapped.
- **No terminal past 12.** Linux allows 63. Twelve is one per function key.
- **Only 240x75 of any screen.** Larger framebuffers work; the grid is clamped
and the remainder stays background. Raising `VT_MAXCOLS`/`VT_MAXROWS` is the
fix, and costs BSS.
- **No UTF-8 the font lacks.** The console draws through the TrueType
rasteriser now, so the limit is the font's `cmap` rather than a 256-entry
table — Latin, Latin Extended, Greek, Cyrillic and the common punctuation and
maths symbols all render. Anything genuinely absent still draws as `?`, which
is deliberate: drawing a *different* character would be worse.
rasteriser, so the limit is the font's `cmap` — Latin, Latin Extended, Greek,
Cyrillic and the common punctuation and maths symbols all render. Anything
genuinely absent still draws as `?`, which is deliberate: drawing a
*different* character would be worse.
- **No shaping, and no wide characters.** One codepoint is one cell of one
advance. Combining marks land on top of their own cell rather than the
preceding one, and CJK draws at half the width it wants. Both need a
width table and a cell that can span two columns.
- **No dead keys**, so French and Spanish layouts are absent rather than
broken. See above.
broken.
- **No per-terminal layout.** One keyboard, one layout. Switching to tty3 does
not move anybody's fingers.
- **Only three layouts.** Adding one is a table in `keymap.c` and a line in the
- **Only three layouts** (US, Polish programmers', German). Adding one is a table in `keymap.c` and a line in the
catalogue; the mechanism is not the limit, the typing is.

10
docs/architecture/usb.md
View file

@ -21,9 +21,10 @@ The specification is public and the controller is not complicated.
> BSD is much more valuable as the reference you check yourself against than as
> the code you paste.
Licensing points the same way: the BSD stacks are BSD-2/3-clause and compatible,
Linux's drivers are GPL-2.0-only and are not — see
[../licensing.md](../licensing.md).
Licensing does not decide it: the BSD stacks are BSD-2/3-clause and compatible, and
since SchOS became GPL-2.0-or-later Linux's GPL-2.0-only drivers are compatible too
— see [../licensing.md](../licensing.md). The argument against porting is the
framework underneath, not the licence.
## What the spec gives you for free
@ -129,7 +130,8 @@ working one.
filesystem's next read, by the rule in `blkdev_rw()`: a filesystem that went on
with missing blocks would be guessing.
- **HID is keyboards and pointers.** Consumer-control interfaces (volume keys),
game pads and the rest are declined. Keyboard LEDs are not driven.
game pads and the rest are declined. Caps, Num and Scroll Lock light their LEDs
on USB and PS/2 keyboards, one state shared between keyboards.
- **USB 3 hubs** are not driven: the hub driver speaks the USB 2 hub protocol.
- **No isochronous transfers**, so no audio and no cameras.
- **No USB 3 SuperSpeed-specific features** beyond what enumeration needs — the