nexoniarz/SchOS
nexoniarz/SchOS
Watch
1
0
Fork
You've already forked SchOS
 0
Code Issues Pull requests Projects Releases 1 Packages Wiki Activity Actions

docs: README, plan, guides and the site describe what is built now

Browse source 
README: rewritten around what runs (Linux software, Xorg, apk, Linux's drivers), the honest gaps, measured numbers.
plan/: 1.0.md is the finish line, where each clause of the acceptance test stands, and the register (one failing
clause: a root that survives a reboot); built plans (foundations, hybrid, alpine, xorg) and the decided-against one
(wayland) are gone, their content lives in architecture/; roadmap.md is only what is still open.
guides: building/layout/testing/adding-a-driver (Linux code may be used). history: four entries (ls -lR, TLB
shootdowns, MDIO 0x02460246, netlink MSG_TRUNC). Website: cards, stack, download page, docs index.
Code comments that pointed at deleted plan docs point at the architecture docs.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EVRhxi9zpdW2uDUN4FVcao
This commit is contained in:
Nexoniarz 2026-10-26 08:08:05 +01:00
commit cb3d28e3d6
57 changed files with 685 additions and 2121 deletions
Download patch file Download diff file Expand all files Collapse all files

31
docs/guides/adding-a-driver.md
View file

@ -12,12 +12,20 @@ virtio, NVMe and the Intel datasheets are public documents with no licence on
what you write from them. `kernel/drivers/bus/xhci/xhci.c` came from the spec,
and so did `net/e1000/` and `gpu/virtio_gpu/`.
**BSD: yes. Linux: no.** This is a licence incompatibility rather than a
preference, and it decides the whole strategy — see
[../licensing.md](../licensing.md). BSD-2 and BSD-3 code can be used directly
with a `NOTICE` entry; GPL-2.0-only code can never be, at any distance. Reading
Linux to understand *hardware* is fine and normal; translating a Linux driver
function by function produces a derivative work whatever the identifiers say.
**Three sources, in the order to try them:**
1. **A native driver, written from the specification.** The default, and what every driver in the
kernel is.
2. **Linux's own driver, unmodified, through LDLS** ([ldls.md](../architecture/ldls.md)), when the
device is one Linux supports and SchOS cannot afford to write a driver for — a NIC with a PHY
behind it, a GPU, a Wi-Fi card. It runs in a process, so a crash costs the interface and not the
machine. This is how the laptop's Realtek NIC is driven.
3. **BSD or Linux code taken into the tree.** Licensing no longer forbids either: SchOS is
GPL-2.0-or-later, so GPL-2.0-only code combines with it, and BSD-2/3 code can be used directly with
a `NOTICE` entry — see [../licensing.md](../licensing.md). The reason to prefer 1 and 2 is
maintenance, not law: a driver copied in is a driver that has to be tracked.
Reading another kernel to understand *hardware* is fine and normal.
One caveat about "port it from BSD" as a plan, learned by looking: FreeBSD's
`sys/dev/usb` is roughly 100k lines welded to newbus, devclass, `bus_dma`,
@ -45,7 +53,9 @@ ours.
kernel/drivers/
registry.c
gpu/vga/ gpu/virtio_gpu/
net/e1000/
net/e1000/ net/rtl8139/ net/r8169/
storage/ahci/ storage/nvme/
audio/ac97.c
bus/xhci/
usb/core/ usb/hid/ usb/storage/ usb/hub/
@ -56,11 +66,12 @@ ours.
| class | vtable | consumer |
|---|---|---|
| `DEVCLASS_DISPLAY` | `struct display_ops` ([display.h](../../include/display.h)) | the compositor |
| `DEVCLASS_NET` | `struct netdev_ops` ([netdev.h](../../include/netdev.h)) | the TCP/IP stack |
| `DEVCLASS_DISPLAY` | `struct display_ops` ([display.h](../../include/gfx/display.h)) | none today — Xorg draws through `/dev/fb0` |
| `DEVCLASS_NET` | `struct netdev_ops` ([netdev.h](../../include/dev/netdev.h)) | the TCP/IP stack |
| `DEVCLASS_AUDIO` | `struct audio_ops` | `/dev/dsp` |
A display driver that owns a linear framebuffer also publishes it with
`gfx_attach_scanout()` ([gfx.h](../../include/gfx.h)). That is what the text
`gfx_attach_scanout()` ([gfx.h](../../include/gfx/gfx.h)). That is what the text
console and the crash screen draw on, and it is separate from the display
class on purpose: the class is about *frames* — acquire, present, vblank —
and the crash screen has to work when nothing else in the system does.

18
docs/guides/building.md
View file

@ -8,8 +8,15 @@ change that only works on one of them is a bug.
```sh
make # kernel + UEFI loader + root filesystem
make qemu # build and boot it
make iso # a bootable ISO (the live system) for VMware or real hardware
./write.sh sdX # write that ISO to a USB stick (Linux; it checks what it is writing over)
```
The first build downloads what it needs (musl, toybox, Alpine's packages -- a few
hundred megabytes, kept in `dist/`) and compiles Xorg's dependencies not at all: the
packages are Alpine's binaries, pinned in `ports/alpine/world.lock`. See
[packages](../architecture/packages.md).
## Prerequisites
| Need | Why | Windows | Linux | macOS | BSD |
@ -17,7 +24,7 @@ make qemu # build and boot it
| `clang`, `ld.lld`, `llvm-objcopy`, `llvm-objdump` | cross-compiles the kernel, the 16/32-bit boot code, and musl userland | LLVM installer | `clang lld llvm` | `brew install llvm` | `pkg install llvm` |
| `make` | — | Git Bash | `make` | Xcode CLT | `gmake` |
| `perl` | generates the interrupt vector table | Git Bash | usually present | present | `pkg install perl5` |
| `python3` | boot logo, crash-screen QR | python.org (`python`) | `python3` | `brew install python` | `pkg install python3` |
| `python3` | boot logo, crash-screen QR, the numbers in the README | python.org (`python`) | `python3` | `brew install python` | `pkg install python3` |
| `qemu-system-x86_64` | running and testing | `scoop install qemu` | `qemu-system-x86` | `brew install qemu` | `pkg install qemu` |
| `curl`, `tar` | fetching ports | Git Bash | present | present | present |
| `xorriso` **or** `mkisofs` | `make iso` only | `scoop install cdrtools` | `xorriso` | `brew install xorriso` | `pkg install xorriso` |
@ -81,10 +88,11 @@ produce.
## Ports
`musl`, `toybox` and `mksh` are built from source into `sysroot/`. **This does
not work on a Windows host** — `make` prints `ports: skipped` and uses whatever
is already staged in `sysroot/`. Building them needs a Linux host (or any other
Unix); do it once and the staged result serves the Windows builds.
`musl`, `toybox`, `mksh`, lwIP, uACPI, zstd, ncurses and the rest of `ports/` are built from source
into `sysroot/`; Alpine's packages are unpacked into the root from `dist/alpine/`. **This does not work
on a Windows host** -- `make` prints `ports: skipped` and uses whatever is already staged in
`sysroot/`. Building them needs a Linux host (or any other Unix); do it once and the staged result
serves the Windows builds.
## Where the tools live

134
docs/guides/layout.md
View file

@ -14,47 +14,46 @@ deliberately kept out of `clean`).
schos/
├─ boot/ uefi/ the PE/COFF loader (splash, F8 menu, kernel load),
│ common/ the UI it shares. UEFI only -- there is no BIOS path.
├─ kernel/ The kernel, by subsystem: arch/ core/ dev/ fs/ proc/ sys/
├─ kernel/ The kernel, by subsystem: arch/ core/ dev/ drivers/ fs/ net/
│ proc/ sys/ acpi/ (uACPI glue, the EC, I2C-HID) gmmos/ (the
│ micro OS under the crash screen) scff/ (the decompressor).
│ drivers/ is the in-kernel drivers, by class: audio bus gpu
│ net storage usb.
├─ ldls/ The Linux driver layer: the .ko loader, the BTF reader, the
│ Linux symbols reimplemented (kpi/), and host-side test tools.
│ The process that runs it is cmd/ldlsd.
├─ include/ Headers shared by kernel, userland, and tools/mkfs, by
│ subsystem -- see "include/" below
├─ lib/ First-party userland runtime: _start glue, syscall stubs,
│ the initcode blob embedded in the kernel, user.ld
├─ cmd/ First-party programs — one directory per program:
│ cmd/<name>/main.c → sysroot/bin/<name>
│ schinit and schinitd are PID 1 and its supervisor; vgpud is
│ the virtio-gpu driver, which runs out here rather than in
│ the kernel; socktest/shmtest/nettest/pcitest are suites.
├─ ports/ Third-party software — one directory per port.
│ Contains ONLY a Makefile and optional patches/.
├─ projects/ Things that run ON SchOS rather than things that ARE it:
│ the compositor, window manager, desktop, and installer.
│ schinit and schinitd are PID 1 and its supervisor; vgpud and
│ ldlsd are drivers that run out here rather than in the
│ kernel; socktest/shmtest/nettest/pcitest/xready are suites.
├─ etc/ The configuration the image starts with: schinit's job
│ manifests (etc/schinit/*.job), motd, os-release, the zsh files.
├─ ports/ Third-party software -- one directory per port, containing
│ ONLY a Makefile and optional patches/. musl, toybox, mksh,
│ lwIP, uACPI, zstd, ncurses, the fonts, fastfetch, kilo; and
│ alpine/, which is the apk root (world, world.lock).
├─ projects/ Things that run ON SchOS rather than things that ARE it.
│ Not part of the kernel build. See projects/README.md.
├─ tools/ Host-side helpers, SOURCE ONLY: mkfs.c, the SchFS test
│ suites, sign.pl, vectors.pl, and the PowerShell harnesses.
│ Their binaries build into build/tools/ -- this directory
│ used to hold both, which is the one rule this layout has.
│ tools/windows/loc.ps1 counts how much of the tree is first-party.
├─ sysroot/ Staging for the disk image. etc/ is hand-maintained;
│ bin/ lib/ sbin/ are populated by `make sysroot-install`.
├─ dist/ Upstream tarball cache. Survives `make clean`.
├─ Refrence/ A FreeBSD checkout (Refrence/freebsd-src), kept as reference
│ material and a parts bin. Never built, never in an image.
│ Read it when a specification is ambiguous; copy from it only
│ after the per-file licence check in docs/licensing.md.
│ A full clone is 2.1 GB and a previous NetBSD one was deleted
│ for that reason. Only sys/dev, sys/net and sys/netinet have
│ ever been read, so a sparse checkout is ~100 MB:
│ git clone --filter=blob:none --sparse <url> Refrence/freebsd-src
│ git -C Refrence/freebsd-src sparse-checkout set │ sys/dev sys/net sys/netinet
├─ src-trunk/ A NetBSD checkout, when present -- reference material and a
│ parts bin. NOT built, NOT installed, must never reach an
│ image. Currently absent; it was deleted for disk space and
│ can be re-fetched sparsely. Copying from it is governed by
│ the per-file licence check in docs/licensing.md §3 --
│ NetBSD is not uniformly licensed.
├─ tools/ Host-side helpers, SOURCE ONLY: mkfs.c, mkschfs.c, mkscff.c,
│ the SchFS test suites, vectors.pl, and the harnesses, grouped
│ by what runs them (windows/ .ps1, posix/ .sh, python/ .py).
│ Their binaries build into build/tools/. tools/windows/loc.ps1
│ and tools/python/stats.py count how much of the tree is
│ first-party.
├─ sysroot/ Staging for the disk image, GENERATED by the build -- edit
│ tools/posix/mkaroot.sh and etc/, not this.
├─ dist/ Upstream tarball and package cache. Survives `make clean`.
├─ shell.nix NixOS development shell: the whole toolchain in one file
├─ write.sh Writes schos.iso to a USB stick, safely (Linux).
├─ run.ps1 The Windows-side QEMU launcher.
├─ docs/ architecture/ what it is plan/ what's next
│ guides/ how to work history/ why it's like this
│ Website/ the site, and the docs reader that renders this
└─ build/ ALL generated output (see below)
```
@ -92,7 +91,10 @@ build/
├─ kernel/ object files, the linked kernel, kernel.asm
├─ lib/ libc glue objects, initcode blob
├─ cmd/ linked first-party programs
├─ fs.img the filesystem image
├─ aroot/ the apk root the image is made from
├─ schfs.img the root as a disk image (make qemu)
├─ fs-ram.img the same root as the live ramdisk, fs-ram.scff compressed
├─ iso/ … what make iso assembles
└─ ports/<name>/
├─ src/ extracted upstream tarball
├─ obj/ out-of-tree build directory
@ -130,57 +132,43 @@ Local modifications to upstream go in `ports/foo/patches/*.patch` and are
applied after extraction — never by editing files under `build/`, which are
wiped on every clean.
## Ramdisk root
## The live root
```sh
make # normal: root on the IDE disk (build/fs.img as hdb)
make ramroot # root from a SCHFS image linked into the kernel
make # normal: the root is a SchFS disk image, build/schfs.img
make iso # the same root as a compressed ramdisk linked into the kernel
make ramroot # the ramdisk, uncompressed
```
`make ramroot` builds `build/fs-ram.img` (default 6000 blocks, override with
`RAMDISK_BLOCKS=`) and links it into the kernel, so the system boots to a shell
with **no data disk attached at all**. That is what makes destructive work on
the real disk survivable — reformatting it, or bringing up a new on-disk format
— and it is the first piece of the installer.
`make iso` builds the root as a SchFS image, compresses it
([SCFF](../architecture/scff.md)) and links it into the kernel, so the system boots
to a shell with **no data disk attached at all** -- which is what the USB stick is.
It is also what makes destructive work on a real disk survivable. Nothing written
to a live root survives a reboot.
Use `make ramroot`, not `make RAMROOT=1`: the flag changes `CFLAGS`, which make
cannot see, so objects compiled for one setting would be silently reused for
the other and `ROOTDEV` would point at the wrong device. The target drops the
compiled objects first. Switching back is the same — plain `make` after a
`ramroot` build reuses stale objects, so run `rm -rf build/kernel build/lib
build/cmd` first (or just `make clean`).
The image costs boot time: the loader reads the kernel through the firmware's
file protocol, so a 6 MB ramroot kernel is visibly slower to start than the
350 KB default.
Use the targets, not `make RAMROOT=1`: the flag changes `CFLAGS`, which make cannot
see, so objects compiled for one setting would be silently reused for the other and
`ROOTDEV` would point at the wrong device. The targets drop the compiled objects
first. Going back to the disk root is the same -- run `make clean` or remove
`build/kernel build/lib build/cmd` first.
**What may go in it is a licensing question, not only a size one.** The image is
linked into the kernel binary, so anything in `sysroot/` when it is built becomes
part of a distributable artifact. A file staged there under a licence that
is not ours to redistribute does not merely ride along — it makes the kernel
itself undistributable. `RAMDISK_EXCLUDE` in the Makefile is the list of
paths kept out, and enforcing it in the build rather than relying on someone
remembering to unstage something first is deliberate. It has been needed once
already: a 33 MB test payload landed in `sysroot/` and both overflowed the image
and would have gone into the kernel binary.
part of a distributable artifact. A file staged there under a licence that is not
ours to redistribute does not merely ride along -- it makes the kernel itself
undistributable. `RAMDISK_EXCLUDE` in the Makefile is the list of paths kept out,
and enforcing it in the build rather than relying on someone remembering to unstage
something first is deliberate. (Linux modules for LDLS are staged by `make
ldls-stage`/`ldls-r8169` only on request, and `.gitignore` refuses `*.ko`.)
Three knobs, all in the Makefile, none of them `FSSIZE`:
The knobs: `RAMDISK_BLOCKS` (image size), `RAMDISK_NINODES` (the inode table is
*preallocated*: the default of 12000 once cost a quarter of a small image to
describe about 260 files), `RAMDISK_EXCLUDE`. If `mkfs` reports the image is full,
read its message: it prints the metadata breakdown and names the right knob.
| | |
|---|---|
| `RAMDISK_BLOCKS` | image size, 6000 (× 1 KiB `BSIZE`) |
| `RAMDISK_NINODES` | inode table size, 2000 |
| `RAMDISK_EXCLUDE` | sysroot-relative paths to omit, with their subtrees |
`RAMDISK_NINODES` is worth knowing about because the inode table is
*preallocated*: mkfs's default of 12000 inodes cost 1501 of the 6000 blocks — a
quarter of the image — to describe about 260 files. At 2000 it costs 251, and
metadata overall fell from 1596 blocks to 345.
If `mkfs` reports the image is full, read its message: it now prints the
metadata breakdown and names the right knob for whichever image overflowed. It
used to say only "increase FSSIZE in param.h", which is a constant that has
nothing to do with the ramdisk root and sends you to the wrong file.
The ISO build is fast on purpose (a few seconds when little changed): it copies with
`cp -p` and `install -C` so unchanged files keep their time stamps, and toggling the
ramdisk does not wipe the tree.
## Two things that will bite you

22
docs/guides/testing.md
View file

@ -12,7 +12,7 @@ reporting that it started.
| | What it runs | What it proves |
|---|---|---|
| `make schfs-check` | Host-native unit suites | The filesystem's logic, in about a second |
| `make schfs-check`, `make kmem-test`, `ldls-inspect`/`ldls-run` | Host-native unit suites and the Linux-module loader run on the build machine | The filesystem's logic, the slab allocator's bookkeeping, and a `.ko`'s layout, in about a second |
| `tools/windows/bootcheck.ps1` | A real boot, driven over the serial console | That the system answers correctly |
| `make crashqr`, `make schcom-test` | A boot, then pixel assertions from the host | That the *screen* is right |
@ -77,6 +77,22 @@ dropped every byte would pass any check made from one side alone.
Under QEMU's user-mode networking the host is `10.0.2.2` seen from inside, and
SLIRP connects to the host's loopback — so the listener binds there.
## Test programs that run on SchOS
Several first-party programs in `cmd/` exist to be run **on** the system and print which of their checks
failed, because only the system can say whether its own interfaces behave. `bootcheck` runs the ones it
can; the rest are run by hand or by a script:
| | Checks |
|---|---|
| `socktest`, `nettest`, `netecho` | Unix and INET sockets, `SCM_RIGHTS` and credentials, `setsockopt`'s refusals |
| `shmtest`, `memfdtest` | System V shared memory; `memfd` mapped by two processes, and one sent over a socket |
| `pcitest` | `/dev/pci`: claim, map, DMA, interrupts, release |
| `evtest`, `locktest`, `sigtest` | the event descriptors; `flock` and record locks; signals |
| `dynhello` | dynamic linking: TLS, the heap, a forked child that still has its libraries |
| `xready` | every kernel call Xorg makes, before Xorg is started — the evdev queries, the fbdev ioctls, the console calls, the `/sys` files |
| `tools/python/ldls_check.py` | what LDLS needs from a kernel's BTF |
## Pixel assertions
```sh
@ -148,10 +164,12 @@ leak.
- **No automated test runs on real hardware.** Everything above is QEMU.
- **SMP is exercised but not stressed** — boots on 1, 2 and 4 processors are
checked; nothing hunts for races.
- **Nothing asserts the desktop by pixel.** `xready` checks the calls Xorg makes; whether Window Maker
and an `xterm` appear on a real screen is looked at in screenshots, not asserted.
- **No fuzzing anywhere**, and no `fsx`/`xfstests` against SchFS. Getting SchFS
under those is the main argument for the FUSE implementation in
[../plan/roadmap.md](../plan/roadmap.md).
- **The SchFS round-trip test needs symlinks and hard links**, so it skips on a
Git Bash host and only runs on Unix.
- **`make ports` does not work on Windows.** musl, toybox and mksh must be built
- **`make ports` does not work on Windows.** musl, toybox, mksh and the rest must be built
once on a Unix host and staged.