nexoniarz/SchOS
nexoniarz/SchOS
Watch
1
0
Fork
You've already forked SchOS
 0
Code Issues Pull requests Projects Releases 1 Packages Wiki Activity Actions

docs: README, plan, guides and the site describe what is built now

Browse source 
README: rewritten around what runs (Linux software, Xorg, apk, Linux's drivers), the honest gaps, measured numbers.
plan/: 1.0.md is the finish line, where each clause of the acceptance test stands, and the register (one failing
clause: a root that survives a reboot); built plans (foundations, hybrid, alpine, xorg) and the decided-against one
(wayland) are gone, their content lives in architecture/; roadmap.md is only what is still open.
guides: building/layout/testing/adding-a-driver (Linux code may be used). history: four entries (ls -lR, TLB
shootdowns, MDIO 0x02460246, netlink MSG_TRUNC). Website: cards, stack, download page, docs index.
Code comments that pointed at deleted plan docs point at the architecture docs.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EVRhxi9zpdW2uDUN4FVcao
This commit is contained in:
Nexoniarz 2026-10-26 08:08:05 +01:00
commit cb3d28e3d6
57 changed files with 685 additions and 2121 deletions
Download patch file Download diff file Expand all files Collapse all files

207
docs/plan/roadmap.md
View file

@ -1,183 +1,120 @@
# Roadmap — everything else still open
The three current decisions have their own documents; see
[README.md](README.md). This file holds what is left: work that is scoped,
agreed and unstarted, with nothing in it that is finished.
[1.0.md](1.0.md) draws the finish line and holds the register of what is half-built.
This file holds what is left beyond it: work that is scoped, agreed and unstarted, with
nothing in it that is finished.
**Finished work is not here.** It moved to
[../architecture/](../architecture/) and was rewritten as a description of what
it produced — see [the rule](../README.md#the-rule-that-keeps-this-from-rotting).
What used to be phases 1 through 4 of this file, several hundred lines of ticked
boxes, is now `usb`, `networking.md`, `graphics.md` and `devices.md` over there.
**Finished work is not here.** It moved to [../architecture/](../architecture/) and was
rewritten as a description of what it produced — see
[the rule](../README.md#the-rule-that-keeps-this-from-rotting).
---
## 1. Storage and the boot path
**Done: a USB disk, a SATA disk and their partitions can be booted from.** The
root is chosen at boot from what is present (`rootdev_mount()` in
`kernel/proc/proc.c`), and `make disk` builds a GPT image with an ESP and a SchFS
partition that a PC boots from a stick or a disk.
A USB disk, a SATA disk, an NVMe SSD and their partitions boot, and the root is chosen at boot from
what is present ([boot.md](../architecture/boot.md)).
NVMe (`kernel/drivers/storage/nvme/`) registers with the same block registry,
so an NVMe SSD boots the same way.
**Next:** an installer that writes that layout to a disk, and interrupt-driven,
queued I/O for AHCI and NVMe.
- **An installer** that writes the layout `make disk` builds (a GPT disk with an ESP and a SchFS
partition) to a machine's disk, so a root survives a reboot. See §5.
- **Interrupt-driven, queued I/O for AHCI and NVMe.** Both poll for completion, one command at a time
per disk: enough to boot and run, not to be fast.
- **An EHCI driver.** USB 2.0 ports wired only to EHCI do nothing; xHCI ports work.
## 2. Graphics
- **A `/dev/dri`-style device with mode setting**, so resolution stops being
whatever the loader asked for. This was to be done *with*
[hybrid.md](hybrid.md)'s GPU eviction, and that eviction has now happened —
`cmd/vgpud` drives virtio-gpu from userspace. The modeset API is the next thing
it makes affordable, and is milestone H3 over there rather than here.
- **Double buffering and vblank.** `display_ops` specifies both and no driver
implements either. Schcom needs both to not tear.
- **Port the console onto the text layer and delete the cell grid.** The layer
exists, the rasteriser exists, and the console still indexes `font_data[256][16]`.
- **Load a font from disk** and point the console at it.
- **CFF/Type 2 charstrings**, so OTF fonts work. A separate `font_ops` provider
and a bytecode interpreter, *not* an extension of the `glyf` path.
- **Text layout** — advance widths, kerning — as a library in `lib/`, so the
terminal and the file manager are not each solving it.
The console keeps its fixed grid until the first item above; proportional text
belongs to the GUI.
- **A `/dev/dri` device with mode setting**, so a real panel's resolution stops being whatever the
loader's menu chose and Mesa has something to open. This is the GPU plan, [gpu.md](gpu.md).
- **Double buffering and vblank**, which `display_ops` specifies and no driver implements; the same plan.
- **CFF/Type 2 charstrings**, so OTF fonts work. A separate `font_ops` provider and a bytecode
interpreter, *not* an extension of the `glyf` path.
- **Text layout** — advance widths, kerning — as a library, so the terminal and the file manager are not
each solving it.
## 3. Networking
Two things the stack owed on the day it landed, both still owed:
- **A routing table** and `/proc/net/route`. With two interfaces up the first is the default route and the
second is reachable only by an address on its own subnet; that is a rule and not routing.
- **Wi-Fi.** The laptop has no other way to be wireless. The route is the same as for the NIC: a Linux
driver (`iwlwifi`, `mt76`…) through LDLS, which needs `nl80211`, a firmware loader that exists, and a
supplicant (`wpa_supplicant` is an Alpine package).
- **IPv6.**
- **A netlink event stream**, for anything that wants to be told a link changed.
- **`accept()` does not report the peer's address**, and there is no
`getsockname`/`getpeername`. Nothing has needed them yet; a server that wants
to log who connected will.
- **No routing table.** With two interfaces up, the first is the default route
and the second is reachable only by an address on its own subnet. That is a
rule rather than an accident, and it is not routing.
## 4. The hardware that makes a laptop a laptop
Two that were on this list are now done and described in
[../architecture/networking.md](../architecture/networking.md): `setsockopt`
honours six options and refuses the rest, and `sendmsg`/`recvmsg` carry
`SCM_RIGHTS`. A third is closer than it was — toybox's `ping` needed `recvmsg`
*and* `IP_RECVTTL`, and only the second is still missing.
## 4. The interactive crash screen
PSOGM renders and stops. The idea is *a small OS inside the crash*, and the
framing decides the design:
> At panic time the kernel is the only thing running, which is exactly why it can
> offer what a live system cannot: walk the stack, read another process's memory,
> dump page tables, inspect the process table — with no scheduler and no locking
> to respect, because nothing else can race you.
What makes it a *system* rather than a print statement is that it keeps the base
machine's drivers. Concretely, the ones that work with interrupts off: the
framebuffer (`gfx_draw_pixel` is a plain memory write), the PS/2 keyboard
(pollable via ports 0x60/0x64), and the serial port. **Those three are a complete
console.**
Everything else — IDE, USB, the timer, anything that sleeps or takes a
sleeplock — is off limits, because the lock it needs may be precisely the one
held by the CPU that just died.
So the shape is a **polled event loop over preallocated state**: no `kalloc`, no
`sleep`/`wakeup`, no `acquire` on anything a live path might hold, one static
command buffer, and that driver set. **That restriction is not a limitation to
work around later; it is the invariant that makes the mode trustworthy when
everything else is not.** Anything wanting more should be answered by rebooting
into a recovery environment.
A useful precedent to read but not copy: DDB, the BSD in-kernel debugger, is
exactly this, and its hard-won rule is the same one — *a debugger that can
deadlock is worse than no debugger.*
- **HDA audio.** AC'97 is what QEMU and old boards have; a laptop built this century has Intel HDA,
which is codec enumeration, CORB/RIRB and stream descriptors. It arrives as a second `struct audio_ops`.
- **Embedded-controller events** (the ACPI SCI), so a lid, a power button on the EC and a battery change are seen.
- **CPU frequency control** (AMD CPPC; Intel HWP/EIST). Idle already reaches `_CST`'s C2; under load the cores run at
the firmware's default clock.
- **Touchpad gestures** — two-finger scroll, palm rejection — on top of I2C-HID's mouse mode.
- **An IOMMU** (VT-d / AMD-Vi), which turns "a driver in a process is isolated against bugs" into "against malice" —
the largest honest caveat on [ldls.md](../architecture/ldls.md).
- **udev's job** — a uevent socket and `/sys/devices` — so X and others are told about devices instead of being
given a list.
## 5. The installer and `.sip`
Scoped in [`projects/schinst/README.md`](../../projects/schinst/README.md),
which holds the detail. In short:
Scoped in [`projects/schinst/README.md`](../../projects/schinst/README.md), which holds the detail. In short:
- [ ] `.sip` format specification, **written down before any code**
- [ ] `tools/mksip` — capture a directory tree into a package
- [ ] The apply path
- [ ] An installation environment that boots from the ISO, partitions, applies
the `.sip`, and installs the loader
- [ ] An installation environment that boots from the ISO, partitions, applies the `.sip`, and installs the loader
`.sip` is modelled on WIM, and the thing worth copying from WIM is that it is
**file-based, not block-based**: one file holds a whole filesystem, applies to a
partition of any size, and several images can share their common files.
`.sip` is modelled on WIM, and the thing worth copying from WIM is that it is **file-based, not block-based**: one
file holds a whole filesystem, applies to a partition of any size, and several images can share their common files.
**Most of this already exists.** `tools/mkschfs.c` and `tools/mkfs.c` both do
content dedup with an `(st_dev, st_ino)` fast path, and SchFS's B-tree is content
addressed. `mksip` should share that code rather than reimplement it — that is
the reason the dedup was factored the way it was.
**Most of this already exists.** `tools/mkschfs.c` and `tools/mkfs.c` both do content dedup with an `(st_dev, st_ino)`
fast path, and SchFS's B-tree is content addressed. `mksip` should share that code rather than reimplement it. And
the live image is already a compressed block image of a SchFS root ([scff.md](../architecture/scff.md)); an installer
may need nothing more than to write that, decompressed, to a partition.
Open questions, all wanting a decision before code:
- **Compression.** Something with a small decompressor is worth more than
something with a good ratio, because the decompressor runs in the installation
environment.
- **Multiple images per `.sip`** — WIM's real trick, install/recovery/boot
sharing one blob store. Worth the complexity now, or not?
- **Can the kernel mount a `.sip` read-only directly?** It is close to a
filesystem already, and if it can, the installation environment's root can
simply *be* the `.sip`.
- **Compression.** Something with a small decompressor is worth more than something with a good ratio, because the
decompressor runs in the installation environment. (zstd is already in the kernel for SCFF.)
- **Multiple images per `.sip`** — WIM's real trick, install/recovery/boot sharing one blob store. Worth the complexity
now, or not?
- **Can the kernel mount a `.sip` read-only directly?** If it can, the installation environment's root can simply *be* the `.sip`.
The GUI for this is a Wayland client; see [wayland.md](wayland.md).
The GUI for this is an X client, like everything else on the desktop.
## 6. SchFS beyond SchOS
In the order they should happen.
**~~An on-disk format version~~, and an upgrade path.** The version policy is
done — see [../architecture/schfs.md](../architecture/schfs.md). What remains is
the **upgrade step**, and it cannot be written until there are two formats to
convert between; the first bump is when that becomes real work rather than
speculation.
**An upgrade step.** The format version policy is done ([schfs.md](../architecture/schfs.md)); the upgrade step cannot be
written until there are two formats to convert between, and the first bump is when it becomes real work.
The format is now pinned, which is what the other two items were waiting on.
**A Linux implementation, FUSE first.** Three reasons in order of weight: it can be tested with `fsx` and `xfstests`, which
are far harsher than anything written alongside the code; it carries no licence entanglement, since libfuse is LGPL and used
across a process boundary; and it can reuse the read/write logic already in `tools/mkschfs` and `tools/schfs_fsck` rather than
being written twice and having the two disagree. A kernel module comes only after FUSE passes `xfstests`; SchOS is GPL-2.0-or-later
([../licensing.md](../licensing.md)), so a Linux module is a technical project now and not a licensing one.
**A Linux implementation, FUSE first.** Three reasons in order of weight: it can
be tested with `fsx` and `xfstests`, which are far harsher than anything written
alongside the code; it carries no licence entanglement, since libfuse is LGPL and
used across a process boundary; and it can reuse the read/write logic already in
`tools/mkschfs` and `tools/schfs_fsck` rather than being written twice and having
the two disagree.
A kernel module is a different project, comes only after FUSE passes `xfstests`,
and carries the licence problem [../licensing.md](../licensing.md) describes.
**The missing features**, listed honestly before adding any: no journal replay
across an unclean shutdown beyond what copy-on-write gives for free, no
snapshots, no sparse-file support, and no online `fsck`. **Snapshots are the one
where the existing design already pays for most of the cost** — the B-tree is
copy-on-write, which is most of the machinery; nothing takes or names a snapshot.
**The missing features**, listed honestly before adding any: no journal replay across an unclean shutdown beyond what
copy-on-write gives for free, no snapshots, no sparse-file support, and no online `fsck`. **Snapshots are the one where the
existing design already pays for most of the cost** — the B-tree is copy-on-write, which is most of the machinery; nothing takes
or names a snapshot.
## 7. The kernel's own gaps
Not phases, and each one improves everything above it:
- **A slab or pool allocator.** `kalloc` hands out one page at a time, and every
subsystem written since has invented its own fixed-size table to work around
that. This is the single change that would most improve the rest of the kernel.
- **A page cache**, which is what a general writable `MAP_SHARED` needs.
[foundations.md](foundations.md) delivers a `memfd` instead, which is narrower
and sufficient for the GUI.
- **Symlinks in procfs**, so `/proc/<pid>/exe` can be one and `ps -o comm` stops
printing `[init]`. Belongs with the rest of the VFS work rather than on its own.
- **The `ls -lR` hang** — **not reproducible**, and kept here because the absence
of a result is a result. Instrumented and hunted at length; see Gate 0 of
[1.0.md](1.0.md). Two real defects were found on the way and fixed, neither
proven to be the cause.
- **No garbage collector for descriptors in flight.** An `AF_UNIX` socket passed
over itself and never received leaks the pair. Bounded at thirty-two sockets,
and nothing in the tree does it — but a third-party program could.
- **A page cache**, which is what a general writable `MAP_SHARED` of a file needs. `memfd` is the narrow answer and has been enough.
- **A working `mprotect`, and NX.** `mprotect` is accepted and ignored and every mapping is executable; RELRO, guard pages
and a W^X JIT all want the real thing.
- **Per-CPU run queues** and work stealing, with a load balancer that is not worse than the lock it replaces.
- **A garbage collector for descriptors in flight.** An `AF_UNIX` socket passed over itself and never received leaks the pair;
bounded at thirty-two, and nothing in the tree does it.
- **Kernel-time accounting**, so `ru_stime` is not zero.
- **A contiguous multi-page allocator**, so `struct proc` and `struct socket` can come from somewhere other than fixed tables.
---
## Kept out of scope on purpose
- Anything that needs pushing or publishing. This work stays local.
- TrueType hinting.
- XWayland.
- DRM/KMS as a *kernel* subsystem; the GPU plan serves `/dev/dri` from a process.