57aff40228
docs/ was ten files where a plan, a description, a guide and a bug report all
sat next to each other. It is now architecture/ (what SchOS is), plan/ (what it
will be), guides/ (how to work on it) and history/ (why it is like this), with
an index that states the one rule holding it together: when something is
finished it moves from plan/ to architecture/ and is rewritten as a description
of the thing rather than a record of building it.
A ticked checkbox says the work happened. It does not say what the work
*produced*, and six months later the checkbox is all that is left. The network
stack proved the point: finished, verified end to end, and for a month the only
description of it anywhere was a checkbox four levels of indentation deep.
So roadmap.md went from 534 lines to 181 -- nothing deleted, the done work
graduated. Seven subsystems that were finished and undescribed now have
descriptions: boot, kernel, syscalls, filesystem, networking, graphics, usb,
init. Every one ends with "What it does not do", which is the section that pays;
a document listing only strengths is marketing, and the reader who needs it most
is the one about to trip over a limitation nobody wrote down.
schfs.md still opened with "phases 1 through 3 1/2 are done, phase 4 is next"
long after all five had landed. It is a description now, with the phase record
and the hard-won bugs moved to history/.
Three corrections found on the way, all load-bearing:
* projects/schcom/README.md claimed SCM_RIGHTS was done. There is no
sendmsg/recvmsg in the kernel at all, and an entire GUI plan rested on it.
* The root README said "No TCP/IP" months after the stack landed.
* history/kernel.md sec.7 was marked "in progress" on a problem that process
groups solved.
Also adds plan/1.0.md, which draws a finish line so the open list stops being
infinite, and marks schwm/ and schde/ superseded: in Wayland the compositor is
the window manager, so that split dissolves rather than being abandoned.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
8.8 KiB
Repository layout
Where everything is, and how to add to it.
One rule holds the whole thing together:
A directory contains either hand-written source or generated output. Never both.
Everything generated lives under build/. Delete build/ and you lose nothing
that can't be rebuilt (except upstream tarballs, which live in dist/ and are
deliberately kept out of clean).
schos/
├─ boot/ Stage 1 MBR + stage 2 loader (splash, F8 menu, kernel load)
├─ kernel/ The kernel, by subsystem: arch/ core/ dev/ fs/ proc/ sys/
├─ include/ Headers shared by kernel, userland, and tools/mkfs
├─ lib/ First-party userland runtime: _start glue, syscall stubs,
│ the initcode blob embedded in the kernel, user.ld
├─ cmd/ First-party programs — one directory per program:
│ cmd/<name>/main.c → sysroot/bin/<name>
├─ ports/ Third-party software — one directory per port.
│ Contains ONLY a Makefile and optional patches/.
├─ projects/ Things that run ON SchOS rather than things that ARE it:
│ the compositor, window manager, desktop, and installer.
│ Not part of the kernel build. See projects/README.md.
├─ tools/ Host-side helpers, SOURCE ONLY: mkfs.c, the SchFS test
│ suites, sign.pl, vectors.pl, and the PowerShell harnesses.
│ Their binaries build into build/tools/ -- this directory
│ used to hold both, which is the one rule this layout has.
│ tools/windows/loc.ps1 counts how much of the tree is first-party.
├─ sysroot/ Staging for the disk image. etc/ is hand-maintained;
│ bin/ lib/ sbin/ are populated by `make sysroot-install`.
├─ dist/ Upstream tarball cache. Survives `make clean`.
├─ Refrence/ A FreeBSD checkout (Refrence/freebsd-src), kept as reference
│ material and a parts bin. Never built, never in an image.
│ Read it when a specification is ambiguous; copy from it only
│ after the per-file licence check in docs/licensing.md.
│ A full clone is 2.1 GB and a previous NetBSD one was deleted
│ for that reason. Only sys/dev, sys/net and sys/netinet have
│ ever been read, so a sparse checkout is ~100 MB:
│ git clone --filter=blob:none --sparse <url> Refrence/freebsd-src
│ git -C Refrence/freebsd-src sparse-checkout set │ sys/dev sys/net sys/netinet
├─ src-trunk/ A NetBSD checkout, when present -- reference material and a
│ parts bin. NOT built, NOT installed, must never reach an
│ image. Currently absent; it was deleted for disk space and
│ can be re-fetched sparsely. Copying from it is governed by
│ the per-file licence check in docs/licensing.md §3 --
│ NetBSD is not uniformly licensed.
├─ docs/ architecture/ what it is plan/ what's next
│ guides/ how to work history/ why it's like this
└─ build/ ALL generated output (see below)
build/
build/
├─ boot/ uefi/ loader, common/ UI
├─ kernel/ object files, the linked kernel, kernel.asm
├─ lib/ libc glue objects, initcode blob
├─ cmd/ linked first-party programs
├─ fs.img the filesystem image
└─ ports/<name>/
├─ src/ extracted upstream tarball
├─ obj/ out-of-tree build directory
└─ prefix/ install prefix (musl's headers and libs land here)
Adding your own program
mkdir cmd/hello
$EDITOR cmd/hello/main.c # include "types.h", "user.h", ...
Then add hello to CMDS in the root Makefile. It gets linked against
lib/ and installed to sysroot/bin/hello by make sysroot-install.
Adding a third-party port
mkdir ports/foo
$EDITOR ports/foo/Makefile # start with `include ../mk/port.mk`
port.mk derives everything from the directory name — it gives you $(SRCDIR),
$(BUILDDIR), $(PREFIX), $(DISTDIR), $(SYSROOT), and $(MUSL) (the
musl install prefix that every port links against). Follow the pattern in
ports/mksh/Makefile: fetch tarball to $(DISTDIR), extract to $(SRCDIR),
build into $(BUILDDIR), install into $(SYSROOT). clean removes
$(PORTDIR) and nothing else.
ports/Makefile picks up the new directory automatically. If it links against
musl, add foo: musl there so ordering is right.
Local modifications to upstream go in ports/foo/patches/*.patch and are
applied after extraction — never by editing files under build/, which are
wiped on every clean.
Ramdisk root
make # normal: root on the IDE disk (build/fs.img as hdb)
make ramroot # root from a SCHFS image linked into the kernel
make ramroot builds build/fs-ram.img (default 6000 blocks, override with
RAMDISK_BLOCKS=) and links it into the kernel, so the system boots to a shell
with no data disk attached at all. That is what makes destructive work on
the real disk survivable — reformatting it, or bringing up a new on-disk format
— and it is the first piece of the installer.
Use make ramroot, not make RAMROOT=1: the flag changes CFLAGS, which make
cannot see, so objects compiled for one setting would be silently reused for
the other and ROOTDEV would point at the wrong device. The target drops the
compiled objects first. Switching back is the same — plain make after a
ramroot build reuses stale objects, so run rm -rf build/kernel build/lib build/cmd first (or just make clean).
The image costs boot time: the loader reads the kernel through the firmware's file protocol, so a 6 MB ramroot kernel is visibly slower to start than the 350 KB default.
What may go in it is a licensing question, not only a size one. The image is
linked into the kernel binary, so anything in sysroot/ when it is built becomes
part of a distributable artifact. A file staged there under a licence that
cannot be combined with Apache-2.0 does not merely ride along — it makes the
kernel itself undistributable. RAMDISK_EXCLUDE in the Makefile is the list of
paths kept out, and enforcing it in the build rather than relying on someone
remembering to unstage something first is deliberate. It has been needed once
already: a 33 MB test payload landed in sysroot/ and both overflowed the image
and would have gone into the kernel binary.
Three knobs, all in the Makefile, none of them FSSIZE:
RAMDISK_BLOCKS |
image size, 6000 (× 1 KiB BSIZE) |
RAMDISK_NINODES |
inode table size, 2000 |
RAMDISK_EXCLUDE |
sysroot-relative paths to omit, with their subtrees |
RAMDISK_NINODES is worth knowing about because the inode table is
preallocated: mkfs's default of 12000 inodes cost 1501 of the 6000 blocks — a
quarter of the image — to describe about 260 files. At 2000 it costs 251, and
metadata overall fell from 1596 blocks to 345.
If mkfs reports the image is full, read its message: it now prints the
metadata breakdown and names the right knob for whichever image overflowed. It
used to say only "increase FSSIZE in param.h", which is a constant that has
nothing to do with the ramdisk root and sends you to the wrong file.
Two things that will bite you
The second is about a bootloader that no longer exists. It is kept because the shape of the bug outlived the code.
ld -r -b binary derives symbol names from the input file's path. The
kernel embeds two blobs this way (initcode, entryother), and the root
Makefile has --redefine-sym lines mapping e.g.
_binary_build_lib_initcode_start → _binary_initcode_start. Move either
blob's build location and those symbol names change; update the Makefile to
match or the kernel fails to link.
STAGE2_SECTORS is gone, with the BIOS loader it sized. It is recorded here
because the shape of the bug outlived it: one number that four files each
believed independently — boot/bios/bootasm.S, stage2main.c's readseg(),
the dd ... seek= offset, and twice inside mkiso.sh. Raising it from 24 to 48
for the boot logo left the dd seek=25 offset behind, so the kernel was written
directly over stage 2's own tail, and nothing said so.
The fix was to derive all five from one Makefile variable. The lesson worth
keeping is the other half: tools/padstage2.pl failed the build when stage 2
overflowed its budget rather than truncating it, which is why the overflow to
25440 bytes against a 24576 limit was found at all. A layout constant that
several artifacts must agree on wants a check that stops the build, not a
comment asking people to remember.