d3c37f82b6
The site and several documents still described the system as it was before virtual terminals, the new schinit, the hybrid work and descriptor passing. docs/architecture/init.md described the wrong program. It documented a single schinit that parses manifests and supervises jobs, which has not been true since PID 1 was split: PID 1 forks a supervisor, spawns jobs on its behalf and reaps; the job table lives in /proc/schinit; a heartbeat runs both ways; and proc_reincarnate() re-execs PID 1 in place rather than panicking. Rewritten around what exists, keeping the reasoning that is still load-bearing -- the five failures of the sibling-init scheme, the escalation ladder, and the three bugs in the reincarnation path that each had to happen before they were obvious. docs/plan/init.md is deleted, which is the tree's own rule for finished work and had not been applied. Fourteen source comments referenced it by part number, so they are repointed at the architecture doc by section name rather than left dangling -- a deleted document nobody notices is worse than a stale one. docs/architecture/devices.md gained the section it has been missing since /dev/pci existed: what a driver needs from a kernel, why mmap is checked against the caller's claims, why DMA is a call and not a mapping, and why the write back after an interrupt is not optional. Also that a released device does not come back by itself, which is true and was nowhere. docs/architecture/networking.md gained a descriptor-passing section, and docs/architecture/README.md's stack diagram gained the two layers and the third seam it was missing. "Two seams hold the whole thing together" has not been true since /dev/pci. docs/plan/README.md said the hybrid work was deliberately last, on the argument that an interface with no consumer is wrong in ways nobody can see. The argument was right; the conclusion was not, and the document now says why it changed -- the alternative to a driver compatibility layer is writing a driver for every device that exists, and the original objection is answered rather than ignored because SchOS's own drivers are the consumers. The site: a Hybrid section with the eviction transcript, cards for virtual terminals, schinit and drivers outside the kernel, and the numbers remeasured rather than estimated -- 46,961 lines, 207 syscalls. The boot log in app.js is from a boot taken today, and the lines are trimmed to what the console can show without clipping, each still a verbatim prefix of real output. Two small rendering fixes found by looking at the pages rather than assuming: .console pre's min-height is now scoped to the hero, so a static transcript sizes to its content instead of reserving eight lines it does not have; and the docs reader's contents panel strips inline marks from headings, so a heading written `proc_reincarnate()` no longer carries its backticks into the sidebar. Every intra-doc link checked; none broken. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
182 lines
9.1 KiB
Markdown
182 lines
9.1 KiB
Markdown
# Repository layout
|
||
|
||
Where everything is, and how to add to it.
|
||
|
||
One rule holds the whole thing together:
|
||
|
||
> **A directory contains either hand-written source or generated output. Never both.**
|
||
|
||
Everything generated lives under `build/`. Delete `build/` and you lose nothing
|
||
that can't be rebuilt (except upstream tarballs, which live in `dist/` and are
|
||
deliberately kept out of `clean`).
|
||
|
||
```
|
||
schos/
|
||
├─ boot/ uefi/ the PE/COFF loader (splash, F8 menu, kernel load),
|
||
│ common/ the UI it shares. UEFI only -- there is no BIOS path.
|
||
├─ kernel/ The kernel, by subsystem: arch/ core/ dev/ fs/ proc/ sys/
|
||
├─ include/ Headers shared by kernel, userland, and tools/mkfs
|
||
├─ lib/ First-party userland runtime: _start glue, syscall stubs,
|
||
│ the initcode blob embedded in the kernel, user.ld
|
||
├─ cmd/ First-party programs — one directory per program:
|
||
│ cmd/<name>/main.c → sysroot/bin/<name>
|
||
│ schinit and schinitd are PID 1 and its supervisor; vgpud is
|
||
│ the virtio-gpu driver, which runs out here rather than in
|
||
│ the kernel; socktest/shmtest/nettest/pcitest are suites.
|
||
├─ ports/ Third-party software — one directory per port.
|
||
│ Contains ONLY a Makefile and optional patches/.
|
||
├─ projects/ Things that run ON SchOS rather than things that ARE it:
|
||
│ the compositor, window manager, desktop, and installer.
|
||
│ Not part of the kernel build. See projects/README.md.
|
||
├─ tools/ Host-side helpers, SOURCE ONLY: mkfs.c, the SchFS test
|
||
│ suites, sign.pl, vectors.pl, and the PowerShell harnesses.
|
||
│ Their binaries build into build/tools/ -- this directory
|
||
│ used to hold both, which is the one rule this layout has.
|
||
│ tools/windows/loc.ps1 counts how much of the tree is first-party.
|
||
├─ sysroot/ Staging for the disk image. etc/ is hand-maintained;
|
||
│ bin/ lib/ sbin/ are populated by `make sysroot-install`.
|
||
├─ dist/ Upstream tarball cache. Survives `make clean`.
|
||
├─ Refrence/ A FreeBSD checkout (Refrence/freebsd-src), kept as reference
|
||
│ material and a parts bin. Never built, never in an image.
|
||
│ Read it when a specification is ambiguous; copy from it only
|
||
│ after the per-file licence check in docs/licensing.md.
|
||
│ A full clone is 2.1 GB and a previous NetBSD one was deleted
|
||
│ for that reason. Only sys/dev, sys/net and sys/netinet have
|
||
│ ever been read, so a sparse checkout is ~100 MB:
|
||
│ git clone --filter=blob:none --sparse <url> Refrence/freebsd-src
|
||
│ git -C Refrence/freebsd-src sparse-checkout set │ sys/dev sys/net sys/netinet
|
||
├─ src-trunk/ A NetBSD checkout, when present -- reference material and a
|
||
│ parts bin. NOT built, NOT installed, must never reach an
|
||
│ image. Currently absent; it was deleted for disk space and
|
||
│ can be re-fetched sparsely. Copying from it is governed by
|
||
│ the per-file licence check in docs/licensing.md §3 --
|
||
│ NetBSD is not uniformly licensed.
|
||
├─ shell.nix NixOS development shell: the whole toolchain in one file
|
||
├─ docs/ architecture/ what it is plan/ what's next
|
||
│ guides/ how to work history/ why it's like this
|
||
└─ build/ ALL generated output (see below)
|
||
```
|
||
|
||
## build/
|
||
|
||
```
|
||
build/
|
||
├─ boot/ uefi/ loader, common/ UI
|
||
├─ kernel/ object files, the linked kernel, kernel.asm
|
||
├─ lib/ libc glue objects, initcode blob
|
||
├─ cmd/ linked first-party programs
|
||
├─ fs.img the filesystem image
|
||
└─ ports/<name>/
|
||
├─ src/ extracted upstream tarball
|
||
├─ obj/ out-of-tree build directory
|
||
└─ prefix/ install prefix (musl's headers and libs land here)
|
||
```
|
||
|
||
## Adding your own program
|
||
|
||
```sh
|
||
mkdir cmd/hello
|
||
$EDITOR cmd/hello/main.c # include "types.h", "user.h", ...
|
||
```
|
||
|
||
Then add `hello` to `CMDS` in the root `Makefile`. It gets linked against
|
||
`lib/` and installed to `sysroot/bin/hello` by `make sysroot-install`.
|
||
|
||
## Adding a third-party port
|
||
|
||
```sh
|
||
mkdir ports/foo
|
||
$EDITOR ports/foo/Makefile # start with `include ../mk/port.mk`
|
||
```
|
||
|
||
`port.mk` derives everything from the directory name — it gives you `$(SRCDIR)`,
|
||
`$(BUILDDIR)`, `$(PREFIX)`, `$(DISTDIR)`, `$(SYSROOT)`, and `$(MUSL)` (the
|
||
musl install prefix that every port links against). Follow the pattern in
|
||
`ports/mksh/Makefile`: fetch tarball to `$(DISTDIR)`, extract to `$(SRCDIR)`,
|
||
build into `$(BUILDDIR)`, install into `$(SYSROOT)`. `clean` removes
|
||
`$(PORTDIR)` and nothing else.
|
||
|
||
`ports/Makefile` picks up the new directory automatically. If it links against
|
||
musl, add `foo: musl` there so ordering is right.
|
||
|
||
Local modifications to upstream go in `ports/foo/patches/*.patch` and are
|
||
applied after extraction — never by editing files under `build/`, which are
|
||
wiped on every clean.
|
||
|
||
## Ramdisk root
|
||
|
||
```sh
|
||
make # normal: root on the IDE disk (build/fs.img as hdb)
|
||
make ramroot # root from a SCHFS image linked into the kernel
|
||
```
|
||
|
||
`make ramroot` builds `build/fs-ram.img` (default 6000 blocks, override with
|
||
`RAMDISK_BLOCKS=`) and links it into the kernel, so the system boots to a shell
|
||
with **no data disk attached at all**. That is what makes destructive work on
|
||
the real disk survivable — reformatting it, or bringing up a new on-disk format
|
||
— and it is the first piece of the installer.
|
||
|
||
Use `make ramroot`, not `make RAMROOT=1`: the flag changes `CFLAGS`, which make
|
||
cannot see, so objects compiled for one setting would be silently reused for
|
||
the other and `ROOTDEV` would point at the wrong device. The target drops the
|
||
compiled objects first. Switching back is the same — plain `make` after a
|
||
`ramroot` build reuses stale objects, so run `rm -rf build/kernel build/lib
|
||
build/cmd` first (or just `make clean`).
|
||
|
||
The image costs boot time: the loader reads the kernel through the firmware's
|
||
file protocol, so a 6 MB ramroot kernel is visibly slower to start than the
|
||
350 KB default.
|
||
|
||
**What may go in it is a licensing question, not only a size one.** The image is
|
||
linked into the kernel binary, so anything in `sysroot/` when it is built becomes
|
||
part of a distributable artifact. A file staged there under a licence that
|
||
cannot be combined with Apache-2.0 does not merely ride along — it makes the
|
||
kernel itself undistributable. `RAMDISK_EXCLUDE` in the Makefile is the list of
|
||
paths kept out, and enforcing it in the build rather than relying on someone
|
||
remembering to unstage something first is deliberate. It has been needed once
|
||
already: a 33 MB test payload landed in `sysroot/` and both overflowed the image
|
||
and would have gone into the kernel binary.
|
||
|
||
Three knobs, all in the Makefile, none of them `FSSIZE`:
|
||
|
||
| | |
|
||
|---|---|
|
||
| `RAMDISK_BLOCKS` | image size, 6000 (× 1 KiB `BSIZE`) |
|
||
| `RAMDISK_NINODES` | inode table size, 2000 |
|
||
| `RAMDISK_EXCLUDE` | sysroot-relative paths to omit, with their subtrees |
|
||
|
||
`RAMDISK_NINODES` is worth knowing about because the inode table is
|
||
*preallocated*: mkfs's default of 12000 inodes cost 1501 of the 6000 blocks — a
|
||
quarter of the image — to describe about 260 files. At 2000 it costs 251, and
|
||
metadata overall fell from 1596 blocks to 345.
|
||
|
||
If `mkfs` reports the image is full, read its message: it now prints the
|
||
metadata breakdown and names the right knob for whichever image overflowed. It
|
||
used to say only "increase FSSIZE in param.h", which is a constant that has
|
||
nothing to do with the ramdisk root and sends you to the wrong file.
|
||
|
||
## Two things that will bite you
|
||
|
||
The second is about a bootloader that no longer exists. It is kept because the
|
||
*shape* of the bug outlived the code.
|
||
|
||
**`ld -r -b binary` derives symbol names from the input file's path.** The
|
||
kernel embeds two blobs this way (`initcode`, `entryother`), and the root
|
||
Makefile has `--redefine-sym` lines mapping e.g.
|
||
`_binary_build_lib_initcode_start` → `_binary_initcode_start`. Move either
|
||
blob's build location and those symbol names change; update the Makefile to
|
||
match or the kernel fails to link.
|
||
|
||
**`STAGE2_SECTORS` is gone, with the BIOS loader it sized.** It is recorded here
|
||
because the shape of the bug outlived it: one number that four files each
|
||
believed independently — `boot/bios/bootasm.S`, `stage2main.c`'s `readseg()`,
|
||
the `dd ... seek=` offset, and twice inside `mkiso.sh`. Raising it from 24 to 48
|
||
for the boot logo left the `dd seek=25` offset behind, so the kernel was written
|
||
directly over stage 2's own tail, and nothing said so.
|
||
|
||
The fix was to derive all five from one Makefile variable. The lesson worth
|
||
keeping is the other half: `tools/padstage2.pl` *failed the build* when stage 2
|
||
overflowed its budget rather than truncating it, which is why the overflow to
|
||
25440 bytes against a 24576 limit was found at all. A layout constant that
|
||
several artifacts must agree on wants a check that stops the build, not a
|
||
comment asking people to remember.
|