SchOS/docs/layout.md

Repository layout

One rule holds the whole thing together:

A directory contains either hand-written source or generated output. Never both.

Everything generated lives under build/. Delete build/ and you lose nothing that can't be rebuilt (except upstream tarballs, which live in dist/ and are deliberately kept out of clean).

schunix/
├─ boot/          Stage 1 MBR + stage 2 loader (splash, F8 menu, kernel load)
├─ kernel/        The kernel, by subsystem: arch/ core/ dev/ fs/ proc/ sys/
├─ include/       Headers shared by kernel, userland, and tools/mkfs
├─ lib/           First-party userland runtime: _start glue, syscall stubs,
│                 the initcode blob embedded in the kernel, user.ld
├─ cmd/           First-party programs — one directory per program:
│                 cmd/<name>/main.c  →  sysroot/bin/<name>
├─ ports/         Third-party software — one directory per port.
│                 Contains ONLY a Makefile and optional patches/.
├─ projects/      Things that run ON Schunix rather than things that ARE it:
│                 the compositor, window manager, desktop, and installer.
│                 Not part of the kernel build. See projects/README.md.
├─ tools/         Host-side helpers, SOURCE ONLY: mkfs.c, the SchFS test
│                 suites, sign.pl, vectors.pl, and the PowerShell harnesses.
│                 Their binaries build into build/tools/ -- this directory
│                 used to hold both, which is the one rule this layout has.
│                 tools/loc.ps1 counts how much of the tree is first-party.
├─ sysroot/       Staging for the disk image. etc/ is hand-maintained;
│                 bin/ lib/ sbin/ are populated by `make sysroot-install`.
├─ dist/          Upstream tarball cache. Survives `make clean`.
├─ Refrence/      A FreeBSD checkout (Refrence/freebsd-src), kept as reference
│                 material and a parts bin. Never built, never in an image.
│                 Read it when a specification is ambiguous; copy from it only
│                 after the per-file licence check in docs/licensing.md.
│                 A full clone is 2.1 GB and a previous NetBSD one was deleted
│                 for that reason. Only sys/dev, sys/net and sys/netinet have
│                 ever been read, so a sparse checkout is ~100 MB:
│                   git clone --filter=blob:none --sparse <url> Refrence/freebsd-src
│                   git -C Refrence/freebsd-src sparse-checkout set │                       sys/dev sys/net sys/netinet
├─ src-trunk/     A NetBSD checkout, kept as reference material and a parts
│                 bin. NOT built, NOT installed, and it must never reach an
│                 image. Copying from it is governed by the per-file licence
│                 check in docs/architecture/schapi.md §5 — NetBSD is not
│                 uniformly licensed.
├─ docs/          Design docs
└─ build/         ALL generated output (see below)

build/

build/
├─ boot/          bootblock, stage2
├─ kernel/        object files, the linked kernel, kernel.asm
├─ lib/           libc glue objects, initcode blob
├─ cmd/           linked first-party programs
├─ fs.img         the filesystem image
└─ ports/<name>/
    ├─ src/       extracted upstream tarball
    ├─ obj/       out-of-tree build directory
    └─ prefix/    install prefix (musl's headers and libs land here)

Adding your own program

mkdir cmd/hello
$EDITOR cmd/hello/main.c          # include "types.h", "user.h", ...

Then add hello to CMDS in the root Makefile. It gets linked against lib/ and installed to sysroot/bin/hello by make sysroot-install.

Adding a third-party port

mkdir ports/foo
$EDITOR ports/foo/Makefile        # start with `include ../mk/port.mk`

port.mk derives everything from the directory name — it gives you $(SRCDIR), $(BUILDDIR), $(PREFIX), $(DISTDIR), $(SYSROOT), and $(MUSL) (the musl install prefix that every port links against). Follow the pattern in ports/mksh/Makefile: fetch tarball to $(DISTDIR), extract to $(SRCDIR), build into $(BUILDDIR), install into $(SYSROOT). clean removes $(PORTDIR) and nothing else.

ports/Makefile picks up the new directory automatically. If it links against musl, add foo: musl there so ordering is right.

Local modifications to upstream go in ports/foo/patches/*.patch and are applied after extraction — never by editing files under build/, which are wiped on every clean.

Ramdisk root

make            # normal: root on the IDE disk (build/fs.img as hdb)
make ramroot    # root from a SCHFS image linked into the kernel

make ramroot builds build/fs-ram.img (default 6000 blocks, override with RAMDISK_BLOCKS=) and links it into the kernel, so the system boots to a shell with no data disk attached at all. That is what makes destructive work on the real disk survivable — reformatting it, or bringing up a new on-disk format — and it is the first piece of the installer.

Use make ramroot, not make RAMROOT=1: the flag changes CFLAGS, which make cannot see, so objects compiled for one setting would be silently reused for the other and ROOTDEV would point at the wrong device. The target drops the compiled objects first. Switching back is the same — plain make after a ramroot build reuses stale objects, so run rm -rf build/kernel build/lib build/cmd first (or just make clean).

The image costs boot time: stage2 loads the kernel by PIO one sector at a time, so a 6 MB ramroot kernel is visibly slower to start than the 350 KB default.

What may go in it is a licensing question, not only a size one. The image is linked into the kernel binary, so anything in sysroot/ when it is built becomes part of a distributable artifact. A file staged there under a licence that cannot be combined with Apache-2.0 does not merely ride along — it makes the kernel itself undistributable. RAMDISK_EXCLUDE in the Makefile is the list of paths kept out, and enforcing it in the build rather than relying on someone remembering to unstage something first is deliberate. It has been needed once already: a 33 MB test payload landed in sysroot/ and both overflowed the image and would have gone into the kernel binary.

Three knobs, all in the Makefile, none of them FSSIZE:

RAMDISK_BLOCKS	image size, 6000 (× 1 KiB BSIZE)
RAMDISK_NINODES	inode table size, 2000
RAMDISK_EXCLUDE	sysroot-relative paths to omit, with their subtrees

RAMDISK_NINODES is worth knowing about because the inode table is preallocated: mkfs's default of 12000 inodes cost 1501 of the 6000 blocks — a quarter of the image — to describe about 260 files. At 2000 it costs 251, and metadata overall fell from 1596 blocks to 345.

If mkfs reports the image is full, read its message: it now prints the metadata breakdown and names the right knob for whichever image overflowed. It used to say only "increase FSSIZE in param.h", which is a constant that has nothing to do with the ramdisk root and sends you to the wrong file.

Two things that will bite you

ld -r -b binary derives symbol names from the input file's path. The kernel embeds two blobs this way (initcode, entryother), and the root Makefile has --redefine-sym lines mapping e.g. _binary_build_lib_initcode_start → _binary_initcode_start. Move either blob's build location and those symbol names change; update the Makefile to match or the kernel fails to link.

STAGE2_SECTORS now lives in exactly one place: the root Makefile. It used to be written out by hand in five — boot/bootasm.S, boot/stage2main.c's readseg(), the dd ... seek= offset, and twice more inside tools/mkiso.sh (its stage2 size assertion and -boot-load-size). This note used to warn you to keep three of them in step and did not know about the other two.

Raising it from 24 to 48 for the boot logo demonstrated the problem: the dd seek=25 offset stayed behind, so the kernel was written directly over stage2's own tail. Now the Makefile passes -DSTAGE2_SECTORS to both boot sources, derives the padding budget and the seek= offset from the same variable, and exports it to mkiso.sh. The #defines that remain in the two boot sources are #ifndef-guarded fallbacks so those files still read sensibly alone; the Makefile always wins.

Two things still fail the build rather than silently misbooting, and should stay that way: tools/padstage2.pl errors if stage2 overflows its budget instead of truncating it, and boot/stage2main.c static-asserts that stage2 cannot reach the multiboot kernel-search buffer at 0x10000 — which puts the hard ceiling at 65 sectors, since stage2 loads at 0x7E00.