SchOS/docs/guides/layout.md

# Repository layout

Where everything is, and how to add to it.

One rule holds the whole thing together:

> **A directory contains either hand-written source or generated output. Never both.**

Everything generated lives under `build/`. Delete `build/` and you lose nothing
that can't be rebuilt (except upstream tarballs, which live in `dist/` and are
deliberately kept out of `clean`).

```
schos/
├─ boot/          uefi/ the PE/COFF loader (splash, F8 menu, kernel load),
│                 common/ the UI it shares. UEFI only -- there is no BIOS path.
├─ kernel/        The kernel, by subsystem: arch/ core/ dev/ drivers/ fs/ net/
│                 proc/ sys/ acpi/ (uACPI glue, the EC, I2C-HID) gmmos/ (the
│                 micro OS under the crash screen) scff/ (the decompressor).
│                 drivers/ is the in-kernel drivers, by class: audio bus gpu
│                 net storage usb.
├─ ldls/          The Linux driver layer: the .ko loader, the BTF reader, the
│                 Linux symbols reimplemented (kpi/), and host-side test tools.
│                 The process that runs it is cmd/ldlsd.
├─ include/       Headers shared by kernel, userland, and tools/mkfs, by
│                 subsystem -- see "include/" below
├─ lib/           First-party userland runtime: _start glue, syscall stubs,
│                 the initcode blob embedded in the kernel, user.ld
├─ cmd/           First-party programs — one directory per program:
│                 cmd/<name>/main.c  →  sysroot/bin/<name>
│                 schinit and schinitd are PID 1 and its supervisor; vgpud and
│                 ldlsd are drivers that run out here rather than in the
│                 kernel; socktest/shmtest/nettest/pcitest/xready are suites.
├─ etc/           The configuration the image starts with: schinit's job
│                 manifests (etc/schinit/*.job), motd, os-release, the zsh files.
├─ ports/         Third-party software -- one directory per port, containing
│                 ONLY a Makefile and optional patches/. musl, toybox, mksh,
│                 lwIP, uACPI, zstd, ncurses, the fonts, fastfetch, kilo; and
│                 alpine/, which is the apk root (world, world.lock).
├─ projects/      Things that run ON SchOS rather than things that ARE it.
│                 Not part of the kernel build. See projects/README.md.
├─ tools/         Host-side helpers, SOURCE ONLY: mkfs.c, mkschfs.c, mkscff.c,
│                 the SchFS test suites, vectors.pl, and the harnesses, grouped
│                 by what runs them (windows/ .ps1, posix/ .sh, python/ .py).
│                 Their binaries build into build/tools/. tools/windows/loc.ps1
│                 and tools/python/stats.py count how much of the tree is
│                 first-party.
├─ sysroot/       Staging for the disk image, GENERATED by the build -- edit
│                 tools/posix/mkaroot.sh and etc/, not this.
├─ dist/          Upstream tarball and package cache. Survives `make clean`.
├─ shell.nix      NixOS development shell: the whole toolchain in one file
├─ write.sh       Writes schos.iso to a USB stick, safely (Linux).
├─ run.ps1        The Windows-side QEMU launcher.
├─ docs/          architecture/ what it is   plan/    what's next
│                 guides/      how to work   history/ why it's like this
│                 Website/     the site, and the docs reader that renders this
└─ build/         ALL generated output (see below)
```

## include/

One folder per subsystem, and every include names it: `#include
"core/proc.h"`, not `"proc.h"`. The compiler is given `-I./include` and
nothing deeper, so the folder is the only way to reach a header and the
include line says where to find it.

| folder  | what is in it |
|---------|---------------|
| `arch/` | the x86-64 machine: `mmu.h`, `x86.h`, `traps.h`, `memlayout.h`, `mp.h`, `ioperm.h`, `vmap.h`, `asm.h`, and `schboot.h`, the loader's handoff block |
| `core/` | the kernel proper: `types.h`, `param.h`, `defs.h`, `proc.h`, the locks, `kmem.h`, `dma.h`, `irq.h`, `klog.h`, `dbg.h`, `sbuf.h`, `syscall.h`, `uaccess.h`, `hwinfo.h`, `elf.h`, `date.h` |
| `fs/`   | the VFS and what is under it: `fs.h`, `file.h`, `buf.h`, `blkdev.h`, `schfs.h`, `fat.h`, `fsstat.h` |
| `dev/`  | devices and buses: `device.h`, `pci.h`, `pciuser.h`, `usb.h`, `virtio.h`, `acpi.h`, `input.h`, `kbd.h`, `keymap.h`, `vt.h`, `audio.h`, `netdev.h` |
| `gfx/`  | pixels: `display.h`, `fb.h`, `gfx.h`, `font.h`, `text.h`, `ttf.h`, `theme.h`, `logo.h` (generated), `qr.h` |
| `net/`  | `socket.h`, `netuser.h` |
| `uapi/` | what userland is built against: `user.h`, `posix.h`, `errno.h`, `stat.h`, `fcntl.h`, `schinit.h` |
| `ldls/` | the Linux driver layer's headers |

A side effect worth having: `errno.h`, `stat.h` and `fcntl.h` are no longer
at the top of `include/`, so a host tool built with `-I./include` gets the
system's versions of them rather than ours.

`tools/fix-includes.py` rewrites a bare `#include "proc.h"` to the folder the
header is in, reading the folders from the tree. Run it with `--modified` on
code written before the split, or after moving a header.

## build/

```
build/
├─ boot/          uefi/ loader, common/ UI
├─ kernel/        object files, the linked kernel, kernel.asm
├─ lib/           libc glue objects, initcode blob
├─ cmd/           linked first-party programs
├─ aroot/         the apk root the image is made from
├─ schfs.img      the root as a disk image (make qemu)
├─ fs-ram.img     the same root as the live ramdisk, fs-ram.scff compressed
├─ iso/ …         what make iso assembles
└─ ports/<name>/
    ├─ src/       extracted upstream tarball
    ├─ obj/       out-of-tree build directory
    └─ prefix/    install prefix (musl's headers and libs land here)
```

## Adding your own program

```sh
mkdir cmd/hello
$EDITOR cmd/hello/main.c          # include "types.h", "user.h", ...
```

Then add `hello` to `CMDS` in the root `Makefile`. It gets linked against
`lib/` and installed to `sysroot/bin/hello` by `make sysroot-install`.

## Adding a third-party port

```sh
mkdir ports/foo
$EDITOR ports/foo/Makefile        # start with `include ../mk/port.mk`
```

`port.mk` derives everything from the directory name — it gives you `$(SRCDIR)`,
`$(BUILDDIR)`, `$(PREFIX)`, `$(DISTDIR)`, `$(SYSROOT)`, and `$(MUSL)` (the
musl install prefix that every port links against). Follow the pattern in
`ports/mksh/Makefile`: fetch tarball to `$(DISTDIR)`, extract to `$(SRCDIR)`,
build into `$(BUILDDIR)`, install into `$(SYSROOT)`. `clean` removes
`$(PORTDIR)` and nothing else.

`ports/Makefile` picks up the new directory automatically. If it links against
musl, add `foo: musl` there so ordering is right.

Local modifications to upstream go in `ports/foo/patches/*.patch` and are
applied after extraction — never by editing files under `build/`, which are
wiped on every clean.

## The live root

```sh
make            # normal: the root is a SchFS disk image, build/schfs.img
make iso        # the same root as a compressed ramdisk linked into the kernel
make ramroot    # the ramdisk, uncompressed
```

`make iso` builds the root as a SchFS image, compresses it
([SCFF](../architecture/scff.md)) and links it into the kernel, so the system boots
to a shell with **no data disk attached at all** -- which is what the USB stick is.
It is also what makes destructive work on a real disk survivable. Nothing written
to a live root survives a reboot.

Use the targets, not `make RAMROOT=1`: the flag changes `CFLAGS`, which make cannot
see, so objects compiled for one setting would be silently reused for the other and
`ROOTDEV` would point at the wrong device. The targets drop the compiled objects
first. Going back to the disk root is the same -- run `make clean` or remove
`build/kernel build/lib build/cmd` first.

**What may go in it is a licensing question, not only a size one.** The image is
linked into the kernel binary, so anything in `sysroot/` when it is built becomes
part of a distributable artifact. A file staged there under a licence that is not
ours to redistribute does not merely ride along -- it makes the kernel itself
undistributable. `RAMDISK_EXCLUDE` in the Makefile is the list of paths kept out,
and enforcing it in the build rather than relying on someone remembering to unstage
something first is deliberate. (Linux modules for LDLS are staged by `make
ldls-stage`/`ldls-r8169` on request, and `.gitignore` refuses `*.ko`: binaries stay out of git. The
r8169 set is meant to be staged before `make iso`, since the ISO ships it.)

The knobs: `RAMDISK_BLOCKS` (image size), `RAMDISK_NINODES` (the inode table is
*preallocated*: the default of 12000 once cost a quarter of a small image to
describe about 260 files), `RAMDISK_EXCLUDE`. If `mkfs` reports the image is full,
read its message: it prints the metadata breakdown and names the right knob.

The ISO build is fast on purpose (a few seconds when little changed): it copies with
`cp -p` and `install -C` so unchanged files keep their time stamps, and toggling the
ramdisk does not wipe the tree.

## Two things that will bite you

The second is about a bootloader that no longer exists. It is kept because the
*shape* of the bug outlived the code.

**`ld -r -b binary` derives symbol names from the input file's path.** The
kernel embeds two blobs this way (`initcode`, `entryother`), and the root
Makefile has `--redefine-sym` lines mapping e.g.
`_binary_build_lib_initcode_start` → `_binary_initcode_start`. Move either
blob's build location and those symbol names change; update the Makefile to
match or the kernel fails to link.

**`STAGE2_SECTORS` is gone, with the BIOS loader it sized.** It is recorded here
because the shape of the bug outlived it: one number that four files each
believed independently — `boot/bios/bootasm.S`, `stage2main.c`'s `readseg()`,
the `dd ... seek=` offset, and twice inside `mkiso.sh`. Raising it from 24 to 48
for the boot logo left the `dd seek=25` offset behind, so the kernel was written
directly over stage 2's own tail, and nothing said so.

The fix was to derive all five from one Makefile variable. The lesson worth
keeping is the other half: `tools/padstage2.pl` *failed the build* when stage 2
overflowed its budget rather than truncating it, which is why the overflow to
25440 bytes against a 24576 limit was found at all. A layout constant that
several artifacts must agree on wants a check that stops the build, not a
comment asking people to remember.