The last three phases of the filesystem, which were written together and
touch the same files throughout -- kmount.c, objw.c and vfs.c are each
rewritten by all three -- so they land as one commit rather than three
that could not be built separately.
Durability. A transaction that reaches the platter out of order is worse
than one that never reached it: the uberblock can name a tree whose
blocks are still in the drive's write cache. bdirty/bflush give the
buffer layer an explicit barrier, ideflush issues ATA FLUSH CACHE and
usb_storage_flush the SCSI SYNCHRONIZE CACHE that means the same thing,
and a commit now waits for the data before it writes the root that
points at it. The uberblock groups are read newest-first with fallback,
so a volume whose newest roots are torn mounts at an older one and says
which generations were lost.
Reflink. Extents carry a refcount, so two objects can name the same
blocks and neither frees them while the other holds them. Exposed as
FICLONE -- Linux's ioctl value, so `cp --reflink` works unmodified --
and as cmd/reflink for the case where there is no cp to hand.
Attributes, and the one genuinely new idea here. attr.c stores named
values on an object, reachable through setxattr/getxattr/removexattr at
Linux's syscall numbers. On top of that, schfs_index declares an index
over an attribute name and schfs_query answers "which objects carry this
value" from the B-tree rather than by walking. A query against an
unindexed attribute is refused with an error, never silently answered by
a full scan: a query that quietly becomes a scan is worse than one that
admits it cannot help. cmd/schattr drives both.
Three bugs fixed on the way, each found by the numbers being wrong
rather than by inspection:
- statfs's f_fsid was declared int64[2]. Linux's fsid_t is two *ints*,
so every field after it landed one slot late: userland read f_frsize
out of f_namelen, got 255, and df reported a 32 MB root filesystem
as 1992 KB. It hid for as long as it did because the fields used to
be invented constants, so wrong numbers looked like the usual ones.
- struct inode's nlink was a short against SchFS's uint32, so a file
past 32767 names came back negative -- and iput frees an object
whose count reaches zero. It is an int now, signed on purpose, so an
underflow lands somewhere a check can see rather than wrapping.
- filewrite split every write to fit v1's log. That limit belongs to
the journal, not to writing, and SchFS makes one transaction of
whatever it is handed: a 1 MB write was becoming 170 commits and
170 flushes for nothing. fs_ops carries max_write now, and SchFS
sets no limit.
Also here: a real mount table, so `mount -t schfs /dev/hdb /mnt` shows
up in /proc/mounts -- df looks a path up there before it calls statfs at
all, so a mount missing from that list was invisible to it -- several
volumes mounted at once, snapshots mounted by name, and /proc/fsstat
with an rdtsc-based counter, which is the only instrument fine enough to
see inside a single syscall.
Verified: `make schfs-check` passes in full, including the sweep that
crashes the volume at each of 119 points mid-transaction and requires
every one to mount with exactly its committed data, and the case where
both newest uberblocks are rubbish and the volume still comes back. The
headless boot check passes with 27 [ok] and no failures, on a SchFS root.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
31 KiB
SchFS — design
Status: phases 1 through 3½ are done, and SchFS is the root filesystem.
kernel/proc/proc.c mounts it before pid 1 has a working directory and panics
if it is not there. The copy-on-write B-tree, the volume layer, the object
layer, the write path, snapshots, an image writer and a checker all exist, and
several volumes can be mounted at once. Phase 4 (extent refcounts) is next.
See §8 for exactly what is built and how to run its tests.
Supersedes the on-disk format described in include/fs.h. Keeps the name SchFS;
the on-disk magic changes so v1 images are detected and rejected rather than
silently misread.
The brief was "the best aspects of BeFS, ZFS and btrfs." Those three disagree with each other in places, so this document says explicitly what is taken from each and what is deliberately left behind.
1. What each one is actually good at
ZFS — never be inconsistent. Copy-on-write: no block that is part of the live tree is ever overwritten. An update writes new blocks, then atomically flips a single root pointer. A crash therefore lands on either the old tree or the new one — never a half-written mix. Consequence: no fsck, ever. Paired with end-to-end checksums where a block's checksum is stored in its parent pointer, not in the block itself, which makes the whole tree self-validating and catches misdirected writes and bit rot that a self-stored checksum cannot.
btrfs — one data structure, reused everywhere. btrfs has essentially one engine: a copy-on-write B-tree. Inodes, directory entries, extent maps, free space, and snapshots are all just different key types in that same tree. For a small team this is the single most important idea in the whole design: you write and debug one tricky data structure, then get five subsystems from it. Also takes extents with reference counts, which is what makes snapshots and reflink copies fall out almost for free.
BeFS — the part that feels like a database, at a size one person can build. BeFS is a 64-bit journaled filesystem whose distinguishing feature is B+tree indexes over file attributes, with a query interface on top. That is the "SQL-like" capability, and unlike WinFS it actually shipped. Critically, BeFS is also the right scale model: a small team, a readable codebase, and a book (Giampaolo, Practical File System Design, free online) that describes it end to end.
2. What is deliberately not taken
- No ZFS storage pools / RAID-Z / vdevs. Multi-disk redundancy is a large subsystem that solves a problem SchOS does not have yet.
- No btrfs-style rebalancing or online format conversion. Enormous complexity, and it is the historical source of btrfs's worst bugs.
- No actual SQL. No parser, no query planner, no relational engine. See §6 — the useful part of "database filesystem" is the index, not the language. WinFS spent 2003-2006 and a large team on the other interpretation and shipped nothing.
- No deduplication. ZFS's dedup is famous for needing more RAM than anyone expects. Extent refcounts give reflink/snapshot benefits without it.
3. On-disk shape
+--------------------+ block 0 boot block (untouched by the FS)
+--------------------+ blocks 1..N uberblock array (128 slots, rotated)
+--------------------+
| CoW B-tree | one tree, many key types:
| | (ino, INODE, 0) -> inode record
| | (ino, DIRENT, hash) -> directory entry
| | (ino, EXTENT, offset) -> extent mapping
| | (ino, ATTR, nameid) -> extended attribute
| | (idx, INDEX, key) -> attribute index entry
+--------------------+
| free space | extent-based, tracked as tree keys, not a bitmap
+--------------------+
Uberblock array (from ZFS). 128 slots written round-robin, each carrying a transaction number and a checksum. Mount scans all slots and takes the highest-numbered one that validates. This is what makes crash recovery "pick the newest good root" instead of "replay a journal."
Everything is one B-tree (from btrfs). Keyed by
(objectid, type, offset), 24 bytes. Adding a feature usually means adding a
key type, not a new on-disk structure.
Checksums live in the parent (from ZFS). Each internal node stores, for
every child pointer, that child's CRC32c. A block that is read is verified
against the pointer that led to it. v1 already chose CRC32c and already has
checksum fields in dinode and superblock — same algorithm, better placement.
Widths — every v1 limit that has to die
| v1 | limit | v2 |
|---|---|---|
dirent.inum is ushort |
65,535 files, total | uint64 object ids |
dinode.size is uint |
4 GB max file | uint64 |
uint block numbers, BSIZE 1024 |
4 TB volume | uint64 blocks, 4096-byte |
DIRSIZ 14 |
14-char filenames | variable, up to 255 |
| fixed hash table per dir | every dir ≥113 KB, capped at 119 KB | B+tree, pay per entry |
addrs[NDIRECT+3] |
block-at-a-time | extents with refcounts |
Block size moves 1024 → 4096 to match modern media and make extents worthwhile.
4. What already exists and gets reused
Not starting from zero:
struct schfs_extentis already defined ininclude/fs.hand unused. The extent concept is already sketched; v2 gives it refcounts and 64-bit fields.- CRC32c checksum fields already exist in
dinodeandsuperblock, and mkfs already computes them. Same algorithm, relocated into parent pointers. kernel/fs/log.cis a working transaction log withbegin_op/end_opcall sites throughout the FS. Under CoW the log is not needed for consistency, butbegin_op/end_opare exactly the right places to bracket a transaction group, so the call sites survive even though the implementation is replaced.- The buffer cache (
bio.c) is format-agnostic and needs no change beyond the block-size constant.
5. Phasing
Each phase leaves a bootable system. Do not start phase 2 before phase 1 is solid — the B-tree is the load-bearing piece and everything else assumes it.
Phase 1 — the B-tree, in userland first. Write the CoW B-tree as a plain C
library with a block-device abstraction, and test it on a host file with a
fuzz/torture harness (random insert/delete/lookup, simulated crash at every
write point, verify the tree always mounts). This is the one component where a
subtle bug is unfixable later, and testing it in the kernel is far harder. It is
also portable straight into tools/mkfs.
Phase 2 — read-only mount. DONE. New magic, uberblock scan, tree walk, inode
and dirent key types. tools/mkschfs writes images; the kernel only reads them.
The system boots from a ramdisk root (see docs/layout.md and the ramdisk work)
so a broken disk cannot stop you from booting.
Went further than planned in two places, both cheaply: data blocks carry
checksums too (a CSUM key type, as btrfs does — without it "end-to-end
checksums" would cover only metadata, which is the part least likely to rot),
and extents landed early because reserving a file's blocks in one go was
simpler than not doing it.
Not yet done: SchFS files are not reachable through open(2). The mount is
observable through /proc/schfs and through the kernel API in schfs_int.h,
but wiring it into namei/readi means the VFS work that phase 3 needs anyway.
Phase 3 — writes, transaction groups. DONE. Every schfs_k* call that
changes anything is one transaction group, committed by writing a new uberblock
before it returns. The full write API — create, link, unlink, rename, write at
an arbitrary offset, truncate — lives in objw.c and is exercised by
tools/schfs_objtest.c.
Two things went differently from the plan above, both for reasons worth recording.
Free space is not tracked as tree keys. Allocating from a free map that lives
in the tree mutates the tree, which allocates, which changes the map; btrfs
resolves that with a fixpoint loop and ZFS with log-structured space maps. SchFS
sidesteps it: the committed tree already says exactly which blocks are live, so
the map is derived by walking the tree once at mount and held only in RAM.
It cannot disagree with the filesystem, because it is computed from it, and a
crash raises no question at all. The cost is mount time proportional to metadata
size, and a ceiling on volume size set by how much bitmap fits in memory. If
mount time ever hurts, the map can be cached on disk purely as a hint,
validated against the generation and discarded on a mismatch — an addition, not
a format change. See kernel/fs/schfs/space.c.
Data blocks needed a generation stamp too. The B-tree already rewrites its own
nodes in place within a group, since a block that no committed root can reach is
nobody's fallback. Data blocks had no such marker, so every hundred-byte write
copied a whole 4 KB block and split an extent record around it. schfs_extent
gained a 64-bit gen, and the same rule now applies to file data.
Phase 3½ — reachable through open(2). DONE. SchFS is a mounted filesystem:
mkdir /mnt
mount -t schfs /dev/hdb /mnt
ls /mnt ; echo hi > /mnt/tmp/w ; cat /mnt/tmp/w ; mkdir /mnt/tmp/d
The VFS grew a real dispatch layer rather than a set of if (dev == SCHFS_DEV)
checks. struct fs_ops (include/file.h) is one table per filesystem, chosen by
iget() from the device number and hung off the inode; v1's implementations
keep their bodies and are simply named v1_*. Adding a third filesystem is a
new table and nothing else. kernel/fs/schfs/vfs.c is SchFS's, and it is pure
translation — object ids in, inodes out — because all the filesystem logic
already lives below it.
Three things had to change on the way, none of them SchFS's fault:
struct inode.inumand.sizewere 32-bit, capping objects at 2³² and files at 4 GiB on a filesystem that promises far more. Both areuint64now, as arereadi/writeioffsets andstruct file.off.struct direntcarries a 16-bit inum and a 14-byte name, so it cannot represent a SchFS entry at all.getdents64is now an fs_op: each filesystem emits the record stream itself, andf->offis whatever position it needs to resume. v1 walks its hash buckets; SchFS walks a key range and synthesises "." and "..", which it does not store.- Mounting is a substitution made during path resolution (
mount_add/mount_crossinfs.c). Nothing on either disk records it. The reverse direction matters as much as the forward one:..at a mounted root has to leave the mount, and the filesystem underneath has no idea it is mounted anywhere.
Three long-standing bugs surfaced the moment real commands ran against it, all in the shared VFS rather than in SchFS:
include/fcntl.hused xv6's open flags while every program in the userland sends Linux's.O_APPEND(0x400) landed onO_TRUNC, socmd >> filetruncated it;O_TRUNC(0x200) landed onO_CREATE, socmd > fileleft whatever was past the new end in place. Nothing about either pointed at a header.sys_openatsilently droppedO_TRUNCandO_APPENDwhilesys_openhonoured both, so the same redirection behaved differently depending on which entry point libc happened to call.rmdirhad no Linux syscall number, so every Linux-ABI program got ENOSYS.
Phase 3¾ — durability, and the cost of it. DONE. Three things that the earlier phases had assumed rather than arranged.
The commit barrier. Copy-on-write promises that a crash lands on either the
old tree or the new one, but only if the blocks a root points at reach the
medium before the block naming that root. Nothing enforced that: bwrite
order is not platter order on any drive with a write cache. struct schfs_bdev
gained a bflush hook, schfs_vol_commit calls it either side of the uberblock
write, and the kernel implements it as "drain this device's dirty buffers, then
ATA FLUSH CACHE" (ideflush, and SCSI SYNCHRONIZE CACHE for a USB stick).
Falling back through the ring. Mount took the highest-generation uberblock
that validated, checked the tree it named, and gave up if that failed — with up
to 127 older, entirely intact roots sitting on the disk. It now walks the ring
newest-first and adopts the first root it can actually read, counting what it
skipped into vol->recovered so the rollback is reported rather than silent.
Deferred writes, and brackets. The B-tree seals and writes a node on every
record it inserts, so a leaf collecting one file's data checksums went to the
disk once per 4 KB of that file. The kernel's block hook marks buffers dirty
instead (bdirty), and the commit barrier is what makes that safe. On its own
that changes little, because a commit contains a flush and every schfs_k*
call was one commit — so schfs_ktx_open/schfs_ktx_close bracket a run of
calls into one group, and schfs_vfs_write wraps a whole write(2) in one.
Measured over 1 MB of file data, in block writes reaching the device:
| before | after | |
|---|---|---|
| committing every 4 KB | 1947 | 1410 |
| one commit for the megabyte | 1798 | 267 |
267 device writes for 256 blocks of data. Note that the two changes are worth almost nothing separately — deferral cannot help while every commit flushes, and batching cannot help while every write goes straight to the disk — which is why the first attempt at this measured an 8% difference and concluded batching was not the problem.
Phase 3⅞ — where the time actually went. DONE. The filesystem was correct
and durable and the system still felt slow: ls /bin took the better part of a
second. Two causes, neither of them the disk.
Measured first, through /proc/fsstat (counters added for exactly this). The
decisive number was 398,999 block reads against 445 actual disk misses —
99.9% of all block reads were re-reading a block already in the buffer cache. So
the disk was never the problem, and every plan that started with "cache more" or
"read ahead" would have been aimed at 1% of the cost.
The cursor re-read its own buffer. cur_load called node_read
unconditionally, and schfs_cursor_val calls cur_load on every access to
the current record — deliberately, so that a callback disturbing the shared
block buffer cannot corrupt a directory walk. But node_read copies 4096 bytes
out of the buffer cache and then runs CRC32c over all 4096. Listing a 243-entry
directory paid both once per entry. The cursor now remembers which block its
buffer holds and skips the reload, cross-checked against the block header (so a
second cursor sharing the page cannot fool it) and against a global epoch that
every mutation bumps (so a block rewritten in place cannot be served stale).
The checksum was bit-at-a-time. schfs_crc32c did eight shifts per byte —
32,768 iterations to checksum one block — to stay usable before any
initialisation. It builds a 1 KB table on first use now, which keeps that
property and is eight times faster. This only became the top cost after the
cursor fix, which is the usual shape: removing the largest item promotes
whatever was beneath it.
| cycles, one session | before | after |
|---|---|---|
getdents |
26.8 G | 0.14 G (196×) |
read |
26.4 G | 5.5 G (4.8×) |
exec |
18.7 G | 4.0 G (4.7×) |
| block reads | 398,999 | 127,713 |
ls /bin went from ~400 ms to ~20 ms. Verified by reading an 802 KB binary
through the whole path and checking its md5 against the host's copy.
There was also a third of the problem that was not the kernel at all: make qemu ran QEMU without -enable-kvm, so every instruction was emulated. The run
targets probe /dev/kvm now. Measure against the KVM number or you are
measuring TCG.
Phase 4 — extents with refcounts. DONE. cp that costs nothing.
A reference count per run of blocks, in its own tree (SCHFS_KEY_EXTREF under
SCHFS_EXTREF_OBJECTID) rather than in the file's own extent record — because a
count kept per file cannot be decremented by the other file sharing the run.
schfs_file_reflink copies one extent record per run and bumps the count; both
files stay ordinary and writable, and the first write to either copies only the
run it touches. Reachable from userland as FICLONE, Linux's ioctl, so
cp --reflink works unmodified; cmd/reflink is a two-line caller of it.
Measured on the running system: cloning a 411 KB file moved df by 24 KB.
Three things had to be right, and each was wrong first:
- The count is "how many extent records point into this run", not "how many files". Splitting a shared extent around a written block leaves a head and a tail, which is one more reference than before. Missing that leaked four blocks per split.
- A file's extent may point into the middle of a run. Records are keyed by
the block a run was allocated at, so every lookup is a range query
(
extref_find), not an exact one. An exact lookup finds nothing for a split tail, "nothing" means unshared, and the tail is then freed while another file still reaches it. - The space scan has to walk the reference records too. A file truncated inside a shared run describes fewer blocks than exist; the rest would be derived as free while still referenced, and the allocator would hand out a block that a later release frees a second time.
None of those is visible until every holder of a run is deleted, which is why the test that matters is the least glamorous one: after deleting every file that shared a run, free space must be back to exactly what it was.
Phase 5 — attribute index and query. DONE. The reason the design was one
B-tree with many key types from the start: the "database filesystem" feature
turned out to be kernel/fs/schfs/attr.c and nothing else.
An attribute is a named value on an object, stored under (ino, ATTR, hash-of-name) and packed exactly as directory entries are — same collision
argument, same shape. An attribute may be indexed, and an indexed attribute
gets a second set of records keyed by value: (hash-of-name, INDEX, hash-of-value) holding the object ids that carry it. So "every object whose
tag is holiday" is one tree descent instead of a walk of every object.
There is no query language, no parser and no planner, and §6 above explains why: the useful half of "query the filesystem" is the index, and WinFS spent three years and a large team on the other interpretation.
schattr index tag # declare the index
schattr set /tmp/a tag holiday
schattr find tag holiday # one descent, whatever the volume holds
Four decisions worth keeping:
- Which attributes are indexed is cached in the volume, read once at mount.
Every
setxattrhas to ask, and the answer is almost always no; a tree descent to say so would make every attribute write pay for a feature it is not using. - The value is stored in the index entry, not just its hash. Two values can hash alike, and a query that returned the wrong file would be far worse than a slow one.
- An index covers values written after it is declared. There is no reindexing pass. Saying so is better than a half-truth that makes a query answer confidently and wrongly.
- A query on an unindexed attribute is refused, not answered by scanning. A query that silently degrades into a full walk teaches you to use it where it will not scale.
Reachable as setxattr/getxattr/removexattr at Linux's syscall numbers, so
anything that already knows how to attach metadata needs no changes, plus two
native calls (schfs_index, schfs_query) that have no Linux equivalent
because asking a filesystem a question does not.
Phase 5 — attribute index and query. See below.
6. The "SQL" part, honestly
What you want from "works like SQL" is: ask a question about a lot of files and get an answer fast, without walking every directory. That is an index, not a query language. BeFS's design:
- Files carry typed attributes (
string,int,time, …). - An attribute can be indexed: a dedicated B+tree keyed by attribute value, mapping value → object ids.
- A query is a predicate over indexed attributes, evaluated by intersecting index ranges.
In SchFS this is just another key type — (idx, INDEX, key) — in the tree
that already exists. That is the whole reason to build phases 1-4 the btrfs way:
the "database filesystem" feature becomes a few hundred lines on top of
machinery already written and debugged, rather than a separate engine.
A concrete query() syscall and a find-like tool come after that, and are
small by comparison.
7. Honest cost
Phases 1-3 are roughly 3,000-4,000 lines and are the bulk of the work. Phase 1 alone (B-tree + torture harness) is worth doing carefully and unhurriedly; every later phase compounds on it.
This is the largest single piece of work in SchOS. It is also entirely ordinary filesystem engineering with excellent published references (Giampaolo for BeFS; the btrfs design papers; the ZFS uberblock/checksum-in- parent scheme) — there is no research risk here, only volume.
Do this before the installer. The installer's entire job is to create and populate this format. Building it against v1 means building it twice.
7½. Limits
Three different things get called "the limit", and they are worth keeping apart: what the on-disk format can express, what this implementation can currently mount, and what the rest of SchOS can carry. The smallest one is what you actually hit.
The format
| limit | where it comes from | |
|---|---|---|
| Volume | 64 ZiB (2⁶⁴ × 4 KiB blocks) | uint64 block numbers |
| File | 16 EiB (2⁶⁴ bytes) | schfs_inode.size |
| Files per volume | 1.8 × 10¹⁹ | uint64 object ids |
| Hard links per file | 4.3 × 10⁹ | uint32 nlink |
| Filename | 255 bytes | SCHFS_NAME_MAX |
| Entries per directory | no fixed limit | one tree key per name hash |
| Extent | 64 ZiB in one record | uint64 nblocks |
| Tree depth | 8 levels | SCHFS_MAX_LEVEL |
| Block size | 4096 bytes, fixed | SCHFS_BLKSIZE; a mismatch is refused at mount |
| Transaction groups | 1.8 × 10¹⁹ | uint64 generation, never reused |
Directory depth is unbounded on disk; path resolution is what limits it.
This implementation
| limit | where it comes from | |
|---|---|---|
| Volume, mounted | 64 GiB | the in-RAM allocation bitmap: SCHFS_SPACE_PAGES (512) pages × 32768 blocks |
| Volumes mounted at once | 4 | SCHFS_VOL_SLOTS; slot 0 is the root |
| Snapshots mounted at once | 4 | SCHFS_SNAP_SLOTS |
| Volume, via the buffer cache | 4 TiB | bio.c addresses blocks with a uint, and one SchFS block spans four of them |
| RAM per mount | 8 KiB per GiB of volume | two bitmaps, one bit per block each |
| Inline value in a leaf | 1016 bytes | SCHFS_MAX_INLINE = a quarter of a node |
| Colliding names in one directory | ~60, sharing a hash | the packed list for one hash must fit in one inline value |
| Directory entries at 8-byte names | ~42 per key, unlimited keys | as above |
| Records per leaf / children per node | 127 / 84 | 4 KiB minus a 32-byte header |
| Mount time | one pass over the metadata | the allocation map is derived, not stored |
The 64 GiB mount ceiling is the one that bites first, and it is one constant:
raising SCHFS_SPACE_PAGES to 4096 makes it 512 GiB at 64 KiB of pointers per
mount. It is set where it is because the page-pointer vector lives inside
struct schfs_vol, which the kernel holds as static storage.
The rest of SchOS
These are VFS and driver limits rather than SchFS ones:
MAXPATHcaps a path at 128 bytes.NMOUNTallows 8 mounts.- The IDE driver handles the primary channel only: 2 ATA devices.
- The IDE driver uses LBA48 past 128 GiB, so the disk is not the limit there.
- A single
read/writecall is capped at 4 GiB (nis auint); the file itself is not.
The two that used to be here — a 16-bit inode number in struct dirent and a
32-bit inum/size in struct inode — are gone. getdents64 is a per-
filesystem operation now, so SchFS emits its own records with 64-bit inode
numbers and names up to 255 bytes, and the inode carries 64-bit numbers and
sizes throughout.
8. What exists now
include/schfs.h on-disk format + compile-time layout assertions
kernel/fs/schfs/btree.{c,h} the copy-on-write B-tree and its cursor
kernel/fs/schfs/schfs_int.h volume + object layer interface
kernel/fs/schfs/space.c allocation bitmap, derived from the tree at mount
kernel/fs/schfs/vol.c uberblock ring: open, format, commit
kernel/fs/schfs/obj.c read side: inodes, dirents, extents, path lookup
kernel/fs/schfs/objw.c write side: create, link, unlink, rename, write
kernel/fs/schfs/kmount.c kernel: block device, mount slots, transaction bracket
kernel/fs/schfs/vfs.c the fs_ops table: SchFS behind open/read/write
tools/mkschfs.c build an image from a directory tree
tools/schfs_fsck.c verify, list, extract
include/fsstat.h counters behind /proc/fsstat: where the time goes
tools/schfs_test.c B-tree torture harness
tools/schfs_objtest.c object layer: files, directories, space accounting
tools/posix/schfs_roundtrip.sh end-to-end filesystem test
tools/posix/mutate.sh mutation-tests the suites themselves
tools/windows/bootcheck.ps1 boots under QEMU and drives the shell
Everything except kmount.c is compiled both into the kernel and into the
host tools, so the writer and the reader cannot drift apart: there is one
implementation of the format and nothing to disagree with.
Running the tests
make schfs-check all three suites
make schfs-test B-tree only
make schfs-objtest object layer only
make schfs-image-test round-trip only
make schfs-check after touching anything under kernel/fs/schfs.
The round-trip's oracle is the host's own filesystem: an image is built from a
directory tree, extracted with a separate program, and diff -r'd against the
original. It also asserts hard-link counts (which diff cannot see), checks
that rebuilding produces byte-identical output, and runs the whole thing a
second time with the name hash cut to 6 bits so the packed-dirent collision path
— which a 64-bit hash would never reach — is actually exercised.
schfs_objtest runs against a memory-backed volume, so a full
create/write/delete/remount cycle takes milliseconds. Its most important check
is the least glamorous: after deleting everything, free space must be back to
exactly what it was on an empty volume. Every other test in the suite still
passes on a filesystem that leaks a block on every delete.
Its section 10 is crash recovery, and it exists because the whole consistency
argument for this format — "mount takes the newest uberblock that validates, so
a crash lands on either the old tree or the new one" — had no end-to-end test at
all. schfs_test injected crashes into the B-tree; nothing did it to a volume.
The crash is a longjmp out of a block write, which is the only faithful model:
returning an error tests error handling instead, and letting execution continue
with the write swallowed has the filesystem reading back blocks it believes it
just wrote, which no real failure does. It runs the same transaction 119 times,
cutting the power one write later each time, and after every one the volume must
mount, check out structurally, and still hold the data committed before it.
tools/posix/mutate.sh injects six deliberate bugs and fails if any of them goes
undetected. It is how the suites are kept honest; see the note in §9.
Verifying on real hardware
make ramroot root from RAM, so device 1 is free
make build/schfs.img
make qemu-schfs then, at the shell: cat /proc/schfs
.\run.ps1 -SchFS does the same on Windows, and tools\bootcheck.ps1 -SchFS
does it unattended and asserts on the transcript.
9. Things learned the hard way
Recorded because each one cost real time and none is obvious from the code.
- The root's object id must be reserved.
next_objectidstarted atSCHFS_FIRST_OBJECTID, which is the root's id, so the first directory created silently became the root. It then contained itself, and every tree walk recursed until the stack ran out.schfs_fsck --checknow detects directory loops explicitly rather than crashing. - Cursor buffers alias. A range callback gets a pointer into the cursor's block buffer, and almost every readdir caller immediately looks the entry up, which overwrites it. Directory iteration copies each entry out before calling back. Extracting a tree with directories reparented under their own children is what this looks like when it goes wrong.
- A 4096-byte block does not fit on a 4096-byte kernel stack. The cursor
exists precisely to avoid one block buffer per tree level, but
schfs_vol_openstill kept a whole uberblock as a local and blew the stack on the first boot. The kstack canary caught it cleanly. - Extents only merge if you allocate the run first. Writing an extent record
allocates blocks for its own copy-on-write path, so allocating data one block
at a time guarantees the blocks are never adjacent and every block needs its
own extent.
schfs_file_reserveclaims the whole run up front; an 822 KB binary is one extent instead of 201. - Freed blocks are reusable during a build. Copy-on-write forbids reusing a released block because the previously committed tree still points at it — but a build has committed nothing. Recycling them cut the test image from 3596 blocks to 374.
- Reproducibility dies from atime. mkschfs reads every source file, which updates its atime, so copying atime into the image made two builds of the same tree differ. All three timestamps come from mtime.
- An optional hook must still be initialised.
struct schfs_bdevgainedbflush, and every host tool built one on the stack field by field. The tools that did not set the new field handedschfs_vol_commita garbage pointer to test and call, which segfaulted at the first commit. Each of those helpers memsets the struct first now — the fix is not "remember to set it". - Deferring writes makes an all-dirty buffer cache normal.
bgetpanicked with "no buffers" when it could find nothing clean to recycle, which was correct while a dirty buffer meant "the log has not committed". It now writes the least recently used victim out and takes it; a transaction touching more thanNBUFblocks reaches that state long before it commits. - The two performance fixes are worthless apart. Deferring tree writes buys nothing while every commit flushes, and batching commits buys nothing while every write goes straight to the disk. Measuring either alone says "this is not where the time goes", and both measurements are correct.
fsid_tis eight bytes.struct statfsdeclaredf_fsidasint64[2], which is one slot too long, so everything after it landed in the wrong place: userland readf_frsizeout off_namelenand got 255. df then computed in units of 255 bytes and called a 32 MB root filesystem 1992 KB. It survived because every field in that struct used to be an invented constant, so no number in it had ever been checked against reality.