A fresh boot spent 2600 ms in schinit's bootstrap() before printing its first
line, and the whole of it was device nodes. Measured rather than guessed: an
instrumented bootstrap reported 2600 ms on a fresh image and 0 ms on the next
boot, because the nodes already existed by then and mknod failed fast. The cost
was therefore invisible to anyone not building images -- which is to say,
invisible to everyone except whoever works on this.
Each node was a real inode on the root filesystem, and SchFS commits every
mutation as its own transaction group: a copy-on-write rewrite of the path to
the root, then an uberblock behind a cache flush. Twenty of those is seconds.
The duplication mattered more than the time. Major and minor numbers are the
kernel's -- devsw[] in kernel/dev/cdev.c is what they dispatch on -- and schinit
held a second copy of that mapping with nothing checking the two agreed. Adding
a character device meant editing a userspace program, and a wrong minor produced
a node that opened successfully and did nothing.
kernel/fs/devfs.c puts the table next to the thing it describes and serves /dev
behind the same fs_ops vtable procfs uses. The namespace is all it provides:
fileread and filewrite see T_DEV and go through devsw themselves, so a read
never reaches it. It is read-only, and dirlink and ialloc answer EROFS rather
than pretending -- a node exists because the kernel has the device, or it does
not exist.
This also makes /proc/filesystems true. It has advertised "nodev devtmpfs" for
as long as it has existed, against a kernel that had no such filesystem.
Two things found while measuring what remained:
* The thirteen mknod() calls for /proc ran *after* the procfs mount, so every
one was resolving a path into a read-only synthetic filesystem and failing.
procfs.c's own note called them a fallback to be cleaned up later; they have
been dead for longer than that.
* The four mkdir() calls for mount points were the rest of it. They are made
by the image builder now (sysroot-install), and the ones in init are a
fallback -- a mkdir that fails because the directory exists costs a path
walk and no writes, where creating one costs a transaction.
bootstrap() is now 0 ms on a fresh image, down from 2600.
init opens /dev/console after the mount rather than mknod'ing a node called
"console" into the root directory, which is what used to leave a stray /console
in `ls /`. The old path is kept as a fallback: if the mount fails there is no
/dev/console and no way to report it.
socktest passes, /proc still lists per-process directories, ps -e still
enumerates.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
Everything created after boot was stamped 1970. know() in SchFS and now_sec() in
the v1 filesystem both returned uptime(), under comments saying the kernel had no
wall clock -- which stopped being true when time_init() started reading the RTC.
The comments outlived the condition they described.
Two errors, and the second is the less obvious: uptime() counts 10 ms *ticks* and
they were being used as *seconds*, so a node created 0.6 s into boot read back as
00:01:00 and the clocks ran a hundred times too fast. Both now go through
wallclock(), which exists precisely so nothing can disagree with gettimeofday
about the time. Confirmed against the RTC line from the same boot.
The occupancy counters are the instrumentation docs/plan/roadmap.md asked for
while hunting the ls -lR hang: inode, file and pinned-buffer counts with a
high-water mark each, maintained at the 0->1 and 1->0 transitions rather than by
scanning, so they cost one increment on a path that already holds the table's
lock.
The peaks are the diagnostic. A live count only describes the moment you looked,
and a reference leak is invisible in it right up until the table is full -- at
which point iget() panics and filealloc() returns 0, which is where the trail
used to go cold.
What they measured: across ten iterations of ls -lR with writes interleaved, 33
execs and 13378 tree descents, inode_peak stayed at 9/200, file_peak at 9/512 and
buf_pinned_peak at 1/256, with the SchFS bracket unlocked throughout. That
eliminates the roadmap's own hypothesis -- "a lock or reference-count leak
somewhere in the *at() paths" -- and the hang did not reproduce. It is very
likely fixed in passing by the *at() dirfd rewrite, the getdents cookie rewrite
or SchFS becoming the root, all of which landed after it was reported.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
The last three phases of the filesystem, which were written together and
touch the same files throughout -- kmount.c, objw.c and vfs.c are each
rewritten by all three -- so they land as one commit rather than three
that could not be built separately.
Durability. A transaction that reaches the platter out of order is worse
than one that never reached it: the uberblock can name a tree whose
blocks are still in the drive's write cache. bdirty/bflush give the
buffer layer an explicit barrier, ideflush issues ATA FLUSH CACHE and
usb_storage_flush the SCSI SYNCHRONIZE CACHE that means the same thing,
and a commit now waits for the data before it writes the root that
points at it. The uberblock groups are read newest-first with fallback,
so a volume whose newest roots are torn mounts at an older one and says
which generations were lost.
Reflink. Extents carry a refcount, so two objects can name the same
blocks and neither frees them while the other holds them. Exposed as
FICLONE -- Linux's ioctl value, so `cp --reflink` works unmodified --
and as cmd/reflink for the case where there is no cp to hand.
Attributes, and the one genuinely new idea here. attr.c stores named
values on an object, reachable through setxattr/getxattr/removexattr at
Linux's syscall numbers. On top of that, schfs_index declares an index
over an attribute name and schfs_query answers "which objects carry this
value" from the B-tree rather than by walking. A query against an
unindexed attribute is refused with an error, never silently answered by
a full scan: a query that quietly becomes a scan is worse than one that
admits it cannot help. cmd/schattr drives both.
Three bugs fixed on the way, each found by the numbers being wrong
rather than by inspection:
- statfs's f_fsid was declared int64[2]. Linux's fsid_t is two *ints*,
so every field after it landed one slot late: userland read f_frsize
out of f_namelen, got 255, and df reported a 32 MB root filesystem
as 1992 KB. It hid for as long as it did because the fields used to
be invented constants, so wrong numbers looked like the usual ones.
- struct inode's nlink was a short against SchFS's uint32, so a file
past 32767 names came back negative -- and iput frees an object
whose count reaches zero. It is an int now, signed on purpose, so an
underflow lands somewhere a check can see rather than wrapping.
- filewrite split every write to fit v1's log. That limit belongs to
the journal, not to writing, and SchFS makes one transaction of
whatever it is handed: a 1 MB write was becoming 170 commits and
170 flushes for nothing. fs_ops carries max_write now, and SchFS
sets no limit.
Also here: a real mount table, so `mount -t schfs /dev/hdb /mnt` shows
up in /proc/mounts -- df looks a path up there before it calls statfs at
all, so a mount missing from that list was invisible to it -- several
volumes mounted at once, snapshots mounted by name, and /proc/fsstat
with an rdtsc-based counter, which is the only instrument fine enough to
see inside a single syscall.
Verified: `make schfs-check` passes in full, including the sweep that
crashes the volume at each of 119 points mid-transaction and requires
every one to mount with exactly its committed data, and the case where
both newest uberblocks are rubbish and the volume still comes back. The
headless boot check passes with 27 [ok] and no failures, on a SchFS root.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two loose ends from the mount-by-name commit, both closed.
A slot is a device number, so supporting more than one snapshot is a
small array rather than a redesign: SCHFS_SNAP_DEV through
SCHFS_SNAP_DEV + SCHFS_SNAP_SLOTS - 1, each with its own volume and its
own scratch pages, allocated on first use. Four of them -- "one at a
time" is an odd thing to have to explain, and the cost is two pages per
slot. Not many, because each one is a device number and those are a
small fixed space.
The leak mattered more. umount removed the mount table entry and left the
slot marked in use, so a snapshot could be unmounted and then never
mounted again until reboot. sys_umount2 releases it now, keyed off the
device it already has in hand.
Verified in the running system: two snapshots mounted at once, both
readable, both unmounted, and a third mount afterwards succeeds and reads
correctly -- which is the case that failed before.
schfs-check passes; bootcheck 108 [ok].
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`mount -t schfs @golden /mnt` mounts a snapshot. The source argument
finally means something: sys_mount ignored it outright, because there was
one volume and nothing to choose between. `@name` rather than a path
because a snapshot is not a device -- there is no node in /dev for it and
there should not be, since it lives inside the volume already mounted.
A second volume rather than a parameter threaded through all 23 schfs_k*
entry points. Only the read side can ever address it, so the write path
keeps talking to the live volume and needed no change; the four read
entry points grew _dev variants and pick the volume from the device the
inode already carries. One fs_ops table serves both, because every entry
takes an inode and dispatches on it -- a second table would be the same
pointers under a different name.
It needs its own device number: a snapshot's object ids are the *same
numbers* as the live tree's, so one device would make the two views alias
in the inode cache and hand a reader whichever was cached first.
Three things this turned up, none of them new but all of them reachable
for the first time:
- sys_mount refused with `dev == SCHFS_DEV`, meaning "do not mount SchFS
onto SchFS". A snapshot's mount point is necessarily on the live root,
so that refused the entire feature. The real condition is mounting a
device onto *itself*, which is `dev == fsdev`.
- createat allocates an inode and then panics if dirlink fails --
reasonable about a writable filesystem, a crash on a read-only one.
The refusal now happens before anything is allocated. Unlink and
rename already handled a refusal properly.
- a refused create reported ENOENT, so writing to a snapshot said "No
such file or directory" -- the sentence a shell prints for every
create failure. It reports EROFS now.
Verified in the running system against an image built with
--snapshot golden: the mount succeeds, ls lists the snapshot's tree,
/mnt2/etc/motd reads back through it, and a write is refused with
"Read-only file system".
One at a time, and no umount path for it yet: mount_del_by_root removes
the mount table entry, but ksnap_unmount is only called on the failure
paths. schfs-check passes; bootcheck 108 [ok].
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An x86-64 operating system built from xv6, now largely rewritten: SMP,
a copy-on-write root filesystem (SchFS), PCI/USB/virtio drivers, a TTF
rasteriser, and a Linux-numbered syscall ABI.
No history before this point -- the tree was never under version control.
This commit exists so the rename that follows has something to undo to.
Nexoniarz