A fresh boot spent 2600 ms in schinit's bootstrap() before printing its first
line, and the whole of it was device nodes. Measured rather than guessed: an
instrumented bootstrap reported 2600 ms on a fresh image and 0 ms on the next
boot, because the nodes already existed by then and mknod failed fast. The cost
was therefore invisible to anyone not building images -- which is to say,
invisible to everyone except whoever works on this.
Each node was a real inode on the root filesystem, and SchFS commits every
mutation as its own transaction group: a copy-on-write rewrite of the path to
the root, then an uberblock behind a cache flush. Twenty of those is seconds.
The duplication mattered more than the time. Major and minor numbers are the
kernel's -- devsw[] in kernel/dev/cdev.c is what they dispatch on -- and schinit
held a second copy of that mapping with nothing checking the two agreed. Adding
a character device meant editing a userspace program, and a wrong minor produced
a node that opened successfully and did nothing.
kernel/fs/devfs.c puts the table next to the thing it describes and serves /dev
behind the same fs_ops vtable procfs uses. The namespace is all it provides:
fileread and filewrite see T_DEV and go through devsw themselves, so a read
never reaches it. It is read-only, and dirlink and ialloc answer EROFS rather
than pretending -- a node exists because the kernel has the device, or it does
not exist.
This also makes /proc/filesystems true. It has advertised "nodev devtmpfs" for
as long as it has existed, against a kernel that had no such filesystem.
Two things found while measuring what remained:
* The thirteen mknod() calls for /proc ran *after* the procfs mount, so every
one was resolving a path into a read-only synthetic filesystem and failing.
procfs.c's own note called them a fallback to be cleaned up later; they have
been dead for longer than that.
* The four mkdir() calls for mount points were the rest of it. They are made
by the image builder now (sysroot-install), and the ones in init are a
fallback -- a mkdir that fails because the directory exists costs a path
walk and no writes, where creating one costs a transaction.
bootstrap() is now 0 ms on a fresh image, down from 2600.
init opens /dev/console after the mount rather than mknod'ing a node called
"console" into the root directory, which is what used to leave a stray /console
in `ls /`. The old path is kept as a fallback: if the mount fails there is no
/dev/console and no way to report it.
socktest passes, /proc still lists per-process directories, ps -e still
enumerates.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
9.5 KiB
The VFS
The layer between open(2) and a filesystem. SchFS is described separately in
schfs.md; this is the machinery it plugs into, plus the buffer cache,
mounts, and procfs.
struct fs_ops
A filesystem is a vtable, chosen by iget() from the device number and never
null once an inode is held.
struct fs_ops {
const char *name;
void (*iload) (struct inode *); // fill type/size/nlink from disk
void (*istore) (struct inode *); // write them back
void (*ifree) (struct inode *); // last link gone, release it
int (*read) (struct inode *, int user_dst, uint64 dst, uint64 off, uint n);
int (*write) (struct inode *, int user_src, uint64 src, uint64 off, uint n);
void (*truncate)(struct inode *, uint64 newsize);
struct inode *(*dirlookup)(struct inode *dp, char *name, uint *poff);
int (*dirlink) (struct inode *dp, char *name, uint64 inum);
int (*dirunlink)(struct inode *dp, char *name, uint off);
int (*dirempty) (struct inode *dp);
struct inode *(*ialloc)(uint dev, short type);
int (*getdents)(struct inode *dp, char *out, int cap, uint64 *off);
};
Two implementations exist: schfs_ops (kernel/fs/schfs/vfs.c) and procfs_ops
(kernel/fs/procfs.c).
poff is deliberately an opaque cookie rather than a byte offset —
meaningful only to the same filesystem's dirunlink. A directory does not have
to be an array of fixed-size records for this interface to work.
ialloc creates an object with no name. The caller sets nlink and calls
dirlink; until it does, nothing can reach the object. If the caller dies in
between, the result is an unreferenced record that is harmless and, in SchFS,
never reaches the disk anyway — nothing is durable until the transaction commits.
The one place the models disagree
The VFS inherited xv6's shape: allocate an inode, fill it in, link a name to it. SchFS has no free-standing inode — an object exists because a record with its id exists.
nlink is the sharper disagreement. The VFS adjusts it itself and then calls
iupdate, so SchFS's own link accounting is deliberately bypassed in the VFS
path: schfs_dir_add/schfs_dir_del are used rather than schfs_link/
schfs_unlink, and iput's ifree hook releases the object once the VFS has
decided there are no names left.
Two pieces of code counting links independently is precisely how a filesystem loses a file that is still referenced.
nlink is an int, not a short. SchFS stores a uint32 and the narrower type
truncated it, so a file past 32767 names came back with a negative count — and
iput frees an object whose count reaches zero. Signed on purpose: nlink-- on
a count that is somehow already zero should land somewhere a check can see it,
not wrap to four billion.
Path resolution
namex() walks a path one component at a time. It was refactored into
namex_from(struct inode *start, ...), parameterising what used to be a
hardcoded idup(proc->cwd).
That refactor is what made the *at() family work. Every one of them — openat,
fstatat, unlinkat, renameat, linkat, symlinkat, mkdirat, mknodat,
readlinkat, faccessat — previously handled only two cases: AT_FDCWD, or an
absolute path with dirfd ignored. A real dirfd with a relative path is
the standard, efficient way any directory-tree walker avoids rebuilding full
path strings, and it was either rejected outright or — worse, in openat —
silently resolved against proc->cwd, which can open the wrong file entirely.
sys_fs.c now has at_dir(dirfd), at_namei() and at_nameiparent(), and
every *at() call uses them.
The buffer cache
kernel/dev/bio.c, xv6's design unchanged: a fixed pool of buffers, an LRU
list, one sleeplock per buffer. bread() returns a locked buffer; bwrite()
flushes it; brelse() releases.
Sleeping is the point. A buffer lock is held across a disk read, which is
why every path that touches a filesystem must be able to sleep — and why the
root mount happens in forkret() rather than in main().
Mounts
mount() and umount2() at their Linux numbers, with a mount table consulted
during path resolution. /proc is mounted by init:
mount("proc", "/proc", "proc", 0, 0);
procfs
kernel/fs/procfs.c. A real filesystem behind the same vtable, not a set of
character devices.
The reason it had to become one is ps. /proc used to be mknod'd nodes —
init ran mknod("/proc/uptime", 6, 2) and friends, so every file was a real
inode on the root filesystem with a major/minor behind it. That is fine for a
fixed list and cannot work at all for the thing /proc is mostly for: one
directory per process. You cannot mknod a directory for a process that does
not exist yet and will be gone in a minute. ps listed /proc, got no numbers,
and reported nothing. It was never a ps problem.
Nothing is stored. Directory listings are generated from the process table when asked; file contents when read.
Three things worth knowing:
- The split is deliberate.
kernel/fs/procfs.cowns the namespace — what exists.kernel/dev/procfs.ckeeps the generators — what the global files say. They change for different reasons, andproc_gen_global()is the only thing that crosses between them. - procfs never walks the process table.
proc_snapshot()copies what is needed out from underptable.lock, so a process that exits between the readdir that listed it and the open that reads it produces a stale line rather than a use-after-free. - Sizes are reported as zero, exactly as Linux does. A synthetic file has no
length until it is generated, and generating every one on every
stat()to fill in a number almost nothing reads would be absurd.
devtmpfs
/dev is generated, not stored. kernel/fs/devfs.c, behind the same fs_ops
vtable, mounted by init with mount("dev", "/dev", "devtmpfs", 0, 0).
It used to be twenty mknod() calls in cmd/schinit, which was wrong twice
over:
The cost. Every node was a real inode on the root filesystem, and SchFS commits each mutation as its own transaction group — a copy-on-write rewrite of the path to the root, then an uberblock behind a cache flush. Measured at 2600 ms of a first boot, spent before init printed its first line. On later boots the nodes already existed and the same code took 0 ms, so the cost was invisible to anyone not building fresh images — which is to say, invisible to everyone except whoever works on the system.
The duplication, which matters more. Major and minor numbers are the
kernel's: they are what devsw[] in kernel/dev/cdev.c dispatches on. init
held a second copy of that mapping and nothing checked the two agreed. Adding a
character device meant editing a userspace program, and a wrong minor there
produced a node that opened successfully and did nothing.
The table now sits next to the thing it describes. The namespace is all devfs
provides — reads and writes never reach it, because fileread/filewrite see
T_DEV and dispatch through devsw themselves.
It is read-only: dirlink and ialloc answer EROFS rather than pretending.
A node exists because the kernel has the device, or it does not exist.
What is there
console, null, keyboard, zero, full, random, urandom, tty,
stdin, stdout, stderr, fb0, mouse, hda, hdb, and input/event0
and input/event1 in one subdirectory. Minors under major 1 follow Linux's
mem-major layout where it costs nothing.
The raw disks are created whether or not a drive is present: opening one with
nothing behind it fails with ENXIO, which is a better answer than a missing
path — and with no node at all, echo x > /dev/hdb silently creates an ordinary
file of that name and appears to work.
What is there
Per-process directories, plus the global files: meminfo, uptime, cpuinfo,
version, stat, loadavg, mounts, filesystems, schfs, pci,
interrupts, devices, netif, and sysrq.
/proc is read-only except sysrq, which is the one deliberate write target,
and the code says so in one place. Marking every procfs inode read-only once
broke echo s > /proc/sysrq and took out three USB checks within a minute —
they drive the disk through sysrq.
sysrq keys: c crash on purpose, u / s storage checks, d display
selftest, t text specimen, n the netdev ARP selftest.
What it does not do
- No page cache.
mmapof a file reads the whole thing in at map time, and writableMAP_SHAREDof a file is refused rather than faked. This is the single largest gap in the layer. - No symlinks in procfs, which is why
/proc/<pid>/exeis a regular file andps -o commprints[init]. - No
AF_UNIXsocket inodes. Socket names live in a table inkernel/net/unix.c, sobind()creates nothing that can bestat()ed orunlink()ed. - No unified inode cache across filesystems beyond the single
itable. - A known, un-root-caused hang: heavy
ls -lRuse across many commands in one long shell session can eventually make a later, unrelated directory hang; the identical command succeeds on a fresh boot. It is not inode or file table exhaustion — both fail cleanly and neither fired. It smells like a lock or refcount leak in the*at()paths or in toybox's own fd handling. The next step is instrumentingitable/ftableoccupancy across a long session to see what is climbing. sys_delete/sys_show/sys_showstatare orphaned entries from the old v1 filesystem's recycle bin. Nothing reaches them and they die withfs.c.