A fresh boot spent 2600 ms in schinit's bootstrap() before printing its first
line, and the whole of it was device nodes. Measured rather than guessed: an
instrumented bootstrap reported 2600 ms on a fresh image and 0 ms on the next
boot, because the nodes already existed by then and mknod failed fast. The cost
was therefore invisible to anyone not building images -- which is to say,
invisible to everyone except whoever works on this.
Each node was a real inode on the root filesystem, and SchFS commits every
mutation as its own transaction group: a copy-on-write rewrite of the path to
the root, then an uberblock behind a cache flush. Twenty of those is seconds.
The duplication mattered more than the time. Major and minor numbers are the
kernel's -- devsw[] in kernel/dev/cdev.c is what they dispatch on -- and schinit
held a second copy of that mapping with nothing checking the two agreed. Adding
a character device meant editing a userspace program, and a wrong minor produced
a node that opened successfully and did nothing.
kernel/fs/devfs.c puts the table next to the thing it describes and serves /dev
behind the same fs_ops vtable procfs uses. The namespace is all it provides:
fileread and filewrite see T_DEV and go through devsw themselves, so a read
never reaches it. It is read-only, and dirlink and ialloc answer EROFS rather
than pretending -- a node exists because the kernel has the device, or it does
not exist.
This also makes /proc/filesystems true. It has advertised "nodev devtmpfs" for
as long as it has existed, against a kernel that had no such filesystem.
Two things found while measuring what remained:
* The thirteen mknod() calls for /proc ran *after* the procfs mount, so every
one was resolving a path into a read-only synthetic filesystem and failing.
procfs.c's own note called them a fallback to be cleaned up later; they have
been dead for longer than that.
* The four mkdir() calls for mount points were the rest of it. They are made
by the image builder now (sysroot-install), and the ones in init are a
fallback -- a mkdir that fails because the directory exists costs a path
walk and no writes, where creating one costs a transaction.
bootstrap() is now 0 ms on a fresh image, down from 2600.
init opens /dev/console after the mount rather than mknod'ing a node called
"console" into the root directory, which is what used to leave a stray /console
in `ls /`. The old path is kept as a fallback: if the mount fails there is no
/dev/console and no way to report it.
socktest passes, /proc still lists per-process directories, ps -e still
enumerates.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
211 lines
9.5 KiB
Markdown
211 lines
9.5 KiB
Markdown
# The VFS
|
|
|
|
The layer between `open(2)` and a filesystem. SchFS is described separately in
|
|
[schfs.md](schfs.md); this is the machinery it plugs into, plus the buffer cache,
|
|
mounts, and procfs.
|
|
|
|
---
|
|
|
|
## `struct fs_ops`
|
|
|
|
A filesystem is a vtable, chosen by `iget()` from the device number and never
|
|
null once an inode is held.
|
|
|
|
```c
|
|
struct fs_ops {
|
|
const char *name;
|
|
|
|
void (*iload) (struct inode *); // fill type/size/nlink from disk
|
|
void (*istore) (struct inode *); // write them back
|
|
void (*ifree) (struct inode *); // last link gone, release it
|
|
|
|
int (*read) (struct inode *, int user_dst, uint64 dst, uint64 off, uint n);
|
|
int (*write) (struct inode *, int user_src, uint64 src, uint64 off, uint n);
|
|
void (*truncate)(struct inode *, uint64 newsize);
|
|
|
|
struct inode *(*dirlookup)(struct inode *dp, char *name, uint *poff);
|
|
int (*dirlink) (struct inode *dp, char *name, uint64 inum);
|
|
int (*dirunlink)(struct inode *dp, char *name, uint off);
|
|
int (*dirempty) (struct inode *dp);
|
|
|
|
struct inode *(*ialloc)(uint dev, short type);
|
|
int (*getdents)(struct inode *dp, char *out, int cap, uint64 *off);
|
|
};
|
|
```
|
|
|
|
Two implementations exist: `schfs_ops` (`kernel/fs/schfs/vfs.c`) and `procfs_ops`
|
|
(`kernel/fs/procfs.c`).
|
|
|
|
`poff` is deliberately **an opaque cookie** rather than a byte offset —
|
|
meaningful only to the same filesystem's `dirunlink`. A directory does not have
|
|
to be an array of fixed-size records for this interface to work.
|
|
|
|
`ialloc` creates an object **with no name**. The caller sets `nlink` and calls
|
|
`dirlink`; until it does, nothing can reach the object. If the caller dies in
|
|
between, the result is an unreferenced record that is harmless and, in SchFS,
|
|
never reaches the disk anyway — nothing is durable until the transaction commits.
|
|
|
|
### The one place the models disagree
|
|
|
|
The VFS inherited xv6's shape: allocate an inode, fill it in, link a name to it.
|
|
SchFS has no free-standing inode — an object exists because a record with its id
|
|
exists.
|
|
|
|
`nlink` is the sharper disagreement. **The VFS adjusts it itself** and then calls
|
|
`iupdate`, so SchFS's own link accounting is deliberately bypassed in the VFS
|
|
path: `schfs_dir_add`/`schfs_dir_del` are used rather than `schfs_link`/
|
|
`schfs_unlink`, and `iput`'s `ifree` hook releases the object once the VFS has
|
|
decided there are no names left.
|
|
|
|
> Two pieces of code counting links independently is precisely how a filesystem
|
|
> loses a file that is still referenced.
|
|
|
|
`nlink` is an `int`, not a `short`. SchFS stores a `uint32` and the narrower type
|
|
truncated it, so a file past 32767 names came back with a *negative* count — and
|
|
`iput` frees an object whose count reaches zero. Signed on purpose: `nlink--` on
|
|
a count that is somehow already zero should land somewhere a check can see it,
|
|
not wrap to four billion.
|
|
|
|
## Path resolution
|
|
|
|
`namex()` walks a path one component at a time. It was refactored into
|
|
`namex_from(struct inode *start, ...)`, parameterising what used to be a
|
|
hardcoded `idup(proc->cwd)`.
|
|
|
|
That refactor is what made the `*at()` family work. Every one of them — `openat`,
|
|
`fstatat`, `unlinkat`, `renameat`, `linkat`, `symlinkat`, `mkdirat`, `mknodat`,
|
|
`readlinkat`, `faccessat` — previously handled only two cases: `AT_FDCWD`, or an
|
|
absolute path with `dirfd` ignored. A **real** dirfd with a **relative** path is
|
|
the standard, efficient way any directory-tree walker avoids rebuilding full
|
|
path strings, and it was either rejected outright or — worse, in `openat` —
|
|
silently resolved against `proc->cwd`, which can open the wrong file entirely.
|
|
|
|
`sys_fs.c` now has `at_dir(dirfd)`, `at_namei()` and `at_nameiparent()`, and
|
|
every `*at()` call uses them.
|
|
|
|
## The buffer cache
|
|
|
|
`kernel/dev/bio.c`, xv6's design unchanged: a fixed pool of buffers, an LRU
|
|
list, one sleeplock per buffer. `bread()` returns a locked buffer; `bwrite()`
|
|
flushes it; `brelse()` releases.
|
|
|
|
**Sleeping is the point.** A buffer lock is held across a disk read, which is
|
|
why every path that touches a filesystem must be able to sleep — and why the
|
|
root mount happens in `forkret()` rather than in `main()`.
|
|
|
|
## Mounts
|
|
|
|
`mount()` and `umount2()` at their Linux numbers, with a mount table consulted
|
|
during path resolution. `/proc` is mounted by init:
|
|
|
|
```c
|
|
mount("proc", "/proc", "proc", 0, 0);
|
|
```
|
|
|
|
## procfs
|
|
|
|
`kernel/fs/procfs.c`. A real filesystem behind the same vtable, not a set of
|
|
character devices.
|
|
|
|
**The reason it had to become one is `ps`.** /proc used to be `mknod`'d nodes —
|
|
init ran `mknod("/proc/uptime", 6, 2)` and friends, so every file was a real
|
|
inode on the root filesystem with a major/minor behind it. That is fine for a
|
|
fixed list and cannot work at all for the thing /proc is mostly for: **one
|
|
directory per process**. You cannot `mknod` a directory for a process that does
|
|
not exist yet and will be gone in a minute. `ps` listed `/proc`, got no numbers,
|
|
and reported nothing. It was never a `ps` problem.
|
|
|
|
Nothing is stored. Directory listings are generated from the process table when
|
|
asked; file contents when read.
|
|
|
|
Three things worth knowing:
|
|
|
|
- **The split is deliberate.** `kernel/fs/procfs.c` owns the *namespace* — what
|
|
exists. `kernel/dev/procfs.c` keeps the *generators* — what the global files
|
|
say. They change for different reasons, and `proc_gen_global()` is the only
|
|
thing that crosses between them.
|
|
- **procfs never walks the process table.** `proc_snapshot()` copies what is
|
|
needed out from under `ptable.lock`, so a process that exits between the
|
|
readdir that listed it and the open that reads it produces a stale line rather
|
|
than a use-after-free.
|
|
- **Sizes are reported as zero**, exactly as Linux does. A synthetic file has no
|
|
length until it is generated, and generating every one on every `stat()` to
|
|
fill in a number almost nothing reads would be absurd.
|
|
|
|
## devtmpfs
|
|
|
|
`/dev` is generated, not stored. `kernel/fs/devfs.c`, behind the same `fs_ops`
|
|
vtable, mounted by init with `mount("dev", "/dev", "devtmpfs", 0, 0)`.
|
|
|
|
It used to be twenty `mknod()` calls in `cmd/schinit`, which was wrong twice
|
|
over:
|
|
|
|
**The cost.** Every node was a real inode on the root filesystem, and SchFS
|
|
commits each mutation as its own transaction group — a copy-on-write rewrite of
|
|
the path to the root, then an uberblock behind a cache flush. Measured at
|
|
**2600 ms of a first boot**, spent before init printed its first line. On later
|
|
boots the nodes already existed and the same code took 0 ms, so the cost was
|
|
invisible to anyone not building fresh images — which is to say, invisible to
|
|
everyone except whoever works on the system.
|
|
|
|
**The duplication, which matters more.** Major and minor numbers are the
|
|
kernel's: they are what `devsw[]` in `kernel/dev/cdev.c` dispatches on. init
|
|
held a second copy of that mapping and nothing checked the two agreed. Adding a
|
|
character device meant editing a userspace program, and a wrong minor there
|
|
produced a node that opened successfully and did nothing.
|
|
|
|
The table now sits next to the thing it describes. The namespace is all devfs
|
|
provides — reads and writes never reach it, because `fileread`/`filewrite` see
|
|
`T_DEV` and dispatch through `devsw` themselves.
|
|
|
|
It is read-only: `dirlink` and `ialloc` answer `EROFS` rather than pretending.
|
|
A node exists because the kernel has the device, or it does not exist.
|
|
|
|
### What is there
|
|
|
|
`console`, `null`, `keyboard`, `zero`, `full`, `random`, `urandom`, `tty`,
|
|
`stdin`, `stdout`, `stderr`, `fb0`, `mouse`, `hda`, `hdb`, and `input/event0`
|
|
and `input/event1` in one subdirectory. Minors under major 1 follow Linux's
|
|
mem-major layout where it costs nothing.
|
|
|
|
The raw disks are created whether or not a drive is present: opening one with
|
|
nothing behind it fails with `ENXIO`, which is a better answer than a missing
|
|
path — and with no node at all, `echo x > /dev/hdb` silently creates an ordinary
|
|
file of that name and appears to work.
|
|
|
|
### What is there
|
|
|
|
Per-process directories, plus the global files: `meminfo`, `uptime`, `cpuinfo`,
|
|
`version`, `stat`, `loadavg`, `mounts`, `filesystems`, `schfs`, `pci`,
|
|
`interrupts`, `devices`, `netif`, and `sysrq`.
|
|
|
|
**/proc is read-only except `sysrq`**, which is the one deliberate write target,
|
|
and the code says so in one place. Marking every procfs inode read-only once
|
|
broke `echo s > /proc/sysrq` and took out three USB checks within a minute —
|
|
they drive the disk through sysrq.
|
|
|
|
`sysrq` keys: `c` crash on purpose, `u` / `s` storage checks, `d` display
|
|
selftest, `t` text specimen, `n` the netdev ARP selftest.
|
|
|
|
---
|
|
|
|
## What it does not do
|
|
|
|
- **No page cache.** `mmap` of a file reads the whole thing in at map time, and
|
|
writable `MAP_SHARED` of a file is refused rather than faked. This is the
|
|
single largest gap in the layer.
|
|
- **No symlinks in procfs**, which is why `/proc/<pid>/exe` is a regular file
|
|
and `ps -o comm` prints `[init]`.
|
|
- **No `AF_UNIX` socket inodes.** Socket names live in a table in
|
|
`kernel/net/unix.c`, so `bind()` creates nothing that can be `stat()`ed or
|
|
`unlink()`ed.
|
|
- **No unified inode cache across filesystems** beyond the single `itable`.
|
|
- **A known, un-root-caused hang:** heavy `ls -lR` use across many commands in
|
|
one long shell session can eventually make a *later*, unrelated directory
|
|
hang; the identical command succeeds on a fresh boot. It is not inode or file
|
|
table exhaustion — both fail cleanly and neither fired. It smells like a lock
|
|
or refcount leak in the `*at()` paths or in toybox's own fd handling. The next
|
|
step is instrumenting `itable`/`ftable` occupancy across a long session to see
|
|
what is climbing.
|
|
- **`sys_delete`/`sys_show`/`sys_showstat`** are orphaned entries from the old
|
|
v1 filesystem's recycle bin. Nothing reaches them and they die with `fs.c`.
|