nexoniarz/SchOS
nexoniarz/SchOS
Watch
1
0
Fork
You've already forked SchOS
 0
Code Issues Pull requests Projects Releases 1 Packages Wiki Activity Actions

schfs: the commit barrier, reflink, and attributes with an index over them

Browse source 
The last three phases of the filesystem, which were written together and
touch the same files throughout -- kmount.c, objw.c and vfs.c are each
rewritten by all three -- so they land as one commit rather than three
that could not be built separately.

Durability. A transaction that reaches the platter out of order is worse
than one that never reached it: the uberblock can name a tree whose
blocks are still in the drive's write cache. bdirty/bflush give the
buffer layer an explicit barrier, ideflush issues ATA FLUSH CACHE and
usb_storage_flush the SCSI SYNCHRONIZE CACHE that means the same thing,
and a commit now waits for the data before it writes the root that
points at it. The uberblock groups are read newest-first with fallback,
so a volume whose newest roots are torn mounts at an older one and says
which generations were lost.

Reflink. Extents carry a refcount, so two objects can name the same
blocks and neither frees them while the other holds them. Exposed as
FICLONE -- Linux's ioctl value, so `cp --reflink` works unmodified --
and as cmd/reflink for the case where there is no cp to hand.

Attributes, and the one genuinely new idea here. attr.c stores named
values on an object, reachable through setxattr/getxattr/removexattr at
Linux's syscall numbers. On top of that, schfs_index declares an index
over an attribute name and schfs_query answers "which objects carry this
value" from the B-tree rather than by walking. A query against an
unindexed attribute is refused with an error, never silently answered by
a full scan: a query that quietly becomes a scan is worse than one that
admits it cannot help. cmd/schattr drives both.

Three bugs fixed on the way, each found by the numbers being wrong
rather than by inspection:

  - statfs's f_fsid was declared int64[2]. Linux's fsid_t is two *ints*,
    so every field after it landed one slot late: userland read f_frsize
    out of f_namelen, got 255, and df reported a 32 MB root filesystem
    as 1992 KB. It hid for as long as it did because the fields used to
    be invented constants, so wrong numbers looked like the usual ones.
  - struct inode's nlink was a short against SchFS's uint32, so a file
    past 32767 names came back negative -- and iput frees an object
    whose count reaches zero. It is an int now, signed on purpose, so an
    underflow lands somewhere a check can see rather than wrapping.
  - filewrite split every write to fit v1's log. That limit belongs to
    the journal, not to writing, and SchFS makes one transaction of
    whatever it is handed: a 1 MB write was becoming 170 commits and
    170 flushes for nothing. fs_ops carries max_write now, and SchFS
    sets no limit.

Also here: a real mount table, so `mount -t schfs /dev/hdb /mnt` shows
up in /proc/mounts -- df looks a path up there before it calls statfs at
all, so a mount missing from that list was invisible to it -- several
volumes mounted at once, snapshots mounted by name, and /proc/fsstat
with an rdtsc-based counter, which is the only instrument fine enough to
see inside a single syscall.

Verified: `make schfs-check` passes in full, including the sweep that
crashes the volume at each of 119 points mid-transaction and requires
every one to mount with exactly its committed data, and the case where
both newest uberblocks are rubbish and the volume still comes back. The
headless boot check passes with 27 [ok] and no failures, on a SchFS root.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Nexoniarz 2026-08-15 16:01:22 +02:00
commit 03cc316001
39 changed files with 3942 additions and 550 deletions
Download patch file Download diff file Expand all files Collapse all files

5
Makefile
View file

@ -54,7 +54,7 @@ KERN_FS = $(addprefix $(BUILD)/kernel/fs/, \
# the kernel and mkschfs cannot disagree about the on-disk format: there is one
# implementation, found and fixed on the host where a test takes a second.
KERN_FS += $(addprefix $(BUILD)/kernel/fs/schfs/, \
btree.o space.o vol.o obj.o objw.o kmount.o vfs.o)
btree.o space.o vol.o obj.o objw.o attr.o kmount.o vfs.o)
# Sockets. Unix domain only for now; an IP stack becomes more files here
# rather than a reshaping of what is already written.
@ -86,7 +86,7 @@ KERN_NOENTRY = $(filter-out $(BUILD)/kernel/arch/x86_64/entry.o, $(KERNOBJS))
# Third-party software does NOT go here — see ports/.
# -----------------------------------------------------------------------
ULIB = $(addprefix $(BUILD)/lib/, start.o glue.o usys.o)
CMDS = init schinit fbinfo shmtest socktest inputtest
CMDS = init schinit fbinfo shmtest socktest inputtest reflink schattr
CMDBINS = $(addprefix $(BUILD)/cmd/, $(CMDS))
# -----------------------------------------------------------------------
@ -719,6 +719,7 @@ SAN := $(shell echo 'int main(void){return 0;}' > /tmp/.santest.c 2>/dev/null &&
SCHFS_SRC = kernel/fs/schfs/btree.c kernel/fs/schfs/btree.h \
kernel/fs/schfs/space.c kernel/fs/schfs/vol.c \
kernel/fs/schfs/obj.c kernel/fs/schfs/objw.c \
kernel/fs/schfs/attr.c \
kernel/fs/schfs/schfs_int.h include/schfs.h
# -iquote, not -I: include/ has its own errno.h, stat.h, fcntl.h and types.h,

59
cmd/reflink/main.c Normal file
View file

@ -0,0 +1,59 @@
// reflink -- copy a file by sharing its blocks instead of duplicating them.
//
// reflink <src> <dst>
//
// The copy is instant and free whatever the file's size: only metadata is
// written. Both files are ordinary and writable afterwards; the first write to
// either copies just the run it touches. `df` before and after is the check
// that it really shared rather than quietly copying.
//
// Uses FICLONE, which is Linux's ioctl for exactly this, so the same call works
// against any filesystem that supports cloning.
#include "types.h"
#include "user.h"
#include "fcntl.h"
#define FICLONE 0x40049409
static void
puts_(const char *s)
{
int n = 0;
while (s[n]) n++;
write(2, (void *)s, n);
}
int
main(int argc, char *argv[])
{
if (argc != 3) {
puts_("usage: reflink <src> <dst>\n");
exit();
}
int src = open(argv[1], O_RDONLY);
if (src < 0) {
puts_("reflink: cannot open "); puts_(argv[1]); puts_("\n");
exit();
}
// O_TRUNC matters: FICLONE replaces the destination's contents entirely,
// and creating it non-empty would leave blocks to release mid-clone.
int dst = open(argv[2], O_CREATE | O_WRONLY | O_TRUNC);
if (dst < 0) {
puts_("reflink: cannot create "); puts_(argv[2]); puts_("\n");
close(src);
exit();
}
if (ioctl(dst, FICLONE, (void *)(long)src) < 0) {
puts_("reflink: FICLONE failed\n");
close(src); close(dst);
exit();
}
close(src);
close(dst);
exit();
}

116
cmd/schattr/main.c Normal file
View file

@ -0,0 +1,116 @@
// schattr -- attach metadata to files, and ask the filesystem about it.
//
// schattr set <file> <name> <value> attach a value
// schattr get <file> <name> read it back
// schattr rm <file> <name> remove it
// schattr index <name> start indexing that attribute
// schattr find <name> <value> every object carrying that value
//
// `find` is the point of the rest. Without an index it is refused rather than
// answered by walking every file, because a query that silently degrades into a
// full scan is worse than one that says it cannot help: the first teaches you to
// use it on a filesystem where it will not scale.
//
// Indexing covers values written *after* the index is declared. There is no
// reindexing pass, so declare the index first.
#include "types.h"
#include "user.h"
static void
puts_(const char *s)
{
int n = 0;
while (s[n]) n++;
write(1, (void *)s, n);
}
static void
putn_(unsigned long long v)
{
char buf[24];
int i = 0;
if (v == 0) { write(1, "0", 1); return; }
while (v > 0 && i < (int)sizeof(buf)) { buf[i++] = '0' + (int)(v % 10); v /= 10; }
while (i--) write(1, &buf[i], 1);
}
static int
streq(const char *a, const char *b)
{
while (*a && *a == *b) { a++; b++; }
return *a == *b;
}
int
main(int argc, char *argv[])
{
if (argc < 2) {
puts_("usage: schattr set|get|rm <file> <name> [value]\n");
puts_(" schattr index <name>\n");
puts_(" schattr find <name> <value>\n");
exit();
}
if (streq(argv[1], "set") && argc == 5) {
int len = 0;
while (argv[4][len]) len++;
if (setxattr(argv[2], argv[3], argv[4], len, 0) < 0) {
puts_("schattr: set failed\n");
exit();
}
exit();
}
if (streq(argv[1], "get") && argc == 4) {
char val[256];
int n = getxattr(argv[2], argv[3], val, sizeof(val) - 1);
if (n < 0) {
puts_("schattr: no such attribute\n");
exit();
}
val[n] = '\0';
puts_(val);
puts_("\n");
exit();
}
if (streq(argv[1], "rm") && argc == 4) {
if (removexattr(argv[2], argv[3]) < 0) {
puts_("schattr: remove failed\n");
exit();
}
exit();
}
if (streq(argv[1], "index") && argc == 3) {
if (schfs_index(argv[2]) < 0) {
puts_("schattr: could not create the index\n");
exit();
}
puts_("schattr: indexing "); puts_(argv[2]);
puts_(" from now on\n");
exit();
}
if (streq(argv[1], "find") && argc == 4) {
unsigned long long ids[64];
int n = schfs_query(argv[2], argv[3], ids, 64);
if (n < 0) {
puts_("schattr: no index on "); puts_(argv[2]);
puts_(" -- declare one with `schattr index` first\n");
exit();
}
putn_((unsigned long long)n);
puts_(n == 1 ? " object:\n" : " objects:\n");
for (int i = 0; i < n; i++) {
puts_(" #");
putn_(ids[i]);
puts_("\n");
}
exit();
}
puts_("schattr: bad usage\n");
exit();
}

204
docs/architecture/schfs.md
View file

@ -1,9 +1,11 @@
# SchFS — design
**Status: phases 1 and 2 are done.** The copy-on-write B-tree, the volume layer,
the object layer, a working image writer and a checker all exist, and the kernel
mounts a SchFS volume read-only at boot. Phase 3 (writes and transaction groups)
is next. See §8 for exactly what is built and how to run its tests.
**Status: phases 1 through 3½ are done, and SchFS is the root filesystem.**
`kernel/proc/proc.c` mounts it before pid 1 has a working directory and panics
if it is not there. The copy-on-write B-tree, the volume layer, the object
layer, the write path, snapshots, an image writer and a checker all exist, and
several volumes can be mounted at once. Phase 4 (extent refcounts) is next.
See §8 for exactly what is built and how to run its tests.
Supersedes the on-disk format described in `include/fs.h`. Keeps the name SchFS;
the on-disk magic changes so v1 images are detected and rejected rather than
@ -211,8 +213,159 @@ in the shared VFS rather than in SchFS:
entry point libc happened to call.
- `rmdir` had no Linux syscall number, so every Linux-ABI program got ENOSYS.
**Phase 4 — extents with refcounts.** Enables reflink copies and snapshots
nearly for free.
**Phase 3¾ — durability, and the cost of it. DONE.** Three things that the
earlier phases had assumed rather than arranged.
*The commit barrier.* Copy-on-write promises that a crash lands on either the
old tree or the new one, but only if the blocks a root points at reach the
medium *before* the block naming that root. Nothing enforced that: `bwrite`
order is not platter order on any drive with a write cache. `struct schfs_bdev`
gained a `bflush` hook, `schfs_vol_commit` calls it either side of the uberblock
write, and the kernel implements it as "drain this device's dirty buffers, then
ATA FLUSH CACHE" (`ideflush`, and SCSI SYNCHRONIZE CACHE for a USB stick).
*Falling back through the ring.* Mount took the highest-generation uberblock
that validated, checked the tree it named, and gave up if that failed — with up
to 127 older, entirely intact roots sitting on the disk. It now walks the ring
newest-first and adopts the first root it can actually read, counting what it
skipped into `vol->recovered` so the rollback is reported rather than silent.
*Deferred writes, and brackets.* The B-tree seals and writes a node on every
record it inserts, so a leaf collecting one file's data checksums went to the
disk once per 4 KB of that file. The kernel's block hook marks buffers dirty
instead (`bdirty`), and the commit barrier is what makes that safe. On its own
that changes little, because a commit *contains* a flush and every `schfs_k*`
call was one commit — so `schfs_ktx_open`/`schfs_ktx_close` bracket a run of
calls into one group, and `schfs_vfs_write` wraps a whole `write(2)` in one.
Measured over 1 MB of file data, in block writes reaching the device:
| | before | after |
|---|---|---|
| committing every 4 KB | 1947 | 1410 |
| one commit for the megabyte | 1798 | **267** |
267 device writes for 256 blocks of data. Note that the two changes are worth
almost nothing separately — deferral cannot help while every commit flushes,
and batching cannot help while every write goes straight to the disk — which is
why the first attempt at this measured an 8% difference and concluded batching
was not the problem.
**Phase 3⅞ — where the time actually went. DONE.** The filesystem was correct
and durable and the system still felt slow: `ls /bin` took the better part of a
second. Two causes, neither of them the disk.
Measured first, through `/proc/fsstat` (counters added for exactly this). The
decisive number was **398,999 block reads against 445 actual disk misses** —
99.9% of all block reads were re-reading a block already in the buffer cache. So
the disk was never the problem, and every plan that started with "cache more" or
"read ahead" would have been aimed at 1% of the cost.
*The cursor re-read its own buffer.* `cur_load` called `node_read`
unconditionally, and `schfs_cursor_val` calls `cur_load` on **every access to
the current record** — deliberately, so that a callback disturbing the shared
block buffer cannot corrupt a directory walk. But `node_read` copies 4096 bytes
out of the buffer cache and then runs CRC32c over all 4096. Listing a 243-entry
directory paid both once per entry. The cursor now remembers which block its
buffer holds and skips the reload, cross-checked against the block header (so a
second cursor sharing the page cannot fool it) and against a global epoch that
every mutation bumps (so a block rewritten in place cannot be served stale).
*The checksum was bit-at-a-time.* `schfs_crc32c` did eight shifts per byte —
32,768 iterations to checksum one block — to stay usable before any
initialisation. It builds a 1 KB table on first use now, which keeps that
property and is eight times faster. This only became the top cost *after* the
cursor fix, which is the usual shape: removing the largest item promotes
whatever was beneath it.
| cycles, one session | before | after |
|---|---|---|
| `getdents` | 26.8 G | **0.14 G** (196×) |
| `read` | 26.4 G | **5.5 G** (4.8×) |
| `exec` | 18.7 G | **4.0 G** (4.7×) |
| block reads | 398,999 | **127,713** |
`ls /bin` went from ~400 ms to ~20 ms. Verified by reading an 802 KB binary
through the whole path and checking its md5 against the host's copy.
There was also a third of the problem that was not the kernel at all: `make
qemu` ran QEMU without `-enable-kvm`, so every instruction was emulated. The run
targets probe `/dev/kvm` now. Measure against the KVM number or you are
measuring TCG.
**Phase 4 — extents with refcounts. DONE.** `cp` that costs nothing.
A reference count per run of blocks, in its own tree (`SCHFS_KEY_EXTREF` under
`SCHFS_EXTREF_OBJECTID`) rather than in the file's own extent record — because a
count kept per file cannot be decremented by the *other* file sharing the run.
`schfs_file_reflink` copies one extent record per run and bumps the count; both
files stay ordinary and writable, and the first write to either copies only the
run it touches. Reachable from userland as `FICLONE`, Linux's ioctl, so
`cp --reflink` works unmodified; `cmd/reflink` is a two-line caller of it.
Measured on the running system: cloning a 411 KB file moved `df` by 24 KB.
Three things had to be right, and each was wrong first:
- **The count is "how many extent records point into this run", not "how many
files".** Splitting a shared extent around a written block leaves a head *and*
a tail, which is one more reference than before. Missing that leaked four
blocks per split.
- **A file's extent may point into the middle of a run.** Records are keyed by
the block a run was *allocated* at, so every lookup is a range query
(`extref_find`), not an exact one. An exact lookup finds nothing for a split
tail, "nothing" means unshared, and the tail is then freed while another file
still reaches it.
- **The space scan has to walk the reference records too.** A file truncated
inside a shared run describes fewer blocks than exist; the rest would be
derived as free while still referenced, and the allocator would hand out a
block that a later release frees a second time.
None of those is visible until every holder of a run is deleted, which is why
the test that matters is the least glamorous one: after deleting every file that
shared a run, free space must be back to exactly what it was.
**Phase 5 — attribute index and query. DONE.** The reason the design was one
B-tree with many key types from the start: the "database filesystem" feature
turned out to be `kernel/fs/schfs/attr.c` and nothing else.
An attribute is a named value on an object, stored under `(ino, ATTR,
hash-of-name)` and packed exactly as directory entries are — same collision
argument, same shape. An attribute may be *indexed*, and an indexed attribute
gets a second set of records keyed by value: `(hash-of-name, INDEX,
hash-of-value)` holding the object ids that carry it. So "every object whose
`tag` is `holiday`" is one tree descent instead of a walk of every object.
There is no query language, no parser and no planner, and §6 above explains why:
the useful half of "query the filesystem" is the index, and WinFS spent three
years and a large team on the other interpretation.
```
schattr index tag # declare the index
schattr set /tmp/a tag holiday
schattr find tag holiday # one descent, whatever the volume holds
```
Four decisions worth keeping:
- **Which attributes are indexed is cached in the volume**, read once at mount.
Every `setxattr` has to ask, and the answer is almost always no; a tree
descent to say so would make every attribute write pay for a feature it is
not using.
- **The value is stored in the index entry, not just its hash.** Two values can
hash alike, and a query that returned the wrong file would be far worse than a
slow one.
- **An index covers values written after it is declared.** There is no
reindexing pass. Saying so is better than a half-truth that makes a query
answer confidently and wrongly.
- **A query on an unindexed attribute is refused, not answered by scanning.** A
query that silently degrades into a full walk teaches you to use it where it
will not scale.
Reachable as `setxattr`/`getxattr`/`removexattr` at Linux's syscall numbers, so
anything that already knows how to attach metadata needs no changes, plus two
native calls (`schfs_index`, `schfs_query`) that have no Linux equivalent
because asking a filesystem a question does not.
**Phase 5 — attribute index and query.** See below.
@ -279,6 +432,8 @@ Directory *depth* is unbounded on disk; path resolution is what limits it.
| | limit | where it comes from |
|---|---|---|
| Volume, mounted | **64 GiB** | the in-RAM allocation bitmap: `SCHFS_SPACE_PAGES` (512) pages × 32768 blocks |
| Volumes mounted at once | **4** | `SCHFS_VOL_SLOTS`; slot 0 is the root |
| Snapshots mounted at once | **4** | `SCHFS_SNAP_SLOTS` |
| Volume, via the buffer cache | **4 TiB** | `bio.c` addresses blocks with a `uint`, and one SchFS block spans four of them |
| RAM per mount | **8 KiB per GiB** of volume | two bitmaps, one bit per block each |
| Inline value in a leaf | **1016 bytes** | `SCHFS_MAX_INLINE` = a quarter of a node |
@ -298,7 +453,8 @@ These are VFS and driver limits rather than SchFS ones:
- `MAXPATH` caps a path at **128 bytes**.
- `NMOUNT` allows **8** mounts.
- The IDE driver is 28-bit LBA: **128 GiB** per disk.
- The IDE driver handles the primary channel only: **2** ATA devices.
- The IDE driver uses LBA48 past 128 GiB, so the disk is not the limit there.
- A single `read`/`write` call is capped at **4 GiB** (`n` is a `uint`); the
file itself is not.
@ -318,10 +474,11 @@ kernel/fs/schfs/space.c allocation bitmap, derived from the tree at mount
kernel/fs/schfs/vol.c uberblock ring: open, format, commit
kernel/fs/schfs/obj.c read side: inodes, dirents, extents, path lookup
kernel/fs/schfs/objw.c write side: create, link, unlink, rename, write
kernel/fs/schfs/kmount.c kernel: block device, mount, transaction bracket
kernel/fs/schfs/kmount.c kernel: block device, mount slots, transaction bracket
kernel/fs/schfs/vfs.c the fs_ops table: SchFS behind open/read/write
tools/mkschfs.c build an image from a directory tree
tools/schfs_fsck.c verify, list, extract
include/fsstat.h counters behind /proc/fsstat: where the time goes
tools/schfs_test.c B-tree torture harness
tools/schfs_objtest.c object layer: files, directories, space accounting
tools/posix/schfs_roundtrip.sh end-to-end filesystem test
@ -357,6 +514,17 @@ is the least glamorous: after deleting everything, free space must be back to
exactly what it was on an empty volume. Every other test in the suite still
passes on a filesystem that leaks a block on every delete.
Its section 10 is crash recovery, and it exists because the whole consistency
argument for this format — "mount takes the newest uberblock that validates, so
a crash lands on either the old tree or the new one" — had no end-to-end test at
all. `schfs_test` injected crashes into the B-tree; nothing did it to a volume.
The crash is a `longjmp` out of a block write, which is the only faithful model:
returning an error tests error handling instead, and letting execution continue
with the write swallowed has the filesystem reading back blocks it believes it
just wrote, which no real failure does. It runs the same transaction 119 times,
cutting the power one write later each time, and after every one the volume must
mount, check out structurally, and still hold the data committed before it.
`tools/posix/mutate.sh` injects six deliberate bugs and fails if any of them goes
undetected. It is how the suites are kept honest; see the note in §9.
@ -401,3 +569,23 @@ Recorded because each one cost real time and none is obvious from the code.
- **Reproducibility dies from atime.** mkschfs reads every source file, which
updates its atime, so copying atime into the image made two builds of the same
tree differ. All three timestamps come from mtime.
- **An optional hook must still be initialised.** `struct schfs_bdev` gained
`bflush`, and every host tool built one on the stack field by field. The tools
that did not set the new field handed `schfs_vol_commit` a garbage pointer to
test and call, which segfaulted at the first commit. Each of those helpers
memsets the struct first now — the fix is not "remember to set it".
- **Deferring writes makes an all-dirty buffer cache normal.** `bget` panicked
with "no buffers" when it could find nothing clean to recycle, which was
correct while a dirty buffer meant "the log has not committed". It now writes
the least recently used victim out and takes it; a transaction touching more
than `NBUF` blocks reaches that state long before it commits.
- **The two performance fixes are worthless apart.** Deferring tree writes buys
nothing while every commit flushes, and batching commits buys nothing while
every write goes straight to the disk. Measuring either alone says "this is
not where the time goes", and both measurements are correct.
- **`fsid_t` is eight bytes.** `struct statfs` declared `f_fsid` as `int64[2]`,
which is one slot too long, so everything after it landed in the wrong place:
userland read `f_frsize` out of `f_namelen` and got 255. df then computed in
units of 255 bytes and called a 32 MB root filesystem 1992 KB. It survived
because every field in that struct used to be an invented constant, so no
number in it had ever been checked against reality.

7
docs/roadmap.md
View file

@ -360,6 +360,13 @@ palette and a layout, none of which care how the pixels were found. What does
not survive is every assumption in `stage2main.c` about where things live in low
memory (see `docs/layout.md`).
### SchFS: all five phases are done (2026-08-15)
Phases 1-3 were finished earlier. Phase 3¾ (the commit barrier, uberblock
fallback, deferred writes), phase 4 (extent refcounts and reflink) and phase 5
(attributes, the index and the query) all landed on 2026-08-15. See
docs/architecture/schfs.md, which is the detailed record.
### SchFS beyond SchOS
Three ideas parked 2026-08-08, in the order they should happen.

151
include/defs.h
View file

@ -23,6 +23,13 @@ void binit(void);
struct buf* bread(uint, uint64);
void brelse(struct buf*);
void bwrite(struct buf*);
// Modify without writing. The buffer stays dirty in the cache until an
// eviction or a bflush() takes it out; the caller is promising to ask for the
// ordering it needs explicitly rather than relying on write order.
void bdirty(struct buf*);
// Write out every dirty buffer for `dev`, then make the device commit them.
// Returns 0, or -1 if the device could not confirm durability.
int bflush(uint dev);
// console.c
struct termios;
@ -69,7 +76,12 @@ void mountinit(void);
// 1 if the filesystem on this device refuses writes -- a mounted snapshot.
// Ask before mutating, not after: the create path panics on a failed dirlink.
int fs_readonly(uint dev);
int mount_add(uint dev, uint64 inum, uint fsdev, uint64 root);
int mount_add(uint dev, uint64 inum, uint fsdev, uint64 root,
const char *path, const char *source, const char *type);
// Walk the mount table for /proc/mounts. `path` and `source` need MAXPATH
// bytes, `type` 16. Returns 0 for a live entry, -1 for an empty slot.
int mount_get(int i, char *path, char *source, char *type,
uint *fsdev);
int mount_del_by_root(uint fsdev, uint64 root);
struct inode* ialloc(uint, short);
struct inode* iget(uint, uint64);
@ -130,6 +142,8 @@ void iderw(struct buf*);
// For probes, not for reads: iderw() panics on an absent drive, so anything
// that is guessing whether a disk exists has to ask here first.
int ide_disk_present(int dev);
// ATA FLUSH CACHE: block until the drive has committed its write cache.
int ideflush(int dev);
// fs/schfs/kmount.c -- SchFS.
//
@ -138,52 +152,18 @@ int ide_disk_present(int dev);
// Every schfs_k* that changes anything is one transaction and is durable by the
// time it returns.
struct schfs_inode;
void schfs_kmount(uint dev);
// Mounting. Slot 0 is the root filesystem, mounted by proc.c before pid 1 has a
// working directory; schfs_kmount_dev takes any other SchFS disk and returns
// the filesystem device number it landed on, or a negative errno.
void schfs_kmount(uint bdev);
int schfs_kmount_dev(uint bdev);
int schfs_kumount(uint dev);
int schfs_kmounted(void);
uint64 schfs_klookup(const char *path);
int schfs_kstat(uint64 ino, struct schfs_inode *out);
long schfs_kread(uint64 ino, uint64 off, void *buf, unsigned n);
int schfs_kreaddir(uint64 dir,
int (*fn)(uint64, unsigned short, const char *,
unsigned, void *),
void *arg);
int schfs_kinfo(char *buf, int cap);
int schfs_kcreate(uint64 dir, const char *name, unsigned namelen,
unsigned short type, uint mode,
unsigned short major, unsigned short minor, uint64 *out);
long schfs_kwrite(uint64 ino, uint64 off, const void *buf,
unsigned n);
int schfs_ktruncate(uint64 ino, uint64 size);
int schfs_kunlink(uint64 dir, const char *name, unsigned namelen);
int schfs_klink(uint64 dir, const char *name, unsigned namelen,
uint64 ino);
int schfs_krename(uint64 olddir, const char *oldname,
unsigned oldlen, uint64 newdir, const char *newname,
unsigned newlen);
int schfs_ksetattr(uint64 ino, const struct schfs_inode *in);
int schfs_ksync(void);
// Pieces the VFS layer (fs/schfs/vfs.c) needs, where a directory entry and the
// object it names are separate operations. The VFS keeps the link count.
uint64 schfs_klookup_at(uint64 dir, const char *name, unsigned len);
int schfs_kdirempty(uint64 dir);
int schfs_kalloc_object(unsigned short type, uint mode,
uint64 *out);
int schfs_kdiradd(uint64 dir, const char *name, unsigned len,
uint64 ino, unsigned short type);
int schfs_kdirdel(uint64 dir, const char *name, unsigned len);
int schfs_kfree(uint64 ino);
uint64 schfs_kroot(void);
// A snapshot mounted read-only: the same block device with an older tree root.
// The read path takes a device number so the VFS can address either view; there
// is no write path, because a snapshot that could be written to would not be
// one. See kernel/fs/schfs/kmount.c.
// Returns the device number of the slot the snapshot landed in, or negative.
int schfs_ksnap_mount(const char *name, unsigned namelen);
void schfs_ksnap_unmount(uint dev);
uint64 schfs_ksnap_root(uint dev);
// Read side. `dev` is a filesystem device number -- a live volume or a mounted
// snapshot -- not the block device underneath it.
uint64 schfs_klookup(uint dev, const char *path);
long schfs_kread_dev(uint dev, uint64 ino, uint64 off, void *buf,
unsigned n);
int schfs_kstat_dev(uint dev, uint64 ino, struct schfs_inode *out);
@ -193,6 +173,85 @@ int schfs_kreaddir_dev(uint dev, uint64 dir,
int (*fn)(uint64, unsigned short, const char *,
unsigned, void *),
void *arg);
// Resume a directory walk at a tree key. The callback gets each entry's hash,
// which is what getdents hands back to userland as its position.
int schfs_kreaddir_from_dev(uint dev, uint64 dir, uint64 start_hash,
int (*fn)(uint64, unsigned short, const char *,
unsigned, uint64, void *),
void *arg);
uint64 schfs_kroot(uint dev);
int schfs_kinfo(char *buf, int cap);
// Write side. Each is one transaction and is durable by the time it returns,
// unless the caller has a bracket open -- see schfs_ktx_open.
int schfs_kcreate(uint dev, uint64 dir, const char *name,
unsigned namelen, unsigned short type, uint mode,
unsigned short major, unsigned short minor, uint64 *out);
long schfs_kwrite(uint dev, uint64 ino, uint64 off, const void *buf,
unsigned n);
int schfs_ktruncate(uint dev, uint64 ino, uint64 size);
int schfs_kunlink(uint dev, uint64 dir, const char *name,
unsigned namelen);
int schfs_klink(uint dev, uint64 dir, const char *name,
unsigned namelen, uint64 ino);
int schfs_krename(uint dev, uint64 olddir, const char *oldname,
unsigned oldlen, uint64 newdir, const char *newname,
unsigned newlen);
int schfs_ksetattr(uint dev, uint64 ino,
const struct schfs_inode *in);
int schfs_ksync(uint dev);
int schfs_ksync_all(void);
// Share src's data blocks with dst without copying. Both stay writable; the
// first write to either copies only the run it touches. Behind FICLONE.
int schfs_kreflink(uint dev, uint64 src, uint64 dst);
// Attributes, and the index over them. See kernel/fs/schfs/attr.c.
int schfs_kattr_set(uint dev, uint64 ino, const char *name,
unsigned namelen, const void *val, unsigned vallen,
unsigned short type);
int schfs_kattr_get(uint dev, uint64 ino, const char *name,
unsigned namelen, void *out, unsigned cap,
unsigned *vallen);
int schfs_kattr_del(uint dev, uint64 ino, const char *name,
unsigned namelen);
int schfs_kindex(uint dev, const char *name, unsigned namelen);
int schfs_kquery(uint dev, const char *name, unsigned namelen,
const void *val, unsigned vallen, uint64 *out, int cap);
// Bracket a run of changes into one transaction group. Every schfs_k* call made
// between open and close joins it and commits once, at the close; the bracket
// nests, and holds the volume lock for its whole extent. Costly to leave open —
// nothing else touches that volume meanwhile — so bracket a bounded piece of
// work such as one write(2), not a whole syscall's worth of unrelated steps.
// schfs_ktx_close returns the commit's status, or -1 if the caller had no
// bracket open.
void schfs_ktx_open(uint dev);
int schfs_ktx_close(uint dev);
// Volume counts for statfs. 0 if `dev` is a mounted SchFS volume (live or
// snapshot) and the counts are filled in, -1 if it is not.
int schfs_kstatfs(uint dev, uint64 *total, uint64 *freeblocks,
uint64 *objects);
// Pieces the VFS layer (fs/schfs/vfs.c) needs, where a directory entry and the
// object it names are separate operations. The VFS keeps the link count.
int schfs_kdirempty(uint dev, uint64 dir);
int schfs_kalloc_object(uint dev, unsigned short type, uint mode,
uint64 *out);
int schfs_kdiradd(uint dev, uint64 dir, const char *name,
unsigned len, uint64 ino, unsigned short type);
int schfs_kdirdel(uint dev, uint64 dir, const char *name,
unsigned len);
int schfs_kfree(uint dev, uint64 ino);
// A snapshot mounted read-only: the same block device with an older tree root.
// The read path takes a device number so the VFS can address either view; there
// is no write path, because a snapshot that could be written to would not be
// one. See kernel/fs/schfs/kmount.c.
// Returns the device number of the slot the snapshot landed in, or negative.
int schfs_ksnap_mount(uint dev, const char *name, unsigned namelen);
void schfs_ksnap_unmount(uint dev);
uint64 schfs_ksnap_root(uint dev);
// ramdisk.c
uint uptime(void);
@ -236,6 +295,8 @@ int ramdisk_present(void);
// usb_storage.c
void usb_storage_rw(struct buf*);
int usb_storage_present(void);
// SCSI SYNCHRONIZE CACHE: the same barrier as ideflush, over bulk transport.
int usb_storage_flush(void);
void usb_storage_dump(void); // `echo s > /proc/sysrq`
// ioapic.c

2
include/errno.h
View file

@ -18,6 +18,7 @@
#define ENOMEM 12 // Out of memory
#define EACCES 13 // Permission denied
#define EFAULT 14 // Bad address
#define ENOTBLK 15 // Block device required
#define EBUSY 16 // Device or resource busy
#define EEXIST 17 // File exists
#define EXDEV 18 // Cross-device link
@ -54,6 +55,7 @@
#define EADDRNOTAVAIL 99 // Cannot assign requested address
#define ECONNRESET 104 // Connection reset by peer
#define EISCONN 106 // Transport endpoint is already connected
#define ENODATA 61 // No data available -- no such attribute
#define ENOTCONN 107 // Transport endpoint is not connected
#define ECONNREFUSED 111 // Connection refused

45
include/file.h
View file

@ -63,6 +63,16 @@ struct fs_ops {
// position that starts at 0 and is echoed back; returns bytes written, 0 at
// the end of the directory, or negative on error.
int (*getdents)(struct inode *dp, char *out, int cap, uint64 *off);
// The largest write this filesystem wants in one call, or 0 for "no limit".
//
// v1 needs one: a write has to fit in the on-disk log, so filewrite splits
// anything larger and each piece is its own transaction. That limit is a
// property of the journal, not of writing, and applying it to SchFS was
// actively harmful -- SchFS makes one transaction out of whatever it is
// handed, so chopping a 1 MB write into 6 KB pieces bought 170 commits and
// the flushes that go with them for nothing.
uint max_write;
};
// In-memory inode.
@ -82,7 +92,12 @@ struct inode {
short type;
short major;
short minor;
short nlink;
// int, not short. SchFS stores a uint32 link count and this truncated it, so
// a file that passed 32767 names came back with a negative one -- and iput
// frees an object whose count reaches zero. Signed rather than unsigned on
// purpose: `nlink--` on a count that is somehow already zero should land
// somewhere a check can see it, not wrap to four billion.
int nlink;
uint64 size;
uint rwmode;
@ -95,13 +110,23 @@ struct inode {
};
#define I_VALID 0x2
// Device number that means "the mounted SchFS volume". Real disks are 0 and 1,
// and this is deliberately far from them so a stray value never looks like a
// SchFS inode.
// Device numbers for filesystems that are not a raw disk. Real disks are 0..3,
// and these are deliberately far from them so a stray value never looks like a
// SchFS inode. Nothing on any disk records them -- they are runtime identities
// for the inode cache, so the numbering can be rearranged freely.
//
// Mounted SchFS volumes: SCHFS_DEV through SCHFS_DEV + SCHFS_VOL_SLOTS - 1.
//
// This was a single number, because there was a single volume: kmount.c held
// one `kvol` and every write entry point named it directly, so `mount -t schfs
// /dev/hdb /mnt` ignored its source argument entirely and handed back whichever
// disk the kernel had found at boot. Slot 0 is that boot volume, which is why
// SCHFS_DEV alone still means "the root" everywhere that only ever wanted it.
#define SCHFS_VOL_SLOTS 4
#define SCHFS_DEV 64
#define SCHFS_IS_VOL_DEV(d) \
((d) >= SCHFS_DEV && (d) < SCHFS_DEV + SCHFS_VOL_SLOTS)
// Device number that means "the synthetic /proc". Like SCHFS_DEV, it is
// deliberately nowhere near a real disk's 0 or 1.
// Mounted snapshots. Same block device, an older tree root, read-only.
//
// Each needs its own device number because an inode is identified by
@ -114,11 +139,15 @@ struct inode {
// the cost is two scratch pages per slot, allocated on first use. Four rather
// than many because each is a device number and they are a fixed, small space.
#define SCHFS_SNAP_SLOTS 4
#define SCHFS_SNAP_DEV 66
#define SCHFS_SNAP_DEV 96
#define SCHFS_IS_SNAP_DEV(d) \
((d) >= SCHFS_SNAP_DEV && (d) < SCHFS_SNAP_DEV + SCHFS_SNAP_SLOTS)
#define PROCFS_DEV 65
// Anything SchFS answers for, live volume or snapshot.
#define SCHFS_IS_ANY_DEV(d) (SCHFS_IS_VOL_DEV(d) || SCHFS_IS_SNAP_DEV(d))
// The synthetic /proc.
#define PROCFS_DEV 80
// table mapping major device number to
// device functions

46
include/fsstat.h Normal file
View file

@ -0,0 +1,46 @@
// SPDX-License-Identifier: Apache-2.0
//
// Counters for the filesystem and block paths, read through /proc/fsstat.
//
// These exist because "the system feels slow" is not a bug report anyone can
// act on, and every guess about where the time goes in a filesystem is wrong
// until it is measured. `ls /bin` taking three seconds could be the directory
// walk, the tree descents, the copy path, or the disk -- and those four call
// for completely different fixes.
//
// Deliberately plain `uint64` with no locking. A counter that is off by one
// because two CPUs raced tells you the same thing as one that is exact, and a
// lock here would change the very timings it is meant to report.
#pragma once
struct fsstat {
// Buffer cache. `bread_miss` is the one that matters: a miss is a real
// transfer, a hit is a memcpy.
uint64 bread_calls;
uint64 bread_miss;
uint64 bget_evict_dirty; // had to write a buffer out to make room
// Actual device transfers, and the cycles spent inside them.
uint64 disk_reads;
uint64 disk_writes;
uint64 disk_cycles;
// The VFS read path: one call is one read(2) reaching a SchFS file.
uint64 read_calls;
uint64 read_bytes;
uint64 read_cycles;
// Directory listing.
uint64 getdents_calls;
uint64 getdents_cycles;
uint64 getdents_walked; // entries stepped over, including ones re-walked
// B-tree descents from the root, and the block reads they cost.
uint64 descents;
// exec(2), which on this system means reading an 800 KB multicall binary.
uint64 exec_calls;
uint64 exec_cycles;
};
extern struct fsstat fsstat;

16
include/posix.h
View file

@ -120,6 +120,9 @@ struct rlimit {
#define TIOCSCTTY 0x540E
#define TIOCGSERIAL 0x541E
#define FIONREAD 0x541B
// FICLONE: share another file's data blocks with this one. The argument is a
// file descriptor, not a pointer. Linux's value, so `cp --reflink` works.
#define FICLONE 0x40049409
#define TIOCEXCL 0x540C
#define TIOCNXCL 0x540D
@ -203,6 +206,17 @@ struct sysinfo {
char _f[8];
};
// Must match Linux's x86-64 struct statfs field for field, because the programs
// reading it were compiled against Linux's.
//
// f_fsid is the trap. Linux's fsid_t is `struct { int val[2]; }` -- eight bytes
// in total, not two eight-byte words -- and declaring it as int64[2] made this
// struct one slot too long from f_fsid onwards. Everything after it landed in
// the wrong place: userland read f_frsize out of f_namelen and got 255, so df
// computed its block counts in units of 255 bytes and reported a 32 MB root
// filesystem as 1992 KB. It went unnoticed for as long as it did because every
// field here used to be an invented constant, so the numbers were meaningless
// whichever slot they came from.
struct statfs {
int64 f_type;
int64 f_bsize;
@ -211,7 +225,7 @@ struct statfs {
int64 f_bavail;
int64 f_files;
int64 f_ffree;
int64 f_fsid[2];
int32 f_fsid[2];
int64 f_namelen;
int64 f_frsize;
int64 f_flags;

94
include/schfs.h
View file

@ -40,6 +40,7 @@
#define SCHFS_KEY_INDEX 5 // (idxid, INDEX, valuehash) -> objectid
#define SCHFS_KEY_CSUM 6 // (CSUM, CSUM, diskblk) -> uint32 CRC32c
#define SCHFS_KEY_SNAP 7 // (SNAP, SNAP, id) -> struct schfs_snap
#define SCHFS_KEY_EXTREF 8 // (EXTREF,EXTREF, diskblk) -> struct schfs_extref
// Data-block checksums live under one reserved object id, exactly as btrfs
// keeps a checksum tree. Tree nodes carry their checksum in their parent
@ -52,6 +53,16 @@
// B-tree is copy-on-write, so the blocks a root reaches are already immutable
// as long as nothing frees them. Keeping one costs a record.
#define SCHFS_SNAP_OBJECTID 2ULL
// Shared extents, keyed by the disk block a run starts at. btrfs calls this the
// extent tree and so, in effect, is this -- one reserved object id whose records
// say how many files point at a run.
//
// Only *shared* runs have a record. A run nothing has cloned has no entry at
// all, and absence means a reference count of one. That is not a space
// optimisation so much as a correctness one: the overwhelmingly common case
// stays exactly as it was, so the write path pays nothing for a feature it is
// not using, and there is no second structure to keep in step on every write.
#define SCHFS_EXTREF_OBJECTID 3ULL
// Object ids below this are reserved; the root directory is the first real one.
#define SCHFS_FIRST_OBJECTID 16ULL
@ -225,10 +236,25 @@ struct schfs_extent {
// mistake this field exists to prevent. Only ever trusted together with
// refs == 1; a shared run belongs to somebody else as well.
uint64_t gen;
// Whether this run is shared, as far as *this* file knows. A hint, not the
// truth: the count that decides whether blocks may be released lives in the
// extent-reference tree (SCHFS_KEY_EXTREF), because a per-file copy cannot
// be decremented by the other file that shares the run.
//
// Kept because it makes the common case free: refs == 1 here means nothing
// has ever cloned this run, so the write path can rewrite it in place
// without consulting the tree at all.
uint32_t refs;
uint32_t pad;
};
// How many files point at a run of blocks. Present only for runs with more than
// one reference; see SCHFS_EXTREF_OBJECTID.
struct schfs_extref {
uint64_t nblocks;
uint64_t refs;
};
// Directory entry payload. The key's offset field holds the name hash, so a
// lookup is one tree descent rather than a directory scan.
//
@ -248,6 +274,71 @@ struct schfs_dirent_rec {
#define SCHFS_NAME_MAX 255
// ---------------------------------------------------------------------------
// Attributes, and the index over them
//
// An attribute is a named value on an object -- the same thing Linux calls an
// extended attribute, and the thing BeFS built its query interface on. Stored
// under (ino, ATTR, namehash), packed exactly as directory entries are, and for
// the same reason: two names can hash alike, and a filesystem that loses one of
// them under a collision is not one to build on.
//
// The point of having them is the *index*. An attribute may be indexed, and an
// indexed attribute gets a second set of records keyed by its value rather than
// by the object -- so "every object whose `type` is `photo`" is a range scan of
// a few keys rather than a walk of every object in the filesystem. That is the
// whole of what "a filesystem you can query" means here, and the reason the
// design is one B-tree with many key types: the feature is a few hundred lines
// on machinery that already exists, rather than a second engine.
struct schfs_attr_rec {
uint32_t namelen;
uint32_t valuelen;
uint16_t type; // SCHFS_ATTR_* below; what a query compares as
uint16_t flags;
uint32_t pad;
/* name bytes, then value bytes; the pair padded to an 8-byte boundary */
};
// What a value *is*, so that a query can compare two of them meaningfully. A
// string and an integer that happen to share bytes are not the same value, and
// ordering differs between them.
#define SCHFS_ATTR_RAW 0 // opaque bytes; only equality is meaningful
#define SCHFS_ATTR_STRING 1
#define SCHFS_ATTR_INT 2 // int64, little-endian
#define SCHFS_ATTR_TIME 3 // seconds since the epoch, int64
#define SCHFS_ATTR_NAME_MAX 63
#define SCHFS_ATTR_VALUE_MAX 255
// Step from one packed attribute record to the next. 8-byte aligned so the
// uint32s of the next one are naturally aligned.
#define SCHFS_ATTR_LEN(namelen, valuelen) \
((unsigned)((sizeof(struct schfs_attr_rec) + (namelen) + (valuelen) + 7) & ~7u))
// An index entry: one object that carries a given value for a given attribute.
//
// Keyed (indexid, INDEX, valuehash), where indexid is the hash of the attribute
// *name*. Several objects can share a value, so the payload is a packed list of
// these, exactly as dirents and attributes are.
struct schfs_index_rec {
uint64_t objectid;
uint32_t valuelen;
uint32_t pad;
/* value bytes follow, so a query can reject a hash collision */
};
#define SCHFS_INDEX_LEN(valuelen) \
((unsigned)((sizeof(struct schfs_index_rec) + (valuelen) + 7) & ~7u))
// Which attributes are indexed. One record, under the index object id, whose
// value is a packed list of attribute names.
//
// A list rather than a flag on each attribute, because the question asked at
// every setxattr is "is *this name* indexed", and answering it from one small
// record is a single cached lookup rather than a search.
#define SCHFS_INDEX_OBJECTID 4ULL
#define SCHFS_INDEX_LIST_OFF 0ULL /* the one record's offset in the key */
// Step from one packed dirent to the next. Records are 8-byte aligned so the
// uint64 objectid of the next one is naturally aligned.
#define SCHFS_DIRENT_LEN(namelen) \
@ -274,5 +365,8 @@ SCHFS_ASSERT_SIZE(schfs_uberblock, SCHFS_BLKSIZE);
SCHFS_ASSERT_SIZE(schfs_inode, 80);
SCHFS_ASSERT_SIZE(schfs_extent, 32);
SCHFS_ASSERT_SIZE(schfs_dirent_rec, 16);
SCHFS_ASSERT_SIZE(schfs_extref, 16);
SCHFS_ASSERT_SIZE(schfs_attr_rec, 16);
SCHFS_ASSERT_SIZE(schfs_index_rec, 16);
#endif /* SCHFS_H */

14
include/user.h
View file

@ -64,6 +64,20 @@ int geteuid(void);
int nanosleep(void*);
int mprotect(void*, uint64, int);
int ioctl(int, uint64, void*);
// Attributes on a file, and the index over them. Only SchFS carries these;
// anything else answers EOPNOTSUPP.
//
// schfs_query returns how many object ids it wrote, or -1 if the attribute has
// no index -- it will not fall back to walking the filesystem, because a query
// that silently becomes a full scan is worse than one that says it cannot help.
int setxattr(const char *path, const char *name, const void *val, int size,
int flags);
int getxattr(const char *path, const char *name, void *val, int size);
int removexattr(const char *path, const char *name);
int schfs_index(const char *name);
int schfs_query(const char *name, const char *value,
unsigned long long *out, int cap);
char* getcwd(char*, int);
int arch_prctl(int, uint64);
int set_tid_address(int*);

12
include/x86.h
View file

@ -13,6 +13,18 @@ inb(ushort port)
return data;
}
// Cycle counter. Not a clock -- it does not convert to time without knowing the
// frequency, and it is not synchronised across cores -- but for "which of these
// two paths costs more" it is exactly the right instrument, and the only one
// with fine enough resolution to see inside a single syscall.
static inline uint64
rdtsc(void)
{
uint lo, hi;
asm volatile("rdtsc" : "=a" (lo), "=d" (hi));
return ((uint64)hi << 32) | lo;
}
static inline void
insl(int port, void *addr, int cnt)
{

127
kernel/dev/bio.c
View file

@ -25,6 +25,10 @@
#include "sleeplock.h"
#include "fs.h"
#include "buf.h"
#include "x86.h"
#include "fsstat.h"
struct fsstat fsstat;
struct {
struct spinlock lock;
@ -59,8 +63,20 @@ binit(void)
// is device-agnostic; everything below picks a backend. RAMDISK_DEV is served
// out of memory (kernel/dev/ramdisk.c), USBDISK_DEV by bulk transfers to a
// USB stick, everything else by the IDE driver.
static void bdevrw_inner(struct buf *b);
static void
bdevrw(struct buf *b)
{
uint64 t0 = rdtsc();
if(b->flags & B_DIRTY) fsstat.disk_writes++; else fsstat.disk_reads++;
bdevrw_inner(b);
fsstat.disk_cycles += rdtsc() - t0;
}
static void
bdevrw_inner(struct buf *b)
{
if(b->dev == RAMDISK_DEV){
ramdiskrw(b);
@ -73,6 +89,19 @@ bdevrw(struct buf *b)
iderw(b);
}
// Tell the device to commit whatever it has accepted but not yet made durable.
// Same dispatch as bdevrw, and the same rule: everything above this line is
// device-agnostic.
static int
bdevflush(uint dev)
{
if(dev == RAMDISK_DEV)
return 0; // memory has no volatile stage below it
if(dev == USBDISK_DEV)
return usb_storage_flush();
return ideflush(dev);
}
// Look through buffer cache for block on device dev.
// If not found, allocate a buffer.
// In either case, return locked buffer.
@ -81,6 +110,7 @@ bget(uint dev, uint64 blockno)
{
struct buf *b;
retry:
acquire(&bcache.lock);
// Is the block already cached?
@ -107,6 +137,47 @@ bget(uint dev, uint64 blockno)
return b;
}
}
// Every buffer is dirty and none is clean enough to take.
//
// This used to be a straight panic, which was right while every writer went
// through bwrite and a dirty buffer therefore meant "the log has not
// committed yet". SchFS marks tree blocks dirty and leaves them there (see
// bdirty), so an all-dirty cache is now an ordinary state: a transaction that
// touches more than NBUF blocks reaches it long before it commits.
//
// Writing one out early is always safe. The only ordering the format needs is
// that every block of the new tree precede the uberblock that names it, and
// the uberblock goes down after an explicit bflush -- so nothing here can
// land on the wrong side of that barrier.
for(b = bcache.head.prev; b != &bcache.head; b = b->prev){
if(b->refcnt == 0){
fsstat.bget_evict_dirty++;
b->refcnt++; // claim it, so nobody else recycles it either
release(&bcache.lock);
acquiresleep(&b->lock);
if(b->flags & B_DIRTY)
bdevrw(b);
// Between dropping bcache.lock and taking b's, another process may have
// found this buffer under its *old* identity and be waiting on it. It is
// clean now either way, so hand it back rather than renaming a block out
// from under them, and look for another victim.
acquire(&bcache.lock);
if(b->refcnt != 1){
b->refcnt--;
release(&bcache.lock);
releasesleep(&b->lock);
goto retry;
}
b->dev = dev;
b->blockno = blockno;
b->flags = 0;
release(&bcache.lock);
return b; // b->lock still held, refcnt 1
}
}
panic("bget: no buffers");
}
@ -116,8 +187,10 @@ bread(uint dev, uint64 blockno)
{
struct buf *b;
fsstat.bread_calls++;
b = bget(dev, blockno);
if(!(b->flags & B_VALID)) {
fsstat.bread_miss++;
bdevrw(b);
}
return b;
@ -133,6 +206,60 @@ bwrite(struct buf *b)
bdevrw(b);
}
// Mark b modified but do not write it. The buffer stays dirty in the cache
// until something evicts it or bflush() comes past.
//
// This is what makes a copy-on-write filesystem affordable. SchFS's B-tree
// seals and writes a node on every single record insert, so a hot leaf -- the
// one collecting a file's data checksums, say -- was written to the disk once
// per 4 KB of file. Measured over 1 MB: 1288 block writes for 256 blocks of
// data. Deferring collapses every one of those repeats into the last one.
//
// It is only safe because the filesystem asks for the ordering it needs
// explicitly, at commit, instead of assuming that write order is medium order.
// A filesystem that did assume that would be broken here -- and was already
// broken on any drive with a write cache, which is all of them.
void
bdirty(struct buf *b)
{
if(!holdingsleep(&b->lock))
panic("bdirty");
b->flags |= B_DIRTY;
b->flags |= B_VALID;
}
// Push every dirty buffer for `dev` out, then make the device itself commit
// what it has taken. Returns 0, or -1 if the device could not confirm it.
//
// Walks the array rather than the MRU list: brelse relinks buffers as it goes,
// and the array is the one order that cannot change underneath the walk.
int
bflush(uint dev)
{
struct buf *b;
for(b = bcache.buf; b < bcache.buf+NBUF; b++){
acquire(&bcache.lock);
if(b->dev != dev || !(b->flags & B_DIRTY)){
release(&bcache.lock);
continue;
}
b->refcnt++;
release(&bcache.lock);
acquiresleep(&b->lock);
if(b->flags & B_DIRTY) // recheck: an eviction may have beaten us to it
bdevrw(b);
releasesleep(&b->lock);
acquire(&bcache.lock);
b->refcnt--;
release(&bcache.lock);
}
return bdevflush(dev);
}
// Release a locked buffer.
// Move to the head of the MRU list.
void

64
kernel/dev/ide.c
View file

@ -30,6 +30,11 @@
#define IDE_CMD_WRITE_EXT 0x34
#define IDE_CMD_RDMUL_EXT 0x29
#define IDE_CMD_WRMUL_EXT 0x39
// FLUSH CACHE, and the form a drive using 48-bit addressing wants. Neither
// takes an address; the difference is only which one a given drive admits to
// supporting, so ideflush tries the plain one and falls back.
#define IDE_CMD_FLUSH 0xe7
#define IDE_CMD_FLUSH_EXT 0xea
// idequeue points to the buf now being read/written to the disk.
// idequeue->qnext points to the next buf to be processed.
@ -248,10 +253,69 @@ ideintr(void)
// Start disk on next buf in queue.
if(idequeue != 0)
idestart(idequeue);
else
wakeup(&idequeue); // ideflush waits for the channel to go quiet
release(&idelock);
}
// FLUSH CACHE: do not return until the drive has committed what it has taken
// to the medium.
//
// Every layer above this driver assumes a write that has returned is on the
// disk. On a drive with a volatile write cache that is simply not true, and it
// is the assumption SchFS's whole crash-safety argument rests on: copy-on-write
// promises that a crash lands on either the old tree or the new one, but only
// if the tree blocks reach the platter before the uberblock that names them.
// The drive is otherwise free to write them in whatever order suits its head,
// and a power cut then leaves a newest uberblock -- correctly checksummed --
// pointing at a tree that was never written. Nothing can recover from that,
// because nothing about it looks wrong.
//
// Polled rather than queued: it moves no data, so it does not fit idestart's
// shape, and it is rare enough that the simplest correct thing wins.
int
ideflush(int dev)
{
int r = 0;
if(dev == 0 ? !havedisk0 : !havedisk1)
return 0; // nothing there is nothing to flush
acquire(&idelock);
// The channel has to be idle. This talks straight to the registers, and
// issuing it underneath a queued transfer would corrupt that transfer's
// handshake rather than merely delay it.
while(idequeue != 0)
sleep(&idequeue, &idelock);
outb(0x3f6, 2); // nIEN: this command answers by status, not IRQ
outb(0x1f6, 0xe0 | ((dev & 1) << 4));
if(idewait(0) < 0){
r = -1;
goto out;
}
outb(0x1f7, IDE_CMD_FLUSH);
// A drive with a full write cache can take seconds over this, which is the
// other reason idewait's bound is as large as it is.
if(idewait(1) < 0){
// Drives that came up in 48-bit mode may abort the 28-bit spelling. Same
// operation, and there is no third option, so a failure here is real.
outb(0x1f7, IDE_CMD_FLUSH_EXT);
if(idewait(1) < 0){
cprintf("ide%d: FLUSH CACHE failed; writes are not durable\n", dev);
r = -1;
}
}
out:
outb(0x3f6, 0); // interrupts back on for ordinary transfers
release(&idelock);
return r;
}
//PAGEBREAK!
// Sync buf with disk.
// If B_DIRTY is set, write buf to disk, clear B_DIRTY, set B_VALID.

60
kernel/dev/procfs.c
View file

@ -31,6 +31,7 @@
#include "proc.h"
#include "uaccess.h"
#include "errno.h"
#include "fsstat.h"
#define PROC_BUFSZ 2048
@ -277,17 +278,68 @@ gen_loadavg(struct sbuf *sb)
sb_puts(sb, " 1\n");
}
// df and mountpoint parse this. One root filesystem, no mount table yet.
// df and mountpoint parse this.
//
// It used to be three fixed lines, which was true while there was one
// filesystem and no way to mount a second. Now `mount -t schfs /dev/hdb /mnt`
// works, and a mount missing from here is invisible to df -- it printed a
// header and no row, because df looks the path up in this list before it calls
// statfs at all.
static void
gen_mounts(struct sbuf *sb)
{
char path[MAXPATH], source[MAXPATH], type[16];
uint fsdev;
sb_puts(sb, "devtmpfs /dev devtmpfs rw 0 0\n");
for (int i = 0; i < 8; i++) {
if (mount_get(i, path, source, type, &fsdev) != 0)
continue;
// "/" is emitted last, below, so skip it if it somehow appears here.
if (path[0] == '/' && path[1] == '\0')
continue;
sb_puts(sb, source[0] ? source : "none");
sb_puts(sb, " ");
sb_puts(sb, path);
sb_puts(sb, " ");
sb_puts(sb, type[0] ? type : "schfs");
// A mounted snapshot is genuinely read-only, and saying so is what stops
// anything trying to write to one and reporting a confusing EROFS later.
sb_puts(sb, SCHFS_IS_SNAP_DEV(fsdev) ? " ro 0 0\n" : " rw 0 0\n");
}
// Root must be last: df keeps the last mount point that prefixes the path
// it was asked about, so listing / first made `df /` report /proc instead.
sb_puts(sb, "devtmpfs /dev devtmpfs rw 0 0\n");
sb_puts(sb, "proc /proc proc rw 0 0\n");
sb_puts(sb, "/dev/root / schfs rw 0 0\n");
}
// Where the time goes in the filesystem and block paths.
//
// Cycles, not seconds: the numbers are only ever compared against each other,
// and converting would mean knowing the TSC frequency, which this kernel does
// not. What matters is the ratios and the miss counts.
static void
gen_fsstat(struct sbuf *sb)
{
sb_puts(sb, "bread_calls "); sb_putu(sb, fsstat.bread_calls);
sb_puts(sb, "\nbread_miss "); sb_putu(sb, fsstat.bread_miss);
sb_puts(sb, "\nbget_evict_dirty "); sb_putu(sb, fsstat.bget_evict_dirty);
sb_puts(sb, "\ndisk_reads "); sb_putu(sb, fsstat.disk_reads);
sb_puts(sb, "\ndisk_writes "); sb_putu(sb, fsstat.disk_writes);
sb_puts(sb, "\ndisk_kcycles "); sb_putu(sb, fsstat.disk_cycles / 1000);
sb_puts(sb, "\nread_calls "); sb_putu(sb, fsstat.read_calls);
sb_puts(sb, "\nread_bytes "); sb_putu(sb, fsstat.read_bytes);
sb_puts(sb, "\nread_kcycles "); sb_putu(sb, fsstat.read_cycles / 1000);
sb_puts(sb, "\ngetdents_calls "); sb_putu(sb, fsstat.getdents_calls);
sb_puts(sb, "\ngetdents_walked "); sb_putu(sb, fsstat.getdents_walked);
sb_puts(sb, "\ngetdents_kcycles "); sb_putu(sb, fsstat.getdents_cycles / 1000);
sb_puts(sb, "\ndescents "); sb_putu(sb, fsstat.descents);
sb_puts(sb, "\nexec_calls "); sb_putu(sb, fsstat.exec_calls);
sb_puts(sb, "\nexec_kcycles "); sb_putu(sb, fsstat.exec_cycles / 1000);
sb_puts(sb, "\n");
}
static void
gen_filesystems(struct sbuf *sb)
{
@ -311,6 +363,7 @@ gen_filesystems(struct sbuf *sb)
#define PROC_PCI 11
#define PROC_INTERRUPTS 12
#define PROC_DEVICES 13
#define PROC_FSSTAT 14
// Render global file `id` (a PROC_* constant) into `buf`. Returns its length.
//
@ -330,6 +383,7 @@ proc_gen_global(int id, char *buf, int cap)
case PROC_STAT: gen_stat(&sb); break;
case PROC_LOADAVG: gen_loadavg(&sb); break;
case PROC_MOUNTS: gen_mounts(&sb); break;
case PROC_FSSTAT: gen_fsstat(&sb); break;
case PROC_FILESYSTEMS: gen_filesystems(&sb); break;
// SchFS renders straight into the buffer rather than through sbuf, because
// it lives in kernel/fs/schfs and must not depend on this file's helpers.

37
kernel/drivers/usb/storage/usb_storage.c
View file

@ -53,6 +53,10 @@ struct csw {
#define SCSI_READ_CAPACITY10 0x25
#define SCSI_READ10 0x28
#define SCSI_WRITE10 0x2A
// SYNCHRONIZE CACHE(10): commit the device's write cache to the medium. The
// SCSI spelling of ATA's FLUSH CACHE, and needed for the same reason -- see
// ideflush() and the barrier in schfs_vol_commit.
#define SCSI_SYNC_CACHE10 0x35
#define SECTOR_SIZE 512
@ -265,6 +269,39 @@ usb_storage_rw(struct buf *b)
b->flags &= ~B_DIRTY;
}
// Make everything written so far durable. Returns 0, or -1 if the stick would
// not confirm it.
//
// A whole-device sync with no LBA range, which is what the callers want: the
// filesystem's barrier is "everything before this point", not "these blocks".
//
// Cheap sticks routinely reject this command outright. That is reported rather
// than ignored, because the caller is about to write an uberblock on the
// strength of it -- and a barrier that silently does nothing is worse than no
// barrier at all, since it makes an unsafe volume look safe.
int
usb_storage_flush(void)
{
uint8 cdb[10];
int r;
if(!ums.present)
return 0;
memset(cdb, 0, sizeof(cdb));
cdb[0] = SCSI_SYNC_CACHE10; // LBA 0, count 0: the entire medium
acquiresleep(&ums.lock);
r = bot_command(cdb, sizeof(cdb), 0, 0, 0);
releasesleep(&ums.lock);
if(r < 0){
cprintf("usb-storage: SYNCHRONIZE CACHE failed; writes are not durable\n");
return -1;
}
return 0;
}
// Read block 1 through the ordinary buffer cache and report what is there.
//
// This is the only test that proves the whole path rather than a piece of it:

5
kernel/fs/exec.c
View file

@ -9,6 +9,8 @@
#include "elf.h"
#include "fcntl.h"
#include "errno.h"
#include "x86.h"
#include "fsstat.h"
// Binaries with entry points at 0x100000+ are musl/Linux-ABI.
// SchOS-native binaries (init) are linked at 0x4000.
@ -19,6 +21,8 @@ extern int linux_syscall_map_size;
int
exec(char *path, char **argv)
{
uint64 _t0 = rdtsc();
fsstat.exec_calls++;
char *s, *last;
int i, off;
uint64 argc, sz, sp, ustack[MAXARG+1];
@ -228,6 +232,7 @@ exec(char *path, char **argv)
switchuvm(myproc());
freevm(oldpgdir);
fsstat.exec_cycles += rdtsc() - _t0;
return 0;
bad:

9
kernel/fs/file.c
View file

@ -216,12 +216,15 @@ filewrite(struct file *f, char *addr, int n)
// Recycle-bin check removed here too; see the note in fileread().
iunlock(f->ip);
int max = ((LOGSIZE-1-1-2) / 2) * 512;
// How much may go in one call is the filesystem's business. v1 is bounded
// by its log; SchFS is not bounded at all and would rather have the lot,
// because it makes one transaction out of whatever it is given.
uint max = f->ip->ops->max_write;
int i = 0;
while(i < n){
int n1 = n - i;
if(n1 > max)
n1 = max;
if(max && (uint)n1 > max)
n1 = (int)max;
begin_op();
ilock(f->ip);

67
kernel/fs/fs.c
View file

@ -268,6 +268,10 @@ static const struct fs_ops v1_ops = {
.dirempty = v1_dirempty,
.ialloc = v1_ialloc,
.getdents = v1_getdents,
// A write has to fit in the on-disk log, with room for the inode block, the
// bitmap block and the log header. This is the constant filewrite used to
// apply to every filesystem, SchFS included.
.max_write = ((LOGSIZE-1-1-2) / 2) * 512,
};
// Defined in fs/schfs/vfs.c, which is only linked when SchFS is built in.
@ -290,10 +294,10 @@ fs_readonly(uint dev)
static const struct fs_ops *
ops_for(uint dev)
{
if (dev == SCHFS_DEV) return &schfs_ops;
// A mounted snapshot is the same filesystem seen through an older root, so it
// is the same operations; they dispatch on the device themselves.
if (SCHFS_IS_SNAP_DEV(dev)) return &schfs_ops;
// Every mounted SchFS volume, and every mounted snapshot. One table serves
// them all: the operations take an inode, an inode carries its device, and
// kmount.c turns that device into the right volume.
if (SCHFS_IS_ANY_DEV(dev)) return &schfs_ops;
if (dev == PROCFS_DEV) return &procfs_ops;
return &v1_ops;
}
@ -399,7 +403,10 @@ v1_istore(struct inode *ip)
dip->type = ip->type;
dip->major = ip->major;
dip->minor = ip->minor;
dip->nlink = ip->nlink;
// v1's on-disk field is a short and cannot be widened without changing the
// format, so the narrowing is the format's, not an accident of this line.
// The in-memory count is an int now because SchFS needs the range.
dip->nlink = (short)ip->nlink;
dip->size = ip->size;
dip->rwmode = ip->rwmode;
dip->supermode = ip->supermode;
@ -1529,9 +1536,27 @@ static struct {
uint64 inum; // the mount point directory itself
uint fsdev; // device of the filesystem mounted on it
uint64 root; // that filesystem's root inode
// What userland called it. Resolution never needs these -- it works in
// (dev, inum) -- but /proc/mounts does, and a path cannot be recovered from
// an inode without walking the whole tree backwards. So the strings are
// kept at mount time, when the caller still has them.
char path[MAXPATH];
char source[MAXPATH];
char type[16];
} m[NMOUNT];
} mtable;
// Copy a string into a fixed field, always NUL-terminating.
static void
mstrcpy(char *dst, const char *src, int cap)
{
int i = 0;
if (src)
for (; i < cap - 1 && src[i]; i++)
dst[i] = src[i];
dst[i] = '\0';
}
void
mountinit(void)
{
@ -1540,7 +1565,8 @@ mountinit(void)
// Attach `fsdev`:`root` over the directory `dev`:`inum`.
int
mount_add(uint dev, uint64 inum, uint fsdev, uint64 root)
mount_add(uint dev, uint64 inum, uint fsdev, uint64 root,
const char *path, const char *source, const char *type)
{
acquire(&mtable.lock);
for (int i = 0; i < NMOUNT; i++)
@ -1556,6 +1582,9 @@ mount_add(uint dev, uint64 inum, uint fsdev, uint64 root)
mtable.m[i].inum = inum;
mtable.m[i].fsdev = fsdev;
mtable.m[i].root = root;
mstrcpy(mtable.m[i].path, path, MAXPATH);
mstrcpy(mtable.m[i].source, source, MAXPATH);
mstrcpy(mtable.m[i].type, type, 16);
release(&mtable.lock);
return 0;
}
@ -1563,6 +1592,30 @@ mount_add(uint dev, uint64 inum, uint fsdev, uint64 root)
return -1; /* table full */
}
// Walk the mount table for /proc/mounts. Returns 0 and fills the caller's
// buffers for entry `i`, or -1 when there is no such entry.
//
// An index rather than a callback because the caller formats into a growing
// string buffer and must not hold this lock while doing it.
int
mount_get(int i, char *path, char *source, char *type, uint *fsdev)
{
if (i < 0 || i >= NMOUNT)
return -1;
acquire(&mtable.lock);
if (!mtable.m[i].used) {
release(&mtable.lock);
return -1;
}
mstrcpy(path, mtable.m[i].path, MAXPATH);
mstrcpy(source, mtable.m[i].source, MAXPATH);
mstrcpy(type, mtable.m[i].type, 16);
if (fsdev)
*fsdev = mtable.m[i].fsdev;
release(&mtable.lock);
return 0;
}
// Detach by naming the *mounted* filesystem's root, which is what a path to
// the mount point resolves to once the mount exists. Naming the covered
// directory instead is not possible from userland: the whole point of the
@ -1663,7 +1716,7 @@ namex_from(struct inode *start, char *path, int nameiparent, char *name)
// Root is a SchFS object, not a v1 inode. ops_for(SCHFS_DEV) then routes
// every operation on it through schfs_ops, which is the whole point: the v1
// format code in this file is no longer reachable from `/`.
ip = (*spath == '/') ? iget(SCHFS_DEV, schfs_kroot()) : idup(start);
ip = (*spath == '/') ? iget(SCHFS_DEV, schfs_kroot(SCHFS_DEV)) : idup(start);
// The root itself can be a mount point, and a relative walk can start on one.
ip = mount_cross(ip);

3
kernel/fs/procfs.c
View file

@ -90,6 +90,7 @@ static const struct global_file globals[] = {
{ "pci", 11 },
{ "interrupts", 12 },
{ "devices", 13 },
{ "fsstat", 14 },
};
#define NGLOBALS ((int)(sizeof(globals) / sizeof(globals[0])))
@ -518,7 +519,7 @@ procfs_mount(void)
cprintf("procfs: /proc does not exist; not mounted\n");
return;
}
if (mount_add(dp->dev, dp->inum, PROCFS_DEV, PROCFS_ROOT) < 0)
if (mount_add(dp->dev, dp->inum, PROCFS_DEV, PROCFS_ROOT, "/proc", "proc", "proc") < 0)
cprintf("procfs: /proc already has something mounted on it\n");
iput(dp);
}

507
kernel/fs/schfs/attr.c Normal file
View file

@ -0,0 +1,507 @@
// SPDX-License-Identifier: Apache-2.0
//
// Attributes, and the index over them.
//
// An attribute is a named value on an object. The interesting part is not
// storing them -- that is another key type in a tree that already exists -- but
// the index: an indexed attribute gets a second set of records keyed by *value*
// rather than by object, so "every object whose `tag` is `holiday`" is a range
// scan of a few keys instead of a walk of the whole filesystem.
//
// That is the whole of what "a filesystem you can query" means here, and the
// reason the design was one B-tree with many key types from the start. There is
// no query engine, no parser and no planner; there is an index, which is the
// part that actually makes the question fast. See docs/architecture/schfs.md §6.
//
// ---------------------------------------------------------------------------
// Packed lists, again
//
// Attributes are keyed (ino, ATTR, hash-of-name) and index entries
// (name-hash, INDEX, hash-of-value). Both hash something that can collide, so
// both store a *packed list* of records rather than a single one -- the same
// shape directory entries use, for the same reason. A 64-bit hash makes a
// collision vanishingly unlikely and "vanishingly unlikely" is not "impossible".
#include "schfs_int.h"
#ifdef SCHFS_HOSTED
#include <string.h>
#else
#include "types.h"
#include "defs.h"
#endif
/* ------------------------------------------------------------------ */
/* Reading */
/* ------------------------------------------------------------------ */
// Find one attribute in the packed list under its name's hash.
//
// Returns 1 and fills the caller's buffers if present, 0 if absent, -2 on
// corruption. `*valuelen` is set to the value's true length even when the
// buffer is too small, so a caller can size a second call -- which is what
// getxattr(2) promises.
int
schfs_attr_get(struct schfs_vol *v, uint64_t ino, const char *name,
unsigned namelen, void *out, unsigned cap, unsigned *valuelen,
uint16_t *type)
{
if (namelen == 0 || namelen > SCHFS_ATTR_NAME_MAX)
return 0;
struct schfs_key k;
schfs_mkkey(&k, ino, SCHFS_KEY_ATTR, schfs_namehash(name, namelen));
unsigned len = SCHFS_MAX_INLINE;
int r = schfs_btree_lookup(&v->tree, &v->ctx, &k, v->dirbuf, &len);
if (r == -1)
return 0;
if (r != 0)
return r;
if (len > SCHFS_MAX_INLINE)
return -2;
unsigned off = 0;
while (off + sizeof(struct schfs_attr_rec) <= len) {
const struct schfs_attr_rec *a =
(const struct schfs_attr_rec *)(v->dirbuf + off);
unsigned step = SCHFS_ATTR_LEN(a->namelen, a->valuelen);
if (a->namelen == 0 || a->namelen > SCHFS_ATTR_NAME_MAX ||
a->valuelen > SCHFS_ATTR_VALUE_MAX || off + step > len)
return -2;
const char *nm = (const char *)a + sizeof(*a);
if (a->namelen == namelen && memcmp(nm, name, namelen) == 0) {
if (valuelen) *valuelen = a->valuelen;
if (type) *type = a->type;
if (out && cap >= a->valuelen)
memmove(out, nm + a->namelen, a->valuelen);
else if (out)
return -3; /* buffer too small; length is set */
return 1;
}
off += step;
}
return 0;
}
// Hand every attribute on an object to a callback, in the order they are
// stored. Used by listxattr and by the reindexing walk.
int
schfs_attr_iterate(struct schfs_vol *v, struct schfs_cursor *c, uint64_t ino,
int (*fn)(const char *name, unsigned namelen,
const void *val, unsigned vallen,
uint16_t type, void *arg),
void *arg)
{
struct schfs_key start;
schfs_mkkey(&start, ino, SCHFS_KEY_ATTR, 0);
// Re-point at the volume's current tree. The root moves on every commit
// under copy-on-write, so a cursor is bound per operation rather than once.
schfs_cursor_init(c, &v->tree, c->blk);
if (schfs_cursor_seek(c, &start) != 0)
return 0;
for (;;) {
const struct schfs_key *k = schfs_cursor_key(c);
if (!k || k->objectid != ino || k->type != SCHFS_KEY_ATTR)
return 0;
unsigned len;
const char *val = (const char *)schfs_cursor_val(c, &len);
if (!val || len > SCHFS_MAX_INLINE)
return -2;
// Copied out before the callback runs: the callback may look something
// up, and that reuses the very buffer this points into. The same trap
// that once reparented directories under their own children.
memmove(v->dirbuf, val, len);
unsigned off = 0;
while (off + sizeof(struct schfs_attr_rec) <= len) {
const struct schfs_attr_rec *a =
(const struct schfs_attr_rec *)(v->dirbuf + off);
unsigned step = SCHFS_ATTR_LEN(a->namelen, a->valuelen);
if (a->namelen == 0 || a->namelen > SCHFS_ATTR_NAME_MAX ||
a->valuelen > SCHFS_ATTR_VALUE_MAX || off + step > len)
return -2;
const char *nm = (const char *)a + sizeof(*a);
int stop = fn(nm, a->namelen, nm + a->namelen, a->valuelen,
a->type, arg);
if (stop)
return 0;
off += step;
}
if (schfs_cursor_next(c) != 0)
return 0;
}
}
/* ------------------------------------------------------------------ */
/* The index */
/* */
/* Which attributes are indexed is one record, and it is consulted on */
/* every attribute write -- so it is read into the volume rather than */
/* fetched each time. See schfs_index_reload. */
/* ------------------------------------------------------------------ */
// Is `name` an indexed attribute?
//
// Answered from the cached list rather than the tree. setxattr asks this every
// time, and the answer is almost always no; a tree descent to say so would make
// every attribute write pay for a feature it is not using.
static int
index_wanted(struct schfs_vol *v, const char *name, unsigned namelen)
{
unsigned off = 0;
while (off + 1 <= v->idxnames_len) {
unsigned l = (unsigned char)v->idxnames[off];
if (l == 0 || off + 1 + l > v->idxnames_len)
return 0;
if (l == namelen && memcmp(v->idxnames + off + 1, name, namelen) == 0)
return 1;
off += 1 + l;
}
return 0;
}
// Read the indexed-attribute list off the disk into the volume.
//
// Called at mount and after any change to the list. A volume with no indexes --
// which is every volume until somebody asks for one -- ends up with an empty
// list and index_wanted answers no without touching anything.
int
schfs_index_reload(struct schfs_vol *v)
{
struct schfs_key k;
schfs_mkkey(&k, SCHFS_INDEX_OBJECTID, SCHFS_KEY_INDEX, SCHFS_INDEX_LIST_OFF);
unsigned len = sizeof(v->idxnames);
int r = schfs_btree_lookup(&v->tree, &v->ctx, &k, v->idxnames, &len);
if (r == -1) {
v->idxnames_len = 0;
return 0;
}
if (r != 0)
return r;
v->idxnames_len = len;
return 0;
}
// Add one object to the index for (attribute name, value).
static int
index_add(struct schfs_vol *v, const char *name, unsigned namelen,
const void *val, unsigned vallen, uint64_t ino)
{
struct schfs_key k;
schfs_mkkey(&k, schfs_namehash(name, namelen), SCHFS_KEY_INDEX,
schfs_namehash((const char *)val, vallen));
unsigned len = SCHFS_MAX_INLINE;
int r = schfs_btree_lookup(&v->tree, &v->ctx, &k, v->dirbuf, &len);
if (r == -1)
len = 0;
else if (r != 0)
return r;
// Already listed? An object that sets the same value twice must not appear
// twice, or a query would return it twice.
unsigned off = 0;
while (off + sizeof(struct schfs_index_rec) <= len) {
const struct schfs_index_rec *ir =
(const struct schfs_index_rec *)(v->dirbuf + off);
unsigned step = SCHFS_INDEX_LEN(ir->valuelen);
if (off + step > len)
return -2;
if (ir->objectid == ino)
return 0;
off += step;
}
unsigned add = SCHFS_INDEX_LEN(vallen);
if (len + add > SCHFS_MAX_INLINE)
return -3; /* too many objects share this value */
struct schfs_index_rec ir;
memset(&ir, 0, sizeof(ir));
ir.objectid = ino;
ir.valuelen = vallen;
memmove(v->dirbuf + len, &ir, sizeof(ir));
memmove(v->dirbuf + len + sizeof(ir), val, vallen);
// The padding is zeroed rather than left as whatever was in the buffer, so
// two identical filesystems produce byte-identical images.
memset(v->dirbuf + len + sizeof(ir) + vallen, 0,
add - sizeof(ir) - vallen);
return schfs_btree_insert(&v->tree, &v->ctx, &k, v->dirbuf, len + add);
}
// Remove one object from the index for (attribute name, value).
static int
index_del(struct schfs_vol *v, const char *name, unsigned namelen,
const void *val, unsigned vallen, uint64_t ino)
{
struct schfs_key k;
schfs_mkkey(&k, schfs_namehash(name, namelen), SCHFS_KEY_INDEX,
schfs_namehash((const char *)val, vallen));
unsigned len = SCHFS_MAX_INLINE;
int r = schfs_btree_lookup(&v->tree, &v->ctx, &k, v->dirbuf, &len);
if (r == -1)
return 0;
if (r != 0)
return r;
unsigned off = 0;
while (off + sizeof(struct schfs_index_rec) <= len) {
struct schfs_index_rec *ir = (struct schfs_index_rec *)(v->dirbuf + off);
unsigned step = SCHFS_INDEX_LEN(ir->valuelen);
if (off + step > len)
return -2;
if (ir->objectid == ino) {
memmove(v->dirbuf + off, v->dirbuf + off + step, len - off - step);
len -= step;
if (len == 0)
return schfs_btree_delete(&v->tree, &v->ctx, &k);
return schfs_btree_insert(&v->tree, &v->ctx, &k, v->dirbuf, len);
}
off += step;
}
return 0;
}
/* ------------------------------------------------------------------ */
/* Writing */
/* ------------------------------------------------------------------ */
// Set an attribute, replacing any previous value of the same name.
//
// The index is kept in step here rather than by a separate pass, because an
// index that can be out of date is worse than none: a query would answer
// confidently and wrongly.
int
schfs_attr_set(struct schfs_vol *v, uint64_t ino, const char *name,
unsigned namelen, const void *val, unsigned vallen,
uint16_t type)
{
if (namelen == 0 || namelen > SCHFS_ATTR_NAME_MAX)
return -1;
if (vallen > SCHFS_ATTR_VALUE_MAX)
return -1;
schfs_cache_invalidate();
// The old value has to come out of the index before the new one goes in,
// and it has to be read before the record is overwritten.
int indexed = index_wanted(v, name, namelen);
char oldval[SCHFS_ATTR_VALUE_MAX];
unsigned oldlen = 0;
int had_old = 0;
if (indexed) {
unsigned vl = 0;
int g = schfs_attr_get(v, ino, name, namelen, oldval, sizeof(oldval),
&vl, 0);
if (g == 1) { had_old = 1; oldlen = vl; }
}
struct schfs_key k;
schfs_mkkey(&k, ino, SCHFS_KEY_ATTR, schfs_namehash(name, namelen));
unsigned len = SCHFS_MAX_INLINE;
int r = schfs_btree_lookup(&v->tree, &v->ctx, &k, v->dirbuf, &len);
if (r == -1)
len = 0;
else if (r != 0)
return r;
// Drop any record with this name; what remains is everything that merely
// hashed the same.
unsigned off = 0;
while (off + sizeof(struct schfs_attr_rec) <= len) {
struct schfs_attr_rec *a = (struct schfs_attr_rec *)(v->dirbuf + off);
unsigned step = SCHFS_ATTR_LEN(a->namelen, a->valuelen);
if (a->namelen == 0 || off + step > len)
return -2;
const char *nm = (const char *)a + sizeof(*a);
if (a->namelen == namelen && memcmp(nm, name, namelen) == 0) {
memmove(v->dirbuf + off, v->dirbuf + off + step, len - off - step);
len -= step;
break;
}
off += step;
}
unsigned add = SCHFS_ATTR_LEN(namelen, vallen);
if (len + add > SCHFS_MAX_INLINE)
return -3;
struct schfs_attr_rec a;
memset(&a, 0, sizeof(a));
a.namelen = namelen;
a.valuelen = vallen;
a.type = type;
memmove(v->dirbuf + len, &a, sizeof(a));
memmove(v->dirbuf + len + sizeof(a), name, namelen);
memmove(v->dirbuf + len + sizeof(a) + namelen, val, vallen);
memset(v->dirbuf + len + sizeof(a) + namelen + vallen, 0,
add - sizeof(a) - namelen - vallen);
r = schfs_btree_insert(&v->tree, &v->ctx, &k, v->dirbuf, len + add);
if (r != 0)
return r;
if (indexed) {
if (had_old) {
r = index_del(v, name, namelen, oldval, oldlen, ino);
if (r != 0)
return r;
}
r = index_add(v, name, namelen, val, vallen, ino);
if (r != 0)
return r;
}
return 0;
}
// Remove an attribute. Absent is not a failure.
int
schfs_attr_del(struct schfs_vol *v, uint64_t ino, const char *name,
unsigned namelen)
{
if (namelen == 0 || namelen > SCHFS_ATTR_NAME_MAX)
return -1;
schfs_cache_invalidate();
int indexed = index_wanted(v, name, namelen);
char oldval[SCHFS_ATTR_VALUE_MAX];
unsigned oldlen = 0;
int had_old = 0;
if (indexed) {
unsigned vl = 0;
int g = schfs_attr_get(v, ino, name, namelen, oldval, sizeof(oldval),
&vl, 0);
if (g == 1) { had_old = 1; oldlen = vl; }
}
struct schfs_key k;
schfs_mkkey(&k, ino, SCHFS_KEY_ATTR, schfs_namehash(name, namelen));
unsigned len = SCHFS_MAX_INLINE;
int r = schfs_btree_lookup(&v->tree, &v->ctx, &k, v->dirbuf, &len);
if (r == -1)
return 0;
if (r != 0)
return r;
unsigned off = 0;
int found = 0;
while (off + sizeof(struct schfs_attr_rec) <= len) {
struct schfs_attr_rec *a = (struct schfs_attr_rec *)(v->dirbuf + off);
unsigned step = SCHFS_ATTR_LEN(a->namelen, a->valuelen);
if (a->namelen == 0 || off + step > len)
return -2;
const char *nm = (const char *)a + sizeof(*a);
if (a->namelen == namelen && memcmp(nm, name, namelen) == 0) {
memmove(v->dirbuf + off, v->dirbuf + off + step, len - off - step);
len -= step;
found = 1;
break;
}
off += step;
}
if (!found)
return 0;
r = (len == 0) ? schfs_btree_delete(&v->tree, &v->ctx, &k)
: schfs_btree_insert(&v->tree, &v->ctx, &k, v->dirbuf, len);
if (r != 0)
return r;
if (indexed && had_old)
return index_del(v, name, namelen, oldval, oldlen, ino);
return 0;
}
/* ------------------------------------------------------------------ */
/* Declaring an index, and querying it */
/* ------------------------------------------------------------------ */
// Start indexing an attribute.
//
// Only affects values written *after* this: there is no reindexing pass, and
// pretending otherwise would be the kind of half-truth that makes a query
// answer confidently and wrongly. A caller that wants existing objects in the
// index rewrites their attributes.
int
schfs_index_create(struct schfs_vol *v, const char *name, unsigned namelen)
{
if (namelen == 0 || namelen > SCHFS_ATTR_NAME_MAX)
return -1;
schfs_cache_invalidate();
if (index_wanted(v, name, namelen))
return 0; /* already indexed */
if (v->idxnames_len + 1 + namelen > sizeof(v->idxnames))
return -3;
v->idxnames[v->idxnames_len] = (char)namelen;
memmove(v->idxnames + v->idxnames_len + 1, name, namelen);
v->idxnames_len += 1 + namelen;
struct schfs_key k;
schfs_mkkey(&k, SCHFS_INDEX_OBJECTID, SCHFS_KEY_INDEX, SCHFS_INDEX_LIST_OFF);
return schfs_btree_insert(&v->tree, &v->ctx, &k, v->idxnames,
v->idxnames_len);
}
// Every object whose `name` attribute equals `val`.
//
// One tree descent, whatever the size of the filesystem -- which is the entire
// point. Returns how many object ids were written to `out`, or negative on
// error. A caller with a full buffer gets `cap` and no way to ask for more,
// which is a limit worth having rather than an unbounded kernel allocation.
int
schfs_query_equal(struct schfs_vol *v, const char *name, unsigned namelen,
const void *val, unsigned vallen, uint64_t *out, int cap)
{
if (namelen == 0 || namelen > SCHFS_ATTR_NAME_MAX)
return -1;
if (!index_wanted(v, name, namelen))
return -1; /* not indexed: there is nothing to scan */
struct schfs_key k;
schfs_mkkey(&k, schfs_namehash(name, namelen), SCHFS_KEY_INDEX,
schfs_namehash((const char *)val, vallen));
unsigned len = SCHFS_MAX_INLINE;
int r = schfs_btree_lookup(&v->tree, &v->ctx, &k, v->dirbuf, &len);
if (r == -1)
return 0; /* no object carries that value */
if (r != 0)
return r;
int n = 0;
unsigned off = 0;
while (off + sizeof(struct schfs_index_rec) <= len && n < cap) {
const struct schfs_index_rec *ir =
(const struct schfs_index_rec *)(v->dirbuf + off);
unsigned step = SCHFS_INDEX_LEN(ir->valuelen);
if (off + step > len)
return -2;
// The value is stored, not just its hash, and this is why: two values
// can hash alike, and a query that returned the wrong file would be a
// far worse failure than a slow one.
if (ir->valuelen == vallen &&
memcmp((const char *)ir + sizeof(*ir), val, vallen) == 0)
out[n++] = ir->objectid;
off += step;
}
return n;
}

86
kernel/fs/schfs/btree.c
View file

@ -30,16 +30,44 @@
/* CRC32c (Castagnoli), table-free so this stays usable very early. */
/* ------------------------------------------------------------------ */
// Table-driven, with the table built on first use.
//
// This was a bit-at-a-time loop -- eight shifts per byte, 32768 iterations to
// checksum one 4096-byte block -- justified by staying usable before anything
// is initialised. The table keeps that property, because it builds itself on
// the first call rather than needing an init hook: the cost is one 1 KB static
// array and a first call that does the work the old loop did for 256 bytes.
//
// It matters because every block read off the medium is checksummed in full.
// After the cursor stopped re-reading resident blocks this became the largest
// remaining cost in the read path, which is the usual shape of these things --
// removing the top item promotes whatever was underneath it.
static uint32_t crc32c_tab[256];
static int crc32c_tab_ready;
static void
crc32c_build(void)
{
for (unsigned i = 0; i < 256; i++) {
uint32_t c = i;
for (int k = 0; k < 8; k++)
c = (c >> 1) ^ (0x82F63B78u & (uint32_t)(-(int32_t)(c & 1)));
crc32c_tab[i] = c;
}
crc32c_tab_ready = 1;
}
uint32_t
schfs_crc32c(uint32_t crc, const void *buf, unsigned len)
{
const uint8_t *p = (const uint8_t *)buf;
if (!crc32c_tab_ready)
crc32c_build();
crc = ~crc;
while (len--) {
crc ^= *p++;
for (int i = 0; i < 8; i++)
crc = (crc >> 1) ^ (0x82F63B78u & (uint32_t)(-(int32_t)(crc & 1)));
}
while (len--)
crc = (crc >> 8) ^ crc32c_tab[(crc ^ *p++) & 0xFF];
return ~crc;
}
@ -264,14 +292,60 @@ schfs_cursor_init(struct schfs_cursor *c, struct schfs_btree *t, void *blk)
c->blk = blk;
c->depth = 0;
c->valid = 0;
// Whatever the previous user of this buffer left in it is not ours to
// trust: a mutation between two cursors could have rewritten that block in
// place, and the copy here would be a generation behind.
c->resident = 0;
c->epoch = 0;
}
// Bumped by schfs_cache_invalidate(), which every write entry point calls
// before it touches anything. A cursor's resident block is only trusted while
// this has not moved: within one transaction group the tree rewrites its own
// nodes *in place*, so a cached copy of block N can be a generation behind
// while still carrying block N's own number in its header. Nothing else
// distinguishes the two, which is why identity checks alone are not enough.
uint64_t schfs_tree_epoch = 1;
// Load path[d]'s block into the cursor buffer, verifying it against the
// checksum recorded alongside it.
//
// The early return is the whole reason this system was slow.
//
// schfs_cursor_val calls this on *every* access to the current record, and
// iter_item calls schfs_cursor_val once per directory entry -- deliberately, so
// that a callback which disturbs the shared buffer cannot corrupt the walk. But
// node_read is not cheap: it copies 4096 bytes out of the buffer cache and then
// runs CRC32c over all 4096. Listing a 243-entry directory did that once per
// entry, and `ls /bin` moved several megabytes through both. Measured across
// one session: 398,999 block reads against 445 actual disk misses, so ~99.9% of
// them were re-reading a block already sitting in the buffer.
//
// Two conditions, and both are needed. `resident` says this cursor put that
// block here; the header says the page still holds it, which is what catches a
// second cursor sharing the same page having loaded something else in between.
// The header alone would be enough for identity, but a block cached from before
// an in-place rewrite has the same blocknr -- so the per-cursor field, cleared
// at init, is what bounds the trust to one entry point's lifetime.
static int
cur_load(struct schfs_cursor *c, int d)
{
return node_read(c->t->dev, c->path[d].blocknr, c->path[d].crc, c->blk);
uint64_t want = c->path[d].blocknr;
if (c->resident == want && c->epoch == schfs_tree_epoch) {
const struct schfs_node_hdr *h = HDR(c->blk);
if (h->magic == SCHFS_NODE_MAGIC && h->blocknr == want &&
(c->path[d].crc == 0 || c->path[d].crc == h->crc))
return 0;
}
if (node_read(c->t->dev, want, c->path[d].crc, c->blk) != 0) {
c->resident = 0;
return -2;
}
c->resident = want;
c->epoch = schfs_tree_epoch;
return 0;
}
// From the node currently in the buffer at depth `d`, walk down to a leaf,

28
kernel/fs/schfs/btree.h
View file

@ -40,6 +40,19 @@ struct schfs_bdev {
// blocks from balloc. NULL means unsupported; callers then fall back to
// allocating singly and accept the extra extent records.
uint64_t (*balloc_run)(void *ctx, uint64_t n);
// Optional: make every write issued so far durable, and do not return until
// it is. NULL means the backing store has no volatile stage to worry about
// -- a memory-backed device, or a host file the tools fsync themselves.
//
// This is the one hook the format's crash-safety argument actually rests
// on. Copy-on-write guarantees that a tree is never half-modified, but only
// if the blocks a root points at reach the medium *before* the block that
// names that root. Nothing in the ordering of bwrite calls guarantees that:
// a drive with a write cache is free to complete them in any order it
// likes. Without a barrier between the two, a power cut lands on a newest
// uberblock whose tree was never written -- exactly the failure copy-on-
// write is supposed to make impossible.
int (*bflush)(void *ctx);
};
struct schfs_btree {
@ -93,9 +106,24 @@ struct schfs_cursor {
void *blk; // caller-owned, SCHFS_BLKSIZE bytes
int depth; // path[depth] is the leaf
int valid; // 0 once the walk has run off either end
// Which block `blk` currently holds, as far as this cursor knows, or 0 for
// "nothing yet". See cur_load: re-reading a block that is already in the
// buffer costs a 4 KB copy and a CRC32c over the whole 4 KB, and the code
// was paying both on every single access to the current record.
//
// Cleared by schfs_cursor_init rather than trusted across cursors, because
// the buffer belongs to the caller and the *next* caller's contents are not
// this cursor's business. Cross-checked against the block header on every
// use, so a second cursor sharing the same page cannot fool it either.
uint64_t resident;
uint64_t epoch; // schfs_tree_epoch when it was loaded
struct schfs_path_ent path[SCHFS_MAX_LEVEL + 1];
};
// Every mutation bumps this, retiring every cursor's cached block. See
// cur_load in btree.c.
extern uint64_t schfs_tree_epoch;
// `blk` must point at SCHFS_BLKSIZE bytes that outlive the cursor.
void schfs_cursor_init(struct schfs_cursor *c, struct schfs_btree *t, void *blk);

1031
kernel/fs/schfs/kmount.c
View file

File diff suppressed because it is too large Load diff

48
kernel/fs/schfs/obj.c
View file

@ -246,7 +246,9 @@ schfs_dir_lookup(struct schfs_vol *v, struct schfs_cursor *c,
}
struct iter_ctx {
int (*fn)(uint64_t, uint16_t, const char *, unsigned, void *);
// The hash-aware form. schfs_dir_iterate's plain callback is wrapped in one
// of these, so there is one walk rather than two.
schfs_dir_fn fn;
void *arg;
struct schfs_cursor *cur;
uint64_t dir;
@ -310,7 +312,7 @@ iter_item(const struct schfs_key *k, const void *val, unsigned len, void *arg)
uint16_t type = d->type;
memmove(name, (const char *)d + sizeof(*d), namelen);
int r = ic->fn(oid, type, name, namelen, ic->arg);
int r = ic->fn(oid, type, name, namelen, k->offset, ic->arg);
if (r) {
ic->stop = r;
return 1;
@ -320,13 +322,17 @@ iter_item(const struct schfs_key *k, const void *val, unsigned len, void *arg)
return 0;
}
// Walk a directory from a given point in its key space.
//
// `start_hash` is where to resume: entries are keyed (dir, DIRENT, hash), so
// seeking straight to a hash is what makes a partial listing continue in one
// descent instead of re-walking everything before it. See schfs_vfs_getdents.
int
schfs_dir_iterate(struct schfs_vol *v, struct schfs_cursor *c, uint64_t dir,
int (*fn)(uint64_t, uint16_t, const char *, unsigned, void *),
void *arg)
schfs_dir_iterate_from(struct schfs_vol *v, struct schfs_cursor *c, uint64_t dir,
uint64_t start_hash, schfs_dir_fn fn, void *arg)
{
struct schfs_key start;
schfs_mkkey(&start, dir, SCHFS_KEY_DIRENT, 0);
schfs_mkkey(&start, dir, SCHFS_KEY_DIRENT, start_hash);
struct iter_ctx ic;
ic.fn = fn; ic.arg = arg; ic.cur = c; ic.dir = dir; ic.stop = 0;
@ -338,6 +344,31 @@ schfs_dir_iterate(struct schfs_vol *v, struct schfs_cursor *c, uint64_t dir,
return ic.stop == 1 ? 0 : ic.stop;
}
// The plain form, for callers that want the whole directory and do not care
// where in the key space an entry sat.
struct plain_ctx {
int (*fn)(uint64_t, uint16_t, const char *, unsigned, void *);
void *arg;
};
static int
plain_entry(uint64_t ino, uint16_t type, const char *name, unsigned namelen,
uint64_t hash, void *arg)
{
struct plain_ctx *p = arg;
(void)hash;
return p->fn(ino, type, name, namelen, p->arg);
}
int
schfs_dir_iterate(struct schfs_vol *v, struct schfs_cursor *c, uint64_t dir,
int (*fn)(uint64_t, uint16_t, const char *, unsigned, void *),
void *arg)
{
struct plain_ctx p = { fn, arg };
return schfs_dir_iterate_from(v, c, dir, 0, plain_entry, &p);
}
/* ------------------------------------------------------------------ */
/* File data */
/* ------------------------------------------------------------------ */
@ -375,6 +406,11 @@ static struct {
void
schfs_cache_invalidate(void)
{
// Retires every cursor's resident block as well as the two caches below:
// an in-place rewrite leaves a stale copy that is indistinguishable from
// the real thing by inspection, so the only safe signal is "something
// changed". See cur_load in btree.c.
schfs_tree_epoch++;
for (unsigned i = 0; i < INO_CACHE_N; i++)
ino_cache[i].valid = 0;
ext_cache.valid = 0;

317
kernel/fs/schfs/objw.c
View file

@ -295,6 +295,135 @@ csum_del(struct schfs_vol *v, uint64_t dblk)
return r == -1 ? 0 : r; /* absent is not a failure */
}
/* ------------------------------------------------------------------ */
/* Extent references */
/* */
/* How many files point at a run of blocks. Only shared runs have a */
/* record; absence means one reference. See SCHFS_EXTREF_OBJECTID. */
/* ------------------------------------------------------------------ */
// Find the reference record *covering* `dblk`, which is not necessarily one
// keyed at it.
//
// This is the whole subtlety of the extent tree. A record is keyed by the block
// a run was allocated at, but a file's extent may point into the middle of that
// run -- splitting a shared extent around a written block leaves a tail whose
// first block is several into the run. Looking up by that block finds nothing,
// and "nothing" means "unshared", so the tail would be freed while another file
// still reached it. Every lookup therefore has to be a range query.
//
// Returns 1 and fills `out` (and `*runstart`) if a record covers dblk, 0 if
// none does -- meaning one reference -- or -2 on corruption.
static int
extref_find(struct schfs_vol *v, uint64_t dblk, uint64_t *runstart,
struct schfs_extref *out)
{
struct schfs_cursor c;
schfs_cursor_init(&c, &v->tree, v->ctx.a);
struct schfs_key probe;
schfs_mkkey(&probe, SCHFS_EXTREF_OBJECTID, SCHFS_KEY_EXTREF, dblk);
int r = schfs_btree_lookup_le(&c, &probe);
if (r == -2)
return -2;
if (r != 0)
return 0;
const struct schfs_key *got = schfs_cursor_key(&c);
unsigned vlen;
const struct schfs_extref *found =
(const struct schfs_extref *)schfs_cursor_val(&c, &vlen);
if (!got || !found)
return 0;
if (got->objectid != SCHFS_EXTREF_OBJECTID || got->type != SCHFS_KEY_EXTREF)
return 0;
if (vlen != sizeof(*out))
return -2;
if (dblk >= got->offset + found->nblocks)
return 0; /* the nearest record ends before dblk */
if (runstart) *runstart = got->offset;
*out = *found;
return 1;
}
static int
extref_put(struct schfs_vol *v, uint64_t dblk, const struct schfs_extref *e)
{
struct schfs_key k;
schfs_mkkey(&k, SCHFS_EXTREF_OBJECTID, SCHFS_KEY_EXTREF, dblk);
return schfs_btree_insert(&v->tree, &v->ctx, &k, e, sizeof(*e));
}
static int
extref_del(struct schfs_vol *v, uint64_t dblk)
{
struct schfs_key k;
schfs_mkkey(&k, SCHFS_EXTREF_OBJECTID, SCHFS_KEY_EXTREF, dblk);
int r = schfs_btree_delete(&v->tree, &v->ctx, &k);
return r == -1 ? 0 : r; /* absent is not a failure */
}
// One more file now points at the run covering `dblk`. `nblocks` is used only
// when no record exists yet, to record the run's length.
static int
extref_inc(struct schfs_vol *v, uint64_t dblk, uint64_t nblocks)
{
struct schfs_extref e;
uint64_t start = dblk;
int have = extref_find(v, dblk, &start, &e);
if (have == -2)
return -2;
if (have == 0) {
// First share: the run went from one holder to two, and this is where a
// record starts existing. `nblocks` is recorded now and never changed,
// because it is the length of the *run as allocated* -- see extref_dec.
e.nblocks = nblocks;
e.refs = 2;
} else {
e.refs++;
}
return extref_put(v, start, &e);
}
// One fewer holder. Returns 1 if the run is now unreferenced, filling *nblocks
// with the length to release; 0 if somebody still holds it; -2 on corruption.
//
// The record survives down to one holder rather than being deleted at that
// point, which is the subtle part. A file that shares a run may later be
// truncated inside it, leaving its extent record describing a *shorter* run
// than was allocated. If the record vanished at one reference, the last holder
// would free only as much as its own shortened record described and the tail
// would leak. Keeping the record keeps the original length, so the final
// release frees exactly what was allocated.
static int
extref_dec(struct schfs_vol *v, uint64_t dblk, uint64_t *runstart,
uint64_t *nblocks)
{
struct schfs_extref e;
uint64_t start = dblk;
int have = extref_find(v, dblk, &start, &e);
if (have == -2)
return -2;
if (have == 0) {
*runstart = dblk;
*nblocks = 0; /* no record: the caller's own view is it */
return 1;
}
*runstart = start;
if (e.refs <= 1) {
*nblocks = e.nblocks;
int r = extref_del(v, start);
return r == 0 ? 1 : r;
}
e.refs--;
*nblocks = e.nblocks;
int r = extref_put(v, start, &e);
return r == 0 ? 0 : r;
}
/* ------------------------------------------------------------------ */
/* Extents */
/* ------------------------------------------------------------------ */
@ -417,14 +546,31 @@ extent_grow(struct schfs_vol *v, uint64_t ino, uint64_t fblk,
}
// Release a run of data blocks and the checksum records that describe them.
// Shared runs are left alone: another file still points at them, and proper
// refcounting needs an extent tree that nothing creates yet.
//
// A run somebody else still points at is not released, only dropped: the
// reference count in the extent tree decides, because this file's own copy of
// `refs` cannot be decremented by the other holder.
//
// Note the asymmetry with a partial release. A file whose extent is being
// truncated in the middle releases only the tail, but the reference count is
// per *run*, not per block -- so a shared run is only ever dropped whole. That
// is why extent_release consults the count using the run's first block, and why
// a partial free of a shared run leaves the blocks alone: they are still
// reachable through the other file, in their entirety.
static int
extent_release(struct schfs_vol *v, const struct schfs_extent *e,
uint64_t first, uint64_t n)
{
if (e->refs != 1)
return 0;
struct schfs_extref xr;
uint64_t runstart = 0;
int shared = extref_find(v, e->disk_block, &runstart, &xr);
if (shared == -2)
return -2;
// Nothing has ever cloned this run, so this file is the only holder and may
// release exactly what it was asked to -- a whole run or the tail of one.
// This is the path every ordinary file takes.
if (shared == 0) {
for (uint64_t i = 0; i < n; i++) {
int r = csum_del(v, first + i);
if (r != 0)
@ -432,6 +578,117 @@ extent_release(struct schfs_vol *v, const struct schfs_extent *e,
v->dev.bfree(v->dev.ctx, first + i);
}
return 0;
}
// Shared. A partial release means this file is keeping the head of the run,
// so its record still points into it and the reference stands: the blocks
// stay, because the other holder reaches all of them anyway.
if (first != e->disk_block || n != e->nblocks)
return 0;
uint64_t runlen = 0;
int last = extref_dec(v, e->disk_block, &runstart, &runlen);
if (last == -2)
return -2;
if (!last)
return 0; /* somebody else still points at it */
// Freed from the run's own start and for its allocated length, not from
// this file's view of it -- which may be a slice several blocks in.
if (runlen == 0) {
runstart = e->disk_block;
runlen = e->nblocks;
}
for (uint64_t i = 0; i < runlen; i++) {
int r = csum_del(v, runstart + i);
if (r != 0)
return r;
v->dev.bfree(v->dev.ctx, runstart + i);
}
return 0;
}
static int extents_drop_from(struct schfs_vol *v, uint64_t ino,
struct schfs_inode *in, uint64_t from);
// Share one file's data with another, without copying any of it.
//
// The reflink. `dst` must be an existing object; whatever it held is released
// first. Afterwards both files name the same blocks, both are ordinary writable
// files, and the first write to either copies only the run it touches -- which
// is what copy-on-write already does for the tree and now does for data.
//
// The whole operation is metadata: one extent record per run in the source,
// duplicated, plus a reference record per run. Cloning a gigabyte costs the
// same as cloning a byte.
//
// Both sides' extent records are updated, not just the destination's. The
// per-record `refs` is the hint the write path consults before rewriting a run
// in place; leaving the source saying "mine alone" would let a later write to
// the source scribble over the clone.
int
schfs_file_reflink(struct schfs_vol *v, uint64_t src, uint64_t dst)
{
struct schfs_inode si, di;
struct schfs_cursor c;
if (src == dst)
return -1;
schfs_cache_invalidate();
schfs_cursor_init(&c, &v->tree, v->ctx.a);
if (schfs_inode_get(v, &c, src, &si) != 0)
return -1;
schfs_cursor_init(&c, &v->tree, v->ctx.a);
if (schfs_inode_get(v, &c, dst, &di) != 0)
return -1;
if (si.type != SCHFS_T_FILE || di.type != SCHFS_T_FILE)
return -1;
// The destination starts empty. Doing this first means a failure part way
// through leaves a short file rather than one holding a mixture.
if (extents_drop_from(v, dst, &di, 0) != 0)
return -2;
di.blocks = 0;
// Walk the source by file block. Not with a cursor held across the loop:
// every iteration inserts records, which rewrites the very nodes a cursor
// would be standing on.
uint64_t fblk = 0;
uint64_t nblk = (si.size + SCHFS_BLKSIZE - 1) / SCHFS_BLKSIZE;
while (fblk < nblk) {
uint64_t start;
struct schfs_extent e;
int m = extent_find(v, src, fblk, &start, &e);
if (m == -2)
return -2;
if (m == 0) {
fblk++; /* a hole: nothing to share */
continue;
}
int r = extref_inc(v, e.disk_block, e.nblocks);
if (r != 0)
return -2;
// Both sides now say "shared", so neither will rewrite the run in
// place. 2 rather than a real count: it is only ever compared with 1.
struct schfs_extent se = e;
se.refs = 2;
if (extent_put(v, src, start, &se) != 0)
return -2;
if (extent_put(v, dst, start, &se) != 0)
return -2;
di.blocks += e.nblocks;
fblk = start + e.nblocks;
}
di.size = si.size;
di.gen = v->tree.generation;
return schfs_inode_put(v, dst, &di);
}
// Drop everything from file block `from` onward. Used by truncate and by the
@ -616,11 +873,59 @@ block_for_write(struct schfs_vol *v, uint64_t ino, uint64_t fblk,
return r;
}
// The block just replaced rejoins the free map at the next commit, not now:
// until then the committed tree still points at it and a crash lands there.
// Account for what the split did to the old run.
//
// The reference count on a shared run is "how many file-extent records
// point into it", so a split that leaves a head *and* a tail has added a
// reference, not kept one. Getting this wrong is invisible until every
// holder is deleted and the run is either freed while still in use or
// never freed at all -- the second is what happened first, leaking four
// blocks per split.
{
int pieces = (fblk > start ? 1 : 0)
+ (fblk + 1 < start + e.nblocks ? 1 : 0);
struct schfs_extref xr;
uint64_t runstart = 0;
int shared = extref_find(v, e.disk_block, &runstart, &xr);
if (shared == -2)
return -2;
if (shared) {
if (pieces == 2) {
if (extref_inc(v, e.disk_block, e.nblocks) != 0)
return -2;
} else if (pieces == 0) {
// A one-block run, entirely replaced: this file's only
// reference to it is gone.
uint64_t runlen = 0;
int last = extref_dec(v, e.disk_block, &runstart, &runlen);
if (last == -2)
return -2;
if (last) {
if (runlen == 0) {
runstart = e.disk_block;
runlen = e.nblocks;
}
for (uint64_t i = 0; i < runlen; i++) {
int rr = csum_del(v, runstart + i);
if (rr != 0)
return rr;
v->dev.bfree(v->dev.ctx, runstart + i);
}
}
}
// The replaced block itself is never freed here: it is part of a
// run somebody else still reaches in full.
} else {
// The block just replaced rejoins the free map at the next commit,
// not now: until then the committed tree still points at it and a
// crash lands there.
int r = extent_release(v, &e, old, 1);
if (r != 0)
return r;
}
}
w->disk = nd;
w->src = old;

71
kernel/fs/schfs/schfs_int.h
View file

@ -75,6 +75,11 @@ struct schfs_vol {
int uber_slot; // ring slot the last commit used
int readonly;
// How many newer uberblocks schfs_vol_open had to reject before it found a
// root it could actually read. Zero on every healthy mount. Anything else
// means transactions were lost, which is a recovery the operator has to be
// told about rather than something to absorb quietly.
int recovered;
char label[SCHFS_LABEL_MAX];
uint8_t uuid[16];
@ -82,6 +87,14 @@ struct schfs_vol {
// ctx buffers: a split reuses ctx.b and ctx.c while the value being
// inserted is still being read from, so the value needs storage of its own.
char dirbuf[SCHFS_MAX_INLINE];
// Which attributes are indexed, as a packed list of (length, name), read
// off the disk at mount. Cached here rather than looked up because every
// attribute write asks the question and the answer is almost always no --
// a tree descent to say so would make every write pay for a feature it is
// not using. See kernel/fs/schfs/attr.c.
char idxnames[512];
unsigned idxnames_len;
};
// ---------------------------------------------------------------------------
@ -141,6 +154,12 @@ void schfs_vol_free(struct schfs_vol *v, uint64_t blk);
int schfs_space_init(struct schfs_space *s, uint64_t nblocks,
void *(*palloc)(void));
// Return an already-sized map to "empty volume", reusing its pages. Mounting
// may try several uberblocks, and each candidate root needs its scan to start
// from a clean map -- but the pages were taken once and must not be taken
// again, which is the one thing schfs_space_init cannot do twice.
void schfs_space_reset(struct schfs_space *s, uint64_t nblocks);
// Record a run as allocated. Idempotent, because an extent range may be shared
// by several files.
void schfs_space_mark(struct schfs_space *s, uint64_t blk, uint64_t n);
@ -187,6 +206,15 @@ int schfs_dir_lookup(struct schfs_vol *v, struct schfs_cursor *c,
// Iterate a directory's real entries in hash order. "." and ".." are not
// stored and are not reported; callers that need them synthesise them.
// A non-zero return from fn stops the walk and is passed back.
// A directory entry, with the tree key's hash it was found under. The hash is
// what a partial listing resumes from -- see schfs_dir_iterate_from.
typedef int (*schfs_dir_fn)(uint64_t ino, uint16_t type, const char *name,
unsigned namelen, uint64_t hash, void *arg);
int schfs_dir_iterate_from(struct schfs_vol *v, struct schfs_cursor *c,
uint64_t dir, uint64_t start_hash,
schfs_dir_fn fn, void *arg);
int schfs_dir_iterate(struct schfs_vol *v, struct schfs_cursor *c, uint64_t dir,
int (*fn)(uint64_t ino, uint16_t type, const char *name,
unsigned namelen, void *arg),
@ -244,6 +272,49 @@ int schfs_dir_empty(struct schfs_vol *v, uint64_t dir);
// schfs_file_write keeps the extent count down.
int schfs_file_reserve(struct schfs_vol *v, uint64_t ino, uint64_t nblocks);
// Share `src`'s data blocks with `dst` without copying them. Both remain
// ordinary writable files; the first write to either copies only the run it
// touches. Costs one metadata record per run, whatever the file's size.
int schfs_file_reflink(struct schfs_vol *v, uint64_t src, uint64_t dst);
// ---------------------------------------------------------------------------
// attr.c -- attributes, the index over them, and the query that uses it
// ---------------------------------------------------------------------------
// One attribute. Returns 1 and fills the buffers if present, 0 if absent, -3 if
// `out` is too small (the length is still reported), -2 on corruption.
int schfs_attr_get(struct schfs_vol *v, uint64_t ino, const char *name,
unsigned namelen, void *out, unsigned cap,
unsigned *valuelen, uint16_t *type);
// Every attribute on an object. The callback returns non-zero to stop.
int schfs_attr_iterate(struct schfs_vol *v, struct schfs_cursor *c, uint64_t ino,
int (*fn)(const char *name, unsigned namelen,
const void *val, unsigned vallen,
uint16_t type, void *arg),
void *arg);
// Set or replace an attribute, keeping any index in step.
int schfs_attr_set(struct schfs_vol *v, uint64_t ino, const char *name,
unsigned namelen, const void *val, unsigned vallen,
uint16_t type);
// Remove one. Absent is not a failure.
int schfs_attr_del(struct schfs_vol *v, uint64_t ino, const char *name,
unsigned namelen);
// Start indexing an attribute. Affects values written after this only.
int schfs_index_create(struct schfs_vol *v, const char *name, unsigned namelen);
// Load the indexed-attribute list from the tree into the volume. Called at
// mount, and after the list changes.
int schfs_index_reload(struct schfs_vol *v);
// Every object whose `name` attribute equals `val`. One tree descent, whatever
// the size of the filesystem. Returns how many ids were written, or negative.
int schfs_query_equal(struct schfs_vol *v, const char *name, unsigned namelen,
const void *val, unsigned vallen, uint64_t *out, int cap);
// Write at an arbitrary offset, allocating blocks, extent records and data
// checksums, copying any block that a committed tree can still reach, and
// splitting the extent record around it. Writing past the end extends the file;

68
kernel/fs/schfs/space.c
View file

@ -93,6 +93,39 @@ popcount64(uint64_t v)
/* Setup */
/* ------------------------------------------------------------------ */
// Bring an already-sized map back to "empty volume": every block free except
// the ones that are never allocatable. Split out of schfs_space_init because
// mounting may have to try more than one uberblock, and each candidate root
// needs a scan starting from a clean map -- while the pages themselves were
// taken once and must not be taken again.
void
schfs_space_reset(struct schfs_space *s, uint64_t nblocks)
{
for (int i = 0; i < s->npages; i++) {
memset(s->inuse[i], 0, SCHFS_BLKSIZE);
memset(s->pend[i], 0, SCHFS_BLKSIZE);
}
s->nblocks = nblocks;
s->free_blocks = nblocks - SCHFS_FIRST_BLOCK;
s->pending = 0;
s->hint = SCHFS_FIRST_BLOCK;
// Cleared, not preserved: schfs_space_scan sets RETAIN_ALL when it finds a
// snapshot, so carrying one candidate root's answer into the next root's
// scan would pin blocks for a snapshot that tree may not even have.
s->flags = 0;
// The boot block and the uberblock ring are never allocatable, and neither
// is anything past the end of the volume: the last bitmap page covers more
// blocks than exist, and a stray allocation out there would be written to a
// sector the device does not have.
for (uint64_t b = 0; b < SCHFS_FIRST_BLOCK; b++)
bit_set(s->inuse, b);
for (uint64_t b = nblocks;
b < (uint64_t)s->npages * SCHFS_SPACE_BITS_PER_PAGE; b++)
bit_set(s->inuse, b);
}
int
schfs_space_init(struct schfs_space *s, uint64_t nblocks,
void *(*palloc)(void))
@ -111,30 +144,16 @@ schfs_space_init(struct schfs_space *s, uint64_t nblocks,
void *p = palloc();
if (p == 0)
return -5; /* caller frees what was taken */
memset(p, 0, SCHFS_BLKSIZE);
s->inuse[i] = (uint64_t *)p;
p = palloc();
if (p == 0)
return -5;
memset(p, 0, SCHFS_BLKSIZE);
s->pend[i] = (uint64_t *)p;
s->npages = (int)(i + 1);
}
s->nblocks = nblocks;
s->free_blocks = nblocks - SCHFS_FIRST_BLOCK;
s->hint = SCHFS_FIRST_BLOCK;
// The boot block and the uberblock ring are never allocatable, and neither
// is anything past the end of the volume: the last bitmap page covers more
// blocks than exist, and a stray allocation out there would be written to a
// sector the device does not have.
for (uint64_t b = 0; b < SCHFS_FIRST_BLOCK; b++)
bit_set(s->inuse, b);
for (uint64_t b = nblocks; b < (uint64_t)npages * SCHFS_SPACE_BITS_PER_PAGE; b++)
bit_set(s->inuse, b);
schfs_space_reset(s, nblocks);
return 0;
}
@ -276,6 +295,25 @@ mark_leaf_extents(struct schfs_space *s, const void *blk)
(const struct schfs_extent *)((const char *)blk + it[i].offset);
schfs_space_mark(s, e->disk_block, e->nblocks);
}
// Shared runs, at the length they were *allocated*.
//
// Not redundant with the loop above, and the difference matters. A file
// that shares a run may since have been truncated inside it, so its extent
// record describes a shorter run than exists -- and the tail would be
// derived as free while the reference record still holds it. The allocator
// would then hand out a block that a later release frees a second time.
for (int i = 0; i < n; i++) {
if (it[i].key.type != SCHFS_KEY_EXTREF)
continue;
if (it[i].size != sizeof(struct schfs_extref))
continue;
if ((int)it[i].offset + (int)it[i].size > SCHFS_BLKSIZE)
continue;
const struct schfs_extref *x =
(const struct schfs_extref *)((const char *)blk + it[i].offset);
schfs_space_mark(s, it[i].key.offset, x->nblocks);
}
}
// Snapshot roots found while walking, so they can be walked in turn.

207
kernel/fs/schfs/vfs.c
View file

@ -39,6 +39,7 @@
#include "buf.h"
#include "file.h"
#include "errno.h"
#include "fsstat.h"
#include "schfs_int.h"
@ -91,7 +92,10 @@ schfs_vfs_iload(struct inode *ip)
ip->type = type_to_vfs(in.type);
ip->major = (short)in.rdev_major;
ip->minor = (short)in.rdev_minor;
ip->nlink = (short)in.nlink;
// Clamped, not cast. The on-disk count is a uint32 and the VFS's is a signed
// int, so the top half of the range has nowhere to go; saturating keeps it
// positive, and a file with two billion names is not going to lose one here.
ip->nlink = in.nlink > 0x7fffffffu ? 0x7fffffff : (int)in.nlink;
ip->size = in.size;
ip->rwmode = RW;
ip->supermode = 0;
@ -123,7 +127,7 @@ schfs_vfs_istore(struct inode *ip)
// Read-modify-write rather than build-from-scratch: the on-disk record holds
// fields the VFS has no slot for (uid, gid, the real mode bits, the
// generation), and rebuilding it here would quietly zero every one of them.
if (schfs_kstat(ip->inum, &in) != 0) {
if (schfs_kstat_dev(ip->dev, ip->inum, &in) != 0) {
memset(&in, 0, sizeof(in));
in.mode = ip->type == T_DIR ? 0755 : 0644;
}
@ -137,7 +141,7 @@ schfs_vfs_istore(struct inode *ip)
in.mtime = ip->mtime;
in.ctime = ip->ctime;
schfs_ksetattr(ip->inum, &in);
schfs_ksetattr(ip->dev, ip->inum, &in);
}
static void
@ -145,7 +149,7 @@ schfs_vfs_ifree(struct inode *ip)
{
if (readonly(ip))
return;
schfs_kfree(ip->inum);
schfs_kfree(ip->dev, ip->inum);
}
/* ------------------------------------------------------------------ */
@ -173,6 +177,9 @@ schfs_vfs_ifree(struct inode *ip)
static int
schfs_vfs_read(struct inode *ip, int user_dst, uint64 dst, uint64 off, uint n)
{
uint64 t0 = rdtsc();
fsstat.read_calls++;
char *tmp = kalloc();
if (tmp == 0)
return -1;
@ -199,6 +206,8 @@ schfs_vfs_read(struct inode *ip, int user_dst, uint64 dst, uint64 off, uint n)
}
kfree(tmp);
fsstat.read_bytes += done;
fsstat.read_cycles += rdtsc() - t0;
return (int)done;
}
@ -212,6 +221,22 @@ schfs_vfs_write(struct inode *ip, int user_src, uint64 src, uint64 off, uint n)
if (tmp == 0)
return -1;
// One transaction for the whole call, not one per 4 KB chunk.
//
// Each schfs_kwrite is its own group otherwise, and a group ends in two cache
// flushes and an uberblock write. A flush is a barrier, so committing per
// chunk forbids the buffer cache to coalesce the tree writes that every chunk
// repeats — the same leaf, holding this file's data checksums, went to the
// disk once per chunk. Measured over 1 MB: 1410 block writes per chunk-commit
// against 267 for one group. It is also better semantics, since a short write
// now leaves the file at a length some prefix of this call actually reached
// rather than at whichever chunk happened to commit last.
//
// copyin under the bracket is safe: it walks the page table and returns -1 on
// an unmapped address rather than faulting, and a page fault here would be a
// SIGSEGV in any case — there is no demand paging to re-enter the filesystem.
schfs_ktx_open(ip->dev);
uint done = 0;
while (done < n) {
uint want = n - done;
@ -225,13 +250,19 @@ schfs_vfs_write(struct inode *ip, int user_src, uint64 src, uint64 off, uint n)
memmove(tmp, (char *)(src + done), want);
}
long put = schfs_kwrite(ip->inum, off + done, tmp, want);
long put = schfs_kwrite(ip->dev, ip->inum, off + done, tmp, want);
if (put <= 0) break;
done += (uint)put;
if ((uint)put < want) break;
}
int cr = schfs_ktx_close(ip->dev);
kfree(tmp);
// A failed commit means nothing durably happened, so report it that way
// rather than claiming bytes that are not on the disk.
if (cr != 0)
return -1;
if (off + done > ip->size)
ip->size = off + done;
return (int)done;
@ -242,7 +273,7 @@ schfs_vfs_truncate(struct inode *ip, uint64 newsize)
{
if (readonly(ip))
return;
if (schfs_ktruncate(ip->inum, newsize) == 0)
if (schfs_ktruncate(ip->dev, ip->inum, newsize) == 0)
ip->size = newsize;
}
@ -297,7 +328,7 @@ schfs_vfs_dirlink(struct inode *dp, char *name, uint64 inum)
if (schfs_kstat_dev(dp->dev, inum, &in) != 0)
return -1;
int r = schfs_kdiradd(dp->inum, name, namelen, inum, in.type);
int r = schfs_kdiradd(dp->dev, dp->inum, name, namelen, inum, in.type);
if (r != 0)
return -1;
@ -306,7 +337,7 @@ schfs_vfs_dirlink(struct inode *dp, char *name, uint64 inum)
// parent -- and every ".." walk out of it lands somewhere else entirely.
if (in.type == SCHFS_T_DIR && in.parent != dp->inum) {
in.parent = dp->inum;
schfs_ksetattr(inum, &in);
schfs_ksetattr(dp->dev, inum, &in);
}
return 0;
}
@ -320,21 +351,21 @@ schfs_vfs_dirunlink(struct inode *dp, char *name, uint off)
(void)off;
unsigned namelen = 0;
while (name[namelen]) namelen++;
return schfs_kunlink(dp->inum, name, namelen) == 0 ? 0 : -1;
return schfs_kunlink(dp->dev, dp->inum, name, namelen) == 0 ? 0 : -1;
}
static int
schfs_vfs_dirempty(struct inode *dp)
{
return schfs_kdirempty(dp->inum);
return schfs_kdirempty(dp->dev, dp->inum);
}
static struct inode *
schfs_vfs_ialloc(uint dev, short type)
{
uint64 ino = 0;
if (schfs_kalloc_object(type_from_vfs(type), type == T_DIR ? 0755 : 0644,
&ino) != 0)
if (schfs_kalloc_object(dev, type_from_vfs(type),
type == T_DIR ? 0755 : 0644, &ino) != 0)
return 0;
return iget(dev, ino);
}
@ -345,44 +376,60 @@ schfs_vfs_ialloc(uint dev, short type)
// One record's worth of state for the callback below.
//
// Entries come back in name-hash order, which has nothing to do with their
// names -- POSIX promises no order, and hash order is what makes a lookup one
// tree descent instead of a scan. The position handed back to userland is
// therefore an *index*: entry N of the walk. Resuming means walking to N again,
// which readdir already does linearly, so a full listing stays linear overall
// only because the common case is resuming exactly where the last call stopped.
// ---------------------------------------------------------------------------
// The resume cookie, and why it is a key rather than a count
//
// getdents is called repeatedly until it returns nothing, and each call has to
// carry on where the last stopped. `*poff` is the only state there is, so
// whatever goes in it must be enough to find that place again.
//
// It used to be an ordinal -- "entry N of the walk" -- and resuming meant
// walking from the first entry and skipping N. That is quadratic in the size of
// the directory, and measurably so: listing /bin, 243 entries, walked 914
// entries across five calls, because each call started over. A larger directory
// gets worse faster than it gets bigger.
//
// It is now the tree key to seek to. Entries are keyed (dir, DIRENT, hash), so
// the cookie is a hash and resuming is one descent to that hash. The three
// values below `SCHFS_DIRENT_FIRST` are reserved for the synthesised "." and
// "..", which are not records and would otherwise have no position at all.
//
// One consequence worth stating: a whole hash bucket is emitted or none of it
// is. Buckets hold colliding names, at most SCHFS_MAX_INLINE packed, which is
// under 3 KB of output records -- so any caller with a 4 KB buffer never
// notices. A caller whose buffer cannot hold one bucket gets EINVAL, which is
// what Linux does when a buffer cannot hold a single record.
#define GD_DOT 0 /* emit "." next */
#define GD_DOTDOT 1 /* emit ".." next */
#define GD_ENTRIES 2 /* emit real entries, from hash (cookie - 2) */
struct gd_ctx {
char *out;
int cap;
int written;
uint64 skip; // entries to step over before emitting
uint64 index; // entries seen so far
int full;
uint64 from; // hash to resume at
uint64 next; // hash the *following* call should resume at
int full; // buffer filled; stop the walk
int toobig; // a single bucket did not fit in an empty buffer
uint64 bucket; // hash of the bucket being emitted
int bucket_started;
};
static int
gd_entry(uint64 ino, unsigned short type, const char *name, unsigned namelen,
void *arg)
gd_emit(struct gd_ctx *g, uint64 ino, unsigned short type,
const char *name, unsigned namelen)
{
struct gd_ctx *g = arg;
if (g->index++ < g->skip)
return 0;
if (g->full)
return 1;
if (namelen > 255)
namelen = 255;
int reclen = (8 + 8 + 2 + 1 + (int)namelen + 1 + 7) & ~7;
if (g->written + reclen > g->cap) {
g->full = 1;
return 1; /* stop; the caller will come back */
}
if (g->written + reclen > g->cap)
return -1;
char *k = g->out + g->written;
uint64 *p64 = (uint64 *)k;
p64[0] = ino;
p64[1] = g->index; /* resume position: entries consumed */
p64[1] = g->next;
*(uint16 *)(k + 16) = (uint16)reclen;
*(uint8 *)(k + 18) = type == SCHFS_T_DIR ? 4 /* DT_DIR */
: type == SCHFS_T_DEV ? 2 /* DT_CHR */
@ -391,44 +438,98 @@ gd_entry(uint64 ino, unsigned short type, const char *name, unsigned namelen,
memmove(k + 19, name, namelen);
memset(k + 19 + namelen, 0, reclen - 19 - namelen);
g->written += reclen;
fsstat.getdents_walked++; // entries emitted; no longer entries re-walked
return 0;
}
static int
gd_entry(uint64 ino, unsigned short type, const char *name, unsigned namelen,
uint64 hash, void *arg)
{
struct gd_ctx *g = arg;
if (g->full)
return 1;
// A new bucket. Everything emitted so far is safe to keep, because the
// cookie already points past the previous bucket.
if (!g->bucket_started || hash != g->bucket) {
g->bucket = hash;
g->bucket_started = 1;
g->next = hash + GD_ENTRIES; // resume *at* this bucket if we stop here
}
// The cookie the emitted records carry has to be "past this bucket", or a
// caller that stops reading mid-buffer would come back and get the bucket
// again. Setting it before the emit is what makes that true.
uint64 saved = g->next;
g->next = hash + 1 + GD_ENTRIES;
if (gd_emit(g, ino, type, name, namelen) < 0) {
// Did not fit. Roll back to the start of this bucket so the whole thing is
// re-read next time, and stop.
g->next = saved;
g->full = 1;
if (g->written == 0)
g->toobig = 1;
return 1;
}
return 0;
}
static int
schfs_vfs_getdents(struct inode *dp, char *out, int cap, uint64 *poff)
{
uint64 t0 = rdtsc();
fsstat.getdents_calls++;
struct gd_ctx g;
g.out = out;
g.cap = cap;
g.written = 0;
g.skip = *poff;
g.index = 0;
g.from = *poff;
g.next = *poff;
g.full = 0;
g.toobig = 0;
g.bucket = 0;
g.bucket_started = 0;
// "." and ".." are synthesised: they are not records, so a walk would never
// produce them, and every readdir caller expects both.
if (g.from <= GD_DOTDOT) {
static const char *dots[2] = { ".", ".." };
for (int i = 0; i < 2; i++) {
if (g.index++ < g.skip)
continue;
int reclen = (8 + 8 + 2 + 1 + (i + 1) + 1 + 7) & ~7;
if (g.written + reclen > cap)
for (int i = (int)g.from; i < 2; i++) {
g.next = (uint64)i + 1;
if (gd_emit(&g, dp->inum, SCHFS_T_DIR, dots[i], (unsigned)(i + 1)) < 0) {
if (g.written == 0)
return -EINVAL;
*poff = g.next;
return g.written;
char *k = out + g.written;
uint64 *p64 = (uint64 *)k;
p64[0] = dp->inum; /* close enough for ".."; nothing reads it */
p64[1] = g.index;
*(uint16 *)(k + 16) = (uint16)reclen;
*(uint8 *)(k + 18) = 4; /* DT_DIR */
memmove(k + 19, dots[i], i + 1);
memset(k + 19 + i + 1, 0, reclen - 19 - i - 1);
g.written += reclen;
}
}
g.next = GD_ENTRIES; // real entries start at hash 0
g.from = GD_ENTRIES;
}
if (schfs_kreaddir_dev(dp->dev, dp->inum, gd_entry, &g) != 0 && g.written == 0)
uint64 start_hash = g.from - GD_ENTRIES;
if (schfs_kreaddir_from_dev(dp->dev, dp->inum, start_hash, gd_entry, &g) != 0
&& g.written == 0)
return -1;
*poff = g.index;
if (g.toobig)
return -EINVAL;
// Nothing emitted and the walk ran to the end: this is the end of the
// directory, and the cookie must not move or the caller loops.
if (!g.full && g.written == 0) {
*poff = g.next;
fsstat.getdents_cycles += rdtsc() - t0;
return 0;
}
*poff = g.next;
fsstat.getdents_cycles += rdtsc() - t0;
return g.written;
}

113
kernel/fs/schfs/vol.c
View file

@ -85,70 +85,118 @@ schfs_vol_open(struct schfs_vol *v, struct schfs_bdev *dev,
struct schfs_ctx *x, void *(*palloc)(void))
{
struct schfs_uberblock *u = (struct schfs_uberblock *)x->a;
int best_slot = -1;
uint64_t best_gen = 0;
int io_errors = 0;
int seen = 0; // valid uberblocks found, at any generation
uint64_t ceiling = ~0ULL; // consider only generations below this
int spaced = 0; // the allocation map is built and owned
v->dev = *dev;
v->ctx = *x;
v->tree.dev = &v->dev;
v->recovered = 0;
// Remember which slot won, not a copy of it.
// Walk the ring newest-first, and keep walking.
//
// An uberblock is a whole SCHFS_BLKSIZE block, and keeping one as a local
// put 4096 bytes on a kernel stack that is 4096 bytes in total — the canary
// in kalloc'd stacks caught it as "kstack overflow" on the very first boot.
// Re-reading the winner costs one block read, once, at mount.
// This used to take the highest valid generation, check its tree, and give
// up if that failed. But an uberblock passing its own checksum only proves
// the uberblock landed -- not the tree it names. That is exactly the state
// a power cut used to leave behind before commit had a barrier, and it is
// still what a misdirected write or a bad sector under the root produces.
//
// A ring of SCHFS_UBER_COUNT slots exists precisely so there is somewhere
// to fall back *to*. Refusing the mount while 127 older, entirely intact
// roots sat on the disk was throwing away the only thing the ring is for.
for (;;) {
int best_slot = -1;
uint64_t best_gen = 0;
for (int i = 0; i < SCHFS_UBER_COUNT; i++) {
if (v->dev.bread(v->dev.ctx, SCHFS_UBER_START + i, u) != 0) {
io_errors++;
if (ceiling == ~0ULL)
io_errors++; // count each slot once, on the first sweep
continue; // an unreadable slot is not fatal: try the rest
}
if (!uber_valid(u))
continue;
if (u->generation >= ceiling)
continue; // already tried, and rejected
if (best_slot < 0 || u->generation > best_gen) {
best_gen = u->generation;
best_slot = i;
}
}
if (ceiling == ~0ULL)
seen = best_slot >= 0;
if (best_slot < 0)
return io_errors == SCHFS_UBER_COUNT ? -2 : -1;
break; // nothing older left to try
ceiling = best_gen; // next round considers strictly older ones
// Re-read the winner rather than keeping a copy of it.
//
// An uberblock is a whole SCHFS_BLKSIZE block, and keeping one as a
// local put 4096 bytes on a kernel stack that is 4096 bytes in total --
// the canary in kalloc'd stacks caught it as "kstack overflow" on the
// very first boot. Re-reading costs one block read per candidate.
if (v->dev.bread(v->dev.ctx, SCHFS_UBER_START + best_slot, u) != 0)
return -2;
goto next;
if (!uber_valid(u))
return -2; /* changed under us, or a flaky device */
goto next; /* changed under us, or a flaky device */
vol_from_uber(v, u, best_slot);
v->readonly = 1; // until an allocation map exists, below
// The newest uberblock passing its own checksum does not prove the tree it
// names is readable — a misdirected write could have landed on the root.
// Check now, while there is still the option of falling back, rather than
// discovering it on the first lookup.
if (v->dev.bread(v->dev.ctx, v->tree.root, x->a) != 0)
return -2;
goto next;
{
struct schfs_node_hdr *h = (struct schfs_node_hdr *)x->a;
if (h->magic != SCHFS_NODE_MAGIC || h->blocknr != v->tree.root)
return -2;
goto next;
}
// Rebuild the allocation map from the tree just adopted. Without a palloc
// the volume stays read-only, which is exactly right for a tool that only
// inspects an image: it can then never allocate, rather than allocating
// from a map that says everything is free.
// Rebuild the allocation map from the tree just adopted. Without a
// palloc the volume stays read-only, which is exactly right for a tool
// that only inspects an image: it can then never allocate, rather than
// allocating from a map that says everything is free.
//
// The scan is also the deepest check this root gets: it visits every
// metadata block and verifies each against the checksum in its parent.
// A root that survives it is not merely present, it is whole -- so a
// scan failure is a reason to try the next candidate, not to give up.
if (palloc) {
if (!spaced) {
// Sizing failures are the caller's problem, not this root's:
// a volume too large to map or a machine out of pages will
// fail identically on every slot in the ring.
int sr = schfs_space_init(&v->space, v->total_blocks, palloc);
if (sr != 0)
return sr;
spaced = 1;
} else {
schfs_space_reset(&v->space, v->total_blocks);
}
if (schfs_space_scan(v) != 0)
return -2;
goto next;
v->free_blocks = v->space.free_blocks;
v->readonly = 0;
}
// Which attributes are indexed, so attr.c can answer without a lookup
// on every write. A volume with no indexes -- every volume until
// somebody asks for one -- ends up with an empty list.
schfs_index_reload(v);
return 0;
next:
v->recovered++; // this candidate was newer, and unusable
continue;
}
// Nothing usable. Distinguish "not a SchFS volume" from "a SchFS volume
// that cannot be read", because the first is an ordinary answer to a probe
// and the second is a damaged disk somebody needs to hear about.
if (io_errors == SCHFS_UBER_COUNT)
return -2;
return seen ? -2 : -1;
}
int
@ -181,9 +229,30 @@ schfs_vol_commit(struct schfs_vol *v)
memmove(u->label, v->label, sizeof(u->label));
u->crc = uber_crc(u);
// The barrier the whole design rests on. Every block the new root reaches
// has been handed to bwrite by now, but "handed to bwrite" is not "on the
// medium": the kernel defers tree writes into the buffer cache, and the
// drive under it has a write cache of its own. Both are free to reorder.
//
// So: drain everything, *then* write the block that makes it live. A crash
// before this point leaves the previous uberblock newest and nothing is
// lost; a crash after it finds a tree that is fully present. Without the
// barrier there is a third outcome -- a newest uberblock naming a tree that
// never landed -- and it is unrecoverable, because the checksum on the
// uberblock is perfectly valid.
if (v->dev.bflush && v->dev.bflush(v->dev.ctx) != 0)
return -2;
if (v->dev.bwrite(v->dev.ctx, SCHFS_UBER_START + v->uber_slot, u) != 0)
return -2;
// And the uberblock itself, before this function claims the transaction is
// durable. Everything below acts on that claim: the blocks this group freed
// are handed back to the allocator, which is only safe once the group that
// still referenced them can no longer be what a crash comes back to.
if (v->dev.bflush && v->dev.bflush(v->dev.ctx) != 0)
return -2;
// Order matters here, and it is the whole crash-safety argument in three
// lines. Only now that the new uberblock is on the disk is the group this
// freed blocks in superseded, so only now may those blocks be handed out

2
kernel/proc/proc.c
View file

@ -493,7 +493,7 @@ forkret(void) {
// hang and is not one.
{
struct inode *stale = myproc()->cwd;
myproc()->cwd = iget(SCHFS_DEV, schfs_kroot());
myproc()->cwd = iget(SCHFS_DEV, schfs_kroot(SCHFS_DEV));
if (stale)
iput(stale);
}

335
kernel/sys/sys_fs.c
View file

@ -14,6 +14,7 @@
#include "syscall.h"
#include "uaccess.h"
#include "errno.h"
#include "schfs.h" /* SCHFS_BLKSIZE, SCHFS_NAME_MAX — statfs reports them */
// -----------------------------------------------------------------------
// Linux-ABI-compatible struct stat (x86_64 "struct stat" from
@ -965,6 +966,44 @@ addr_t sys_ioctl(void)
argaddr(1, &request); argaddr(2, &argp);
int is_dev = (f->type == FD_INODE && f->ip && f->ip->type == 3);
// FICLONE: make this file share another's data blocks. Linux's spelling,
// because `cp --reflink` and every other tool that knows about cloning
// already asks for it by this number -- inventing our own would mean
// nothing could use the feature without being patched.
//
// The argument is a *file descriptor*, not a pointer, which is why it is
// read before the argp-shaped handling below.
if (request == FICLONE) {
struct file *sf;
struct proc *pr = myproc();
int sfd = (int)argp;
if (f->type != FD_INODE || !f->ip || !f->writable)
return -EBADF;
if (sfd < 0 || sfd >= NOFILE || (sf = pr->ofile[sfd]) == 0)
return -EBADF;
if (sf->type != FD_INODE || !sf->ip || !sf->readable)
return -EBADF;
// Both sides must be on the same filesystem: a clone shares blocks, and
// blocks belong to one volume.
if (sf->ip->dev != f->ip->dev)
return -EXDEV;
if (!SCHFS_IS_VOL_DEV(f->ip->dev))
return -EOPNOTSUPP;
if (f->ip->type != T_FILE || sf->ip->type != T_FILE)
return -EINVAL;
int r = schfs_kreflink(f->ip->dev, sf->ip->inum, f->ip->inum);
if (r != 0)
return -EIO;
// The destination inode's size changed underneath the VFS's copy.
ilock(f->ip);
f->ip->size = sf->ip->size;
iunlock(f->ip);
return 0;
}
// /dev/fb0 (major 2) — geometry queries. This has to come before the
// catch-all below, which answers every non-tty ioctl with 0: a program
// asking FBIOGET_VSCREENINFO would have been told "success" and then
@ -1055,9 +1094,36 @@ addr_t sys_rmdir(void)
}
// fsync / fdatasync / sync — stubs (log-based FS commits on every op).
addr_t sys_fsync(void) { return 0; }
addr_t sys_fdatasync(void){ return 0; }
addr_t sys_sync(void) { return 0; }
// sync / fsync / fdatasync.
//
// These returned 0 without doing anything, which was defensible while every
// write reached the disk before its syscall returned. It is not defensible now:
// tree blocks are marked dirty in the buffer cache and pushed out at commit, so
// "flush what is outstanding" finally names a real action. An empty transaction
// drives the same barrier a real one does -- the dirty buffers go out and the
// device is told to commit them.
addr_t sys_sync(void)
{
schfs_ksync_all();
return 0;
}
addr_t sys_fsync(void)
{
struct file *f;
if (argfd(0, 0, &f) < 0) return -EBADF;
// Whole-volume rather than per-file: SchFS has no way to make one object
// durable on its own. A transaction group is the unit that commits, and
// this file's last write is in whichever group is open.
if (f->type == FD_INODE && f->ip && SCHFS_IS_VOL_DEV(f->ip->dev))
return schfs_ksync(f->ip->dev) == 0 ? 0 : -EIO;
return 0;
}
// No separate data-only path: the inode record and the data it describes are
// records in the same tree and commit together. Reporting success for a weaker
// guarantee than was asked for would be the wrong way round; this is stronger.
addr_t sys_fdatasync(void){ return sys_fsync(); }
// pread: read from fd at given offset without changing file position.
addr_t sys_pread(void)
@ -1106,16 +1172,43 @@ addr_t sys_pwritev2(void){ return sys_writev(); }
// fallocate: preallocate space — we don't track this; stub success.
addr_t sys_fallocate(void) { return 0; }
// statfs / fstatfs: return fake filesystem stats.
static void fill_statfs(struct statfs *sf) {
// statfs / fstatfs.
//
// These used to return the same six invented constants whatever they were
// asked about -- 150000 blocks, half of them free -- so df reported a
// filesystem that did not exist and reported it identically for every mount.
// The numbers are real now, and which ones you get depends on which filesystem
// the path landed on.
static void fill_statfs(struct statfs *sf, uint dev) {
uint64 total = 0, freeb = 0, objects = 0;
memset(sf, 0, sizeof(*sf));
sf->f_type = 0x534b4658; // 'SKFX'
sf->f_bsize = 512;
sf->f_blocks = 150000;
sf->f_bfree = 75000;
sf->f_bavail = 75000;
if (schfs_kstatfs(dev, &total, &freeb, &objects) == 0) {
sf->f_bsize = SCHFS_BLKSIZE;
sf->f_frsize = SCHFS_BLKSIZE;
sf->f_blocks = (int64)total;
sf->f_bfree = (int64)freeb;
sf->f_bavail = (int64)freeb; // no reserve, so nothing held back
// Both zero, deliberately. There is no inode table to run out of and
// the object id space is 2^64, so there is no honest pair of numbers
// here -- and df computes a "% inodes used" from them, which against a
// real f_files and a zero f_ffree would show every SchFS mount as
// completely full. Zeroes are what filesystems that allocate inodes
// dynamically report, and what df is written to expect from them.
sf->f_files = 0;
sf->f_ffree = 0;
sf->f_namelen = SCHFS_NAME_MAX;
(void)objects;
return;
}
// v1, /proc, or anything else: the old shape, but at least with a block
// size that matches the one the buffer cache actually uses.
sf->f_bsize = BSIZE;
sf->f_frsize = BSIZE;
sf->f_namelen = DIRSIZ;
sf->f_frsize = 512;
}
addr_t sys_statfs(void)
@ -1125,7 +1218,17 @@ addr_t sys_statfs(void)
if (argstr(0, &path) < 0 || argaddr(1, &bufp) < 0) return -1;
struct statfs *sf = (struct statfs*)user_ptr(bufp, sizeof(*sf));
if (!sf) return -EFAULT;
fill_statfs(sf);
// Resolve the path: which filesystem it lands on is the whole question,
// and mount crossing happens during resolution.
begin_op();
struct inode *ip = namei(path);
if (!ip) { end_op(); return -ENOENT; }
uint dev = ip->dev;
iput(ip);
end_op();
fill_statfs(sf, dev);
return 0;
}
@ -1136,7 +1239,8 @@ addr_t sys_fstatfs(void)
if (argfd(0, 0, &f) < 0 || argaddr(1, &bufp) < 0) return -1;
struct statfs *sf = (struct statfs*)user_ptr(bufp, sizeof(*sf));
if (!sf) return -EFAULT;
fill_statfs(sf);
if (f->type != FD_INODE || !f->ip) return -EBADF;
fill_statfs(sf, f->ip->dev);
return 0;
}
@ -1404,9 +1508,8 @@ addr_t sys_mount(void)
return 0;
}
// Only SchFS, and only the one volume the kernel found at boot. Checking
// the name still matters: a caller that asked for something else should be
// told no rather than quietly handed this.
// Only SchFS. Checking the name still matters: a caller that asked for
// something else should be told no rather than quietly handed this.
if (strncmp(fstype, "schfs", 6) != 0)
return -ENODEV;
@ -1429,7 +1532,11 @@ addr_t sys_mount(void)
// Distinct codes, because "resource busy" for every possible failure
// tells the caller nothing and cost an hour of guessing once already.
int r = schfs_ksnap_mount(nm, nlen);
//
// Snapshots are taken of the root volume: a snapshot is named inside
// the volume that holds it, and there is no syntax yet for saying which
// volume that is.
int r = schfs_ksnap_mount(SCHFS_DEV, nm, nlen);
if (r < 0) {
switch (r) {
case -1: return -ENODEV; /* no volume mounted to look in */
@ -1444,7 +1551,41 @@ addr_t sys_mount(void)
if (!schfs_kmounted())
return -ENODEV;
uint64 root = snapmount ? schfs_ksnap_root(snapdev) : schfs_kroot();
// The source finally names a disk.
//
// It was ignored outright before -- there was one volume and nothing to
// choose between -- so `mount -t schfs /dev/hdb /mnt` mounted whatever the
// kernel had probed at boot and said nothing about it. Now an explicit
// source is resolved, and mounting a second SchFS disk works.
//
// An empty source, or the bare word "schfs", still means the root volume:
// that is what /etc/fstab and every existing script pass.
uint fsdev_src = SCHFS_DEV;
if (!snapmount && source && source[0] && strncmp(source, "schfs", 6) != 0) {
begin_op();
struct inode *sp = namei(source);
if (!sp) { end_op(); return -ENOENT; }
ilock(sp);
// A raw disk is a character device in the devsw table (major 8, see
// kernel/dev/cdev.c) whose minor is the IDE device number.
if (sp->type != T_DEV || sp->major != 8) {
iunlockput(sp); end_op();
return -ENOTBLK;
}
uint bdev = (uint)sp->minor;
iunlockput(sp);
end_op();
int r = schfs_kmount_dev(bdev);
if (r < 0) {
// -EBUSY also covers "already mounted", which is the common typo
// and deserves to be distinguishable from "no free slot" one day.
return r;
}
fsdev_src = (uint)r;
}
uint64 root = snapmount ? schfs_ksnap_root(snapdev) : schfs_kroot(fsdev_src);
if (root == 0) {
if (snapmount)
schfs_ksnap_unmount(snapdev);
@ -1469,12 +1610,14 @@ addr_t sys_mount(void)
// the only volume there could be. A snapshot is a second device, and its
// mount point is necessarily on the live root -- there is nowhere else to
// put it -- so the old spelling refused every snapshot mount.
uint fsdev = snapmount ? snapdev : SCHFS_DEV;
uint fsdev = snapmount ? snapdev : fsdev_src;
if (dev == fsdev)
return -EBUSY;
if (mount_add(dev, inum, fsdev, root) != 0) {
if (mount_add(dev, inum, fsdev, root, target, source, "schfs") != 0) {
if (snapmount)
schfs_ksnap_unmount(snapdev);
else if (fsdev_src != SCHFS_DEV)
schfs_kumount(fsdev_src); /* undo the volume mount we just did */
return -EBUSY;
}
return 0;
@ -1499,10 +1642,158 @@ addr_t sys_umount2(void)
if (mount_del_by_root(dev, inum) != 0)
return -EINVAL;
// Release the snapshot slot with the mount. Without this the table entry
// went away and the slot stayed taken, so a volume could be unmounted and
// never mounted again until reboot.
// Release the slot with the mount. Without this the table entry went away
// and the slot stayed taken, so a volume could be unmounted and never
// mounted again until reboot.
if (SCHFS_IS_SNAP_DEV(dev))
schfs_ksnap_unmount(dev);
else if (SCHFS_IS_VOL_DEV(dev) && dev != SCHFS_DEV)
// Pushes the volume's dirty buffers out and drives a cache flush before
// letting go. The root volume (SCHFS_DEV) refuses -- there is nothing
// to fall back to -- and mount_del_by_root has already failed for it,
// since nothing mounts the root over anything.
schfs_kumount(dev);
return 0;
}
// -----------------------------------------------------------------------
// Attributes, and the index over them (SchFS phase 5)
//
// setxattr/getxattr/removexattr keep Linux's shape, so a program that already
// knows how to attach metadata to a file needs no changes. schfs_query is the
// one call with no Linux equivalent: it asks the *filesystem* a question
// instead of walking it, which is the whole point of building the index.
// -----------------------------------------------------------------------
// Resolve a path to a SchFS inode, and refuse anything that is not one.
// Attributes live in the tree, so there is nowhere to put them on v1 or /proc.
static int
xattr_inode(char *path, struct inode **ip)
{
begin_op();
struct inode *p = namei(path);
if (!p) { end_op(); return -ENOENT; }
if (!SCHFS_IS_VOL_DEV(p->dev)) { iput(p); end_op(); return -EOPNOTSUPP; }
end_op();
*ip = p;
return 0;
}
addr_t sys_setxattr(void)
{
char *path, *name;
addr_t uval;
int size, flags;
if (argstr(0, &path) < 0 || argstr(1, &name) < 0) return -EFAULT;
if (argaddr(2, &uval) < 0 || argint(3, &size) < 0) return -EFAULT;
argint(4, &flags); /* XATTR_CREATE/REPLACE: unused */
if (size < 0 || size > SCHFS_ATTR_VALUE_MAX) return -E2BIG;
int namelen = strlen(name);
if (namelen <= 0 || namelen > SCHFS_ATTR_NAME_MAX) return -ERANGE;
char val[SCHFS_ATTR_VALUE_MAX];
if (size > 0 && copy_from_user(val, uval, (uint64)size) < 0) return -EFAULT;
struct inode *ip;
int r = xattr_inode(path, &ip);
if (r < 0) return r;
// Stored as a string: nothing here knows the caller's intent, and a string
// is the type whose equality comparison is the one a query wants.
r = schfs_kattr_set(ip->dev, ip->inum, name, (unsigned)namelen,
val, (unsigned)size, SCHFS_ATTR_STRING);
iput(ip);
return r == 0 ? 0 : -EIO;
}
addr_t sys_getxattr(void)
{
char *path, *name;
addr_t uval;
int size;
if (argstr(0, &path) < 0 || argstr(1, &name) < 0) return -EFAULT;
if (argaddr(2, &uval) < 0 || argint(3, &size) < 0) return -EFAULT;
int namelen = strlen(name);
if (namelen <= 0 || namelen > SCHFS_ATTR_NAME_MAX) return -ERANGE;
struct inode *ip;
int r = xattr_inode(path, &ip);
if (r < 0) return r;
char val[SCHFS_ATTR_VALUE_MAX];
unsigned vallen = 0;
// A zero size means "how big is it?", which is how every caller sizes its
// buffer before asking again.
r = schfs_kattr_get(ip->dev, ip->inum, name, (unsigned)namelen,
size > 0 ? val : 0, (unsigned)(size > 0 ? size : 0),
&vallen);
iput(ip);
if (r == 0) return -ENODATA;
if (r == -3) return -ERANGE;
if (r != 1) return -EIO;
if (size == 0) return (addr_t)vallen;
if (copy_to_user(uval, val, vallen) < 0) return -EFAULT;
return (addr_t)vallen;
}
addr_t sys_removexattr(void)
{
char *path, *name;
if (argstr(0, &path) < 0 || argstr(1, &name) < 0) return -EFAULT;
int namelen = strlen(name);
if (namelen <= 0 || namelen > SCHFS_ATTR_NAME_MAX) return -ERANGE;
struct inode *ip;
int r = xattr_inode(path, &ip);
if (r < 0) return r;
r = schfs_kattr_del(ip->dev, ip->inum, name, (unsigned)namelen);
iput(ip);
return r == 0 ? 0 : -EIO;
}
// Declare an index on an attribute name. Covers values written afterwards.
addr_t sys_schfs_index(void)
{
char *name;
if (argstr(0, &name) < 0) return -EFAULT;
int namelen = strlen(name);
if (namelen <= 0 || namelen > SCHFS_ATTR_NAME_MAX) return -ERANGE;
return schfs_kindex(SCHFS_DEV, name, (unsigned)namelen) == 0 ? 0 : -EIO;
}
// Every object whose `name` attribute equals `value`, as object ids.
//
// Object ids rather than paths: the index maps values to objects, and a
// filesystem keyed that way has no cheap way back to a name. Turning an id into
// a path is a caller's problem, and a rarer one than the query itself.
addr_t sys_schfs_query(void)
{
char *name, *value;
addr_t uout;
int cap;
if (argstr(0, &name) < 0 || argstr(1, &value) < 0) return -EFAULT;
if (argaddr(2, &uout) < 0 || argint(3, &cap) < 0) return -EFAULT;
if (cap <= 0 || cap > 256) return -EINVAL;
int namelen = strlen(name), vallen = strlen(value);
if (namelen <= 0 || namelen > SCHFS_ATTR_NAME_MAX) return -ERANGE;
if (vallen > SCHFS_ATTR_VALUE_MAX) return -E2BIG;
uint64 ids[64];
if (cap > (int)(sizeof(ids) / sizeof(ids[0])))
cap = (int)(sizeof(ids) / sizeof(ids[0]));
int n = schfs_kquery(SCHFS_DEV, name, (unsigned)namelen,
value, (unsigned)vallen, ids, cap);
if (n < 0) return -ENOENT; /* no such index */
if (n > 0 && copy_to_user(uout, ids, (uint64)n * sizeof(uint64)) < 0)
return -EFAULT;
return (addr_t)n;
}

11
kernel/sys/table.inc
View file

@ -215,6 +215,17 @@ SYSCALL(155, 137, statfs)
SYSCALL(156, 138, fstatfs)
SYSCALL(157, 161, chroot)
// --- attributes, and the index over them (SchFS phase 5) ---
//
// Linux's numbers for the xattr calls, so a program that already knows how to
// set an extended attribute needs no changes. schfs_query has no Linux
// equivalent -- it is the one genuinely new call -- so it is native-only.
SYSCALL(193, 188, setxattr)
SYSCALL(194, 191, getxattr)
SYSCALL(195, 197, removexattr)
SYSCALL(196, NO_LINUX, schfs_index)
SYSCALL(197, NO_LINUX, schfs_query)
// --- memory extensions ---
SYSCALL(136, 28, madvise)
SYSCALL(137, NO_LINUX, mremap)

9
lib/usys.S
View file

@ -109,3 +109,12 @@ SYSCALL(setsockopt)
SYSCALL(getsockopt)
SYSCALL(mount)
SYSCALL(getdents64)
// Attributes and the index over them (SchFS phase 5). setxattr/getxattr/
// removexattr are Linux's; schfs_index and schfs_query are native, because
// asking a filesystem a question has no Linux equivalent.
SYSCALL(setxattr)
SYSCALL(getxattr)
SYSCALL(removexattr)
SYSCALL(schfs_index)
SYSCALL(schfs_query)

20
tools/mkschfs.c
View file

@ -69,6 +69,7 @@ static long readlink(const char *p, char *b, unsigned long n)
#include "../kernel/fs/schfs/vol.c"
#include "../kernel/fs/schfs/obj.c"
#include "../kernel/fs/schfs/objw.c"
#include "../kernel/fs/schfs/attr.c"
/* ------------------------------------------------------------------ */
/* File-backed block device */
@ -111,6 +112,23 @@ img_write(void *ctx, uint64_t b, const void *buf)
return 0;
}
// Make what has been written actually reach the file.
//
// The kernel's version of this hook drains the buffer cache and then tells the
// drive to commit its own cache; here the equivalent stages are stdio's buffer
// and the operating system's page cache. The image mkschfs is building is
// normally consumed by something else the moment it exits, so this is mostly
// belt and braces -- but the commit path calls it, and a hook that lies about
// having flushed is exactly the bug this whole change set is about.
static int
img_flush(void *ctx)
{
struct imgdev *d = ctx;
if (fflush(d->f) != 0)
return -1;
return 0;
}
/* ------------------------------------------------------------------ */
/* Allocation */
/* */
@ -661,12 +679,14 @@ main(int argc, char **argv)
IMG.nblocks = nblocks;
struct schfs_bdev bd;
memset(&bd, 0, sizeof(bd)); /* optional hooks must read as absent */
bd.ctx = &IMG;
bd.bread = img_read;
bd.bwrite = img_write;
bd.balloc = img_alloc;
bd.bfree = img_free;
bd.balloc_run = img_alloc_run;
bd.bflush = img_flush;
int r = schfs_vol_format(&VOL, &bd, nblocks, label, &CTX, img_palloc);
if (r != 0) die("formatting", r);

5
tools/schfs_fsck.c
View file

@ -73,6 +73,10 @@ static int link(const char *from, const char *to)
#include "../kernel/fs/schfs/space.c"
#include "../kernel/fs/schfs/vol.c"
#include "../kernel/fs/schfs/obj.c"
// Included for schfs_index_reload, which schfs_vol_open calls at mount. This
// tool never writes, so the rest of attr.c is dead weight here -- but linking
// against a filesystem whose mount path is missing a function is worse.
#include "../kernel/fs/schfs/attr.c"
/* ------------------------------------------------------------------ */
@ -771,6 +775,7 @@ main(int argc, char **argv)
IMG.nblocks = (uint64_t)sz / SCHFS_BLKSIZE;
struct schfs_bdev bd;
memset(&bd, 0, sizeof(bd)); /* optional hooks must read as absent */
bd.ctx = &IMG;
bd.bread = img_read;
bd.bwrite = img_write;

438
tools/schfs_objtest.c
View file

@ -17,12 +17,14 @@
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <setjmp.h> /* the crash in section 10 is a longjmp out of a write */
#include "../kernel/fs/schfs/btree.c"
#include "../kernel/fs/schfs/space.c"
#include "../kernel/fs/schfs/vol.c"
#include "../kernel/fs/schfs/obj.c"
#include "../kernel/fs/schfs/objw.c"
#include "../kernel/fs/schfs/attr.c"
/* ------------------------------------------------------------------ */
/* Memory-backed device */
@ -50,11 +52,28 @@ md_read(void *ctx, uint64_t b, void *buf)
return 0;
}
// Simulated power loss.
//
// md_fail_after counts writes down; when it reaches zero the write does not
// happen and control leaves the filesystem entirely, through the longjmp. That
// is what a power cut is: the disk keeps whatever reached it and *nothing else
// runs*. The obvious cheaper models are both wrong in ways that test the wrong
// thing -- returning an error tests error handling, and silently swallowing the
// write while letting execution continue leaves the code reading back blocks it
// believes it just wrote, which no real failure does.
static long md_fail_after = -1;
static int md_crash_armed;
static jmp_buf md_crash_jmp;
static long md_writes;
static int
md_write(void *ctx, uint64_t b, const void *buf)
{
(void)ctx;
if (b >= NBLOCKS) return -1;
md_writes++;
if (md_crash_armed && md_fail_after >= 0 && md_fail_after-- == 0)
longjmp(md_crash_jmp, 1);
memcpy(DISK + b * SCHFS_BLKSIZE, buf, SCHFS_BLKSIZE);
return 0;
}
@ -67,6 +86,11 @@ static void *md_palloc(void) { return calloc(1, SCHFS_BLKSIZE
static void
mkbdev(struct schfs_bdev *bd)
{
// Zero first. Every field has to be set even when the answer is "no such
// hook", because callers declare these on the stack: leaving the optional
// bflush uninitialised handed schfs_vol_commit a garbage function pointer
// to test and call, which is a segfault at the first commit.
memset(bd, 0, sizeof(*bd));
bd->ctx = 0;
bd->bread = md_read;
bd->bwrite = md_write;
@ -663,6 +687,420 @@ main(void)
(unsigned long long)VOL.free_blocks);
}
/* ---- 9b. reflink ---- */
//
// The thing that makes a reflink worth having is that it costs no space,
// and the thing that makes it dangerous is that both files are writable
// afterwards. So both are checked: the clone must be free, and a write to
// either side must not be visible on the other.
printf("\n reflink\n");
{
uint64_t before = VOL.free_blocks;
uint64_t a = mkfile(ROOT, "clone-src");
char big[16 * 1024];
for (unsigned i = 0; i < sizeof(big); i++)
big[i] = (char)(i * 7 + 1);
schfs_file_write(&VOL, a, 0, big, sizeof(big), DATABLK);
remount();
uint64_t used_by_a = before - VOL.free_blocks;
ok(used_by_a > 0, "the source file occupies space");
uint64_t b = mkfile(ROOT, "clone-dst");
uint64_t pre = VOL.free_blocks;
ok(schfs_file_reflink(&VOL, a, b) == 0, "reflink succeeds");
remount();
// A handful of metadata blocks is expected; a full copy is not.
uint64_t cost = pre - VOL.free_blocks;
ok(cost * 4 < used_by_a,
"the clone costs far less than the data it shares");
if (cost * 4 >= used_by_a)
printf(" (clone cost %llu blocks, data is %llu)\n",
(unsigned long long)cost, (unsigned long long)used_by_a);
ok(size_of(b) == sizeof(big), "the clone has the source's size");
char rb[sizeof(big)];
ok(readall(b, rb, sizeof(rb)) == (long)sizeof(big) &&
memcmp(rb, big, sizeof(big)) == 0,
"the clone reads back exactly what the source held");
// Writing to the clone must not disturb the source. This is the check
// that fails if the write path rewrites a shared run in place.
const char *poke = "CLONE-WAS-WRITTEN";
schfs_file_write(&VOL, b, 0, poke, 17, DATABLK);
remount();
char ra[sizeof(big)];
ok(readall(a, ra, sizeof(ra)) == (long)sizeof(big) &&
memcmp(ra, big, sizeof(big)) == 0,
"writing the clone leaves the source untouched");
ok(readall(b, rb, sizeof(rb)) == (long)sizeof(big) &&
memcmp(rb, poke, 17) == 0 &&
memcmp(rb + 17, big + 17, sizeof(big) - 17) == 0,
"and the clone holds the new bytes with the rest unchanged");
// And the other direction, on a fresh pair: writing the *source*.
uint64_t c2 = mkfile(ROOT, "clone-dst2");
ok(schfs_file_reflink(&VOL, a, c2) == 0, "a second clone of the same file");
remount();
schfs_file_write(&VOL, a, 0, "SOURCE-WAS-WRITTEN", 18, DATABLK);
remount();
ok(readall(c2, rb, sizeof(rb)) == (long)sizeof(big) &&
memcmp(rb, big, sizeof(big)) == 0,
"writing the source leaves the clone untouched");
// Deleting every holder must give all the space back. A refcount that
// is decremented once too often frees blocks another file still uses;
// one decremented too rarely leaks them. Only this catches either.
schfs_unlink(&VOL, ROOT, "clone-src", 9);
schfs_unlink(&VOL, ROOT, "clone-dst", 9);
schfs_unlink(&VOL, ROOT, "clone-dst2", 10);
remount();
ok(VOL.free_blocks == before,
"deleting every holder returns exactly the space they shared");
if (VOL.free_blocks != before)
printf(" (started %llu, ended %llu)\n",
(unsigned long long)before,
(unsigned long long)VOL.free_blocks);
ok(schfs_btree_check(&VOL.tree) == 0, "the tree is sound after all that");
}
/* ---- 9c. attributes, and the index over them ---- */
//
// Two things are being checked, and only the second is interesting. That
// attributes store and reload is table stakes. That a *query* returns
// exactly the right objects -- including after a value changes, which has
// to remove the old index entry as well as add the new one -- is the part
// that makes the feature worth having and the part that quietly breaks.
printf("\n attributes and query\n");
{
uint64_t d = mkdirent(ROOT, "photos");
uint64_t f1 = mkfile(d, "beach.jpg");
uint64_t f2 = mkfile(d, "hills.jpg");
uint64_t f3 = mkfile(d, "notes.txt");
ok(schfs_attr_set(&VOL, f1, "tag", 3, "holiday", 7,
SCHFS_ATTR_STRING) == 0, "set an attribute");
char buf[64];
unsigned vl = 0;
uint16_t ty = 0;
ok(schfs_attr_get(&VOL, f1, "tag", 3, buf, sizeof(buf), &vl, &ty) == 1 &&
vl == 7 && memcmp(buf, "holiday", 7) == 0 &&
ty == SCHFS_ATTR_STRING,
"it reads back with its value and type");
ok(schfs_attr_get(&VOL, f2, "tag", 3, buf, sizeof(buf), &vl, &ty) == 0,
"an object without it says so");
remount();
ok(schfs_attr_get(&VOL, f1, "tag", 3, buf, sizeof(buf), &vl, &ty) == 1 &&
memcmp(buf, "holiday", 7) == 0,
"and it survives a remount");
// Two attributes on one object, and one replaced.
schfs_attr_set(&VOL, f1, "rating", 6, "5", 1, SCHFS_ATTR_INT);
schfs_attr_set(&VOL, f1, "tag", 3, "work", 4, SCHFS_ATTR_STRING);
remount();
ok(schfs_attr_get(&VOL, f1, "tag", 3, buf, sizeof(buf), &vl, &ty) == 1 &&
vl == 4 && memcmp(buf, "work", 4) == 0,
"replacing a value replaces it rather than appending");
ok(schfs_attr_get(&VOL, f1, "rating", 6, buf, sizeof(buf), &vl, &ty) == 1 &&
vl == 1 && buf[0] == '5',
"and the other attribute is untouched");
ok(schfs_attr_del(&VOL, f1, "rating", 6) == 0 &&
schfs_attr_get(&VOL, f1, "rating", 6, buf, sizeof(buf), &vl, &ty) == 0,
"removing one leaves the rest");
ok(schfs_attr_get(&VOL, f1, "tag", 3, buf, sizeof(buf), &vl, &ty) == 1,
"specifically, the one not removed");
/* --- the index --- */
uint64_t hits[8];
ok(schfs_query_equal(&VOL, "tag", 3, "holiday", 7, hits, 8) < 0,
"querying an attribute with no index is refused, not answered slowly");
ok(schfs_index_create(&VOL, "tag", 3) == 0, "declare an index on `tag`");
// Written *after* the index exists, which is the only thing it covers.
schfs_attr_set(&VOL, f1, "tag", 3, "holiday", 7, SCHFS_ATTR_STRING);
schfs_attr_set(&VOL, f2, "tag", 3, "holiday", 7, SCHFS_ATTR_STRING);
schfs_attr_set(&VOL, f3, "tag", 3, "work", 4, SCHFS_ATTR_STRING);
remount();
int n = schfs_query_equal(&VOL, "tag", 3, "holiday", 7, hits, 8);
ok(n == 2, "a query returns exactly the objects carrying that value");
if (n != 2)
printf(" (got %d)\n", n);
ok(n == 2 && ((hits[0] == f1 && hits[1] == f2) ||
(hits[0] == f2 && hits[1] == f1)),
"and they are the right two");
n = schfs_query_equal(&VOL, "tag", 3, "work", 4, hits, 8);
ok(n == 1 && hits[0] == f3, "a different value returns a different set");
n = schfs_query_equal(&VOL, "tag", 3, "nonesuch", 8, hits, 8);
ok(n == 0, "a value nothing carries returns nothing");
// Changing a value must take the object *out* of the old set. An index
// that only ever gains entries answers confidently and wrongly.
schfs_attr_set(&VOL, f2, "tag", 3, "work", 4, SCHFS_ATTR_STRING);
remount();
n = schfs_query_equal(&VOL, "tag", 3, "holiday", 7, hits, 8);
ok(n == 1 && hits[0] == f1,
"changing a value removes the object from the old value's set");
n = schfs_query_equal(&VOL, "tag", 3, "work", 4, hits, 8);
ok(n == 2, "and adds it to the new one");
// And removing the attribute entirely.
schfs_attr_del(&VOL, f1, "tag", 3);
remount();
n = schfs_query_equal(&VOL, "tag", 3, "holiday", 7, hits, 8);
ok(n == 0, "removing the attribute removes it from the index too");
ok(schfs_btree_check(&VOL.tree) == 0, "the tree is sound after all that");
}
/* ---- 10. crash recovery ---- */
//
// The whole consistency argument for this filesystem is one sentence:
// mount takes the newest uberblock that validates, so a crash lands on
// either the old tree or the new one and never between them. Every test
// above this point runs on a volume that was never interrupted, so not one
// of them touches that sentence -- schfs_test proves it for the B-tree in
// isolation, and nothing proved it for a whole volume until here.
printf("\n crash recovery\n");
{
// A known-good starting point, and a note of where it is.
{
uint64_t d = mkdirent(ROOT, "survivor");
uint64_t f = mkfile(d, "keep");
schfs_file_write(&VOL, f, 0, "before the crash", 16, DATABLK);
}
remount();
uint64_t good_gen = VOL.generation;
unsigned char *saved = malloc(NBLOCKS * SCHFS_BLKSIZE);
memcpy(saved, DISK, NBLOCKS * SCHFS_BLKSIZE);
/* --- 10a. cut the power in the middle of a transaction --- */
//
// Enough writes to be well into the tree and nowhere near the commit,
// which is the case that matters: the uberblock never lands, so the
// previous one is still newest and every block this group touched is
// one no committed root points at.
md_writes = 0;
md_writes = 0;
if (setjmp(md_crash_jmp) == 0) {
md_crash_armed = 1;
md_fail_after = 12;
uint64_t d = mkdirent(ROOT, "doomed");
for (int i = 0; i < 40; i++) {
char nm[16];
snprintf(nm, sizeof(nm), "f%d", i);
uint64_t f = mkfile(d, nm);
if (f) schfs_file_write(&VOL, f, 0, "junk", 4, DATABLK);
}
schfs_vol_commit(&VOL);
printf(" FAIL: the simulated crash never fired\n");
failures++;
}
md_crash_armed = 0;
md_fail_after = -1;
struct schfs_bdev bd3;
mkbdev(&bd3);
memset(&VOL, 0, sizeof(VOL));
int r = schfs_vol_open(&VOL, &bd3, &CTX, md_palloc);
ok(r == 0, "the volume mounts after a crash mid-transaction");
ok(VOL.generation == good_gen,
"it comes back at the last committed generation, not a partial one");
ok(VOL.recovered == 0,
"and needs no fallback to do it: the newest uberblock is the good one");
/* --- 10b. the pre-crash filesystem is exactly as it was --- */
{
struct schfs_cursor c;
schfs_cursor_init(&c, &VOL.tree, CURBLK);
uint64_t d = 0, f = 0;
uint16_t ty = 0;
ok(schfs_dir_lookup(&VOL, &c, ROOT, "survivor", 8, &d, &ty) == 0,
"what was committed before the crash is still there");
schfs_cursor_init(&c, &VOL.tree, CURBLK);
ok(d && schfs_dir_lookup(&VOL, &c, d, "keep", 4, &f, &ty) == 0,
"and so is its content");
char buf[32];
memset(buf, 0, sizeof(buf));
schfs_cursor_init(&c, &VOL.tree, CURBLK);
long got = f ? schfs_file_read(&VOL, &c, f, 0, buf, 16, DATABLK) : -1;
ok(got == 16 && memcmp(buf, "before the crash", 16) == 0,
"the file reads back byte for byte");
schfs_cursor_init(&c, &VOL.tree, CURBLK);
uint64_t gone = 0;
ok(schfs_dir_lookup(&VOL, &c, ROOT, "doomed", 6, &gone, &ty) != 0,
"nothing the interrupted transaction did is visible");
}
ok(schfs_btree_check(&VOL.tree) == 0,
"the tree is structurally sound after the crash");
/* --- 10b'. crash at every write point, not one --- */
//
// One crash point proves the mechanism exists. It does not prove there
// is no window -- and a window is exactly what this kind of bug is: a
// handful of writes somewhere in the middle after which the volume
// comes back subtly wrong rather than obviously broken. So: run the
// same transaction again and again, cutting the power one write later
// each time, and check the whole story every time.
{
int points = 0, bad = 0;
for (long cut = 1; cut < 120; cut++) {
memcpy(DISK, saved, NBLOCKS * SCHFS_BLKSIZE);
memset(&VOL, 0, sizeof(VOL));
mkbdev(&bd3);
if (schfs_vol_open(&VOL, &bd3, &CTX, md_palloc) != 0) {
bad++; break;
}
int fired = 0;
if (setjmp(md_crash_jmp) == 0) {
md_crash_armed = 1;
md_fail_after = cut;
uint64_t d = mkdirent(ROOT, "doomed");
for (int i = 0; i < 40; i++) {
char nm[16];
snprintf(nm, sizeof(nm), "f%d", i);
uint64_t f = mkfile(d, nm);
if (f) schfs_file_write(&VOL, f, 0, "junk", 4, DATABLK);
}
schfs_vol_commit(&VOL);
} else {
fired = 1;
}
md_crash_armed = 0;
md_fail_after = -1;
if (!fired)
break; /* ran out of writes to cut at */
points++;
// Whatever the disk looks like now, all three of these must
// hold: it mounts, it is a whole tree, and the data committed
// before the crash is exactly as it was.
memset(&VOL, 0, sizeof(VOL));
mkbdev(&bd3);
if (schfs_vol_open(&VOL, &bd3, &CTX, md_palloc) != 0) {
printf(" cut %ld: does not mount\n", cut);
bad++; continue;
}
if (schfs_btree_check(&VOL.tree) != 0) {
printf(" cut %ld: tree is not sound\n", cut);
bad++; continue;
}
struct schfs_cursor c;
schfs_cursor_init(&c, &VOL.tree, CURBLK);
uint64_t d = 0, f = 0;
uint16_t ty = 0;
if (schfs_dir_lookup(&VOL, &c, ROOT, "survivor", 8, &d, &ty) != 0) {
printf(" cut %ld: lost a committed directory\n", cut);
bad++; continue;
}
schfs_cursor_init(&c, &VOL.tree, CURBLK);
if (schfs_dir_lookup(&VOL, &c, d, "keep", 4, &f, &ty) != 0) {
printf(" cut %ld: lost a committed file\n", cut);
bad++; continue;
}
char buf[32];
memset(buf, 0, sizeof(buf));
schfs_cursor_init(&c, &VOL.tree, CURBLK);
if (schfs_file_read(&VOL, &c, f, 0, buf, 16, DATABLK) != 16 ||
memcmp(buf, "before the crash", 16) != 0) {
printf(" cut %ld: committed file came back changed\n", cut);
bad++; continue;
}
}
char msg[96];
snprintf(msg, sizeof(msg),
"%d crash points, every one mounts intact with its "
"committed data", points);
ok(bad == 0 && points > 20, msg);
}
/* --- 10c. a newest uberblock naming a tree that never landed --- */
//
// This is the failure the commit barrier exists to prevent, and it is
// still reachable through a misdirected write or a bad sector under
// the root. Before the fallback existed the mount simply refused, with
// 127 intact older roots sitting on the disk untouched.
memcpy(DISK, saved, NBLOCKS * SCHFS_BLKSIZE);
memset(&VOL, 0, sizeof(VOL));
mkbdev(&bd3);
if (schfs_vol_open(&VOL, &bd3, &CTX, md_palloc) != 0) {
printf(" FAIL: reopen for the misdirected-write case\n");
failures++;
} else {
uint64_t base_gen = VOL.generation;
// Two more commits, then destroy the block each of their roots
// points at -- so the two newest uberblocks are both intact and
// both name rubbish.
uint64_t roots[2];
for (int i = 0; i < 2; i++) {
char nm[16];
snprintf(nm, sizeof(nm), "later%d", i);
mkfile(ROOT, nm);
schfs_vol_commit(&VOL);
roots[i] = VOL.tree.root;
}
for (int i = 0; i < 2; i++)
memset(DISK + roots[i] * SCHFS_BLKSIZE, 0xA5, SCHFS_BLKSIZE);
memset(&VOL, 0, sizeof(VOL));
mkbdev(&bd3);
int rr = schfs_vol_open(&VOL, &bd3, &CTX, md_palloc);
ok(rr == 0, "a volume whose two newest roots are rubbish still mounts");
// Strictly older, not "exactly two older". Copy-on-write shares
// every unchanged node between generations, so a block that is part
// of the newest tree is very often part of several older ones too --
// destroying it takes them all with it. How far back the ring has to
// reach is therefore not something a test can pin down, and an
// earlier version of this asserted exactly two and was simply wrong
// once the filesystem had enough history to share nodes.
//
// What must hold is that it landed on neither corrupted generation
// and said so.
ok(VOL.generation < base_gen,
"it falls back past both of them to an older readable root");
ok(VOL.recovered >= 2,
"and says that at least those two groups were lost");
ok(schfs_btree_check(&VOL.tree) == 0,
"what it recovered is structurally sound");
}
/* --- 10d. every uberblock destroyed is a failure, not a guess --- */
memcpy(DISK, saved, NBLOCKS * SCHFS_BLKSIZE);
memset(DISK + SCHFS_UBER_START * SCHFS_BLKSIZE, 0,
SCHFS_UBER_COUNT * SCHFS_BLKSIZE);
memset(&VOL, 0, sizeof(VOL));
mkbdev(&bd3);
ok(schfs_vol_open(&VOL, &bd3, &CTX, md_palloc) != 0,
"a volume with no valid uberblock at all is refused, not invented");
// Put the good image back so anything after this runs on a sane volume.
memcpy(DISK, saved, NBLOCKS * SCHFS_BLKSIZE);
memset(&VOL, 0, sizeof(VOL));
mkbdev(&bd3);
schfs_vol_open(&VOL, &bd3, &CTX, md_palloc);
free(saved);
}
printf("\n%s\n", failures ? "FAILED" : "all checks passed");
return failures ? 1 : 0;
}

1
tools/schfs_test.c
View file

@ -259,6 +259,7 @@ static void
mkdev(struct schfs_bdev *bd, struct memdev *d)
{
memset(d, 0, sizeof(*d));
memset(bd, 0, sizeof(*bd)); /* optional hooks must read as absent */
d->next = SCHFS_UBER_START + 1;
d->fail_after = -1;
bd->ctx = d;