nexoniarz/SchOS
nexoniarz/SchOS
Watch
1
0
Fork
You've already forked SchOS
 0
Code Issues Pull requests Projects Releases 1 Packages Wiki Activity Actions

schfs: commit every 5 s, survive a full disk, mount up to 64 TiB

Browse source 
Transaction groups commit from a schfsd thread every 5 seconds, on sync
and fsync, on unmount and shutdown, and early when freed-but-pending
blocks crowd the free ones -- the contract ext4 and ZFS have. A crash
still lands on the last group whole, and loses at most those seconds.
Committing before every write(2) returned cost two cache flushes and an
uberblock a call: 4 KB writes go from 2.8 to 6.6 MB/s on AHCI, and from
66 KB/s to 3.9 MB/s on QEMU's IDE disk.

Filling the disk made the root volume unmountable, on the last commit
as well: a tree update that ran out of blocks halfway was published by
the next commit. 1% of the volume (1 to 64 MiB) is now a metadata
reserve, as btrfs and ZFS keep one; data stops at it with ENOSPC as a
short write, new names at half of it, and deletes may use the rest. A
write that runs out part way records what it wrote in the inode rather
than leaving blocks past an end of file nothing moved.

Deleting a large file goes in 4 MiB steps from the end with a commit
allowed between them, as ext4 restarts its transaction through a long
truncate. Deleting 80 MB on a full disk ran out of room halfway and
leaked the rest.

The VFS's dirunlink called the object layer's whole unlink, which freed
the object with its last name, so a file removed while open lost its
data under the reader. It removes the name only now, and the object is
freed on the last iput. An object still open when the machine went down
is found by the mount's space scan and freed, like ext4's orphan list.

The allocation bitmap has a second level of pointer pages, so the mount
ceiling is 64 TiB instead of 64 GiB, and the freed-this-group bitmap
has pages only where frees happened: a 1 TiB volume takes 32 MiB.

A freed block written after the newest snapshot is reused at once
(ZFS's birth-time rule, so bfree carries the block's generation); only
older ones are kept while a snapshot exists. Deleting one snapshot no
longer stops retaining for the others, which let their blocks be
handed out again.

Checked under QEMU: crash after sync, after 8 s and within 5 s; fill to
ENOSPC, rm, remount; unlink while open; an orphan freed at the next
mount; a 1 TiB volume. schfs-check passes, with a new test for the
snapshot rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
This commit is contained in:
Nexoniarz 2026-10-23 12:20:05 +02:00
commit edc4e0537d
13 changed files with 572 additions and 144 deletions
Download patch file Download diff file Expand all files Collapse all files

39
kernel/fs/schfs/objw.c
View file

@ -84,8 +84,11 @@ schfs_snap_create(struct schfs_vol *v, uint64_t id, const char *name,
struct schfs_key k;
schfs_mkkey(&k, SCHFS_SNAP_OBJECTID, SCHFS_KEY_SNAP, id);
int r = schfs_btree_insert(&v->tree, &v->ctx, &k, &sn, sizeof sn);
if (r == 0)
if (r == 0) {
v->space.flags |= SCHFS_SPACE_RETAIN_ALL;
if (sn.created > v->space.snap_gen)
v->space.snap_gen = sn.created;
}
return r;
}
@ -103,12 +106,11 @@ schfs_snap_delete(struct schfs_vol *v, uint64_t id)
if (r != 0)
return r;
// Stop retaining. If other snapshots remain, the next mount's derivation
// sets the flag again -- it is the walk that knows, not this function, and
// guessing here would mean keeping a second count to disagree with it.
// Until that mount the allocator is merely conservative, which is the safe
// direction.
v->space.flags &= ~(unsigned)SCHFS_SPACE_RETAIN_ALL;
// Keep retaining. Another snapshot may hold the same old blocks, and only
// the walk at mount knows which remain -- so until then the allocator stays
// conservative, which is the safe direction. This used to clear the flag,
// the unsafe one: deleting one of two snapshots let the other's blocks be
// freed, handed out and overwritten.
return 0;
}
@ -575,7 +577,7 @@ extent_release(struct schfs_vol *v, const struct schfs_extent *e,
int r = csum_del(v, first + i);
if (r != 0)
return r;
v->dev.bfree(v->dev.ctx, first + i);
v->dev.bfree(v->dev.ctx, first + i, e->gen);
}
return 0;
}
@ -603,7 +605,7 @@ extent_release(struct schfs_vol *v, const struct schfs_extent *e,
int r = csum_del(v, runstart + i);
if (r != 0)
return r;
v->dev.bfree(v->dev.ctx, runstart + i);
v->dev.bfree(v->dev.ctx, runstart + i, e->gen);
}
return 0;
}
@ -754,6 +756,9 @@ schfs_file_reserve(struct schfs_vol *v, uint64_t ino, uint64_t nblocks)
// next data block interleaves metadata with data, so consecutive file
// blocks are never physically adjacent. Grabbing the run up front keeps it
// contiguous and a whole file becomes one record.
if (schfs_space_low(&v->space, 1, nblocks))
return -3;
uint64_t first;
if (v->dev.balloc_run) {
first = v->dev.balloc_run(v->dev.ctx, nblocks);
@ -806,6 +811,8 @@ block_for_write(struct schfs_vol *v, uint64_t ino, uint64_t fblk,
return -2;
if (m == 0) {
if (schfs_space_low(&v->space, 1, 1))
return -3;
uint64_t d = v->dev.balloc(v->dev.ctx);
if (d == 0)
return -3;
@ -842,6 +849,8 @@ block_for_write(struct schfs_vol *v, uint64_t ino, uint64_t fblk,
// it: a head covering what came before, the new block, and a tail covering
// what came after. No deletion is needed — when the block being replaced is
// the run's first, the middle record simply overwrites the original key.
if (schfs_space_low(&v->space, 1, 1))
return -3;
uint64_t nd = v->dev.balloc(v->dev.ctx);
if (nd == 0)
return -3;
@ -911,7 +920,7 @@ block_for_write(struct schfs_vol *v, uint64_t ino, uint64_t fblk,
int rr = csum_del(v, runstart + i);
if (rr != 0)
return rr;
v->dev.bfree(v->dev.ctx, runstart + i);
v->dev.bfree(v->dev.ctx, runstart + i, e.gen);
}
}
}
@ -956,6 +965,12 @@ schfs_file_write(struct schfs_vol *v, uint64_t ino, uint64_t off,
struct wblk w;
r = block_for_write(v, ino, fblk, &w);
// Out of room, at a block boundary and before anything for this block
// was touched: keep what was written, as a short write, and record it
// in the inode. Returning the error alone left those blocks in the
// tree past an end of file nothing had moved.
if (r == -3 && done > 0)
break;
if (r != 0)
return r;
if (w.fresh)
@ -986,8 +1001,8 @@ schfs_file_write(struct schfs_vol *v, uint64_t ino, uint64_t off,
done += want;
}
if (off + n > in.size)
in.size = off + n;
if (off + done > in.size)
in.size = off + done;
in.gen = v->tree.generation;
r = schfs_inode_put(v, ino, &in);
if (r != 0)