SCFF: the live root ships compressed, with Xorg and Window Maker

SCFF, the SchOS Compressed File Format (include/fs/scff.h), is a block
image cut into 128 KB chunks, each its own zstd frame, behind an index.
tools/mkscff writes it from fs-ram.img on every core (-19, 11 s) and
checks each chunk decompresses to itself. The kernel carries the .scff
as its ramdisk and unpacks a chunk the first time a block in it is
read; zero blocks share one page. The kernel and the ISO are the
compressed size, and the root costs memory only for what is touched.
See docs/architecture/scff.md.

zstd 1.5.7 comes from ports/zstd like uACPI: the decoder is compiled
into the kernel at -O2, with a static context and a 64 KB stack of its
own, and mkscff builds the whole library from the same release.

world now has xorg-server, the fbdev and evdev drivers, xinit, Window
Maker, xterm and DejaVu, and /root/.xinitrc runs wmaker. That root is
358 MB, 102 MB as SCFF.

The ISO no longer carries the root a third time as an ISO 9660 tree,
which nothing read. schos.iso: 1389 MB -> 318 MB. El Torito keeps its
own copy of the EFI image, the layout real machines have booted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
This commit is contained in:
Nexoniarz 2026-10-24 21:17:31 +02:00
commit df1394eaa3
18 changed files with 832 additions and 38 deletions

80
kernel/scff/scff.c Normal file
View file

@ -0,0 +1,80 @@
// SPDX-License-Identifier: GPL-2.0-or-later
// The zstd decoder for SCFF chunks (include/fs/scff.h).
//
// One static decompression context in a fixed workspace, so nothing here
// allocates, and one caller at a time: kernel/dev/ramdisk.c holds its chunk
// lock across every call.
//
// **The decoder runs on a stack of its own.** A kernel stack is 8 KB, and a
// chunk is decompressed at the bottom of a read -- syscall, namei, SchFS,
// bread, ramdiskrw -- with most of it already used. zstd's sequence decoder
// keeps several KB of state in locals, which Linux gives it on a 16 KB stack.
// Here that would run off the end of the process's stack into whatever is
// below it. So the call is made on scff_stack, which the chunk lock makes
// this caller's alone. An interrupt taken meanwhile lands on it too, which
// is fine: it is a kernel stack like any other, and a big one.
//
// This file sees zstd's headers and none of the kernel's, which keeps the two
// sets of type definitions from meeting.
#define ZSTD_STATIC_LINKING_ONLY
#include "zstd.h"
#include "scff_zstd.h"
static unsigned char scff_work[256 * 1024] __attribute__((aligned(64)));
static unsigned char scff_stack[64 * 1024] __attribute__((aligned(16)));
static ZSTD_DCtx *dctx;
// call_on_stack(fn, arg, top): fn(arg) with %rsp at top, then back.
// %rbp keeps the old stack pointer across the call; it is callee-saved.
long call_on_stack(long (*fn)(void *), void *arg, void *top);
__asm__(
".text\n"
".globl call_on_stack\n"
"call_on_stack:\n"
" pushq %rbp\n"
" movq %rsp, %rbp\n"
" movq %rdx, %rsp\n"
" movq %rdi, %rax\n"
" movq %rsi, %rdi\n"
" callq *%rax\n"
" movq %rbp, %rsp\n"
" popq %rbp\n"
" retq\n");
struct job {
void *dst;
unsigned long cap;
const void *src;
unsigned long n;
};
static long
run(void *arg)
{
struct job *j = arg;
size_t r = ZSTD_decompressDCtx(dctx, j->dst, j->cap, j->src, j->n);
return ZSTD_isError(r) ? -1 : (long)r;
}
int
scff_zstd_init(void)
{
if (dctx)
return 0;
if (ZSTD_estimateDCtxSize() > sizeof(scff_work))
return -1;
dctx = ZSTD_initStaticDCtx(scff_work, sizeof(scff_work));
return dctx ? 0 : -1;
}
long
scff_zstd_decompress(void *dst, unsigned long cap, const void *src, unsigned long n)
{
struct job j = { dst, cap, src, n };
if (!dctx)
return -1;
return call_on_stack(run, &j, scff_stack + sizeof(scff_stack));
}