PSOGM mode will put the kernel log on screen as QR codes, a page per
symbol, and version 6 held 134 bytes. Version 40 at level L holds 2,953.
What grew: blocks of two lengths and the interleave that steps around
them, the alignment-pattern grid, the version-information blocks from
version 7, the 16-bit character count from version 10. The per-version
table is down to the two columns that are genuinely transcription (EC
codewords per block, block count); total codewords come from the
geometry.
Checked against segno module for module at every version (exact-fill
payloads), and 120 symbols across all 40 versions decode with zbar.
OpenCV's detector cannot find symbols above about version 24 at all,
segno's included, so qrcheck reports those as the detector's limit.
The crash card stays a version-6 symbol (QR_CARD_BYTES), so a phone still
reads it from across the desk.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
A release on the Forgejo carries the ISO and its sha256. The server is a
small VPS shared with other projects, so publishing a release deletes the
ISO from every older one; their notes, tags and checksums stay, with a
line saying how to rebuild the image from the tag.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
The README led with a wallpaper and described the system of two months
ago: no desktop, no dynamic linking, a handful of drivers. It now leads
with Shift and covers Xorg and Window Maker, apk and Alpine packages,
SCFF, HTTPS, the storage, network and input drivers, and the machines it
has booted on. "Not there yet" lists what is actually missing today:
threads, udev, EHCI, touchpad gestures, an installer.
The numbers -- lines of code and of comments, syscalls, drivers, packages,
commits -- come from tools/python/stats.py, which rewrites them in the
README and on the website between stats markers instead of somebody
typing them once and letting them go stale.
.gitattributes keeps the generated font and logo tables and the vendored
Niveus out of the repository page's language bar.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
Xorg uses VTs only when stat("/dev/tty0") reports Linux's tty major 4.
SchOS's terminals sit under major 1, so X ran on no VT at all: the console
kept drawing over it, Ctrl+Alt+F-keys did nothing, and nothing ended the
session. stat now reports terminals under Linux's numbers.
startx picks the VT only when uname says Linux, so root's xserverrc passes
the current one itself. An xorg.conf.d file names X's input devices, since
Alpine's Xorg looks them up through udev and SchOS has none yet.
Alt+F1..F6 and Ctrl+Alt+F1..F6 now switch terminals as on Linux, alongside
the settable chord. USB keyboards gain arrows, Home/End/PgUp/PgDn/Ins/Del
and the switch keys, which the PS/2 path always had.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
SCFF, the SchOS Compressed File Format (include/fs/scff.h), is a block
image cut into 128 KB chunks, each its own zstd frame, behind an index.
tools/mkscff writes it from fs-ram.img on every core (-19, 11 s) and
checks each chunk decompresses to itself. The kernel carries the .scff
as its ramdisk and unpacks a chunk the first time a block in it is
read; zero blocks share one page. The kernel and the ISO are the
compressed size, and the root costs memory only for what is touched.
See docs/architecture/scff.md.
zstd 1.5.7 comes from ports/zstd like uACPI: the decoder is compiled
into the kernel at -O2, with a static context and a 64 KB stack of its
own, and mkscff builds the whole library from the same release.
world now has xorg-server, the fbdev and evdev drivers, xinit, Window
Maker, xterm and DejaVu, and /root/.xinitrc runs wmaker. That root is
358 MB, 102 MB as SCFF.
The ISO no longer carries the root a third time as an ISO 9660 tree,
which nothing read. schos.iso: 1389 MB -> 318 MB. El Torito keeps its
own copy of the EFI image, the layout real machines have booted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
Xorg finds its VT through /dev/tty0, and without one ran on none: the
console went on drawing into the framebuffer over the X screen. tty0 is
the terminal on screen, as on Linux. FBIOPUTCMAP succeeds, as it does on
a true-colour Linux framebuffer, instead of 256 logged errors per start.
schos-base gives root a .xserverrc with -dumbSched, Xorg's switch for
input on the main loop: SchOS has no threads, and Xorg crashed in
pthread_join on its input thread when the first client left. It adds
the hostname to /etc/hosts, so xauth can resolve schos:0.
zsh loses PROMPT_SP: on a serial terminal narrower than the console its
padding wrapped and left a # above every prompt. bootcheck knows zsh's
prompt and matches without colour codes, so busybox's ls reads as text.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
make aroot builds a SchFS root from Alpine's minirootfs, the packages
ports/alpine/world lists, and SchOS itself as the signed apk package
schos-base, in its own repository inside the root. apk add installs from
Alpine; apk upgrade updates both. Booted: zsh is Alpine's, apk update
sees 28669 packages, apk add htop works, and so does xorg-server.
world.lock pins every package with its sha256; a build fetches only what
dist/alpine/ lacks and never looks for newer versions. make alpine-lock
is the one step that does.
Alpine's zsh, busybox commands and hwdata replace SchOS's copies; SchOS
keeps its own tools, toybox's other commands, and static mksh, which
schinit's recovery shell now tries first.
Also: the dependency-file include matched Alpine's usr/lib/sysctl.d, a
directory, and stopped every make. See docs/plan/alpine.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
-isohybrid-gpt-basdat does nothing without an MBR template, so the ISO
had no partition table at all: written to a stick, it was a CD
filesystem at sector 0. AMI on a B550 (and OVMF) read El Torito off any
block device and booted it anyway. An HP all-in-one from 2012 looks for
an EFI system partition on a USB disk, as the spec says to, found none,
and listed the stick under Legacy only.
The FAT image now goes in a second time as MBR partition 2, type 0xEF,
and is FAT16 by cluster count rather than FAT12, the type old firmware
reads worst. Booted under OVMF both as a CD and as a USB disk. The ISO
also rebuilds when mkiso.sh changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
tools/wallpapers.py writes them from the Shift geometry, so they follow
the mark: Graphite (the default), Strata, Porcelain (light) and Dial.
Each is a 3840x2160 SVG with a JPEG render beside it. JPEG is the right
format for gradients under grain, and the branding README now says why
this is the one exception to its no-JPEG rule.
Removed: wallpaper1 (drawn flat, and built on the retired Guru mark),
the site's unused copy of it, and the two superseded wordmark PNGs. The
root README showed one of those, and now shows Graphite.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
Sixty-five headers sat side by side in include/, so `#include "irq.h"`
said nothing about where irq.h was or what it belonged to, and errno.h,
stat.h and fcntl.h at the top shadowed the system's for any host tool
built with -I./include. They are in folders now -- arch/ core/ fs/
dev/ gfx/ net/ uapi/ (and ldls/ for the Linux driver layer) -- and
every include names the folder, as Linux's do: "core/proc.h".
Done mechanically and checked the hard way: the kernel built from this
tree disassembles identically to the one built from the tree before
it. The full image boots, and userland is the same set of programs.
Paths in docs and comments follow; docs/guides/layout.md has a table
of what is where, and tools/fix-includes.py rewrites old-style
includes from the folders in the tree, for code written before this.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
Transaction groups commit from a schfsd thread every 5 seconds, on sync
and fsync, on unmount and shutdown, and early when freed-but-pending
blocks crowd the free ones -- the contract ext4 and ZFS have. A crash
still lands on the last group whole, and loses at most those seconds.
Committing before every write(2) returned cost two cache flushes and an
uberblock a call: 4 KB writes go from 2.8 to 6.6 MB/s on AHCI, and from
66 KB/s to 3.9 MB/s on QEMU's IDE disk.
Filling the disk made the root volume unmountable, on the last commit
as well: a tree update that ran out of blocks halfway was published by
the next commit. 1% of the volume (1 to 64 MiB) is now a metadata
reserve, as btrfs and ZFS keep one; data stops at it with ENOSPC as a
short write, new names at half of it, and deletes may use the rest. A
write that runs out part way records what it wrote in the inode rather
than leaving blocks past an end of file nothing moved.
Deleting a large file goes in 4 MiB steps from the end with a commit
allowed between them, as ext4 restarts its transaction through a long
truncate. Deleting 80 MB on a full disk ran out of room halfway and
leaked the rest.
The VFS's dirunlink called the object layer's whole unlink, which freed
the object with its last name, so a file removed while open lost its
data under the reader. It removes the name only now, and the object is
freed on the last iput. An object still open when the machine went down
is found by the mount's space scan and freed, like ext4's orphan list.
The allocation bitmap has a second level of pointer pages, so the mount
ceiling is 64 TiB instead of 64 GiB, and the freed-this-group bitmap
has pages only where frees happened: a 1 TiB volume takes 32 MiB.
A freed block written after the newest snapshot is reused at once
(ZFS's birth-time rule, so bfree carries the block's generation); only
older ones are kept while a snapshot exists. Deleting one snapshot no
longer stops retaining for the others, which let their blocks be
handed out again.
Checked under QEMU: crash after sync, after 8 s and within 5 s; fill to
ENOSPC, rm, remount; unlink while open; an orphan freed at the next
mount; a 1 TiB volume. schfs-check passes, with a new test for the
snapshot rule.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
q35 stopped with "no root filesystem" because its disk is SATA, like
every PC since 2008, and SchOS only had IDE. And SchFS could only sit
on a whole raw device, so nothing could boot from a disk that also
carried an ESP.
- include/blkdev.h, kernel/dev/blkdev.c: a registry of block devices
found at probe time, numbered from 4. A disk's GPT (or MBR) is read
when it registers, and each partition becomes a device whose block 0
is the partition's first sector. SchFS needs no format change to live
in a partition; a Linux SchFS driver will see the same bytes on
/dev/sdX2. SchFS has its own GPT type:
47138FA9-B79C-47A6-9F7D-AC1F7F572D5E.
- AHCI driver, written from the AHCI 1.3.1 spec: BIOS handoff,
COMRESET when the link is down, IDENTIFY, 48-bit DMA read/write,
FLUSH. Completion is polled, so the root disk does not depend on
interrupt routing.
- USB sticks also register as usb0, so their partitions are found.
- The root probe tries registered devices between USB and IDE,
SchFS-typed partitions first.
- make disk / tools/posix/mkdisk.sh: a GPT image with a 64 MB ESP and
a SchFS partition, ready to dd to a stick or a disk.
Tested in QEMU q35: the old two-disk setup roots on sata1; disk.img as
the only SATA disk roots on sata0p2; disk.img as a USB stick (firmware
boots from it) roots on usb0p2. Writes survive sync in each case.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HH7xukt4aaiGj1nQEAoQf1
Nexoniarz is the sole author and SchOS was never published under
Apache-2.0, so this is a straight relicence. LICENSE is the FSF GPLv2 text,
every first-party SPDX header moves to GPL-2.0-or-later, and NOTICE,
README.md, docs/licensing.md and the site say so. projects/schfs-linux
follows the rest of the tree.
The GPL gives copyleft: a fork has to ship its source. It also lifts the
old "Linux drivers, never" rule, since GPL-2.0-only code combines with
GPL-2.0-or-later. "Or later" leaves room for GPLv3, but one binary still
has to pick one version, so docs/licensing.md now keeps Apache-2.0 and
GPLv3 code out of the kernel to keep Linux code possible there. BSD
4-clause code goes from "avoid" to "no", because the advertising clause
is GPL-incompatible.
LDLS still ships no .ko. That boundary is now project policy rather than
a legal requirement, and ldls.md, devices.md and licensing.md section 6
say so. refmod declares license=GPL, and ldls-check passes (27 checks).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Gate 0's last structural row. kernel/core/kmem.c: slabs are pages from
kalloc() with a header at the front, and free objects hold the free list in
their own first eight bytes.
That last detail is the whole design. **An object carries no header and
freeing performs no search**: the page base of any object *is* its slab, and
the slab names its cache. It is also exactly why an object cannot exceed a
page, which matters below.
Two interfaces. kmem_cache_create/alloc/free for a subsystem with one kind of
thing -- it gets its own line in /proc/slab, which is most of why naming it is
worth the argument -- and kmalloc/kmfree over power-of-two classes from 16 to
2048 for code with no single type to build a cache around. Each cache keeps
partial and full lists; a slab that empties goes straight back to kalloc()
rather than being kept, so /proc/slab never overstates what is held.
`make kmem-test` asserts the bookkeeping on the build machine. Same arrangement
as schfs-objtest, and for the same reason: slab lists, free-list threading, the
page-to-cache lookup and whether pages actually come back are all things that
can be wrong without a kernel to be wrong in. 26 checks, including one that a
permissive allocator cannot pass -- write a distinct pattern through 512
objects and read them all back, because two objects sharing storage shows up
there and nowhere else.
One bug the test found immediately: kmem_max_obj() returned the raw page
remainder, which kmem_cache_create() then rounded *up* past its own limit. The
largest legal size was reported as illegal.
**It does not do what the register said it would.** The claim was that every
fixed table exists to work around kalloc. Measured:
struct proc 10656 NPROC 128 larger than a page
struct socket 4928 NSOCK 32 larger than a page
struct buf 4272 NBUF 256 larger than a page
struct inode 280 NINODE 200 a cache, bounded on purpose
struct device 120 NDEVICE 32 3.8 KB, never hit
struct file 48 NFILE 512 24 KB, never hit
The three largest cannot go in a page-backed slab whatever it does. NINODE is
a cache and is meant to be bounded. NFILE and NDEVICE are small enough that
their tables are rounding errors and their ceilings have never been reached.
Freeing the big ones needs a contiguous multi-page allocator, which is a
different thing and is not written. That is now in the register and in
docs/architecture/kernel.md rather than left as a claim nobody checked.
What it does fix is real, and was measured rather than assumed:
- **A pipe is 640 bytes and came from a whole 4096-byte page.** Every `|` in
a shell command burned three and a half kilobytes it never touched. Six
pipes per page now, and /proc/slab shows the page going back when they
close.
- **procread() built its answer in a 2048-byte array on the kernel stack**,
under a comment that had warned about exactly that since it was written --
it stayed a local because a whole page for 2 KB felt worse, and kmalloc is
the answer that was missing. Its frame is 64 bytes now instead of ~2100.
- procfs's other generator took a page from kalloc for the same 2 KB.
Verified on a boot: /proc/slab tracks pipe and kmalloc-2048 through real
pipelines, allocs and frees balance, and $(), ps, time, readlink, sigtest and
rename are all still green with no traps.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
"Ą Ę Ś Ł Ż" drew as "?????" and the obvious diagnosis -- the font has no
glyphs for them -- was wrong twice over. `struct vtcell.ch` was **one byte**
holding a CP437 index, and the console mapped UTF-8 down to it on the way in.
U+0104 does not fit in a byte. No font could have fixed that; the cell had to
get wider first.
So: `ch` is a uint32 codepoint, the mapping on the way in is gone, and the cell
draws through kernel/dev/ttf.c against JetBrains Mono linked into the image.
Cell size comes from the font's measured advance instead of a hardcoded 8x16.
The grid costs 864 KB of BSS instead of 432 KB, which is the honest price of
holding a codepoint per cell for six terminals at 240x75.
The rasteriser needed three fixes before it could be trusted with this.
**21 KB of locals on a 4 KB kernel stack.** `ttf_render` held an outline (512
points as three arrays) and a raster (1024 edges) as automatics. KSTACKSIZE is
4096. They walked off the bottom of the stack into whatever was underneath,
which surfaced as user processes faulting on addresses they never touched and
then a panic inside freevm() on an overwritten page directory. Nothing pointed
anywhere near a font.
**And then again, recursively.** load_outline() held a 4736-byte outline as a
local *and recursed* for composite glyphs. Composite glyphs are exactly Ą, Ę,
Ś, Ł, Ż -- so the second crash arrived the moment the first fix let Unicode
reach the screen, looking like a fresh bug. Now a static outline per recursion
level with a depth cap of 4 (the font's deepest real nesting is 2), and the
leaf parse split into load_simple() so its 512-byte flag array is not
multiplied by the depth. Worst chain measured from the object file: ~2.3 KB.
Neither showed in tools/ttftest, which has an 8 MB stack. That is the standing
hazard of one source file built for host and kernel: the host is most forgiving
about exactly the resource the kernel is strictest about.
**Component transforms were being skipped**, on the stated theory that a wrong
transform is worse than none. It is not. `‘` in this font is the comma glyph
with a scale of -1 -- a 180-degree rotation -- and the offset that ships with
it only lands correctly after the rotation. Without it the mark went twelve
pixels into a ten-pixel cell and was clipped away entirely, so the character
read as missing from the font rather than as misplaced. F2Dot14 2x2 now, with
SCALED_COMPONENT_OFFSET honoured.
The glyph cache was ASCII-only, so every accented letter re-rasterised on every
repaint: a screenful of Polish did hundreds of bezier flattenings per scroll
where the same screenful of English did none. 95 reserved ASCII slots plus 224
shared slots keyed by codepoint, 64 KB pool. Verified 13680 cached renders
byte-identical to direct ones across collisions and a pool flush.
tools/ttftest speaks UTF-8 now. Iterating bytes meant it could only ever ask
for ASCII, and ASCII is entirely simple glyphs -- it could not reach the code
path that mattered.
Also: resolvconf's success line moves behind /proc/bootmode, same rule as
vgpud. A quiet boot opened with "resolvconf: nameserver 10.0.2.3" where the
one line was supposed to be "Welcome to SchOS!". Failures stay loud.
Verified on screen, not just on the host: Ą Ę Ś Ł Ż ć ń ó ź, Ω λ π Œ Ð Æ ß,
∞ ‘x’ → ≤ all render, panic=0 across four boots.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
schdbgd bridges TCP to the debugger, which is the one way in that does not
need you to be sitting at the machine. It is a bridge and nothing else: a line
in becomes a line written to /dev/dbgro and the answer comes back, so `nc` is a
complete client and there is no clever client to be broken on the day you need
this.
# schdbgd &
schdbgd: debugger on port 4444, loopback only
$ nc 127.0.0.1 4444
dma
arena 0xffff80003da41000-0xffff80003ea41000 16384 KB total, ...
A kernel debugger plus a TCP socket is a remote root hole, so three things were
decided rather than discovered:
- It does not start by itself. schdbgd.job ships Disabled. SchOS has no
kernel command line, so the boot flag the plan asked for is the job
manifest, which is the same thing in this system's own idiom.
- It binds loopback; -a names an address and says so in a paragraph.
- It cannot reach a view that changes state, enforced in the kernel.
That last one is /dev/dbgro: /dev/dbg on minor 1. The minor is the one piece of
per-open state a character device already sees, so which node you opened is the
permission -- nothing to latch, nothing to remember between the write and the
read, no way across. Which views are refused comes from dbg_register_rw() on
the registration itself, because a central list of dangerous views goes stale
in the direction where a new view is quietly treated as safe. Today that is
`watch <addr>`; `watch` with no argument is still the listing and still
answered.
Also: socket() now accepts IPPROTO_TCP with SOCK_STREAM and IPPROTO_UDP with
SOCK_DGRAM. They name the default rather than departing from it, and toybox's
nc writes them out -- so TCP was "Protocol not supported" on a system whose
whole network is TCP.
docs/plan/debugger.md retires into docs/architecture/debugging.md; 16 files
repointed. Every plan in docs/plan/ that was open is now closed, so the next
thing is Gate 0 itself: the survey, with the instrument built.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
L4 was listed as "free -- H2 built all of this" and never actually tested,
which is a claim rather than a result. Tested now, and it holds:
ldlsd running ping 2/2; holds index 7 and DMA slots 3 and 4
killed interface DOWN, device released, bus mastering
cleared, DMA reclaimed -- back to vgpud's 3 blocks
ping fails honestly: sendto: -5
a second ldlsd ping 3/3, and the arena hands back the same addresses
pciuser_release_all() and netuser_release_all() sit beside each other in
exit() for the same reason, and that they apply unchanged to a foreign driver
is the whole point of having built them against our own.
With that, every milestone on ldls.md is done, so it leaves docs/plan/ for
docs/architecture/ -- the rule docs/plan/README.md states and the one that
keeps the two directories meaning different things. The new document describes
what is, not what was planned: the five rules, the four pieces, what it costs
(polling, no stack protector, locks that are no-ops), that it can be killed,
and a "what it does not do" that leads with one driver, one kernel version,
tried once.
Sixteen source files pointed at docs/plan/ldls.md and would have been dangling
the moment it moved. All repointed, and the milestone references -- which the
architecture doc deliberately does not have -- rewritten to name sections
instead, so a reader lands somewhere real.
The register row closes with it. hybrid.md freed a slot, ldls.md took it, and
it is now free again.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
docs/plan/ldls.md milestone L2, less the half that needs /dev/pci.
In Linux, __pci_register_driver ends at the PCI core, which matches a
driver's id_table against the bus and calls probe(). There is no PCI core
here, so LDLS keeps the pointer that call was given and plays the part.
To do that it must know where pci_driver.probe is, and it asks the kernel
rather than being told by us. ldls/btf.c reads /sys/kernel/btf/vmlinux --
122,279 types, every struct layout and every function prototype, as generated
data describing the kernel that is installed. Implemented from
Documentation/bpf/btf.rst, which is a published specification.
$ ldls-types /sys/kernel/btf/vmlinux --need
pci_driver probe offset 16 size 8
pci_dev resource offset 1000 size 1088
resource start offset 0 size 8
all present
Cross-checked against pahole on the same file, field for field. No offset in
this is ours, which is rule 3's uncomfortable row disappearing rather than
being argued about.
And then the driver:
driver name "8139too"
devices it claims: 10ec:8139 ... and 16 more
calling probe(pci_dev 10ec:8139, ...)
1 alloc_etherdev_mqs(0x368, 1, 1)
2 pci_enable_device / 3 pci_request_regions / 4 pci_set_master
5 pci_iomap(pdev, 1, 0)
6..544 ioread8 / iowrite8 -- 0x88, 0x8a, 0x8c, 0x8e, over and over
545 dev_addr_mod / 547 netif_napi_add_weight_locked
549 timer_init_key / 550 register_netdev / 551 netdev_info
probe returned 0 after 552 stub call(s)
10ec:8139 is the real RTL8139 id. It mapped BAR 1, the one marked
IORESOURCE_MEM. Those 538 alternating byte accesses stepping 0x88/0x8a/0x8c/
0x8e are serial EEPROM clocking -- the driver bit-banging the chip for its MAC
address and handing the result to dev_addr_mod. Then NAPI, a watchdog timer,
register_netdev, its "found it" line, and zero.
Fifteen distinct symbols carry that entire probe, and three of them are 540 of
the 552 calls. L1 said 44 symbols carry 110 modules through init; L2 says 15
carry this one through probe. Neither is a number anyone would have guessed
and both came from running the thing.
Recorded as not done, because it is not: probe receives a pci_dev whose shape
is right and whose contents are invented. The BARs are made up, pci_iomap
returns a zeroed page, and 8139too read a MAC address of zeroes and was
perfectly happy about it. The "all through /dev/pci" half of L2 belongs to the
SchOS side. Also recorded: BTF has to come from somewhere, and SchOS has no
/sys/kernel/btf/vmlinux -- the user brings it with the module, from the kernel
the module was built for.
make ldls-check is 27 checks, four of them L2's, and it says so out loud when
it skips them on a machine with no BTF rather than passing silently.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
docs/plan/ldls.md milestone L1. Every undefined symbol resolves to a
trampoline -- twenty generated bytes that load the symbol's index and jump to
a shared thunk, which saves the argument registers and reports the call. Then
init_module is simply called.
$ ldls-run 8139too.ko
module 8139too
license GPL
wants 94 symbols
calling init_module at 0x416fc020
1 _printk(0x416fb6c0, 0, 0)
| 8139too: 8139too Fast Ethernet driver 0.9.28
2 __pci_register_driver(0x416fb400, 0x416fff80, 0x416fbdbb)
init_module returned 0 after 2 stub call(s)
That second line is the driver's own banner, printed through a _printk that
is ours, from a string in the module's own .rodata that a relocation put
there. There is no stronger evidence that a load worked than the loaded thing
speaking.
Stubs return zero rather than stopping, which departs from what the milestone
originally said and does so deliberately: stopping at the first call tells you
one name, carrying on tells you the sequence, and the sequence is the thing a
symbol table cannot give you. A driver that then dereferences the zero faults,
and that is a result too -- it says how far it got before it needed the kernel
to have really done something.
The first run failed on neither of those. It failed on a relocation:
ldls-run: relocate: a signed 32-bit relocation does not fit
A module is compiled -mcmodel=kernel, so the compiler emits R_X86_64_32S for
anything it believes lives in the top 2 GB, and an ordinary mmap at 0x7f...
has nowhere to put it. Linux gives modules a region within 2 GB of the kernel
text; the harness uses MAP_32BIT to reach the same window from the other end.
On SchOS this costs nothing -- a process's mmap area starts at 0x40000000 and
is already inside it -- but it is a real constraint on where the host may load
a module, and a loader that did not check its own arithmetic would have found
it much later and much more confusingly.
Swept over 500 stock modules: 449 ran init to completion, 22 faulted on a
stub's zero, 29 have no init_module, and zero hit a loader error.
Recorded honestly in "what it will not do": running init is not working. Every
symbol is a stub, so what has been shown is that drivers ask for things in a
readable order, not that anything answers. Also recorded: every undefined
symbol in a module is NOTYPE, so the stubs cannot tell jiffies from
pci_iomap and both get a trampoline -- BTF distinguishes VAR from FUNC and
that is where the fix belongs rather than in a guess.
make ldls-check is 23 checks now, five of them L1's, all against
ldls/test/refmod.c which is ours and now has an init_module of its own.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
docs/plan/ldls.md milestone L0. ldls/ko.c indexes the sections of an ELF64
relocatable object, places the allocatable ones, resolves undefined symbols
against a caller's table and applies the relocations. Nothing is executed.
It is freestanding -- no libc, no allocation, no I/O -- so the same source
builds as a host tool now and as the SchOS-side host later. That is the
arrangement mkschfs and the kernel already share, and for the same reason:
the failures at this layer are format failures, and one found on the host
costs a second instead of a boot.
$ ldls-inspect 8139too.ko
module 8139too
license GPL
loads to 33988 bytes
relocations: 1980
R_X86_64_64 363, PC32 991, PLT32 428, 32S 169, PC64 29
undefined symbols: 94
stops at: unresolved: __fentry__
The licence is printed before anything is loaded, because it is the one
property that decides whether the object may be on the machine at all and it
is a plain string in .modinfo. Rule 1 is easier to keep when the answer is on
screen.
94 symbols is the measured API surface of 8139too on 6.18.46 -- the list,
exactly, rather than an estimate -- and it is L1's work queue. It is smaller
than it looks: __fentry__, __stack_chk_fail, __ref_stack_chk_guard,
__x86_return_thunk and __fortify_panic are compiler and instrumentation
artefacts that stub out in a line each.
Swept over every module in a stock 6.18.46 tree: 2,440 objects, all of them
laid out and relocated to completion, none with a relocation type the loader
does not handle. That sweep is also what retired the fixed section table --
nvidia-modeset has 7,069 sections and 74,305 relocations, against a median of
50 -- so the caller supplies the table and there is no limit to run into
rather than a generous one that fails later.
Loading is not running, and the doc says so: not one symbol is implemented,
and the distance between "relocates" and "transmits a packet" is the rest of
the plan.
No .ko is in this tree and .gitignore now refuses one outright. make
ldls-check runs 18 checks against ldls/test/refmod.c, which is ours, written
to contain one of each thing the loader has to get right.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
docs/plan/debugger.md, milestones D0, D1 and D2. What they replace is the
ninety-second loop -- add a cprintf, rebuild, boot, read the serial log --
which answers exactly the one question you thought of before the rebuild.
The rule the shape follows: **it observes, it does not control.** No
stepping, no software breakpoints, no stopping the world. Half the bugs
this kernel produces are timing -- an interrupt storm, a level-triggered
line nobody acknowledges, a driver that faults only on its second start --
and every one of them disappears when you halt the machine to look. And
the layering rule, which is the circularity rule the display already has:
a debugger that needs lwIP cannot debug lwIP.
D0 -- symbols. tools/ksyms.pl turns the first link pass into a table the
second pass carries. Two passes is enough because .ksym is placed after
.text and .rodata, so carrying the table cannot move anything recorded in
it; the build lists both passes' text symbols and refuses to link if they
ever disagree, because a symbol table that is subtly wrong gives confident
false answers. ksym.c reads it through weak symbols, so pass one -- which
has no table -- still links. A crash chain now reads panic+0x5f <-
proc_sysrq_cmd+0x99 <- ... <- syscall+0xf6.
D1 -- a log that outlives the kernel. Every character through consputc()
lands in a byte ring with a timestamp per line, readable as /proc/kmsg.
It sits in a region carved before the page allocator gets anything, with a
magic header, so a kernel finds the previous kernel's log still there --
including the whole PSOGM report. panic.c's own comment said "there is no
core dump and no log to go back to"; that is no longer true.
The obvious place for it is the top of RAM beside the DMA arena. That
was the first attempt and the magic came back as ASCII on the reboot,
because the top of usable memory is exactly where the firmware makes its
own allocations. It lives above the kernel image instead, which is a
stable address across boots of the same build.
D2 -- the introspection registry. A subsystem publishes its state with one
line at the bottom of the file that owns the data:
dbg_register("dma", dma_dump, "the DMA arena's free extents");
Seven views -- dma, mem, proc, vma, claims, pci, sock -- reachable three
ways: sysrq 'g' (always, no process and no filesystem), /dev/dbg (echo a
line, cat the answer), and schdbg. The protocol is lines of text because
the same bytes have to work where nothing can allocate or frame a record,
and because a protocol you cannot drive by hand is one you cannot debug.
struct sbuf moved out of kernel/dev/procfs.c into include/sbuf.h rather
than being written a second time.
Several views exist because of a specific bug. `vma` is the table that
decides whether exit() frees a page, and the DMA ownership bug that made
this worth building was a block that was never in it.
Also fixed, found by building with clang for the first time in a while:
struct iovec was defined in both posix.h and socket.h, and every file that
included both was a redefinition. GCC 15 defaults to C23, which permits an
identical struct redeclaration, so the gcc path never noticed -- while
clang defaults to gnu17 and rejected it, breaking the one toolchain
shell.nix exists to provide and the UEFI loader requires. Both are now
guarded.
D3 (`whose <addr>`, the cross-subsystem ownership query) and D4 (hardware
watchpoints on DR0-DR3) are not built and are described honestly as
missing in the architecture doc's "What it does not do".
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
Shift is the logo, Guru is the icon, SchCotie is the mascot, and the difference
between them is not stylistic. An icon works once, at one size, in one state. A
mascot has to be able to *do* things -- confused, pleased, asleep, seen from
behind -- which is why it costs more and is drawn last. Written down because
this is exactly the kind of decision that gets re-derived badly.
Assets/ becomes docs/branding/, which also fixes two things:
* Filenames had spaces in them, so every reference needed quoting. They are
lowercase and hyphenated now.
* Assets/ held hand-written SVG and generated PNG in one directory, which is
the one rule docs/guides/layout.md states.
The guide records what was previously only in people's heads:
* The constraint everything answers to -- include/logo.h is a 128x128
single-channel coverage mask, so a mark has to survive as a solid shape in
one colour and be recognisable at 24 px. Colour is the last decision, never
the first.
* Guru's purple is load-bearing. PSOGM_BG #4B1D9A is deliberately not the
accent #7C5CFF, and theme.h's rule is that the full-bleed purple belongs
entirely to failure. If Guru appears in accent violet on a sticker, the deep
purple stops meaning "the machine stopped" -- and it is the only colour in
the system that means anything.
* Why shift-mask.svg is a separate file from shift.svg. mklogo.py reads ink as
(255 - luminance) * alpha, so tones survive as coverage and the relationship
inverts: darkest here is brightest on screen. Copying the violet fills would
light the object from below.
* Why a raster is committed beside its SVG. inkscape is not a build
prerequisite, so requiring a vector rasteriser would break the build on most
hosts for a file that changes about once a year.
* That the BSD daemon is copyrighted and SchCotie must be drawn from the
concept rather than from Beastie. A daemon is a Greek daimon and a Unix
background process; neither is ownable. The pitchfork, the sneakers and that
face are.
Format rule: SVG, or PNG at high resolution with no lossy compression, nothing
else. SVG is the source of truth for every mark and a raster is generated from
it, never edited.
include/logo.h regenerates byte-identical from the new path -- only the source
comment differs. The two old wordmark SVGs were deleted outside this session and
are left deleted; their PNGs are kept as superseded-*.png for reference.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
A leaf's value area holds variable-length records end to end, so a record begins
wherever the one before it finished -- a multiple of 8 only by luck. The code
reads them by casting a pointer into the block, which tells the compiler they
are aligned when they are not.
That works on x86, which is why it has never shown as a bug, and it is undefined
in C. A compiler entitled to assume 8-byte alignment may emit an instruction
that requires it, and the failure would arrive at a different optimisation level
or on a different architecture looking like filesystem corruption rather than
like codegen. UndefinedBehaviorSanitizer reported 31 of them across obj.c,
objw.c, attr.c and space.c.
Seven types now carry SCHFS_PACKED: dirent_rec, extent, extref, index_rec,
attr_rec, snap and inode. Two of those -- attr_rec and inode -- were never
reported, because schfs_objtest does not reach an unaligned instance of either.
They were latent rather than absent, and casting into a leaf is the same lie
whether or not a test has caught it yet.
NO ON-DISK LAYOUT CHANGED, and that was measured before the annotation was
applied rather than assumed: every one of these already carries explicit `pad`
fields and therefore has no implicit padding, so sizes and member offsets are
identical with and without it. Seven _Static_asserts now pin the sizes, because
a record that silently grows misreads every existing volume from that point on
and looks like corruption rather than like a struct that changed.
schfs_node_hdr, schfs_item, schfs_kptr and schfs_uberblock are deliberately left
alone. The node header is exactly 32 bytes, so the items and key pointers after
it are already 8-aligned, and annotating them would cost byte-wise access on the
tree's hottest paths to fix a problem they do not have.
Verified by reading an image written before the change: schfs_fsck --check
passes on structure, checksums, key ordering, data blocks and derived free
space, and the kernel mounts its own root unchanged.
Also here: mkschfs passed a null pointer to qsort for an empty directory. Every
implementation ignores it when the count is zero; the standard does not say they
have to.
make schfs-check now runs with no sanitizer output at all.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
Started as "fix the ASAN leak in schfs_objtest" and the test turned out to be
the messenger. The allocation map was taken through a palloc callback and there
was no way to return it: no schfs_space_release, and a comment in
schfs_space_init reading "caller frees what was taken" for an API the caller
did not have.
Three separate leaks from that one gap:
* The kernel leaked the entire map on every unmount -- up to 2 MB, at 8 KiB
per GiB of volume -- because kmount_unmount flushed, cleared `mounted` and
walked away. This is the one that matters; the others are test-only.
* schfs_space_init stranded a page on its own error path. npages was raised
only after *both* of a pair of allocations succeeded, so a failure on the
second left the first outside the count -- unreachable to any cleanup that
could later exist.
* schfs_objtest leaked 2121728 bytes in 518 allocations, which is what made
the whole SchFS suite exit non-zero regardless of its assertions.
The map now carries its own pfree rather than taking one at release time, so it
can only ever be freed by the allocator that filled it -- the kernel's pages
come from kalloc and a host tool's from calloc, and the two are not
interchangeable. schfs_vol_open and schfs_vol_format release any map the volume
is already holding before building another, which is what stops a re-open
leaking; struct schfs_vol must therefore be zeroed before first use, and the
header now says so.
The test's own leak was the last one found and the simplest: it does
memset(&VOL, 0, sizeof(VOL)) before reopening, which wiped the map's pointers
before vol_open could release them. Eight of those, each now releasing first.
schfs_objtest asserts the balance -- 520 pages taken, 520 given back -- because
a suite that leaks cannot detect a leak: the filesystem's own allocations would
arrive as noise inside the harness's. The next one fails the run instead.
make schfs-check now exits 0. The 31 UBSAN misalignment reports are still
there and are still a Gate 0 item.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
uber_valid() answered one question -- valid or not -- and a version mismatch
returned the same as a failed checksum. Those two states need opposite handling:
A damaged uberblock means STEP OVER IT. That is what the 128-slot ring is for,
and the older root is a consistent filesystem. This is crash recovery.
A sound uberblock from a newer format means STOP. Falling back mounts a STALE
filesystem: every commit made since the volume was upgraded is discarded, the
mount reports success, and nobody finds out.
Demonstrated rather than asserted. A forged image with two uberblocks --
(version 1, generation 1, root 129) and (version 2, generation 2, root 130) --
made the old code mount root 129 and exit 0, silently rolling the filesystem
back to before the upgrade.
uber_check() returns BAD / OK / RO / TOO_NEW, and asks its questions in order:
is it ours (magic), is it intact (crc), can we read it (version, features), does
it make sense (root in range). Compatibility is asked before sanity on purpose --
the fields a sanity check reads are only meaningfully placed if the format is one
we know.
Plus feature flags in three classes, which is what stops a version number being a
blunt instrument: compat (ignore unknown bits), ro_compat (mount read-only),
incompat (refuse). The test for which class is one question -- what happens if a
driver that does not know this feature writes to the volume? Nothing -> compat.
Something goes stale -> ro_compat. Something is unreadable -> incompat.
They occupy what was reserved0, so every later field keeps its offset and
existing images -- which have zeros there -- read unchanged. schfs_vol_commit
memsets the uberblock before filling it in, which is why v1 volumes already say
"no features" without having been written by code that knew the field existed.
That trick is available exactly once. Four _Static_asserts now pin the layout.
Verified on forged images: version bump refuses, incompat refuses, blocksize
mismatch refuses, ro_compat mounts the NEWEST root read-only rather than falling
back.
Also here, because they are the same file: SchFS's transaction owner is a pid
alongside the pointer, since a bare struct proc * does not identify a process --
ptable slots are recycled, and an exited owner's pointer compares equal to
whichever unrelated process lands in that slot next, at which point klock() skips
the lock entirely and two callers share one volume's scratch pages. And
schfs_lockinfo(), so /proc/fsstat can report a wedged volume instead of leaving
every path asleep in acquiresleep with no way to ask why.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
The TCP/IP stack reached straight into the first driver anyone wrote.
net.c called e1000_present(), the lwIP glue was named netif_e1000.c and
called e1000_transmit(), and include/defs.h declared the driver's
internals to the whole kernel. The second NIC driver would therefore
have arrived with a near-identical copy of the glue file and a patch to
the protocol stack, which is the arrangement include/device.h was
written to prevent -- the e1000 even said so, with `.ops = 0, // no
class vtable yet; there is no net class`.
There is now. A driver publishes a struct netdev_ops -- transmit,
receive, mac, link_up -- and is finished: it never mentions lwIP, and
nothing above it learns its name. The display class was already this
shape; this is the same shape for networking, and the two now read the
same way in kernel/dev/device.c.
What that made possible immediately:
- One netif per bound device instead of one per driver. lwIP's glue is
netif_eth.c, it takes a struct device, and it holds up to NETIF_MAX
interfaces. net.c attaches every network device it finds and starts
DHCP on each, because which interface the network answers on is not
ours to decide.
- /proc/netif lists all of them, one block of fields each, with the
interface name and the driver that claimed it.
- The ARP self-test moved from the e1000 to the class. Nothing in it
was chip-specific, and it is the first thing anyone will want to run
against the next driver. Still `echo n > /proc/sysrq`.
- Private headers live with their driver: e1000.h was in include/,
where exactly one file could use it. The driver build now adds the
driver's own directory to the include path.
The e1000 also drives more than one card. Its probe() refused the second
outright, because there was a single static instance -- so a machine
with two of them drove one and silently ignored the other. Instances are
an array now, and the instance is hung on dev->priv by a new optional
`attached` hook that the bus calls once the device exists. probe() runs
before device_add(), so there was previously nowhere to put it; that is
the whole reason the hook exists rather than a line inside probe.
QEMU gets an explicit NIC on every target (QEMU_NET). The default device
happens to be an e1000 on this machine type today, which is exactly the
sort of thing that stays true until a QEMU release changes it and the
network stops being exercised without anyone noticing.
Verified:
- bootcheck -Net passes, 48 [ok], no failures -- including the moved
self-test and two new checks that /proc/netif names both the
interface and the driver behind it, which together say the stack
went through the class rather than through a driver it was compiled
to know.
- Two NICs attached by hand (-device e1000 twice, second with its own
MAC): eth0 and eth1 both attach, each reports its own MAC in
/proc/netif, and each gets its own DHCP lease. Before this commit
the same machine produced one interface.
- make schfs-check still passes in full.
One trap worth recording: moving a header broke the build for anyone
with stale .d files -- make stopped with "No rule to make target
'include/e1000.h'". CFLAGS gains -MP, which emits a phony target per
header so a moved or deleted one can never wedge the build again.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The second of the two things the stack owed. recvfrom accepted the
address arguments and ignored them, so every caller was told nothing
about where a datagram came from -- which for UDP is most of the point:
a server that cannot see the sender cannot reply to it.
Fixing it properly meant fixing something worse that was sitting
underneath. UDP payloads went into the same byte ring TCP uses, with no
record of where one datagram ended and the next began, so two datagrams
that arrived together came back from one read as one. That is a bug
whose appearance depends on timing and whose blame lands on the sender.
Datagrams now carry a header alongside the bytes: source address, source
port, length. Sixteen of them per socket, in a ring beside the existing
one. Whole buffers would have been 16 sockets x 16 datagrams x an MTU of
static memory for something almost always empty; only the boundaries need
keeping, and those are eight bytes each. A datagram too big for the
buffer it is read into is truncated and the remainder discarded, rather
than left in the ring to be delivered as the head of the next one. One
that arrives with the queue full is dropped whole -- that is UDP's own
contract, and half a datagram in the ring would corrupt every one after.
An empty queue and a zero-length datagram are kept apart, because a
sender really can send zero bytes and the right answer to that is 0
rather than "nothing has arrived yet".
sendto grew the mirror image: a destination address on a datagram
socket, which is what a server replying to whoever just wrote to it
needs. On TCP a destination is ignored rather than refused -- the
connection decides where the bytes go, and programs pass one out of
habit. For a stream, recvfrom's source is read from the pcb: it is the
peer, and it is the same answer for every byte.
Verified: bootcheck -Net now covers UDP too, 45 [ok] and no failures.
The address check is real rather than cosmetic -- nettest deliberately
leaves its sockaddr uninitialised, so a kernel that filled in nothing
would print stack rubbish instead of 10.0.2.2 and the port the responder
is actually on. The boundaries are checked by byte count: the responder
sends 20 bytes and then 16, and the guest reports reading 20 and then
16. Treated as a stream the first read would have taken all 36.
Left for later, and deliberately: accept() still does not report the
peer's address, and getsockname/getpeername do not exist.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The TCP/IP stack landed with its end-to-end test performed by hand: start
a listener, boot, type nettest, read the output. That is evidence once
and nothing afterwards, and it is exactly the kind of check that quietly
stops being run.
bootcheck -Net does the whole exchange. A listener runs on this host on a
port picked by binding one and letting go, the guest is told to connect
out to 10.0.2.2 at that port, and both ends are asserted: the guest
reports what it received, the listener reports what arrived. One side
alone proves much less than it looks -- a stack that completes a
handshake and then drops every byte satisfies any check made from inside
the guest.
The listener is a background job, a separate process, because this
harness has one hard rule: nothing may block the writer feeding QEMU's
stdin. A socket accept is precisely the wait that deadlocked two earlier
attempts at reading QEMU's output. It answers the protocol nettest
speaks and then closes -- the close is load-bearing, since nettest reads
until end of file and a listener that stayed open would hang the guest
rather than fail it.
The NIC is spelled out (-netdev user + -device e1000) rather than left to
QEMU's default. Other runs can take the default and still answer "did a
driver bind"; a run that asserts on an address should be reading its own
configuration.
/proc/netif is new, and the address is why. Until now the only report was
a line on the console at boot, which userland cannot read and a test can
only find by scraping the transcript. It gives interface, MAC, state,
address, netmask, gateway, MTU and frame counts. Named netif rather than
net on purpose: /proc/net is a *directory* on Linux, and a plain file
there would give ENOTDIR to anything reaching for /proc/net/dev, which is
worse than not having it. The counters are kept at the lwIP seam rather
than in the driver, because a frame the driver received and the netif
dropped never reached the stack.
Verified: 38 [ok], no failures. Eleven of them are new -- lwIP up, DHCP
obtaining 10.0.2.15 from SLIRP (a number nothing in the guest knows in
advance, so it is the exchange completing rather than a guess), the four
/proc/netif fields, frames counted in both directions, the handshake, the
reply read back, and the listener confirming HELLO-FROM-SCHOS arrived.
One bug found by writing it: $netPort was chosen after the command list
was built, so the guest ran `nettest 10.0.2.2` with no port and printed
its usage. An unset PowerShell variable interpolates to nothing rather
than failing, which is worth remembering next time a command in this
file looks right and behaves oddly.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The last three phases of the filesystem, which were written together and
touch the same files throughout -- kmount.c, objw.c and vfs.c are each
rewritten by all three -- so they land as one commit rather than three
that could not be built separately.
Durability. A transaction that reaches the platter out of order is worse
than one that never reached it: the uberblock can name a tree whose
blocks are still in the drive's write cache. bdirty/bflush give the
buffer layer an explicit barrier, ideflush issues ATA FLUSH CACHE and
usb_storage_flush the SCSI SYNCHRONIZE CACHE that means the same thing,
and a commit now waits for the data before it writes the root that
points at it. The uberblock groups are read newest-first with fallback,
so a volume whose newest roots are torn mounts at an older one and says
which generations were lost.
Reflink. Extents carry a refcount, so two objects can name the same
blocks and neither frees them while the other holds them. Exposed as
FICLONE -- Linux's ioctl value, so `cp --reflink` works unmodified --
and as cmd/reflink for the case where there is no cp to hand.
Attributes, and the one genuinely new idea here. attr.c stores named
values on an object, reachable through setxattr/getxattr/removexattr at
Linux's syscall numbers. On top of that, schfs_index declares an index
over an attribute name and schfs_query answers "which objects carry this
value" from the B-tree rather than by walking. A query against an
unindexed attribute is refused with an error, never silently answered by
a full scan: a query that quietly becomes a scan is worse than one that
admits it cannot help. cmd/schattr drives both.
Three bugs fixed on the way, each found by the numbers being wrong
rather than by inspection:
- statfs's f_fsid was declared int64[2]. Linux's fsid_t is two *ints*,
so every field after it landed one slot late: userland read f_frsize
out of f_namelen, got 255, and df reported a 32 MB root filesystem
as 1992 KB. It hid for as long as it did because the fields used to
be invented constants, so wrong numbers looked like the usual ones.
- struct inode's nlink was a short against SchFS's uint32, so a file
past 32767 names came back negative -- and iput frees an object
whose count reaches zero. It is an int now, signed on purpose, so an
underflow lands somewhere a check can see rather than wrapping.
- filewrite split every write to fit v1's log. That limit belongs to
the journal, not to writing, and SchFS makes one transaction of
whatever it is handed: a 1 MB write was becoming 170 commits and
170 flushes for nothing. fs_ops carries max_write now, and SchFS
sets no limit.
Also here: a real mount table, so `mount -t schfs /dev/hdb /mnt` shows
up in /proc/mounts -- df looks a path up there before it calls statfs at
all, so a mount missing from that list was invisible to it -- several
volumes mounted at once, snapshots mounted by name, and /proc/fsstat
with an rdtsc-based counter, which is the only instrument fine enough to
see inside a single syscall.
Verified: `make schfs-check` passes in full, including the sweep that
crashes the volume at each of 119 points mid-transaction and requires
every one to mount with exactly its committed data, and the case where
both newest uberblocks are rubbish and the volume still comes back. The
headless boot check passes with 27 [ok] and no failures, on a SchFS root.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reported from a real laptop booting through Ventoy: the F8 menu appears,
selecting Boot SchOS gives a black screen and a white dash in the corner.
Both halves of that are bugs.
The disc could not be chainloaded. Booting it directly, firmware runs the
El Torito FAT image and the loader opens \schos\kernel on that volume,
which is where mkiso.sh put it. Ventoy does not use El Torito at all: it
mounts the ISO 9660 tree and runs \EFI\BOOT\BOOTX64.EFI from there. The
loader then looks for the kernel on the volume it was itself loaded from
-- the ISO tree -- and the kernel was only ever inside the FAT image. It
is now in both. The comment in mkiso.sh had even described this failure,
as the reason the kernel is in the FAT image, without noticing the ISO
tree had the same requirement.
The error was invisible. print() redirects to the serial port once the
splash is up, which is right for a development machine and useless on a
laptop: the loader died with a perfectly good explanation and sent it to
a wire nobody was holding. What is left on screen is the firmware's own
cursor -- the white dash. die() now draws the message on the framebuffer
too, over a cleared band so it does not land on the logo.
Verified by reproducing the exact failure: the ISO tree extracted and
booted as a plain volume, as a chainloader presents it. With the kernel
removed it dies visibly with "no kernel at \schos\kernel or \kernel, EFI
status 0x000E" -- EFI_NOT_FOUND, which is what the laptop was hitting
silently. With it present, that same volume boots through to a shell.
110 [ok] on the ESP path, the ISO check passes, no failures.
ui_panel was written for this screen during the loader restyle and then
never called -- dead code sitting in boot/common/ui.c waiting for the row
that would use it. This is that row: F8 -> System Information, showing
firmware vendor and revision, UEFI revision, CPU brand, usable memory,
current mode and whether ACPI was found.
Everything is read when the screen is drawn rather than captured at
startup, because the resolution row above it can have changed the mode,
and a panel still reporting what the machine booted with is worse than no
panel at all.
Two facts are deliberately read through the same code that feeds the
kernel -- efi_type_to_e820() for what counts as usable memory, and
find_rsdp() for ACPI. A panel with its own idea of either would be a
second implementation to keep in step, and the ACPI line exists precisely
because a missing RSDP means no MADT, no IOAPIC and no SMP, which has
already been debugged once from the far end.
The screenshot caught a bug in the screen it was taken to verify: UEFI
revision read "21587.2286". EFI_TABLE_HEADER begins with a UINT64
Signature, so Revision is at offset 8 and I had read offset 4 -- the
upper half of "IBI SYST", formatted as a version number. It now reads 2.7
under OVMF, which is correct.
Verified by driving the menu over the QEMU monitor and reading the
pixels: five rows with the hint updated to 1-5, the panel centred, and
every value cross-checkable -- EDK II, firmware 1.0, UEFI 2.7, "QEMU
Virtual CPU version 2.5+" against -cpu qemu64, 1018 MB of the 1 GB given,
1280x800 matching what fbshot captures, ACPI present. The highlight sits
9 px inside both panel borders, symmetric.
F8 has a 1.5 s window, so the capture presses it repeatedly through boot
rather than guessing when OVMF finishes -- and only F8, since any other
key breaks the wait and boots.
Also: every ESP drive now says format=raw. QEMU warned about probing the
format on each of the fifteen or so places that attach it, which is new
noise on what is now the only boot path. 110 [ok], no failures, no
warnings.
./mkiso.sh from tools/posix/ failed with "missing build/kernel/kernel --
run make first" on a tree that was built perfectly well: the paths were
relative to the caller's directory rather than the repo. mkiso.sh and
schcom_test.sh now resolve the root from their own location, the way
memory-sync.sh already did. An output path given to mkiso.sh is made
absolute first, so `mkiso.sh out.iso` still writes where the caller is
standing and not where the repo is.
mutate.sh had it worse: it began with `cd /mnt/d/DEV/SchOS`, an absolute
WSL path from the Windows machine it was written on, so it could not run
against any other checkout at all.
Fixing that exposed why it mattered. Every one of its nine mutations
reported "rejected at compile time" -- and none of them had reached a
compiler. The host tools moved to build/tools/ during the bt/ -> tools/
migration and this script still asked for build/mkschfs, so make answered
"No rule to make target" and the harness read that as the mutation being
caught by the build. A mutation test whose every case is silently skipped
is precisely the thing its own header warns about: a test that cannot
fail, believed anyway.
With the paths corrected all nine are caught by the suites, which is what
this was always supposed to demonstrate:
file_read ignores the end-of-file clamp -> extracted file differs
dirent_walk advances past the next entry -> entries lost
dir_lookup returns the wrong objectid -> fsck: no such file
extents point one block past their data -> mkschfs write fails
an overwrite drops the tail of the split -> FAIL: CoW keeps the run
truncate leaves stale bytes in the tail block -> FAIL: reads as zeros
committed frees never return to the free map -> FAIL: create hog -> -3
hard links do not bump nlink -> FAIL: nlink=1, expected 2
data checksum covers only the first 16 bytes -> FAIL: tree walk
The build failure is now printed rather than swallowed, so the next time
a target is renamed it says so instead of reporting nine passes. Two
mutations also reported a bare "caught:" with no reason -- the suite had
failed but nothing matched the quote patterns -- so the fallback is now
the last thing the suite actually said.
schfs-check and schfs-test still pass, and the tree restores clean.
1449 lines of boot/bios/, tools/sign.pl and tools/padstage2.pl, plus the
build rules, the disk image and the ISO entry that existed to carry them.
Everything since roughly 2012 boots UEFI, and CSM -- the firmware-side
BIOS emulation that lets a legacy loader run on such a machine at all --
has been going away since Intel dropped support for it in 2020. So this
is not a path being retired early; it is one whose remaining hardware is
already the exception.
The cost was never the code, it was the tax on everything near it. Stage
2 had a 65-sector ceiling it overflowed silently -- 25440 bytes against
24576 -- and because the kernel is written at sector 1 + STAGE2_SECTORS,
an overflowing loader was overwritten by the very kernel it was loading.
The resolution list could not be shared with UEFI either: VBE mode
setting is INT 10h, so matching it needed unreal mode, which is the
riskiest code in the tree for a feature that already worked elsewhere.
What replaces schos.img is the staged ESP, handed to QEMU as a virtual
FAT volume. That is a development arrangement and not an installable
disk; a real GPT image with a real FAT partition is the installer's job,
and until it exists nothing here writes a disk a machine boots unaided.
Secure Boot is untouched too -- the loader is unsigned.
The ISO keeps one El Torito entry, the FAT image holding loader and
kernel. It never booted from a USB stick via BIOS anyway: the firmware
presents the stick as a hard disk while stage2main.c read the kernel back
over ATAPI.
The 32-bit multiboot entry in entry.S now has no caller and is left in
place. Deleting kernel entry code is its own change with its own way of
failing silently, and it should not ride along with this one.
Ported with it: run.ps1, the six screenshot tools, and schcom_test.sh,
each with its own OVMF varstore copy so no run inherits another's UEFI
variables.
Two things this shook out. OVMF now belongs in `all`: nothing boots
without firmware, `make clean` wipes build/ including the copy fetched
from the host, and a clean followed by a build left every run script
throwing. And it cannot be listed as a prerequisite at the top of the
file -- make expands prerequisites when it reads the line, and
$(OVMF_CODE) is defined further down, so naming it there appends nothing
and says nothing.
Verified from a clean tree: 110 [ok] and no failures on the full suite,
the ISO boots with no hard disk attached, schfs-check passes, schcom's
end-to-end pixel test passes, and fbshot captures a real 1280x800 frame.
bootcheck has only ever booted through the BIOS loader. That is the path
being removed, so the suite had to reach OVMF before the removal rather
than after it -- deleting the boot path first would leave nothing running
during the work that needs watching most.
-Uefi swaps the drives for a pflash pair and the staged ESP, keeping the
same layout: ESP on index 0, the SchFS volume on index 1. Every check
runs unaltered, so a difference between the two transcripts is a
difference in firmware and not in what was asked. The varstore is copied
per run; sharing one would let a run that changed a UEFI variable decide
how the next one behaves.
Both paths now report 110 [ok] with no failures.
Three checks did fail on UEFI, and were right to. They asserted xres 800,
yres 600 and line_length 3200 -- the mode boot/bios/stage2.S asks VBE for.
Under OVMF the firmware picks, here 1280x800 with a 5120-byte stride,
which is correct and self-consistent. The checks were pinning the BIOS
loader's fixed choice, and could do so only because that choice was fixed.
They now assert the invariants the original comment was actually after: a
plausible mode, a stride that covers a full row, and a width that is a
multiple of 8 -- the last being the mode that sheared under OVMF, where
the firmware advertised 1366 and the hardware scanned out 1360.
Also reaped a QEMU left holding schfs.img by a run started outside the
harness, which its pidfile reaper could not have known about.
--snapshots is a range scan over SCHFS_SNAP_OBJECTID, printing each
retained root's id, name, block and the generation it was taken at. The
name is printed with an explicit precision: it is bounded but a damaged
image need not have terminated it.
mkschfs grows --snapshot NAME, which takes one of the finished tree. It
runs after the commit, because a snapshot names a *committed* root --
taking it before would record a root that exists only in memory.
Verified end to end: an image built with --snapshot golden lists the
snapshot and still passes --check. An image without one prints "no
snapshots" rather than nothing at all.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A snapshot is a tree root that is kept. The B-tree is copy-on-write, so
the blocks a committed root reaches are already immutable -- nothing
overwrites a block a committed tree can reach -- and retaining one costs
a record: (SNAP_OBJECTID, KEY_SNAP, id) -> struct schfs_snap.
The record names the root as it was *before* the record was added.
Inserting it rewrites the path under copy-on-write and produces a new
root, so a snapshot that included its own creation would name a block
nobody had written yet.
What makes it survive is the derivation. There is no free-space map on
disk; it is rebuilt at mount by walking the tree, and that walk now
covers every snapshot root as well as the live one. Without it a block
reachable only from a snapshot looks free, gets handed out, and
overwrites the data the snapshot exists to keep -- silently, surfacing
only when the snapshot is read. A snapshot's own tree may hold older
snapshot records; those are deliberately not followed, or deleting a
snapshot could be undone by an older one's memory of it.
Runtime frees are suspended while a snapshot exists. Knowing which freed
blocks a snapshot still needs requires per-extent reference counting,
which is a design and not a flag; until that exists the safe answer is to
free nothing and let the next mount's derivation reclaim what is
genuinely unreachable. Allocation is unaffected and stays correct -- it
works from the in-use bitmap the derivation filled. The running
free-blocks *counter* does drift from the truth while a snapshot is
present, which is cosmetic and is reported by the test rather than
asserted away.
Tested in schfs_objtest: after a snapshot, the live tree shows new
contents while the snapshot still shows the old ones; a file deleted from
the live tree is still readable through the snapshot; deleting the
snapshot removes the record and the space comes back at the next mount.
schfs-check passes including the round-trip; bootcheck 108 [ok].
Not yet: snapshots are not reachable by name from userspace -- there is
no mount-a-snapshot path, so this is the format and the mechanism, not
the feature a user sees.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The compositor already walked a surface table, but nothing had ever run
two clients at once, so "it composites" rested on a single rectangle.
schcom-test now starts two overlapping clients and checks the overlap
region from the host: the later surface has to win there. A compositor
that cannot order two surfaces cannot host a window manager, and that is
the property SchWM will be built on.
Two bugs in the test itself, both of which would have read as compositor
faults:
- it waited for the first "committed" line and screenshotted then, so
the second client had not drawn yet. It waits for both now.
- one probe point sat at 400,300 -- dead centre of an 800x600 screen,
which is exactly where the pointer is drawn. Moved clear of it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Clients connect over a Unix socket, are handed a shared buffer, draw into
it and commit; Schcom composites and presents. Both of the hard blockers
its README listed -- unix domain sockets and raw input events -- have been
done for a while, so nothing was in the way.
The model is Wayland's, on purpose. A surface is not a window: it is a
rectangle of pixels with no policy attached, because placement belongs to
SchWM and the moment this program starts deciding where windows go the
split has failed. Nothing is shown until COMMIT, so a client halfway
through drawing is never presented. The compositor tells clients when to
draw rather than letting them render frames nobody sees.
Wayland is MIT-licensed and supporting it later is intended, so the
divergences are the ones that are cheap to undo: a buffer handle is a
System V shm id rather than an fd over SCM_RIGHTS, and the message
carries a handle_kind field so switching is a new value rather than a
protocol break. Every message carries its own length, so an unknown
message is skipped instead of desynchronising the stream -- which is what
lets a v0 compositor survive a v1 client.
The compositor owns the shm segment, not the client: it has to survive a
client crashing mid-frame, and it must not be something a client can
resize underneath it. A dropped connection takes its surfaces and their
segments with it.
Composites into a back buffer and copies once, because compositing
straight into the framebuffer shows every intermediate state -- the
flicker would be the whole desktop being repainted under the user's eyes.
projects/ gets build glue for the first time: `make projects` builds
against the sysroot's libc and stages into sysroot/bin. It is not a
dependency of anything above it, and `make` never descends there.
Verified by `make schcom-test`, which boots SchOS, runs the compositor and
a client, screendumps the framebuffer and asserts the pixels on the host.
That last part is the point -- the compositor reporting that it
composited something proves nothing, and a screenshot nobody inspects
proves less. It checks the client's colour mid-surface, the compositor's
background, and the white border exactly at the requested origin, so
"arrived" and "arrived at the wrong offset" cannot pass for each other.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The disc carried one file -- boot/schos.bin -- with 8.3 uppercase names,
one El Torito entry, and no way to boot a UEFI machine. Now:
/boot/schos.bin BIOS entry, unchanged: no-emulation, 1+STAGE2_SECTORS
/boot/efiboot.img UEFI entry: a FAT image built with mtools
/EFI/BOOT/BOOTX64.EFI the loader, also visible as a file
/bin /etc /usr ... the staged sysroot as an ordinary tree
Rock Ridge and Joliet restore real names, permissions and symlinks;
-cache-inodes matters more than it looks, because the sysroot is a toybox
multicall binary with a couple of hundred hard links to it and without
inode caching each link becomes its own copy.
-isohybrid-gpt-basdat adds a GPT so the .iso can be dd'd to a stick and
booted by UEFI. That is UEFI only and deliberately so: the BIOS isohybrid
trick needs syslinux's isohdpfx.bin, and even with it this loader reads
the kernel back over ATAPI, which a stick presented as a hard disk cannot
serve. usbstick.img is what that is for.
The FAT image carries the kernel as well as the loader, because
BOOTX64.EFI opens \schos\kernel on the volume it was loaded from, and for
a UEFI boot that is the FAT image rather than the ISO 9660 tree. Without
it the loader halted with EFI_NOT_FOUND.
Also fixes `make iso` and `make ramroot`, which were already broken:
fs-ram.img was still built by the v1 mkfs while the kernel accepts only
SchFS as root, so the ramdisk was a format it could not mount and it
panicked with "schfs root: no SchFS volume on the root device". It is a
mkschfs image now, which needed --exclude in mkschfs -- dropping the
exclude list was not an option, because the ramdisk is linked into the
kernel binary and what goes in it is a licensing question. RAMDISK_NINODES
is gone with the v1 image: SchFS has no preallocated inode table.
Verified: BIOS ISO boots to a shell and runs commands over serial
(241 files in /bin, /etc/motd reads back). UEFI ISO boots to a prompt.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three bugs, all of which surfaced as a failed check somewhere other than
the command that was actually slow.
Wait-Prompt waited a fixed 20 s. These tests run under TCG, not KVM, and
a cold `ls /bin | wc -l` takes 23 s on the Fedora laptop measured with
the guest's own /proc/uptime -- it was over the wall every time. Raised
to 60 s; a long timeout only costs time when something is genuinely
wrong, a short one costs a false failure elsewhere.
It also matched a prompt anywhere in the transcript. Between writing a
command and the guest echoing it the log still ends with the previous
prompt, so it could return before the command had started and the next
one was typed on top of it. Anchor the match to text produced after the
command was sent.
On timeout it slept two seconds and typed regardless, which is what
turned one slow command into an interleaved transcript and a failure
attributed to a later check. Wait for output to stop arriving instead.
Report any command that never produced a prompt next to the result:
after one overrun every later check is reading a transcript that may be
interleaved, and that should not be something you have to notice in the
scrollback.
With this, -Fb -Tty -Usb -Sock -Evdev -Ps is 108 [ok], 0 misses.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The `finally` stops QEMU on every path the script controls, but not when
the script itself is killed -- a CI timeout, Ctrl-C at the wrong moment,
pkill. QEMU then keeps running with no parent. Because these tests run
under TCG rather than KVM, a spare emulator pinning a core slows every
later run, and one was found still executing the guest at 86% of a core
eight minutes after its parent was gone.
That is the bad kind of failure: the suite does not break, it gets
slower, and commands drift past Wait-Prompt's timeout until the next one
is typed into a busy shell and the transcript interleaves. The check
that then fails is unrelated to whatever actually changed.
Record the pid on startup and reap the previous run's leftover before
starting a new one, the same self-healing approach already used for the
monitor port. The name is checked before killing, because pids are
recycled and killing an unrelated process would be worse than the leak.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Neither is Fedora-specific. A different host and a freshly built
userland simply drove code that Windows never reached.
tools/mkfs.c did not compile at all. The DT_UNKNOWN/DT_DIR/DT_REG
fallbacks sat above <dirent.h>, so on glibc -- which spells them as
enumerators -- the #ifndef asked before the header was read, always
answered "missing", and then the header's own enum expanded to
`enum { 0 = 0, ... }`. Move the fallbacks below the include. The
include cannot move up instead: it has to follow the #undef that ends
the `#define dirent schfs_dirent` block.
getstackpcs() page-faulted. It dereferenced rbp[1] without validating
rbp, and the guard rejected 0xffffffff -- dead code on x86-64, already
caught by the KERNBASE test -- while the value that actually turns up
is all-ones, which is above KERNBASE. rbp[1] then wrapped to 0x7 and
took trap 14. `cp` panicked the kernel every time. Bound rbp from
above and require 8-byte alignment.
That second one masks other bugs: getcallerpcs() runs on every lock
acquisition and again from the trap handler, so a walker that can
fault replaces a real diagnosis with a bogus one.
Baseline on the Fedora laptop is now green: make ports, make,
make schfs-check including the round-trip that had never run, and
bootcheck at 27/27 plus 108 [ok] across the extended suites.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
tools/posix/setup-fedora.sh installs the toolchain and reports what it
cannot: PowerShell has no Fedora package, and bootcheck is the test
suite, so the script says how to get pwsh rather than failing quietly.
It also checks /dev/kvm, since a Ryzen box can run the tests at native
speed.
docs/moving-to-linux.md is the actual procedure: copy the repo, restore
the memory, install, build. The memory travels inside the repo and
`memory-sync.sh import` puts it under the key this machine derives from
its own path -- /home/nexoniarz/Documents/SchOS becomes
-home-nexoniarz-Documents-SchOS, where D:\DEV\SchOS was D--DEV-SchOS.
OVMF discovery no longer assumes QEMU's Windows layout. Fedora ships
edk2-ovmf to /usr/share/edk2/ovmf/, Debian and Arch differ again, so the
Makefile searches a list and names the packages when it finds nothing.
Tested through the shell rather than $(wildcard): on Windows make is a
native binary using Win32 file calls, while `command -v` returns an MSYS
path like /c/Users/..., which $(wildcard) cannot see -- it returned
nothing on a machine that had the firmware.
Two things only that machine can test: `make ports` builds musl, toybox
and mksh from source, which Windows cannot do at all; and the SchFS
round-trip needs symlinks and hard links, so it has never actually run.
Milestone 1 of three: manifests, RunAtLoad, KeepAlive with throttling,
and reaping. Replaces the hardcoded fork-a-shell loop in cmd/init, which
stays in the tree for now.
Jobs are sectioned key=value files in /etc/schinit/*.job, kept in the
repo under etc/ and installed into the sysroot -- sysroot/ is generated
and gitignored, so anything written only there is lost on a fresh clone,
which is already true of /etc/passwd and should not become true of the
boot configuration. Unknown keys and sections are ignored rather than
fatal: a manifest written for a newer schinit must not stop this one
booting the machine.
Socket sections are parsed and recorded but not yet held. That is
milestone 2 and it is the whole reason for copying launchd rather than
sysvinit -- the supervisor holds the listener and starts the job on first
connect, which inverts the dependency graph and makes runlevels a
non-concept. The key exists now so manifests need not change later.
getdents64 is exposed to userland for this. The kernel has had it since
the Linux ABI landed, but nothing declared it in user.h or lib/usys.S, so
first-party programs had no way to list a directory at all.
Two things found by testing rather than reasoning:
- setsid() gives a job its own session, and a session has no
controlling terminal. The console then delivers input only to the
foreground process group it tracks, so the shell came up, echoed
every keystroke through the line discipline, and executed none of
them -- indistinguishable from a hung shell. ControllingTerminal=yes
claims it with TIOCSCTTY/TIOCSPGRP; off by default, since a daemon
has no business seizing the console.
- the first bootstrap() dropped /dev/input/event0-1 and most of /proc,
because it was transcribed from the part of the old init that had
been read rather than all of it. -Evdev caught it.
build/usbstick.img is built by `all` now: without it QEMU refuses the
-drive and the entire -Usb run produces no output, which reads as a
broken USB driver.
Seven suites green: base 27, Tty 51, Ps 32, Sock 64, Fb 36, Usb 31,
Evdev 29. make schfs-check passes.
boot/bios/ now holds the MBR, the El Torito stub and the real-mode stage
2; boot/uefi/ is empty and documented.
Separate directories rather than #ifdefs, because the two share almost
nothing. A BIOS loader lives in 16-bit real mode, reaches the firmware
through software interrupts, and walks itself up into long mode. A UEFI
loader is a 64-bit PE that the firmware has already placed in long mode
with paging on, calling services through a table of function pointers.
What they do share is the hand-off contract, which boot/README.md now
writes down: ELF segments placed, long mode, a 24-byte-entry memory map
at E820_MAP_ADDR, framebuffer geometry at VBE_INFO_ADDR.
That contract is the point of documenting it now. UEFI's GetMemoryMap
returns EFI_MEMORY_DESCRIPTORs, and the right move is to translate them
into the entries kernel/arch/x86_64/e820.c already parses rather than
teach the kernel a second format.
Build output paths under $(BUILD)/boot are unchanged; only sources moved.
Two bugs reported from actual use, plus a test harness that had stopped
testing.
init could be killed by Ctrl-C, taking the machine with it. SIGINT goes
to the foreground process group, init is pgid 1, and any window where
tty_fg_pgrp is 1 -- before a shell claims the terminal, after one exits,
or between the two while the key is held -- lands it on init. init has
no handler, so killed was set, exit() ran, and exit() panics outright on
initproc. Spamming Ctrl-C at a frozen program is exactly the input that
finds that window. POSIX and Linux both discard a default-action signal
for pid 1; init_ignores() now does the same, SIGKILL included, because
init is not killable from userspace on any Unix.
Everything was slow because BSIZE was 1024 while a SchFS block is 4096,
so every block cost four IDE requests, each with its own interrupt and
sleep/wakeup. exec() of the 800 KB toybox binary meant ~800 round trips.
The giveaway: `seq 1 200`, which touches no file and prints nothing,
cost the same 2.0s as `cat /bin/toybox`, and `ls | wc -l` cost exactly
double -- both dominated by loading the same binary twice.
BSIZE is 4096 now, one request per SchFS block, and NBUF is 256 blocks
(1 MB) rather than 90 KB -- smaller than the binary every command execs,
so nothing was ever still resident. ideinit() issues SET MULTIPLE MODE,
which is what makes a 4096-byte transfer legal: READ/WRITE MULTIPLE move
the drive's multiple count per interrupt and nothing had ever set it,
which is the real reason idestart() carried a sector_per_block > 7 panic.
seq 1 200 2.02s -> 0.06s
ls /bin 2.76s -> 0.68s
ls /bin | wc -l 4.68s -> 0.73s
And the harness. bootcheck called Find-Qemu from a param() default,
which is bound before any function in the body exists -- so it died on
its first line and printed an error containing neither "FAIL" nor
"Exception", the two things being grepped for. It had not run since it
was made host-portable. Worse, an earlier index-based edit had deleted
the Expect blocks for -Tty, -Usb, -Sock, -Ps and -Evdev, so five of the
seven modes asserted nothing at all. Restored, and the USB run gets its
own 16 MB build/usbstick.img: it had been pointed at build/fs.img, which
nothing builds since SchFS became root, so QEMU refused to start.
242 checks across seven modes, all passing, and make schfs-check green.
tools/windows/ .ps1 (pwsh, so usable from Linux and macOS too)
tools/posix/ .sh
tools/python/ .py
tools/ .c and .pl
The .c and .pl files stay at the top level because they are not
host-specific: the build compiles or invokes them on every platform.
Call sites updated in the Makefile, ports/mk/port.mk, docs and the
project memory. The one thing that does not survive a move on its own is
a script deriving the repo root from its own location -- all nine
PowerShell tools used Split-Path $PSScriptRoot -Parent, and memory-sync.sh
used dirname/.., both of which quietly become tools/ one level deeper.
Two levels now.
Two pre-existing Windows failures surfaced while verifying, both real and
neither caused by the move:
- schfs_fsck.c did not compile under mingw at all: --extract uses
POSIX mkdir(path,mode), symlink() and link(), none of which mingw
has. That took the whole tool with it, and `make schfs-check` with
it. mkdir maps to _mkdir; symlink and link fail loudly rather than
writing a regular file in their place, because an extracted tree
that differs from the image while reporting success defeats the
point of --extract.
- the round-trip test then failed at `ln -s`, since Git Bash cannot
create symlinks without developer mode. It skips with a message now.
Not a fudge: mkschfs's mingw build cannot see a symlink either, so
the test would have compared a tree with none against an image with
none and called it a pass -- coverage reporting itself as present
while being absent.
Verified on Windows: build clean, all seven bootcheck suites green,
make schfs-check passes.
Claude Code keeps per-project memory at <home>/.claude/projects/<key>/,
where <key> is the project's absolute path with every separator and the
drive colon turned into a dash. That makes it both rename-fragile and
machine-specific, and this project has hit each of those once:
- D:\DEV\Schunix -> D:\DEV\SchOS changed the key, and 43 memories were
invisible until moved across by hand.
- Development happens from Windows and from Linux, and each looks in a
different place for the same checkout.
Committing .claude/memory/ fixes both -- git carries it across renames
and between hosts -- and `import` puts it wherever the local machine
expects. .claude/settings.local.json stays ignored: those are per-machine
permissions.
The colon maps to a dash rather than being dropped. Deleting it gives
D-DEV-SchOS, which matches nothing and silently finds zero memories; that
was the first attempt. On Git Bash the key comes from `cygpath -w`,
because Claude Code is a Windows program and sees D:\DEV\SchOS, not
/d/DEV/SchOS.
Developed from both Windows and Linux, so nothing in the build should
care which host it runs on. It did, in four places.
Toolchain: one path instead of two. clang is a cross-compiler by
construction, so a single LLVM install targets bare-metal ELF, 16/32-bit
code below 1 MB, and Linux/musl userland from any host -- which is how
the Windows build already worked. The GNU branch beside it had drifted
into passing --no-warn-rwx-segments, a hard error on lld, which is the
default linker on both FreeBSD and macOS; and it required a brewed
x86_64-elf-gcc before anything built on a Mac at all. GNU is kept as a
fallback (USE_GNU=1) but clang+lld is chosen whenever both are present.
Host tools use the host's own cc; only mingw needs naming explicitly.
tools/mkschfs.c included <sys/sysmacros.h> unconditionally on non-mingw
hosts. That header is glibc/musl only -- macOS and the BSDs have neither
the header nor a need for it, since major()/minor() come from
<sys/types.h> there. Now guarded on __linux__.
Ports verified tarballs with `sha256sum -c -`, which does not exist on
macOS (shasum), FreeBSD/NetBSD (sha256), or OpenBSD (cksum -a sha256),
and each spells its check mode differently. tools/sha256check.sh picks
whichever of the five the host has and compares digests itself.
nproc is coreutils, so absent on macOS and BSD; falls through to sysctl
-n hw.ncpu before defaulting to 4.
The PowerShell tools now run under pwsh on any OS: QEMU is found on PATH
(the scoop location is a Windows-only fallback), and path literals use
forward slashes -- Join-Path $root 'build\x' produces a *filename
containing a backslash* on a Unix host. The backslash sweep is limited
to known directory prefixes on purpose: bootcheck sends
printf 'caf\303\251' to the guest to test UTF-8, and a blanket
substitution would have quietly changed what that asserts.
Also fixes six tools left booting build/fs.img, which stopped being
bootable when SchFS became the root filesystem: crashshot, dispshot,
fbshot, splashshot, textshot and ttyshot all take build/schfs.img now,
and the -SchFS switches that used to select it are gone.
Verified on Windows: full build, and bootcheck green. The Linux, macOS
and BSD paths are written from their documented behaviour and are NOT
executed here -- this machine has no WSL and no second host.
Removes the "-unix" element and with it any trademark exposure to The
Open Group.
The split follows the ecosystem layout: SchOS is the system, and the
kernel has its own name. Kernel-identity strings are the only places
that take SchIFT -- uname(2) sysname, /proc/version, and the two licence
sentences that say "kernel" outright. That is what every Unix does here:
`uname -s` on Linux reports the kernel, not the distribution. Everything
a user reads as the product -- panic screen, boot menu, error prefixes,
docs, LICENSE, NOTICE -- is SchOS.
schunix.img is schos.img, the logo assets and the musl TLS patch are
renamed, and __SCHUNIX__ is __SCHOS__ (the define and the patch that
consumes it moved together).
Two things the rename broke, both found by the bootcheck suite:
- shmtest wrote "SCHUNIX-MMAP-OK" with a hardcoded length of 15 and
compared it back by hand-counted offsets (m[3]=='U', m[14]=='K').
The shorter marker left the write running two bytes past the literal
and the compare reading past the data, failing a file-backed mmap
that was working correctly. Now written and compared through one
constant with sizeof.
- bootcheck asserted a 31-byte file after appending to
"hello-from-schunix"; the shorter string makes it 29.
Both are the same mistake: a string literal whose length is written out
somewhere else. Full suite green -- base, Tty, Ps, Sock, Fb, Usb, Evdev.
Nexoniarz