`∞`, `→`, `≤` and `'` could not be typed. `ą` and `ę` could. That pairing is
the whole diagnosis: KBD_UP was 0xE2 and its neighbours filled 0xE0..0xE9,
which are exactly the UTF-8 lead bytes for three-byte characters. Every
character in U+0800..U+FFFF -- most punctuation, every arrow, every maths
symbol, the whole of CJK -- arrived as a value the console read as an arrow key
and helpfully turned into `ESC [ A`. Two-byte sequences lead with 0xC4 and
nothing claimed 0xC4, so Polish came through and the rest did not.
The chord codes already knew this rule and said so in a comment. The navigation
keys predate them and were never brought in line. Stated once, in kbd.h: a key
produces either a Unicode codepoint or a value above 0x10FFFF, and nothing in
between means anything. KEY_* -> KBD_*, which also settles a silent collision
-- include/input.h defines its own KEY_HOME, KEY_UP and KEY_END as evdev
numbers, and kbd.c includes both headers, so three macros were being quietly
redefined in the one file that used them.
Then the layouts, which is what the codepoint-sized keycodes were for.
kernel/dev/keymap.c: us, pl, de. A layout is four tables -- plain, Shift,
AltGr, Shift+AltGr -- of uint32 codepoints. uint32 and not uchar is the reason
the file exists: `ó` fits in a byte and `ą` does not, and half a Polish
alphabet is not a Polish layout.
Two things deliberately are *not* tables:
- **Ctrl is derived from the plain codepoint.** The table it replaces was
indexed by scancode, which quietly assumed US -- on QWERTZ the Y and Z keys
swap, so Ctrl+Z would have suspended a job only on some layouts.
- **Caps Lock is a case toggle on the result**, so AltGr+a with Caps Lock is
`Ą`. No second table could say that, because Caps Lock combines with AltGr
rather than replacing it. Latin Extended-A alternates upper/lower by
codepoint but not all the same way round -- Ł is U+0141 (odd) and Ą is
U+0104 (even) -- so Polish lands in both halves and the parity matters.
kbdgetc() encodes to UTF-8 and returns the bytes one call at a time.
consoleintr()'s contract is bytes: the serial line hands it UTF-8 already and
it could not tell an encoded codepoint from a byte of the same value.
The layouts live in the kernel. Linux's loadkeys compiles a text keymap and
uploads it a key at a time through KDSKBENT, which puts the data in userspace
-- and a machine whose /usr has not mounted then has no layout at all. Here the
kernel always has a working keyboard and `loadkeys` only names which one.
KB_GETLAYOUT/KB_SETLAYOUT/KB_LISTLAYOUT; /etc/vconsole.conf and an
os.schos.keymap job for the boot-time choice. No dead keys, so French and
Spanish are absent rather than wrong.
/etc/locale.conf and LANG=C.UTF-8 in profile and zshenv. Only the charset half
of that is real -- musl decodes UTF-8 whatever the locale says and ships no
locale data -- but an unset LANG reads as "ASCII only" to everything that asks,
which is its own way of producing `?`.
And two symlink bugs found while chasing
zsh: locking failed for /root/.zsh_history: operation not permitted
on every boot. symlink() is the classic atomic lock -- create the link, and if
it exists someone else holds it -- and zsh distinguishes EEXIST ("contended,
retry") from anything else ("this cannot work, give up"). sys_symlink returned
a bare -1, which is -EPERM, so every zsh on the machine took the second branch
on a filesystem that supports symlinks perfectly well. Same errno class as
commit 75ccf98.
The second one the first was hiding: **divesymlink() has been in fs.c since the
fork and nothing ever called it.** A symlink behaved as a small file containing
its own target, so `cat link` printed the target's *name*. open() and openat()
follow the final component now (O_NOFOLLOW gives ELOOP), and stat() and lstat()
stop being the same function, since which one follows is the only difference
between them. Mid-path symlinks are still not resolved -- that belongs in
namex(), which would re-enter lookup holding a directory lock.
Verified on the emulated keyboard, not just over serial: AltGr+a/e/s gives
ąęś, Shift+AltGr gives ĄĘŚ, Caps Lock+AltGr gives ĄĘŚ, de types zy for the
physical y/z keys and @€µ on AltGr. `∞ 'x' → ≤ • — …` typed over serial now
survives. Three fresh boots: 0 lock failures, 0 faults.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
"Ą Ę Ś Ł Ż" drew as "?????" and the obvious diagnosis -- the font has no
glyphs for them -- was wrong twice over. `struct vtcell.ch` was **one byte**
holding a CP437 index, and the console mapped UTF-8 down to it on the way in.
U+0104 does not fit in a byte. No font could have fixed that; the cell had to
get wider first.
So: `ch` is a uint32 codepoint, the mapping on the way in is gone, and the cell
draws through kernel/dev/ttf.c against JetBrains Mono linked into the image.
Cell size comes from the font's measured advance instead of a hardcoded 8x16.
The grid costs 864 KB of BSS instead of 432 KB, which is the honest price of
holding a codepoint per cell for six terminals at 240x75.
The rasteriser needed three fixes before it could be trusted with this.
**21 KB of locals on a 4 KB kernel stack.** `ttf_render` held an outline (512
points as three arrays) and a raster (1024 edges) as automatics. KSTACKSIZE is
4096. They walked off the bottom of the stack into whatever was underneath,
which surfaced as user processes faulting on addresses they never touched and
then a panic inside freevm() on an overwritten page directory. Nothing pointed
anywhere near a font.
**And then again, recursively.** load_outline() held a 4736-byte outline as a
local *and recursed* for composite glyphs. Composite glyphs are exactly Ą, Ę,
Ś, Ł, Ż -- so the second crash arrived the moment the first fix let Unicode
reach the screen, looking like a fresh bug. Now a static outline per recursion
level with a depth cap of 4 (the font's deepest real nesting is 2), and the
leaf parse split into load_simple() so its 512-byte flag array is not
multiplied by the depth. Worst chain measured from the object file: ~2.3 KB.
Neither showed in tools/ttftest, which has an 8 MB stack. That is the standing
hazard of one source file built for host and kernel: the host is most forgiving
about exactly the resource the kernel is strictest about.
**Component transforms were being skipped**, on the stated theory that a wrong
transform is worse than none. It is not. `‘` in this font is the comma glyph
with a scale of -1 -- a 180-degree rotation -- and the offset that ships with
it only lands correctly after the rotation. Without it the mark went twelve
pixels into a ten-pixel cell and was clipped away entirely, so the character
read as missing from the font rather than as misplaced. F2Dot14 2x2 now, with
SCALED_COMPONENT_OFFSET honoured.
The glyph cache was ASCII-only, so every accented letter re-rasterised on every
repaint: a screenful of Polish did hundreds of bezier flattenings per scroll
where the same screenful of English did none. 95 reserved ASCII slots plus 224
shared slots keyed by codepoint, 64 KB pool. Verified 13680 cached renders
byte-identical to direct ones across collisions and a pool flush.
tools/ttftest speaks UTF-8 now. Iterating bytes meant it could only ever ask
for ASCII, and ASCII is entirely simple glyphs -- it could not reach the code
path that mattered.
Also: resolvconf's success line moves behind /proc/bootmode, same rule as
vgpud. A quiet boot opened with "resolvconf: nameserver 10.0.2.3" where the
one line was supposed to be "Welcome to SchOS!". Failures stay loud.
Verified on screen, not just on the host: Ą Ę Ś Ł Ż ć ń ó ź, Ω λ π Œ Ð Æ ß,
∞ ‘x’ → ≤ all render, panic=0 across four boots.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three things, all in the first minute of using a shell.
**Arrow keys corrupted the two keystrokes after them.** They arrive as single
keycodes -- KEY_UP is 0xE2 -- and were pushed into the input buffer exactly
like that. No terminal works that way, and 0xE2 is the worst possible byte to
have chosen: it is a UTF-8 lead byte for a three-byte character, so a reader
decoding UTF-8 swallowed the next two keys waiting for continuations that never
arrived. consoleintr() sends the DEC/xterm sequences now -- ESC[A, ESC[D,
ESC[5~ and the rest -- which every terminfo entry and every line editor already
knows. Not echoed: in canonical mode echoing them would move the cursor for
real, which was the other half of what "weird glitches" looked like.
**Then the end of the line vanished as you typed into it**, with the buffer
perfectly correct. One line in vt_putc(): backspace erased the cell it moved
onto. That is a line-discipline erase, not the BS control character -- a
program rubbing a character out sends `\b \b`, and the separation matters
because programs also use plain `\b` to go back. zsh does, on every mid-line
edit:
echo alpha \b\b zha \b\b
back two, rewrite the tail, back two again to park the cursor. Against a
destructive backspace that last pair deleted the `ha` it had just drawn.
`\b` moves and nothing else now, 0x7f is ignored as on a real terminal, and
consputc() spells its own erase out as `\b \b` as it already did for serial.
ESC[@, ESC[P and ESC[X went in alongside, because TERM=linux declares them.
**VT3 to VT6 were blank** because nothing ran on them. NVT is 6 and the kernel
made all six; only two had a shell, so Alt+F3 switched to a terminal that was
working perfectly and had nobody to talk to. Four more manifests.
Six shells starting at once is enough concurrency to expose an intermittent
fault in one of them -- one boot in five, a wild jump into .data that runs
until it writes to 0, which nothing catches because SchOS never sets EFER.NXE.
Three boots with two shells, none. Recorded rather than fixed: it is not caused
by the manifests, only made likelier by them.
Also: QEMU_NET forwards host 8080/8081/8000 to guest 80/8080/8000, so the httpd
is reachable from the machine running QEMU instead of being sealed behind NAT.
Verified end to end. And the greeting prints once again -- a replaced
supervisor was announcing a boot that had already happened.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
Ctrl-C killed the shell. consoleread() breaks off the read in progress when
SIGINT arrives and returned -1 to do it -- and -1 is -EPERM by the time it
reaches a program, so every program that handles SIGINT by carrying on saw a
permanent error instead of an interrupted call:
zsh: error on TTY read: operation not permitted
schinitd: restarting os.schos.console-shell
and the next Ctrl-C did it to the replacement. EINTR now, which is what a
signal-interrupted read is required to give. Verified: Ctrl-C an interruptible
command, shell survives, zero restarts.
Same mistake, same file family: link, unlink, rename and renameat returned -1
on every error path. Callers branch on errno -- `if (unlink(p) < 0 && errno !=
ENOENT)` is the standard shape, and against EPERM it always complains -- which
is why zsh reported failures for things that had worked. They return ENOENT,
EEXIST, EXDEV, EISDIR, ENOTEMPTY and EFAULT now. About ninety other `return -1`s
remain in sys_fs.c; these four are done because these four were what a program
tripped over. /dev/mem's unknown minors return ENODEV rather than -1 too.
/dev/urandom was a linear congruential generator seeded with a **constant**, so
every boot produced the same bytes in the same order -- not weak, the same. A
key, a nonce, a temporary filename: shared with every SchOS machine that had
ever run. Seeded at boot from RDRAND where the CPU has it, from the cycle
counter and the wall clock where it does not, and re-stirred per byte. Still an
LCG and still not a CSPRNG; what changed is the part that was dangerous rather
than merely poor.
**Not fixed, and recorded as such:** rename() still will not replace an existing
target, so zsh's history save fails on every exit after the first. An attempt to
unlink the target first was reverted -- the directory is an HTree, dirunlink
followed by dirlink of the same name did not behave, and it panicked in
filewrite with the old file already gone. Losing both files is worse than
refusing, so it refuses until the HTree's slot reuse is understood.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
`make qemu` gives the machine two display devices, which is deliberate --
stdvga carries the splash and text console, virtio-gpu is what the display
class binds and what a compositor is developed against. What the comment above
QEMU_DISPLAY did not say is that this means **two QEMU displays**, and a window
shows one at a time.
Once vgpud runs they show completely different things: the console and your
shell are on stdvga, and virtio-gpu has whatever vgpud drew, which today is a
test pattern. So a window sitting on the virtio-gpu display shows colour bars
and no prompt, and one that opened before vgpud drew shows black -- and neither
is a failure, though both look exactly like one.
`make qemu-console` boots with one display and no GPU to hand over: vgpud finds
nothing to claim and exits, and the single window is the console. Mostly it is
a way to tell "I am looking at the other screen" apart from "the screen is
broken" in the ten seconds before assuming the second one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
`make` had been failing since the e1000 work and I did not see it, because I
was checking build output with `grep error` -- which matches neither "Błąd" nor
mkfs's "image full". Checking the exit status instead found all of this in a
minute.
**usbstick.img was full.** `make ldls-stage` puts a module and its BTF in
sysroot/lib/modules, and a vmlinux BTF is 5 MB on its own; the 16 MB test image
could not hold it. The size cannot move -- tools/windows/bootcheck.ps1 asserts
READ CAPACITY returns exactly 32768 sectors -- so the exclusions grew instead:
lib and usr as well as bin. That image exists to prove the USB driver can read
one block, so its contents were never the point. It also stopped a GPL module
riding into a test artifact, which RAMDISK_EXCLUDE does for the ramdisk and
nothing was doing for this one.
**toybox's httpd was shadowing ours.** /bin/httpd answered `-p` with toybox's
"Unknown option 'p'", because whichever install rule ran second won the name.
TOYBOX_SKIP existed for exactly this and held one entry, `ping`, found the same
hard way. It is now generated from cmd/ -- a list you have to remember is a
list that is wrong the next time cmd/ grows, and it already was: `chvt` had
been silently toybox's, whose VT ioctls are Linux's and not ours.
**ldlsd would not link with clang.** `*out = in;` on a 96-byte struct is a call
to memcpy as far as the C standard is concerned; gcc unrolled it and clang
emitted the call, so a freestanding program linked for months and then did not
on a machine where nothing about it had changed but the compiler. lib/glue.c
now provides memcpy, memmove, memset and memcmp -- the four an implementation
may call without being asked.
**OVMF was never found on NixOS.** `$(dir $(QEMU))share` is the *bin*
directory plus "share", so it looked in /usr/bin/share. The absolute paths
after it covered that up on distributions that keep firmware in /usr; on one
that keeps nothing there it meant the firmware was undiscoverable on a machine
that had it, and the only way to build was to copy the file in by hand. It
resolves the prefix properly now and looks in <prefix>/share/qemu too.
Verified by exit status, from a clean tree and a no-op: both 0. ldls-check
passes 27. The guest boots, serves /srv to the host and answers 403 to
%2e%2e/etc/passwd.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
Four complaints, and only one of them was the thing it looked like.
**"toybox is too small."** Half true, and the fix cost no licensing at all:
236 of toybox's applets are what defconfig builds, and the rest live in
toys/pending/ -- awk, bc, diff, more, expr, tr, hexdump, stty among them. Same
0BSD tarball, switched off by a config default. 26 of them are on now, 236 ->
268. Not everything: strace wants ptrace, syslogd wants BSD's facilitynames[],
git wants OpenSSL, and those three break the link.
The toybox build also works again on this machine. Its scripts start
#!/bin/bash, which does not exist on NixOS, so the binary in the tree was one
nobody could reproduce; the shebangs are rewritten to /usr/bin/env bash on the
way in. Pending applets are enabled by editing .config directly, because
`make oldconfig` is the obvious way and undoes it -- every pending symbol is
`default n`.
**BusyBox was asked for and is not here.** It is GPL-2.0-only, and `make
ramroot` links the root image *into the kernel binary*, so it would put GPLv2
inside an Apache-2.0 executable -- which is the exact case ports/Makefile's
NOTPORTS filter and docs/licensing.md §4 exist for. nano is the same answer one
version later. Both recorded in licensing.md with what was done instead, since
"no" without an alternative is how a rule gets quietly broken later.
**An editor that is not vi.** kilo, BSD-2-Clause, one 1300-line file, no
dependencies beyond termios and TIOCGWINSZ, both of which zsh already needed.
Installed as /bin/edit as well as /bin/kilo, because somebody with a file to
fix will type `edit`.
**"wget doesn't work."** Not a userland problem, and BusyBox's wget would have
failed identically: nothing on this system could turn a name into an address,
so every program taking a hostname failed in its own way and each looked like
a separate bug. There is still no DNS client in SchOS and there does not need
to be -- musl has a complete resolver and it works over ordinary UDP sockets.
The one missing fact was which nameserver, which the kernel has had all along
from DHCP and never published. /proc/resolv.conf publishes it in resolv.conf's
exact syntax; cmd/resolvconf decides whether it becomes /etc/resolv.conf, and
will not overwrite a hand-written one without -f.
# wget -O - http://example.com/ | head -1
<!doctype html><html lang="en"><head><title>Example Domain</title>
**Something to host with.** cmd/httpd: GET and HEAD, static files, index.html
for a directory and a listing otherwise, a child per connection. No CGI, no
keep-alive, no TLS, no config file. The path check is the only
security-relevant code in it and it refuses rather than normalises -- no
component may be `..`, and percent-decoding happens first, because %2e%2e is
`..` and a check before decoding does not see it. Both forms answer 403.
And the disk is 100 MB, from 32. It is sparse, so the image is 21 MB on the
host. RAMDISK_BLOCKS went 6000 -> 8000 with it: the ramdisk was at 86% and the
next thing added would have broken `make ramroot`, which nobody would have
noticed until they used it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
schdbgd bridges TCP to the debugger, which is the one way in that does not
need you to be sitting at the machine. It is a bridge and nothing else: a line
in becomes a line written to /dev/dbgro and the answer comes back, so `nc` is a
complete client and there is no clever client to be broken on the day you need
this.
# schdbgd &
schdbgd: debugger on port 4444, loopback only
$ nc 127.0.0.1 4444
dma
arena 0xffff80003da41000-0xffff80003ea41000 16384 KB total, ...
A kernel debugger plus a TCP socket is a remote root hole, so three things were
decided rather than discovered:
- It does not start by itself. schdbgd.job ships Disabled. SchOS has no
kernel command line, so the boot flag the plan asked for is the job
manifest, which is the same thing in this system's own idiom.
- It binds loopback; -a names an address and says so in a paragraph.
- It cannot reach a view that changes state, enforced in the kernel.
That last one is /dev/dbgro: /dev/dbg on minor 1. The minor is the one piece of
per-open state a character device already sees, so which node you opened is the
permission -- nothing to latch, nothing to remember between the write and the
read, no way across. Which views are refused comes from dbg_register_rw() on
the registration itself, because a central list of dangerous views goes stale
in the direction where a new view is quietly treated as safe. Today that is
`watch <addr>`; `watch` with no argument is still the listing and still
answered.
Also: socket() now accepts IPPROTO_TCP with SOCK_STREAM and IPPROTO_UDP with
SOCK_DGRAM. They name the default rather than departing from it, and toybox's
nc writes them out -- so TCP was "Protocol not supported" on a system whose
whole network is TCP.
docs/plan/debugger.md retires into docs/architecture/debugging.md; 16 files
repointed. Every plan in docs/plan/ that was open is now closed, so the next
thing is Gate 0 itself: the survey, with the instrument built.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
e1000 -- Linux's, unmodified -- probes, opens, gets an address by DHCP and
carries ping traffic in a SchOS process. The kernel was driving that NIC and
gave it up, which makes it the first real handover as well as the second
driver.
None of what it needed was a missing feature. Every one was a call that
succeeded, returned zero and did nothing:
_find_next_bit "bit 0 is set" -- for_each_cpu never ended
netif_carrier_off carrier already up -- TCTL.EN never set
__x86_indirect_thunk_r8 the jump discarded -- 255 rx buffers, none made
dma_map_page_attrs bus address 0 -- NIC aimed at physical 0
so the rule from memcpy is now general: a stub returning zero is unsafe
wherever zero is a meaningful answer.
The loader grew ko_fixup_percpu(). The stack guard turned out to be one
per-CPU variable among several; `current` cannot be defused because the driver
dereferences it. Taking the %gs off instead leaves the same opcode reading the
same width from storage the host supplies -- a per-CPU variable in a
single-threaded host is just a variable. Eight bytes become seven and a nop,
and the displacement is not always last, so there is now just enough of an x86
decoder in ko.c to know how long the instruction is.
ldls/kpi/thunk.c is fifteen lines of assembly and the highest-value file here:
retpolines are control flow, and a stub cannot stand in for a jump.
The kernel side:
- ioperm.c, the TSS I/O permission bitmap. A driver's outl is an instruction,
not a call, and e1000's reset path picks it deliberately. IOPL=3 would hand
over the PIC and the disk with it, so it is per-port. Writing it found that
the I/O map base had been written at offset 64 rather than 102, leaving the
CPU checking ports 0-831 against the TSS's own RSP0.
- claim masks the device's IRQ line: 450,000 a second, otherwise.
- net_detach(), the missing half of net_attach_late(). An interface left in
the stack keeps its route and sends every frame to a driver that refuses
it, which looks exactly like a userspace driver that does not work.
- e1000_release() gives up the interface, not just the hardware.
8139too still passes 4/4; ldls-check still passes 27; four CPUs, no panic.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
docs/plan/ldls.md L3d, and with it L3 -- the milestone the plan calls the one
that proves the whole idea and the one that cannot be skipped.
# ldlsd /lib/modules/8139too.ko /lib/modules/btf &
kpi: the driver read a MAC off the hardware: 52:54:00🆎cd:ef
ndo_open returned 0
carrying frames. the interface is ldls0.
# cat /proc/netif
interface ldls0 state up address 10.9.0.15 gateway 10.9.0.2
# ping 10.9.0.2 4
64 bytes from 10.9.0.2: icmp_seq=1 time=30 ms
64 bytes from 10.9.0.2: icmp_seq=2 time=20 ms
64 bytes from 10.9.0.2: icmp_seq=3 time=20 ms
64 bytes from 10.9.0.2: icmp_seq=4 time=20 ms
10.9.0.0/24 is reachable only through ldls0, and ldls0 is 8139too running in a
SchOS process. The address was obtained by DHCP over the same driver first.
lwIP -> netdev_ops -> /dev/netdev -> ldlsd -> ndo_start_xmit -> a BAR mapped by
/dev/pci -> the wire, and the whole way back. Nothing in the kernel knows a
Linux driver is involved; nothing in the driver knows it is not in Linux.
It polls rather than taking interrupts. /dev/pci delivers them by blocking
read(), which needs a thread this program does not have, so the driver's NAPI
poll is called from a loop instead -- the same work in a context LDLS can
provide, at the cost of latency and a spinning process.
Four things Linux's alloc_etherdev does that allocating does not, each found by
the driver faulting on it: dev->dev_addr is a pointer aimed at perm_addr;
dev->mc and dev->uc are list heads and are empty only when they point at
themselves; dev->_tx is an array of netdev_queue, because netif_start_queue
clears a bit in _tx[0].state; and __LINK_STATE_START is set by dev_open before
ndo_open is called.
Two did not announce themselves, and they are the ones worth recording.
netif_running(dev) gates 8139too's receive loop. Without it the chip received
the frame, the driver acknowledged the interrupt and left the loop, and nothing
came out -- while every counter said packets were arriving. Found by reading
the chip's own CBR register through the BAR and seeing the hardware had written
600 bytes the driver never collected.
memcpy was still a logging stub. The driver copied each frame into its sk_buff,
the copy did not happen, and the stack was handed exactly the right length of
zeroes; DHCP never completed and nothing reported an error. A stub that returns
zero is not safe for a function whose whole job is a side effect, and the
memory primitives are the worst case because they fail silently and plausibly.
ldls/kpi/string.c exists because of it.
Still one driver, one kernel version, tried once, against an emulated card --
which is what the plan has always promised and is worth repeating next to a
working ping.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
docs/plan/ldls.md milestone L3c. ldls/kpi/ implements 41 of 8139too's 94
symbols rather than logging them: register access to a mapped BAR, the PCI
calls to /dev/pci, allocation to mmap and PCIIOC_DMA, register_netdev to
/dev/netdev.
# ldlsd /lib/modules/8139too.ko /lib/modules/btf
found 10ec:8139 at 00:05.0
bar 1 0x0000000081060000 256 mem
calling probe, with 41 of 94 symbols implemented
kpi: mapped BAR 1 -- phys 0x0000000081060000, 256 bytes
kpi: the driver read a MAC off the hardware: 52:54:00🆎cd:ef
netuser: ldls0 is a process now, 52:54:0🆎cd:ef
probe returned 0 after 0 stub call(s)
52:54:00🆎cd:ef is what the machine was configured with. Nothing invented
it: the driver bit-banged the card's EEPROM, serially, through a BAR mapped by
/dev/pci, and handed back what it found. That it matches means the loader, the
relocations, the BTF offsets, the BAR mapping and the register accessors are
all correct at once, which is why it was the thing to aim at. Zero stub calls:
every symbol probe reaches is implemented.
Two bugs, one of them the kernel's.
Mine first: pci_iomap checked a failed mmap for -1 only, while SchOS returns
-errno. It returned -13 as a pointer, the driver took that as its register
base, and the first write landed at 0x4e -- thirteen below where it meant to
be.
Underneath it, the real one. sys_mmap rounds a length up to a page, the only
granularity an MMU has, and pciuser_mmio_ok() compared that against the BAR's
own size. rtl8139's registers are 256 bytes, the request became 4096, and the
answer was -EACCES for a device the process legitimately held. A BAR smaller
than a page could never be mapped at all, and nothing had noticed because the
only consumer until now was virtio-gpu, whose BARs are large. The check is now
against the pages the BAR lies in, and the widening is named where it happens:
a sub-page BAR hands over the whole page, forced by the hardware rather than
chosen.
That is exactly what hybrid.md said a foreign driver would be good for --
"what it asks for is what a foreign driver would ask for too" -- arriving as a
concrete bug rather than a prediction.
kpi_io_ok() now bounds every ioread/iowrite against the mapping pci_iomap
returned. A stray access would otherwise be a page fault with no context, and
with no IOMMU a hosted driver's wild writes have to be caught early or not at
all. It is also what turned the fault above into two lines naming the offsets.
Not done, and said so in the output rather than only in the doc: no frame has
moved. ndo_open is never called and there is no transmit or receive path, so
ldlsd releases the interface instead of leaving it up -- a link that reports
itself up with nothing draining its queue is a black hole, and the stack would
send DHCP into it for ever. Down is the true answer until packets move. That
is L3d, now listed separately.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
docs/plan/ldls.md milestone L3b. ldls/ko.c, btf.c and stub.c were written
freestanding so they would build unchanged on both sides; cmd/ldlsd collects
that promise -- the same three files, linked against lib/ instead of a libc,
in an ordinary SchOS process.
# ldlsd /lib/modules/8139too.ko /lib/modules/btf
module 8139too license GPL wants 94 symbols
defused 6 %gs stack-guard reference(s)
calling init_module
1 _printk(...)
| 8139too: 8139too Fast Ethernet driver 0.9.28
2 __pci_register_driver(...)
btf 122279 types
layout pci_driver.probe +16 pci_dev 2680 bytes
driver "8139too" devices it claims: 10ec:8139 ... and 18 more
calling probe
1 alloc_etherdev_mqs(0x368, 0x1)
probe returned -12 after 1 stub call(s)
Byte for byte the host harness's behaviour, which is what the shared source
was for. Two earlier predictions came good with no work: a process's mmap area
starts at 0x40000000, inside the window R_X86_64_32S needs, and the kernel
never sets EFER.NXE so every mapping is executable. The second is a gap rather
than a feature and is now recorded as one -- LDLS is relying on the absence of
a protection an OS should have.
One thing did not just work. The first run faulted before probe's first call,
err 5 -- present, read, user -- at a kernel address, with the rip resolving to
mov %gs:0x0(%rip),%rax # rtl8139_init_one+0x26
Modules are built -mstack-protector-guard-reg=gs, so the canary comes through
the GS segment, and that is an ABI between the kernel and its own per-CPU area
which a host that is not that kernel cannot provide. ko_defuse_gs() rewrites
those references to xor %reg,%reg.
There are two forms and the second is easy to miss: the prologue loads the
guard with mov (8b) and the epilogue compares it with sub (2b) before a jne to
__stack_chk_fail. xor is right for both -- it zeroes the saved canary and sets
ZF, so the epilogue's branch falls through exactly as a passing check does. Six
sites in 8139too.
It checks every site before patching any and refuses the whole operation on
anything it does not recognise. That refusal is what found the sub form, by
declining to guess -- a loader that half-understands an instruction should not
be editing it.
make ldls-stage/ldls-unstage put a module and its kernel's BTF into the local
sysroot and take them out again. Not part of `all`, and .gitignore refuses
*.ko, so an image only carries somebody else's GPL code between two commands
you typed on purpose.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
docs/plan/ldls.md L3a -- the class bridge, and the first of the three pieces
L3 turned out to be.
/dev/netdev is /dev/pci with the arrows reversed. That hands a device to a
process; this lets a process be one. The kernel side is netdev_ops and nothing
else -- transmit, receive, mac, link_up, the same four entries e1000
publishes -- so a process claims the device, supplies a MAC, and from then on
read() gives it frames the stack wants sent and write() hands it frames that
arrived.
cmd/netecho proves it and is deliberately not a driver: no hardware, none
claimed, every frame answered by the program itself. DHCP so the interface
gets an address at all, ARP so the stack can find it, ICMP so it can be
pinged. All built by hand from RFCs 826, 791, 792 and 2131.
# netecho &
netuser: ldls0 is a process now, 2:4c:44:4c:53:1
netecho: I am 10.9.0.1 and will offer 10.9.0.2
netecho: answered DHCP, offering an address
# cat /proc/netif
interface ldls0 driver netuser address 10.9.0.2 gateway 10.9.0.1
# ping 10.9.0.1 3
netecho: answered an ARP
netecho: answered a ping
3 packets transmitted, 3 received, 0% packet loss
The interface got its address by DHCP from a process and then answered ping
from one. lwIP never learns that the thing behind netdev_ops is not a driver,
which is the whole property the bridge exists to have.
It dies the way it should, too. kill the process and link_up() answers false
immediately, so the stack refuses to send -- "ping: sendto: -5" -- rather than
swallowing packets into a driver that is not there. Start another and the
*same* interface comes back, keeping its address. That came free:
netuser_release_all() sits beside pciuser_release_all() in exit() for exactly
the same reason, and the reason is that a process holding a device is one that
may crash.
One thing the kernel needed: netinit() starts DHCP on the devices that exist
when it runs, so an interface created later -- which is how an LDLS-hosted
driver arrives -- would have come up with no address and no way to get one.
net_attach_late() fixes that, and lives in kernel/net/net.c because that is
the only file allowed to include lwIP's headers.
Recorded as not done, because it is not: this proves the bridge, not a driver.
Nothing in that test loads a module. That is why it was done first -- if ping
had not worked here the bridge would be wrong, and now a failure with a real
driver can only be the driver's side. L3b is the SchOS-side host, L3c is the
fifteen symbols for real, and L3 is the two joined.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
docs/plan/ldls.md milestone L2, less the half that needs /dev/pci.
In Linux, __pci_register_driver ends at the PCI core, which matches a
driver's id_table against the bus and calls probe(). There is no PCI core
here, so LDLS keeps the pointer that call was given and plays the part.
To do that it must know where pci_driver.probe is, and it asks the kernel
rather than being told by us. ldls/btf.c reads /sys/kernel/btf/vmlinux --
122,279 types, every struct layout and every function prototype, as generated
data describing the kernel that is installed. Implemented from
Documentation/bpf/btf.rst, which is a published specification.
$ ldls-types /sys/kernel/btf/vmlinux --need
pci_driver probe offset 16 size 8
pci_dev resource offset 1000 size 1088
resource start offset 0 size 8
all present
Cross-checked against pahole on the same file, field for field. No offset in
this is ours, which is rule 3's uncomfortable row disappearing rather than
being argued about.
And then the driver:
driver name "8139too"
devices it claims: 10ec:8139 ... and 16 more
calling probe(pci_dev 10ec:8139, ...)
1 alloc_etherdev_mqs(0x368, 1, 1)
2 pci_enable_device / 3 pci_request_regions / 4 pci_set_master
5 pci_iomap(pdev, 1, 0)
6..544 ioread8 / iowrite8 -- 0x88, 0x8a, 0x8c, 0x8e, over and over
545 dev_addr_mod / 547 netif_napi_add_weight_locked
549 timer_init_key / 550 register_netdev / 551 netdev_info
probe returned 0 after 552 stub call(s)
10ec:8139 is the real RTL8139 id. It mapped BAR 1, the one marked
IORESOURCE_MEM. Those 538 alternating byte accesses stepping 0x88/0x8a/0x8c/
0x8e are serial EEPROM clocking -- the driver bit-banging the chip for its MAC
address and handing the result to dev_addr_mod. Then NAPI, a watchdog timer,
register_netdev, its "found it" line, and zero.
Fifteen distinct symbols carry that entire probe, and three of them are 540 of
the 552 calls. L1 said 44 symbols carry 110 modules through init; L2 says 15
carry this one through probe. Neither is a number anyone would have guessed
and both came from running the thing.
Recorded as not done, because it is not: probe receives a pci_dev whose shape
is right and whose contents are invented. The BARs are made up, pci_iomap
returns a zeroed page, and 8139too read a MAC address of zeroes and was
perfectly happy about it. The "all through /dev/pci" half of L2 belongs to the
SchOS side. Also recorded: BTF has to come from somewhere, and SchOS has no
/sys/kernel/btf/vmlinux -- the user brings it with the module, from the kernel
the module was built for.
make ldls-check is 27 checks, four of them L2's, and it says so out loud when
it skips them on a machine with no BTF rather than passing silently.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
docs/plan/ldls.md milestone L1. Every undefined symbol resolves to a
trampoline -- twenty generated bytes that load the symbol's index and jump to
a shared thunk, which saves the argument registers and reports the call. Then
init_module is simply called.
$ ldls-run 8139too.ko
module 8139too
license GPL
wants 94 symbols
calling init_module at 0x416fc020
1 _printk(0x416fb6c0, 0, 0)
| 8139too: 8139too Fast Ethernet driver 0.9.28
2 __pci_register_driver(0x416fb400, 0x416fff80, 0x416fbdbb)
init_module returned 0 after 2 stub call(s)
That second line is the driver's own banner, printed through a _printk that
is ours, from a string in the module's own .rodata that a relocation put
there. There is no stronger evidence that a load worked than the loaded thing
speaking.
Stubs return zero rather than stopping, which departs from what the milestone
originally said and does so deliberately: stopping at the first call tells you
one name, carrying on tells you the sequence, and the sequence is the thing a
symbol table cannot give you. A driver that then dereferences the zero faults,
and that is a result too -- it says how far it got before it needed the kernel
to have really done something.
The first run failed on neither of those. It failed on a relocation:
ldls-run: relocate: a signed 32-bit relocation does not fit
A module is compiled -mcmodel=kernel, so the compiler emits R_X86_64_32S for
anything it believes lives in the top 2 GB, and an ordinary mmap at 0x7f...
has nowhere to put it. Linux gives modules a region within 2 GB of the kernel
text; the harness uses MAP_32BIT to reach the same window from the other end.
On SchOS this costs nothing -- a process's mmap area starts at 0x40000000 and
is already inside it -- but it is a real constraint on where the host may load
a module, and a loader that did not check its own arithmetic would have found
it much later and much more confusingly.
Swept over 500 stock modules: 449 ran init to completion, 22 faulted on a
stub's zero, 29 have no init_module, and zero hit a loader error.
Recorded honestly in "what it will not do": running init is not working. Every
symbol is a stub, so what has been shown is that drivers ask for things in a
readable order, not that anything answers. Also recorded: every undefined
symbol in a module is NOTYPE, so the stubs cannot tell jiffies from
pci_iomap and both get a trampoline -- BTF distinguishes VAR from FUNC and
that is where the fix belongs rather than in a guess.
make ldls-check is 23 checks now, five of them L1's, all against
ldls/test/refmod.c which is ours and now has an init_module of its own.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
docs/plan/ldls.md milestone L0. ldls/ko.c indexes the sections of an ELF64
relocatable object, places the allocatable ones, resolves undefined symbols
against a caller's table and applies the relocations. Nothing is executed.
It is freestanding -- no libc, no allocation, no I/O -- so the same source
builds as a host tool now and as the SchOS-side host later. That is the
arrangement mkschfs and the kernel already share, and for the same reason:
the failures at this layer are format failures, and one found on the host
costs a second instead of a boot.
$ ldls-inspect 8139too.ko
module 8139too
license GPL
loads to 33988 bytes
relocations: 1980
R_X86_64_64 363, PC32 991, PLT32 428, 32S 169, PC64 29
undefined symbols: 94
stops at: unresolved: __fentry__
The licence is printed before anything is loaded, because it is the one
property that decides whether the object may be on the machine at all and it
is a plain string in .modinfo. Rule 1 is easier to keep when the answer is on
screen.
94 symbols is the measured API surface of 8139too on 6.18.46 -- the list,
exactly, rather than an estimate -- and it is L1's work queue. It is smaller
than it looks: __fentry__, __stack_chk_fail, __ref_stack_chk_guard,
__x86_return_thunk and __fortify_panic are compiler and instrumentation
artefacts that stub out in a line each.
Swept over every module in a stock 6.18.46 tree: 2,440 objects, all of them
laid out and relocated to completion, none with a relocation type the loader
does not handle. That sweep is also what retired the fixed section table --
nvidia-modeset has 7,069 sections and 74,305 relocations, against a median of
50 -- so the caller supplies the table and there is no limit to run into
rather than a generous one that fails later.
Loading is not running, and the doc says so: not one symbol is implemented,
and the distance between "relocates" and "transmits a packet" is the rest of
the plan.
No .ko is in this tree and .gitignore now refuses one outright. make
ldls-check runs 18 checks against ldls/test/refmod.c, which is ours, written
to contain one of each thing the loader has to get right.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
H3 of docs/plan/hybrid.md -- "a modeset API, which is the thing an
out-of-kernel GPU driver makes affordable and an in-kernel one never got".
# schmode modes
1280x800 *
1280x720
1024x768
800x600
640x480
# schmode 1024 768
1024x768 ok
Captured off the virtio-gpu head, the frames really come back 1280x800,
1024x768, 640x480 and 1280x800 again, with the checkerboard band showing no
shear at any of them.
Nothing in the kernel changed for this. No syscall on the path has heard of
a scanout: schmode writes a line to a socket and vgpud sends five commands
to the device. The comparison is the point -- the same five commands in the
kernel would have been the same five commands *and a reason to be nervous*,
because a wrong one there takes the machine instead of one process. That is
why the in-kernel driver never grew one: not difficulty, exposure.
The framebuffer is allocated once, at the largest mode, and every smaller
mode reuses it. PCIIOC_DMA has no free -- a driver's blocks come back when
it exits and not before -- so a modeset that allocated would spend sixteen
slots and then the arena, and the failure would look like a GPU bug rather
than like counting. A larger mode is refused and says why.
vgpud used to end in for(;;) sleep(100) under a comment arguing that
building a control socket before there is a compositor to judge it is the
mistake this plan warns about. That was right about a *compositor protocol*
and wrong as a reason to have no socket: a modeset has a consumer that
exists, somebody changing the mode and looking at the screen, and schmode
is small enough to judge an interface this small. What is deliberately not
on that socket is everything a compositor would need -- no buffer
submission, no damage, no vblank -- which still waits, for the original
reason.
H4, the USB eviction, is now the only milestone left on the plan and is by
some distance the largest thing on the register.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
docs/plan/debugger.md D3 and D4, which finish the plan apart from the
network transport.
D3 -- `whose <addr>`. The views each speak for one subsystem, which is what
keeps them cheap and current, and it is exactly why none of them can see
the bug that matters: a page on the DMA arena's free list looks fine from
dma.c, the same page on kalloc's free list looks fine from kalloc.c, and
that it is on both is a fact no single subsystem holds.
# schdbg whose 0x3e239000
phys 0x000000003e239000 kernel va 0xffff80003e239000
dma arena, handed out
pcidma slot 2, 0 KB into a 4000 KB block given to pid 4 (vgpud)
mapped pid 4 (vgpud) at 0x000000004000d000 rwu vma[5] no_free
consistent: 1 owner, 1 mapping(s)
Claimants register the way views do -- one line in the file that owns the
data -- so the kernel image, the log region, the arena, the page allocator,
the pciuser DMA blocks and every page table each answer for themselves.
`mapped` is the one no allocator can answer: there is no reverse map, so
vm_walk_user() goes forward over every mapped page of every live process.
Two verdicts are shouted and everything else is reported as fact. More than
one owner means the second allocator to hand the page out is handing out
memory the first is still using -- the H2 bug, which would have printed as
three lines. Free-but-still-mapped means a page will be given away from
under the page tables that point at it. Deliberately *not* a finding: no
owner at all, because kalloc keeps a free list and no record of what it has
handed out, so an ordinary allocated page has nobody to speak for it and
saying otherwise would be the query crying wolf.
D4 -- watchpoints, on the four debug registers that had been sitting unused
since this kernel was written. Armed against the same framebuffer page and
then killing the driver:
watch 0: 0xffff80003e239000 written, 8 byte(s), by stosq+0x1f, pid 12 vgpud
#0 memset+0xba
#1 dma_alloc+0x2e2
#2 do_dma+0x22a
#3 pciuser_ioctl+0x13b
#4 sys_ioctl+0x3c7
#5 syscall+0xf6
The chain is the answer, not the rip. It reports and continues -- a data
watchpoint is a trap, delivered after the store completes, so returning
carries on with the program none the wiser, which is what lets this be used
on a machine that is still running.
Debug registers are per-CPU and there is no general IPI here, so the table
carries a generation number and each CPU reloads from its timer interrupt
when it notices the number has moved: one comparison per tick, one tick of
latency at worst. `watch` prints which CPUs have caught up, because "this
watchpoint is watching" and "this watchpoint is watching one CPU" must not
look the same.
Also fixed while testing: watch_init() was called from mpmain(), which runs
on every processor, so the command appeared in the list once per CPU.
dbg_register() now refuses a duplicate name outright, so the next init that
runs per-CPU fails loudly in the list rather than quietly doubling it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
docs/plan/debugger.md, milestones D0, D1 and D2. What they replace is the
ninety-second loop -- add a cprintf, rebuild, boot, read the serial log --
which answers exactly the one question you thought of before the rebuild.
The rule the shape follows: **it observes, it does not control.** No
stepping, no software breakpoints, no stopping the world. Half the bugs
this kernel produces are timing -- an interrupt storm, a level-triggered
line nobody acknowledges, a driver that faults only on its second start --
and every one of them disappears when you halt the machine to look. And
the layering rule, which is the circularity rule the display already has:
a debugger that needs lwIP cannot debug lwIP.
D0 -- symbols. tools/ksyms.pl turns the first link pass into a table the
second pass carries. Two passes is enough because .ksym is placed after
.text and .rodata, so carrying the table cannot move anything recorded in
it; the build lists both passes' text symbols and refuses to link if they
ever disagree, because a symbol table that is subtly wrong gives confident
false answers. ksym.c reads it through weak symbols, so pass one -- which
has no table -- still links. A crash chain now reads panic+0x5f <-
proc_sysrq_cmd+0x99 <- ... <- syscall+0xf6.
D1 -- a log that outlives the kernel. Every character through consputc()
lands in a byte ring with a timestamp per line, readable as /proc/kmsg.
It sits in a region carved before the page allocator gets anything, with a
magic header, so a kernel finds the previous kernel's log still there --
including the whole PSOGM report. panic.c's own comment said "there is no
core dump and no log to go back to"; that is no longer true.
The obvious place for it is the top of RAM beside the DMA arena. That
was the first attempt and the magic came back as ASCII on the reboot,
because the top of usable memory is exactly where the firmware makes its
own allocations. It lives above the kernel image instead, which is a
stable address across boots of the same build.
D2 -- the introspection registry. A subsystem publishes its state with one
line at the bottom of the file that owns the data:
dbg_register("dma", dma_dump, "the DMA arena's free extents");
Seven views -- dma, mem, proc, vma, claims, pci, sock -- reachable three
ways: sysrq 'g' (always, no process and no filesystem), /dev/dbg (echo a
line, cat the answer), and schdbg. The protocol is lines of text because
the same bytes have to work where nothing can allocate or frame a record,
and because a protocol you cannot drive by hand is one you cannot debug.
struct sbuf moved out of kernel/dev/procfs.c into include/sbuf.h rather
than being written a second time.
Several views exist because of a specific bug. `vma` is the table that
decides whether exit() frees a page, and the DMA ownership bug that made
this worth building was a block that was never in it.
Also fixed, found by building with clang for the first time in a while:
struct iovec was defined in both posix.h and socket.h, and every file that
included both was a redefinition. GCC 15 defaults to C23, which permits an
identical struct redeclaration, so the gcc path never noticed -- while
clang defaults to gnu17 and rejected it, breaking the one toolchain
shell.nix exists to provide and the UEFI loader requires. Both are now
guarded.
D3 (`whose <addr>`, the cross-subsystem ownership query) and D4 (hardware
watchpoints on DR0-DR3) are not built and are described honestly as
missing in the architecture doc's "What it does not do".
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
The failure this is against is not scope creep. It is abandonment by promotion:
a thing gets built to the point where it works, the next thing becomes more
interesting, and the first is left in a corner -- in the tree, not finished, not
deleted, and not in any list. Enough of those and the tree stops matching its
own documentation.
docs/plan/1.0.md now holds the register, and it is the only place anything
half-built is listed. Four sections, because the kinds are genuinely different:
built and connected to nothing (the text layer, a font from disk, socket
activation, wait_vblank); started and stopped part-way (foundations 2-5, hybrid
H2, Schcom v0, a released PCI device that nothing brings back); named in a "what
it does not do" and still true; and frozen -- scoped, unstarted, not to be
started. The rule is that nothing enters except by being finished or deleted,
and that adding a row requires closing one.
1.0.md already said "two things in flight, never three". It was right and it was
not being followed.
Then the corners themselves, because a register written over a tree full of them
is just a longer list:
- cmd/init is deleted. Nothing had execed /bin/init since initcode.S changed
to /bin/schinit; it was built and installed on every image as an unreachable
fallback, and an unreachable fallback rots rather than saves.
- projects/schwm/ and projects/schde/ are deleted. Both said "to be deleted"
in their own first paragraph, which proves only that a directory marked for
deletion is not deleted by being marked. README-only, no code lost, and the
arguments each made are moved into wayland.md where they are still true.
- projects/schcom/schcom and schcom-demo were tracked binaries. Untracked, and
.gitignore'd.
Three code comments and one document described a boot that has not happened
since 7d44eb7. cons_serial_only starts set, but kmain clears it on its first
lines now, so the splash does not survive the boot and the kernel log is on
screen from the start. That reversal was earned by a Chromebox with no serial
port freezing at boot showing nothing at all, and the comments still argued the
losing side of it.
The site:
- a download page. One ISO, what it has actually been run on -- one machine,
under test -- Ventoy and dd, what the first boot looks like step by step,
and the build if you would rather. The support table names one Chromebox
rather than listing hardware, because one Chromebox is the entire bare-metal
record and a list would be fiction.
- Assets/usb.svg, drawn to the same rule as the other marks: two tones,
geometry only, the accent spent in one place, and that place is the activity
light because it is the part that tells you something is happening.
- Download is in both navs and is the hero's primary action; a closing band
ends the front page on it.
- the uberblock-slots and drivers-outside-the-kernel figures are gone. Three
numbers left, including the fastfetch one, which stays because it is funny.
Two things caught by rendering the pages instead of assuming: .hero p was
beating .filemeta on specificity and the file line was coming out at body size,
and the dd command was one line long enough that the card clipped it at `conv=`
-- a destructive command that can scroll half of itself out of view is a good
way to take somebody's disk. It is two lines now.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
docs/plan/hybrid.md's first eviction. cmd/vgpud is the virtio-gpu driver as a
process: it finds the device by vendor and device id, asks the kernel for it,
and does the whole virtio 1.0 bring-up from outside -- reset, feature
negotiation, a virtqueue in DMA memory, GET_DISPLAY_INFO, RESOURCE_CREATE_2D,
ATTACH_BACKING, SET_SCANOUT, TRANSFER and FLUSH.
vgpud: virtio-gpu at 00:03.0, currently held by virtio-gpu
virtio-gpu: released to userspace; screen is the VBE framebuffer
vgpud: claimed
vgpud: display is 1280x800
vgpud: scanout is ours, framebuffer at physical 0x3eceb000
vgpud: drew a test pattern from userspace
Verified by screendumping the virtio-gpu head: the pattern is there, with a
checkerboard that would shear if the stride were wrong. The console carries on
in VBE on the other head at the same time, which is the fallback rule from the
plan working rather than being described.
No syscall knows this is a GPU. The kernel handed over a PCI device and the
process decided what it was -- which is the property that makes this a general
mechanism, and the reason the evictions come before anything foreign.
Four things the first real driver needed that the test program had not:
- PCI config space (PCIIOC_CFG). A virtio device says where its registers
are in vendor capabilities, which live in config space, so a driver that
cannot read it cannot find its own registers.
- a way for the incumbent to let go: struct pci_driver gained a release
hook. A driver without one keeps its device, which is always safe and for
anything on the path to a shell is also correct.
- the outgoing driver freeing its DMA. Releasing the device but keeping its
4 MB framebuffer is half a handover -- the incoming driver asks for one and
is told there is no room.
- a 16 MB DMA arena. 4 MB was exactly one framebuffer at this resolution,
and either side of a handover there are two.
And one bug it reproduced on its first working run: vgpud polls the used ring,
so it registered no interrupt handler, and a virtio device interrupts on every
completion. The storm detector said so immediately --
irq 11: 77780 interrupts/second -- a device on this line is not
acknowledging it. handler 'e1000' is attached
-- naming the line, the rate and the neighbour it was drowning. One line fixes
it (VRING_AVAIL_F_NO_INTERRUPT before the queue is enabled), and that detector
was worth every minute it cost.
include/virtio.h moved out of the driver directory. Its own comment said it
should when a second reader appeared; one has.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
docs/plan/hybrid.md mechanisms 2 and 4: MMIO and DMA to a process.
A process can enumerate the bus, claim a device exclusively, map its BARs and
get memory the device can be pointed at. Proven end to end against an e1000e
-- the exact device class we cannot drive and would want a foreign driver for:
device 7: 8086:10d3 (network)
claimed
BAR0 at 0x81060000, 128 KB
regs[0..3] = 0x00140261 0x00000000 0x00080283 0x00000000
MMIO reaches the device
DMA 8 KB at virtual 0x40001000, physical 0x3f8eb000
Those are the e1000e's CTRL and STATUS with full-duplex and link-up set, which
is how you tell a real mapping from one reading 0xFFFFFFFF off an unmapped
address. Claiming a device the kernel drives is refused -- two writers of one
register set looks like hardware that has started lying -- and everything a
process held is released when it exits, because a driver that crashed must not
keep its hardware away from its replacement.
mmap's offset is the BAR's physical address, checked against the BARs of
devices *this process has claimed*. Without that check the device is /dev/mem
with extra steps and a process could map the kernel's own memory. Mappings are
write-through: a cached write to a doorbell arrives whenever the cache feels
like it, which from a driver's side is indistinguishable from hardware that
ignores you.
The interface is about PCI devices, not about any driver model, and its first
users will be SchOS's own drivers when virtio-gpu is evicted. That ordering is
the design rather than an accident: an interface whose first consumers are
native drivers is a general mechanism something foreign can use later, instead
of a hole shaped like something foreign.
Also: PROT_* and MAP_* moved to include/posix.h. They were defined only in
sys_mem.c, so cmd/fbinfo carried a private copy and this wanted a third --
two copies of a number the kernel compares against, where a mismatch shows up
as a mapping with the wrong permissions rather than an error.
No IOMMU, so this isolates a driver against its own bugs and not against its
own DMA. The plan says so at length and it is still true.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
QEMU carries a gdb stub and exposing it costs two flags. `make qemu-gdb` holds
the machine before its first instruction and waits on port 26000; attach and
you get breakpoints, single-stepping and symbols in kernel code -- including
the early boot the crash screen cannot reach and the serial console has not
started for.
.gdbinit does the connecting, and documents the one confusing part rather than
leaving it to be rediscovered: at the reset vector a breakpoint on a kernel
symbol fails with "Cannot access memory at 0xffff8000...", because the kernel
links into the high half and nothing is mapped there until it enables paging.
Symbols resolve fine -- gdb knows kmain is main.c:32 -- the address just is not
addressable yet. Continue, let it boot, interrupt, and everything works.
This is for QEMU. On hardware with no serial port the tools are the on-screen
kernel log and PSOGM's QR code, which is what that QR code has always been for.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
ports/toybox already installed its applets as hard links to one multicall
binary, with a cp fallback. The links had never happened: the install is
stamped, the stamp predated the ln, and 236 separate 784 KB copies had been
sitting in sysroot/bin ever since. Forcing the install: 194 MB -> 13 MB.
Which immediately broke everything, in a way worth writing down.
**/bin/ping exists twice.** toybox has a ping applet and cmd/ping is a
first-party program that talks to lwIP directly. The install rule for cmd/
programs was `cp`, and cp onto an existing hard link writes *through* it -- so
copying cmd/ping over the applet of the same name replaced the contents of the
shared inode and all 236 links became ping. `cat /proc/uptime` answered
"ping: not a dotted-quad address".
Invisible until the links existed: while the applets were separate copies,
overwriting one affected exactly one. Fixed from both directions, because it
can be hit either way -- the cmd/ rule removes the target before copying, and
toybox skips names SchOS provides itself.
**The USB test image stopped fitting.** mkfs writes each hard link as its own
copy, so /bin expanded to over 60 MB on the way in. That image exists to prove
the USB driver can read one block, and bootcheck asserts on its 16 MB size, so
it excludes /bin rather than growing.
The ISO is the point of all this: 196 MB before, 62 MB now, and it boots to a
shell with RAMROOT=1.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
docs/plan/init.md milestones I0 and I1.
**The split.** schinit is PID 1 and does four things: bootstrap the
filesystems, fork schinitd, fork jobs on schinitd's behalf, reap children. No
manifest parsing, no filesystem walking after boot, no allocation. It is the
one program on the machine with no supervisor, so it is written to be too
boring to fail.
schinitd is PID 2 and holds everything else -- /etc/schinit, respawn policy,
throttling -- and is restartable, which is where the bugs are allowed to be.
**Jobs are forked by PID 1, not by schinitd.** That is the load-bearing
detail. schinitd decides and asks; PID 1 acts. So every job is a child of PID
1 at all times, and a supervisor that dies orphans nothing, reparents nothing
and restarts nothing.
**/proc/schinit** is the job table, written by PID 1 whenever it changes. The
kernel holds the text and does not interpret it: a procfs generator runs in
the kernel, and walking PID 1's memory to answer a read would put a userspace
program's format into the kernel, where it would rot.
Verified the way the plan asks for -- `kill 3`, then look:
schinitd: adopting os.schos.console-shell
schinitd: adopting os.schos.tty2-shell
/proc/schinit byte-identical across the restart, same pids, same cookies, same
start times, and the shell being typed into never noticed.
The first attempt got I1 wrong in the instructive way: PID 1 published the
table and schinitd never read it back, so the restarted supervisor started
everything a second time -- two console shells on one terminal, which is the
split-brain this plan rejects the two-init scheme for, reached by a different
road. Publishing state is half of it; coming back and believing it is the
other half.
The protocol is include/schinit.h: fixed-size messages, versioned in the
hello, entirely internal. Fixed-size so PID 1 never allocates, and a
supervisor that has gone mad cannot make it run a parser.
Still to come: I2 (heartbeat both ways, so a hang is as detectable as a death)
and I3 (proc_reincarnate, the actual law change).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
/dev/tty1..tty6, Alt+F1..F6 or `chvt N`. Each is an independent terminal --
its own screen, cursor, line discipline, foreground process group and
session. Two shells on two terminals are unrelated processes that share a
keyboard and a screen.
What made it possible is a backing store. The console drew straight into the
framebuffer and scrolled by moving pixels, which works exactly as long as
nothing else wants the screen: there was nowhere to put terminal 2 while
terminal 1 was visible, and nothing to repaint terminal 1 from. Each terminal
owns a grid of cells now and the framebuffer shows whichever grid is active,
so vt_switch() is set-index, repaint, return -- nothing is saved on the way
out because the outgoing grid has been current all along.
The rule the layer is built on: a terminal does not know whether it is
visible. Writes land in the grid, the cursor moves, the grid scrolls, all
unconditionally; drawing is one question asked at the bottom of vt.c. Every
alternative spreads "am I on screen" through the escape parser, and then a
terminal switched away from mid-sequence comes back wrong.
Everything that was a single global is per-terminal now: tty_fg_pgrp (deleted),
termios, the input ring, the escape parser, the UTF-8 decoder, and the cursor
-- which also stops being pixels in gfx.c and becomes cells in the terminal.
TIOCSCTTY was `return 0` and now records proc->ctty, which is what /dev/tty
resolves through; /dev/tty was an alias for /dev/console with the same minor,
and they are different questions.
kbd.c names Alt+F1..F6 as KEY_ALT_F1+n, above 0xFF on purpose -- every byte
value is already a character or a UTF-8 continuation, and a chord colliding
with one would be typed into the terminal instead of switching away from it.
What the chord means is decided in console.c, where the terminals are.
VT_ACTIVATE / VT_WAITACTIVE / VT_GETSTATE / VT_OPENQRY, Linux's numbers.
The argument to ACTIVATE is by value, not a pointer -- same shape as TCFLSH.
Two bugs found by screenshotting the result rather than reading the log:
- zsh emits 0x0F to leave alt-charset mode, because that is what the `linux`
terminfo says rmacs is, and the console drew CP437 glyph 0x0F: a sun, in
the prompt, on every line. A terminal never draws a C0 control character.
- SGR 7 was unimplemented, so zsh's partial-line marker rendered as a bare
'#' instead of a reversed block. Reverse video swaps the colours at draw
time, not at store time, so turning it off restores what was underneath.
Verified by screendump: tty1 and tty2 show different screens, and switching
back repaints tty1's earlier content from its grid. schfs-check passes.
docs/architecture/terminals.md, including what it does not do -- no
scrollback, no reflow, no VT_PROCESS mode for a display server to veto a
switch, which Schcom will need before it owns Alt+F7.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
Porting zsh found four defects in the kernel and one in the filesystem. None
of them were about zsh; all of them had been there the whole time.
**exec threw the environment away.** kernel/fs/exec.c wrote a single zero for
envp with the comment "(no environment)" -- not yet, but ever, for every
process. musl has been passing a perfectly good envp to execve (Linux 59, which
SchOS answers) and it was never read. So $TERM was unset, $PATH was whatever
each program invented, and a variable exported in a shell did not survive the
exec meant to carry it. exec() takes an envp, sys_exec reads argument three,
and schinit manifests grew an Environment key. The userland two-argument
exec(path, argv) stays as a wrapper that passes null explicitly -- the raw stub
is execve now, so a short caller cannot leave %rdx holding a stray pointer.
**Which overflowed the kernel stack.** Three pointer arrays, ~1.8 KB, on a
4096-byte stack, under exec -> loaduvm -> readi -> SchFS -> btree. Pid 1 went
down on the first exec. The canary caught it and named the process, which is
what it is for. They live in one kalloc'd page now.
**waitpid returned a bare -1** for both "no children" and "interrupted". musl
reads [-4095,-1] as -errno, so userspace saw EPERM; zsh printed "wait failed:
operation not permitted" and abandoned the rest of /etc/zsh/zshrc. -ECHILD and
-EINTR now. Third time a bare -1 has cost real time -- on a system with no
permission model, EPERM should be read as "a syscall returned -1".
**NVMA was 64.** musl's allocator takes a mapping per size class and does not
release them eagerly, so compinit ran out of slots and got -1, which is EPERM,
which malloc turns into NULL, which zsh reports as "fatal error: out of memory"
-- on a machine with 1000 MB free. 256 slots, and -ENOMEM.
**And a small write to SchFS costs 58 ms.** 200 writes to one open file take
11.75 s; the same to /dev/null take 0.14 s. That is the design stated at the
top of kmount.c -- one transaction group per operation, and a commit is two
flushes and an uberblock -- not a bug. But compdump writes its digest with
thousands of separate prints, so it takes minutes and looks like a hang.
compinit -D, which skips the dump, finishes in 3.6 s. This one is a decision
rather than a fix, so it is written up and left: /etc/zsh/zshrc keeps compinit
switched off and explains exactly why, with the measurements.
zsh runs as /bin/zsh and is the console shell. ZLE works -- arrow keys recall
history through terminfo, $COLUMNS reads the real 160 because TIOCGWINSZ was
fixed, colour and prompt expansion work, startup is 0.14 s. mksh keeps /bin/sh,
so schinit's no-jobs fallback is a different program from the one that could
have failed.
Also fixed while here: AT_CLKTCK was the literal 100 and is USER_HZ; the
extract rule in four ports could move a source tree inside itself when make
re-ran it, which is how zsh's configure came to write modfile=zsh-5.9/... into
config.modules and fail three steps later.
schfs-check passes. Full writeup in docs/history/kernel.md section 14.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
fastfetch reported 3 seconds of uptime after 15 real ones, `time` claimed
3.12s for a 20-second command, and ping's one-second interval was closer to
six. One cause, not three: `ticks` was advancing at ~14.5 Hz instead of 100.
It was not the timer. virtio-gpu-pci shares IRQ 11 with the e1000 and
registered no interrupt handler at all. PCI INTx is level-triggered, so the
device held the line asserted, the kernel signalled EOI, and it fired again
immediately -- 72,000 interrupts a second, 83% of CPU0, starving the tick.
The e1000's handler read ICR as zero half a million times, correctly
answering "not mine", which is what made it look like a network-free problem.
Reading virtio's ISR byte clears it and drops the line. That is the fix.
irq_dispatch now measures each line's rate against the TSC -- not ticks,
which is the first casualty of a storm -- and names the attached handlers
once above 20000/s. It reproduces the original fault as a single line.
Found along the way, and fixed because they were real:
- lapicinit loaded TICR with xv6's uncalibrated 10000000, whose own comment
admitted it was never measured. calibrate_timers() now measures the LAPIC
and the TSC against the 8254 PIT. (It came back at 999.9 MHz, so this was
not the bug -- but the next machine is not obliged to be 1 GHz.)
- microdelay() was an empty function body. Every microdelay(1000) in the
xHCI reset path and the USB hub power-on sequence returned instantly.
- HZ is a constant in param.h now, not the literal 100 written out at a
dozen ticks-to-seconds conversions. USER_HZ is separate and deliberately
does not follow it: musl answers sysconf(_SC_CLK_TCK) with 100 always, so
times() and /proc/stat scale at the syscall boundary the way Linux does.
- TIOCGWINSZ answered a hardcoded 24x80. gfx_cols()/gfx_rows() have known
the real geometry all along; at 1024x768 the console is 128x48, so every
program that lays out a screen was formatting for the wrong one.
- TCFLSH fell through to the catch-all `return 0`, claiming to discard
input without discarding any.
uptime now advances 5.01 seconds per 5 real seconds. schfs-check passes.
Full investigation, including the two wrong hypotheses and the measurement
flaw that protected them, in docs/history/kernel.md section 13.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
A fresh boot spent 2600 ms in schinit's bootstrap() before printing its first
line, and the whole of it was device nodes. Measured rather than guessed: an
instrumented bootstrap reported 2600 ms on a fresh image and 0 ms on the next
boot, because the nodes already existed by then and mknod failed fast. The cost
was therefore invisible to anyone not building images -- which is to say,
invisible to everyone except whoever works on this.
Each node was a real inode on the root filesystem, and SchFS commits every
mutation as its own transaction group: a copy-on-write rewrite of the path to
the root, then an uberblock behind a cache flush. Twenty of those is seconds.
The duplication mattered more than the time. Major and minor numbers are the
kernel's -- devsw[] in kernel/dev/cdev.c is what they dispatch on -- and schinit
held a second copy of that mapping with nothing checking the two agreed. Adding
a character device meant editing a userspace program, and a wrong minor produced
a node that opened successfully and did nothing.
kernel/fs/devfs.c puts the table next to the thing it describes and serves /dev
behind the same fs_ops vtable procfs uses. The namespace is all it provides:
fileread and filewrite see T_DEV and go through devsw themselves, so a read
never reaches it. It is read-only, and dirlink and ialloc answer EROFS rather
than pretending -- a node exists because the kernel has the device, or it does
not exist.
This also makes /proc/filesystems true. It has advertised "nodev devtmpfs" for
as long as it has existed, against a kernel that had no such filesystem.
Two things found while measuring what remained:
* The thirteen mknod() calls for /proc ran *after* the procfs mount, so every
one was resolving a path into a read-only synthetic filesystem and failing.
procfs.c's own note called them a fallback to be cleaned up later; they have
been dead for longer than that.
* The four mkdir() calls for mount points were the rest of it. They are made
by the image builder now (sysroot-install), and the ones in init are a
fallback -- a mkdir that fails because the directory exists costs a path
walk and no writes, where creating one costs a transaction.
bootstrap() is now 0 ms on a fresh image, down from 2600.
init opens /dev/console after the mount rather than mknod'ing a node called
"console" into the root directory, which is what used to leave a stray /console
in `ls /`. The old path is kept as a fallback: if the mount fails there is no
/dev/console and no way to report it.
socktest passes, /proc still lists per-process directories, ps -e still
enumerates.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
Two ANSI bugs, both found by running fastfetch rather than by reading the
parser, and both invisible to anything that spells its escapes out in full.
ESC[m with no parameters means ESC[0m. ECMA-48 is explicit that an omitted
parameter takes its default and SGR's default is 0, and sgr() iterated its
parameter list -- which is empty in exactly that case, so the loop body never
ran and the reset was silently dropped.
Programs emit the bare form constantly. fastfetch writes "\x1b[32m1%\x1b[m", so
everything after the first coloured character stayed green forever. With a
background colour it is much worse: its palette swatch sets a white background
and resets, the reset did nothing, and the console went white-on-white and
stayed there -- filling the screen with white rectangles as it scrolled, which
looks like a corrupted framebuffer rather than one missing branch.
OSC sequences were not a state at all, so ESC ] fell through and the payload
printed as text: a hyperlinked path arrived as "Disk (]8;;file:///\]8;;\)".
Nothing here acts on an operating-system command -- there is no window title and
no hyperlink to follow -- but *discarding* one is not optional, because the
alternative is printing somebody's URL into the middle of their output. ESC \
ends it, and an ESC followed by anything else is handed back to the escape state
rather than eaten, so a malformed OSC does not swallow the sequence after it.
Verified by screendumping the framebuffer after two consecutive fastfetch runs:
identical output both times, near-white coverage down from most of the screen to
2% (the palette swatch's own white cell), and the disk line reading "Disk (/)".
Also installs docs/branding/Ascii/ascii_50.txt as /usr/share/schos/logo.txt --
Shift as characters, at the width that leaves room for an info block on an
80-column console. Installed from the branding directory rather than copied into
sysroot by hand, so there is one source.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
fastfetch reported "getifaddrs(&ifAddrStruct) failed", and the right answer to
that is not a patch to fastfetch. Asking what interfaces a machine has is
something every networked program does, and on Linux there is exactly one way:
a NETLINK_ROUTE socket. musl implements getifaddrs() over it, so a libc call
that looks like a syscall is really a small protocol conversation -- and without
this side of it every caller fails the same way.
kernel/net/netlink.c is a responder rather than a general socket family. No
event stream, no multicast groups, nothing subscribes to link state: a request
arrives, the whole answer is generated into the socket's buffer, and reads drain
it. That shape is chosen, not a shortcut -- getifaddrs sends a dump and reads
until NLMSG_DONE with MSG_DONTWAIT, so generating at send time means a read
never blocks and the non-blocking flag is honoured for free.
RTM_GETLINK returns one RTM_NEWLINK per interface with IFLA_IFNAME and
IFLA_ADDRESS; RTM_GETADDR returns one RTM_NEWADDR per address with IFA_ADDRESS,
IFA_LOCAL and IFA_LABEL. Anything else gets a bare NLMSG_DONE, because the
caller's next move is a read either way and an unexpected error message is worse
than nothing to read.
The data is not netlink's. Interfaces come from the netdev class and addresses
from lwIP's netifs, exactly as /proc/netif reads them; nothing in the file knows
what an e1000 is. Interface indices matter more than they look -- getifaddrs
matches address records to link records by index, so the two dumps have to
agree. Loopback is 1, netdevs follow from 2.
Verified by reading it back through musl rather than by inspecting the bytes:
lo flags=0x49 fam=-1
eth0 flags=0x1043 fam=17 (AF_PACKET, the MAC)
lo flags=0x49 fam=2 addr=127.0.0.1 mask=255.0.0.0
eth0 flags=0x1043 fam=2 addr=10.0.2.15 mask=255.255.255.0
Correct flags, addresses and netmasks, and the link records pair with the
address records.
One thing this does not close: fastfetch's Local IP module defaults to
default-route-only, which calls ffNetifGetDefaultRouteIfName() and reads
/proc/net/route. SchOS has no routing table and no such file, so the module
still shows nothing -- now for a reason that is about routing rather than about
enumeration. That is the next file to provide.
socktest still passes; the AF_UNIX and AF_INET paths are untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
PSOGM drew ":(" -- two dots and a parenthesis. It now draws Guru, the icon from
docs/branding/: a chamfered head with two eye knockouts and a flat mouth, in the
:| register.
The old reasoning is kept in the file rather than deleted, because it is the
thing being traded away: ":(" was "legible as a mood without being a picture of
anything, and does not pretend the machine has feelings about what just
happened." That is a real argument and a face is a louder claim. What it buys is
identity -- ":(" is recognisably Microsoft's, and a crash screen is the single
screenshot most likely to leave the machine, so it is the worst place to be
wearing somebody else's face. The octagon also quotes the chamfered corners of
the old SchOS letterforms, which is the only part of that mark still in use.
Still drawn from primitives, for the original reason: it scales with the display
and costs no space in the kernel image. A scanline fill for the head, two
rectangles and one more knocked out of it -- no allocation, no locks, nothing
that can fail halfway, which is the whole constraint of panic context. The
geometry is expressed in the same 200-unit box docs/branding/guru.svg uses so
the two cannot drift without somebody editing a number on purpose.
The eyes and mouth are knocked out of the head rather than drawn on top of it,
which is what keeps them legible small: a hole in a solid reads at sizes a
stroke does not.
Verified by crashing on purpose -- echo c > /proc/sysrq -- and screendumping the
framebuffer through the QEMU monitor. The serial report still says ":(", and
should: Guru cannot be drawn in ASCII, and a mood marker in a log is not the
icon.
Separately, and found by looking at the boot splash rather than at the code:
boot/common/ui.c includes logo.h, theme.h and font.h, and the Makefile rule for
the loader listed none of them. A new logo therefore did not rebuild the loader.
The splash had been drawing a mask compiled six hours earlier while `make`
reported nothing to do, and the only symptom was a boot screen quietly
disagreeing with the kernel about what the logo is. The kernel half never showed
it because those objects are built with -MD -MP and track their own headers.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
Shift is the logo, Guru is the icon, SchCotie is the mascot, and the difference
between them is not stylistic. An icon works once, at one size, in one state. A
mascot has to be able to *do* things -- confused, pleased, asleep, seen from
behind -- which is why it costs more and is drawn last. Written down because
this is exactly the kind of decision that gets re-derived badly.
Assets/ becomes docs/branding/, which also fixes two things:
* Filenames had spaces in them, so every reference needed quoting. They are
lowercase and hyphenated now.
* Assets/ held hand-written SVG and generated PNG in one directory, which is
the one rule docs/guides/layout.md states.
The guide records what was previously only in people's heads:
* The constraint everything answers to -- include/logo.h is a 128x128
single-channel coverage mask, so a mark has to survive as a solid shape in
one colour and be recognisable at 24 px. Colour is the last decision, never
the first.
* Guru's purple is load-bearing. PSOGM_BG #4B1D9A is deliberately not the
accent #7C5CFF, and theme.h's rule is that the full-bleed purple belongs
entirely to failure. If Guru appears in accent violet on a sticker, the deep
purple stops meaning "the machine stopped" -- and it is the only colour in
the system that means anything.
* Why shift-mask.svg is a separate file from shift.svg. mklogo.py reads ink as
(255 - luminance) * alpha, so tones survive as coverage and the relationship
inverts: darkest here is brightest on screen. Copying the violet fills would
light the object from below.
* Why a raster is committed beside its SVG. inkscape is not a build
prerequisite, so requiring a vector rasteriser would break the build on most
hosts for a file that changes about once a year.
* That the BSD daemon is copyrighted and SchCotie must be drawn from the
concept rather than from Beastie. A daemon is a Greek daimon and a Unix
background process; neither is ownable. The pitchfork, the sneakers and that
face are.
Format rule: SVG, or PNG at high resolution with no lossy compression, nothing
else. SVG is the source of truth for every mark and a raster is generated from
it, never edited.
include/logo.h regenerates byte-identical from the new path -- only the source
comment differs. The two old wordmark SVGs were deleted outside this session and
are left deleted; their PNGs are kept as superseded-*.png for reference.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
The old logo was a thin monoline wordmark in a ring, with a grey gradient. The
kernel draws include/logo.h as a 128x128 four-bit coverage mask, so hairlines at
that size go ragged, the ring spent most of the area on nothing, and at small
sizes the S read as a 5.
Three marks, because each depicts the thing it belongs to and so they stop
competing:
Shift three slabs, displaced. The layer diagram in
docs/architecture/README.md as a solid object, and the kernel's
name as a shape. Dimensional rather than flat -- the tradition
Paul Rand's NeXT cube belongs to, which is what the architecture
already claims as its model.
Uberblock the SchFS ring: twelve slots, one of them newest. Twelve so the
slots divide 360 exactly; an arbitrary count leaves a part slot
crammed against its neighbour where the ring closes. The newest
is marked by REACHING FURTHER OUT as well as by colour, because
colour is exactly what the coverage mask discards.
Guru the PSOGM face, drawn for reference. Not wired in: panic.c argues
for the ":(" on the grounds that it is "legible as a mood without
being a picture of anything", and Guru spends that. Worth deciding
on purpose.
mklogo.py reads ink as (255 - luminance) * alpha, so tones survive as *coverage*
rather than being flattened -- which means the old grey gradient was not
discarded, it was turned into partial transparency. It also inverts: darkest is
most ink, which the kernel draws brightest. The mask source is therefore a
separate greyscale file with the top slab darkest, or the object would be lit
from below. Two files sharing one geometry is a drift hazard and the desc in
both says so.
The splash gets it for free: ui_logo() shades per row with metal() and the
mask's own tones reinforce that gradient rather than fighting it, so the slabs
come out looking machined.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
Same disease as the network stack, one subsystem over. kernel/dev/vga.c
was three things wearing one name: the owner of the framebuffer, a
bitmap drawing library, and the text console. Only the first is a
driver, and because they shared a file they shared a prefix -- so
panic.c drew the crash screen with vga_fill_rect, console.c printed with
vga_putc, /dev/fb0 handed out vga_fb_phys, and every one of those files
named a display driver it had no business knowing about.
Split three ways:
- kernel/drivers/gpu/vga/vga_fb.c owns the VBE framebuffer -- the mode
the loader programmed and handed over, the aperture, and the display
class vtable. It is a driver and now lives with the driver that
finds the card on the bus.
- kernel/dev/gfx.c draws: pixels, shapes, glyphs, the cursor, scroll,
the ANSI erase operations. It names no driver. The framebuffer
arrives through gfx_attach_scanout(), so a driver publishes what it
owns and this file draws on it without knowing what it is.
- include/gfx.h is the interface, and says why it is deliberately not
the display class: that one is about frames -- acquire, present,
vblank -- and the crash screen has to work when nothing else does.
main() no longer calls a display driver. platform_register_devices()
takes the framebuffer address and brings up whatever owns it, which is
where driver names belong; kernel/drivers/registry.c is the file whose
entire job is naming every driver in the kernel.
virtio-gpu got the e1000 treatment as well: instances in an array, the
instance hung on dev->priv by an `attached` hook, and every op reading
it rather than a file-static. It refused the second adapter outright
before, which is not hypothetical -- integrated plus discrete is an
ordinary laptop. The VGA driver deliberately has no such hook, and says
so: there is one framebuffer the loader programmed however many
VGA-class controllers the bus turns up, and a second one publishing the
same scanout is the truth rather than a bug.
Private headers moved next to their drivers: include/virtio.h and
include/xhci.h each had exactly one includer. include/ is for headers
the rest of the kernel needs.
Deleted rather than moved: consoleinit() programmed the sixteen ANSI
colours into the VGA DAC and disabled text-mode blink, by writing ports
0x3C0-0x3C9 from the console. The comment three lines above the colour
table already said this does nothing in a 32bpp linear mode -- there is
no palette to index -- so it was both dead and in the wrong file.
Verified, and this is a refactor so the evidence is that nothing moved:
- bootcheck -Fb -Net: 59 [ok], no failures. That covers the console
through every command, /dev/fb0's geometry, writing pixels through
write() and mapping the framebuffer through mmap.
- make crashshot: the crash screen renders exactly as before --
shapes, glyphs, logo and QR code -- which is the whole of gfx.c
exercised at once, in a panic, with the scanout attached by a driver.
- make schcom-test: five pixel assertions from the host oracle still
pass, so the display class and the compositor above it are unmoved.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The TCP/IP stack reached straight into the first driver anyone wrote.
net.c called e1000_present(), the lwIP glue was named netif_e1000.c and
called e1000_transmit(), and include/defs.h declared the driver's
internals to the whole kernel. The second NIC driver would therefore
have arrived with a near-identical copy of the glue file and a patch to
the protocol stack, which is the arrangement include/device.h was
written to prevent -- the e1000 even said so, with `.ops = 0, // no
class vtable yet; there is no net class`.
There is now. A driver publishes a struct netdev_ops -- transmit,
receive, mac, link_up -- and is finished: it never mentions lwIP, and
nothing above it learns its name. The display class was already this
shape; this is the same shape for networking, and the two now read the
same way in kernel/dev/device.c.
What that made possible immediately:
- One netif per bound device instead of one per driver. lwIP's glue is
netif_eth.c, it takes a struct device, and it holds up to NETIF_MAX
interfaces. net.c attaches every network device it finds and starts
DHCP on each, because which interface the network answers on is not
ours to decide.
- /proc/netif lists all of them, one block of fields each, with the
interface name and the driver that claimed it.
- The ARP self-test moved from the e1000 to the class. Nothing in it
was chip-specific, and it is the first thing anyone will want to run
against the next driver. Still `echo n > /proc/sysrq`.
- Private headers live with their driver: e1000.h was in include/,
where exactly one file could use it. The driver build now adds the
driver's own directory to the include path.
The e1000 also drives more than one card. Its probe() refused the second
outright, because there was a single static instance -- so a machine
with two of them drove one and silently ignored the other. Instances are
an array now, and the instance is hung on dev->priv by a new optional
`attached` hook that the bus calls once the device exists. probe() runs
before device_add(), so there was previously nowhere to put it; that is
the whole reason the hook exists rather than a line inside probe.
QEMU gets an explicit NIC on every target (QEMU_NET). The default device
happens to be an e1000 on this machine type today, which is exactly the
sort of thing that stays true until a QEMU release changes it and the
network stops being exercised without anyone noticing.
Verified:
- bootcheck -Net passes, 48 [ok], no failures -- including the moved
self-test and two new checks that /proc/netif names both the
interface and the driver behind it, which together say the stack
went through the class rather than through a driver it was compiled
to know.
- Two NICs attached by hand (-device e1000 twice, second with its own
MAC): eth0 and eth1 both attach, each reports its own MAC in
/proc/netif, and each gets its own DHCP lease. Before this commit
the same machine produced one interface.
- make schfs-check still passes in full.
One trap worth recording: moving a header broke the build for anyone
with stale .d files -- make stopped with "No rule to make target
'include/e1000.h'". CFLAGS gains -MP, which emits a phony target per
header so a moved or deleted one can never wedge the build again.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lwIP 2.2.0, fetched by ports/lwip and compiled straight into the kernel.
Not written from scratch, and the reason is not the packet formats: it is
congestion control, retransmission, window management and twenty years of
corner cases that only show up against a real peer. Not FreeBSD's
netinet either, though the licence would have allowed it -- that stack is
not a library, it reaches into mbufs, epoch(9), netisr, VNET and uma(9),
so adopting it means porting FreeBSD's kernel infrastructure first. lwIP
was written for exactly this situation: a small kernel with a raw
Ethernet driver and a port layer of a few hundred lines.
It is kernel code, so nothing is installed into the sysroot and there is
no second build system: the top-level Makefile compiles the files it
wants against our port headers. The file list is explicit rather than a
wildcard, because lwIP ships IPv6, 6LoWPAN, PPP, SLIP, DNS-SD and a dozen
apps in the same tree, and building what is not configured yields either
dead code or a link error naming a port function nobody meant to write.
Its sources are unmodified and its warnings are silenced for its files
only -- patching them would mean maintaining a fork.
The port layer is ours: a lock, a clock, the netif that hands frames to
e1000, lwipopts.h, and a shim/ directory of tiny headers so lwIP's
<string.h> and friends resolve to the kernel's functions instead of the
host's, whose prototypes disagree and would otherwise be found on the
default include path.
NO_SYS=1, so lwIP has no threads of its own and one kernel thread feeds
it. A thread rather than the interrupt handler, because a call into lwIP
can reach the driver, the driver can sleep, and an interrupt handler
cannot -- so the interrupt still only clears the cause register and the
thread polls. kthread_create is new and generally useful: a process with
a kernel-only page directory, no files, no cwd, no parent, dispatched
from forkret, which can therefore sleep and be preempted like anything
else. DHCP obtains an address at boot and the console says which one.
AF_INET went in exactly where phase 2 said it would. socknew dispatches
on the domain, kernel/net/inet.c implements the internet side on lwIP's
raw API, and not one syscall changed shape: bind, connect, listen,
accept, read and write reach the same descriptors they always did. TCP
and UDP both work. cmd/nettest opens a connection, writes, and reads the
answer back.
Two things are owed and named here so they are not discovered later:
recvfrom does not yet report the source address, and the socket paths
poll on the timer tick instead of sleeping on the socket. Both are
follow-ups, not redesigns.
NOTICE carries lwIP's BSD-3 text. Clause 3 is no-endorsement, not the
old advertising clause, so it sits with Apache-2.0 without difficulty.
Verified: the tree builds clean, and the headless boot check passes with
27 [ok] and no failures with the stack compiled in and the thread
running -- which is the part that would break first if netinit deadlocked
or the thread starved anything. An end-to-end connection test belongs in
that suite and is the next commit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The last three phases of the filesystem, which were written together and
touch the same files throughout -- kmount.c, objw.c and vfs.c are each
rewritten by all three -- so they land as one commit rather than three
that could not be built separately.
Durability. A transaction that reaches the platter out of order is worse
than one that never reached it: the uberblock can name a tree whose
blocks are still in the drive's write cache. bdirty/bflush give the
buffer layer an explicit barrier, ideflush issues ATA FLUSH CACHE and
usb_storage_flush the SCSI SYNCHRONIZE CACHE that means the same thing,
and a commit now waits for the data before it writes the root that
points at it. The uberblock groups are read newest-first with fallback,
so a volume whose newest roots are torn mounts at an older one and says
which generations were lost.
Reflink. Extents carry a refcount, so two objects can name the same
blocks and neither frees them while the other holds them. Exposed as
FICLONE -- Linux's ioctl value, so `cp --reflink` works unmodified --
and as cmd/reflink for the case where there is no cp to hand.
Attributes, and the one genuinely new idea here. attr.c stores named
values on an object, reachable through setxattr/getxattr/removexattr at
Linux's syscall numbers. On top of that, schfs_index declares an index
over an attribute name and schfs_query answers "which objects carry this
value" from the B-tree rather than by walking. A query against an
unindexed attribute is refused with an error, never silently answered by
a full scan: a query that quietly becomes a scan is worse than one that
admits it cannot help. cmd/schattr drives both.
Three bugs fixed on the way, each found by the numbers being wrong
rather than by inspection:
- statfs's f_fsid was declared int64[2]. Linux's fsid_t is two *ints*,
so every field after it landed one slot late: userland read f_frsize
out of f_namelen, got 255, and df reported a 32 MB root filesystem
as 1992 KB. It hid for as long as it did because the fields used to
be invented constants, so wrong numbers looked like the usual ones.
- struct inode's nlink was a short against SchFS's uint32, so a file
past 32767 names came back negative -- and iput frees an object
whose count reaches zero. It is an int now, signed on purpose, so an
underflow lands somewhere a check can see rather than wrapping.
- filewrite split every write to fit v1's log. That limit belongs to
the journal, not to writing, and SchFS makes one transaction of
whatever it is handed: a 1 MB write was becoming 170 commits and
170 flushes for nothing. fs_ops carries max_write now, and SchFS
sets no limit.
Also here: a real mount table, so `mount -t schfs /dev/hdb /mnt` shows
up in /proc/mounts -- df looks a path up there before it calls statfs at
all, so a mount missing from that list was invisible to it -- several
volumes mounted at once, snapshots mounted by name, and /proc/fsstat
with an rdtsc-based counter, which is the only instrument fine enough to
see inside a single syscall.
Verified: `make schfs-check` passes in full, including the sweep that
crashes the volume at each of 119 points mid-transaction and requires
every one to mount with exactly its committed data, and the case where
both newest uberblocks are rubbish and the volume still comes back. The
headless boot check passes with 27 [ok] and no failures, on a SchFS root.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every qemu target spelled `-cpu qemu64,+rdtscp` and nothing asked for
acceleration, so the default way to run this system was the slowest one
available. That is not a small constant: without KVM, QEMU emulates each
instruction in software, and on this tree `ls /bin` measured 120 ms
emulated against 20 ms with KVM on identical code. Every guess about the
kernel being slow has to be made against the KVM number or it is really
a measurement of TCG.
The flags are probed rather than assumed. /dev/kvm is absent on a host
without the extension, inside another VM, and when the user is not in
the kvm group -- and a hard -enable-kvm turns each of those into a QEMU
that refuses to start, which is worse than a slow one. `-cpu host` only
goes with KVM: it means the physical CPU, which an emulator cannot
present, so the fallback keeps the old spelling exactly.
qemu-kvm stays, and is now the only target that *insists*. That is what
you want when the question is why the host has no /dev/kvm, rather than
quietly getting a slow machine.
Verified both branches on this host: `make -n qemu` expands to
"-enable-kvm -cpu host" where /dev/kvm exists, and overriding QEMU_ACCEL
reproduces the previous "-cpu qemu64,+rdtscp" verbatim.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ui_panel was written for this screen during the loader restyle and then
never called -- dead code sitting in boot/common/ui.c waiting for the row
that would use it. This is that row: F8 -> System Information, showing
firmware vendor and revision, UEFI revision, CPU brand, usable memory,
current mode and whether ACPI was found.
Everything is read when the screen is drawn rather than captured at
startup, because the resolution row above it can have changed the mode,
and a panel still reporting what the machine booted with is worse than no
panel at all.
Two facts are deliberately read through the same code that feeds the
kernel -- efi_type_to_e820() for what counts as usable memory, and
find_rsdp() for ACPI. A panel with its own idea of either would be a
second implementation to keep in step, and the ACPI line exists precisely
because a missing RSDP means no MADT, no IOAPIC and no SMP, which has
already been debugged once from the far end.
The screenshot caught a bug in the screen it was taken to verify: UEFI
revision read "21587.2286". EFI_TABLE_HEADER begins with a UINT64
Signature, so Revision is at offset 8 and I had read offset 4 -- the
upper half of "IBI SYST", formatted as a version number. It now reads 2.7
under OVMF, which is correct.
Verified by driving the menu over the QEMU monitor and reading the
pixels: five rows with the hint updated to 1-5, the panel centred, and
every value cross-checkable -- EDK II, firmware 1.0, UEFI 2.7, "QEMU
Virtual CPU version 2.5+" against -cpu qemu64, 1018 MB of the 1 GB given,
1280x800 matching what fbshot captures, ACPI present. The highlight sits
9 px inside both panel borders, symmetric.
F8 has a 1.5 s window, so the capture presses it repeatedly through boot
rather than guessing when OVMF finishes -- and only F8, since any other
key breaks the wait and boots.
Also: every ESP drive now says format=raw. QEMU warned about probing the
format on each of the fifteen or so places that attach it, which is new
noise on what is now the only boot path. 110 [ok], no failures, no
warnings.
1449 lines of boot/bios/, tools/sign.pl and tools/padstage2.pl, plus the
build rules, the disk image and the ISO entry that existed to carry them.
Everything since roughly 2012 boots UEFI, and CSM -- the firmware-side
BIOS emulation that lets a legacy loader run on such a machine at all --
has been going away since Intel dropped support for it in 2020. So this
is not a path being retired early; it is one whose remaining hardware is
already the exception.
The cost was never the code, it was the tax on everything near it. Stage
2 had a 65-sector ceiling it overflowed silently -- 25440 bytes against
24576 -- and because the kernel is written at sector 1 + STAGE2_SECTORS,
an overflowing loader was overwritten by the very kernel it was loading.
The resolution list could not be shared with UEFI either: VBE mode
setting is INT 10h, so matching it needed unreal mode, which is the
riskiest code in the tree for a feature that already worked elsewhere.
What replaces schos.img is the staged ESP, handed to QEMU as a virtual
FAT volume. That is a development arrangement and not an installable
disk; a real GPT image with a real FAT partition is the installer's job,
and until it exists nothing here writes a disk a machine boots unaided.
Secure Boot is untouched too -- the loader is unsigned.
The ISO keeps one El Torito entry, the FAT image holding loader and
kernel. It never booted from a USB stick via BIOS anyway: the firmware
presents the stick as a hard disk while stage2main.c read the kernel back
over ATAPI.
The 32-bit multiboot entry in entry.S now has no caller and is left in
place. Deleting kernel entry code is its own change with its own way of
failing silently, and it should not ride along with this one.
Ported with it: run.ps1, the six screenshot tools, and schcom_test.sh,
each with its own OVMF varstore copy so no run inherits another's UEFI
variables.
Two things this shook out. OVMF now belongs in `all`: nothing boots
without firmware, `make clean` wipes build/ including the copy fetched
from the host, and a clean followed by a build left every run script
throwing. And it cannot be listed as a prerequisite at the top of the
file -- make expands prerequisites when it reads the line, and
$(OVMF_CODE) is defined further down, so naming it there appends nothing
and says nothing.
Verified from a clean tree: 110 [ok] and no failures on the full suite,
the ISO boots with no hard disk attached, schfs-check passes, schcom's
end-to-end pixel test passes, and fbshot captures a real 1280x800 frame.
The BIOS loader drew its own splash and the UEFI loader drew a different
one, which is how two implementations of a single screen end up looking
nothing alike: only one of them ever got a change. stage2main.c's
fourteen drawing functions are deleted and it calls boot/common/ui.c,
which is what the UEFI loader already used. There is one splash now.
What changed in the look, in order of how much it mattered:
- No product name under the mark. The name is *in* the mark, and
setting it again underneath read as a placeholder nobody removed.
- The mark is scaled to the bar's width and sits directly above it, so
the two are one object rather than a large logo with an unrelated
line beneath. Box-filtered down from the source mask, never up:
point-sampling a coverage mask drops half the edge pixels and puts
holes in the thin strokes.
- Metal rather than flat grey. A vertical gradient with a knee near the
top where a highlight falls, applied to the mark, the bar fill and
panel headings. A single colour on a dark ground looks printed; the
knee is what makes it look lit. No texture -- at this size noise
reads as dirt.
- The boot manager is a panel with a lit selected row, replacing a
screen laid out in literals with a magic y of 340 that only ever
looked right at 800x600.
Three defects found by reading pixels rather than code:
- the UEFI loader kept calling print() after the splash, and ConOut on
a graphical firmware renders into the same framebuffer -- a
screenshot showed firmware text across our mark from the left edge.
Diagnostics go to COM1 directly now, which is where they were being
read from anyway, and the console is cleared before drawing so the
firmware has nothing to put back.
- SPLASH_UP was declared and read but never assigned, so that redirect
was dead code and the "serial works" reading was OVMF's own console.
- stage2 overflowed its allocation -- 25440 bytes against 24576 -- and
the build did not stop. The kernel is written at sector
1 + STAGE2_SECTORS, so a stage2 past its allocation is overwritten by
the kernel it is loading. STAGE2_SECTORS is 64 now, and the padding
tool's check does fire; it had simply not been re-run.
Verified from the host on both: BIOS 800x600 and UEFI 1280x800 both draw
a centred group with zero stray pixels, UEFI's spanning x 576-703 which
is exactly the mark width centred. bootcheck 108 [ok], iso builds.
Still text-mode: the resolution menu. On BIOS it necessarily is -- it
runs in real mode before any framebuffer exists, which is the whole
reason it can set one. On UEFI it could be graphical and is not yet.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The first piece of the two loaders that is genuinely common code rather
than a contract they both honour. It can be, because the difference
between the firmwares is entirely in how a framebuffer is *obtained* --
VBE in real mode, or GOP -- and not in what you do with one afterwards.
boot/common/ui.c takes a base, a size and a stride and draws; it knows
nothing about disks, memory maps or mode setting, which is what lets it
compile into a 16/32/64-bit BIOS loader and a PE/COFF UEFI application
without either leaking into it.
The look is deliberately quiet: dark ground, the mark centred, one thin
bar beneath it, nothing on screen that is not load-bearing. The whole
group is laid out from the centre outwards rather than from the top down,
so it sits identically at 800x600 and at 1600x1200 instead of drifting up
the screen as the mode grows.
UEFI had no graphical splash at all -- it printed text to a console the
QEMU window is usually not even showing. It draws the same screen as the
BIOS path now, with the bar tracking bytes of kernel actually written
rather than a segment count, which says nothing about how long each takes.
Two ordering bugs found by looking at the pixels rather than the code:
setup_framebuffer ran *after* the kernel was read, so every progress tick
went to a framebuffer that did not exist yet and the splash then painted
over the result; and that also put the resolution prompt after the
slowest step instead of at the start. The display is set up first now.
Verified from the host with screendump: background is exactly SCH_BG, the
bar is 4px tall and width/4 wide at y=495 on a 1280x800 screen, which is
where centring the group puts it. UEFI still boots to cpu0; BIOS
bootcheck 27/27.
Not yet shared: the BIOS loader still draws with its own copy of this
code. Deleting that copy in favour of this one is the next step and the
point of the exercise -- until then this is one loader using shared code
and one duplicating it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The BIOS loader handed the kernel a 32-bit entry that climbed into long
mode itself; the UEFI loader arrived already there and used a separate
64-bit entry. Two entries meant two copies of the same preconditions, and
that is not a hypothetical cost -- it is exactly where EFER.SCE went
missing for months, set on one path and not the other, invisible until a
program tried to return from a syscall.
stage2 does the climb now. It builds the page tables where the UEFI
loader builds them and meaning the same thing -- 4 GB identity-mapped
with 1 GB pages, and the same 4 GB again at 0xffff800000000000, because
entry64low's first instruction is a movabs to a high address and identity
alone triple faults on it -- sets PAE, LME and SCE, enables paging, and
far jumps through a GDT with an L-bit code descriptor, which a 32-bit
descriptor cannot express.
stage2main.c is compiled for x86_64 and runs in long mode. It still reads
the multiboot header to find where the kernel's segments go, but enters
through the SCH64 header rather than the entry named there: that entry is
32-bit code whose only job is the climb, and dropping back to 32 bits to
redo work already done is a mode switch for no reason. If the SCH64
header is absent it halts, because there is no longer anything to fall
back to.
So both loaders now hand over identically -- long mode, interrupts off,
CR3 mapping the low 4 GB twice, framebuffer in RBX -- and there is one
set of preconditions instead of two that can drift.
Verified on every path that exists: BIOS to schinit; BIOS with -smp 2,
which brings up cpu1 through the AP trampoline's own separate climb; the
resolution menu still selecting 1024x768 over serial; the ISO, which
loads through ATAPI and a ramdisk root; UEFI unchanged at 1018 MB in 7
regions; bootcheck 108 [ok]. stage2 is 21988 of 24576 bytes.
The 32-bit mboot_entry stays in entry.S. Nothing of ours uses it now, but
it is what makes the kernel loadable by a multiboot loader that is not
ours, and removing it is a separate decision from this one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Groundwork for usb-tablet, which is the fix for the guest cursor and the
host's never being in the same place: a PS/2 mouse reports how far it
moved, two pointers accumulating deltas independently drift apart, and no
amount of care in the compositor can make them agree. An absolute device
reports where it is, so they coincide by construction.
- the HID driver no longer matches the boot subclass only. usb-tablet
declares subclass 0 and was refused at the door; hid_probe decides
what an interface actually is, and still declines what it cannot make
sense of.
- a non-boot HID interface is taken to be an absolute pointer and is
not asked for boot protocol, which it does not have -- that request
failing is what stopped it binding.
- tablet_report decodes the fixed 6-byte layout (buttons, 16-bit X,
16-bit Y, wheel) and scales the logical range to pixels there,
because it is the only place that knows the range.
- input_mouse_abs reports EV_ABS/ABS_X/ABS_Y so a reader can tell an
absolute position from a delta, and mouse_set_abs places the pointer
rather than nudging it.
- Schcom treats EV_ABS as authoritative and EV_REL as a nudge.
The device binds and identifies correctly: "usb-hid: tablet (absolute) on
slot 1, endpoint 0x81 every 4 ms".
It is NOT finished, and QEMU_INPUT is empty by default because of it. No
reports have yet been observed arriving on the tablet's interrupt
endpoint, so making it the default pointer would trade a cursor that
drifts for one that does not move at all. Opt in with
make qemu QEMU_INPUT='-device qemu-xhci,id=xhci -device usb-tablet,bus=xhci.0'
Next step is to find out why nothing arrives: whether the first transfer
is queued but never completes, whether a completion needs re-arming for
this endpoint, or whether QEMU is not routing motion to the tablet at all
(`info mice`).
bootcheck 108 [ok], schcom-test passes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Clients connect over a Unix socket, are handed a shared buffer, draw into
it and commit; Schcom composites and presents. Both of the hard blockers
its README listed -- unix domain sockets and raw input events -- have been
done for a while, so nothing was in the way.
The model is Wayland's, on purpose. A surface is not a window: it is a
rectangle of pixels with no policy attached, because placement belongs to
SchWM and the moment this program starts deciding where windows go the
split has failed. Nothing is shown until COMMIT, so a client halfway
through drawing is never presented. The compositor tells clients when to
draw rather than letting them render frames nobody sees.
Wayland is MIT-licensed and supporting it later is intended, so the
divergences are the ones that are cheap to undo: a buffer handle is a
System V shm id rather than an fd over SCM_RIGHTS, and the message
carries a handle_kind field so switching is a new value rather than a
protocol break. Every message carries its own length, so an unknown
message is skipped instead of desynchronising the stream -- which is what
lets a v0 compositor survive a v1 client.
The compositor owns the shm segment, not the client: it has to survive a
client crashing mid-frame, and it must not be something a client can
resize underneath it. A dropped connection takes its surfaces and their
segments with it.
Composites into a back buffer and copies once, because compositing
straight into the framebuffer shows every intermediate state -- the
flicker would be the whole desktop being repainted under the user's eyes.
projects/ gets build glue for the first time: `make projects` builds
against the sysroot's libc and stages into sysroot/bin. It is not a
dependency of anything above it, and `make` never descends there.
Verified by `make schcom-test`, which boots SchOS, runs the compositor and
a client, screendumps the framebuffer and asserts the pixels on the host.
That last part is the point -- the compositor reporting that it
composited something proves nothing, and a screenshot nobody inspects
proves less. It checks the client's colour mid-surface, the compositor's
background, and the white border exactly at the requested origin, so
"arrived" and "arrived at the wrong offset" cannot pass for each other.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`make ramroot` wiped the objects on the way in, and nothing wiped on the
way out. So a plain `make` afterwards kept every RAMDISK_ROOT object and
linked the 24 MB ramdisk back into a kernel that no longer used it: 25 MB
where a clean build is 836 KB. It still booted -- from the ramdisk -- and
the only outward symptom was stage2 having thirty times as much to load
through PIO, one sector at a time. That is the "booting got slower"
report this came from.
A stamp file records which setting the objects were built with, and the
tree is dropped when it changes in either direction.
qemu-kvm is opt-in and the default stays TCG, because the suite's timings
were established there. Measured with the guest timing itself,
`ls /bin | wc -l` goes 2.9s -> 1.1s. Boot does not improve and gets
slightly worse: every PIO port access in stage2 is a VM exit under KVM.
Retracts a claim from earlier today. The "2.2x speedup" attributed to the
region-based allocator was a measurement artifact -- the probe sent a
command, slept a fixed 50s on the host, then read /proc/uptime, so it
timed the sleep rather than the command, through a guest clock that drifts
under load. Bracketing the command inside one guest command line shows no
measurable difference. The e820 change is still correct; it is simply not
a performance change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Nothing outside the kernel could touch a disk. sys_mount does not even
look at the device it is handed -- it mounts the one SchFS volume found
at boot and ignores `source` -- so a program had no way to read a
partition table, format a volume or write a boot sector. An installer is
a program that needs all three, so this is the first piece of it.
A character device in devsw rather than a separate block layer, because
that is where the offset already lives: fileread/filewrite pass f->off
and advance it by what comes back, which is everything a seekable device
needs. The minor is the IDE device number, the same numbering iderw()
uses; two schemes for the same disks would be a bug waiting to happen.
Writes bypass every filesystem above them, which is inherent to a raw
device and why an installer must target a disk it has not mounted.
The nodes are created unconditionally, like every other entry in /dev.
That matters more than it looks: with no node, `echo x > /dev/hdb`
silently creates an ordinary file of that name and appears to work. It
did, and it agreed with itself on read-back; only the host's copy of the
disk showed nothing had been written.
st_rdev was never filled, so every node in /dev reported "0, 0" under
ls -l. Harmless while nothing read it, wrong as soon as something has to
tell two disks apart. Filled with the full Linux dev_t encoding rather
than (major << 8 | minor), which only works while both stay small.
Verified against the host as an independent oracle, both directions: the
guest wrote to /dev/hdb and the host saw those bytes at offset 0; the
host wrote at offset 4096 and the guest read them back with dd skip=.
ls -l now shows 1,1 for console, 2,0 for fb0 and 8,1 for hdb.
Known limit, commented in disk.c: devsw passes the offset as a 32-bit
uint, so a raw disk is addressable to 4 GB. That must be widened before
anyone installs onto a real disk.
bootcheck 108 [ok] with -Fb -Tty -Usb -Sock -Evdev -Ps.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The disc carried one file -- boot/schos.bin -- with 8.3 uppercase names,
one El Torito entry, and no way to boot a UEFI machine. Now:
/boot/schos.bin BIOS entry, unchanged: no-emulation, 1+STAGE2_SECTORS
/boot/efiboot.img UEFI entry: a FAT image built with mtools
/EFI/BOOT/BOOTX64.EFI the loader, also visible as a file
/bin /etc /usr ... the staged sysroot as an ordinary tree
Rock Ridge and Joliet restore real names, permissions and symlinks;
-cache-inodes matters more than it looks, because the sysroot is a toybox
multicall binary with a couple of hundred hard links to it and without
inode caching each link becomes its own copy.
-isohybrid-gpt-basdat adds a GPT so the .iso can be dd'd to a stick and
booted by UEFI. That is UEFI only and deliberately so: the BIOS isohybrid
trick needs syslinux's isohdpfx.bin, and even with it this loader reads
the kernel back over ATAPI, which a stick presented as a hard disk cannot
serve. usbstick.img is what that is for.
The FAT image carries the kernel as well as the loader, because
BOOTX64.EFI opens \schos\kernel on the volume it was loaded from, and for
a UEFI boot that is the FAT image rather than the ISO 9660 tree. Without
it the loader halted with EFI_NOT_FOUND.
Also fixes `make iso` and `make ramroot`, which were already broken:
fs-ram.img was still built by the v1 mkfs while the kernel accepts only
SchFS as root, so the ramdisk was a format it could not mount and it
panicked with "schfs root: no SchFS volume on the root device". It is a
mkschfs image now, which needed --exclude in mkschfs -- dropping the
exclude list was not an option, because the ramdisk is linked into the
kernel binary and what goes in it is a licensing question. RAMDISK_NINODES
is gone with the v1 image: SchFS has no preallocated inode table.
Verified: BIOS ISO boots to a shell and runs commands over serial
(241 files in /bin, /etc/motd reads back). UEFI ISO boots to a prompt.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
tools/posix/setup-fedora.sh installs the toolchain and reports what it
cannot: PowerShell has no Fedora package, and bootcheck is the test
suite, so the script says how to get pwsh rather than failing quietly.
It also checks /dev/kvm, since a Ryzen box can run the tests at native
speed.
docs/moving-to-linux.md is the actual procedure: copy the repo, restore
the memory, install, build. The memory travels inside the repo and
`memory-sync.sh import` puts it under the key this machine derives from
its own path -- /home/nexoniarz/Documents/SchOS becomes
-home-nexoniarz-Documents-SchOS, where D:\DEV\SchOS was D--DEV-SchOS.
OVMF discovery no longer assumes QEMU's Windows layout. Fedora ships
edk2-ovmf to /usr/share/edk2/ovmf/, Debian and Arch differ again, so the
Makefile searches a list and names the packages when it finds nothing.
Tested through the shell rather than $(wildcard): on Windows make is a
native binary using Win32 file calls, while `command -v` returns an MSYS
path like /c/Users/..., which $(wildcard) cannot see -- it returned
nothing on a machine that had the firmware.
Two things only that machine can test: `make ports` builds musl, toybox
and mksh from source, which Windows cannot do at all; and the SchFS
round-trip needs symlinks and hard links, so it has never actually run.
Nexoniarz