Every path component was copied into a DIRSIZ buffer and every path
into a MAXPATH one, so names were cut at 29 bytes and paths at 127
without an error. Two names sharing their first 29 bytes opened the
same file, on SchFS and FAT alike, though both hold 255.
VFS_NAME_MAX (255) and VFS_PATH_MAX (4096) are the limits now, and
anything longer is ENAMETOOLONG. namex_from() takes its path buffers
from kalloc(), a page each, rather than putting two of them on an 8 KB
kernel stack. The v1 format still stores DIRSIZ bytes, and refuses a
longer name instead of matching on its first 30.
Checked: 255-byte names on SchFS and FAT (fsck.fat clean, and the FAT
name answers to its uppercase form), 256 refused, a 200-byte path and a
link to it, and no pages lost over 700 lookups through links.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
kd_write() went through bread(), so every block a write allocated was
read from the disk first and then replaced whole. That was a quarter of
the disk operations of a SchFS write. bgetblk() is bread() without the
read, as Linux's getblk() is.
1 MB on AHCI under QEMU: 2.0 -> 2.8 MB/s in 4 KB writes, 4.5 -> 5.6
MB/s in 64 KB ones. 3 MB of random data written and copied through the
cache's evictions reads back with the same md5.
The 63 KB/s seen earlier is the legacy IDE driver on QEMU's default
machine, at about 10 ms per emulated PIO write; the same disk on AHCI
(q35) is 40 times faster.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
Symlinks are followed in namex_from() now, in the middle of a path as
well as at the end, and a relative target resolves from the directory
that holds the link rather than from the caller's cwd. `ln -s f d1/s;
cat d1/s` said "Symbolic link loop" because it looked for ./f. A loop is
ELOOP after 40 links, a dangling link ENOENT, and open(O_CREAT) through
a link to a file opens the file. divesymlink() is gone.
mknod from a Linux program passes (mode, dev) where the native call
passes (major, minor), and was read as the native one: `mknod n c 1 2`
made node 432,258, and mkfifo made a character device. Both are decoded
now; a FIFO is EPERM, and an existing name is EEXIST.
/proc/mounts listed devtmpfs twice: once from a fixed line and once
from the mount table. The fixed line is gone, and mount_add() refuses
a filesystem mounted on its own root, which is where a second devtmpfs
mount lands. A read-only FAT mount says ro.
/proc/partitions exists, in Linux's format, and /proc/filesystems lists
vfat.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
The filesystem every other system can read: a USB stick, an SD card, the
EFI system partition. It is here first for the hardware report, which has
to leave a machine on something that can be read on another one.
mount -t vfat /dev/usb0p1 /mnt (msdos and fat work too; -o ro)
Written from Microsoft's specification, and matched to Linux vfat where
the specification leaves a choice: 8.3 names with the NT lowercase bits,
long-name entries otherwise, Linux's alias rule (REPORTS, then BIGNUM~1),
case-insensitive lookup, UTC times, the boot-sector dirty bit, every FAT
copy and FSInfo kept up to date. It uses its own sector cache over the
new blkdev_xfer, since a partition need not be a whole number of 4 KB
blocks, and every operation that changes something reaches the disk
before it returns. Inode numbers come from the entry position, or the
first cluster for a directory, plus a small table for entries that moved.
Around it:
- /dev lists every disk and partition the block layer found (major 8,
minor = block device), and raw reads and writes on them go by sector.
- The ESP and MBR FAT partitions are registered, typed so the root probe
passes over them. MBR entries with a bad status byte are skipped, so a
superfloppy's boot code is not read as a partition table.
- statfs, sync, fsync, umount (EBUSY while in use), shutdown, reboot and
the power button all know about FAT.
- create undoes and reports a failed ialloc or dirlink (ENOSPC, EINVAL,
EPERM) instead of panicking; fs_ops gains refuse_types and no_hardlinks.
- filewrite returns a short write instead of panicking on it.
- utimensat is implemented. As a stub returning 0 it made toybox touch
believe the file existed, so `touch newfile` created nothing on any
filesystem.
Tested in QEMU against mkfs.vfat images (FAT12 floppy, FAT16 on MBR,
FAT32 superfloppy and GPT ESP) by writing, renaming, deleting, filling to
ENOSPC and truncating there, then checking with fsck.fat -n and mtools on
the host: all clean, contents match.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
consoleinit() put consoleread/consolewrite in devsw[1], over memread and
memwrite. Major 1 holds the whole mem major (null, zero, full, random,
urandom) and memread already sends the console's minors to the console,
so the override only broke the others: a read of /dev/zero waited for a
typed line. dd and head on either device hung, and `head -c 10 /dev/zero`
only finished because it swallowed the next command.
The console now installs just its open and close hooks. /dev/keyboard
writes are discarded, as consolewrite used to do.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
"acpi: 0 PCI interrupts routed through _PRT" on an AMD board meant that
every PCIe bridge's devices are wired to the second I/O APIC, which is not
driven. It says so now, and that those devices need MSI.
"vga: framebuffer disagreement" on an NVIDIA card compared the GOP's
framebuffer against BAR0, a 16 MB register block big enough to pass for a
screen. The BAR that contains the framebuffer is preferred now.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
From the first boot on real hardware: a Razer mouse's control transfer
failed with a babble, and an ITE controller's late endpoint-0 completion
was reported as an "unexpected event" during a command.
A failed control transfer left endpoint 0 halted (or, on a timeout, still
running with TRBs queued), so every request after it to that device failed
too. It is now reset or stopped and its dequeue pointer moved past what was
left on the ring. A STALL -- a device declining a request it does not
implement -- is logged only on a verbose boot.
HID interfaces without the boot subclass were turned away unless they had a
pointer. The report descriptor's keyboard page is now read too -- modifier
bits, a key array, an NKRO bitmap -- and each report rewritten into the boot
layout for the existing path. Checked with QEMU's usb-kbd forced into report
protocol: "echo Hi", Shift included, typed and ran.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
At 800x600 with the 21-pixel console cell there was not room for the logo,
the facts and the call chain above the three closing lines. The chain ran
into the footer, the footer was drawn over it, and what remained was the
tail of a trace line after "asks more." -- the "0x5d" in the photo -- with
the tops of "Call chain" underlining the line above.
The layout is measured first now; the top margin gives way, then the logo,
and the chain stops above the footer. Text in the rows the QR card shares
stops short of it rather than running underneath. The screen is flushed
unconditionally: the CPU that panicked may have stopped another halfway
through a flush.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
On the first real machine with a discrete card (an RTX 3060, 800x600 from
the UEFI GOP) the console was slow enough to photograph mid-scroll, and the
boot died on CPU 1's "cpu1: starting": CPU 0 spun 200M times on the console
lock while CPU 1 held it, scrolling, and called it a deadlock.
The framebuffer is device memory in the uncached 3-4 GB page of the direct
map. QEMU's stdvga is RAM underneath, so nothing showed there; on a card,
gfx_scroll_up's memmove read the whole screen back across the bus one byte
at a time, and every anti-aliased glyph read back each pixel it blended.
So the console now draws into a copy of the screen in RAM (vmap_alloc: kalloc
pages behind a page table of their own, since nothing needs it contiguous),
and gfx_flush() copies the changed rectangle onto the card through an
ioremap_wc() mapping -- PAT entry 4 set to write-combining on every CPU.
Flushes happen where drawing ends: a message, a terminal write, an echo, a
terminal switch. Scrolling moves eight bytes at a time instead of one.
/dev/fb0 reads and writes the card itself, as before. The early console's
fb_info now carries the physical address it always should have.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
The firmware's AML is now interpreted. uACPI 6.1.1 (MIT) is fetched by
ports/uacpi, checksum-verified, and compiled into the kernel unmodified,
the same way lwIP is. kernel/acpi/host.c is its host layer: memory,
mapping, port and PCI config I/O, time, mutexes and events that spin
before the scheduler exists, the SCI, and an "acpi" thread for the
work the SCI defers.
PCI interrupts go where the board says. pci_init() now routes every
device's INTx through the root bridge's _PRT before drivers bind. It
follows link devices' _CRS, uses a bridge's own _PRT where the namespace
has one, and swizzles through bridges that don't. Before this, INTx
waited on the 8259's line number, which is the wrong pin in APIC mode.
irq_register() takes a trigger mode rather than a level/low flag,
because q35's links and most SCIs are level, active-high. The MADT's
interrupt source overrides are now parsed, and are what places the SCI.
The power button syncs the filesystems and powers off. shutdown() and
reboot() go through the firmware's S5 and reset register first, and
only then fall back to the emulators' ports, which a real board does
not answer.
Tested under QEMU: q35 with MSI off routes xHCI to GSI 22,
level/active-high, and it takes real interrupts with nothing polled.
system_powerdown powers off q35 and i440fx (QEMU exits 0), and q35
with 4 CPUs boots from NVMe with USB hot-plug working alongside.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
A native driver written from the NVMe 1.4 base specification. It resets
the controller the firmware left running, then sets up an admin queue and
one I/O queue pair. Completion is polled, with interrupts disabled at the
queue, for the same reason AHCI polls: without AML, interrupt routing on
real hardware cannot be trusted. Each active namespace with 512-byte to
4 KB sectors and no metadata is registered as nvme<C>n<N>, the name
Linux uses, and its GPT is read like any other disk's.
Tested under QEMU q35: disk.img on NVMe as the only disk boots through
OVMF, roots on nvme0n1p2, and writes read back after sync. A
4 KB-sector namespace registers.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
Devices can now arrive and leave after boot. A "usbhotplug" kernel thread
handles root-port changes, which it is told about by Port Status Change
events and also checks for every quarter second. It also handles changes
behind hubs, which each hub reports on its status-change endpoint. An
unplugged device has its endpoints closed and any transfer in flight
failed. Then its drivers' disconnect hooks run, the slot is disabled and
its memory is freed. Everything behind a hub that goes goes with it. This
needed the interrupt handler to file command and endpoint-0 completions
for their waiters, so that enumeration can run while keyboards interrupt.
On real hardware: the controller is taken from the firmware through USB
Legacy Support before reset, with SMIs off. On Intel 7/8/9 series parts
the ports are switched over from EHCI, which is where firmware leaves
them.
Every interface of a device is offered to the class drivers, not just
the first, so a receiver's mouse on interface 1 gets a driver. Configure
Endpoint now copies the slot context from the controller. Before, it
rebuilt it and lost the route string and TT fields of devices behind
hubs.
HID: pointers are read in report protocol from their report descriptor.
That covers buttons, X/Y and wheel, relative or absolute, and report IDs.
Boot protocol is kept as the fallback for mice and for keyboards.
Non-pointer HID interfaces are declined rather than treated as tablets.
USB keyboards get software key repeat. Mass storage handles up to four
sticks, each unregistered from the block layer when it is pulled out,
along with its cached blocks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HH7xukt4aaiGj1nQEAoQf1
/proc/interrupts on q35 said "LOSING 526" ticks: 5 s of boot with
interrupts off. Timing each step of kmain() found two waits:
- AHCI sent a COMRESET to every empty port and waited a second for a
link. DET = 0 is "no device", so only DET = 1 is worth a reset now
(after 20 ms of grace for a drive that is still spinning up).
- ideinit() read 0xFF from a bus with no IDE controller, took the BSY
bit for a busy drive, and waited out its 1 s timeout. A status of
0xFF means no controller and is answered at once.
That is every PC with SATA disks. q35 now loses 33 ticks, under the
flag; i440fx still roots on IDE and loses none.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HH7xukt4aaiGj1nQEAoQf1
q35 stopped with "no root filesystem" because its disk is SATA, like
every PC since 2008, and SchOS only had IDE. And SchFS could only sit
on a whole raw device, so nothing could boot from a disk that also
carried an ESP.
- include/blkdev.h, kernel/dev/blkdev.c: a registry of block devices
found at probe time, numbered from 4. A disk's GPT (or MBR) is read
when it registers, and each partition becomes a device whose block 0
is the partition's first sector. SchFS needs no format change to live
in a partition; a Linux SchFS driver will see the same bytes on
/dev/sdX2. SchFS has its own GPT type:
47138FA9-B79C-47A6-9F7D-AC1F7F572D5E.
- AHCI driver, written from the AHCI 1.3.1 spec: BIOS handoff,
COMRESET when the link is down, IDENTIFY, 48-bit DMA read/write,
FLUSH. Completion is polled, so the root disk does not depend on
interrupt routing.
- USB sticks also register as usb0, so their partitions are found.
- The root probe tries registered devices between USB and IDE,
SchFS-typed partitions first.
- make disk / tools/posix/mkdisk.sh: a GPT image with a 64 MB ESP and
a SchFS partition, ready to dd to a stick or a disk.
Tested in QEMU q35: the old two-disk setup roots on sata1; disk.img as
the only SATA disk roots on sata0p2; disk.img as a USB stick (firmware
boots from it) roots on usb0p2. Writes survive sync in each case.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HH7xukt4aaiGj1nQEAoQf1
PCI drivers took their interrupt from INTERRUPT_LINE. On real hardware
that is the 8259's number, not the IOAPIC pin the device is wired to
(that is in ACPI _PRT, which needs AML), so nothing ever arrived. xHCI
is entirely interrupt-driven, so the USB keyboard and a USB root would
have hung even on a clean boot. Under QEMU the two numbers agree,
which is why it never showed.
- pci_irq_attach()/pci_irq_detach(): MSI, then MSI-X if the driver
allows it, then INTx. The handler is registered before MSI is
enabled, so no edge is lost. xHCI, e1000, r8169 and rtl8139 use it.
- Lines 24-29 are MSI vectors. The LAPIC error vector moves from 19,
which is a PCI pin on Intel chipsets, to 30. Nothing is placed at or
above vector 64 (T_SYSCALL).
- A storming line is masked and its handlers are polled instead of
being left to eat the CPU. A level line that has had a handler for 2 s
and never fired is polled every tick by a kernel thread.
Tested in QEMU q35 with the root on a USB stick: xHCI and e1000e on
MSI; then MSI disabled and INTx deliberately misrouted, where xHCI got
zero interrupts and the root still mounted through the poller. i440fx
INTx and the rtl8139 hand-over are unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HH7xukt4aaiGj1nQEAoQf1
Every one of these works under QEMU and stops a real machine:
- MADT processors that are "online capable" but not enabled were
counted as CPUs. AMD firmware lists absent cores that way (so does
QEMU with maxcpus), and startothers() then waited for ever for an
AP that does not exist. Only enabled entries count now, x2APIC
(type 9) entries are read too, and the wait is bounded to a second;
an AP that misses it is parked with INIT and the boot goes on.
- The PIT calibration loop had no bound. Recent boards clock-gate the
8254: the output never rises (a hang) or floats high (TSC "0 MHz"
and every clock thousands of times fast). The loop is bounded and
checked, and falls back to the ACPI PM timer, then CPUID 0x15/0x16.
Under QEMU pit=off the PM timer gives the same figures the PIT did.
- Firmware that hands over in x2APIC mode left lapic[] reading
garbage: cpunum() panicked or the timer never started. Each CPU is
switched back to xAPIC first (x2APIC -> disabled -> xAPIC).
- The IOAPIC was always at 0xFEC00000 and interrupts were aimed at
APIC ID "0". Both now come from the firmware's tables.
- INIT-to-STARTUP was 100 us ("Bochs is slow"); it is the 10 ms the
MP specification asks for.
- mouseinit() drained the 8042 with an unbounded loop. With no
controller the port reads 0xFF and the loop never ends. It now
notices there is no controller and skips the mouse.
Tested in QEMU: i440fx and q35, -smp 4, -smp 2,maxcpus=4, pit=off.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HH7xukt4aaiGj1nQEAoQf1
The kernel's rtl8139 driver had no release hook, so /dev/pci refused
every request for the card ("the kernel would not release it") and a
hosted 8139too could never run on a machine where this driver probed
first.
release() follows e1000_release(): silence the chip before unhooking the
interrupt, reset it and clear bus mastering so it stops writing into its
buffers, take the interface out of the stack, then give the DMA buffers
back.
Tested in QEMU: ldlsd takes the card from the kernel, 8139too probes it,
DHCP binds ldls0, and ping 10.9.0.2 gets 3 of 3 replies.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HH7xukt4aaiGj1nQEAoQf1
Nexoniarz is the sole author and SchOS was never published under
Apache-2.0, so this is a straight relicence. LICENSE is the FSF GPLv2 text,
every first-party SPDX header moves to GPL-2.0-or-later, and NOTICE,
README.md, docs/licensing.md and the site say so. projects/schfs-linux
follows the rest of the tree.
The GPL gives copyleft: a fork has to ship its source. It also lifts the
old "Linux drivers, never" rule, since GPL-2.0-only code combines with
GPL-2.0-or-later. "Or later" leaves room for GPLv3, but one binary still
has to pick one version, so docs/licensing.md now keeps Apache-2.0 and
GPLv3 code out of the kernel to keep Linux code possible there. BSD
4-clause code goes from "avoid" to "no", because the advertising clause
is GPL-incompatible.
LDLS still ships no .ko. That boundary is now project policy rather than
a legal requirement, and ldls.md, devices.md and licensing.md section 6
say so. refmod declares license=GPL, and ldls-check passes (27 checks).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Three things, and the first one is the freeze.
**The loader jumped out from under itself.** A machine with a lot of RAM
stopped at "Starting the kernel" with the splash still up: no panic, no reset,
nothing. The firmware's own exception handler had the answer the whole time,
once a machine was found that printed it:
X64 Exception Type - 0E(#PF - Page-Fault)
ExceptionData - 0x10 I:1 P:0
RIP - 0000000140004DEE <- the loader's own code, at 5.4 GB
RAX - 0000000000001000 <- the page tables it had just installed
RCX - 00000000001000F0 <- the kernel it was about to jump to
I:1 P:0 is an instruction fetch from a page that is not present. The loader
built page tables covering four gigabytes, loaded cr3 -- and the firmware had
placed the loader image *above* four gigabytes, so the next instruction fetch
was unmapped. It reproduces exactly on `-m 8G` and never on `-m 1G`, which is
why every test here passed.
The PDPT is one page whether four entries are filled or all 512, so all 512
are: 512 GB of identity map, replaced by kvmalloc() a few hundred instructions
later.
**And 3 GB was never the limit, it was a circle.** PHYSTOP_MAX was 0xC0000000
because kvmalloc() mapped exactly four 1 GB pages, and kvmalloc mapped four
because that was the cap -- each holding the other up, throwing away thirteen
gigabytes of a sixteen gigabyte machine. kvmalloc now maps as many gigabytes as
the machine has RAM and tells ioremap where the direct map stopped, instead of
ioremap assuming it stopped at four.
Freeing memory becomes two passes, and the ordering is forced rather than
chosen: kinit1() runs *before* kvmalloc because kvmalloc calls kalloc for its
own page tables, and freeing a page means writing into it -- so kinit1 can only
touch what the loader mapped. kinit_high() adds the rest afterwards.
`phystop` and the new `lowtop` are now two different numbers: the DMA arena
stays below 4 GB because a device is not obliged to reach the top of memory,
and an RTL8139's buffer address register is thirty-two bits wide.
**And the 0 MB.** The System Information screen reported "0 MB usable" on a
machine with sixteen gigabytes in it, because memory_mb() had its own 16 KB
guess at how big a memory map gets, and returned zero when GetMemoryMap said
EFI_BUFFER_TOO_SMALL. That is the same bug as the previous commit's, in a
second copy, and this one failed silently. There is one buffer and one
map_read() now.
Verified at 1 GB, 8 GB and 16 GB: all three boot, no traps, no panics, socktest
and memfdtest green in each. 8 GB and 16 GB did not boot at all before -- and
16 GB now reports 17408 MB with "mem: 13312 MB above 4 GB added to the
allocator", where the old kernel would have used three. Both display
configurations still clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
EFI status 0x5 is EFI_BUFFER_TOO_SMALL, and that is the whole bug: the loader
asked the firmware for its memory map with a 32 KB buffer, which is about 680
descriptors -- generous next to QEMU's map and not generous at all next to a
machine whose firmware has carved out every reserved range it has ever needed.
The descriptor count is a property of the vendor, not of the RAM, which is why
this was invisible in every test here and fatal on two real machines.
The fix is not a bigger guess. A call with a **zero size** fails on purpose and
fills in the size actually needed, and it allocates nothing -- so the map key
from the real call immediately after is still valid, which is the constraint
this whole sequence is written around and the reason the buffer has to be
static. 256 KB now, which is 5461 descriptors, and a machine that somehow
exceeds even that is told in numbers rather than in a status code:
schos: the firmware's memory map needs 5664 bytes and this loader has 512
schos: memory map larger than the loader's buffer (status 0x...0005)
That line is real: the failure path was provoked by shrinking the buffer to 512
and booting, which is also how the 5664 was measured -- QEMU needs 118
descriptors, so hardware needs more.
**A second limit is nearby, and it is only a warning.** SCHBOOT_E820_MAX is 40
entries, fixed by the hand-off block's layout; build_e820() merges adjacent
regions of the same type, so 40 survives a long map. If it ever does truncate,
what is dropped is the high end -- the map arrives sorted and phystop is capped
below 4 GB regardless -- so it stays a warning. But it now appears on screen as
well as on the serial line, because after the splash print() goes to a port a
laptop does not have, and "some RAM was quietly unused" is not a thing to
discover months later from a total that never looked right.
Both supported configurations: one boot each, no traps, no panics, socktest and
memfdtest green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A laptop has no serial port, and the machine we are trying to boot stops with
the loader's splash still on screen. Three completely different things look
exactly like that: the jump never landed, the kernel started and hung, or the
kernel started and panicked with a full report drawn into a framebuffer nobody
can see. They call for different next steps, so the kernel now says which
without needing a console at all.
**A bar, as the kernel's first act.** Eight lines of accent colour across the
top of the framebuffer, written straight through the handoff block before a
single subsystem exists -- no gfx, no scanout, no allocator, nothing that can
itself fail. If it appears over the splash, the jump landed and the kernel is
executing. If it does not, nothing below that point is worth looking at.
Skipped when the framebuffer is above the direct map, because a fault there
leaves the machine quieter still, and free on a boot that works: the console
clears the screen a few lines later.
**Blinking keyboard lights, on panic.** Scroll, Num and Caps together, about
three times a second, for ever. It is the one output every PC has had since
before any of this existed -- no driver, no interrupt, no memory -- and it
works when the graphics do not. Linux does the same thing for the same reason.
Every wait in it is bounded. It runs after `panicked` is set so there is
nothing left to protect, but a machine that hangs inside its own death rattle
tells you less than one that stops quietly. The delay is microdelay, which
spins on the TSC: no timer, no interrupt, no calibration -- all three of which
a kernel that died during boot may not have.
That makes the three cases distinguishable from across the room: no bar and no
blink is a jump that did not land; a bar and no blink is a kernel that hung; a
blinking keyboard is a panic, and the report is in /proc/kmsg after a restart.
**And the loader now describes the mode it is actually handing over**, not only
the one the firmware started in. On a machine where changing the resolution
loses the splash entirely, the difference between those two lines is the bug
report; on a boot that works nobody sees either, because ClearScreen runs a few
lines later.
Verified: `echo c > /proc/sysrq`, then QMP polled for five and a half seconds
afterwards -- the machine answers throughout and a screendump still shows the
panic report intact, so the blink loop runs without faulting, resetting, or
scribbling on what it is there to accompany. Both supported configurations
boot clean with the early bar in place.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two NICs, and unlike the RTL8111 last commit, QEMU emulates both -- so both
are tested rather than reasoned about.
**RTL8139** is its own driver and not a mode of the 8169, because it is its own
chip. No descriptor rings: the receiver is one circular buffer the chip writes
packets into back to back behind four-byte headers, and the transmitter is four
fixed slots. Putting both programming models behind one set of function
pointers is how a mode flag ends up threaded through every path.
Three details are the whole driver. CAPR runs sixteen bytes behind the true
read position -- a quirk of the part, not an off-by-one. WRAP means the chip
runs a packet past the end of the buffer rather than splitting it, so the 8 KB
buffer is allocated as 8 KB + 16 + one maximum frame; getting that wrong is a
chip writing past its allocation at line rate. And a length field of 0xFFF0
means "still writing this one", so treating it as a length copies a partial
packet and loses the alignment for everything after it.
**The e1000 now matches the PCIe 8257x parts.** Its header refused them, with
a note saying they "differ in NVM access and initialisation, and claiming a
device this driver has not been shown to initialise is worse than not claiming
it". That was right, and the second half is answerable: QEMU emulates the
82574L, so it can be *shown* to initialise.
What differs turned out to be one register. EERD moved its address field from
bits 8-15 to bits 2-15 and its done bit from 4 to 1; the rings, RCTL, TCTL and
ICR are identical. So there is a per-instance flag for the NVM layout and seven
more ids -- not a second copy of four hundred lines of ring code. The wrong
pair of constants fails silently, reading the wrong word or waiting for a bit
that never sets, and the MAC comes out as zeroes; that is why it is named in
the header rather than papered over.
Still refused: the ICH/PCH parts (82562, 82577-82579, I217-I219), which put the
PHY behind a MAC-PHY interconnect and need a different bring-up nothing here
can test.
Tested, each booted alone: rtl8139 (10ec:8139) and 82574L (8086:10d3) both bind,
read a real MAC, complete DHCP and ping 4 of 4 with no loss. The classic
82540EM does the same, which is the regression that mattered for the NVM
change. Both supported display configurations: one boot each, no traps, no
panics, socktest and memfdtest green.
Also recorded: the Chromebox CN62 carries an RTL8111, not an Intel part, so
last commit's driver covers it too.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The laptop's NIC is an RTL8111H, and the request was to add it to LDLS.
Measuring it is what changed the answer, so the measurement is the commit
message: r8169.ko (6.18.52) has **183 undefined symbols, 130 of them not
implemented by ldls/kpi/, and 35 of those 130 are phylib** --
phy_connect_direct, phy_start, mdiobus_read, genphy_soft_reset and the rest.
phylib is a subsystem with a link state machine, not a set of leaf calls, and
LDLS's usual answer for an unimplemented symbol -- a stub that reports itself
and returns zero -- is exactly wrong for it. phy_start returning zero means the
link never comes up and there is nothing to look at. The 8139too that ldls.md
uses as its model has no phylib dependency, which is why that one was
tractable and this one is not.
So: a native driver. It reaches the PHY through the chip's own MII window --
PHYAR is one register that is a whole MII bus -- and skips the subsystem. Less
work than the port, no GPL entanglement, no firmware blob, no coupling to one
Linux version, and it covers the whole RTL8111/8168 family because all of it
answers to 10EC:8168. The register layout is a fact about the hardware, taken
the way ldls.md section 3 says facts may be taken; the expression is this
tree's, in the same shape as the e1000 beside it.
**It has not run on hardware, and that is not a hedge.** QEMU emulates rtl8139
and nothing else from Realtek, and the 8139 is a different programming model --
a circular receive buffer rather than descriptors -- so it is deliberately not
in the match table. What is verified: it builds, its presence changes nothing
on a machine without the device, and with an rtl8139 on the bus it correctly
leaves it unclaimed. What is not: every register write, on a real part.
The probe therefore says everything a first bug report needs, with cprintf
rather than kinfo so it reaches the screen of a machine with no serial port:
r8169: xid 5c0, 1c:1b:b5:xx:xx:xx, irq 11, link up (bmsr 796d)
xid is the hardware version from the top of TxConfig -- the PCI device id has
been 8168 for twenty years and says nothing. bmsr is MII register 1: bit 2 is
link, bit 5 is autonegotiation complete. Those four facts separate "the chip is
not there" from "the PHY sees nothing" from "the rings are wrong".
Two constants were wrong in the first draft and are recorded because both would
have worked quietly. RxConfig had 0x7700 for "unlimited": MXDMA is bits 8-10
and RXFTH is bits 13-15, so that was one bit short of a full burst and a
half-full FIFO threshold. And CPCR bit 3 was being set under a comment calling
it "PCI multiple read/write" -- bit 3 is PCIDAC, 64-bit addressing, and bit 4
is what the comment described; neither is wanted, so the write is gone.
Both supported configurations: one boot each, no traps, no panics, socktest and
memfdtest green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
schdbgd ships Disabled, and its manifest explains why: a debugger that can read
the process table should not run on a machine whose owner never asked for one.
That default is right and has one bad consequence -- **the machines that most
need debugging are the ones you cannot get a shell on to enable it.** A laptop
with no serial port that stops before it can draw has nowhere to say why, and
`schinitd enable` needs a prompt that does not exist.
So `make DEBUG=1` adds a sixth entry to the F8 menu. Picking it toggles;
booting carries the choice through SCHBOOT_DEBUG_FLAG; the kernel publishes
/proc/bootdebug; schinitd starts the one job that flag may un-disable, with
the manifest's Args now naming 0.0.0.0 -- a debugger reachable only from the
machine being debugged is no use when that machine is the one that will not
talk to you.
Four decisions, since this is a network-reachable debugger:
compiled out, not hidden A release build cannot set the flag because the
code that would set it is not in the binary. A
menu item that is present and refuses is an
invitation to make it not refuse.
consent at the machine Somebody standing at the keyboard during boot is
the strongest consent this system can ask for and
the weakest one that is any use on hardware that
will not boot.
one job, named in schinitd Not a manifest key like EnableIfDebug: the general
version of "the boot menu starts things that ship
off" is worse than the specific one, and this way
what the flag can start is one readable line.
the kernel only publishes Nothing in the kernel behaves differently, so a
debug kernel and a release kernel are the same
kernel and a bug cannot be the flag's fault.
/proc/bootdebug is its own file rather than a second line in /proc/bootmode,
because bootmode's whole promise is that reading it is a string compare and
not a parser.
**Also: `tone` stopped lying.** On a machine with no sound card it reported
"the device will not take 16-bit samples", which sends you looking at the
format code instead of at the empty PCI slot -- /dev/dsp exists whether or not
hardware is behind it, and the ioctl is where the absence shows up. ENODEV is
now told apart from a format refusal.
Verified end to end: F8 -> item 6 -> boot -> schinitd says it is starting
schdbgd -> a client outside the VM connected to the forwarded port and ran
`dma`, getting the real arena dump back. The release build has no such entry
in the binary at all, and boots clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A program that wants to make a sound opens /dev/dsp, writes PCM, and closes
it. That is what UNIX audio has always been and it now exists here:
tone 440 2
Three pieces, in the order a sample passes through them. **/dev/dsp** is a
character device where write() is audio and a handful of OSS ioctls say what
the samples are. **AudioEngine** is everything above a driver that is not
specific to a chip -- the format, whether there is room, and what a write does
when there is not. **A driver** is bound by class, exactly as a display driver
is: device_find_class(DEVCLASS_AUDIO).
OSS numbering rather than something new, because every UNIX audio program ever
written already knows it and the only thing a different set would achieve is
making ports harder. They are read/write ioctls, which is the part that
matters: a program asks for 44100, the device sets 48000 and writes 48000
back, the program resamples. One that ignores the answer plays 9% fast, and
that is its bug rather than the driver's.
**The engine is deliberately one stream.** Mixing several programs is a real
feature with a real design behind it, and building it before anything had
played a single sound would be exactly what docs/plan/1.0.md keeps a register
for: work that exists and is connected to nothing. The second program that
wants to play at the same time is what should cause it to be written.
**audio_wait() sleeps on the timer tick, not on the device.** That looks like
the lazy choice and is not: a ring that drains is a ring with room, so asking
again in ten milliseconds always makes progress, whereas sleeping on the card
makes audio depend on an interrupt arriving. On a machine whose line is
misrouted that is not a stutter, it is a writer that never wakes. The handler
still accounts for finished buffers -- deleting it would cost efficiency, not
correctness -- and it retires descriptors through the same `tail` pointer
queue() uses, so the two cannot disagree about what is in flight.
**AC'97 first because HDA is two unproven things at once.** It is the simplest
audio device that exists on real machines and in QEMU both. Intel HDA is what
the laptop actually has and is a much larger driver; writing it against an
interface nothing had ever played through would have meant debugging the
driver and the interface simultaneously. HDA arrives as a second
struct audio_ops and nothing above it changes -- which is the entire point of
the class.
Two things the wiring needed: PCI class 0x04 now maps to DEVCLASS_AUDIO (the
driver bound and /proc/devices showed it, but audio_get() found nothing,
because the class byte had never been translated), and include/x86.h grew
inw() -- AC'97's mixer is all 16-bit registers and reading one a byte at a
time returns the low half twice on some codecs.
**Tested by measuring what came out, not by not crashing.** QEMU's wav backend
writes the guest's audio to a file; `tone 440 2` produced 1.97 seconds of
44.1 kHz stereo with peak amplitude 14177 against the generator's table
maximum of 14178, and 880 zero crossings per second -- 440 Hz exactly. wav is
on every QEMU target rather than a host backend, because a build machine may
have no sound card and it is the only backend whose output can be checked.
Both supported configurations: one boot each, no traps, no panics, socktest
and memfdtest green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
"Leaving the firmware, and the bar is full" -- so on the laptop
ExitBootServices was reached, and everything after it was silent. die() draws
on screen, so a failed ExitBootServices would have said so; it did not, which
means the firmware call succeeded and the machine stopped somewhere in the
handful of instructions between there and the kernel's first.
That window had no diagnostics because of a claim in a comment: "nothing below
can report anything". Half true. ConOut is gone, but the *framebuffer* is not
-- it is memory, the firmware has stopped using it, and ui_progress_label() is
a write to it and nothing more. So the window is now split: "Starting the
kernel" is drawn after the page tables are built and immediately before the
jump. A machine that stops on "Leaving the firmware" stopped in
ExitBootServices or the page-table build; one that stops on "Starting the
kernel" stopped in the jump or in the kernel itself, before it clears the
screen.
**And one speculative change, reverted.** On the theory that hardcoded low
memory is a poor assumption under UEFI -- the firmware is entitled to have
anything at 0x1000 -- the page tables were moved to pages asked of
AllocatePages. It cost two triple faults: cr3 still loaded the constant, and
then the PML4 entries still pointed at the constant PDPT. Both were fixed, and
it still reboot-looped, which is when the actual objection turned up:
kernel/arch/x86_64/entry.S:85 # bootstrap page tables (0x1000 - 0x3000)
kernel/arch/x86_64/entry.S:111 mov $0x1000, %eax
mov %eax, %cr3
Those addresses are a *contract between the loader and the kernel's assembly*,
not this file's private choice. Moving them in one of the two places was never
going to work, and the theory that started it had no evidence behind it -- no
report says 0x1000 is the laptop's problem. So it is reverted, with the
contract written down where the constants are, and if it ever does move it
moves in both files at once.
Both supported configurations: one boot each, no traps, no panics, socktest
and memfdtest green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two reports from an Asus TUF FA607NU, which now gets as far as the splash:
the progress bar does not move, and after a resolution change the screen keeps
the background colour with no mark and no bar. Both point at the same code,
and it is code this loader did not have a week ago.
**Modes were being matched on every frame.** mirrors_match() ran from inside
mirrors_show(), so every menu redraw and every segment of the kernel load
asked another display adapter to change mode. On QEMU that is instant and
idempotent. On real hardware a mode set can take the best part of a second, can
blank a panel while it happens, and -- on a laptop -- is quite often being
asked of the *same* panel reached through a second GOP. A loader that does that
in the middle of drawing a progress bar is a loader that shows a background
colour and then waits on a firmware call that is not coming back.
It now happens exactly once, at discovery, before anything is drawn. The
second head is centred and cropped afterwards rather than re-moded, and the
booted system matches resolutions anyway because the kernel driver and vgpud
both follow the console.
**And mirroring no longer runs per segment of the kernel load.** Copying a
megabyte-and-a-half framebuffer to every other adapter, three times, to animate
a bar on a screen nobody is looking at.
**One refusal now turns it off for the rest of the boot**, and
`\schos\nomirror` -- an empty file whose only job is to exist -- turns it off
before it starts. There is no console to ask a question on at that point and no
menu to reach if the menu is what is broken, so the answer has to already be on
the volume. That also makes it a one-file experiment on hardware we cannot
instrument.
**The loader names its stages on screen.** After the splash, print() goes to
the serial port and nowhere else, so on a machine with no serial port a stall
leaves a progress bar and no information at all. Every stage after the kernel
load now labels itself under the bar -- Reading the console font, Reading the
memory map, Leaving the firmware -- so the last thing on the screen names the
call that did not return. Past ExitBootServices there is nothing left to report
with, which is itself worth knowing: a machine stopped on the last label
stopped in the jump.
Opening the boot volume moved ahead of the display, because what is on the
volume now decides some of what the display does.
Both supported configurations: one boot each, no traps, no panics, socktest and
memfdtest green. Mirroring still verified end to end -- F8, a resolution
change, boot, and both heads byte-identical at the new mode.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`jmp mpenter`.
x86-64 requires rsp to be 16-byte aligned immediately before a `call`, so a
function starts with rsp ending in 8 -- the return address -- and its
`push %rbp; mov %rsp,%rbp` prologue leaves %rbp 16-byte aligned. Compilers
build on that: an aligned vector store to a frame slot is `movaps`, and
`movaps` to a misaligned address is #GP.
entry64mp loaded the AP's stack pointer, which is page-aligned, and *jumped*.
Nothing pushed a return address, so %rbp came out eight bytes off -- on every
frame, on every application processor, for the life of the machine.
It cost nothing for as long as no function an AP ran needed an aligned vector
store. Then the console's glyph path started returning a small struct by value
(the cell-as-a-surface change), clang spilled it with movaps, and CPU 1 took a
#GP the instant it announced itself on a verbose boot:
unexpected trap 13 from cpu 1 rip ffff800000120a3b <draw_glyph_px+0x1cb>
120a37: 0f 10 45 90 movups -0x70(%rbp), %xmm0
120a3b: 0f 29 45 b0 movaps %xmm0, -0x50(%rbp) <- #GP
movups on the line above loads fine; movaps stores and faults. The
displacement is a multiple of 16, so what is misaligned is %rbp itself, which
is a fact about how the function was entered.
`call mpenter`, not `jmp`. Eight bytes for an ABI the whole toolchain assumes.
**It was hiding behind a second bug.** The first report was not a #GP at all:
Stop code ACQUIRE
#2 cprintf+0xae
#3 trap+0x2c4
trap() reported the fault with cprintf(), cprintf() takes the console lock,
and the faulting code was already holding it -- on a verbose boot every
character is drawn under that lock. acquire() panicked about a lock its own
call chain owned, and the trap number, the faulting address and the registers
were all lost. A kernel-mode trap ends in panic() a dozen lines later anyway,
so the console goes into panic mode *before* the report rather than during it.
**And a third, in the same boot.** The console reported "JetBrains Mono, 8x16
cells" -- a name from one font and metrics from another. The built-in bitmap
became a provider like any other in the previous commit, so draw_glyph_px()
installs it lazily on the first glyph; on a verbose boot that happens before
gfx_font_init() runs, which then returned early because a font was already
installed. The console spent the whole boot in the fallback while claiming not
to be. "A font is installed" and "a typeface was chosen" had been the same test
and quietly stopped being; they are two flags now.
Verified: verbose boot reaches a shell twice running with no traps and no
panics, and reports 10x21 cells -- the real typeface -- where it reported 8x16
before. Both supported configurations quiet: one boot each, no traps, no
panics, socktest and memfdtest green.
Written up as section 22 of docs/history/kernel.md, because the interesting
part is not the missing eight bytes but that three separate faults had to be
peeled off in order before the first one could be seen.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The first piece of real-hardware work, and it is three bugs that QEMU cannot
show you. The report was an ASUS laptop -- Ryzen 7000-series APU, Radeon
integrated, RTX discrete -- booting to a white underscore on a black screen,
and unable to be offered 1920x1080.
**The framebuffer address was truncated.** The loader handed over
`(UINT32)gop->Mode->FrameBufferBase` and the kernel read a 32-bit field out of
a VBE mode-info block -- a *BIOS* structure the UEFI path had been filling in
because the kernel already knew how to read one. A machine that puts its
aperture above 4 GB, which is ordinary on a laptop with a discrete GPU, got a
plausible-looking address pointing at nothing. The splash went into it, the
kernel was handed it, and the firmware's own cursor kept blinking on an
otherwise black screen. Every step in that sequence succeeded on a number that
was wrong, which is why there is no error message anywhere in it.
SCHBOOT_FB_BASE is the loader's own display block: 64-bit address, 64-bit
size, 32-bit width, height and stride, and the pixel format. The VBE block is
still written for the BIOS loader, but anything that does not fit is left at
zero rather than wrapped -- a truncated address is this bug, and a zero is at
least honest.
The kernel then has a problem the loader does not: its direct map covers the
low 4 GB, so a framebuffer above it needs ioremap(), which needs kpdpt, which
kvmalloc() has not allocated when the early console starts. A low framebuffer
is attached exactly as before; a high one is attached immediately after
kvmalloc(), still thousands of lines before anything else.
**There was more than one adapter, and the first one was taken.**
LocateProtocol answers with *a* GOP and the specification says nothing about
which. QEMU with a stdvga and a virtio-gpu has **three** handles and two of
them are Blt-only with a framebuffer address of zero -- so this is not a
hardware-only hazard, it was a one-in-three chance of drawing into address 0
that happened to keep coming up heads. Every handle is scored now: no
aperture, no candidacy; a readable current mode beats one that needs changing;
size breaks the tie.
**mode_usable() rejected modes real machines have.** A padded scanline (every
GPU that aligns its rows -- and very likely the reason 1920x1080 could not be
offered), a width not divisible by eight (1366x768, which is half the laptops
in the world), and anything below 800x600 (the only mode some firmware gives
over a KVM). All three passed on QEMU, which is why they survived. The mode
list now offers 1366x768 and 640x480 where it did not.
The loader also prints what it found, to the firmware's text console, which is
cleared before the splash: invisible on a boot that works, and the only thing
on the screen on a boot that dies before the splash.
One process note, recorded because it cost a wrong test: `make UEFI_KEEP=1`
ships the loader that is already built. Its own comment says "a build that
quietly ships a stale bootloader is how you spend an evening debugging code
that is not running", and that is exactly what happened -- a regression run
that looked clean had tested the old loader. Loader work gets a full build.
Both supported configurations verified after a clean build: one boot each, no
traps, no panics, socktest and memfdtest green, and the adapter report shows
the right one being chosen out of three.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Gate 0's last item, "delete the cell grid" -- done as far as it should be, and
the rest of it retired, which is the part worth explaining.
include/text.h made the case against the grid in four charges: it cannot do
variable-width glyphs, cannot position text off a multiple of eight, cannot
anti-alias, and cannot hold a codepoint above 255. Every one of those was
about the *drawing layer*, and every one is now false -- glyphs carry metrics,
PSOGM positions text at arbitrary pixels, the TrueType path blends 8-bit
coverage, and vtcell.ch is a uint32.
What is left is a grid in vt.c, and deleting that would not modernise the
console; it would stop it being a terminal. VT100 escapes address columns,
TIOCGWINSZ reports rows and columns, and every program that draws a box
assumes both. So the item is checked with its remainder rewritten to say what
was actually wrong, rather than ticked as if the original wording had been
carried out.
**What was actually wrong was the duplicate.** gfx.c walked font_data and
blitted glyphs its own way; text.c did the same job into a struct surface, for
no caller but a sysrq self-test. Two implementations of bearings, coverage and
clipping, which had to agree about a cell size only one of them was told had
changed -- and they did not agree, which is why every line of PSOGM once drew
the next character's bitmap underneath itself.
There is one now, and the mechanism is the pleasing part: the console hands
text_draw_glyph() **the cell as a surface of its own** -- same pixels, same
stride, origin moved and bounds set to that cell. Clipping a tall ascender to
its line is then the same bounds check that clips to the edge of a screen, and
there is no second notion of clipping in the tree. `surface_sub()` is six
lines and replaced fourteen of hand-written comparisons.
The bitmap is reached through font_builtin() like any other provider, so the
early console and the real one are one path with different metrics, and the
`if (!cfont)` branch that walked the array is gone. gfx.c no longer includes
font.h; text.c does, being the only file left that reads it -- which also
retires the note in text.c explaining why it could not.
struct surface gained a bpp, because gfx_draw_pixel handles 16, 24 and 32 and
whatever replaces it has to as well. Zero means 32: a UEFI GOP has no other
format and the loader publishes 32 unconditionally, so callers that do not
care say nothing.
One behaviour change, and it is the documented one: a codepoint the font
cannot draw is now a substituted glyph rather than a blank cell. text.h argues
for that directly -- "a missing glyph that draws nothing makes text silently
shorter and the fault impossible to see".
Verified by screendump: the console renders correctly, PSOGM renders
correctly, and `echo t > /proc/sysrq` still draws its specimen through the
surface path with the back buffer at 32bpp. Both supported configurations: one
boot each, no traps, no panics, socktest and memfdtest green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Gate 0's last-but-one open item. Drop a TrueType file at \schos\console.ttf on
the ESP and the console is drawn with it; with no file, the face linked into
the kernel image, which is what an early panic needs and is not a degraded
mode.
**The loader reads it, and that is the decision worth writing down.** The
kernel opening the file once the root filesystem is mounted is the obvious
design. It is also wrong: the console has been drawing since the kernel's
first line, so a font arriving later changes the cell size, which changes the
column count, which re-lays out every terminal -- and blanks them. The kernel
log would vanish off the screen at the moment the root filesystem mounted,
which is exactly the log a machine being brought up for the first time is
being read for. Since the next thing on the list is real hardware, losing that
was not a trade worth making.
So the choice happens before anything is drawn. Three details make it cheap:
- **EfiReservedMemoryType**, not EfiLoaderData. The E820 handoff turns
loader data into usable memory, on the argument that the loader's own
pages are the kernel's the moment it runs -- true of the loader, not true
of a buffer it wants the kernel to keep reading from.
- **Below 4 GB**, because the kernel reaches this through its direct map and
that is how far the direct map goes. A font above it would need an ioremap
the console cannot do before there is a console.
- **Used in place.** No copy, which matters because the kernel has no
contiguous allocator to copy 112 KB into -- the only one is the DMA arena,
and that belongs to devices.
Sized without EFI_FILE_INFO and its GUID: seek past the end and ask where that
landed, which the specification requires to clamp.
Every failure falls back and keeps going -- no file, implausible size, no
memory, short read, unparseable file. The last says so in the log, because a
font that silently did not load looks exactly like a font that did.
Verified by booting with JetBrainsMono-ExtraBold.ttf on the ESP and
screendumping the console: visibly the bold face, against the Regular of a
boot with no file. Three boots each way, no traps and no panics, socktest and
memfdtest green in all six.
Nothing ships a console.ttf: the file would be the file already in the kernel,
and two copies of a font are two things that can disagree.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Eleven cards in a grid is a list people stop reading somewhere around the
fourth one. A row you move through is one you go along, and the section is a
survey rather than a comparison -- nothing in it needs to be read beside
anything else, which is the test for whether a grid was carrying information
or just filling space.
So it is a `.nv-scroller`: snapping, hidden scrollbar, arrows that appear only
at the end that has something behind it, and an edge fade that says "more this
way" without a scrollbar having to. All of that is the component's.
**This site's part is one declaration.** `--nv-scroller-item: 21rem`, because
18rem is a width for a card carrying a line and these carry a paragraph. I had
also written `align-self: stretch` to make the heights match, then checked:
they already do, because a flex row stretches by default. The rule came back
out -- restating something the system already guarantees is exactly the drift
the split between Niveus and `app.css` exists to prevent.
The honest cost: a reader who does not scroll sees three cards rather than
eleven, and the section's own claim ("every card marked runs is in the tree")
is about all of them. The fade and the arrows are what pay for that, and they
are the reason this is a scroller rather than a row with a scrollbar.
Verified in a browser: 11 items, equal 256px heights with no rule of ours,
`data-nv-at-start` set and `data-nv-at-end` clear on load, and the arrows page
the rail and swap over as it reaches each end.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Niveus is the interface language the project uses from here on -- this site
first, applications after it. Rebuilding the site on it is what proves the
system is one, rather than a folder of CSS with a README.
**`style.css` is gone.** 21 KB that transcribed include/theme.h by hand:
palette, type scale, shadows, spacing, every component. The intent was right --
"a site describing an operating system should look like the operating system"
-- and the mechanism was a copy, which means two files that have to be kept in
step by remembering to. What replaced it is `app.css`, which is layout and
nothing else, and one declaration in each page:
:root { --nv-brand: #7C5CFF; }
That is SCH_ACCENT. Niveus derives the accent and its states, the link colour,
the focus ring, the tint in the greys and how far the status colours lean, for
light and dark both. One number to keep true instead of forty.
**The site is now adaptive.** Dark by default, because the OS is dark, with a
Light/Dark/Auto group in the bar that is real. The choice is kept in
localStorage and re-applied by an inline script before first paint, so the page
never renders in one appearance and corrects itself while you watch.
**PSOGM keeps its one purple, and now says so structurally.** #4B1D9A is
spelled the way Niveus spells a subtree with its own colour -- `data-nv-brand`
with its own `--nv-brand` on that section -- so every role inside re-derives
from it rather than being painted on. Still exactly one place, which is what
branding/README.md asks for.
Content is unchanged; what changed is what it is made of. Cards are `.nv-card`,
tags are `.nv-badge`, the support table is `.nv-table`, the crash section is a
`.nv-bed`, and the docs reader's two rails are `.nv-nav--vertical` with the
current document marked by `aria-current` rather than a class of our own.
`docs.js` emits `.nv-table` and `.nv-code`, so a change to the system reaches
the documentation too.
Three things were found by looking at it in a browser rather than by reading
it, and all three are the same mistake: **a `.nv-card` carries no padding of
its own -- its parts do.** The stat tiles, the "One ISO, one stick" band and
the card headers were all built from bare children, so numbers ran to the card
edge, the USB mark hung off it, and a title and its badge sat at opposite ends
of a row meant for a title and a trailing slot. Where the component had a right
way to say it, the markup now says it that way; where it did not -- a tile has
no parts -- `app.css` supplies the padding and says why.
Checked: every nv- class used across the three pages and docs.js exists in the
CSS, every token referenced by app.css is defined, and app.css contains no hex
value at all. Niveus's own tools/check.py passes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two things.
**The F8 resolution now applies to the other displays as well.** It already
reached them once vgpud started -- that was the last commit but one -- but not
before: choosing 1024x768 changed the screen it was chosen on and left the
other at 1280x800, so the menu and then the splash appeared on it as a small
rectangle in the middle of a larger black one. Which is a fair reading of "the
resolution change is not changing resolution".
The resolution offered in that menu is a statement about the machine's display,
not about one adapter of it, so both stages before vgpud now follow it:
the loader mirrors_match() puts every mirror GOP into the primary's mode,
called from mirrors_show() so it cannot be forgotten at a new
call site. Two integer comparisons in the common case.
the kernel setup_scanout() takes gfx's dimensions over GET_DISPLAY_INFO's.
Same argument as vgpud's bring_up(): the host window's size is
a fact about QEMU, and the console's mode is the one somebody
chose.
**And the greeting is no longer a race.** "Welcome to SchOS!" was printed after
every job had been started, including the shell on the console -- so which of
the two reached the screen first was up to the scheduler, and it went both ways
about evenly. A boot that sometimes greets you above the prompt and sometimes
below it does not read as two orderings of one boot; it reads as a line printed
by mistake.
schinitd now holds back the job that owns the console -- `own_tty`, which is
exactly the job whose output lands there -- until after the greeting is
written. That is enough to settle it: this process writes the line, and only
then asks PID 1 to spawn the shell, so the fork has not happened yet. Nothing
waits on it that was not already waiting.
Verified: F8 -> Resolution -> 1024x768 -> boot, with the virtio head
screendumped over QMP at three points during startup and once at the prompt --
1024x768 throughout, and byte-identical to the stdvga head. Three consecutive
boots put the greeting at the same offset, ahead of the prompt every time.
Both supported configurations: one boot each, no traps, no panics, socktest and
memfdtest green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two things, both asked for.
**The crash report no longer says which monitors it reached.** "The report is
on the virtio-gpu display too" was a line about the crash screen printed on the
crash screen. The report says what happened and where to read it again; that is
not part of it.
**And the second head now shows the F8 menu and the boot.** It used to be black
until vgpud started several seconds into userspace -- through the splash,
through the menu, through every boot message -- which from the outside is
indistinguishable from a display that does not work. Three stages now cover the
whole boot and hand over to each other without coordinating:
*The loader.* LocateProtocol() answers with the first GOP and offers no way to
ask for the rest, so the rest are found by handle and fed the primary's
framebuffer as a Blt source at the points where a screen is finished: the menu,
the resolution chooser, system information, the splash, each segment of the
kernel load. Blt rather than a second framebuffer to draw into, because a
virtio GOP does not have one -- the same fact that made a virtio-only machine
reboot-loop two commits ago.
*The kernel.* gfx remembers which rows were drawn, as one span rather than a
list of rectangles, because console output is lines and a span is what lines
make. display_mirror_console() copies that span across: a line of boot text is
one cell-row transfer, not a screen. Primed once after pci_init(), because a
display that has just appeared has a black scanout and the console it should be
showing was drawn by the loader into a framebuffer it knows nothing about.
*Then vgpud*, from the same console framebuffer, unchanged.
**Two bugs found building it, both worth the retelling.**
The mirror ran per character from inside vt_feed -- which is inside the console
lock -- and vga_disp_acquire() announces its back buffer with a cprintf. A CPU
taking a lock it already holds, with a call chain that said exactly that. It
runs at the end of consolewrite() and cprintf() now, outside the lock, which is
also the better batching: one transfer per write instead of one per character.
And "is this the console's own screen" was asked of the pointer acquire()
returned. vga_fb hands back a back buffer it copies onto the console's
framebuffer afterwards, so on a one-screen machine the answer was "a different
screen" -- and the console was mirrored onto itself through two megabytes of
intermediate, allocated for the purpose. display_ops grew scanout_phys, which
is a fact about the hardware rather than a policy a driver can get subtly
wrong, and it is asked first: it has no side effects and acquire() has two.
Verified with two heads under QEMU, screendumping the virtio head every 400 ms
from power-on: splash, splash, splash, then the console, and it stays. The F8
menu appears on it; choosing 1024x768 there brings both heads up at 1024x768
byte-identical. A panic still reaches both. Both supported configurations: one
boot each, no traps, no panics, socktest and memfdtest green.
One gap remains and is documented rather than hidden: while the kernel's driver
resets the device during probe, QEMU shows "Display output is not active" for a
fraction of a second. A device being taken over has no scanout until the new
driver sets one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
"it also doesn't show crash screen on virtio gpu or just other gpu than default
vga" -- true, and on the virtio-gpu it was worse than nothing. The head kept
showing the console from a second before the panic: one screen saying the
kernel is dead, the other showing a shell prompt.
panic() painted into the console framebuffer and stopped there. Two cases were
missing, and they are not the same problem.
**The kernel still drives it.** display_acquire() hands back a buffer; the
report is copied in and presented. Ordinary, and a no-op when the display
class's buffer *is* the console's, so a one-screen machine does not do the work
twice.
**A process drives it.** The hard one, and the one hybrid.md owed an answer
for. The kernel let go of the device deliberately, and the process holding it
will never run again -- there is no scheduling after a panic -- so there is
nobody left to ask. It is not a race that can be won by being careful.
So the userspace driver says where it is scanning out *in advance*:
PCIIOC_PANICFB, registered by vgpud after bring-up and after every modeset. On
a panic the kernel paints the report into that memory and asks the kernel
driver the device was taken from to put it on screen one last time --
`panic_present` in struct pci_driver, which virtio-gpu implements by resetting
the device out from under a driver that is still nominally running. That is
safe here and nowhere else, because nothing is going to run again.
The panic path allocates nothing. It cannot: a corrupted heap is one of the
ways a machine gets here, and an allocator that deadlocks takes the crash
screen *and* the serial log's last chance to be read. So the driver keeps its
BAR mappings across the handover, rebuilds one queue from 20 KB of static
memory, and bounds every spin -- including the reset acknowledge, which is an
unbounded `while` in probe() and correct there.
**The registration is checked like the promise it is.** The range must lie
inside a block that process got from PCIIOC_DMA on that device. Without that,
a claimed device would be a way to nominate any physical address in the machine
-- page tables, kernel text -- to be overwritten with four megabytes of crash
report at the one moment nothing is left to notice. That is a worse bug than
the one this fixes. It goes away with the claim and with the process.
What it buys is worth stating plainly: a device can be evicted to userspace
without the kernel losing its ability to tell you the machine died. The
alternative was a driver boundary that is also a blind spot.
Verified both ways under QEMU with two heads, `echo c > /proc/sysrq`, and a
screendump of each over QMP:
vgpud holding the device both heads byte-identical, serial says
"The report is on the virtio-gpu display too."
kernel holding it same, via display_acquire/display_present
(/proc/pci shows 1af4:1050 [virtio-gpu])
Both supported configurations: one boot each, no traps, no panics, socktest and
memfdtest green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
"virtio gpu doesn't follow f8 menu default resolution" -- it did not, and now
that the head mirrors the console that is visible rather than academic.
vgpud sized itself from GET_DISPLAY_INFO, which answers with the size of the
*host's window*: a number set by dragging a corner, that says nothing about the
machine and everything about QEMU. Meanwhile the resolution the F8 menu offered
at boot is the one that was actually chosen, and it is the console's.
So picking 1024x768 at the menu gave a second display still at 1280x800, with
the console letterboxed inside it on the theme ground -- the right pixels in
the wrong mode, on a display that could simply have been the right size.
console_open() now runs *before* bring_up(), and bring_up() takes the console's
mode when there is a console. That ordering is the whole fix and it is not
cosmetic: the DMA allocation is sized from those numbers and fixed for the life
of the process, so a mode chosen after the fact could only ever be smaller than
the window already paid for. A machine with no console to mirror brings up
whatever the device reports, exactly as before.
Verified by driving the F8 menu over the serial line -- 'o', Resolution,
1024x768, boot -- and screendumping both heads over QMP: both come up 1024x768
and are byte-identical, where the virtio head used to stay 1280x800. `schmode
1024 768` from a 1280x800 boot still works. Both supported configurations: one
boot each, no traps, no panics, socktest and memfdtest green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
"but Virtio GPU is still non functional. it only shows logo" -- and that was
a fair reading of it. A second monitor showing a mark for ever is
indistinguishable from a second monitor that does not work.
The mark was the right answer to the question hybrid.md asked: H2 and H3 needed
a screenshot that proved a *userspace process* had drawn something, and that
the driver came back after being killed. Both are closed. What was left was a
display whose entire output was a logo, on a machine whose console was sitting
on the other head.
So vgpud mirrors the console onto it.
**A copy with a diff.** /dev/fb0 maps the console's framebuffer; the virtio
backing store is already mapped, because vgpud allocated it. The backing
therefore holds, at all times, exactly what was last sent to the device --
which means it *is* the previous frame, and "what changed" needs no third
buffer. Ten times a second: compare rows, skip the ones that match, copy the
ones that do not, transfer the one rectangle spanning the rows that changed.
Hence present() becoming present_rect(). Moving one line of text through a
full-screen transfer of a 1280x800 resource is most of a megabyte of copy for
a few hundred bytes of difference, and this loop runs whether or not anything
is happening.
Mapping /dev/fb0 does not take the screen from the console: the pages are the
ones the console draws into, and fb_mapped() only arranges a redraw when the fd
is closed. This one never is.
Sizes differ, so a head larger than the console centres it on the theme ground
the mark already draws, and a smaller head shows the top-left corner -- which
is where a terminal puts everything worth seeing. Scaling is the alternative
and it is the wrong one; text resampled to a non-integer ratio is text you
cannot read. The blocking accept() became poll() with the frame interval as its
timeout, because holding the device is no longer the only thing to do.
Not the endpoint: a compositor will want this head for itself, and when there
is one this goes away. Until then the honest thing for a working display to
show is the system that is running on it.
**And a bug older than any of it.** schmode reported "cannot reach vgpud on
/run/vgpud -- it is not running" about a process that was running perfectly
well. Every boot after the first, on any image whose root is writable, and
nothing to do with this change -- it reproduces on the previous commit, 3 runs
in 4.
bind() creates a node at the path, and /run is a directory on the root
filesystem rather than a tmpfs. The node outlived the process that bound it and
the boot it was bound in, so the next boot's vgpud claimed the device, drew,
held it -- and then failed bind() on a name that already existed. A driver
nobody could talk to, and an error message naming the wrong cause.
It unlinks before binding now, which is what a Unix daemon does. The usual
objection, that this lets a second instance steal a live socket, does not apply
here: two vgpuds cannot coexist, because the first holds the PCI claim and the
second is refused the device long before it reaches bind(). The claim is the
mutual exclusion; the socket node never was.
Verified by screendumping both heads over QMP: with the console at 1280x800 the
two are byte-identical, and after `schmode 1024 768` the virtio head carries
the console at the new mode. Three consecutive boots of one image all reach a
working control socket, where the previous commit managed one. Both supported
configurations: one boot each, no traps, no panics, socktest and memfdtest
green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Booting with a virtio-gpu as the *only* display reboot-looped. Not a hang, not
a panic: the firmware came back round to the boot menu, started the loader
again, and did it again, for ever, with nothing to say why.
virtio-gpu has no linear aperture, so OVMF's virtio GOP implements Blt and
nothing else. On a machine with one and nothing else, the firmware reports a
width and a height and a FrameBufferBase of **zero**:
schos: framebuffer 0x0000000000000000 0x500x0x320
Two places took that at face value.
**kmain() tested `if (bi.ptr && ...)` with bi.ptr already run through P2V()**,
and P2V(0) is KERNBASE, which is not zero. So the guard could not fail -- and
its own comment, three lines above, said "drawing into address zero is a worse
failure than staying quiet". The intent was exactly right and the test could
never enforce it. It reads the physical value first now and tests that.
**vga_fb_init() attached the scanout regardless**, one layer down, so fixing
kmain alone just moved the same write.
Either one had gfx_clear() write four megabytes over physical 0: the real-mode
IVT, the BIOS data area, the AP trampoline at 0x7000, the early page tables.
Hence a reset with no output -- there was nothing left to run.
Both refuse now, and the refusal says what it is:
vga: firmware reported a 1280x800 display with no framebuffer address;
no graphical console
A machine with no usable linear framebuffer gets no graphical console. That is
the truth about it, and the serial line still has everything.
**It still does not reach a shell on such a machine**, and that is recorded
rather than implied: the console path assumes a framebuffer -- vt_init()
reports a terminal of `0 by 0` cells -- and that path has never been exercised.
What this fixes is a machine that destroyed low memory and reset; what it does
not do is make a headless boot work. Supported remains stdvga, with virtio-gpu
optional beside it, which is what docs/architecture/graphics.md already said.
Both supported configurations verified: stdvga alone and stdvga with
virtio-gpu, one boot each, no traps, socktest and memfdtest green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The three F8 screens painted into the framebuffer and nowhere else. That is
right on a machine with a screen and useless on one driven over a serial cable
-- which is how this loader is usually driven, and which f8_window() already
went out of its way to support: it accepts 'o' as well as F8 because, in its
own words, "a serial console sends no scan codes at all".
So the prompt invited a serial user in, drew them a menu they could not see,
and then blocked for ever on a key they had no way to choose. The whole visible
behaviour was:
Press F8 for boot options...
<nothing, for ever>
Each screen echoes itself to COM1 now -- the boot menu, the resolution list and
System Information. The whole thing once on entry, then one line per *move*;
not on every keypress, because a serial terminal has no cursor to paint over
and an unrecognised key would scroll the selection past again for nothing.
Straight to COM1 rather than through print(): print() goes to ConOut until the
splash is up, and ConOut on a machine with graphics draws its text across the
menu that was just painted there.
Two details found by actually driving it, neither cosmetic:
- **The item number is shown only where it is a key you can press.** The boot
menu takes 1-5. The resolution list takes only W/S and Enter and is longer
than nine entries, so numbering it printed ':', ';', '<' past the ninth --
a character nobody can type, offering to select something.
- **Q goes back as well as Esc.** A lone ESC byte on a serial line is
indistinguishable from the start of an escape sequence, so the terminal
driver holds it back waiting for the rest and it never arrives as a key.
Esc is the obvious thing to press on a screen and unreachable on a cable --
the same shape of problem as F8, and the same answer.
Driven end to end over serial: open the menu, walk it with W/S, open System
Information and come back, open Resolution and come back with Q, then boot.
The graphical menu is unchanged apart from the hint now naming Q.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two reports, and the first one was not an F8 bug at all.
**Choosing "Boot SchOS (verbose)" appeared to do nothing.** The menu worked,
the flag crossed, and the kernel really did boot verbose -- and then the
machine *reset* partway through and the second boot was quiet. So what anybody
ended up looking at was an ordinary boot, which reads as "verbose is broken".
PROBE before startothers, ncpu=2 <- verbose boot
SchOS UEFI loader <- the firmware, again
PROBE before startothers, ncpu=2 <- quiet boot
PROBE after startothers
mpmain() opened with
kinfo("cpu%d: starting\n", cpunum());
enable_sse();
idtinit();
and the order is the bug. enable_sse() clears CR0.EM; until it runs, *any* SSE
instruction on that CPU faults, and the kernel is not built with -mno-sse, so
the compiler emits them freely for struct copies and inline memset. idtinit()
loads that CPU's IDT; until it runs there is nowhere for a fault to go. So the
#UD is not a panic -- it is a double fault with no handler, then a triple
fault, then the firmware comes back up. A triple fault prints nothing, which is
why there was no last line to go on.
**Why only on a verbose boot**, which is what disguised it as an F8 problem:
kinfo() is log-only when the boot is quiet, so the work is a few bytes into a
ring and no XMM is involved. A verbose boot renders that same line through the
TrueType rasteriser, and that is where the SSE lands.
Setup first, announcement second. The general rule, stated where the next
person will meet it: a processor announces that it is up after it is able to
run, not before.
Verbose now boots once and puts the whole log on the screen -- `verbose boot:`
lines, `cpu1: starting`, the mount, `schinit: ready`, the jobs starting -- which
is what F8 was for and what a machine with no serial port needs.
**And vgpud drew a test card.** Three colour bands, four gradients, a
checkerboard, and the driver's own pid in binary along the bottom edge. That
was scaffolding for hybrid.md's H2 and H3: a screenshot had to answer "did a
*userspace process* put this here", and "did the driver come back" needed a
restarted instance to look different from one that never stopped. Both
milestones closed a while ago and the test card stayed, so a second display
showed something garish with nothing to explain it.
It draws the Shift mark on SCH_BG now, from the same include/logo.h the loader
splash and the kernel console use. That answers the same question a black
screen cannot -- something is driving this device -- as the system rather than
as a test rig.
Also: vgpud's "no control socket" line is vputs, not puts. It is progress, not
news, and it was printing on the console of every quiet boot.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Gate 1, item 3, and the last of foundations.md.
bind() creates a T_SOCK node at the path; connect() resolves the path and looks
for the socket bound to **that inode**. Matching on the inode rather than on
the path *string* is the part that matters: the name used to live only in the
socket table, so two spellings of one path missed each other, a relative path
never matched an absolute one, and there was nothing to stat() or unlink().
An inode is the thing a path resolves *to*, so connect() can walk the path
however it likes and arrive at the same socket.
# ls -l /tmp
srwxr-xr-x 1 root root 0 2026-09-22 08:04 socktest
The node is a marker, not the binding. It holds no data and the live
association stays in the socket table, so a socket file left by a crash is
what Linux leaves behind -- a dead file that connect() refuses until something
binds again -- and unlink()ing a live socket's path does not disturb anyone
already connected.
EADDRINUSE comes from the *directory* rather than from a scan of the table,
deliberately: a stale socket file has to refuse a rebind for the same reason a
running server's does, or two servers both believe they own it. The creation
lives in sys_bind() and not in unix.c, because that is the layer with both --
kernel/net/unix.c knows about sockets and has no business calling create().
T_SOCK is a compat addition to SchFS by the rule in schfs.h: a driver that does
not know SCHFS_T_SOCK maps it to a zero-length regular file, so nothing goes
stale and nothing becomes unreadable.
cmd/socktest section 15 asserts the plan's own wording -- bind, stat, see an
`s`, unlink -- plus the two a name table could never fail: a second bind to a
live path is refused, and connecting to a path that is a *regular file* is
refused rather than matched by name.
**And the semantics being real broke socktest on its second run**, correctly:
the socket files persisted and every bind() hit EADDRINUSE. Every real server
unlinks before binding; socktest now does too, which is the change the new
behaviour demands rather than a workaround for it.
Chasing that turned up something worse. `socktest` reported rc=1 with every
check passing, and so did every other first-party program:
- **lib/start.S called main and threw %eax away.** sys_exit() reads the
status from %rdi and nothing put it there, so every program exited with
whatever that register happened to hold. Every test suite ends in
`return failures ? 1 : 0` and none of it reached the shell.
- **exit(void) passed no status either**, for the same reason: the
declaration takes no argument, so %rdi was the caller's litter.
`if socktest; then` works now, and a failing command reports 1 -- checked both
ways, because a fix that makes everything return 0 would look identical from
the success side.
Also factored: unlink_path() out of sys_unlink, so a failed bind can take its
own node back out without going through the syscall entry, and create_sock()
as a named door rather than exporting create() with an arbitrary type.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Gate 1, item 2, and the thing a Wayland buffer crosses on.
memfd_create(2) is the same segment machinery shmget() already had -- pages,
refcounts, no_free VMAs, the fork and exit accounting -- reached by
**descriptor** rather than by System V key. That difference is the entire
point and the reason it exists on Linux at all: two processes share memory by
passing the fd over a socket, so neither has to agree on a path or race to
invent a key. It is also why SCM_RIGHTS had to be item 1.
docs/plan/foundations.md proposed exactly this and it was right: what memfd
adds on top of shmget is a segment that starts empty and is sized by
ftruncate(), and a file that owns one reference to it. FD_MEMFD, and
read/write/lseek/fstat all learn about it -- a descriptor that refuses read()
is a surprise nobody needs.
Three things are refused rather than faked, each the honest version of
something this kernel cannot do:
- MAP_PRIVATE on a memfd. Every caller is sharing on purpose; a private
mapping of an anonymous object is an expensive MAP_ANONYMOUS.
- Mapping past the end. Linux faults those pages in; there is no handler
here that could.
- Shrinking while mapped. Linux truncates the pages out from under the
mapper; here a freed page would stay mapped to memory somebody else now
owns.
Writable MAP_SHARED of a *real file* is still ENODEV. That needs a page cache,
and an anonymous object has no such contract to break -- which is precisely
why memfd is the narrow answer.
**cmd/memfdtest**, seventeen checks, written so a permissive implementation
cannot pass: an mmap that quietly returned private zeroed pages satisfies every
return code and fails the one assertion that matters, because the reader is a
different process. It ends with the combination the plan singled out -- a memfd
sent over a socketpair to a receiver that closes every inherited copy first,
maps only the descriptor that arrived, reads what the sender wrote and writes
back where the sender can see it.
Two things were found by writing that test, both of which had been refusing
everything that was not FD_INODE:
- sys_fstat() never reached filestat(), so the memfd branch there was
unreachable from the syscall that needs it. It also refused pipes and
sockets, and returned -1 (which musl reads as EPERM) rather than an errno.
- sys_lseek() the same, and SEEK_END needs a length that is not in an inode.
Verified: 40 consecutive runs leave MemFree at exactly the value it started
at -- 936656 kB before and after -- so nothing leaks across creation, dual
mapping, descriptor passing and exit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Gate 0's last structural row. kernel/core/kmem.c: slabs are pages from
kalloc() with a header at the front, and free objects hold the free list in
their own first eight bytes.
That last detail is the whole design. **An object carries no header and
freeing performs no search**: the page base of any object *is* its slab, and
the slab names its cache. It is also exactly why an object cannot exceed a
page, which matters below.
Two interfaces. kmem_cache_create/alloc/free for a subsystem with one kind of
thing -- it gets its own line in /proc/slab, which is most of why naming it is
worth the argument -- and kmalloc/kmfree over power-of-two classes from 16 to
2048 for code with no single type to build a cache around. Each cache keeps
partial and full lists; a slab that empties goes straight back to kalloc()
rather than being kept, so /proc/slab never overstates what is held.
`make kmem-test` asserts the bookkeeping on the build machine. Same arrangement
as schfs-objtest, and for the same reason: slab lists, free-list threading, the
page-to-cache lookup and whether pages actually come back are all things that
can be wrong without a kernel to be wrong in. 26 checks, including one that a
permissive allocator cannot pass -- write a distinct pattern through 512
objects and read them all back, because two objects sharing storage shows up
there and nowhere else.
One bug the test found immediately: kmem_max_obj() returned the raw page
remainder, which kmem_cache_create() then rounded *up* past its own limit. The
largest legal size was reported as illegal.
**It does not do what the register said it would.** The claim was that every
fixed table exists to work around kalloc. Measured:
struct proc 10656 NPROC 128 larger than a page
struct socket 4928 NSOCK 32 larger than a page
struct buf 4272 NBUF 256 larger than a page
struct inode 280 NINODE 200 a cache, bounded on purpose
struct device 120 NDEVICE 32 3.8 KB, never hit
struct file 48 NFILE 512 24 KB, never hit
The three largest cannot go in a page-backed slab whatever it does. NINODE is
a cache and is meant to be bounded. NFILE and NDEVICE are small enough that
their tables are rounding errors and their ceilings have never been reached.
Freeing the big ones needs a contiguous multi-page allocator, which is a
different thing and is not written. That is now in the register and in
docs/architecture/kernel.md rather than left as a claim nobody checked.
What it does fix is real, and was measured rather than assumed:
- **A pipe is 640 bytes and came from a whole 4096-byte page.** Every `|` in
a shell command burned three and a half kilobytes it never touched. Six
pipes per page now, and /proc/slab shows the page going back when they
close.
- **procread() built its answer in a 2048-byte array on the kernel stack**,
under a comment that had warned about exactly that since it was written --
it stayed a local because a whole page for 2 KB felt worse, and kmalloc is
the answer that was missing. Its frame is 64 bytes now instead of ~2100.
- procfs's other generator took a page from kalloc for the same 2 KB.
Verified on a boot: /proc/slab tracks pipe and kmalloc-2048 through real
pipelines, allocs and frees balance, and $(), ps, time, readlink, sigtest and
rename are all still green with no traps.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`ps` listed the shell and nothing the shell had started -- background jobs
included, which is how it was noticed.
p->ctty was set by TIOCSCTTY and never inherited, so every child of a shell had
ctty 0. `ps` with no arguments shows only processes on *this* terminal, so a
job running on your terminal was invisible because the only process that had
ever claimed one was the shell itself.
The controlling terminal is a property of the session, not of the process that
happened to call the ioctl. It crosses fork() with pgid and sid, three lines
above where it now sits.
# sleep 60 & ; sleep 61 &
# ps
PID TTY TIME CMD
14 1:11 00:00:00 sleep
9 1:11 00:00:00 zsh
15 1:11 00:00:00 sleep
17 1:11 00:00:00 ps
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four Gate 0 rows, and a fifth found while checking one of them.
**getrusage() did not distinguish RUSAGE_CHILDREN from RUSAGE_SELF**, which
undersells it: it ignored `who` entirely and reported the machine's *uptime*
as the caller's user time. Every process of every age answered with the same
number. Per-process CPU time is sampled at the timer tick now and split on the
interrupted privilege level; children's time is rolled up in wait() rather than
exit(), because RUSAGE_CHILDREN means *reaped* children and a running one has
not contributed yet.
**And wait4()'s fourth argument is a `struct rusage *` that was read off the
stack and thrown away.** That is the one shells actually use -- wait3() is the
same syscall with pid -1 -- so zsh's `time` reported
16777216.01s user 1073.78s system 927983249% cpu
for every command. 2^24 seconds is uninitialised memory, and it did not vary
with the workload because it was never written. Now:
busy loop 2.66s user 97% cpu 2.720 total
sleep 2 0.02s user 0% cpu 2.248 total
ls -lR / 0.70s user 65% cpu 1.070 total
**ru_stime is a structural zero and says so.** MSR_SFMASK includes FL_IF, so
SYSCALL clears interrupts and no tick can land while a process is inside a
syscall. Tick sampling cannot see kernel time at all here. Charging it needs
TSC deltas closed and reopened around every sched(), which is scheduler
accounting rather than sampling; the limitation is written down in
docs/architecture/kernel.md rather than papered over.
**/proc/<pid>/exe is a symlink.** It also needed a real st_size, and every
other procfs file deliberately reports zero -- readlink(2) bounds its copy by
it, so a symlink of size 0 resolves to the empty string, which looks like it
worked and is worse than the regular file was. Generating that one path in
ilock() costs one small render per stat() of one file.
**OVMF memory reporting is closed**: 1 GB reports 1017 MB usable, 2 GB reports
2041 MB. The firmware keeps about 7 MB either way, which is the right shape.
**And `ps` printed a header and nothing else**, which is not in any list
because nobody had run it. `ps` with no arguments shows only processes on
*this* terminal; /proc/<pid>/stat hardcoded tty_nr to 0, so nothing could ever
match. `ps -e` always worked, which is exactly what hid it. p->ctty was already
tracked -- procfs simply was not asked. Fields 14..17 were hardcoded zero too,
so TIME read 00:00:00 however long a process had run; they carry the new
counters now, and a shell that has burned 653 ticks reports 00:00:06.
Verified: utime climbs 18 -> 653 across a loop in the shell itself, `ps` lists
its own terminal, `ps -e` still lists everything, `readlink /proc/1/exe` gives
/bin/schinit, and sigtest/$()/rename/dmesg are all still green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`#define ROOTDEV 1` -- IDE disk 1 -- so a kernel could only ever root on the
disk it was *built* expecting. Booting the same image from a USB stick was
impossible by construction, and that was the one thing standing between this
kernel and its own acceptance test, which begins "boot on real hardware from a
USB stick".
rootdev_mount() probes. schfs_kmount() was already written for exactly this --
its own comment says "the uberblock scan doubles as the probe: no valid
uberblock means this is simply not a SchFS volume, which is not an error worth
reporting loudly" -- and a failed probe leaves the slot unmounted and reusable,
which is what makes a loop over candidates legal at all. So the change is one
function and one call site; the only real use of ROOTDEV was `schfs_kmount()`.
Order is preference, not confidence: a RAMDISK_ROOT build wins outright, then
USB, then IDE 1, then IDE 0. USB before the fixed disks because a stick that is
plugged in was plugged in on purpose. iderw() panics on an absent drive rather
than failing, so each candidate is checked with ide_disk_present() /
usb_storage_present() first -- which is what those predicates are for.
Verified with **no IDE disk attached at all**: boots, mounts the stick, writes
a file, and the file is still there on the next boot.
[ 0.000021] root: USB disk (device 3)
And when nothing matches, it names what it looked at, because "no root
filesystem" with no list is the least useful thing a kernel can say at the one
moment somebody needs to know which disk it expected:
root: no SchFS volume on any candidate device. Looked at:
device 3, USB disk: not present
device 1, IDE disk 1: not present
device 0, IDE disk 0: present, but no SchFS on it
**That message is also how dmesg turned out to be broken**, because it did not
appear in `dmesg` and should have. `dmesg` printed *nothing at all*, on a
system whose boot banner says "or dmesg once you are in" and whose entire
quiet-boot design is "route the detail into the kernel log and say how to read
it". The log was there the whole time -- `cat /proc/kmsg` has always worked --
but the one command anybody would reach for could not see it.
Two separate faults, each alone enough:
- **sys_syslog() was `return 0`.** dmesg does not read /proc/kmsg; it calls
klogctl(10) for the buffer size and klogctl(3) to read. 0 to the first one
means "the log is zero bytes", and a stub that answers 0 is
indistinguishable from a machine that logged nothing. Implemented for the
actions that mean something here; the rest return -EINVAL rather than 0,
because a console log level this kernel does not have is not a request that
succeeded.
- **With that fixed, dmesg was still empty.** It parses every line with
`sscanf(msg, "<%u>[%llu.%llu] %n", ...)` and silently drops anything that
does not match. The ring's `[ 0.00] text` already matches everything
after the priority -- scanf skips the leading spaces -- so the only missing
thing was the `<6>`. klog_render_syslog() adds it on the way out rather
than storing it: the ring is also what a person reads with `cat
/proc/kmsg` and what PSOGM points at after a crash, and an ABI detail
belongs at the boundary that has the ABI.
Also: the first version of the no-root diagnostic printed
%-11s device 1596400
because cprintf() has no field-width specifier -- it prints "%-11s" literally
and then reads the next argument for the %d. Plain %s now, with the note.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Checking the rest of ZSH_CACHE after the killesrch fix, on the principle that
one wrong answer in a hand-written block is rarely the only one. Two more, both
`yes`, both measurable on the running system in a line each.
**zsh_cv_sys_path_dev_fd=/dev/fd.** Neither /dev/fd nor /proc/self/fd exists --
/proc/self does not exist at all. So `<(cmd)` failed at the point of use with
cat: /dev/fd/11: No such file or directory
which reads as a broken command rather than as a feature the system does not
have.
**zsh_cv_sys_fifo=yes.** mkfifo() does not fail here. It *succeeds*, and
produces a character device: mknod() turns S_IFIFO into a T_DEV and keeps the
major/minor, so `ls -l` shows `crw-rw-rw- 438, 0` where a `p` belongs, and
nothing written to it comes back out. Succeeding wrongly is what made `yes`
look defensible.
Both `no` now. `<(cmd)` says "doesn't look like your system supports FIFOs",
which names the missing thing instead of blaming cat, and `$( )` is unaffected.
The two gaps are recorded in docs/architecture/filesystem.md rather than left
implied by a build flag -- named pipes and a path to one's own descriptors are
both real features, and neither exists.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`X=$(echo hi)` never returned. It has been in docs/history/kernel.md section 7
as "mksh's job control still doesn't reliably progress ... root cause not yet
isolated" with three suspects, and it was none of them, and it was not the
kernel at all.
ports/zsh/Makefile carries a block of `zsh_cv_*` answers for the checks
autoconf cannot run when cross-compiling. One of them was
zsh_cv_sys_killesrch=no
which reads like a reassuring property and is in fact the answer to the
question configure asks: *does kill(pid, 0) return ESRCH correctly?* `no` makes
configure emit BROKEN_KILL_ESRCH, and Src/zsh_system.h answers that with
#ifdef BROKEN_KILL_ESRCH
#undef ESRCH
#define ESRCH EINVAL
#endif
zsh's waitforpid() ends its wait on `errno != ESRCH` (Src/jobs.c:1569). With
ESRCH redefined to EINVAL (22) and the kernel correctly returning ESRCH (3),
that condition was true for ever: the shell sat in sigsuspend waiting for a
child it had already reaped. An instrumented zsh says the whole thing in one
line -- kill returned -1, errno is 3, and the constant it is compared against
is 22:
WFPkill -1 3 22
**Autoconf's own cross-compilation default for that check is `yes`.** Deleting
the line would have been correct. Writing it down as "a fact about SchOS" is
what got it wrong, and the block's comment now says to read each name as the
question rather than as a property.
Verified: `X=$(echo inline)`, `$(/bin/echo external)`, backticks, nested
`$(echo $(echo deep))`, a slow child, and `$(ls /bin | wc -l)` all return.
mksh -- as /bin/sh -- was never affected.
Two corrections to yesterday's note, both confident and both wrong:
- "Not zsh, mksh hangs the same way" was false. mksh installs as /bin/sh,
so that test was a `command not found` whose output scrolled past. Had the
binary been named right, this would have localised to zsh's build in one
step instead of a day.
- cmd/sigtest had already proven the kernel's SIGCHLD/sigsuspend contract,
and a musl-linked probe had already shown kill(pid,0) returning -1 with
errno == ESRCH == 3 and zsh's own loop condition evaluating false. Both
results were right; both were read as "so it must be subtler in the
kernel". What they actually pointed at was the one place the probe and the
shell differed -- how each was compiled.
The lesson is narrow enough to state: when a probe that reproduces a program's
exact syscall sequence behaves correctly and the program does not, stop looking
under the syscalls and look at the build.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Nexoniarz