The kernel now checks owner/group/other on open, create, unlink, rename, exec
and path search; honours set-user-id and set-group-id; restricts kill, mount,
reboot and mknod; keeps supplementary groups; and implements chown. devtmpfs
nodes carry owners and modes that chmod/chown keep. Terminals run login, the
image has root and schos, X starts as the ordinary user.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EVRhxi9zpdW2uDUN4FVcao
Kernel devices (input, mem, tty, graphics, block) get /sys/devices/virtual
entries with dev/uevent/subsystem, /sys/class and /sys/dev links, input
capabilities, and writable uevent files that announce on the netlink socket.
eudev now runs, processes a full trigger and settles.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EVRhxi9zpdW2uDUN4FVcao
README: rewritten around what runs (Linux software, Xorg, apk, Linux's drivers), the honest gaps, measured numbers.
plan/: 1.0.md is the finish line, where each clause of the acceptance test stands, and the register (one failing
clause: a root that survives a reboot); built plans (foundations, hybrid, alpine, xorg) and the decided-against one
(wayland) are gone, their content lives in architecture/; roadmap.md is only what is still open.
guides: building/layout/testing/adding-a-driver (Linux code may be used). history: four entries (ls -lR, TLB
shootdowns, MDIO 0x02460246, netlink MSG_TRUNC). Website: cards, stack, download page, docs index.
Code comments that pointed at deleted plan docs point at the architecture docs.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EVRhxi9zpdW2uDUN4FVcao
The cursor now costs nothing for a byte inside an escape sequence and
one cell fewer for a plain glyph: the grid is the truth, so the old
cursor cell is repainted after the byte, and not at all when a glyph
has just been drawn over it.
__fortify_panic says "r8169: rtl_open.cold+0x..." through ldlsd's module
list, so a photograph of the screen is enough, and exits instead of
spinning on a core and holding /dev/netdev.
docs/plan/finch.md: only integrating Finch is later; finishing it is
not. The self-hosting gap measured against Finch's own bootstrap test.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015f73FesGZGYPkmDvUcj6QX
The virtual terminals never drew a cursor. Each now paints a steady
block (the cell with its colours swapped), erased before and redrawn
after every output byte, redrawn by a repaint, and hidden by
ESC [ ? 25 l as curses programs ask.
r8169 under LDLS on the TUF A15 got through probe and died in ndo_open
on __fortify_panic. netdev_priv() is `dev->priv` now, a flexible array
counted by net_device.priv_len, and rtl_open's memset of its ring
bookkeeping is bounds-checked against it. The allocator left priv_len
zero, so the driver saw an area of no bytes. It is set from BTF when the
kernel has the member. __fortify_panic also says which check, how many
bytes it had and wanted, and where from.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015f73FesGZGYPkmDvUcj6QX
The RTL8168 is now driven by Linux's r8169, unmodified, in a process.
`make ldls-r8169` stages it from the running kernel together with libphy,
mdio-bus, mdio_devres and the Realtek PHY driver, the kernel's BTF and
the RTL PHY firmware, and etc/schinit/ldls-r8169.job starts it at boot
with its log in /var/log/ldls-r8169.log. The native driver is out of the
registry; its code, now with Linux's 8168H setup, stays as the fallback.
LDLS grows what that needed:
- ldlsd loads a comma-separated module set in dependency order, each
linking against the ones before it, with BTF read before any
init_module runs;
- a driver core (bus match, driver probe), devres, kernel printf, real
udelay/msleep/ktime and 1000 Hz jiffies, phylib's bitmaps and
link_mode_params, firmware from /lib/firmware, and the symbols whose
stub zero was a lie (list validity, try_module_get, dev_err_probe,
eth_platform_get_mac_address);
- pdev->bus is a real pci_bus, and cmovcc from per-CPU memory is
rewritten like the other %gs forms.
In QEMU all five modules load, link and initialise, and r8169 registers
and finds no device. Probe and traffic are untested until real hardware.
The kernel turns ASPM off on Realtek NIC links at boot, as r8169 does
through pci_disable_link_state, which a process cannot.
Idle CPUs enter ACPI _CST's C2 (CC6 on Zen) by port read or MWAIT when
the APIC timer keeps running, instead of only ever halting at C1.
docs/plan/gpu.md: amdgpu and nouveau through LDLS, with measurements.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
Caps, Num and Scroll Lock now light their LEDs on PS/2 and USB keyboards,
share one state between keyboards, and stop toggling on key repeat. X sets
them through evdev writes while it holds the keyboard.
Unmapping memory in a process with threads on other CPUs now flushes their
TLBs with an IPI. Without it a freed page stayed reachable through stale
translations, which corrupted Epiphany's heap on SMP.
ACPI batteries and AC adapters appear under /sys/class/power_supply.
/proc/<pid>/maps lists file mappings, and a process killed by a fault
names the file and offset it died in. r8169 gets a 256-entry RX ring.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
- Anonymous memory can be reserved without being backed: large mappings,
MAP_NORESERVE and PROT_NONE are recorded and each page appears, zeroed,
on first touch -- from trap() for the program, from uva2ka() for the
kernel on its behalf. WebKit reserves 4 GB at start and touches a sliver;
backing it all failed with ENOMEM and Epiphany aborted.
- r8169: the 8168G generation and later (the laptop's 8168H, XID 541) are
taken from the firmware's out-of-band mode, get their PLL powered, the
receive gate opened, and their PHY reached through the OCP window. The
PHY's power-down bit is cleared; the old code kept it while restarting
autonegotiation. Untested on the hardware so far.
- The network thread follows each device's link into lwIP. The link was
read once at attach, 0.02 s into boot; real autonegotiation takes
seconds, so on hardware the interface stayed down for good.
- Twelve terminals, switched with Ctrl+Alt+F1..F12 by default: the keys X
uses, so the console and the desktop agree. AltGr+F-key is gone from
the defaults (chvt -c altgr+F brings it back).
- SO_PASSCRED on Unix sockets, with SCM_CREDENTIALS delivered by recvmsg.
WebKit sets it on its IPC sockets and aborted when refused.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
SDL 3 has no OSS backend and SchOS has no ALSA kernel side, so sound now
goes SDL -> alsa-lib -> alsa-plugins' OSS plugin -> /dev/dsp. That path
found two bugs underneath it:
- ac97: resetting the PCM-out box also clears the descriptor list address,
and nothing set it again. Every stream after the first stop played a
"list" at physical address 0 -- silence -- and ALSA resets before each
stream. If the engine halts at the end of the queue, the run bit is set
again.
- /dev/dsp: SETFRAGMENT, GETFMTS, GETCAPS, SETTRIGGER, GETOPTR and GETODELAY
for the plugin's ring-and-pointer model, and poll(), writable once a whole
fragment fits (ALSA spun at 76% CPU otherwise). speaker-test now plays in
real time at 0% CPU, verified in a recording of QEMU's AC'97.
- /etc/asound.conf: the default device is a plug to the OSS device at 16-bit
stereo 48 kHz, what every SchOS audio driver plays.
Also:
- getstackpcs: each frame link must go up the same stack by less than
KSTACKSIZE. The chain ends at the syscall entry with the user's rbp,
which lbreakouthd uses as a register (-1024); the walk dereferenced it
under the syscall tracer and the kernel panicked inside acquire().
- fb: FBIOGET_VSCREENINFO reports nominal timings, as Linux's efifb does,
instead of zeros, which Xorg's fbdev turned into a 0 Hz RandR mode.
- procdump (Ctrl-P) shows where runnable threads are in user space.
- World: lbreakouthd instead of lbreakout2 (its stk.c passes vsnprintf a
size of 511 plus the prefix length, and Alpine's fortified headers trap on
that), plus curl, alsa-lib and alsa-plugins.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
A thread is a struct proc that shares three counted objects with others of
its group: the address space (struct mm: page tables, heap top, VMAs and a
lock for changing them), the descriptor table, and the signal dispositions.
clone() shares what its flags say and fork() copies.
- clone(): pthread_create's shape (VM, FILES, SIGHAND, THREAD, SETTLS,
PARENT_SETTID, CHILD_CLEARTID) and posix_spawn's (VM, VFORK, a stack).
- futex: WAIT, WAIT_BITSET with timeouts, WAKE, REQUEUE, CMP_REQUEUE, keyed
by the word's kernel address so shared pages meet. set_tid_address keeps
the word that exit zeroes and wakes, for pthread_join.
- exit ends a thread; exit_group, a fatal signal, or the leader exiting ends
them all. reapd frees exited threads; the leader is reaped by its parent
once the rest are gone. getpid is the group, gettid the thread.
- Stops and resumes reach every thread; other process signals the leader.
- exec kills the other threads and unshares the dispositions.
- No TLB shootdown, so pages unmapped from a shared address space are freed
only once every CPU has reloaded CR3 or is idle.
- procfs: /proc lists processes, task/ lists threads, comm, Threads:, Tgid.
- The console honours VMIN 0 / VTIME (btop's input drain hung on it).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
- SIGSTOP/SIGTSTP/SIGTTIN/SIGTTOU stop, SIGCONT resumes; the parent gets
SIGCHLD and wait4 reports WUNTRACED/WCONTINUED; ps shows T. Ctrl-Z works,
and mc no longer hangs waiting for its subshell to stop.
- Blocking reads (console, pty, pipe) and sleeps take a pending stop and
carry on after SIGCONT; a caught signal without SA_RESTART ends them
with EINTR.
- nanosleep reads tv_nsec (sleep 0.5 used to return at once), writes the
remainder, and clock_nanosleep understands TIMER_ABSTIME.
- rt_sigaction uses the k_sigaction layout for Linux-ABI processes: the old
action's restorer was written 112 bytes past the caller's struct. Flags
are kept; SIGKILL and SIGSTOP cannot be caught.
- F1..F12 reach programs, as the Linux console sends them (PS/2, USB, I2C).
- TIOCLINUX subcode 6 answers "no modifiers" instead of echoing the 6 back.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
- /dev/ptmx and /dev/pts/N: Unix98 pairs with a full line discipline,
window sizes, job control and hangup (kernel/dev/pty.c).
- The console speaks scroll regions, insert/delete line, SU/SD, RI/IND/NEL,
DECSC/DECRC; swallows charset designations and private CSI sequences;
and LF only returns the carriage under ONLCR.
- setsid() drops the controlling terminal, so a pane's pty becomes its ctty.
- pipe2() honours O_NONBLOCK and O_CLOEXEC (libevent's signal pipe).
- File modes are stored and enforced: chmod, fchmod(at), umask, creation
modes; /tmp is 1777.
- /proc/stat has real user/system/idle; /proc/<pid>/task; kthread flag.
- SO_PEERCRED, IOV_MAX 1024.
- /proc/strace: name a program and its syscalls go to the kernel log.
- World gains htop, btop, tmux, mc, ncdu, fastfetch, games, X toys, NetSurf.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
ACPI, DMI, SMP, xHCI, USB, I2C, NVMe, r8169 and ramdisk progress lines are
kinfo now: in dmesg and on a verbose boot, off the screen of a working
machine. Failures stay cprintf.
vt_repaint also paints the strip right of the last column and below the
last row, which kept the bottom of the X desktop after Ctrl+Alt+F-key.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
On the Ryzen 4600G desktop and the ASUS TUF A15, AMD's GOP "sets" the
smaller modes without leaving the panel's native timing and reports the
mode's own row length. Every check the loader made passed, and the screen
showed three or four squashed copies side by side (1920/640, 2560/640).
Nothing the firmware reports can tell such a mode from a real one.
So a mode change now asks "Keep this resolution?" in the new mode and keeps
it only on Enter within ten seconds. On a mode that does not really work the
question cannot be read, and the old mode comes back by itself.
And the reason to want a smaller mode was the text: the console font was a
fixed 16 px, unreadable on a 1440p monitor. It now scales to about 45 rows,
16 to 40 px per em, on whatever mode the firmware booted in.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
Xorg uses VTs only when stat("/dev/tty0") reports Linux's tty major 4.
SchOS's terminals sit under major 1, so X ran on no VT at all: the console
kept drawing over it, Ctrl+Alt+F-keys did nothing, and nothing ended the
session. stat now reports terminals under Linux's numbers.
startx picks the VT only when uname says Linux, so root's xserverrc passes
the current one itself. An xorg.conf.d file names X's input devices, since
Alpine's Xorg looks them up through udev and SchOS has none yet.
Alt+F1..F6 and Ctrl+Alt+F1..F6 now switch terminals as on Linux, alongside
the settable chord. USB keyboards gain arrows, Home/End/PgUp/PgDn/Ins/Del
and the switch keys, which the PS/2 path always had.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
SCFF, the SchOS Compressed File Format (include/fs/scff.h), is a block
image cut into 128 KB chunks, each its own zstd frame, behind an index.
tools/mkscff writes it from fs-ram.img on every core (-19, 11 s) and
checks each chunk decompresses to itself. The kernel carries the .scff
as its ramdisk and unpacks a chunk the first time a block in it is
read; zero blocks share one page. The kernel and the ISO are the
compressed size, and the root costs memory only for what is touched.
See docs/architecture/scff.md.
zstd 1.5.7 comes from ports/zstd like uACPI: the decoder is compiled
into the kernel at -O2, with a static context and a 64 KB stack of its
own, and mkscff builds the whole library from the same release.
world now has xorg-server, the fbdev and evdev drivers, xinit, Window
Maker, xterm and DejaVu, and /root/.xinitrc runs wmaker. That root is
358 MB, 102 MB as SCFF.
The ISO no longer carries the root a third time as an ISO 9660 tree,
which nothing read. schos.iso: 1389 MB -> 318 MB. El Torito keeps its
own copy of the EFI image, the layout real machines have booted.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
busybox wget hands TLS to ssl_client over a socketpair and then does
shutdown(SHUT_WR). That cut the AF_UNIX link both ways, so ssl_client's
reply was a SIGPIPE and every https:// download hung. SHUT_WR is now a
half-close: the peer reads EOF and can still answer. TCP sockets get
shutdown() at all (tcp_shutdown for the FIN); it used to do nothing.
- exec keeps ignored signals and the signal mask, as execve(2) says.
- /proc/<pid>/fd: a link per descriptor -- the path it was opened by,
socket:[n], pipe:[n], anon_inode:[eventpoll]. /proc/self too. The
global inode bit moves above pid << 8; at 0x10000 it was /proc/256.
- fstat on a socket or pipe answers S_IFSOCK / S_IFIFO instead of EINVAL.
- poll, select and epoll wake as soon as a pipe, socket or input device
changes, not on the next tick.
- Pipes are 64 KB of lazily allocated pages, copied in blocks, with
PIPE_BUF atomicity: 300 MB through `yes | head | wc` in 7 s.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
Xorg finds its VT through /dev/tty0, and without one ran on none: the
console went on drawing into the framebuffer over the X screen. tty0 is
the terminal on screen, as on Linux. FBIOPUTCMAP succeeds, as it does on
a true-colour Linux framebuffer, instead of 256 logged errors per start.
schos-base gives root a .xserverrc with -dumbSched, Xorg's switch for
input on the main loop: SchOS has no threads, and Xorg crashed in
pthread_join on its input thread when the first client left. It adds
the hostname to /etc/hosts, so xauth can resolve schos:0.
zsh loses PROMPT_SP: on a serial terminal narrower than the console its
padding wrapped and left a # above every prompt. bootcheck knows zsh's
prompt and matches without colour codes, so busybox's ls reads as text.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
The live root was a fixed 96 MB image in the kernel: 60 MB of zeros on
the ISO, twice, and room for 60 MB of packages whatever the machine had.
Now the image is only what the root holds, plus slack, measured by
building it once large. Past its end the ramdisk goes on to half of the
memory free at mount, as pages taken the first time each block is
written; an unwritten block reads as zeros and costs nothing. SchFS
keeps no free-space map on disk, so growing a volume is a larger total
before open builds the map: grow_to, set for the ramdisk root.
8 GB machine: a 4 GB root, Xorg installed into it in 77 s with 7.6 GB
still free. The ISO is 172 MB instead of 344.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
Xorg on a framebuffer -- the fbdev video driver, the evdev input driver,
libpciaccess -- asks the kernel a fixed set of questions before it draws
anything, and gives up on the first wrong answer. These are those
answers.
evdev: EVIOCGVERSION, GID, GNAME, GPHYS, GUNIQ, GPROP, GBIT, GKEY, GABS,
GREP and the accepted setters. Xorg and libinput decide what a device
*is* from the capability bitmaps; /dev/input/eventN answered only
EVIOCGRAB, which reads as no device at all. Each device's bitmaps are
its defaults plus whatever it has reported, so a USB tablet makes the
pointer absolute.
fbdev: FBIOPUT_VSCREENINFO succeeds for the mode already set, as on
Linux -- the fbdev driver sets it every time it starts -- and still
refuses any other. FBIOPAN_DISPLAY to the origin; FBIOBLANK.
The console: KDSETMODE(KD_GRAPHICS) stops it drawing on a terminal,
KDSKBMODE(K_OFF) stops keys reaching its tty, and VT_PROCESS now does
the switch handshake: leaving the terminal sends relsig and waits for
VT_RELDISP, arriving sends acqsig. A terminal whose owner has died goes
back to the kernel whole, so a crashed X cannot leave the console dead.
/sys: boot_vga, a writable enable, and graphics/fb0 under the display
device, which is how the fbdev driver pairs a card with /dev/fb0.
ports/linux-headers: <linux/*.h> into musl's prefix, from sabotage-
linux's kernel-headers 4.19.88-2. Its checksum is recorded from the
first download; upstream publishes none.
cmd/xready makes each call and checks the answer, including the switch
handshake on tty2 and an owner that dies holding it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
An ASUS TUF A15 booted with its internal keyboard dead and a USB one
fine. These AMD laptops' firmware describes IRQ 1's polarity wrongly --
Linux keeps a quirk list -- and an edge line programmed the wrong way
up never fires: the controller holds its byte, and only the interrupt
that never comes would read it.
irq_watch() lets a driver say how to ask its device whether something
is waiting. The poller asks every tick; 50 ms of waiting with nothing
delivered turns the line's polarity over and drains the device, which
re-arms the edge, and if it still says nothing the line is polled.
Both are announced. The keyboard and the PS/2 mouse are watched.
ISA lines now also take their mode from the MADT's override entries,
which were parsed and never read.
Tested in QEMU, which ignores polarity on edge lines, with the keyboard
line masked: the first key turns it over and typing works; masked again
after the flip, it is polled and typing still works.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
- schinit, schinitd and vgpud build every message from pieces, one
write() each, so a job starting on another CPU landed mid-line:
"schinitd: starting vgpud: os.schos.resolvconfno virtio-gpu on this
machine". They now collect a line and write it at the newline;
resolvconf's verbose line is one write with its prefix.
- The ACPI SCI registers IRQ_QUIET. A desktop's SCI never fires, and the
poller took two seconds of silence for a misrouted line ("irq 9:
nothing has arrived on it") and called the handler every tick. A line
is only exempt while every handler on it is quiet.
- The loader claims each segment from where the last one's claim ended,
so two segments sharing a page no longer ask for it twice. When a
claim is still refused, it names the memory-map entries in the way.
- Stale divesymlink() declaration gone; the kernel.md struct table
re-measured (inode, buf and file pools grow from slabs now).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
Sixty-five headers sat side by side in include/, so `#include "irq.h"`
said nothing about where irq.h was or what it belonged to, and errno.h,
stat.h and fcntl.h at the top shadowed the system's for any host tool
built with -I./include. They are in folders now -- arch/ core/ fs/
dev/ gfx/ net/ uapi/ (and ldls/ for the Linux driver layer) -- and
every include names the folder, as Linux's do: "core/proc.h".
Done mechanically and checked the hard way: the kernel built from this
tree disassembles identically to the one built from the tree before
it. The full image boots, and userland is the same set of programs.
Paths in docs and comments follow; docs/guides/layout.md has a table
of what is where, and tools/fix-includes.py rewrites old-style
includes from the folders in the tree, for code written before this.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
Every page of RAM went onto the free list at boot, which means writing
each one -- a link, and kfree()'s 4 KB of junk -- before the first
process ran: 64 GB of memset on a 64 GB machine to say it was free.
Memory nobody has used yet is kept as a cursor and an end per region
now, and kalloc() takes from those once the list of given-back pages
is empty. A page is first touched by whoever asked for it.
That makes the cap on RAM cheap to raise, and PHYSTOP_MAX goes from 64
to 256 GB, which still leaves 248 GB of the kernel PDPT for device
windows. MEMREGION_MAX goes from 32 to 64 for the firmware maps that
cut RAM into many pieces.
kalloc_free_bytes() walked the whole free list under the allocator's
lock -- two million links on 8 GB, every second top or fastfetch was
running, with every CPU that wanted a page waiting behind it. It reads
a counter now. And the buffer cache, which keeps what it grows to, is
reported as Buffers in /proc/meminfo and sysinfo(2) rather than as
memory used by nothing.
Checked under QEMU at 1, 2 and 6 GB: MemFree right, zsh splitting
200000 words, fork loops, a 74 MB write.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
The fixed tables that a bigger machine than the one they were written
on reaches first, each raised to where it costs little:
NPCI 64 -> 256 a desktop with cards and Thunderbolt passes 64,
and the devices past it -- often the disk or
NIC last in bus order -- were ignored; 44 KB
NSOCK 32 -> 128 Unix sockets, which a desktop session uses by
the dozen; 4.9 KB each, 470 KB in all
NDEVICE 32 -> 128 120 bytes each
NIFDEV 16 -> 64 one dock is a hub, Ethernet, audio and a card
reader at two or three interfaces apiece
NETIF_MAX 4 -> 8
Device names took one digit and clamped there, so the tenth device of
a kind and every one after it was named "...9". Two digits now.
Checked under QEMU with two bridges of test devices: 68 functions,
all in lspci, /proc/pci and /sys/bus/pci.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
NCPU was 16, under what a desktop has today. 64 costs 48 more TSS
I/O bitmaps and scheduler stacks, about 800 KB of BSS. Past 254 the
APIC would need x2APIC mode, which is a separate change.
With more than 16 there were two other ceilings. sched_getaffinity
wrote "CPU 0" and returned 0, and musl clears every byte past the
count returned -- so the mask came back empty and nproc, make -j and
sysconf(_SC_NPROCESSORS_ONLN) all said 1, on any machine. It returns
the mask of processors that started, and its size, now. And /proc
rendered every file into 2 KB, which /proc/cpuinfo ran out of at the
eighteenth processor without a word; /proc/pci, /proc/interrupts and
/proc/mounts were each one bigger machine from the same cut. There is
one 64 KB buffer for it now behind a sleeplock, since there is no
allocator for contiguous pages, and the generators that write raw
are given the real capacity instead of a constant.
Checked under QEMU with -smp 24: nproc 24, /proc/cpuinfo whole.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
The inode table was NINODE (200) entries, searched end to end on every
iget(), and a panic -- "iget: no inodes" -- when all of them were held.
Held means open, a cwd, a running binary or a directory mid-walk, so a
few busy processes reached it. iget() now finds an inode through a hash
on (dev, inum), takes a free one from a list, and makes more from a
slab cache once the NINODE in the image are all in use.
The open-file table had the same shape, NFILE (512) and then ENFILE,
and gets the same treatment: a free list, and the slab past NFILE.
/proc/fsstat's peaks now read against how many exist.
getrlimit(RLIMIT_NOFILE) said unlimited, which musl turns into
sysconf(_SC_OPEN_MAX) == -1, after which zsh believed it had no
descriptors and `exec {fd}<file` failed on the first one. It reports
NOFILE now. And a comment in sys_fs.c held a "*/" that ended it early.
Checked under QEMU: six processes holding 115 files each, 713 open
descriptions and 341 inodes at once, all back to the baseline after.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
The cache was NBUF buffers, 1 MB, on every machine: enough for one with
tens of megabytes and nothing on one with gigabytes, which re-read the
same directory and binary blocks off the disk for every command. It
still starts as those 256 in the kernel image, and a miss with every
buffer busy now makes a new one rather than evicting, up to 1/32 of RAM
or 256 MB. Growing on demand means nothing is spent that is not used,
and binit() needs nothing from the page allocator.
Lookups go through a hash on (dev, blockno) instead of walking the LRU
list, and bflush() walks a creation-order list with an unlocked filter
first, so neither costs more at 65536 buffers than it did at 256.
struct buf's data is a pointer to its own page now; the drivers copy
through their own DMA buffers, so where it sits does not matter.
On a 2 GB QEMU guest: a 40 MB file read back in 0.89 s instead of 1.83,
the big binaries in 0.17 s instead of 0.26. /proc/fsstat shows the
cache as buf_cached now / most.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
hwreport writes one folder to the first read-write FAT mount, or mounts
the first USB stick that takes vfat itself and unmounts it afterwards:
summary.txt (machine, firmware, CPU, memory, every PCI and USB device
with its driver or "-- no driver"), dmesg.txt, lspci.txt in the shape of
lspci -nnk -xxx, lsusb.txt, dmi.txt, memmap.txt, the ACPI tables as raw
.dat files iasl -d reads, and copies of /proc. Serials, the UUID and the
MSDM/SLIC tables (a Windows licence key) are left out.
It reads /sys, which is new: sysfs at Linux's paths and in Linux's
formats -- /sys/bus/pci/devices with uevent, config and resource,
/sys/bus/usb/devices, /sys/firmware/acpi/tables, dmi/tables, memmap
and efi, /sys/class/dmi/id. toybox's lspci and lsusb work unmodified
against it. The USB core now keeps a table of attached devices for it.
The machine's name comes from SMBIOS (kernel/dev/dmi.c). The UEFI loader
hands over the entry point from the configuration table, the kernel
falls back to the 0xF0000 scan, and the log gets Linux's "DMI:" line.
Two bugs found on the way:
- The loader handed over at most 40 memory-map entries, in firmware
order. A laptop's UEFI map is 60-150 descriptors with RAM above 4 GB
listed last, so that is what was dropped. The entries now have their
own area at 0x31000 with room for 256, are sorted before merging, and
the kernel's 32 usable regions keep the largest when there are more.
- MemTotal and sysinfo's totalram were PHYSTOP, an address, which counts
the PCI hole: a 6 GB machine reported 8. Now the usable RAM, summed.
usb.ids moves to /usr/share/hwdata, where lsusb looks for it, and
pci.ids joins it (`make pci-ids`, BSD-3-Clause like usb.ids).
Checked under QEMU q35: the report written to a FAT32 stick, fsck.fat
clean, DSDT.dat disassembled by iasl on the host, lspci -k and lsusb
showing names, 6 GB of RAM seen as 6137 MB.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
NBLKDEV was 16 for every disk and partition together, which a laptop's
NVMe drive with its eight partitions and a USB stick come close to. The
mount table held 8, SchFS 4 volumes and 4 snapshots, FAT 4 volumes.
Now 32, 32, 16, 8 and 8; the device numbers still fit between the ones
already fixed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
kd_write() went through bread(), so every block a write allocated was
read from the disk first and then replaced whole. That was a quarter of
the disk operations of a SchFS write. bgetblk() is bread() without the
read, as Linux's getblk() is.
1 MB on AHCI under QEMU: 2.0 -> 2.8 MB/s in 4 KB writes, 4.5 -> 5.6
MB/s in 64 KB ones. 3 MB of random data written and copied through the
cache's evictions reads back with the same md5.
The 63 KB/s seen earlier is the legacy IDE driver on QEMU's default
machine, at about 10 ms per emulated PIO write; the same disk on AHCI
(q35) is 40 times faster.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
Symlinks are followed in namex_from() now, in the middle of a path as
well as at the end, and a relative target resolves from the directory
that holds the link rather than from the caller's cwd. `ln -s f d1/s;
cat d1/s` said "Symbolic link loop" because it looked for ./f. A loop is
ELOOP after 40 links, a dangling link ENOENT, and open(O_CREAT) through
a link to a file opens the file. divesymlink() is gone.
mknod from a Linux program passes (mode, dev) where the native call
passes (major, minor), and was read as the native one: `mknod n c 1 2`
made node 432,258, and mkfifo made a character device. Both are decoded
now; a FIFO is EPERM, and an existing name is EEXIST.
/proc/mounts listed devtmpfs twice: once from a fixed line and once
from the mount table. The fixed line is gone, and mount_add() refuses
a filesystem mounted on its own root, which is where a second devtmpfs
mount lands. A read-only FAT mount says ro.
/proc/partitions exists, in Linux's format, and /proc/filesystems lists
vfat.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
The filesystem every other system can read: a USB stick, an SD card, the
EFI system partition. It is here first for the hardware report, which has
to leave a machine on something that can be read on another one.
mount -t vfat /dev/usb0p1 /mnt (msdos and fat work too; -o ro)
Written from Microsoft's specification, and matched to Linux vfat where
the specification leaves a choice: 8.3 names with the NT lowercase bits,
long-name entries otherwise, Linux's alias rule (REPORTS, then BIGNUM~1),
case-insensitive lookup, UTC times, the boot-sector dirty bit, every FAT
copy and FSInfo kept up to date. It uses its own sector cache over the
new blkdev_xfer, since a partition need not be a whole number of 4 KB
blocks, and every operation that changes something reaches the disk
before it returns. Inode numbers come from the entry position, or the
first cluster for a directory, plus a small table for entries that moved.
Around it:
- /dev lists every disk and partition the block layer found (major 8,
minor = block device), and raw reads and writes on them go by sector.
- The ESP and MBR FAT partitions are registered, typed so the root probe
passes over them. MBR entries with a bad status byte are skipped, so a
superfloppy's boot code is not read as a partition table.
- statfs, sync, fsync, umount (EBUSY while in use), shutdown, reboot and
the power button all know about FAT.
- create undoes and reports a failed ialloc or dirlink (ENOSPC, EINVAL,
EPERM) instead of panicking; fs_ops gains refuse_types and no_hardlinks.
- filewrite returns a short write instead of panicking on it.
- utimensat is implemented. As a stub returning 0 it made toybox touch
believe the file existed, so `touch newfile` created nothing on any
filesystem.
Tested in QEMU against mkfs.vfat images (FAT12 floppy, FAT16 on MBR,
FAT32 superfloppy and GPT ESP) by writing, renaming, deleting, filling to
ENOSPC and truncating there, then checking with fsck.fat -n and mtools on
the host: all clean, contents match.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
consoleinit() put consoleread/consolewrite in devsw[1], over memread and
memwrite. Major 1 holds the whole mem major (null, zero, full, random,
urandom) and memread already sends the console's minors to the console,
so the override only broke the others: a read of /dev/zero waited for a
typed line. dd and head on either device hung, and `head -c 10 /dev/zero`
only finished because it swallowed the next command.
The console now installs just its open and close hooks. /dev/keyboard
writes are discarded, as consolewrite used to do.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
On the first real machine with a discrete card (an RTX 3060, 800x600 from
the UEFI GOP) the console was slow enough to photograph mid-scroll, and the
boot died on CPU 1's "cpu1: starting": CPU 0 spun 200M times on the console
lock while CPU 1 held it, scrolling, and called it a deadlock.
The framebuffer is device memory in the uncached 3-4 GB page of the direct
map. QEMU's stdvga is RAM underneath, so nothing showed there; on a card,
gfx_scroll_up's memmove read the whole screen back across the bus one byte
at a time, and every anti-aliased glyph read back each pixel it blended.
So the console now draws into a copy of the screen in RAM (vmap_alloc: kalloc
pages behind a page table of their own, since nothing needs it contiguous),
and gfx_flush() copies the changed rectangle onto the card through an
ioremap_wc() mapping -- PAT entry 4 set to write-combining on every CPU.
Flushes happen where drawing ends: a message, a terminal write, an echo, a
terminal switch. Scrolling moves eight bytes at a time instead of one.
/dev/fb0 reads and writes the card itself, as before. The early console's
fb_info now carries the physical address it always should have.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
The firmware's AML is now interpreted. uACPI 6.1.1 (MIT) is fetched by
ports/uacpi, checksum-verified, and compiled into the kernel unmodified,
the same way lwIP is. kernel/acpi/host.c is its host layer: memory,
mapping, port and PCI config I/O, time, mutexes and events that spin
before the scheduler exists, the SCI, and an "acpi" thread for the
work the SCI defers.
PCI interrupts go where the board says. pci_init() now routes every
device's INTx through the root bridge's _PRT before drivers bind. It
follows link devices' _CRS, uses a bridge's own _PRT where the namespace
has one, and swizzles through bridges that don't. Before this, INTx
waited on the 8259's line number, which is the wrong pin in APIC mode.
irq_register() takes a trigger mode rather than a level/low flag,
because q35's links and most SCIs are level, active-high. The MADT's
interrupt source overrides are now parsed, and are what places the SCI.
The power button syncs the filesystems and powers off. shutdown() and
reboot() go through the firmware's S5 and reset register first, and
only then fall back to the emulators' ports, which a real board does
not answer.
Tested under QEMU: q35 with MSI off routes xHCI to GSI 22,
level/active-high, and it takes real interrupts with nothing polled.
system_powerdown powers off q35 and i440fx (QEMU exits 0), and q35
with 4 CPUs boots from NVMe with USB hot-plug working alongside.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S1giNvrHTrPMPw7V6x1kQ4
Devices can now arrive and leave after boot. A "usbhotplug" kernel thread
handles root-port changes, which it is told about by Port Status Change
events and also checks for every quarter second. It also handles changes
behind hubs, which each hub reports on its status-change endpoint. An
unplugged device has its endpoints closed and any transfer in flight
failed. Then its drivers' disconnect hooks run, the slot is disabled and
its memory is freed. Everything behind a hub that goes goes with it. This
needed the interrupt handler to file command and endpoint-0 completions
for their waiters, so that enumeration can run while keyboards interrupt.
On real hardware: the controller is taken from the firmware through USB
Legacy Support before reset, with SMIs off. On Intel 7/8/9 series parts
the ports are switched over from EHCI, which is where firmware leaves
them.
Every interface of a device is offered to the class drivers, not just
the first, so a receiver's mouse on interface 1 gets a driver. Configure
Endpoint now copies the slot context from the controller. Before, it
rebuilt it and lost the route string and TT fields of devices behind
hubs.
HID: pointers are read in report protocol from their report descriptor.
That covers buttons, X/Y and wheel, relative or absolute, and report IDs.
Boot protocol is kept as the fallback for mice and for keyboards.
Non-pointer HID interfaces are declined rather than treated as tablets.
USB keyboards get software key repeat. Mass storage handles up to four
sticks, each unregistered from the block layer when it is pulled out,
along with its cached blocks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HH7xukt4aaiGj1nQEAoQf1
/proc/interrupts on q35 said "LOSING 526" ticks: 5 s of boot with
interrupts off. Timing each step of kmain() found two waits:
- AHCI sent a COMRESET to every empty port and waited a second for a
link. DET = 0 is "no device", so only DET = 1 is worth a reset now
(after 20 ms of grace for a drive that is still spinning up).
- ideinit() read 0xFF from a bus with no IDE controller, took the BSY
bit for a busy drive, and waited out its 1 s timeout. A status of
0xFF means no controller and is answered at once.
That is every PC with SATA disks. q35 now loses 33 ticks, under the
flag; i440fx still roots on IDE and loses none.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HH7xukt4aaiGj1nQEAoQf1
q35 stopped with "no root filesystem" because its disk is SATA, like
every PC since 2008, and SchOS only had IDE. And SchFS could only sit
on a whole raw device, so nothing could boot from a disk that also
carried an ESP.
- include/blkdev.h, kernel/dev/blkdev.c: a registry of block devices
found at probe time, numbered from 4. A disk's GPT (or MBR) is read
when it registers, and each partition becomes a device whose block 0
is the partition's first sector. SchFS needs no format change to live
in a partition; a Linux SchFS driver will see the same bytes on
/dev/sdX2. SchFS has its own GPT type:
47138FA9-B79C-47A6-9F7D-AC1F7F572D5E.
- AHCI driver, written from the AHCI 1.3.1 spec: BIOS handoff,
COMRESET when the link is down, IDENTIFY, 48-bit DMA read/write,
FLUSH. Completion is polled, so the root disk does not depend on
interrupt routing.
- USB sticks also register as usb0, so their partitions are found.
- The root probe tries registered devices between USB and IDE,
SchFS-typed partitions first.
- make disk / tools/posix/mkdisk.sh: a GPT image with a 64 MB ESP and
a SchFS partition, ready to dd to a stick or a disk.
Tested in QEMU q35: the old two-disk setup roots on sata1; disk.img as
the only SATA disk roots on sata0p2; disk.img as a USB stick (firmware
boots from it) roots on usb0p2. Writes survive sync in each case.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HH7xukt4aaiGj1nQEAoQf1
PCI drivers took their interrupt from INTERRUPT_LINE. On real hardware
that is the 8259's number, not the IOAPIC pin the device is wired to
(that is in ACPI _PRT, which needs AML), so nothing ever arrived. xHCI
is entirely interrupt-driven, so the USB keyboard and a USB root would
have hung even on a clean boot. Under QEMU the two numbers agree,
which is why it never showed.
- pci_irq_attach()/pci_irq_detach(): MSI, then MSI-X if the driver
allows it, then INTx. The handler is registered before MSI is
enabled, so no edge is lost. xHCI, e1000, r8169 and rtl8139 use it.
- Lines 24-29 are MSI vectors. The LAPIC error vector moves from 19,
which is a PCI pin on Intel chipsets, to 30. Nothing is placed at or
above vector 64 (T_SYSCALL).
- A storming line is masked and its handlers are polled instead of
being left to eat the CPU. A level line that has had a handler for 2 s
and never fired is polled every tick by a kernel thread.
Tested in QEMU q35 with the root on a USB stick: xHCI and e1000e on
MSI; then MSI disabled and INTx deliberately misrouted, where xHCI got
zero interrupts and the root still mounted through the poller. i440fx
INTx and the rtl8139 hand-over are unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HH7xukt4aaiGj1nQEAoQf1
Every one of these works under QEMU and stops a real machine:
- MADT processors that are "online capable" but not enabled were
counted as CPUs. AMD firmware lists absent cores that way (so does
QEMU with maxcpus), and startothers() then waited for ever for an
AP that does not exist. Only enabled entries count now, x2APIC
(type 9) entries are read too, and the wait is bounded to a second;
an AP that misses it is parked with INIT and the boot goes on.
- The PIT calibration loop had no bound. Recent boards clock-gate the
8254: the output never rises (a hang) or floats high (TSC "0 MHz"
and every clock thousands of times fast). The loop is bounded and
checked, and falls back to the ACPI PM timer, then CPUID 0x15/0x16.
Under QEMU pit=off the PM timer gives the same figures the PIT did.
- Firmware that hands over in x2APIC mode left lapic[] reading
garbage: cpunum() panicked or the timer never started. Each CPU is
switched back to xAPIC first (x2APIC -> disabled -> xAPIC).
- The IOAPIC was always at 0xFEC00000 and interrupts were aimed at
APIC ID "0". Both now come from the firmware's tables.
- INIT-to-STARTUP was 100 us ("Bochs is slow"); it is the 10 ms the
MP specification asks for.
- mouseinit() drained the 8042 with an unbounded loop. With no
controller the port reads 0xFF and the loop never ends. It now
notices there is no controller and skips the mouse.
Tested in QEMU: i440fx and q35, -smp 4, -smp 2,maxcpus=4, pit=off.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HH7xukt4aaiGj1nQEAoQf1
Nexoniarz is the sole author and SchOS was never published under
Apache-2.0, so this is a straight relicence. LICENSE is the FSF GPLv2 text,
every first-party SPDX header moves to GPL-2.0-or-later, and NOTICE,
README.md, docs/licensing.md and the site say so. projects/schfs-linux
follows the rest of the tree.
The GPL gives copyleft: a fork has to ship its source. It also lifts the
old "Linux drivers, never" rule, since GPL-2.0-only code combines with
GPL-2.0-or-later. "Or later" leaves room for GPLv3, but one binary still
has to pick one version, so docs/licensing.md now keeps Apache-2.0 and
GPLv3 code out of the kernel to keep Linux code possible there. BSD
4-clause code goes from "avoid" to "no", because the advertising clause
is GPL-incompatible.
LDLS still ships no .ko. That boundary is now project policy rather than
a legal requirement, and ldls.md, devices.md and licensing.md section 6
say so. refmod declares license=GPL, and ldls-check passes (27 checks).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
schdbgd ships Disabled, and its manifest explains why: a debugger that can read
the process table should not run on a machine whose owner never asked for one.
That default is right and has one bad consequence -- **the machines that most
need debugging are the ones you cannot get a shell on to enable it.** A laptop
with no serial port that stops before it can draw has nowhere to say why, and
`schinitd enable` needs a prompt that does not exist.
So `make DEBUG=1` adds a sixth entry to the F8 menu. Picking it toggles;
booting carries the choice through SCHBOOT_DEBUG_FLAG; the kernel publishes
/proc/bootdebug; schinitd starts the one job that flag may un-disable, with
the manifest's Args now naming 0.0.0.0 -- a debugger reachable only from the
machine being debugged is no use when that machine is the one that will not
talk to you.
Four decisions, since this is a network-reachable debugger:
compiled out, not hidden A release build cannot set the flag because the
code that would set it is not in the binary. A
menu item that is present and refuses is an
invitation to make it not refuse.
consent at the machine Somebody standing at the keyboard during boot is
the strongest consent this system can ask for and
the weakest one that is any use on hardware that
will not boot.
one job, named in schinitd Not a manifest key like EnableIfDebug: the general
version of "the boot menu starts things that ship
off" is worse than the specific one, and this way
what the flag can start is one readable line.
the kernel only publishes Nothing in the kernel behaves differently, so a
debug kernel and a release kernel are the same
kernel and a bug cannot be the flag's fault.
/proc/bootdebug is its own file rather than a second line in /proc/bootmode,
because bootmode's whole promise is that reading it is a string compare and
not a parser.
**Also: `tone` stopped lying.** On a machine with no sound card it reported
"the device will not take 16-bit samples", which sends you looking at the
format code instead of at the empty PCI slot -- /dev/dsp exists whether or not
hardware is behind it, and the ioctl is where the absence shows up. ENODEV is
now told apart from a format refusal.
Verified end to end: F8 -> item 6 -> boot -> schinitd says it is starting
schdbgd -> a client outside the VM connected to the forwarded port and ran
`dma`, getting the real arena dump back. The release build has no such entry
in the binary at all, and boots clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A program that wants to make a sound opens /dev/dsp, writes PCM, and closes
it. That is what UNIX audio has always been and it now exists here:
tone 440 2
Three pieces, in the order a sample passes through them. **/dev/dsp** is a
character device where write() is audio and a handful of OSS ioctls say what
the samples are. **AudioEngine** is everything above a driver that is not
specific to a chip -- the format, whether there is room, and what a write does
when there is not. **A driver** is bound by class, exactly as a display driver
is: device_find_class(DEVCLASS_AUDIO).
OSS numbering rather than something new, because every UNIX audio program ever
written already knows it and the only thing a different set would achieve is
making ports harder. They are read/write ioctls, which is the part that
matters: a program asks for 44100, the device sets 48000 and writes 48000
back, the program resamples. One that ignores the answer plays 9% fast, and
that is its bug rather than the driver's.
**The engine is deliberately one stream.** Mixing several programs is a real
feature with a real design behind it, and building it before anything had
played a single sound would be exactly what docs/plan/1.0.md keeps a register
for: work that exists and is connected to nothing. The second program that
wants to play at the same time is what should cause it to be written.
**audio_wait() sleeps on the timer tick, not on the device.** That looks like
the lazy choice and is not: a ring that drains is a ring with room, so asking
again in ten milliseconds always makes progress, whereas sleeping on the card
makes audio depend on an interrupt arriving. On a machine whose line is
misrouted that is not a stutter, it is a writer that never wakes. The handler
still accounts for finished buffers -- deleting it would cost efficiency, not
correctness -- and it retires descriptors through the same `tail` pointer
queue() uses, so the two cannot disagree about what is in flight.
**AC'97 first because HDA is two unproven things at once.** It is the simplest
audio device that exists on real machines and in QEMU both. Intel HDA is what
the laptop actually has and is a much larger driver; writing it against an
interface nothing had ever played through would have meant debugging the
driver and the interface simultaneously. HDA arrives as a second
struct audio_ops and nothing above it changes -- which is the entire point of
the class.
Two things the wiring needed: PCI class 0x04 now maps to DEVCLASS_AUDIO (the
driver bound and /proc/devices showed it, but audio_get() found nothing,
because the class byte had never been translated), and include/x86.h grew
inw() -- AC'97's mixer is all 16-bit registers and reading one a byte at a
time returns the low half twice on some codecs.
**Tested by measuring what came out, not by not crashing.** QEMU's wav backend
writes the guest's audio to a file; `tone 440 2` produced 1.97 seconds of
44.1 kHz stereo with peak amplitude 14177 against the generator's table
maximum of 14178, and 880 zero crossings per second -- 440 Hz exactly. wav is
on every QEMU target rather than a host backend, because a build machine may
have no sound card and it is the only backend whose output can be checked.
Both supported configurations: one boot each, no traps, no panics, socktest
and memfdtest green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`jmp mpenter`.
x86-64 requires rsp to be 16-byte aligned immediately before a `call`, so a
function starts with rsp ending in 8 -- the return address -- and its
`push %rbp; mov %rsp,%rbp` prologue leaves %rbp 16-byte aligned. Compilers
build on that: an aligned vector store to a frame slot is `movaps`, and
`movaps` to a misaligned address is #GP.
entry64mp loaded the AP's stack pointer, which is page-aligned, and *jumped*.
Nothing pushed a return address, so %rbp came out eight bytes off -- on every
frame, on every application processor, for the life of the machine.
It cost nothing for as long as no function an AP ran needed an aligned vector
store. Then the console's glyph path started returning a small struct by value
(the cell-as-a-surface change), clang spilled it with movaps, and CPU 1 took a
#GP the instant it announced itself on a verbose boot:
unexpected trap 13 from cpu 1 rip ffff800000120a3b <draw_glyph_px+0x1cb>
120a37: 0f 10 45 90 movups -0x70(%rbp), %xmm0
120a3b: 0f 29 45 b0 movaps %xmm0, -0x50(%rbp) <- #GP
movups on the line above loads fine; movaps stores and faults. The
displacement is a multiple of 16, so what is misaligned is %rbp itself, which
is a fact about how the function was entered.
`call mpenter`, not `jmp`. Eight bytes for an ABI the whole toolchain assumes.
**It was hiding behind a second bug.** The first report was not a #GP at all:
Stop code ACQUIRE
#2 cprintf+0xae
#3 trap+0x2c4
trap() reported the fault with cprintf(), cprintf() takes the console lock,
and the faulting code was already holding it -- on a verbose boot every
character is drawn under that lock. acquire() panicked about a lock its own
call chain owned, and the trap number, the faulting address and the registers
were all lost. A kernel-mode trap ends in panic() a dozen lines later anyway,
so the console goes into panic mode *before* the report rather than during it.
**And a third, in the same boot.** The console reported "JetBrains Mono, 8x16
cells" -- a name from one font and metrics from another. The built-in bitmap
became a provider like any other in the previous commit, so draw_glyph_px()
installs it lazily on the first glyph; on a verbose boot that happens before
gfx_font_init() runs, which then returned early because a font was already
installed. The console spent the whole boot in the fallback while claiming not
to be. "A font is installed" and "a typeface was chosen" had been the same test
and quietly stopped being; they are two flags now.
Verified: verbose boot reaches a shell twice running with no traps and no
panics, and reports 10x21 cells -- the real typeface -- where it reported 8x16
before. Both supported configurations quiet: one boot each, no traps, no
panics, socktest and memfdtest green.
Written up as section 22 of docs/history/kernel.md, because the interesting
part is not the missing eight bytes but that three separate faults had to be
peeled off in order before the first one could be seen.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Gate 0's last item, "delete the cell grid" -- done as far as it should be, and
the rest of it retired, which is the part worth explaining.
include/text.h made the case against the grid in four charges: it cannot do
variable-width glyphs, cannot position text off a multiple of eight, cannot
anti-alias, and cannot hold a codepoint above 255. Every one of those was
about the *drawing layer*, and every one is now false -- glyphs carry metrics,
PSOGM positions text at arbitrary pixels, the TrueType path blends 8-bit
coverage, and vtcell.ch is a uint32.
What is left is a grid in vt.c, and deleting that would not modernise the
console; it would stop it being a terminal. VT100 escapes address columns,
TIOCGWINSZ reports rows and columns, and every program that draws a box
assumes both. So the item is checked with its remainder rewritten to say what
was actually wrong, rather than ticked as if the original wording had been
carried out.
**What was actually wrong was the duplicate.** gfx.c walked font_data and
blitted glyphs its own way; text.c did the same job into a struct surface, for
no caller but a sysrq self-test. Two implementations of bearings, coverage and
clipping, which had to agree about a cell size only one of them was told had
changed -- and they did not agree, which is why every line of PSOGM once drew
the next character's bitmap underneath itself.
There is one now, and the mechanism is the pleasing part: the console hands
text_draw_glyph() **the cell as a surface of its own** -- same pixels, same
stride, origin moved and bounds set to that cell. Clipping a tall ascender to
its line is then the same bounds check that clips to the edge of a screen, and
there is no second notion of clipping in the tree. `surface_sub()` is six
lines and replaced fourteen of hand-written comparisons.
The bitmap is reached through font_builtin() like any other provider, so the
early console and the real one are one path with different metrics, and the
`if (!cfont)` branch that walked the array is gone. gfx.c no longer includes
font.h; text.c does, being the only file left that reads it -- which also
retires the note in text.c explaining why it could not.
struct surface gained a bpp, because gfx_draw_pixel handles 16, 24 and 32 and
whatever replaces it has to as well. Zero means 32: a UEFI GOP has no other
format and the loader publishes 32 unconditionally, so callers that do not
care say nothing.
One behaviour change, and it is the documented one: a codepoint the font
cannot draw is now a substituted glyph rather than a blank cell. text.h argues
for that directly -- "a missing glyph that draws nothing makes text silently
shorter and the fault impossible to see".
Verified by screendump: the console renders correctly, PSOGM renders
correctly, and `echo t > /proc/sysrq` still draws its specimen through the
surface path with the back buffer at 32bpp. Both supported configurations: one
boot each, no traps, no panics, socktest and memfdtest green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Gate 0's last-but-one open item. Drop a TrueType file at \schos\console.ttf on
the ESP and the console is drawn with it; with no file, the face linked into
the kernel image, which is what an early panic needs and is not a degraded
mode.
**The loader reads it, and that is the decision worth writing down.** The
kernel opening the file once the root filesystem is mounted is the obvious
design. It is also wrong: the console has been drawing since the kernel's
first line, so a font arriving later changes the cell size, which changes the
column count, which re-lays out every terminal -- and blanks them. The kernel
log would vanish off the screen at the moment the root filesystem mounted,
which is exactly the log a machine being brought up for the first time is
being read for. Since the next thing on the list is real hardware, losing that
was not a trade worth making.
So the choice happens before anything is drawn. Three details make it cheap:
- **EfiReservedMemoryType**, not EfiLoaderData. The E820 handoff turns
loader data into usable memory, on the argument that the loader's own
pages are the kernel's the moment it runs -- true of the loader, not true
of a buffer it wants the kernel to keep reading from.
- **Below 4 GB**, because the kernel reaches this through its direct map and
that is how far the direct map goes. A font above it would need an ioremap
the console cannot do before there is a console.
- **Used in place.** No copy, which matters because the kernel has no
contiguous allocator to copy 112 KB into -- the only one is the DMA arena,
and that belongs to devices.
Sized without EFI_FILE_INFO and its GUID: seek past the end and ask where that
landed, which the specification requires to clamp.
Every failure falls back and keeps going -- no file, implausible size, no
memory, short read, unparseable file. The last says so in the log, because a
font that silently did not load looks exactly like a font that did.
Verified by booting with JetBrainsMono-ExtraBold.ttf on the ESP and
screendumping the console: visibly the bold face, against the Regular of a
boot with no file. Three boots each way, no traps and no panics, socktest and
memfdtest green in all six.
Nothing ships a console.ttf: the file would be the file already in the kernel,
and two copies of a font are two things that can disagree.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two things, both asked for.
**The crash report no longer says which monitors it reached.** "The report is
on the virtio-gpu display too" was a line about the crash screen printed on the
crash screen. The report says what happened and where to read it again; that is
not part of it.
**And the second head now shows the F8 menu and the boot.** It used to be black
until vgpud started several seconds into userspace -- through the splash,
through the menu, through every boot message -- which from the outside is
indistinguishable from a display that does not work. Three stages now cover the
whole boot and hand over to each other without coordinating:
*The loader.* LocateProtocol() answers with the first GOP and offers no way to
ask for the rest, so the rest are found by handle and fed the primary's
framebuffer as a Blt source at the points where a screen is finished: the menu,
the resolution chooser, system information, the splash, each segment of the
kernel load. Blt rather than a second framebuffer to draw into, because a
virtio GOP does not have one -- the same fact that made a virtio-only machine
reboot-loop two commits ago.
*The kernel.* gfx remembers which rows were drawn, as one span rather than a
list of rectangles, because console output is lines and a span is what lines
make. display_mirror_console() copies that span across: a line of boot text is
one cell-row transfer, not a screen. Primed once after pci_init(), because a
display that has just appeared has a black scanout and the console it should be
showing was drawn by the loader into a framebuffer it knows nothing about.
*Then vgpud*, from the same console framebuffer, unchanged.
**Two bugs found building it, both worth the retelling.**
The mirror ran per character from inside vt_feed -- which is inside the console
lock -- and vga_disp_acquire() announces its back buffer with a cprintf. A CPU
taking a lock it already holds, with a call chain that said exactly that. It
runs at the end of consolewrite() and cprintf() now, outside the lock, which is
also the better batching: one transfer per write instead of one per character.
And "is this the console's own screen" was asked of the pointer acquire()
returned. vga_fb hands back a back buffer it copies onto the console's
framebuffer afterwards, so on a one-screen machine the answer was "a different
screen" -- and the console was mirrored onto itself through two megabytes of
intermediate, allocated for the purpose. display_ops grew scanout_phys, which
is a fact about the hardware rather than a policy a driver can get subtly
wrong, and it is asked first: it has no side effects and acquire() has two.
Verified with two heads under QEMU, screendumping the virtio head every 400 ms
from power-on: splash, splash, splash, then the console, and it stays. The F8
menu appears on it; choosing 1024x768 there brings both heads up at 1024x768
byte-identical. A panic still reaches both. Both supported configurations: one
boot each, no traps, no panics, socktest and memfdtest green.
One gap remains and is documented rather than hidden: while the kernel's driver
resets the device during probe, QEMU shows "Display output is not active" for a
fraction of a second. A device being taken over has no scanout until the new
driver sets one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
"it also doesn't show crash screen on virtio gpu or just other gpu than default
vga" -- true, and on the virtio-gpu it was worse than nothing. The head kept
showing the console from a second before the panic: one screen saying the
kernel is dead, the other showing a shell prompt.
panic() painted into the console framebuffer and stopped there. Two cases were
missing, and they are not the same problem.
**The kernel still drives it.** display_acquire() hands back a buffer; the
report is copied in and presented. Ordinary, and a no-op when the display
class's buffer *is* the console's, so a one-screen machine does not do the work
twice.
**A process drives it.** The hard one, and the one hybrid.md owed an answer
for. The kernel let go of the device deliberately, and the process holding it
will never run again -- there is no scheduling after a panic -- so there is
nobody left to ask. It is not a race that can be won by being careful.
So the userspace driver says where it is scanning out *in advance*:
PCIIOC_PANICFB, registered by vgpud after bring-up and after every modeset. On
a panic the kernel paints the report into that memory and asks the kernel
driver the device was taken from to put it on screen one last time --
`panic_present` in struct pci_driver, which virtio-gpu implements by resetting
the device out from under a driver that is still nominally running. That is
safe here and nowhere else, because nothing is going to run again.
The panic path allocates nothing. It cannot: a corrupted heap is one of the
ways a machine gets here, and an allocator that deadlocks takes the crash
screen *and* the serial log's last chance to be read. So the driver keeps its
BAR mappings across the handover, rebuilds one queue from 20 KB of static
memory, and bounds every spin -- including the reset acknowledge, which is an
unbounded `while` in probe() and correct there.
**The registration is checked like the promise it is.** The range must lie
inside a block that process got from PCIIOC_DMA on that device. Without that,
a claimed device would be a way to nominate any physical address in the machine
-- page tables, kernel text -- to be overwritten with four megabytes of crash
report at the one moment nothing is left to notice. That is a worse bug than
the one this fixes. It goes away with the claim and with the process.
What it buys is worth stating plainly: a device can be evicted to userspace
without the kernel losing its ability to tell you the machine died. The
alternative was a driver boundary that is also a blind spot.
Verified both ways under QEMU with two heads, `echo c > /proc/sysrq`, and a
screendump of each over QMP:
vgpud holding the device both heads byte-identical, serial says
"The report is on the virtio-gpu display too."
kernel holding it same, via display_acquire/display_present
(/proc/pci shows 1af4:1050 [virtio-gpu])
Both supported configurations: one boot each, no traps, no panics, socktest and
memfdtest green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Gate 0's last structural row. kernel/core/kmem.c: slabs are pages from
kalloc() with a header at the front, and free objects hold the free list in
their own first eight bytes.
That last detail is the whole design. **An object carries no header and
freeing performs no search**: the page base of any object *is* its slab, and
the slab names its cache. It is also exactly why an object cannot exceed a
page, which matters below.
Two interfaces. kmem_cache_create/alloc/free for a subsystem with one kind of
thing -- it gets its own line in /proc/slab, which is most of why naming it is
worth the argument -- and kmalloc/kmfree over power-of-two classes from 16 to
2048 for code with no single type to build a cache around. Each cache keeps
partial and full lists; a slab that empties goes straight back to kalloc()
rather than being kept, so /proc/slab never overstates what is held.
`make kmem-test` asserts the bookkeeping on the build machine. Same arrangement
as schfs-objtest, and for the same reason: slab lists, free-list threading, the
page-to-cache lookup and whether pages actually come back are all things that
can be wrong without a kernel to be wrong in. 26 checks, including one that a
permissive allocator cannot pass -- write a distinct pattern through 512
objects and read them all back, because two objects sharing storage shows up
there and nowhere else.
One bug the test found immediately: kmem_max_obj() returned the raw page
remainder, which kmem_cache_create() then rounded *up* past its own limit. The
largest legal size was reported as illegal.
**It does not do what the register said it would.** The claim was that every
fixed table exists to work around kalloc. Measured:
struct proc 10656 NPROC 128 larger than a page
struct socket 4928 NSOCK 32 larger than a page
struct buf 4272 NBUF 256 larger than a page
struct inode 280 NINODE 200 a cache, bounded on purpose
struct device 120 NDEVICE 32 3.8 KB, never hit
struct file 48 NFILE 512 24 KB, never hit
The three largest cannot go in a page-backed slab whatever it does. NINODE is
a cache and is meant to be bounded. NFILE and NDEVICE are small enough that
their tables are rounding errors and their ceilings have never been reached.
Freeing the big ones needs a contiguous multi-page allocator, which is a
different thing and is not written. That is now in the register and in
docs/architecture/kernel.md rather than left as a claim nobody checked.
What it does fix is real, and was measured rather than assumed:
- **A pipe is 640 bytes and came from a whole 4096-byte page.** Every `|` in
a shell command burned three and a half kilobytes it never touched. Six
pipes per page now, and /proc/slab shows the page going back when they
close.
- **procread() built its answer in a 2048-byte array on the kernel stack**,
under a comment that had warned about exactly that since it was written --
it stayed a local because a whole page for 2 KB felt worse, and kmalloc is
the answer that was missing. Its frame is 64 bytes now instead of ~2100.
- procfs's other generator took a page from kalloc for the same 2 KB.
Verified on a boot: /proc/slab tracks pipe and kmalloc-2048 through real
pipelines, allocs and frees balance, and $(), ps, time, readlink, sigtest and
rename are all still green with no traps.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
