lwIP 2.2.0, fetched by ports/lwip and compiled straight into the kernel.
Not written from scratch, and the reason is not the packet formats: it is
congestion control, retransmission, window management and twenty years of
corner cases that only show up against a real peer. Not FreeBSD's
netinet either, though the licence would have allowed it -- that stack is
not a library, it reaches into mbufs, epoch(9), netisr, VNET and uma(9),
so adopting it means porting FreeBSD's kernel infrastructure first. lwIP
was written for exactly this situation: a small kernel with a raw
Ethernet driver and a port layer of a few hundred lines.
It is kernel code, so nothing is installed into the sysroot and there is
no second build system: the top-level Makefile compiles the files it
wants against our port headers. The file list is explicit rather than a
wildcard, because lwIP ships IPv6, 6LoWPAN, PPP, SLIP, DNS-SD and a dozen
apps in the same tree, and building what is not configured yields either
dead code or a link error naming a port function nobody meant to write.
Its sources are unmodified and its warnings are silenced for its files
only -- patching them would mean maintaining a fork.
The port layer is ours: a lock, a clock, the netif that hands frames to
e1000, lwipopts.h, and a shim/ directory of tiny headers so lwIP's
<string.h> and friends resolve to the kernel's functions instead of the
host's, whose prototypes disagree and would otherwise be found on the
default include path.
NO_SYS=1, so lwIP has no threads of its own and one kernel thread feeds
it. A thread rather than the interrupt handler, because a call into lwIP
can reach the driver, the driver can sleep, and an interrupt handler
cannot -- so the interrupt still only clears the cause register and the
thread polls. kthread_create is new and generally useful: a process with
a kernel-only page directory, no files, no cwd, no parent, dispatched
from forkret, which can therefore sleep and be preempted like anything
else. DHCP obtains an address at boot and the console says which one.
AF_INET went in exactly where phase 2 said it would. socknew dispatches
on the domain, kernel/net/inet.c implements the internet side on lwIP's
raw API, and not one syscall changed shape: bind, connect, listen,
accept, read and write reach the same descriptors they always did. TCP
and UDP both work. cmd/nettest opens a connection, writes, and reads the
answer back.
Two things are owed and named here so they are not discovered later:
recvfrom does not yet report the source address, and the socket paths
poll on the timer tick instead of sleeping on the socket. Both are
follow-ups, not redesigns.
NOTICE carries lwIP's BSD-3 text. Clause 3 is no-endorsement, not the
old advertising clause, so it sits with Apache-2.0 without difficulty.
Verified: the tree builds clean, and the headless boot check passes with
27 [ok] and no failures with the stack compiled in and the thread
running -- which is the part that would break first if netinit deadlocked
or the thread starved anything. An end-to-end connection test belongs in
that suite and is the next commit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
struct file already carried a uint64 offset and bread already took a
uint64 block number. The narrowing was one boundary in the middle:
devsw's read and write took a 32-bit uint, so fileread passed a 64-bit
f->off into a 32-bit parameter and the top half was discarded.
The effect was not a refusal at 4 GB but silent wraparound past it. A
read at 5 GB returned the bytes at 1 GB, reporting success. disk.c had a
DISK_MAX_BYTES constant documenting the limit, and nothing anywhere
referenced it -- the cap was described but never enforced, which is the
worst of the three options.
The signature is now uint64 across all nine files that carry it, and
diskread/diskwrite no longer narrow the block number on the way to
bread. Widening this is a prerequisite for NVMe and for any real disk:
namespaces start well above 4 GB and the failure mode reads valid data
from the wrong place.
Verified end to end rather than by inspection. A 6 GB sparse image with
"PAST4GB-MARKER-OK" written at byte 5368709120, attached as /dev/hdb to a
ramdisk-root kernel so device 1 was free:
# dd if=/dev/hdb bs=4096 skip=1310720 count=1 | od -c | head -2
0000000 P A S T 4 G B - M A R K E R - O
Under the old code that offset truncates to 1 GB, where the image is
zeroes. The whole path is exercised: f->off, devsw, diskread, bread and
ide.c's LBA48.
Worth recording for next time: `make ramroot` stages the ESP itself, and
running `make esp` afterwards rebuilds the kernel without RAMROOT and
silently undoes it. The tell is a 855 KB kernel where a 25 MB one is
expected.
110 [ok], no failures, on the default build.
Groundwork for the unified loader: before the two paths can share code,
they have to agree on what they are producing, and until now they only
agreed by coincidence.
Every field of the loader/kernel hand-off was #defined in whichever file
happened to need it -- kernel/core/main.c, kernel/dev/vga.c,
kernel/arch/x86_64/e820.c, boot/bios/cdboot.S, boot/bios/stage2main.c and
boot/uefi/main.c, six copies of the same addresses. schboot.h existed for
exactly this and held only the entry magic and the RSDP slot, with a
comment describing the rest of the block it did not define.
The whole block lives there now: the VBE info fields, the verbose and
media flags (with the real-mode segment:offset spelling cdboot.S needs),
the e820 count, map and cap, and the RSDP. Every one of those files
includes it instead of restating it.
This is not a tidy-up. It is the same hazard as defining the entry magic
twice, only quieter -- nothing breaks until one copy moves, and the two
loaders diverging is precisely what produced the EFER.SCE and RSDP bugs,
where a fix on one path silently did not exist on the other.
Verified on both firmwares: BIOS boots to schinit with the same e820,
ACPI and VGA lines as before; UEFI reaches cpu0 with 1018 MB in 7 regions
and its own 1280x800 mode. `make iso` still builds. bootcheck 108 [ok].
Next: the loaders become starters that gather firmware-specific facts and
hand to one common stage. That needs the BIOS side to climb to long mode
first, since common code can only be compiled for one CPU mode -- which
is the risky part, and is why the contract went first.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
e820_init() took the usable run *containing the kernel* and treated its
end as the top of RAM, and kinit1() walked a single range from the end
of the kernel image to there. That is the shape of a BIOS map. It is
not the shape of a UEFI one: OVMF places an ACPI NVS region at 8 MB,
so the kernel's run ended there, the map failed the PHYSTOP_MIN check,
and the kernel fell back to asserting 224 MB it had not measured --
then freed straight through the reserved hole anyway.
Keep the usable regions as a list as well as a top address. kinit1()
now frees each one, clipped to the window main() passes in, so the
holes are skipped rather than either freed or fatal. PHYSTOP still
means what it always did -- the top of what kalloc may hand out, and
the bound kfree() rejects pages above -- so no call site changes.
phystop comes from the largest region rather than the highest. The DMA
arena is carved off the top of PHYSTOP and must be physically
contiguous, and the largest region is the only one certain to have
room; a smaller region above it is left unfreed rather than raising
the bound past a hole, because kfree()'s check is worth more than the
pages it costs. Report the usable total separately from the top
address, since with a fragmented map they are different numbers and
printing one as the other is how 8 MB went unnoticed.
Verified on both paths. BIOS: unchanged at 1023 MB in 2 regions,
bootcheck 27/27. UEFI under OVMF: 1018 MB in 7 regions, up from the
8 MB that triggered the fallback, booting through vga, e1000, the RTC
and on to cpu0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Neither is Fedora-specific. A different host and a freshly built
userland simply drove code that Windows never reached.
tools/mkfs.c did not compile at all. The DT_UNKNOWN/DT_DIR/DT_REG
fallbacks sat above <dirent.h>, so on glibc -- which spells them as
enumerators -- the #ifndef asked before the header was read, always
answered "missing", and then the header's own enum expanded to
`enum { 0 = 0, ... }`. Move the fallbacks below the include. The
include cannot move up instead: it has to follow the #undef that ends
the `#define dirent schfs_dirent` block.
getstackpcs() page-faulted. It dereferenced rbp[1] without validating
rbp, and the guard rejected 0xffffffff -- dead code on x86-64, already
caught by the KERNBASE test -- while the value that actually turns up
is all-ones, which is above KERNBASE. rbp[1] then wrapped to 0x7 and
took trap 14. `cp` panicked the kernel every time. Bound rbp from
above and require 8-byte alignment.
That second one masks other bugs: getcallerpcs() runs on every lock
acquisition and again from the trap handler, so a walker that can
fault replaces a real diagnosis with a bogus one.
Baseline on the Fedora laptop is now green: make ports, make,
make schfs-check including the round-trip that had never run, and
bootcheck at 27/27 plus 108 [ok] across the extended suites.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
boot/bios/ now holds the MBR, the El Torito stub and the real-mode stage
2; boot/uefi/ is empty and documented.
Separate directories rather than #ifdefs, because the two share almost
nothing. A BIOS loader lives in 16-bit real mode, reaches the firmware
through software interrupts, and walks itself up into long mode. A UEFI
loader is a 64-bit PE that the firmware has already placed in long mode
with paging on, calling services through a table of function pointers.
What they do share is the hand-off contract, which boot/README.md now
writes down: ELF segments placed, long mode, a 24-byte-entry memory map
at E820_MAP_ADDR, framebuffer geometry at VBE_INFO_ADDR.
That contract is the point of documenting it now. UEFI's GetMemoryMap
returns EFI_MEMORY_DESCRIPTORs, and the right move is to translate them
into the entries kernel/arch/x86_64/e820.c already parses rather than
teach the kernel a second format.
Build output paths under $(BUILD)/boot are unchanged; only sources moved.
Removes the "-unix" element and with it any trademark exposure to The
Open Group.
The split follows the ecosystem layout: SchOS is the system, and the
kernel has its own name. Kernel-identity strings are the only places
that take SchIFT -- uname(2) sysname, /proc/version, and the two licence
sentences that say "kernel" outright. That is what every Unix does here:
`uname -s` on Linux reports the kernel, not the distribution. Everything
a user reads as the product -- panic screen, boot menu, error prefixes,
docs, LICENSE, NOTICE -- is SchOS.
schunix.img is schos.img, the logo assets and the musl TLS patch are
renamed, and __SCHUNIX__ is __SCHOS__ (the define and the patch that
consumes it moved together).
Two things the rename broke, both found by the bootcheck suite:
- shmtest wrote "SCHUNIX-MMAP-OK" with a hardcoded length of 15 and
compared it back by hand-counted offsets (m[3]=='U', m[14]=='K').
The shorter marker left the write running two bytes past the literal
and the compare reading past the data, failing a file-backed mmap
that was working correctly. Now written and compared through one
constant with sizeof.
- bootcheck asserted a 31-byte file after appending to
"hello-from-schunix"; the shorter string makes it 29.
Both are the same mistake: a string literal whose length is written out
somewhere else. Full suite green -- base, Tty, Ps, Sock, Fb, Usb, Evdev.
An x86-64 operating system built from xv6, now largely rewritten: SMP,
a copy-on-write root filesystem (SchFS), PCI/USB/virtio drivers, a TTF
rasteriser, and a Linux-numbered syscall ABI.
No history before this point -- the tree was never under version control.
This commit exists so the rename that follows has something to undo to.
