nexoniarz/SchOS
nexoniarz/SchOS
Watch
1
0
Fork
You've already forked SchOS
 0
Code Issues Pull requests Projects Releases 1 Packages Wiki Activity Actions

ldls: a second driver, and everything it found

Browse source 
e1000 -- Linux's, unmodified -- probes, opens, gets an address by DHCP and
carries ping traffic in a SchOS process. The kernel was driving that NIC and
gave it up, which makes it the first real handover as well as the second
driver.

None of what it needed was a missing feature. Every one was a call that
succeeded, returned zero and did nothing:

  _find_next_bit          "bit 0 is set"          -- for_each_cpu never ended
  netif_carrier_off       carrier already up      -- TCTL.EN never set
  __x86_indirect_thunk_r8 the jump discarded      -- 255 rx buffers, none made
  dma_map_page_attrs      bus address 0           -- NIC aimed at physical 0

so the rule from memcpy is now general: a stub returning zero is unsafe
wherever zero is a meaningful answer.

The loader grew ko_fixup_percpu(). The stack guard turned out to be one
per-CPU variable among several; `current` cannot be defused because the driver
dereferences it. Taking the %gs off instead leaves the same opcode reading the
same width from storage the host supplies -- a per-CPU variable in a
single-threaded host is just a variable. Eight bytes become seven and a nop,
and the displacement is not always last, so there is now just enough of an x86
decoder in ko.c to know how long the instruction is.

ldls/kpi/thunk.c is fifteen lines of assembly and the highest-value file here:
retpolines are control flow, and a stub cannot stand in for a jump.

The kernel side:

  - ioperm.c, the TSS I/O permission bitmap. A driver's outl is an instruction,
    not a call, and e1000's reset path picks it deliberately. IOPL=3 would hand
    over the PIC and the disk with it, so it is per-port. Writing it found that
    the I/O map base had been written at offset 64 rather than 102, leaving the
    CPU checking ports 0-831 against the TSS's own RSP0.
  - claim masks the device's IRQ line: 450,000 a second, otherwise.
  - net_detach(), the missing half of net_attach_late(). An interface left in
    the stack keeps its route and sends every frame to a driver that refuses
    it, which looks exactly like a userspace driver that does not work.
  - e1000_release() gives up the interface, not just the hardware.

8139too still passes 4/4; ldls-check still passes 27; four CPUs, no panic.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2
This commit is contained in:
Nexoniarz 2026-09-14 15:20:07 +02:00
commit 071499a378
26 changed files with 2009 additions and 46 deletions
Download patch file Download diff file Expand all files Collapse all files

165
kernel/arch/x86_64/ioperm.c Normal file
View file

@ -0,0 +1,165 @@
// The TSS I/O permission bitmap. See include/ioperm.h for why.
#include "types.h"
#include "defs.h"
#include "param.h"
#include "mmu.h"
#include "proc.h"
#include "ioperm.h"
// The 64-bit TSS, written out rather than indexed.
//
// The old code reached it as `uint *tss` and set `tss[16]` to put 0x68 in the
// I/O map base -- but the I/O map base is at offset 102, which is the high half
// of the dword at index 25, not 16. Index 16 is offset 64, in the middle of the
// interrupt stack table.
//
// That was not harmless. The field it meant to write stayed 0, and a base of 0
// with a limit of 0x67 does not mean "no bitmap" -- it means the bitmap starts
// at the top of the TSS, so the CPU checked ports 0-831 against the TSS's own
// bytes. Those bytes are RSP0: a kernel stack pointer, whose zero bits are
// *permission*. Some low ports were reachable from user mode and had been
// since the kernel was 64-bit, which nothing had any reason to notice.
//
// Naming the fields is what makes that visible, so they are named.
struct taskstate64 {
uint32 res0;
uint64 rsp0;
uint64 rsp1;
uint64 rsp2;
uint64 res1;
uint64 ist[7];
uint64 res2;
uint16 res3;
uint16 iomb; // offset from the base of the TSS
} __attribute__((packed));
#define IOMAP_BITS 65536
#define IOMAP_BYTES (IOMAP_BITS / 8)
// The TSS and its bitmap have to be one object: `iomb` is a 16-bit offset from
// the TSS base, and the TSS descriptor's limit is what cuts the bitmap off.
// So they are declared together and there is no way for them to drift apart.
//
// 8 KB per CPU of permanently resident kernel memory is the cost of being able
// to say "these ports and no others", and it is charged whether or not any
// process ever uses it. A bitmap sized to the highest port actually granted
// would be smaller and would have to be resized at runtime, under a descriptor
// the CPU is using; this is the version that is obviously correct.
struct tssblk {
struct taskstate64 ts;
uint8 map[IOMAP_BYTES];
// Intel requires a byte of all-ones past the end, so an access that straddles
// the final port still reads a deny bit rather than whatever follows.
uint8 tail;
} __attribute__((packed, aligned(16)));
static struct tssblk tssblk[NCPU];
// What each CPU currently has open. Kept per CPU rather than read back from the
// process, because switching *away* has to close what was opened and the
// process that opened it may be part-way through exit() by then.
static struct {
struct ioport_range r[NIOPORT];
int n;
} installed[NCPU];
static void
map_set(uint8 *map, uint16 base, uint16 len, int allow)
{
uint32 p, end = (uint32)base + len;
if (end > IOMAP_BITS)
end = IOMAP_BITS;
for (p = base; p < end; p++) {
if (allow)
map[p >> 3] &= ~(1 << (p & 7));
else
map[p >> 3] |= (1 << (p & 7));
}
}
void *
ioperm_tss(int cpu, uint64 *limit)
{
struct tssblk *b = &tssblk[cpu];
// Deny everything to start with. A zero bit is permission, so a zeroed
// bitmap is an open machine -- this memset is not initialisation tidiness.
memset(b->map, 0xFF, sizeof(b->map));
b->tail = 0xFF;
b->ts.iomb = (uint16)__builtin_offsetof(struct tssblk, map);
installed[cpu].n = 0;
// The limit is the offset of the last valid byte, and it has to include the
// trailing 0xFF -- stopping one byte short is the difference between a
// straddling access being denied and it reading the next object.
*limit = (uint64)__builtin_offsetof(struct tssblk, tail);
return b;
}
// The same block, without the reset. seginit() calls ioperm_tss() once per CPU
// to build it; everything after that just wants the pointer.
void *
ioperm_tss_of(int cpu)
{
return &tssblk[cpu];
}
void
ioperm_set_rsp0(void *tss, addr_t stktop)
{
((struct taskstate64 *)tss)->rsp0 = (uint64)stktop;
}
void
ioperm_grant(struct proc *p, uint16 base, uint16 len)
{
int i;
if (len == 0)
return;
for (i = 0; i < p->nioport; i++)
if (p->ioport[i].base == base && p->ioport[i].len == len)
return;
if (p->nioport >= NIOPORT)
return; // refused quietly; the driver will fault loudly
p->ioport[p->nioport].base = base;
p->ioport[p->nioport].len = len;
p->nioport++;
}
void
ioperm_revoke_all(struct proc *p)
{
p->nioport = 0;
// The bitmap is not touched here. This can run on a CPU that is not the one
// holding the process's ranges open, and the next switchuvm() on each CPU
// closes what that CPU opened -- which is the only place the two can be made
// to agree without an IPI.
}
void
ioperm_switch(struct proc *p)
{
int c = cpunum();
struct tssblk *b = &tssblk[c];
int i;
// Close whatever was open. Doing this unconditionally rather than diffing
// against the incoming process costs a few hundred bit clears on a machine
// where nobody uses ports at all -- which is `installed[c].n == 0`, and then
// this loop does nothing.
for (i = 0; i < installed[c].n; i++)
map_set(b->map, installed[c].r[i].base, installed[c].r[i].len, 0);
installed[c].n = 0;
if (!p || p->nioport == 0)
return;
for (i = 0; i < p->nioport && i < NIOPORT; i++) {
map_set(b->map, p->ioport[i].base, p->ioport[i].len, 1);
installed[c].r[i] = p->ioport[i];
}
installed[c].n = i;
}

25
kernel/arch/x86_64/vm.c
View file

@ -6,6 +6,7 @@
#include "memlayout.h"
#include "mmu.h"
#include "proc.h"
#include "ioperm.h"
int mappages(pml4e_t *pgdir, void *va, addr_t size, addr_t pa, int perm);
static pte_t * walkpgdir(pml4e_t *pml4, const void *va, int alloc);
@ -68,7 +69,7 @@ seginit(void)
{
struct segdesc *gdt;
uint *tss;
uint64 addr;
uint64 addr, tsslim;
void *local;
struct cpu *c;
@ -91,8 +92,12 @@ seginit(void)
}
gdt = (struct segdesc*) local;
tss = (uint*) (((char*) local) + 1024);
tss[16] = 0x00680000;
// The TSS moved out of this page. It now carries an I/O permission bitmap
// (include/ioperm.h) which does not fit here, and it has to be one object
// with it. ioperm_tss() returns it and the descriptor limit that reaches the
// end of the bitmap.
tss = (uint*) ioperm_tss(cpunum(), &tsslim);
kernel_fs_base = ((uint64) local) + 2048;
wrmsr(0xC0000100, kernel_fs_base);
@ -157,14 +162,14 @@ seginit(void)
gdt[SEG_KCPU] = (struct segdesc) {0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
struct tassdesc *td = (struct tassdesc*)(gdt + SEG_TSS);
td->lim_15_0 = 0x67;
td->lim_15_0 = tsslim & 0xffff;
td->base_15_0 = addr & 0xffff;
td->base_23_16 = (addr >> 16) & 0xff;
td->type = STS_T64A;
td->s = 0;
td->dpl = 0;
td->p = 1;
td->lim_19_16 = 0;
td->lim_19_16 = (tsslim >> 16) & 0xf;
td->avl = 0;
td->l = 0;
td->d = 0;
@ -300,10 +305,14 @@ switchuvm(struct proc *p)
pushcli();
if(p->pgdir == 0)
panic("switchuvm: no pgdir");
uint *tss = (uint*) (((char*) mycpu()->local) + 1024);
void *tss = ioperm_tss_of(cpunum());
const addr_t stktop = (addr_t)p->kstack + KSTACKSIZE;
tss[1] = (uint)stktop;
tss[2] = (uint)(stktop >> 32);
ioperm_set_rsp0(tss, stktop);
// Which ports this process may reach. Cheap when nobody uses any, which is
// every process that is not hosting a driver.
ioperm_switch(p);
lcr3(v2p(p->pgdir));
popcli();
}

66
kernel/dev/pciuser.c
View file

@ -24,6 +24,7 @@
#include "memlayout.h"
#include "mmu.h"
#include "proc.h"
#include "ioperm.h"
#include "pci.h"
#include "errno.h"
#include "uaccess.h"
@ -342,6 +343,10 @@ pciuser_release_all(struct proc *p)
pciuser_stop(i);
}
// And its ports. The bitmap itself is closed by the next switchuvm() on each
// CPU; see ioperm_revoke_all().
ioperm_revoke_all(p);
acquire(&pciuser_lock);
// And its interrupt line -- unregistered, and left MASKED.
@ -485,6 +490,56 @@ do_claim(uint64 argp)
// it back. The two are a pair, and the second driver to run is the one that
// finds out if they are not.
pci_enable((struct pci_dev *)d, PCI_CMD_MEMORY | PCI_CMD_BUS_MASTER);
// And its interrupt line: masked.
//
// The kernel driver that just let go had a handler on this line and no
// longer does, but the device is about to be brought up by a driver in a
// process and *will* assert it. With nothing to acknowledge the cause at the
// device, every EOI re-enters an empty dispatch immediately:
//
// irq 11: 449905 interrupts/second -- a device on this line is not
// acknowledging it.
//
// which is the machine spending all of its time noticing. The line comes
// back when the process asks for it with PCIIOC_IRQ, which is the same
// arrangement pciuser_release_all() leaves behind and for the same reason.
if (d->irq != 0xFF && d->irq < 24) {
ioapicmask(d->irq);
acquire(&pciuser_lock);
if (irqdev.proc == 0)
irqdev.masked = 1;
release(&pciuser_lock);
}
// And the ports, if it has any.
//
// A Linux driver does not always go through a function we can intercept:
// e1000's reset path issues `out` directly, because on an 82540 that is the
// only way the chip will acknowledge the write. So a process holding a
// device gets that device's I/O BARs and nothing else -- see
// include/ioperm.h. PCI_CMD_IO goes with them for the same reason memory
// space goes with the mapping above.
{
int granted = 0;
for (int b = 0; b < PCIUSER_MAXBAR; b++) {
if (!d->bar[b].is_io || !d->bar[b].base || !d->bar[b].size)
continue;
if (d->bar[b].base > 0xffff || d->bar[b].base + d->bar[b].size > 0x10000)
continue; // above the port space; nothing to grant
ioperm_grant(myproc(), (uint16)d->bar[b].base, (uint16)d->bar[b].size);
granted = 1;
}
if (granted) {
pci_enable((struct pci_dev *)d, PCI_CMD_IO);
// Install them on this CPU now, rather than at the next switchuvm().
// The process asked for them in a system call and returns to user mode
// from here, which need not go through the scheduler at all.
pushcli();
ioperm_switch(myproc());
popcli();
}
}
return 0;
}
@ -507,8 +562,17 @@ do_release(uint64 argp)
// politely has usually quiesced its device already, but "usually" is not a
// property the kernel can check, and the cost of being sure is one config
// write.
if (mine)
if (mine) {
pciuser_stop(idx);
// Ports go back too. Coarse on purpose: this drops *every* range the
// process had rather than only this device's, and the next claim puts
// them back. A process releasing one of several devices is not a case
// that exists yet, and a revoke that is too wide fails safe.
ioperm_revoke_all(myproc());
pushcli();
ioperm_switch(myproc());
popcli();
}
return 0;
}

63
kernel/drivers/net/e1000/e1000.c
View file

@ -49,6 +49,11 @@ struct e1000 {
// the two calls are separate, and nothing else connects them.
struct pci_dev *pdev;
// And the network device the stack knows it by, so that letting go of the
// hardware can also take the interface out of the stack. Set by attached(),
// which is where the two are first introduced.
struct device *ndev;
struct spinlock lock;
};
@ -387,6 +392,62 @@ e1000_link_up(struct device *dev)
return e && e->ready && (rd(e, E1000_STATUS) & E1000_STATUS_LU) != 0;
}
// Let go of the device, so a userspace driver can have it.
//
// The same hook virtio-gpu grew for H1, on a NIC. docs/plan/hybrid.md's rule
// is that the kernel keeps a minimal driver for everything on the path to a
// shell -- and a network interface is not on that path, so this one may leave.
//
// "Letting go" is more than saying yes. The device is still bus-mastering into
// descriptor rings this driver allocated, and the process about to claim it
// will be handed those pages back by the arena. So the receiver and the
// transmitter are switched off and the interrupt is masked *at the device*
// before the handler is unregistered: masking after unregistering leaves a
// window where a level-triggered line is asserted with nobody to clear it,
// which is the failure docs/history/kernel.md section 13 is about.
//
// `ready` going false is what makes the class vtable safe afterwards --
// transmit, receive and link_up all check it, so the stack sees an interface
// whose link is down rather than one that swallows packets.
static int
e1000_release(struct pci_dev *d)
{
struct e1000 *e = 0;
// Found by the device it is bound to. struct pci_dev carries no private
// pointer -- this driver keeps its instances in a table and matches on
// `pdev`, the same way e1000_attached() does.
for (int i = 0; i < E1000_MAX_NICS; i++)
if (nics[i].pdev == d) { e = &nics[i]; break; }
if (!e || !e->ready)
return 0; // never came up, or already let go
e->ready = 0;
wr(e, E1000_IMC, 0xFFFFFFFF); // no more interrupts from it
wr(e, E1000_RCTL, 0); // stop receiving
wr(e, E1000_TCTL, 0); // stop transmitting
rd(e, E1000_STATUS); // post the writes
if (d->irq != 0xFF)
irq_unregister(d->irq, e1000_intr, e);
// And take the interface out of the stack.
//
// Leaving it there is worse than it sounds. The netif keeps its address and
// the route to its subnet, so the stack goes on choosing it -- and every
// frame it sends reaches e1000_transmit, which now correctly refuses. The
// packets stop and nothing reports an error, which is indistinguishable
// from a driver in a process that does not work. It was mistaken for
// exactly that for an afternoon.
if (e->ndev)
net_detach(e->ndev);
cprintf("e1000: released to userspace; the interface is down\n");
return 0;
}
// Hang the instance on the device the bus has just published. See the comment
// on `attached` in include/pci.h for why this cannot happen inside probe().
static void
@ -395,6 +456,7 @@ e1000_attached(struct pci_dev *d, struct device *dev)
for (int i = 0; i < E1000_MAX_NICS; i++)
if (nics[i].ready && nics[i].pdev == d) {
dev->priv = &nics[i];
nics[i].ndev = dev;
return;
}
}
@ -414,4 +476,5 @@ const struct pci_driver e1000_driver = {
.ops = &e1000_netdev_ops,
.probe = e1000_probe,
.attached = e1000_attached,
.release = e1000_release,
};

46
kernel/net/lwip/netif_eth.c
View file

@ -138,6 +138,52 @@ net_eth_attach(struct device *dev)
return nifs++;
}
// An interface whose device has gone away.
//
// The missing half of net_eth_attach, and it went missing because nothing had
// ever taken a device back. /dev/pci does: the kernel's e1000 driver hands the
// NIC to a process, and until this existed the interface stayed in lwIP --
// up, holding an address, holding the route to its subnet, and attached to a
// driver that now refuses every frame. Packets went to it and stopped, which
// looks exactly like a driver that is not working.
//
// The slot is left occupied rather than compacted. Interface indices are
// visible in /proc/netif and in the names the stack has already handed out,
// and renumbering them because a device left would be worse than a gap.
int
net_eth_detach(struct device *dev)
{
int i;
for (i = 0; i < nifs; i++) {
if (!ifs[i].used || ifs[i].dev != dev)
continue;
// Down first, then out. netif_remove() does stop DHCP and send the
// release, but a netif removed while still up leaves lwIP's default
// route pointing at freed storage if this was interface 0.
netif_set_link_down(&ifs[i].netif);
netif_set_down(&ifs[i].netif);
if (netif_default == &ifs[i].netif)
netif_set_default(NULL);
netif_remove(&ifs[i].netif);
ifs[i].used = 0;
ifs[i].dev = 0;
// Something else becomes the default, if anything can. A machine with
// a second interface should not lose its route because the first one
// was handed to a driver in a process.
if (netif_default == NULL) {
int j;
for (j = 0; j < nifs; j++)
if (ifs[j].used) { netif_set_default(&ifs[j].netif); break; }
}
return i;
}
return -1;
}
int
net_eth_count(void)
{

46
kernel/net/net.c
View file

@ -208,6 +208,42 @@ net_attach_late(struct device *d)
return idx;
}
// Which interfaces have had their address reported. File scope rather than
// local to net_thread(), because an interface that goes away has to be able to
// be announced again if one comes back in its slot.
static int announced[NETIF_MAX];
static void
net_unannounce(int idx)
{
if (idx >= 0 && idx < NETIF_MAX)
announced[idx] = 0;
}
// The other direction: a device the kernel no longer drives.
//
// Called when /dev/pci hands a NIC to a process. Same rule about lwIP headers
// as net_attach_late above -- the caller is a device file and must not see
// them.
int
net_detach(struct device *d)
{
int idx;
if (!net_ready)
return -1;
lwip_sys_lock();
idx = net_eth_detach(d);
lwip_sys_unlock();
if (idx >= 0) {
net_unannounce(idx);
cprintf("net: %s is gone; interface %d removed\n", d->name, idx);
}
return idx;
}
static void
net_thread(void)
{
@ -253,7 +289,7 @@ net_thread(void)
attached, attached == 1 ? "" : "s");
// One bit per interface, so a second one binding later is announced too.
int announced[NETIF_MAX] = { 0 };
for (;;) {
int moved;
@ -269,8 +305,14 @@ net_thread(void)
// Report each address once, when it first arrives. Anyone who has spent
// an evening wondering whether the guest has an address at all wants
// this on the console rather than in a file they have to know about.
for (int i = 0; i < attached; i++) {
// Over every interface there is now, not over the ones there were at
// boot: an interface can arrive later. /dev/netdev makes one whenever a
// process becomes a network device, and until this counted them an
// address on `ldls0` was never reported at all.
for (int i = 0; i < net_eth_count() && i < NETIF_MAX; i++) {
struct netif *nif = net_eth_netif(i);
if (nif == 0 || net_eth_device(i) == 0)
continue;
if (announced[i] || ip4_addr_isany_val(*netif_ip4_addr(nif)))
continue;