ldls: a second driver, and everything it found

e1000 -- Linux's, unmodified -- probes, opens, gets an address by DHCP and
carries ping traffic in a SchOS process. The kernel was driving that NIC and
gave it up, which makes it the first real handover as well as the second
driver.

None of what it needed was a missing feature. Every one was a call that
succeeded, returned zero and did nothing:

  _find_next_bit          "bit 0 is set"          -- for_each_cpu never ended
  netif_carrier_off       carrier already up      -- TCTL.EN never set
  __x86_indirect_thunk_r8 the jump discarded      -- 255 rx buffers, none made
  dma_map_page_attrs      bus address 0           -- NIC aimed at physical 0

so the rule from memcpy is now general: a stub returning zero is unsafe
wherever zero is a meaningful answer.

The loader grew ko_fixup_percpu(). The stack guard turned out to be one
per-CPU variable among several; `current` cannot be defused because the driver
dereferences it. Taking the %gs off instead leaves the same opcode reading the
same width from storage the host supplies -- a per-CPU variable in a
single-threaded host is just a variable. Eight bytes become seven and a nop,
and the displacement is not always last, so there is now just enough of an x86
decoder in ko.c to know how long the instruction is.

ldls/kpi/thunk.c is fifteen lines of assembly and the highest-value file here:
retpolines are control flow, and a stub cannot stand in for a jump.

The kernel side:

  - ioperm.c, the TSS I/O permission bitmap. A driver's outl is an instruction,
    not a call, and e1000's reset path picks it deliberately. IOPL=3 would hand
    over the PIC and the disk with it, so it is per-port. Writing it found that
    the I/O map base had been written at offset 64 rather than 102, leaving the
    CPU checking ports 0-831 against the TSS's own RSP0.
  - claim masks the device's IRQ line: 450,000 a second, otherwise.
  - net_detach(), the missing half of net_attach_late(). An interface left in
    the stack keeps its route and sends every frame to a driver that refuses
    it, which looks exactly like a userspace driver that does not work.
  - e1000_release() gives up the interface, not just the hardware.

8139too still passes 4/4; ldls-check still passes 27; four CPUs, no panic.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VQ7Qx8E1q1M1bbCq8QLHS2

ldls/kpi/skb.c

@@ -23,6 +23,19 @@
 // somewhere to go.
 #define KPI_HEADROOM 64
 
+// Room past `end` for `struct skb_shared_info`.
+//
+// `skb_shinfo(skb)` is `(struct skb_shared_info *)(skb->head + skb->end)` --
+// it is not a field, it lives immediately after the data, and every allocator
+// in Linux reserves space for it. 8139too never looked, so an skb that ended
+// exactly at `end` worked; e1000 reads `nr_frags` on every single transmit,
+// and read it from whatever followed the allocation.
+//
+// The struct is 320 bytes on x86-64 and this is deliberately more. It is not
+// read from BTF only because getting it wrong in the small direction is
+// silent, and a fixed size that is comfortably too large cannot be.
+#define KPI_SHINFO 512
+
 static unsigned int
 rd32(void *base, unsigned int off)
 {
@@ -67,15 +80,26 @@ void *
 kpi_skb_alloc(unsigned int len)
 {
   unsigned char *p;
-  unsigned int total = kpi_l.skb_size + KPI_HEADROOM + len + 64;
+  unsigned int total = kpi_l.skb_size + KPI_HEADROOM + len + 64 + KPI_SHINFO;
   unsigned char *buf;
 
   if (kpi_l.skb_size == 0)
     return 0;
 
-  p = mmap(0, total, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
-  if (p == 0 || (long)p == -1)
+  // From the DMA arena, not from mmap.
+  //
+  // A driver that maps an skb for the device -- e1000 does, 8139too did not --
+  // needs the bus address of its data, and a process can only know that for
+  // memory the kernel handed it through PCIIOC_DMA. Putting the control block
+  // in there too wastes a little arena and keeps one buffer one allocation.
+  p = kpi_dma_heap(total, 64);
+  if (p == 0)
     return 0;
+  {
+    unsigned int i;
+    for (i = 0; i < total; i++)
+      p[i] = 0;                  // the arena is recycled; mmap's zeroes are not
+  }
 
   buf = p + kpi_l.skb_size;
   wrptr(p, kpi_l.skb_head, buf);
@@ -86,6 +110,48 @@ kpi_skb_alloc(unsigned int len)
   return p;
 }
 
+// napi_build_skb(data, frag_len) -- an skb wrapped around a buffer that
+// already exists.
+//
+// The other half of the receive path, and the one that carries anything
+// larger than `copybreak` (256 bytes by default). e1000 hands the device a
+// raw fragment, and when a frame arrives it wraps that same fragment rather
+// than copying it -- so the control block is allocated here and the data is
+// not. A DHCP offer is 590 bytes, which is why nothing larger than a ping
+// reply worked until this existed.
+//
+// `skb_shared_info` lives at the *end of the buffer* in this arrangement,
+// which is what makes `end` smaller than the fragment: the driver picked a
+// fragment size that already includes room for it.
+void *
+kpi_build_skb(void *data, unsigned int frag_len)
+{
+  unsigned char *p;
+  unsigned int i, room;
+
+  if (kpi_l.skb_size == 0 || data == 0 || frag_len <= KPI_SHINFO)
+    return 0;
+
+  p = kpi_dma_heap(kpi_l.skb_size, 64);
+  if (!p)
+    return 0;
+  for (i = 0; i < kpi_l.skb_size; i++)
+    p[i] = 0;
+
+  room = frag_len - KPI_SHINFO;
+  // And zero the shared info, because this buffer is recycled arena and the
+  // driver reads nr_frags out of it without writing it first.
+  for (i = 0; i < KPI_SHINFO; i++)
+    ((unsigned char *)data)[room + i] = 0;
+
+  wrptr(p, kpi_l.skb_head, data);
+  wrptr(p, kpi_l.skb_data, data);
+  wr32(p, kpi_l.skb_tail, 0);
+  wr32(p, kpi_l.skb_end,  room);
+  wr32(p, kpi_l.skb_len,  0);
+  return p;
+}
+
 void *kpi_napi_alloc_skb(void *napi, unsigned int len)
 { (void)napi; return kpi_skb_alloc(len); }
 
@@ -146,6 +212,18 @@ kpi_netif_receive_skb(void *skb)
   return 0;
 }
 
+// gro_receive_skb(gro, skb) -- the same thing, with the GRO node in front.
+//
+// `napi_gro_receive` is an inline wrapper in this kernel, so what a module
+// actually calls is this, and the skb is the *second* argument. Taking the
+// first would hand the stack a `struct gro_node` and call it a frame.
+int
+kpi_gro_receive_skb(void *gro, void *skb)
+{
+  (void)gro;
+  return kpi_netif_receive_skb(skb);
+}
+
 // eth_type_trans(skb, dev) -- consume the Ethernet header and report the
 // protocol. The stack below wants the whole frame including that header, so
 // unlike Linux's this does not pull it off; what it returns is the type field,