Map device registers with ioremap instead of assuming a low BAR

Every driver reached its registers with P2V(bar.base), which assumes the
BAR sits inside the 0-4 GB the direct map covers. Under SeaBIOS it always
does. Under OVMF it does not: firmware may place a 64-bit BAR anywhere,
and it puts virtio-gpu's at about 768 GB. P2V produced an address with no
translation behind it, so the driver's first register write took a page
fault, then a double fault, then a triple fault and a reboot loop --
which, coming from a driver bind, reads as broken hardware rather than a
missing mapping.

ioremap() carries each region on a 1 GB page in PDPT entries 4-511, which
were unused: the direct map occupies 0-3, and kpml4[0], kpml4[KERNBASE]
and every process's PML4 all share that one table, so a mapping appears
in every address space at once with no page table to allocate. The pages
are cache-disabled and write-through for the same reason the top GB of
the direct map is -- these are registers, and a write sitting in a cache
line is a write the device never sees.

A physical address below 4 GB returns the direct-map address unchanged,
so the BIOS path, where every BAR is low, is byte for byte what it was.

virtio-gpu now initialises under OVMF (1280x800, queue 64) instead of
triple faulting. e1000 and xhci had the identical latent bug; xhci was
already refusing to bind above 4 GB with a comment saying the mapping did
not exist yet, and now it does.

BIOS unchanged: bootcheck 108 [ok] with -Fb -Tty -Usb -Sock -Evdev -Ps,
e1000 and xhci both binding.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

kernel/arch/x86_64/vm.c

@@ -224,6 +224,75 @@ kvmalloc(void)
   switchkvm();
 }
 
+// Map a device's registers so the kernel can reach them.
+//
+// Every driver used to say P2V(bar.base), which quietly assumes the BAR lands
+// inside the 0-4 GB the loop above maps. Under SeaBIOS it always does. Under
+// OVMF it does not: firmware is free to put a 64-bit BAR anywhere, and it puts
+// virtio-gpu's at about 768 GB. P2V then produces an address with no
+// translation behind it, and the driver's first register write takes a page
+// fault -- which, coming from a driver bind, looks like broken hardware rather
+// than a missing mapping.
+//
+// The direct map uses PDPT entries 0-3 of a page directory pointer table that
+// kpml4[0], kpml4[KERNBASE] and every process's PML4 (see setupkvm) all share.
+// Entries 4-511 are free, so each one can carry a 1 GB page of device memory:
+// no page table to allocate, nothing to free, and the mapping appears in every
+// address space at once because they all point at this one table.
+//
+// Cache-disabled and write-through, for the same reason the top GB of the
+// direct map is: these are registers, and a write that sits in a cache line
+// waiting for company is a write the device never sees.
+#define IOREMAP_FIRST_PDPT 4
+#define IOREMAP_LAST_PDPT  511
+#define GIGABYTE           (1ULL << 30)
+
+static int ioremap_next = IOREMAP_FIRST_PDPT;
+
+void *
+ioremap(uint64 pa, uint64 size)
+{
+  if (size == 0)
+    return 0;
+
+  // Below 4 GB the direct map already covers it, so hand back the same address
+  // the old code would have produced. This is what keeps the BIOS path -- where
+  // every BAR is low -- byte for byte what it was.
+  if (pa + size <= 4 * GIGABYTE)
+    return P2V(pa);
+
+  uint64 base = pa & ~(GIGABYTE - 1);          // 1 GB pages, so align to one
+  uint64 last = (pa + size - 1) & ~(GIGABYTE - 1);
+  int    span = (int)((last - base) / GIGABYTE) + 1;
+
+  // Already mapped? A second BAR in the same gigabyte should not consume a
+  // second window, and drivers do re-map the same device.
+  for (int i = IOREMAP_FIRST_PDPT; i < ioremap_next; i++) {
+    // Clearing the low 30 bits drops every flag as well, leaving the frame.
+    if ((kpdpt[i] & PTE_P) && (kpdpt[i] & ~(GIGABYTE - 1)) == base)
+      return (void *)(KERNBASE + ((uint64)i << 30) + (pa - base));
+  }
+
+  if (ioremap_next + span > IOREMAP_LAST_PDPT) {
+    cprintf("ioremap: out of device window for %p\n", (void *)pa);
+    return 0;
+  }
+
+  int first = ioremap_next;
+  for (int i = 0; i < span; i++) {
+    kpdpt[ioremap_next] = (base + (uint64)i * GIGABYTE)
+                        | PTE_PS | PTE_P | PTE_W | PTE_PCD | PTE_PWT;
+    ioremap_next++;
+  }
+
+  // The entries were not present a moment ago. Reloading CR3 is heavier than
+  // invlpg but this runs a handful of times at boot, and it cannot be the thing
+  // that is subtly wrong.
+  switchkvm();
+
+  return (void *)(KERNBASE + ((uint64)first << 30) + (pa - base));
+}
+
 void
 switchuvm(struct proc *p)
 {