mm: the loader jumped out from under itself, and 3 GB was never the limit

Three things, and the first one is the freeze.

**The loader jumped out from under itself.** A machine with a lot of RAM
stopped at "Starting the kernel" with the splash still up: no panic, no reset,
nothing. The firmware's own exception handler had the answer the whole time,
once a machine was found that printed it:

    X64 Exception Type - 0E(#PF - Page-Fault)
    ExceptionData - 0x10   I:1 P:0
    RIP - 0000000140004DEE      <- the loader's own code, at 5.4 GB
    RAX - 0000000000001000      <- the page tables it had just installed
    RCX - 00000000001000F0      <- the kernel it was about to jump to

I:1 P:0 is an instruction fetch from a page that is not present. The loader
built page tables covering four gigabytes, loaded cr3 -- and the firmware had
placed the loader image *above* four gigabytes, so the next instruction fetch
was unmapped. It reproduces exactly on `-m 8G` and never on `-m 1G`, which is
why every test here passed.

The PDPT is one page whether four entries are filled or all 512, so all 512
are: 512 GB of identity map, replaced by kvmalloc() a few hundred instructions
later.

**And 3 GB was never the limit, it was a circle.** PHYSTOP_MAX was 0xC0000000
because kvmalloc() mapped exactly four 1 GB pages, and kvmalloc mapped four
because that was the cap -- each holding the other up, throwing away thirteen
gigabytes of a sixteen gigabyte machine. kvmalloc now maps as many gigabytes as
the machine has RAM and tells ioremap where the direct map stopped, instead of
ioremap assuming it stopped at four.

Freeing memory becomes two passes, and the ordering is forced rather than
chosen: kinit1() runs *before* kvmalloc because kvmalloc calls kalloc for its
own page tables, and freeing a page means writing into it -- so kinit1 can only
touch what the loader mapped. kinit_high() adds the rest afterwards.
`phystop` and the new `lowtop` are now two different numbers: the DMA arena
stays below 4 GB because a device is not obliged to reach the top of memory,
and an RTL8139's buffer address register is thirty-two bits wide.

**And the 0 MB.** The System Information screen reported "0 MB usable" on a
machine with sixteen gigabytes in it, because memory_mb() had its own 16 KB
guess at how big a memory map gets, and returned zero when GetMemoryMap said
EFI_BUFFER_TOO_SMALL. That is the same bug as the previous commit's, in a
second copy, and this one failed silently. There is one buffer and one
map_read() now.

Verified at 1 GB, 8 GB and 16 GB: all three boot, no traps, no panics, socktest
and memfdtest green in each. 8 GB and 16 GB did not boot at all before -- and
16 GB now reports 17408 MB with "mem: 13312 MB above 4 GB added to the
allocator", where the old kernel would have used three. Both display
configurations still clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

kernel/arch/x86_64/vm.c

@@ -32,6 +32,16 @@ _Static_assert(PERCPU_SELF == 0 && PERCPU_SCRATCH == 8 && PERCPU_CPU == 16 &&
                "per-CPU block layout changed -- update kernel/arch/x86_64/trapasm.S");
 
 static pml4e_t *kpml4;
+// Where the device windows begin is now decided by kvmalloc(), which knows how
+// much of the address space the direct map took. The constant that used to be
+// here said 4, and would have handed out windows on top of RAM the moment the
+// direct map grew past four gigabytes.
+#define IOREMAP_LAST_PDPT  511
+#define GIGABYTE           (1ULL << 30)
+
+// The first PDPT entry the direct map did not take. Set by kvmalloc().
+static int ioremap_next = 4;
+
 static pdpe_t *kpdpt;
 
 struct tassdesc {
@@ -231,14 +241,34 @@ kvmalloc(void)
   kpml4[0] = v2p(kpdpt) | PTE_P | PTE_W;
   kpml4[PMX(KERNBASE)] = v2p(kpdpt) | PTE_P | PTE_W;
   
-  for(uint64 i = 0; i < 4; i++) {
+  // As many gigabytes as the machine has RAM, not four.
+  //
+  // Four was the number for as long as PHYSTOP was capped at 3 GB, and the cap
+  // existed because this loop stopped at four -- each holding the other up. A
+  // laptop with sixteen gigabytes lost thirteen of them to that circle.
+  //
+  // The 3-4 GB entry stays uncached because that is where the PCI hole is and
+  // always has been. Everything above 4 GB is RAM -- the hole is below it on
+  // every machine this runs on -- so those are cached, and a device BAR up
+  // there still gets its own uncached window from ioremap().
+  uint64 gigs = (phystop + GIGABYTE - 1) / GIGABYTE;
+  if (gigs < 4)
+    gigs = 4;                       // always map the low four
+  if (gigs > IOREMAP_LAST_PDPT - 8)
+    gigs = IOREMAP_LAST_PDPT - 8;   // leave room for device windows
+
+  for(uint64 i = 0; i < gigs; i++) {
     uint64 flags = PTE_PS | PTE_P | PTE_W;
-    if(i >= 3) {
+    if(i == 3) {
       flags |= PTE_PCD | PTE_PWT;
     }
     kpdpt[i] = (i << 30) | flags;
   }
 
+  // ioremap takes what is left. It used to start at a constant 4, which was
+  // correct only while the direct map was exactly four gigabytes.
+  ioremap_next = (int)gigs;
+
   switchkvm();
 }
 
@@ -261,11 +291,6 @@ kvmalloc(void)
 // Cache-disabled and write-through, for the same reason the top GB of the
 // direct map is: these are registers, and a write that sits in a cache line
 // waiting for company is a write the device never sees.
-#define IOREMAP_FIRST_PDPT 4
-#define IOREMAP_LAST_PDPT  511
-#define GIGABYTE           (1ULL << 30)
-
-static int ioremap_next = IOREMAP_FIRST_PDPT;
 
 void *
 ioremap(uint64 pa, uint64 size)
@@ -285,7 +310,7 @@ ioremap(uint64 pa, uint64 size)
 
   // Already mapped? A second BAR in the same gigabyte should not consume a
   // second window, and drivers do re-map the same device.
-  for (int i = IOREMAP_FIRST_PDPT; i < ioremap_next; i++) {
+  for (int i = 4; i < ioremap_next; i++) {
     // Clearing the low 30 bits drops every flag as well, leaving the frame.
     if ((kpdpt[i] & PTE_P) && (kpdpt[i] & ~(GIGABYTE - 1)) == base)
       return (void *)(KERNBASE + ((uint64)i << 30) + (pa - base));