mm: the loader jumped out from under itself, and 3 GB was never the limit
Three things, and the first one is the freeze.
**The loader jumped out from under itself.** A machine with a lot of RAM
stopped at "Starting the kernel" with the splash still up: no panic, no reset,
nothing. The firmware's own exception handler had the answer the whole time,
once a machine was found that printed it:
X64 Exception Type - 0E(#PF - Page-Fault)
ExceptionData - 0x10 I:1 P:0
RIP - 0000000140004DEE <- the loader's own code, at 5.4 GB
RAX - 0000000000001000 <- the page tables it had just installed
RCX - 00000000001000F0 <- the kernel it was about to jump to
I:1 P:0 is an instruction fetch from a page that is not present. The loader
built page tables covering four gigabytes, loaded cr3 -- and the firmware had
placed the loader image *above* four gigabytes, so the next instruction fetch
was unmapped. It reproduces exactly on `-m 8G` and never on `-m 1G`, which is
why every test here passed.
The PDPT is one page whether four entries are filled or all 512, so all 512
are: 512 GB of identity map, replaced by kvmalloc() a few hundred instructions
later.
**And 3 GB was never the limit, it was a circle.** PHYSTOP_MAX was 0xC0000000
because kvmalloc() mapped exactly four 1 GB pages, and kvmalloc mapped four
because that was the cap -- each holding the other up, throwing away thirteen
gigabytes of a sixteen gigabyte machine. kvmalloc now maps as many gigabytes as
the machine has RAM and tells ioremap where the direct map stopped, instead of
ioremap assuming it stopped at four.
Freeing memory becomes two passes, and the ordering is forced rather than
chosen: kinit1() runs *before* kvmalloc because kvmalloc calls kalloc for its
own page tables, and freeing a page means writing into it -- so kinit1 can only
touch what the loader mapped. kinit_high() adds the rest afterwards.
`phystop` and the new `lowtop` are now two different numbers: the DMA arena
stays below 4 GB because a device is not obliged to reach the top of memory,
and an RTL8139's buffer address register is thirty-two bits wide.
**And the 0 MB.** The System Information screen reported "0 MB usable" on a
machine with sixteen gigabytes in it, because memory_mb() had its own 16 KB
guess at how big a memory map gets, and returned zero when GetMemoryMap said
EFI_BUFFER_TOO_SMALL. That is the same bug as the previous commit's, in a
second copy, and this one failed silently. There is one buffer and one
map_read() now.
Verified at 1 GB, 8 GB and 16 GB: all three boot, no traps, no panics, socktest
and memfdtest green in each. 8 GB and 16 GB did not boot at all before -- and
16 GB now reports 17408 MB with "mem: 13312 MB above 4 GB added to the
allocator", where the old kernel would have used three. Both display
configurations still clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
96e10adfcb
commit
3b26575b4a
9 changed files with 246 additions and 55 deletions
|
|
@ -420,6 +420,7 @@ char* kalloc(void);
|
|||
uint64 kalloc_free_bytes(void);
|
||||
void kfree(char*);
|
||||
void kinit1(void*, void*);
|
||||
void kinit_high(void);
|
||||
void kinit2(void);
|
||||
|
||||
// kbd.c
|
||||
|
|
|
|||
|
|
@ -37,16 +37,39 @@ struct memregion {
|
|||
|
||||
extern struct memregion mem_usable[MEMREGION_MAX];
|
||||
extern int nmem_usable;
|
||||
|
||||
// The top of the low region: where the DMA arena is carved from, and the
|
||||
// ceiling for kinit1(). Always below DIRECTMAP_BOOT, and separate from
|
||||
// `phystop` because a device is not obliged to be able to reach the top of
|
||||
// memory -- an RTL8139's buffer address register is thirty-two bits wide.
|
||||
extern uint64 lowtop;
|
||||
#endif
|
||||
|
||||
// What to assume when the firmware gives us nothing usable: the old constant,
|
||||
// so that path behaves exactly as the kernel always did.
|
||||
#define PHYSTOP_FALLBACK 0xE000000ULL
|
||||
|
||||
// kvmalloc() maps physical 0-4 GB as four 1 GB pages and marks the top one
|
||||
// uncached, because that is where PCI BARs and the framebuffer sit. 3 GB is
|
||||
// therefore the most that can be treated as RAM.
|
||||
#define PHYSTOP_MAX 0xC0000000ULL
|
||||
// The most RAM the kernel will use.
|
||||
//
|
||||
// This was 0xC0000000 -- three gigabytes -- because kvmalloc() mapped exactly
|
||||
// four 1 GB pages and the fourth was marked uncached for the PCI hole. On a
|
||||
// laptop with sixteen gigabytes that threw away thirteen of them, silently.
|
||||
//
|
||||
// kvmalloc() now maps as many 1 GB pages as the machine has RAM, so the limit
|
||||
// is the page table rather than a constant: the kernel PDPT has 512 entries
|
||||
// and ioremap takes what the direct map does not, so 64 GB of RAM still leaves
|
||||
// 448 GB of device window. A machine with more than this gets the rest
|
||||
// ignored, with a line saying so.
|
||||
#define PHYSTOP_MAX (64ULL << 30)
|
||||
|
||||
// What the *loader's* page tables cover, and therefore the ceiling on anything
|
||||
// that runs before kvmalloc() builds the kernel's own.
|
||||
//
|
||||
// kinit1() frees pages by writing a free-list pointer into each one, so every
|
||||
// page it touches has to be mapped at the time. Before kvmalloc that means the
|
||||
// low four gigabytes and nothing else, which is why freeing memory happens in
|
||||
// two passes -- see kinit_high().
|
||||
#define DIRECTMAP_BOOT (4ULL << 30)
|
||||
|
||||
// Below this the kernel image, its page tables and the DMA arena do not fit.
|
||||
#define PHYSTOP_MIN (24ULL * 1024 * 1024)
|
||||
|
|
|
|||
Loading…
Reference in a new issue