nexoniarz/SchOS
nexoniarz/SchOS
Watch
1
0
Fork
You've already forked SchOS
 0
Code Issues Pull requests Projects Releases 1 Packages Wiki Activity Actions

kbd: three layouts, and the arrow keys were sitting on top of UTF-8

Browse source 
`∞`, `→`, `≤` and `'` could not be typed. `ą` and `ę` could. That pairing is
the whole diagnosis: KBD_UP was 0xE2 and its neighbours filled 0xE0..0xE9,
which are exactly the UTF-8 lead bytes for three-byte characters. Every
character in U+0800..U+FFFF -- most punctuation, every arrow, every maths
symbol, the whole of CJK -- arrived as a value the console read as an arrow key
and helpfully turned into `ESC [ A`. Two-byte sequences lead with 0xC4 and
nothing claimed 0xC4, so Polish came through and the rest did not.

The chord codes already knew this rule and said so in a comment. The navigation
keys predate them and were never brought in line. Stated once, in kbd.h: a key
produces either a Unicode codepoint or a value above 0x10FFFF, and nothing in
between means anything. KEY_* -> KBD_*, which also settles a silent collision
-- include/input.h defines its own KEY_HOME, KEY_UP and KEY_END as evdev
numbers, and kbd.c includes both headers, so three macros were being quietly
redefined in the one file that used them.

Then the layouts, which is what the codepoint-sized keycodes were for.

kernel/dev/keymap.c: us, pl, de. A layout is four tables -- plain, Shift,
AltGr, Shift+AltGr -- of uint32 codepoints. uint32 and not uchar is the reason
the file exists: `ó` fits in a byte and `ą` does not, and half a Polish
alphabet is not a Polish layout.

Two things deliberately are *not* tables:

  - **Ctrl is derived from the plain codepoint.** The table it replaces was
    indexed by scancode, which quietly assumed US -- on QWERTZ the Y and Z keys
    swap, so Ctrl+Z would have suspended a job only on some layouts.
  - **Caps Lock is a case toggle on the result**, so AltGr+a with Caps Lock is
    `Ą`. No second table could say that, because Caps Lock combines with AltGr
    rather than replacing it. Latin Extended-A alternates upper/lower by
    codepoint but not all the same way round -- Ł is U+0141 (odd) and Ą is
    U+0104 (even) -- so Polish lands in both halves and the parity matters.

kbdgetc() encodes to UTF-8 and returns the bytes one call at a time.
consoleintr()'s contract is bytes: the serial line hands it UTF-8 already and
it could not tell an encoded codepoint from a byte of the same value.

The layouts live in the kernel. Linux's loadkeys compiles a text keymap and
uploads it a key at a time through KDSKBENT, which puts the data in userspace
-- and a machine whose /usr has not mounted then has no layout at all. Here the
kernel always has a working keyboard and `loadkeys` only names which one.
KB_GETLAYOUT/KB_SETLAYOUT/KB_LISTLAYOUT; /etc/vconsole.conf and an
os.schos.keymap job for the boot-time choice. No dead keys, so French and
Spanish are absent rather than wrong.

/etc/locale.conf and LANG=C.UTF-8 in profile and zshenv. Only the charset half
of that is real -- musl decodes UTF-8 whatever the locale says and ships no
locale data -- but an unset LANG reads as "ASCII only" to everything that asks,
which is its own way of producing `?`.

And two symlink bugs found while chasing

    zsh: locking failed for /root/.zsh_history: operation not permitted

on every boot. symlink() is the classic atomic lock -- create the link, and if
it exists someone else holds it -- and zsh distinguishes EEXIST ("contended,
retry") from anything else ("this cannot work, give up"). sys_symlink returned
a bare -1, which is -EPERM, so every zsh on the machine took the second branch
on a filesystem that supports symlinks perfectly well. Same errno class as
commit 75ccf98.

The second one the first was hiding: **divesymlink() has been in fs.c since the
fork and nothing ever called it.** A symlink behaved as a small file containing
its own target, so `cat link` printed the target's *name*. open() and openat()
follow the final component now (O_NOFOLLOW gives ELOOP), and stat() and lstat()
stop being the same function, since which one follows is the only difference
between them. Mid-path symlinks are still not resolved -- that belongs in
namex(), which would re-enter lookup holding a directory lock.

Verified on the emulated keyboard, not just over serial: AltGr+a/e/s gives
ąęś, Shift+AltGr gives ĄĘŚ, Caps Lock+AltGr gives ĄĘŚ, de types zy for the
physical y/z keys and @€µ on AltGr. `∞ 'x' → ≤ • — …` typed over serial now
survives. Three fresh boots: 0 lock failures, 0 faults.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Nexoniarz 2026-09-20 01:09:32 +02:00
commit 4e3e345b41
15 changed files with 994 additions and 208 deletions
Download patch file Download diff file Expand all files Collapse all files

91
docs/architecture/terminals.md
View file

@ -109,17 +109,74 @@ Linux's numbers, because they do Linux's job.
| `VT_GETSTATE` | active terminal and a bitmask of those in use |
| `VT_OPENQRY` | the first terminal nobody has open |
| `VT_GETMODE`/`VT_SETMODE` | accepted and ignored — the kernel drives switching and there is no process mode to set |
| `VT_GETCHORD`/`VT_SETCHORD` | SchOS's own: which key combination switches. See below |
| `KB_GETLAYOUT`/`KB_SETLAYOUT`/`KB_LISTLAYOUT` | SchOS's own: the keyboard layout, by name |
## The keyboard
`kbd.c` recognises Ctrl+Alt+Shift+F1..F6 and returns `KEY_VT_SWITCH + n`, which is above
0xFF on purpose: every value that fits in a byte is already either a character
or a UTF-8 continuation byte, and a chord that collided with one would be typed
into the terminal instead of switching away from it.
`kbd.c` turns scancodes into characters and chords. `keymap.c` holds the
layouts. Between them is one rule, and it is the rule the whole keycode space
is built around:
> **A key produces either a Unicode codepoint or a value above `0x10FFFF`.
> Nothing in between means anything.**
The chord codes always obeyed it. The navigation keys did not: `KEY_UP` was
`0xE2`, and `0xE0..0xE9` are exactly the UTF-8 lead bytes for three-byte
characters. Typing any of U+0800..U+FFFF — most punctuation, every arrow, every
maths symbol, the whole of CJK — produced a value the console read as an arrow
key and turned into `ESC [ A`. `∞`, `→`, `≤` and `'` vanished from the input
line while `ą` and `ę` came through, because those are two-byte sequences whose
lead byte is `0xC4` and nothing claimed `0xC4`. They are `KBD_*` now and sit
above every codepoint there is.
What the chord *means* is decided in `console.c`, where the terminals are. The
keyboard driver names it and stops.
### Layouts
Three, in the kernel: `us`, `pl` (Polish programmers), `de` (QWERTZ). `loadkeys`
lists and switches them; `/etc/vconsole.conf` records the choice and the
`os.schos.keymap` job applies it at boot.
**In the kernel, not in files.** Linux's `loadkeys` compiles a text keymap and
uploads it a key at a time through `KDSKBENT`, which puts the data in userspace
— and a machine whose `/usr` has not mounted then has no layout at all. Here
the kernel always has a working keyboard and userspace only names which one.
A layout is four tables of codepoints — plain, Shift, AltGr, Shift+AltGr —
indexed by scancode. Two things are deliberately *not* tables:
- **Ctrl is derived from the plain codepoint**, not looked up. The table it
replaced was indexed by scancode, which quietly assumed US: on QWERTZ the Z
and Y keys swap, so Ctrl+Z would have suspended a job only on some layouts.
- **Caps Lock is a case toggle on the result**, so `ą` from the AltGr level
becomes `Ą` — which no second table could have said, because Caps Lock
combines with AltGr rather than replacing it.
Codepoints and not bytes is the entire point: `ó` fits in Latin-1 and `ą` does
not, and half a Polish alphabet is not a Polish layout. `kbdgetc()` encodes to
UTF-8 and hands the bytes to `consoleintr()` one call at a time, because that
function's contract is bytes — the serial line delivers UTF-8 already and
nothing could tell an encoded codepoint from a byte of the same value.
**No dead keys.** On a real German layout the acute and circumflex keys wait
for the next key and combine; here they type the accent alone. It needs
per-keyboard pending state that this interface has no room for, so layouts that
depend on it — French AZERTY, Spanish — are not shipped rather than shipped
wrong.
### Locale
`/etc/locale.conf`, read by `/etc/profile` and `/etc/zsh/zshenv`, default
`LANG=C.UTF-8`.
**Only the charset half of it is real.** musl decodes and encodes UTF-8
whatever the locale says and ships no locale data, so collation stays codepoint
order and messages stay English whichever language is named. What `LANG` does
is tell the programs that ask that the terminal is UTF-8 — an unset `LANG`
reads as "ASCII only", and that is what makes a program fall back to `?`.
### The chord is a setting, and that is the point
**There is no combination that is free for everyone on every host**, and a
@ -273,13 +330,15 @@ Two bugs, both in the same half-second of using a shell, and both worth
recording because the symptom pointed away from the cause.
**Arrow keys corrupted the next two keystrokes.** They arrive from the keyboard
as single keycodes -- `KEY_UP` is `0xE2` -- and were pushed into the input
buffer exactly like that. No terminal has ever worked that way, and `0xE2` is
the worst byte to have picked: it is a UTF-8 lead byte for a three-byte
character, so a reader decoding UTF-8 swallowed the *next two keys* waiting for
continuations that never came. `consoleintr()` translates them to the DEC/xterm
sequences now -- `ESC[A`, `ESC[D`, `ESC[5~` and the rest -- which is what every
terminfo entry and every line editor already expects.
as single keycodes and were pushed into the input buffer exactly like that. No
terminal has ever worked that way. `consoleintr()` translates them to the
DEC/xterm sequences now -- `ESC[A`, `ESC[D`, `ESC[5~` and the rest -- which is
what every terminfo entry and every line editor already expects.
The *value* those keycodes had was a second bug wearing the same coat, and it
outlived the fix: `KEY_UP` was `0xE2`, a UTF-8 lead byte, so a character typed
from the other direction was eaten just as thoroughly. See the keycode rule
above.
**Then the end of the line vanished as you typed into it.** With the arrows
working, editing mid-line left the buffer perfectly correct and the screen
@ -331,7 +390,9 @@ waiting for a different editor.
advance. Combining marks land on top of their own cell rather than the
preceding one, and CJK draws at half the width it wants. Both need a
width table and a cell that can span two columns.
- **Typed input above U+07FF is unreliable.** Two-byte UTF-8 arrives intact;
three-byte sequences typed at the shell come through mangled. The console's
own decoder is not the problem — the same characters `cat`ed from a file
render correctly — so this belongs with keyboard and locale handling.
- **No dead keys**, so French and Spanish layouts are absent rather than
broken. See above.
- **No per-terminal layout.** One keyboard, one layout. Switching to tty3 does
not move anybody's fingers.
- **Only three layouts.** Adding one is a table in `keymap.c` and a line in the
catalogue; the mechanism is not the limit, the typing is.