nexoniarz/Finch
Watch
1
0
Fork
You've already forked Finch
 0
mirror of https://github.com/Nexoniarz/Finch.git synced 2026-11-03 08:54:25 +00:00
Code Issues Projects Releases Packages Wiki Activity

docs (en, pl) and editors: masks, tails, shuffles, lookup, dot_add, @cpu, cpu_has, Nix

Browse source 
- technicians' guides: section 31 rewritten around kernels (masks, the end of a list, moving lanes,
  int8 dot products, choosing instructions at build and at run time)
- cheat sheets and engineers' guides: the new operations and their IR; how features are found
- Kate and VS Code: @cpu, boolxN and the new built-ins highlighted

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Psc42moPLwP43Cmb9oZqSR
This commit is contained in:
Claude 2026-11-01 15:46:04 +00:00
commit 1f45bd8a01
No known key found for this signature in database
12 changed files with 333 additions and 30 deletions
Download patch file Download diff file Expand all files Collapse all files

8
docs/en/cheatsheet.md
View file

@ -255,13 +255,19 @@ Types: `f32x4/8/16`, `f64x2/4/8`, `i8x16/32/64`, `i16x8/16/32`, `i32x4/8/16`, `i
| `f32x8(v)` | an 8-lane vector converted lane by lane (`f32x8(ints)`, `i32x8(floats)`) |
| `v + w`, `v * 2.0`, `-v` | lane by lane; a plain number counts for every lane (`/` only on floats) |
| `& \| ^ << >> ~` | lane by lane, whole-number lanes |
| `v == w`, `v != w` | bool: every lane equal / not |
| `v < w`, `v == w`, … | a mask `boolx8`: `m.all()`, `m.any()`, `m.count()`, `m & n`, `!m`, `m[i]` |
| `select(m, a, b)` | lane by lane: `a` where `m` is true, else `b` |
| `v[i]`, `v[i] = x`, `v.len` | a lane; the number of lanes |
| `min(v, w)`, `max`, `abs(v)`, `clamp(v, lo, hi)` | lane by lane |
| `v.sum()`, `v.min()`, `v.max()`, `v.dot(w)` | one number from all the lanes |
| `v.sqrt()`, `v.floor()`, `v.ceil()`, `v.round()` | every lane (float lanes) |
| `v.fma(b, c)`, `v.muladd(b, c)` | `v * b + c`: one rounding always / the fastest way here |
| `v.store(list, i)` | lanes into `list[i]` … |
| `f32x8(list, i, fill)`, `v.store_part(list, i)` | the end of a list: only the lanes inside it (the others are `fill`) |
| `v.shuffle(3, 2, 1, 0)`, `v.lo()`, `v.hi()`, `concat(a, b)` | lanes rearranged (shuffle takes numbers written in the code) |
| `table.lookup(idx)` | `table[idx % table.len]` per lane (`pshufb`, `tbl`) |
| `acc.dot_add(a, b)` | `i32x8` sums gain 4 byte products each (`u8x32` × `i8x32`): `vpdpbusd`, `usdot` where there |
| `@cpu("avx2,fma") fn …`, `cpu_has("avx2")` | a function that may use these features; check at run time before calling it |
---

24
docs/en/for-engineers.md
View file

@ -304,7 +304,13 @@ type goes; a call `f32x8(…)` goes to `vecMake()`.
| `f32x8(i32x8 v)` | `sitofp`, `fptosi`, `sext`, `trunc`… on the whole vector |
| `v + w`, `v * 2.0` | `fadd`/`add`… on vectors; a scalar operand is coerced to the lane type, then splatted |
| `v[i]` | a place: `getelementptr <lane>, ptr %v, i64 i` after a bounds check; or `extractelement` on a temporary |
| `v == w` | `fcmp oeq`/`icmp eq` → `<N x i1>` → `llvm.vector.reduce.and` |
| `v < w`, `v == w` … | `fcmp`/`icmp` → `<N x i1>`, a mask (`boolxN`: `vec` with `bool` lanes) |
| `select(m, a, b)` | `select <N x i1>` |
| `m.any()`, `.all()`, `.count()` | `llvm.vector.reduce.or`/`.and`; `zext` + `llvm.vector.reduce.add` |
| `f32x8(list, i, fill)`, `v.store_part(list, i)` | `llvm.masked.load`/`llvm.masked.store` with the mask `<0, 1, …> < len - i` |
| `v.shuffle(…)`, `.lo()`, `.hi()`, `concat(a, b)` | `shufflevector` with a constant mask |
| `table.lookup(idx)` | `and` with `len - 1`, then `extractelement`/`insertelement` per lane: the pattern the x86 and ARM backends turn into `pshufb`/`tbl` |
| `acc.dot_add(a, b)` | `llvm.x86.avx512.vpdpbusd.{128,256,512}` or `llvm.aarch64.neon.{usdot,sdot,udot}` when the function's features have them; else `zext`/`sext`, `mul`, four strided `shufflevector`s and `add`s |
| `min`, `max`, `abs` | `llvm.minnum`/`maxnum`, `llvm.smin`/`umin`…, `llvm.fabs`/`llvm.abs` |
| `v.sum()`, `.min()`, `.max()` | `llvm.vector.reduce.fadd` (with `reassoc`), `.add`, `.fmin`, `.smin`… |
| `v.sqrt()`, `.floor()`, `.ceil()`, `.round()` | the `llvm.*` intrinsic of that name |
@ -323,6 +329,20 @@ aligned address and crash.
**Which instructions** is clang's choice: `--cpu <name>` adds `-march=<name>` (x86-64) or `-mcpu=<name>` (ARM64) to the
IR's compile, so the same IR becomes SSE2, AVX2 + FMA, AVX-512 or NEON code. Lanes wider than the registers are split.
Some operations need to know the features at code generation time (`dot_add` picks `vpdpbusd` only where it exists).
`linker.cpuFeatures()` asks clang for its predefined macros with the same target and `--cpu`
(`clang -march=X -dM -E empty.c`: `__AVXVNNI__`, `__AVX512VNNI__`, `__ARM_FEATURE_MATMUL_INT8` …), maps them to Finch's
feature names and caches the answer per clang and flags; they reach `gen` as `Options.features`. `@cpu("…")` on a function
(`Node.cpu`) adds its own: `define … "target-features"="+avx2,+fma"` (with `+evex512` for any AVX-512 feature, which
LLVM 18+ needs for 512-bit registers), and `g.curFeatures` is the union while that function is generated. Features of the
other architecture are dropped (`cpuOf`). `checkCpuCall` refuses a vector wider than 128 bits as a parameter or result
between functions whose `@cpu` differ, because caller and callee would lower it to different registers.
`cpu_has(name)` calls `finch_cpu_has` in the runtime: CPUID leaves 1 and 7 with XGETBV for the AVX/AVX-512 state on x86,
`getauxval(AT_HWCAP/AT_HWCAP2)` on Linux ARM64, `sysctlbyname("hw.optional.arm.FEAT_…")` on macOS.
Clang is always run as `NIX_ENFORCE_NO_NATIVE=0 clang …` on Unix: Nix's cc-wrapper otherwise drops `-march=native`
inside `nix-shell`. If `--cpu native` still yields no SSE4.2 macro on x86, the driver warns.
---
## 8. Ownership and memory
@ -572,5 +592,5 @@ evaluated twice and temporaries are never pointed to.
- Generics are monomorphised (code size grows with instantiations); there are no trait bounds, a wrong use is an error
inside the instantiated body.
- A failed test's owned memory is not freed (the test runner `longjmp`s out of it).
- Vectors have no lane masks (`v < w`), shuffles or integer division.
- Vectors have no integer division, and `lookup` on baseline x86-64 (no SSSE3) is done lane by lane.

130
docs/en/for-technicians.md
View file

@ -1400,18 +1400,15 @@ arrays, maps and function parameters, and `print` and `str` show it.
| `+ - * /` | lane by lane (`/` on floats only: x86 and ARM can't divide whole numbers lane by lane) |
| `& \| ^ << >> ~` | lane by lane, on whole-number lanes |
| `-v` | every lane negated |
| `v == w`, `v != w` | `true` if every lane is equal / if any differs |
| `v[i]`, `v[i] = x`, `v.len` | one lane; the number of lanes |
| `min(v, w)`, `max(v, w)`, `abs(v)`, `clamp(v, lo, hi)` | lane by lane; `lo`, `hi` and `w` can be plain numbers |
| `v.sum()`, `v.min()`, `v.max()` | all the lanes made into one number |
| `v.dot(w)` | `(v * w).sum()` |
| `v.sqrt()`, `v.floor()`, `v.ceil()`, `v.round()` | every lane (float lanes) |
| `v.fma(b, c)` | `v * b + c` with one rounding, always |
| `v.muladd(b, c)` | `v * b + c` the fastest way here: one instruction where the processor can |
| `v.fma(b, c)` | `v * b + c` with one rounding, always (the same on every processor) |
| `v.muladd(b, c)` | `v * b + c` the fastest way here: one instruction where the processor can (the last bit may differ between processors) |
| `v.store(list, i)` | writes the lanes to `list[i]` … (or through a `ptr`) |
`<` and the other comparisons would give an answer per lane, so they aren't there; use `min`, `max` or `v[i] < w[i]`.
**A real loop.** The usual shape: as many whole vectors as fit, then the rest one by one.
```c
@ -1433,10 +1430,125 @@ fn dot([]f32 a, []f32 b) -> f32 {
`examples/simd.fch` times this against the plain loop.
**Which instructions.** By default a program runs on every processor of its system (on x86-64 that means SSE2, so an
`f32x8` is done as two halves). `--cpu native` (or `cpu = "native"` in `finch.toml`) lets LLVM use everything the processor
that builds it has: AVX2 and FMA, AVX-512, or NEON's newer parts on ARM. Such a program needs a processor like that one to
run. To pick a level instead: `--cpu x86-64-v3` (AVX2, most PCs since 2015), `--cpu x86-64-v4` (AVX-512), `--cpu apple-m1`.
### Masks: comparing, choosing, the end of a list
Comparing vectors compares every lane, so the answer is a **mask**: one `true`/`false` a lane, of type `boolx8` (for 8 lanes).
```c
v := f32x8(-2.0, -1.0, 0.0, 1.0, 2.0, 3.0, -4.0, 5.0)
m := v > 0.0 // boolx8(false, false, false, true, true, true, false, true)
relu := select(m, v, 0.0) // v where the mask is true, 0 elsewhere
print(m.count(), m.any(), m.all()) // 4 true false
if (v == w).all() { ... } // a condition needs one bool: all() or any()
```
| | |
|---|---|
| `v < w`, `<=`, `>`, `>=`, `==`, `!=` | a mask (a plain number on one side counts for every lane) |
| `select(m, a, b)` | lane by lane: `a` where `m` is true, `b` where false (either may be a plain number) |
| `m.any()`, `m.all()`, `m.count()` | is any lane true, are all, how many |
| `m & n`, `m \| n`, `m ^ n`, `!m` | masks combined |
| `m[i]` | one lane, as a `bool` |
| `i32x8(m)`, `boolx8(v)` | true is 1; a lane that isn't 0 is true |
**The end of a list.** A list is rarely a whole number of vectors long. Two forms touch only the lanes that are inside it:
| | |
|---|---|
| `f32x8(list, i, fill)` | `list[i]` … as far as the list goes; the lanes past its end are `fill` |
| `v.store_part(list, i)` | writes only the lanes that fit in the list |
```c
fn add_one([]f32 xs) {
i := 0
while i < xs.len { // no separate loop for the rest
(f32x8(xs, i, 0.0) + 1.0).store_part(xs, i)
i += 8
}
}
```
### Moving lanes
| | |
|---|---|
| `v.shuffle(3, 2, 1, 0)` | the lanes with these numbers (written in the code), in this order; as many lanes as numbers |
| `v.lo()`, `v.hi()` | the lower and the upper half |
| `concat(a, b)` | `a`'s lanes, then `b`'s |
| `table.lookup(idx)` | for every lane of `idx`: `table[idx % table.len]`. A byte table (`u8x16`, `i8x16`) is one `pshufb` on x86, `tbl` on ARM |
Unpacking 4-bit numbers (two to a byte, as quantized models store them) is shuffle, mask and lookup:
```c
fn unpack4(u8x8 packed, i8x16 meaning) -> i8x16 {
pairs := packed.shuffle(0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7) // each byte twice
odd := u8x16(0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1) == 1
nibbles := select(odd, pairs >> 4, pairs & 15) // low nibble, then high
return meaning.lookup(nibbles) // 0..15 -> what it stands for
}
```
### Int8 dot products: dot_add
`acc.dot_add(a, b)` is the step of a quantized model's matrix multiply. `acc` is an `i32x8` of sums; `a` and `b` have four
times as many bytes (`u8x32` and `i8x32`; `i8` × `i8` and `u8` × `u8` work too). Each lane of `acc` gains the sum of four
products, `a[4k]*b[4k] + … + a[4k+3]*b[4k+3]`, exactly (no saturation), and the sums stay in the vector:
```c
acc := i32x8(0)
i := 0
while i + 32 <= weights.len {
acc = acc.dot_add(u8x32(activations, i), i8x32(weights, i))
i += 32
}
total := acc.sum() // once, at the end
```
It is one instruction where the processor has one: `vpdpbusd` with AVX-VNNI (Intel since Alder Lake, AMD Zen 5) or
AVX-512 VNNI, `usdot`/`sdot`/`udot` on ARM with the dot-product features. Elsewhere it is a few instructions
(`pmaddwd` and adds), with the same result. Which one is used depends on `--cpu` or the function's `@cpu` (below).
### Choosing the instructions: --cpu, @cpu and cpu_has
By default a program runs on every processor of its system (on x86-64 that means SSE2, so an `f32x8` is done as two
halves). Three ways to use more:
**`--cpu`, for the whole program.** `--cpu native` (or `cpu = "native"` in `finch.toml`) lets LLVM use everything the
processor that builds it has: AVX2 and FMA, AVX-512, VNNI, or NEON's newer parts on ARM. The program then needs a processor
like that one. To pick a level instead: `--cpu x86-64-v3` (AVX2, most PCs since 2015), `--cpu x86-64-v4` (AVX-512),
`--cpu alderlake`, `--cpu apple-m1`, `--cpu neoverse-v1`.
**`@cpu`, for one function, and `cpu_has` to choose at run time.** One program can carry an AVX-512 version, an AVX2
version and a plain one, and pick the best the computer it runs on has:
```c
@cpu("avx512f,avx512bw,avx512vl,avx512vnni")
fn matmul_avx512([]u8 a, []i8 b, []i32 out) { ... } // may use AVX-512 inside
@cpu("x86-64-v3")
fn matmul_avx2([]u8 a, []i8 b, []i32 out) { ... } // a level works too
fn matmul([]u8 a, []i8 b, []i32 out) {
if cpu_has("avx512vnni") && cpu_has("avx512vl") {
matmul_avx512(a, b, out)
} else if cpu_has("avx2") {
matmul_avx2(a, b, out)
} else {
matmul_plain(a, b, out)
}
}
```
- The features are x86's `sse3 ssse3 sse4.1 sse4.2 popcnt avx avx2 fma f16c bmi1 bmi2 avx512f avx512bw avx512dq avx512vl
avx512vnni avx512bf16 avx512fp16 avxvnni`, the levels `x86-64-v2`/`v3`/`v4`, and ARM's `neon fp16 dotprod i8mm bf16 sve sve2`.
- Names of the other architecture are left out when building, so the x86 and ARM versions sit side by side in one file
(and `cpu_has` says `false` for them).
- Call a `@cpu` function only after `cpu_has` said yes: on a processor without the feature it stops with *Illegal instruction*.
- A vector wider than 16 bytes (`f32x8`, `i32x16` …) can't be passed between functions with different `@cpu`: the two
sides would expect it in different registers. Pass the list (or a `ptr`) instead; Finch says so if you try.
Lanes wider than the processor's registers are split by LLVM, and the same code builds for x86-64 and ARM64 alike.
`v.sum()` on float lanes adds them in whatever order is fastest, so the last digits can differ from adding them one by one.
On NixOS, Nix's clang wrapper removes `-march=native` inside `nix-shell`. Finch tells the wrapper to keep it, so
`--cpu native` works there too; if it ever has no effect, Finch warns.