mirror of
https://github.com/Nexoniarz/Finch.git
synced 2026-11-03 00:44:27 +00:00
docs (en, pl) and editors: masks, tails, shuffles, lookup, dot_add, @cpu, cpu_has, Nix
- technicians' guides: section 31 rewritten around kernels (masks, the end of a list, moving lanes, int8 dot products, choosing instructions at build and at run time) - cheat sheets and engineers' guides: the new operations and their IR; how features are found - Kate and VS Code: @cpu, boolxN and the new built-ins highlighted Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Psc42moPLwP43Cmb9oZqSR
This commit is contained in:
Claude
parent
04f1f94a21
commit
1f45bd8a01
12 changed files with 333 additions and 30 deletions
|
|
@ -1400,18 +1400,15 @@ arrays, maps and function parameters, and `print` and `str` show it.
|
|||
| `+ - * /` | lane by lane (`/` on floats only: x86 and ARM can't divide whole numbers lane by lane) |
|
||||
| `& \| ^ << >> ~` | lane by lane, on whole-number lanes |
|
||||
| `-v` | every lane negated |
|
||||
| `v == w`, `v != w` | `true` if every lane is equal / if any differs |
|
||||
| `v[i]`, `v[i] = x`, `v.len` | one lane; the number of lanes |
|
||||
| `min(v, w)`, `max(v, w)`, `abs(v)`, `clamp(v, lo, hi)` | lane by lane; `lo`, `hi` and `w` can be plain numbers |
|
||||
| `v.sum()`, `v.min()`, `v.max()` | all the lanes made into one number |
|
||||
| `v.dot(w)` | `(v * w).sum()` |
|
||||
| `v.sqrt()`, `v.floor()`, `v.ceil()`, `v.round()` | every lane (float lanes) |
|
||||
| `v.fma(b, c)` | `v * b + c` with one rounding, always |
|
||||
| `v.muladd(b, c)` | `v * b + c` the fastest way here: one instruction where the processor can |
|
||||
| `v.fma(b, c)` | `v * b + c` with one rounding, always (the same on every processor) |
|
||||
| `v.muladd(b, c)` | `v * b + c` the fastest way here: one instruction where the processor can (the last bit may differ between processors) |
|
||||
| `v.store(list, i)` | writes the lanes to `list[i]` … (or through a `ptr`) |
|
||||
|
||||
`<` and the other comparisons would give an answer per lane, so they aren't there; use `min`, `max` or `v[i] < w[i]`.
|
||||
|
||||
**A real loop.** The usual shape: as many whole vectors as fit, then the rest one by one.
|
||||
|
||||
```c
|
||||
|
|
@ -1433,10 +1430,125 @@ fn dot([]f32 a, []f32 b) -> f32 {
|
|||
|
||||
`examples/simd.fch` times this against the plain loop.
|
||||
|
||||
**Which instructions.** By default a program runs on every processor of its system (on x86-64 that means SSE2, so an
|
||||
`f32x8` is done as two halves). `--cpu native` (or `cpu = "native"` in `finch.toml`) lets LLVM use everything the processor
|
||||
that builds it has: AVX2 and FMA, AVX-512, or NEON's newer parts on ARM. Such a program needs a processor like that one to
|
||||
run. To pick a level instead: `--cpu x86-64-v3` (AVX2, most PCs since 2015), `--cpu x86-64-v4` (AVX-512), `--cpu apple-m1`.
|
||||
### Masks: comparing, choosing, the end of a list
|
||||
|
||||
Comparing vectors compares every lane, so the answer is a **mask**: one `true`/`false` a lane, of type `boolx8` (for 8 lanes).
|
||||
|
||||
```c
|
||||
v := f32x8(-2.0, -1.0, 0.0, 1.0, 2.0, 3.0, -4.0, 5.0)
|
||||
m := v > 0.0 // boolx8(false, false, false, true, true, true, false, true)
|
||||
relu := select(m, v, 0.0) // v where the mask is true, 0 elsewhere
|
||||
print(m.count(), m.any(), m.all()) // 4 true false
|
||||
if (v == w).all() { ... } // a condition needs one bool: all() or any()
|
||||
```
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| `v < w`, `<=`, `>`, `>=`, `==`, `!=` | a mask (a plain number on one side counts for every lane) |
|
||||
| `select(m, a, b)` | lane by lane: `a` where `m` is true, `b` where false (either may be a plain number) |
|
||||
| `m.any()`, `m.all()`, `m.count()` | is any lane true, are all, how many |
|
||||
| `m & n`, `m \| n`, `m ^ n`, `!m` | masks combined |
|
||||
| `m[i]` | one lane, as a `bool` |
|
||||
| `i32x8(m)`, `boolx8(v)` | true is 1; a lane that isn't 0 is true |
|
||||
|
||||
**The end of a list.** A list is rarely a whole number of vectors long. Two forms touch only the lanes that are inside it:
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| `f32x8(list, i, fill)` | `list[i]` … as far as the list goes; the lanes past its end are `fill` |
|
||||
| `v.store_part(list, i)` | writes only the lanes that fit in the list |
|
||||
|
||||
```c
|
||||
fn add_one([]f32 xs) {
|
||||
i := 0
|
||||
while i < xs.len { // no separate loop for the rest
|
||||
(f32x8(xs, i, 0.0) + 1.0).store_part(xs, i)
|
||||
i += 8
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Moving lanes
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| `v.shuffle(3, 2, 1, 0)` | the lanes with these numbers (written in the code), in this order; as many lanes as numbers |
|
||||
| `v.lo()`, `v.hi()` | the lower and the upper half |
|
||||
| `concat(a, b)` | `a`'s lanes, then `b`'s |
|
||||
| `table.lookup(idx)` | for every lane of `idx`: `table[idx % table.len]`. A byte table (`u8x16`, `i8x16`) is one `pshufb` on x86, `tbl` on ARM |
|
||||
|
||||
Unpacking 4-bit numbers (two to a byte, as quantized models store them) is shuffle, mask and lookup:
|
||||
|
||||
```c
|
||||
fn unpack4(u8x8 packed, i8x16 meaning) -> i8x16 {
|
||||
pairs := packed.shuffle(0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7) // each byte twice
|
||||
odd := u8x16(0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1) == 1
|
||||
nibbles := select(odd, pairs >> 4, pairs & 15) // low nibble, then high
|
||||
return meaning.lookup(nibbles) // 0..15 -> what it stands for
|
||||
}
|
||||
```
|
||||
|
||||
### Int8 dot products: dot_add
|
||||
|
||||
`acc.dot_add(a, b)` is the step of a quantized model's matrix multiply. `acc` is an `i32x8` of sums; `a` and `b` have four
|
||||
times as many bytes (`u8x32` and `i8x32`; `i8` × `i8` and `u8` × `u8` work too). Each lane of `acc` gains the sum of four
|
||||
products, `a[4k]*b[4k] + … + a[4k+3]*b[4k+3]`, exactly (no saturation), and the sums stay in the vector:
|
||||
|
||||
```c
|
||||
acc := i32x8(0)
|
||||
i := 0
|
||||
while i + 32 <= weights.len {
|
||||
acc = acc.dot_add(u8x32(activations, i), i8x32(weights, i))
|
||||
i += 32
|
||||
}
|
||||
total := acc.sum() // once, at the end
|
||||
```
|
||||
|
||||
It is one instruction where the processor has one: `vpdpbusd` with AVX-VNNI (Intel since Alder Lake, AMD Zen 5) or
|
||||
AVX-512 VNNI, `usdot`/`sdot`/`udot` on ARM with the dot-product features. Elsewhere it is a few instructions
|
||||
(`pmaddwd` and adds), with the same result. Which one is used depends on `--cpu` or the function's `@cpu` (below).
|
||||
|
||||
### Choosing the instructions: --cpu, @cpu and cpu_has
|
||||
|
||||
By default a program runs on every processor of its system (on x86-64 that means SSE2, so an `f32x8` is done as two
|
||||
halves). Three ways to use more:
|
||||
|
||||
**`--cpu`, for the whole program.** `--cpu native` (or `cpu = "native"` in `finch.toml`) lets LLVM use everything the
|
||||
processor that builds it has: AVX2 and FMA, AVX-512, VNNI, or NEON's newer parts on ARM. The program then needs a processor
|
||||
like that one. To pick a level instead: `--cpu x86-64-v3` (AVX2, most PCs since 2015), `--cpu x86-64-v4` (AVX-512),
|
||||
`--cpu alderlake`, `--cpu apple-m1`, `--cpu neoverse-v1`.
|
||||
|
||||
**`@cpu`, for one function, and `cpu_has` to choose at run time.** One program can carry an AVX-512 version, an AVX2
|
||||
version and a plain one, and pick the best the computer it runs on has:
|
||||
|
||||
```c
|
||||
@cpu("avx512f,avx512bw,avx512vl,avx512vnni")
|
||||
fn matmul_avx512([]u8 a, []i8 b, []i32 out) { ... } // may use AVX-512 inside
|
||||
|
||||
@cpu("x86-64-v3")
|
||||
fn matmul_avx2([]u8 a, []i8 b, []i32 out) { ... } // a level works too
|
||||
|
||||
fn matmul([]u8 a, []i8 b, []i32 out) {
|
||||
if cpu_has("avx512vnni") && cpu_has("avx512vl") {
|
||||
matmul_avx512(a, b, out)
|
||||
} else if cpu_has("avx2") {
|
||||
matmul_avx2(a, b, out)
|
||||
} else {
|
||||
matmul_plain(a, b, out)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- The features are x86's `sse3 ssse3 sse4.1 sse4.2 popcnt avx avx2 fma f16c bmi1 bmi2 avx512f avx512bw avx512dq avx512vl
|
||||
avx512vnni avx512bf16 avx512fp16 avxvnni`, the levels `x86-64-v2`/`v3`/`v4`, and ARM's `neon fp16 dotprod i8mm bf16 sve sve2`.
|
||||
- Names of the other architecture are left out when building, so the x86 and ARM versions sit side by side in one file
|
||||
(and `cpu_has` says `false` for them).
|
||||
- Call a `@cpu` function only after `cpu_has` said yes: on a processor without the feature it stops with *Illegal instruction*.
|
||||
- A vector wider than 16 bytes (`f32x8`, `i32x16` …) can't be passed between functions with different `@cpu`: the two
|
||||
sides would expect it in different registers. Pass the list (or a `ptr`) instead; Finch says so if you try.
|
||||
|
||||
Lanes wider than the processor's registers are split by LLVM, and the same code builds for x86-64 and ARM64 alike.
|
||||
`v.sum()` on float lanes adds them in whatever order is fastest, so the last digits can differ from adding them one by one.
|
||||
|
||||
On NixOS, Nix's clang wrapper removes `-march=native` inside `nix-shell`. Finch tells the wrapper to keep it, so
|
||||
`--cpu native` works there too; if it ever has no effect, Finch warns.
|
||||
|
|
|
|||
Loading…
Reference in a new issue