diff --git a/README.md b/README.md index 2480e4d..58a0f18 100644 --- a/README.md +++ b/README.md @@ -14,10 +14,17 @@ encoder does the hard work, so the decoder only follows instructions. and natural textures — the price of having no entropy coder. - **Made for games and tools too**: mipmaps, GPU format hint, sRGB / premultiplied flags, and random access to 64×64 tiles. +- **A real texture format (YAPF 2)**: GPU-ready BC1–BC7 / BC6H / ETC2 / + ASTC blocks, 16-bit and float pixels, cubemaps, arrays and volumes, + channel meanings (normal X/Y, roughness, AO, …), coverage-preserving + mips, small-mips-first streaming, per-tile checksums, notes and + animation — see [Textures](#textures-yapf-2). - **Tiny and portable**: two C99 files, no dependencies. Plus a CLI, Python, JavaScript and plugins for the tools you already use. -This is **YAPF version 1**, the first stable format. +Two format versions: **version 1** (an 8-bit image, the first stable +format) and **version 2** (the texture container). Plain 8-bit images are +still written as version 1, so every existing YAPF reader keeps working. --- @@ -67,7 +74,64 @@ yapf compare photo.png ui.png # size and decode speed vs the original fo ``` Options when writing `.yapf`: `--mips`, `--linear` (pixels are not sRGB), -`--premult` (alpha is premultiplied). +`--premult` (alpha is premultiplied), and the texture options below. + +--- + +## Textures (YAPF 2) + +Version 2 turns YAPF into a texture container for engines, designed our own +way rather than copied from KTX2 or DDS (the design and the full +specification are in [docs/FORMAT.md](docs/FORMAT.md)): + +| Need | YAPF 2 | +| :--- | :----- | +| **GPU-compressed payloads** | BC1, BC2, BC3, BC4, BC5, BC6H, BC7, ETC2 / EAC and ASTC 4×4–12×12 blocks, stored as the GPU takes them. Built-in encoders for BC1, BC3, BC4, BC5, BC7 and BC6H; or bring blocks from your own encoder. | +| **More than 8 bits** | lossless 16-bit, half and float pixels, coded as byte planes with the same fast machinery | +| **Cubes, arrays, volumes** | cubemaps and cube arrays (reflection probes, skies), texture arrays (terrain layers), 3D textures (colour-grading LUTs) | +| **Channel meaning** | per channel: normal X/Y/Z, roughness, metalness, AO, height, opacity, …; OpenGL or DirectX normal convention; linear channels inside sRGB images | +| **Streaming** | small mips are stored first: a file prefix is a complete low-resolution texture (`yapf info` prints the prefix sizes); single 64 × 64 tiles decode on their own for virtual texturing | +| **Alpha-tested mips** | `--coverage REF` keeps fences and foliage from thinning out with distance; the filter used is recorded | +| **Entropy coding (optional)** | `--entropy`: shared per-surface Huffman code books with a neighbourhood context — about 10 % smaller photos, about 3× smaller GPU blocks on disk; tiles stay independent | +| **Metadata** | notes (author, licence, source, tool, ICC profile, orientation, DPI, HDR light levels, your own tags), colour primaries and transfer (sRGB, Display P3, BT.2020, PQ, HLG, …) | +| **Integrity** | yapf-sum checksums on the header, tables and every tile; *needs* bits make an older decoder refuse cleanly what it cannot read | +| **Animation** | frames with per-frame durations and a loop count | + +```sh +yapf albedo.png albedo.yapf --format bc7 --mips # GPU-ready colour +yapf normal.png normal.yapf --format bc5 --mips --normal dx # normal map, DirectX Y +yapf orm.png orm.yapf --map ao,rough,metal --linear --mips # packed masks +yapf foliage.png foliage.yapf --format bc7 --coverage 128 # alpha-tested mips +yapf sky.hdr sky.yapf --format bc6h --mips # HDR, no RGBM tricks +yapf sky.hdr sky_lossless.yapf # lossless half float +yapf pack cube px.png nx.png py.png ny.png pz.png nz.png probe.yapf --mips +yapf pack array grass.png rock.png sand.png terrain.yapf --format bc7 --mips +yapf pack volume lut_00.png lut_01.png ... grading.yapf --linear +yapf pack anim f0.png f1.png f2.png fx.yapf --frame-ms 50 +yapf albedo.png albedo.yapf --entropy --author "Keel" --license CC-BY-4.0 +yapf info albedo.yapf # everything in the header, notes, stream prefix sizes +yapf verify albedo.yapf # check every checksum +yapf probe.yapf face2.png --face 2 --mip 1 # export one surface +``` + +In C: + +```c +int err; +yapf_texture_t *t = yapf_tex_load("albedo.yapf", 0, 0, &err); +for (uint32_t m = 0; m < t->mip_levels; m++) { + uint32_t s = yapf_tex_surface(t, 0, 0, 0, m), w, h; + yapf_tex_mip_size(t, m, &w, &h, NULL); + if (t->payload == YAPF_PAYLOAD_BLOCKS) /* e.g. YAPF_GPU_BC7 */ + glCompressedTexImage2D(GL_TEXTURE_2D, m, gl_format(t->format, t->flags), + w, h, 0, (GLsizei)yapf_tex_surface_bytes(t, m), t->data[s]); +} +yapf_tex_free(t); +``` + +The language bindings and the JavaScript port speak version 1; the C +library's v1 functions also read 8-bit version 2 files (2D, cube face 0, +array layer 0), so plugins built on them open those too. --- @@ -198,6 +262,47 @@ void yapf_free(yapf_image_t *img); ``` Releases memory returned by the loaders. Passing `NULL` is safe. +### Textures (version 2) + +```c +yapf_texture_t *yapf_tex_load(const char *file, int flags, int threads, int *err); +yapf_texture_t *yapf_tex_load_memory(const void *buf, size_t size, int flags, int threads, int *err); +int yapf_tex_save(const char *file, const yapf_texture_t *t, int flags); +int yapf_tex_encode(const yapf_texture_t *t, int flags, void **out, size_t *size); +yapf_texture_t *yapf_tex_create(const yapf_texture_t *desc); /* zeroed storage */ +void yapf_tex_free(yapf_texture_t *t); +``` +A texture describes its shape (width, height, depth, layers, frames, +cube / volume, mips), storage (pixels with u8 / u16 / half / float samples, +or GPU blocks), meaning (channel map, linear mask, normal convention, +primaries, transfer, mip filter, alpha reference) and holds `data[s]` per +surface `s = yapf_tex_surface(t, frame, layer, face, mip)`. Load flag +`YAPF_LOAD_TRUST` skips checksum checks; encode flag `YAPF_ENC_ENTROPY` +allows the entropy stage. + +```c +yapf_texture_t *yapf_tex_open(const void *buf, size_t avail, int flags, int *err); +uint64_t yapf_tex_stream_bytes(const yapf_texture_t *t, uint32_t first_mip); +int yapf_tex_decode_surface(yapf_texture_t *t, const void *buf, size_t avail, uint32_t s, int threads); +int yapf_tex_decode_tile(const yapf_texture_t *t, const void *buf, size_t avail, uint32_t s, + uint32_t tx, uint32_t ty, uint32_t slice, void *out, size_t stride, + uint32_t *x, uint32_t *y, uint32_t *w, uint32_t *h); +``` +Streaming and virtual texturing: open a file from its first bytes, ask how +many bytes hold mip *m* and smaller, decode surfaces as their bytes arrive +(`YAPF_ERR_TRUNCATED` until then), or decode single tiles. + +```c +int yapf_tex_build_mips(yapf_texture_t *t, uint32_t levels, uint8_t alpha_ref); +int yapf_tex_compress(const yapf_texture_t *src, uint8_t format, yapf_texture_t **out); +int yapf_blocks_decode(uint8_t format, const void *blocks, uint32_t w, uint32_t h, void *out); +int yapf_tex_add_note(yapf_texture_t *t, const char tag[4], const void *data, uint32_t size); +const yapf_note_t *yapf_tex_find_note(const yapf_texture_t *t, const char tag[4]); +``` +Mip chains (linear-light sRGB, optional coverage preservation), the +built-in BCn encoders, BC1–BC7 decoding for previews, and notes. The +complete list, with sizes and helpers, is in [`yapf.h`](yapf.h). + ### Return codes | Constant | Value | Meaning | @@ -206,6 +311,10 @@ Releases memory returned by the loaders. Passing `NULL` is safe. | `YAPF_ERR_INVALID` | –1 | NULL or logically invalid argument | | `YAPF_ERR_IO` | –2 | File could not be opened or written | | `YAPF_ERR_OOM` | –3 | Out of memory | +| `YAPF_ERR_CORRUPT` | –4 | Corrupt or not a YAPF file | +| `YAPF_ERR_UNSUPPORTED` | –5 | Needs a feature this decoder lacks | +| `YAPF_ERR_CHECKSUM` | –6 | A checksum does not match (v2) | +| `YAPF_ERR_TRUNCATED` | –7 | The data is not (yet) all there | ### Channel constants @@ -256,7 +365,8 @@ domain [stb](https://github.com/nothings/stb) headers in `third_party/`. 7. **Independent tiles** — any tile can be decoded on its own, in any order, on any thread. -The complete specification is the comment at the top of [`yapf.h`](yapf.h): +The version 1 specification is the comment at the top of [`yapf.h`](yapf.h); +version 2 (textures) is specified in [docs/FORMAT.md](docs/FORMAT.md). Version 1: ``` HEADER (20 bytes) "YAPF", version 1, channels, GPU hint, mip count, @@ -273,7 +383,7 @@ TILE (64×64) header byte: stored raw, or constant-plane mask + colour tr ## Testing ```sh -make test # C suite: 786 checks, plus JavaScript if node is installed +make test # C suite: 2631 checks, plus JavaScript if node is installed node bindings/js/test.js # JavaScript: round-trips, sample file, corrupt input ``` @@ -286,6 +396,16 @@ What has been verified: C++, and under AddressSanitizer, UndefinedBehaviorSanitizer and ThreadSanitizer; - thousands of corrupted and truncated files are rejected without crashing; +- version 2: lossless round-trips of u8 / u16 / half / float pixels with 1–4 + channels, with and without entropy coding; cubes, cube arrays, arrays, + volumes and animations with notes and every description field; GPU block + payloads; checksum, needs-bit and truncation handling; streaming from file + prefixes and single-tile decoding; BCn encoder quality; linear-light and + coverage-preserving mips; fuzzing of entropy-coded and packed-block files; +- the BC1–BC7 decoders were checked against an independent reference decoder + (bcdec) on 300,000 random blocks: BC6H and BC7 bit-exact, BC1–BC5 within + the ±1 rounding the formats allow (a development check, not part of + `make test`); - every language binding runs its example; JavaScript, Python, C++, C#, Rust, Go, Java and Lua re-encode the sample to byte-identical output; - the GIMP plugin in GIMP 3.0.8 and the Blender extension in Blender 5.1; @@ -293,14 +413,20 @@ What has been verified: **Not yet verified:** running on a Mac (macOS builds are cross-compiled only), Windows on ARM, and running inside Photoshop, VS Code, Figma or a -live Dolphin / GNOME file manager. +live Dolphin / GNOME file manager. The version 2 code has been run on +Linux x64 (with and without SIMD, as C and C++, under ASan, UBSan and +TSan); its NEON path is compiled for ARM64 but has not been run yet, and +the Windows builds have not been rerun. ETC2 / EAC / ASTC blocks are +stored and loaded, but there is no built-in encoder or preview decoder +for them. --- ## Project layout ``` -yapf.h, yapf.c the library (format spec in yapf.h) +yapf.h, yapf.c the library (v1 spec in yapf.h) +docs/FORMAT.md the full format specification, version 2 tools/yapf_cli.c the `yapf` command-line tool tools/build_release.sh builds every download for all platforms tests/test_yapf.c C test suite diff --git a/docs/FORMAT.md b/docs/FORMAT.md new file mode 100644 index 0000000..1099f1e --- /dev/null +++ b/docs/FORMAT.md @@ -0,0 +1,450 @@ +# YAPF file format + +This is the complete specification of YAPF files. `yapf.c` is the +reference implementation; where this text and the code disagree, that is a +bug in one of them. + +YAPF has two versions: + +| Version | What it holds | Who writes it | +| :------ | :------------ | :------------ | +| **1** | one 8-bit image (1–4 channels) with optional mips | `yapf_save` / `yapf_encode`, and the CLI for plain 8-bit images | +| **2** | a texture: GPU blocks or lossless pixels in 8/16-bit or float, cubes, arrays, volumes, animation, channel meanings, colour description, notes, checksums, optional entropy coding | `yapf_tex_save` / `yapf_tex_encode`, and the CLI whenever a v2 feature is used | + +Version 1 is specified at the top of [`yapf.h`](../yapf.h) and is not +repeated here, except that version 2 reuses its tiles, filters and plane +payloads (the "group form") byte for byte. + +--- + +## Design: how YAPF 2 does things its own way + +YAPF 2 is not a wrapper around someone else's container. A few ideas +carry the whole format: + +1. **Everything is tiles.** Pixels are cut into 64 × 64 tiles, GPU + blocks into tiles of 16 × 16 blocks. Each tile has its own size and + checksum in a table, so any tile can be found, checked and decoded on its + own: for multithreaded loading, for partial updates, and for virtual + texturing / "megatexture" streaming. +2. **Small mips first.** Surface data is stored from the smallest mip + to the largest. The first *N* bytes of a file are a complete, lower + resolution texture, so an engine can show something after reading a + few kilobytes and stream in the large mips later. The header carries + the file size, so a loader knows how much is still to come. +3. **Needs, not versions.** New capabilities are announced by *needs* + bits. A decoder refuses a file with a needs bit it does not know, and + can ignore everything else safely. No more version bumps that break + every reader. +4. **Wide samples are byte planes.** 16-bit and float samples are split + into byte planes (high bytes, low bytes) and coded with the same fast + 8-bit machinery as version 1: smooth high bytes compress very well, and + the decoder stays branch-light and SIMD friendly. +5. **Shared code books.** The optional entropy stage keeps its Huffman + tables once per surface, keyed by a small neighbourhood context, so + tiles stay independent and their own headers stay tiny. For GPU + blocks the tables are per byte lane, so endpoint bytes and index bytes + are coded separately. +6. **Meaning travels with the data.** Channel meanings, the normal-map + convention, linear channels inside sRGB images, colour primaries and + transfer, and how the mips were made are header fields, not file-name + suffixes. + +--- + +## Conventions + +All integers are little-endian. `u8`, `u16`, `u32` and `u64` are unsigned +integers of that many bits. Offsets are from the start of the file. + +### yapf-sum + +The checksum of a byte string of length *n*. `K = 0x9E3779B97F4A7C15`; +all arithmetic is modulo 2⁶⁴. + +``` +lane[i] = (i + 1)·K XOR n for i = 0, 1, 2, 3 +for each 8-byte word w_j of the input (little-endian; the last word is +padded with zero bytes): + a = lane[j mod 4] + a = (a XOR w_j) · K + a = a XOR (a >> 32) + lane[j mod 4] = a +h = lane[0] XOR rotl(lane[1], 16) XOR rotl(lane[2], 32) XOR rotl(lane[3], 48) +h = h · K +h = h XOR (h >> 32) +result = low 32 bits of h +``` + +Test vectors: `""` → `0xd7550ed0`, `"YAPF"` → `0xc8cc41d4`, +`"The quick brown fox jumps over the lazy dog"` → `0xbce4f410`. + +--- + +## Version 2 layout + +``` +HEADER 64 bytes +SURFACE TABLE 16 bytes per surface +FRAME TABLE 4 bytes per frame, only when frames > 1 +NOTES notes size bytes +SURFACE DATA in stream order: smallest mip first +``` + +The *front matter* is everything before the surface data: +`64 + 16·surfaces + (frames > 1 ? 4·frames : 0) + notes size` bytes. + +### Header + +| Offset | Size | Field | Notes | +| -----: | ---: | :---- | :---- | +| 0x00 | 4 | magic | `"YAPF"` | +| 0x04 | 1 | version | 2 | +| 0x05 | 1 | channels | 1–4 | +| 0x06 | 1 | format | block format (block payload) or GPU hint (pixel payload); see [GPU formats](#gpu-formats) | +| 0x07 | 1 | mip levels | 1–16 | +| 0x08 | 1 | flags | bit 0 premultiplied alpha, bit 1 sRGB, bit 2 coverage-preserving mips; other bits 0 | +| 0x09 | 1 | payload | 0 pixels (lossless tiles), 1 GPU blocks | +| 0x0A | 1 | sample | pixel payload: 0 u8, 1 u16, 2 half float, 3 float. Block payload: type of the decoded texels (0, or 2 for BC6H) | +| 0x0B | 1 | shape | 0 2D, 1 cube, 2 volume | +| 0x0C | 4 | width | 1–65535 | +| 0x10 | 4 | height | 1–65535; cubes: width = height | +| 0x14 | 4 | depth | volumes: 1–65535; otherwise 1 | +| 0x18 | 2 | layers | ≥ 1 (array layers; with a cube, a cube array); volumes: 1 | +| 0x1A | 2 | frames | ≥ 1 (animation frames) | +| 0x1C | 4 | needs | see [Needs](#needs) | +| 0x20 | 4 | channel map | meaning of channels 0–3, see [Channel meanings](#channel-meanings) | +| 0x24 | 1 | linear mask | bit *c*: channel *c* is linear even when the sRGB flag is set | +| 0x25 | 1 | normal | bits 0–1 convention (0 none, 1 OpenGL +Y, 2 DirectX −Y); bits 2–3 encoding (0 XYZ, 1 XY with Z reconstructed, 2 octahedral) | +| 0x26 | 1 | primaries | 0 unspecified, 1 BT.709/sRGB, 2 Display P3, 3 BT.2020, 4 ACES AP0, 5 ACEScg (AP1), 6 Adobe RGB | +| 0x27 | 1 | transfer | 0 unspecified (the sRGB flag decides), 1 sRGB, 2 linear, 3 gamma 2.2, 4 PQ (ST 2084), 5 HLG, 6 BT.1886 | +| 0x28 | 1 | mip filter | bits 0–6: 0 unknown, 1 box, 2 tent, 3 Kaiser, 4 Lanczos, 5 Mitchell, 6 authored; bit 7: sRGB channels were filtered in linear light | +| 0x29 | 1 | alpha ref | with the coverage flag: alpha-test reference 0–255 | +| 0x2A | 2 | loop count | animation repeats, 0 = forever | +| 0x2C | 4 | notes size | bytes of the notes area | +| 0x30 | 8 | file size | total bytes of the file | +| 0x38 | 4 | tables check | yapf-sum of bytes [0x40, front matter end) | +| 0x3C | 4 | header check | yapf-sum of bytes [0x00, 0x3C) | + +The header check covers the tables check, so the whole front matter is +protected. + +### Surfaces + +A *surface* is one mip level of one face of one layer of one frame. With +`faces = 6` for cubes (order +X, −X, +Y, −Y, +Z, −Z) and 1 otherwise, +surface number + +``` +s = ((frame · layers + layer) · faces + face) · mip_levels + mip +``` + +Mip *m* is `max(1, width >> m) × max(1, height >> m)`, and for volumes +`max(1, depth >> m)` slices deep. A volume surface holds all its slices. + +**Surface table entry** (16 bytes, one per surface, in surface order): + +| Offset | Size | Field | +| -----: | ---: | :---- | +| +0 | 8 | offset of the surface data | +| +8 | 4 | size of the surface data | +| +12 | 4 | tile-table check: yapf-sum of the surface's book size, book and tile table | + +**Stream order.** Writers store surface data for mip `mip_levels − 1` +first (all frames, layers and faces in surface order), then the next larger +mip, and so on, ending with mip 0. Readers must use the offsets and must not +rely on the order. The number of leading bytes that hold every surface of +mip *m* and smaller is the largest `offset + size` over those surfaces. + +### Frame table + +When `frames > 1`: one `u32` per frame, its display time in microseconds. + +### Notes + +A sequence of entries that exactly fills the notes area: + +``` +tag 4 bytes +length u32 +data length bytes +``` + +| Tag | Content | +| :-- | :------ | +| `AUTH` | author (UTF-8) | +| `LICN` | licence (UTF-8, e.g. an SPDX id) | +| `SRCE` | source / origin (UTF-8, e.g. a URL or path) | +| `TOOL` | the tool that wrote the file (UTF-8) | +| `TITL` | title (UTF-8) | +| `DESC` | description (UTF-8) | +| `DATE` | date, ISO 8601 (UTF-8) | +| `ICCP` | an ICC colour profile | +| `ORNT` | orientation, 1 byte: bits 0–1 clockwise quarter turns, bit 2 mirror left–right after turning | +| `DPI ` | 2 × u32: horizontal and vertical dots per inch × 1000 | +| `CLL ` | 2 × u16: maximum content light level, maximum frame-average light level (cd/m²) | + +Tags that begin with a lowercase letter belong to applications (for +example `keel`). Readers ignore tags they do not know. Tags may repeat; +order is kept. + +### Needs + +| Bit | Name | Set when | +| --: | :--- | :------- | +| 0 | WIDE | pixel payload with sample ≠ u8 | +| 1 | BLOCKS | block payload | +| 2 | ENTROPY | any surface has a code book | +| 3 | LAYOUT | shape ≠ 2D, or layers > 1 | +| 4 | FRAMES | frames > 1 | + +A decoder must refuse a file that sets a needs bit it does not implement +(this library: any bit above 4). A file that uses a feature without its +bit is corrupt. Everything that is not a needs feature — channel map, +colour fields, mip filter, notes, frame times — may be ignored. + +A decoder that only implements 8-bit 2D pixels can therefore read every +version 2 file whose needs are 0, and cleanly refuse the rest. + +--- + +## Surface data + +``` +book size u32 0 when the surface has no code book +book book size bytes +tile table 8 bytes per tile: u32 size (bit 31: packed), u32 yapf-sum of the tile bytes +tiles back to back; their sizes must add up exactly to the rest of the surface +``` + +Tiles are numbered slice by slice, rows top to bottom, tiles left to right. +For pixels the grid is `ceil(w / 64) × ceil(h / 64)` tiles per slice (edge +tiles are smaller, no padding); for blocks it is +`ceil(bx / 16) × ceil(by / 16)` tiles of 16 × 16 blocks, where +`bx = ceil(w / block width)` and `by = ceil(h / block height)`. + +Bit 31 of a tile size (*packed*) is allowed only in block tiles of a +surface with a code book. + +--- + +## Pixel tiles + +Let *B* be the bytes per sample (1, 2 or 4), *ch* the channels, and the +tile `tw × th` pixels. + +### Tile header + +``` +byte 0 0x80 stored: the raw samples follow, interleaved in + channel order, each B bytes little-endian; + exactly tw·th·ch·B bytes + otherwise coded: + bits 0–3 B = 1: bit c set = plane c is constant; B > 1: 0 + bit 4 colour transform subtract-green + bit 5 an entropy mask follows + bit 6 no colour transform + bit 7 0 + bits 4 and 6 are exclusive and need ch >= 3; with neither + set, an RGB(A) tile uses YCoCg-R +B > 1 u16 constant mask, bit p = plane p is constant +bit 5 entropy mask: u8 (B = 1) or u16 (B > 1); bit p = plane p is + entropy coded. It must not include constant planes. +then one value byte per constant plane, in plane order; + one payload per other plane, in plane order; + the tile ends exactly where its last payload ends +``` + +With B = 1 and neither bit 5 nor bit 6 set, a coded tile is exactly a +version 1 tile. + +### Planes + +The colour transform turns R, G, B (+ A) into transformed channels; it is +skipped for 1–2 channels and with bit 6. + +* **YCoCg-R** at the sample width *n* = 8·B bits, all arithmetic mod 2ⁿ, + `>>` an arithmetic shift of the *n*-bit two's complement value, `h = 2ⁿ⁻¹`: + + ``` + forward: o = R − B; t = B + (o >> 1); q = G − t; + Y = t + (q >> 1); Co = o + h; Cg = q + h + inverse: o = Co − h; q = Cg − h; t = Y − (q >> 1); + G = q + t; B = t − (o >> 1); R = B + o + ``` +* **subtract-green**: `G, R − G + h, B − G + h` (+ A), mod 2ⁿ. + +Before the transform, float samples (half or float) are mapped to +integers that keep their order: if the sign bit is set every bit is +flipped, otherwise the sign bit is set. The decoder undoes this after the +inverse transform. + +Transformed channel *c* gives B planes: plane `c·B + k` holds byte +`B − 1 − k` of the value (the most significant byte first). For B = 1 +the planes are the transformed channels. + +### Plane payloads + +A plane payload has one of two forms, chosen per plane by the entropy mask. + +**Group form** — exactly the version 1 plane payload: a filter byte per +band of 8 rows, 4-bit control codes, and groups of 8 residuals at the +group's bit width. See `yapf.h`. + +**Entropy form** + +``` +filter[ceil(th / 8)] as in the group form +u16 L bytes of the bit stream +L bytes the folded residuals of the plane in raster order +``` + +Each residual is coded with table `(slot, context)` of the surface's code +book: + +* `slot = t · 16 + p`, where *p* is the plane and *t* is 1 with + subtract-green, 2 with no transform, else 0; +* `context = min(6, floor(log2(s + 1)))`, `s = left + above`, the folded + residuals of the left and upper neighbours in this plane (0 outside the + tile) — so 0, 1–2, 3–6, 7–14, 15–30, 31–62, 63 and up. + +The stream must end in its last byte. Folded residuals then go through +the filters exactly as in the group form. + +--- + +## Block tiles + +A block tile of `tbw × tbh` blocks of *S* bytes is either + +* **raw**: the blocks, rows top to bottom, exactly `tbw·tbh·S` bytes — as + the GPU takes them; or +* **packed** (bit 31 of its size): one bit stream holding byte 0 of every + block (in raster order), then byte 1 of every block, …, byte S − 1. + Byte lane *k* is coded with table `(slot k, context 0)` of the surface's + code book. The stream must end in the last byte of the tile. + +Decoded block surfaces are row-major blocks, slice after slice, ready for +`glCompressedTexImage*` / `vkCmdCopyBufferToImage`. + +--- + +## Code book + +``` +u16 count number of tables, 1–336 +count × { + u8 slot 0–47 + u8 context 0–6 + code table +} +``` + +Each `(slot, context)` pair appears at most once. The book must fill its +size exactly. + +**Code table**: `u8 n − 1`, then *n* 4-bit code lengths (symbols +0 … n − 1, low nibble first, an odd count padded with a zero nibble). +Length 0 = no code; otherwise 1–11. At least one code; Kraft sum ≤ 1. +Codes are canonical — by length, then by symbol, the first code of each +length one more than the last code of the previous length, shifted left — +and stored least significant bit first (the first bit of a code is the +lowest unread bit of the stream), so a decoder can look up 11 bits at a +time. + +--- + +## GPU formats + +Pixel payload hints (`format` field): + +| Value | Name | Vulkan | +| ----: | :--- | :----- | +| 0x00 | RGBA8 | R8G8B8A8_UNORM | +| 0x01 | RGB8 | R8G8B8_UNORM | +| 0x02 | RG8 | R8G8_UNORM | +| 0x03 | R8 | R8_UNORM | +| 0x04 | SRGB8_A8 | R8G8B8A8_SRGB | +| 0x05 | SRGB8 | R8G8B8_SRGB | +| 0x06–0x09 | RGBA16, RGB16, RG16, R16 | R16G16B16A16_UNORM … R16_UNORM | +| 0x0A–0x0D | RGBA16F, RGB16F, RG16F, R16F | R16G16B16A16_SFLOAT … R16_SFLOAT | +| 0x0E–0x11 | RGBA32F, RGB32F, RG32F, R32F | R32G32B32A32_SFLOAT … R32_SFLOAT | + +Block formats (block payload). With the sRGB flag, use the `_SRGB` +variant where one exists. + +| Value | Name | Block | Bytes | Vulkan / DXGI | +| ----: | :--- | :---: | ----: | :------------ | +| 0x40 | BC1 | 4×4 | 8 | BC1_RGB(A)_UNORM / BC1_UNORM | +| 0x41 | BC2 | 4×4 | 16 | BC2_UNORM | +| 0x42 | BC3 | 4×4 | 16 | BC3_UNORM | +| 0x43 | BC4 | 4×4 | 8 | BC4_UNORM | +| 0x44 | BC4_SNORM | 4×4 | 8 | BC4_SNORM | +| 0x45 | BC5 | 4×4 | 16 | BC5_UNORM | +| 0x46 | BC5_SNORM | 4×4 | 16 | BC5_SNORM | +| 0x47 | BC6H | 4×4 | 16 | BC6H_UFLOAT / BC6H_UF16 | +| 0x48 | BC6H_SF | 4×4 | 16 | BC6H_SFLOAT / BC6H_SF16 | +| 0x49 | BC7 | 4×4 | 16 | BC7_UNORM | +| 0x50 | ETC2_RGB8 | 4×4 | 8 | ETC2_R8G8B8_UNORM | +| 0x51 | ETC2_RGB8A1 | 4×4 | 8 | ETC2_R8G8B8A1_UNORM | +| 0x52 | ETC2_RGBA8 | 4×4 | 16 | ETC2_R8G8B8A8_UNORM | +| 0x53 | EAC_R11 | 4×4 | 8 | EAC_R11_UNORM | +| 0x54 | EAC_R11_SNORM | 4×4 | 8 | EAC_R11_SNORM | +| 0x55 | EAC_RG11 | 4×4 | 16 | EAC_R11G11_UNORM | +| 0x56 | EAC_RG11_SNORM | 4×4 | 16 | EAC_R11G11_SNORM | +| 0x60–0x6D | ASTC 4×4, 5×4, 5×5, 6×5, 6×6, 8×5, 8×6, 8×8, 10×5, 10×6, 10×8, 10×10, 12×10, 12×12 | as named | 16 | ASTC_*_UNORM (HDR: transfer linear, sample half) | + +--- + +## Channel meanings + +| Value | Meaning | Value | Meaning | +| ----: | :------ | ----: | :------ | +| 0 | default for the channel count (Y / YA / RGB / RGBA) | 11 | metalness | +| 1 | red | 12 | ambient occlusion | +| 2 | green | 13 | height | +| 3 | blue | 14 | specular | +| 4 | alpha | 15 | emissive | +| 5 | luma | 16 | opacity (alpha-test mask) | +| 6 | normal X | 17 | mask | +| 7 | normal Y | 18 | cavity | +| 8 | normal Z | 19 | thickness | +| 9 | roughness | 20 | id | +| 10 | gloss (1 − roughness) | 255 | unused | + +Alpha (an explicit 4 or 16, or the last channel of a 2- or 4-channel image +with default meanings) is always linear. The linear mask marks further +channels that are linear inside an sRGB image, such as masks packed next to +colour. + +--- + +## Mips + +`mip filter` records how the mips were made. This library's +`yapf_tex_build_mips` uses a 2 × 2 box (2 × 2 × 2 for volumes, edges +repeat), averages sRGB 8-bit channels in linear light (bit 7), and with an +alpha reference rescales alpha on every mip so that the share of texels +with alpha *above* the reference stays that of mip 0 (coverage flag, +`alpha ref`). Alpha-tested foliage and fences then keep their density at +a distance instead of thinning out. + +--- + +## Validation summary + +A decoder rejects a file when: + +* the magic, version, a size, shape or count is out of range, or a cube is + not square; +* the header or tables check does not match (unless the reader was told to + trust the file); +* a needs bit is unknown (*unsupported*), or a feature is used without its + needs bit (*corrupt*); +* a surface lies outside the file, its book is malformed or present without + ENTROPY, or its tiles do not fill it exactly; +* a tile's yapf-sum does not match, or the tile does not decode to exactly + its size (reserved bits, invalid filters, codes, masks or lengths). + +Because every structure is validated before use, corrupt or hostile input +is rejected, never read out of bounds. diff --git a/docs/LANGUAGES.md b/docs/LANGUAGES.md index 7029222..17df03f 100644 --- a/docs/LANGUAGES.md +++ b/docs/LANGUAGES.md @@ -22,6 +22,14 @@ the same native code, so they are all about equally fast; the JavaScript port needs no native library at all. Every example also re-encodes the image and checks that the result is byte-identical to the original file. +**YAPF 2 textures.** The bindings wrap the version 1 API (`yapf_load`, +`yapf_save`, …). Through it they also read 8-bit version 2 files (2D, or +face 0 / layer 0 of cubes and arrays). GPU blocks, 16-bit / float pixels, +volumes and the other version 2 features are available from C and C++ +through the `yapf_tex_*` functions in [`yapf.h`](../yapf.h) (see +[FORMAT.md](FORMAT.md)); any language with a C FFI can call them the same +way. The JavaScript port reads and writes version 1 only. + ## Getting the native library Bindings other than C, C++, Rust, Go, Zig and JavaScript load the shared diff --git a/tests/test_yapf.c b/tests/test_yapf.c index 2412684..000a457 100644 --- a/tests/test_yapf.c +++ b/tests/test_yapf.c @@ -7,6 +7,8 @@ * Round-trips many sizes, channel counts and image kinds, checks mip * chains, single- vs multi-threaded decode, the bundled sample file, and * that corrupted or truncated files are rejected without crashing. + * Version 2: wide samples, shapes, GPU blocks, notes, checksums, needs, + * streaming, tile decode, block encoders and mip generation. * Exit code 0 means every test passed. */ #include "../yapf.h" @@ -14,6 +16,7 @@ #include #include #include +#include /* FNV-1a of the decoded pixels of other/YAPF.YAPF (the YAPF logo). */ #define SAMPLE_HASH 0x8891c2b4u @@ -171,6 +174,494 @@ static uint32_t fnv(const uint8_t *p, size_t n) { return h; } + +/* ====================================================================== + * Version 2 + * ====================================================================== */ + +static yapf_texture_t desc(uint32_t w, uint32_t h, int ch, int sample) { + yapf_texture_t d; + memset(&d, 0, sizeof(d)); + d.width = w; d.height = h; d.depth = 1; d.layers = 1; d.frames = 1; + d.mip_levels = 1; d.channels = (uint8_t)ch; d.sample = (uint8_t)sample; + return d; +} + +/* Fills every surface: noise, gradient or noisy curves, in the sample + * type; block payloads get bytes with some structure. */ +static void tex_fill(yapf_texture_t *t, int kind) { + uint32_t S = yapf_tex_surface_count(t); + for (uint32_t s = 0; s < S; s++) { + uint32_t m = s % t->mip_levels, w, h, d; + yapf_tex_mip_size(t, m, &w, &h, &d); + uint8_t *p = (uint8_t *)t->data[s]; + if (t->payload == YAPF_PAYLOAD_BLOCKS) { + size_t n = yapf_tex_surface_bytes(t, m); + for (size_t i = 0; i < n; i++) + p[i] = kind ? (uint8_t)(i % 16 < 8 ? (i / 64) & 0xFF : rnd()) : (uint8_t)rnd(); + continue; + } + size_t cnt = (size_t)w * h * d * t->channels; + for (size_t i = 0; i < cnt; i++) { + size_t px = i / t->channels; + uint32_t x = (uint32_t)(px % w), y = (uint32_t)((px / w) % h); + int c = (int)(i % t->channels); + double v = kind == 0 ? (rnd() % 1000) / 1000.0 + : kind == 1 ? (x * 3 + y * 5 + c * 40) % 600 / 600.0 + : ((x * x + y * y) / 50 % 700 + rnd() % 5) / 705.0; + switch (t->sample) { + case YAPF_SAMPLE_U8: p[i] = (uint8_t)(v * 255); break; + case YAPF_SAMPLE_U16: ((uint16_t *)p)[i] = (uint16_t)(v * 65535); break; + case YAPF_SAMPLE_F16: ((uint16_t *)p)[i] = yapf_float_to_half((float)(v * 20 - 2)); break; + default: ((float *)p)[i] = (float)(v * 100 - 10) + (kind == 0 ? (float)rnd() : 0.0f); break; + } + } + } +} + +static int tex_equal(const yapf_texture_t *a, const yapf_texture_t *b) { + uint32_t S = yapf_tex_surface_count(a); + if (S != yapf_tex_surface_count(b)) return 0; + for (uint32_t s = 0; s < S; s++) + if (!a->data[s] || !b->data[s] || + memcmp(a->data[s], b->data[s], yapf_tex_surface_bytes(a, s % a->mip_levels)) != 0) + return 0; + return 1; +} + +/* Encode, decode, compare; returns the encoded file (caller frees). */ +static uint8_t *tex_roundtrip(yapf_texture_t *t, int flags, size_t *len, const char *what) { + void *buf = NULL; + *len = 0; + int e = yapf_tex_encode(t, flags, &buf, len); + CHECK(e == YAPF_OK, "v2 encode %s (%d)", what, e); + if (e != YAPF_OK) return NULL; + int err = 0; + yapf_texture_t *a = yapf_tex_load_memory(buf, *len, 0, 1, &err); + yapf_texture_t *b = yapf_tex_load_memory(buf, *len, 0, 0, &err); + CHECK(a && tex_equal(a, t), "v2 roundtrip %s", what); + CHECK(b && tex_equal(b, t), "v2 roundtrip (threads) %s", what); + yapf_tex_free(a); + yapf_tex_free(b); + uint8_t *copy = (uint8_t *)malloc(*len); + memcpy(copy, buf, *len); + yapf_free_buffer(buf); + return copy; +} + +static void v2_pixels(void) { + static const int sizes[][2] = { {1, 1}, {7, 3}, {64, 64}, {65, 63}, {130, 70}, {200, 201} }; + static const char *sn[] = { "u8", "u16", "f16", "f32" }; + for (size_t si = 0; si < sizeof(sizes) / sizeof(sizes[0]); si++) + for (int ch = 1; ch <= 4; ch++) + for (int sample = 0; sample < 4; sample++) + for (int kind = 0; kind < 3; kind++) + for (int fl = 0; fl < 2; fl++) { + yapf_texture_t d = desc((uint32_t)sizes[si][0], (uint32_t)sizes[si][1], ch, sample); + yapf_texture_t *t = yapf_tex_create(&d); + char what[96]; + sprintf(what, "%dx%d ch%d %s kind%d%s", sizes[si][0], sizes[si][1], ch, + sn[sample], kind, fl ? " entropy" : ""); + tex_fill(t, kind); + size_t len; + free(tex_roundtrip(t, fl ? YAPF_ENC_ENTROPY : 0, &len, what)); + yapf_tex_free(t); + } +} + +/* Shapes, frames, notes and every description field survive. */ +static void v2_layout(void) { + struct { int w, h, ch, sample, shape, layers, frames, mips, depth; } c[] = { + { 64, 64, 4, 0, YAPF_SHAPE_CUBE, 1, 1, 7, 1 }, + { 32, 32, 3, 2, YAPF_SHAPE_CUBE, 2, 1, 6, 1 }, + { 40, 30, 4, 0, YAPF_SHAPE_2D, 5, 1, 3, 1 }, + { 33, 17, 1, 1, YAPF_SHAPE_VOLUME, 1, 1, 4, 9 }, + { 50, 50, 4, 0, YAPF_SHAPE_2D, 1, 6, 1, 1 }, + }; + for (size_t i = 0; i < sizeof(c) / sizeof(c[0]); i++) { + yapf_texture_t d = desc((uint32_t)c[i].w, (uint32_t)c[i].h, c[i].ch, c[i].sample); + d.shape = (uint8_t)c[i].shape; d.layers = (uint16_t)c[i].layers; d.frames = (uint16_t)c[i].frames; + d.mip_levels = (uint8_t)c[i].mips; d.depth = (uint32_t)c[i].depth; + d.channel_map[0] = YAPF_CH_NORMAL_X; d.channel_map[1] = YAPF_CH_ROUGHNESS; + d.linear_mask = 0x08; d.normal = YAPF_NORMAL_DX | YAPF_NORMAL_XY; + d.primaries = YAPF_PRIMARIES_DISPLAY_P3; d.transfer = YAPF_TRANSFER_PQ; + d.mip_filter = YAPF_MIP_KAISER; d.alpha_ref = 77; d.loop_count = 3; + d.flags = YAPF_FLAG_SRGB | YAPF_FLAG_COVERAGE; + yapf_texture_t *t = yapf_tex_create(&d); + tex_fill(t, 2); + if (t->frames > 1) for (int f = 0; f < t->frames; f++) t->frame_us[f] = 16667u * (uint32_t)(f + 1); + yapf_tex_add_note(t, "AUTH", "Keel", 4); + yapf_tex_add_note(t, "ICCP", "\0\1\2", 3); + size_t len; + char what[32]; + sprintf(what, "layout case %d", (int)i); + uint8_t *file = tex_roundtrip(t, (int)(i & 1), &len, what); + yapf_texture_t *b = file ? yapf_tex_load_memory(file, len, 0, 1, NULL) : NULL; + int ok = b && b->shape == d.shape && b->layers == d.layers && b->frames == d.frames && + b->depth == d.depth && b->mip_levels == d.mip_levels && + memcmp(b->channel_map, d.channel_map, 4) == 0 && b->linear_mask == 0x08 && + b->normal == d.normal && b->primaries == d.primaries && b->transfer == d.transfer && + b->mip_filter == d.mip_filter && b->alpha_ref == 77 && b->loop_count == 3 && + b->flags == d.flags && b->note_count == 2; + const yapf_note_t *n = b ? yapf_tex_find_note(b, "ICCP") : NULL; + ok = ok && n && n->size == 3 && memcmp(n->data, "\0\1\2", 3) == 0; + if (ok && b->frames > 1) ok = b->frame_us[b->frames - 1] == 16667u * b->frames; + CHECK(ok, "v2 fields, notes and frame times survive (%s)", what); + yapf_tex_free(b); + free(file); + yapf_tex_free(t); + } +} + +static void v2_blocks(void) { + static const uint8_t fmts[] = { YAPF_GPU_BC1, YAPF_GPU_BC7, YAPF_GPU_BC4, YAPF_GPU_ASTC_6x6, + YAPF_GPU_ASTC_12x12, YAPF_GPU_ETC2_RGBA8 }; + for (size_t f = 0; f < sizeof(fmts); f++) + for (int fl = 0; fl < 2; fl++) { + yapf_texture_t d = desc(300, 200, 4, 0); + d.payload = YAPF_PAYLOAD_BLOCKS; d.format = fmts[f]; d.mip_levels = 3; + if (f == 3) { d.shape = YAPF_SHAPE_VOLUME; d.depth = 5; } + yapf_texture_t *t = yapf_tex_create(&d); + tex_fill(t, 1); + char what[48]; + sprintf(what, "blocks 0x%02x%s", fmts[f], fl ? " packed" : ""); + size_t len; + free(tex_roundtrip(t, fl, &len, what)); + yapf_tex_free(t); + } +} + +/* Recomputes both header checks after editing a file. */ +static void reseal(uint8_t *f, size_t front) { + uint32_t a = yapf_checksum(f + 64, front - 64), b; + f[0x38] = (uint8_t)a; f[0x39] = (uint8_t)(a >> 8); f[0x3A] = (uint8_t)(a >> 16); f[0x3B] = (uint8_t)(a >> 24); + b = yapf_checksum(f, 0x3C); + f[0x3C] = (uint8_t)b; f[0x3D] = (uint8_t)(b >> 8); f[0x3E] = (uint8_t)(b >> 16); f[0x3F] = (uint8_t)(b >> 24); +} + +/* Flips, truncation, scribbles: never crash; truncation always fails. */ +static void v2_fuzz(const uint8_t *file, size_t len, const char *what) { + uint8_t *bad = (uint8_t *)malloc(len); + int truncated_ok = 1, err; + for (int it = 0; it < 2000; it++) { + memcpy(bad, file, len); + size_t n = len; + switch (it % 3) { + case 0: for (int k = 0; k < 1 + it % 7; k++) bad[rnd() % len] ^= (uint8_t)(1u << (rnd() % 8)); break; + case 1: n = rnd() % len; break; + default: bad[rnd() % len] = (uint8_t)rnd(); break; + } + int fl = (it & 4) ? YAPF_LOAD_TRUST : 0; + yapf_texture_t *x = yapf_tex_load_memory(bad, n, fl, (it & 1) ? 2 : 1, &err); + if (it % 3 == 1 && x) truncated_ok = 0; + yapf_tex_free(x); + if (it % 50 == 0) yapf_free(yapf_load_memory(bad, n)); + } + CHECK(truncated_ok, "truncated v2 files are rejected (%s)", what); + free(bad); +} + +static void v2_integrity(void) { + yapf_texture_t d = desc(150, 90, 4, YAPF_SAMPLE_F16); + yapf_texture_t *t = yapf_tex_create(&d); + tex_fill(t, 2); + size_t len; + uint8_t *file = tex_roundtrip(t, YAPF_ENC_ENTROPY, &len, "integrity setup"); + if (!file) { yapf_tex_free(t); return; } + uint8_t *bad = (uint8_t *)malloc(len); + size_t front = 64 + 16; /* header + one surface entry */ + int err; + + /* a flipped bit in tile data is caught by its yapf-sum ... */ + memcpy(bad, file, len); + bad[len - 10] ^= 0x10; + CHECK(!yapf_tex_load_memory(bad, len, 0, 1, &err) && err == YAPF_ERR_CHECKSUM, + "corrupt tile data reports YAPF_ERR_CHECKSUM (%d)", err); + yapf_texture_t *trusted = yapf_tex_load_memory(bad, len, YAPF_LOAD_TRUST, 1, &err); + yapf_tex_free(trusted); /* may or may not decode; must not crash */ + /* ... and in the header by the header check */ + memcpy(bad, file, len); + bad[0x0C] ^= 1; + CHECK(!yapf_tex_load_memory(bad, len, 0, 1, &err) && err == YAPF_ERR_CHECKSUM, + "corrupt header reports YAPF_ERR_CHECKSUM"); + /* an unknown needs bit: refused as unsupported */ + memcpy(bad, file, len); + bad[0x1F] |= 0x80; + reseal(bad, front); + CHECK(!yapf_tex_load_memory(bad, len, 0, 1, &err) && err == YAPF_ERR_UNSUPPORTED, + "unknown needs bit is refused (%d)", err); + /* a feature in use but not announced: corrupt */ + memcpy(bad, file, len); + bad[0x1C] &= (uint8_t)~YAPF_NEED_WIDE; + reseal(bad, front); + CHECK(!yapf_tex_load_memory(bad, len, 0, 1, &err) && err == YAPF_ERR_CORRUPT, + "unannounced feature is refused (%d)", err); + /* the old API refuses what it cannot represent */ + CHECK(yapf_load_memory(file, len) == NULL, "yapf_load refuses half-float textures"); + + free(bad); + v2_fuzz(file, len, "half float + entropy"); + free(file); + yapf_tex_free(t); + + /* 8-bit RGBA with entropy planes (decoded several streams at once) */ + d = desc(140, 75, 4, 0); + t = yapf_tex_create(&d); + tex_fill(t, 2); + file = tex_roundtrip(t, YAPF_ENC_ENTROPY, &len, "fuzz setup rgba8"); + if (file) v2_fuzz(file, len, "rgba8 + entropy"); + free(file); + /* BC7 blocks with packed tiles */ + yapf_texture_t *c = NULL; + if (yapf_tex_compress(t, YAPF_GPU_BC7, &c) == YAPF_OK) { + file = tex_roundtrip(c, YAPF_ENC_ENTROPY, &len, "fuzz setup bc7"); + if (file) v2_fuzz(file, len, "packed BC7"); + free(file); + } + yapf_tex_free(c); + yapf_tex_free(t); +} + +/* Streaming: a file prefix holds the small mips; tiles decode alone. */ +static void v2_streaming(void) { + yapf_texture_t d = desc(300, 260, 4, 0); + d.shape = YAPF_SHAPE_CUBE; d.width = d.height = 260; d.mip_levels = 5; + yapf_texture_t *t = yapf_tex_create(&d); + tex_fill(t, 2); + void *buf = NULL; + size_t len = 0; + yapf_tex_encode(t, 0, &buf, &len); + uint8_t *file = (uint8_t *)buf; + int ok = 1, err = 0; + for (uint32_t m = 0; m < 5 && ok; m++) { + yapf_texture_t *probe = yapf_tex_open(file, len, 0, &err); + uint64_t need = probe ? yapf_tex_stream_bytes(probe, m) : 0; + yapf_tex_free(probe); + yapf_texture_t *s = yapf_tex_open(file, (size_t)need, 0, &err); + ok = s != NULL && need <= len && (m > 0 || need == len); + for (uint32_t f = 0; ok && f < 6; f++) + for (uint32_t mm = 0; ok && mm < 5; mm++) { + uint32_t sf = yapf_tex_surface(s, 0, 0, f, mm); + int rc = yapf_tex_decode_surface(s, file, (size_t)need, sf, 1); + if (mm >= m) ok = rc == YAPF_OK && + memcmp(s->data[sf], t->data[sf], yapf_tex_surface_bytes(t, mm)) == 0; + else ok = rc == YAPF_ERR_TRUNCATED; + } + yapf_tex_free(s); + } + CHECK(ok, "stream prefixes decode exactly their small mips"); + + /* tile decode equals the matching part of the full surface */ + yapf_texture_t *s = yapf_tex_open(file, len, 0, &err); + uint32_t sf = yapf_tex_surface(s, 0, 0, 3, 0), tx, ty; + yapf_tex_tile_grid(s, 0, &tx, &ty); + uint8_t tile[64 * 64 * 4]; + ok = s && tx == 5 && ty == 5; + for (uint32_t y = 0; ok && y < ty; y++) + for (uint32_t x = 0; ok && x < tx; x++) { + uint32_t px, py, pw, ph; + ok = yapf_tex_decode_tile(s, file, len, sf, x, y, 0, tile, 64 * 4, &px, &py, &pw, &ph) == YAPF_OK; + for (uint32_t r = 0; ok && r < ph; r++) + ok = memcmp(tile + r * 64 * 4, (uint8_t *)t->data[sf] + ((size_t)(py + r) * 260 + px) * 4, + (size_t)pw * 4) == 0; + } + CHECK(ok, "single tiles decode to the same pixels"); + yapf_tex_free(s); + yapf_free_buffer(buf); + yapf_tex_free(t); +} + +/* v1 and v2 work through either API where the data allows it. */ +static void v2_compat(const char *sample) { + yapf_texture_t *t = yapf_tex_load(sample, 0, 0, NULL); + if (t) { + uint32_t h = fnv((const uint8_t *)t->data[0], yapf_tex_surface_bytes(t, 0)); + CHECK(h == SAMPLE_HASH, "yapf_tex_load reads the v1 sample"); + size_t len; + uint8_t *file = tex_roundtrip(t, YAPF_ENC_ENTROPY, &len, "sample as v2"); + yapf_image_t *img = file ? yapf_load_memory(file, len) : NULL; + CHECK(img && img->width == 512 && img->channels == 4 && + fnv(img->pixels, 512u * 512u * 4u) == SAMPLE_HASH, "yapf_load reads an 8-bit v2 file"); + yapf_free(img); + free(file); + yapf_tex_free(t); + } + /* cube: the v1 API sees face 0 with its mips */ + yapf_texture_t d = desc(40, 40, 3, 0); + d.shape = YAPF_SHAPE_CUBE; d.mip_levels = 3; + yapf_texture_t *c = yapf_tex_create(&d); + tex_fill(c, 1); + size_t len; + uint8_t *file = tex_roundtrip(c, 0, &len, "cube for v1 API"); + yapf_image_t *img = file ? yapf_load_memory_mt(file, len, 2) : NULL; + CHECK(img && img->mip_levels == 3 && memcmp(img->mips[2], c->data[2], 10 * 10 * 3) == 0 && + memcmp(img->pixels, c->data[0], 40 * 40 * 3) == 0, "yapf_load reads face 0 of a cube"); + yapf_free(img); + free(file); + yapf_tex_free(c); +} + +static double psnr8(const uint8_t *a, int sa, const uint8_t *b, int sb, size_t n, int ch) { + double se = 0; + for (size_t i = 0; i < n; i++) + for (int c = 0; c < ch; c++) { + double v = (double)a[i * sa + c] - b[i * sb + c]; + se += v * v; + } + if (se == 0) return 99.0; + return 10.0 * log10(255.0 * 255.0 * (double)(n * ch) / se); +} + +static void v2_gpu(void) { + const uint32_t w = 61, h = 47; /* not multiples of 4 */ + yapf_texture_t d = desc(w, h, 4, 0); + yapf_texture_t *t = yapf_tex_create(&d); + uint8_t *p = (uint8_t *)t->data[0]; + for (uint32_t y = 0; y < h; y++) + for (uint32_t x = 0; x < w; x++) { + uint8_t *q = p + ((size_t)y * w + x) * 4; + q[0] = (uint8_t)(x * 4); q[1] = (uint8_t)(y * 5); q[2] = (uint8_t)(255 - x * 2); + q[3] = (uint8_t)(((x / 8 + y / 8) & 1) ? 255 : 40 + x); + } + static const struct { uint8_t f; int ch; double min; } F[] = { + { YAPF_GPU_BC1, 3, 30 }, { YAPF_GPU_BC3, 4, 30 }, { YAPF_GPU_BC7, 4, 34 }, + { YAPF_GPU_BC4, 1, 38 }, { YAPF_GPU_BC5, 2, 38 } + }; + uint8_t *out = (uint8_t *)malloc((size_t)w * h * 4); + for (size_t i = 0; i < sizeof(F) / sizeof(F[0]); i++) { + yapf_texture_t *c = NULL; + int rc = yapf_tex_compress(t, F[i].f, &c); + int ok = rc == YAPF_OK && c->payload == YAPF_PAYLOAD_BLOCKS && + yapf_blocks_decode(F[i].f, c->data[0], w, h, out) == YAPF_OK; + double q = 0; + if (ok && F[i].f == YAPF_GPU_BC1) { + /* BC1 keeps alpha only as cut-out: compare opaque texels */ + for (size_t k = 0; k < (size_t)w * h; k++) if (p[k * 4 + 3] < 128) memcpy(out + k * 4, p + k * 4, 3); + } + if (ok) q = psnr8(p, 4, out, 4, (size_t)w * h, F[i].ch); + CHECK(ok && q >= F[i].min, "GPU format 0x%02x encodes (PSNR %.1f dB)", F[i].f, q); + if (c) { + size_t len; + char what[32]; + sprintf(what, "compressed 0x%02x", F[i].f); + free(tex_roundtrip(c, YAPF_ENC_ENTROPY, &len, what)); + } + yapf_tex_free(c); + } + CHECK(yapf_blocks_decode(YAPF_GPU_ASTC_4x4, out, 4, 4, out) == YAPF_ERR_UNSUPPORTED, + "ASTC blocks are carried, not decoded"); + free(out); + yapf_tex_free(t); + + /* BC6H: relative error of a smooth HDR sky gradient */ + yapf_texture_t dh = desc(32, 32, 3, YAPF_SAMPLE_F16); + yapf_texture_t *th = yapf_tex_create(&dh), *ch = NULL; + uint16_t *hp = (uint16_t *)th->data[0]; + for (int i = 0; i < 32 * 32; i++) + for (int k = 0; k < 3; k++) hp[i * 3 + k] = yapf_float_to_half((0.3f + 0.4f * (float)k) * + (1.0f + (float)(i % 32) * 0.02f + (float)(i / 32) * 0.15f)); + int ok = yapf_tex_compress(th, YAPF_GPU_BC6H, &ch) == YAPF_OK; + uint16_t back[32 * 32 * 3]; + double worst = 0; + if (ok) ok = yapf_blocks_decode(YAPF_GPU_BC6H, ch->data[0], 32, 32, back) == YAPF_OK; + for (int i = 0; ok && i < 32 * 32 * 3; i++) { + float a = yapf_half_to_float(hp[i]), b = yapf_half_to_float(back[i]); + double r = (a > b ? a - b : b - a) / a; + if (r > worst) worst = r; + } + CHECK(ok && worst < 0.05, "BC6H encodes HDR within 5%% (worst %.4f)", worst); + yapf_tex_free(ch); + yapf_tex_free(th); +} + +static void v2_mips(void) { + /* sRGB: a black / white checker averages to sRGB 188, not 128 */ + yapf_texture_t d = desc(64, 64, 4, 0); + d.flags = YAPF_FLAG_SRGB; + yapf_texture_t *t = yapf_tex_create(&d); + uint8_t *p = (uint8_t *)t->data[0]; + for (int i = 0; i < 64 * 64; i++) { + uint8_t v = ((i % 64 + i / 64) & 1) ? 255 : 0; + p[i * 4] = p[i * 4 + 1] = p[i * 4 + 2] = v; + p[i * 4 + 3] = v; /* alpha stays linear */ + } + int ok = yapf_tex_build_mips(t, 0, 0) == YAPF_OK && t->mip_levels == 7; + const uint8_t *m1 = ok ? (const uint8_t *)t->data[1] : NULL; + CHECK(ok && m1[0] == 188 && m1[3] == 128 && (t->mip_filter & YAPF_MIP_LINEAR_LIGHT), + "sRGB mips average in linear light (got %d, alpha %d)", m1 ? m1[0] : -1, m1 ? m1[3] : -1); + yapf_tex_free(t); + + /* alpha test: speckled foliage (30 % of texels opaque). A plain box + * filter thins it out with distance; with alpha_ref = 128 every mip + * keeps about the same share of texels above 128. */ + double plain[5] = {0}, kept[5] = {0}; + for (int pass = 0; pass < 2; pass++) { + d = desc(128, 128, 4, 0); + t = yapf_tex_create(&d); + p = (uint8_t *)t->data[0]; + rng = 99; + for (int i = 0; i < 128 * 128; i++) p[i * 4 + 3] = rnd() % 100 < 30 ? 255 : 0; + ok = yapf_tex_build_mips(t, 5, pass ? 128 : 0) == YAPF_OK && t->mip_levels == 5; + if (pass) ok = ok && (t->flags & YAPF_FLAG_COVERAGE) && t->alpha_ref == 128; + for (uint32_t m = 1; ok && m < 5; m++) { + uint32_t w, h, n = 0; + yapf_tex_mip_size(t, m, &w, &h, NULL); + const uint8_t *q = (const uint8_t *)t->data[m]; + for (uint32_t i = 0; i < w * h; i++) n += q[i * 4 + 3] > 128; + (pass ? kept : plain)[m] = (double)n / (w * h); + } + if (pass) { + size_t len; + free(tex_roundtrip(t, 0, &len, "mipped texture")); + } + yapf_tex_free(t); + } + for (uint32_t m = 1; ok && m < 5; m++) ok = kept[m] > 0.2 && kept[m] < 0.4; + CHECK(ok && plain[2] < 0.15, "coverage-preserving mips keep alpha-test coverage " + "(mip 2: %.2f, plain box %.2f)", kept[2], plain[2]); + + /* float volume: 2×2×2 box */ + d = desc(4, 4, 1, YAPF_SAMPLE_F32); + d.shape = YAPF_SHAPE_VOLUME; d.depth = 4; + t = yapf_tex_create(&d); + float *f = (float *)t->data[0]; + for (int i = 0; i < 64; i++) f[i] = (float)i; + ok = yapf_tex_build_mips(t, 0, 0) == YAPF_OK && t->mip_levels == 3; + CHECK(ok && ((float *)t->data[1])[0] == (0 + 1 + 4 + 5 + 16 + 17 + 20 + 21) / 8.0f && + ((float *)t->data[2])[0] == 31.5f, "volume mips filter in 3D"); + yapf_tex_free(t); +} + +static void v2_misc(void) { + int ok = 1; + for (uint32_t h = 0; h < 0x10000 && ok; h++) { + if ((h & 0x7C00) == 0x7C00 && (h & 0x3FF)) continue; /* NaN payloads */ + ok = yapf_float_to_half(yapf_half_to_float((uint16_t)h)) == h; + } + CHECK(ok, "every half survives half -> float -> half"); + CHECK(yapf_float_to_half(1.0f) == 0x3C00 && yapf_float_to_half(65520.0f) == 0x7C00 && + yapf_float_to_half(1e-8f) == 0 && yapf_float_to_half(-2.0f) == 0xC000, "float -> half rounding"); + /* fixed vectors pin yapf-sum to the specification */ + CHECK(yapf_checksum("", 0) == 0xd7550ed0u && yapf_checksum("YAPF", 4) == 0xc8cc41d4u && + yapf_checksum("The quick brown fox jumps over the lazy dog", 43) == 0xbce4f410u, + "yapf-sum test vectors"); + /* encoding is deterministic */ + yapf_texture_t d = desc(90, 70, 3, YAPF_SAMPLE_U16); + yapf_texture_t *t = yapf_tex_create(&d); + tex_fill(t, 2); + void *a = NULL, *b = NULL; + size_t la = 0, lb = 0; + yapf_tex_encode(t, YAPF_ENC_ENTROPY, &a, &la); + yapf_tex_encode(t, YAPF_ENC_ENTROPY, &b, &lb); + CHECK(a && la == lb && memcmp(a, b, la) == 0, "v2 encoding is deterministic"); + yapf_free_buffer(a); + yapf_free_buffer(b); + yapf_tex_free(t); + d = desc(10, 12, 4, 0); + d.shape = YAPF_SHAPE_CUBE; + CHECK(yapf_tex_create(&d) == NULL, "a cube must be square"); +} + int main(int argc, char **argv) { static const int sizes[][2] = { {1, 1}, {1, 7}, {7, 1}, {63, 63}, {64, 64}, {65, 65}, {127, 1}, @@ -197,6 +688,16 @@ int main(int argc, char **argv) { printf("note: sample %s not found, skipped\n", sample); } + v2_pixels(); + v2_layout(); + v2_blocks(); + v2_integrity(); + v2_streaming(); + v2_compat(sample); + v2_gpu(); + v2_mips(); + v2_misc(); + printf("%d tests, %d failures\n", tests, fails); return fails != 0; } diff --git a/tools/yapf_cli.c b/tools/yapf_cli.c index effec05..af60926 100644 --- a/tools/yapf_cli.c +++ b/tools/yapf_cli.c @@ -18,20 +18,17 @@ * ------------------------------------------------------------------------- * * yapf [options] convert; formats chosen by extension - * yapf info print header information + * yapf pack [options] + * yapf info print header, tables and notes + * yapf verify check every checksum and decode * yapf bench time decoding * yapf thumbnail [size] small preview (default 256) * yapf compare [more…] size and decode speed: original vs YAPF * - * Reads PNG, JPEG, BMP, TGA, GIF (first frame), PSD (composite), PNM, HDR - * (tone-clamped) and YAPF. Writes YAPF, PNG, BMP, TGA and JPEG. - * - * Options for writing YAPF: - * --mips also store a full mip chain (2×2 box filter) - * --linear mark pixels as linear instead of sRGB - * --premult mark alpha as premultiplied - * Options for reading YAPF: - * --mip N export mip level N instead of the base image + * Reads PNG (8 / 16-bit), JPEG, BMP, TGA, GIF (first frame), PSD, PNM, + * HDR (float) and YAPF. Writes YAPF, PNG, BMP, TGA, JPEG and HDR. + * Plain 8-bit 2D images are written as YAPF version 1; anything that + * needs version 2 (see `yapf` without arguments) is written as version 2. * * Build: cc -O2 -std=c99 -Ithird_party tools/yapf_cli.c yapf.c -o yapf -lm -pthread */ @@ -84,124 +81,532 @@ static double now_s(void) { #endif } -static uint8_t gpu_hint(int ch, int srgb) { - switch (ch) { - case 1: return YAPF_GPU_R8; - case 2: return YAPF_GPU_RG8; - case 3: return srgb ? YAPF_GPU_SRGB8 : YAPF_GPU_RGB8; - default: return srgb ? YAPF_GPU_SRGB8_A8 : YAPF_GPU_RGBA8; - } + +/* ── Options ─────────────────────────────────────────────────────────── */ + +typedef struct { + int mips, srgb, premult, v2, entropy, eight_bit, float32; + int coverage; /* alpha_ref, 0 = off */ + int format; /* block format, -1 = none */ + int normal; /* YAPF_NORMAL_* */ + int linear_alpha; + int primaries, transfer; + int have_map; + uint8_t map[4]; + double frame_ms; + int loops; + /* export selection */ + int mip, layer, face, frame, slice; + const char *author, *license, *source; + const char *notes[16]; + int nnotes; +} opts_t; + +static const struct { const char *name; uint8_t v; } ch_names[] = { + { "r", YAPF_CH_RED }, { "g", YAPF_CH_GREEN }, { "b", YAPF_CH_BLUE }, { "a", YAPF_CH_ALPHA }, + { "luma", YAPF_CH_LUMA }, { "nx", YAPF_CH_NORMAL_X }, { "ny", YAPF_CH_NORMAL_Y }, + { "nz", YAPF_CH_NORMAL_Z }, { "rough", YAPF_CH_ROUGHNESS }, { "gloss", YAPF_CH_GLOSS }, + { "metal", YAPF_CH_METALNESS }, { "ao", YAPF_CH_OCCLUSION }, { "height", YAPF_CH_HEIGHT }, + { "spec", YAPF_CH_SPECULAR }, { "emissive", YAPF_CH_EMISSIVE }, { "opacity", YAPF_CH_OPACITY }, + { "mask", YAPF_CH_MASK }, { "cavity", YAPF_CH_CAVITY }, { "thickness", YAPF_CH_THICKNESS }, + { "id", YAPF_CH_ID }, { "-", YAPF_CH_UNUSED }, { "default", YAPF_CH_DEFAULT } +}; + +static const char *ch_name(uint8_t v) { + for (size_t i = 0; i < sizeof(ch_names) / sizeof(ch_names[0]); i++) + if (ch_names[i].v == v) return ch_names[i].name; + return "?"; } -/* Full mip chain with a 2×2 box filter (edges clamp for odd sizes). */ -static int build_mips(yapf_image_t *img) { - int levels = 1; - while (levels < (int)YAPF_MAX_MIPS && - ((img->width >> levels) > 0 || (img->height >> levels) > 0)) - levels++; - img->mips = (uint8_t **)calloc((size_t)levels, sizeof(uint8_t *)); - if (!img->mips) return -1; - img->mips[0] = img->pixels; - img->mip_levels = (uint8_t)levels; - - int ch = img->channels; - for (int m = 1; m < levels; m++) { - uint32_t pw = img->width >> (m - 1), ph = img->height >> (m - 1); - uint32_t w = img->width >> m, h = img->height >> m; - if (pw < 1) pw = 1; - if (ph < 1) ph = 1; - if (w < 1) w = 1; - if (h < 1) h = 1; - uint8_t *src = img->mips[m - 1]; - uint8_t *dst = (uint8_t *)malloc((size_t)w * h * ch); - if (!dst) return -1; - for (uint32_t y = 0; y < h; y++) - for (uint32_t x = 0; x < w; x++) - for (int c = 0; c < ch; c++) { - uint32_t x0 = 2 * x < pw ? 2 * x : pw - 1, x1 = 2 * x + 1 < pw ? 2 * x + 1 : pw - 1; - uint32_t y0 = 2 * y < ph ? 2 * y : ph - 1, y1 = 2 * y + 1 < ph ? 2 * y + 1 : ph - 1; - unsigned s = src[((size_t)y0 * pw + x0) * ch + c] + src[((size_t)y0 * pw + x1) * ch + c] - + src[((size_t)y1 * pw + x0) * ch + c] + src[((size_t)y1 * pw + x1) * ch + c]; - dst[((size_t)y * w + x) * ch + c] = (uint8_t)((s + 2) / 4); - } - img->mips[m] = dst; - } - return 0; -} - -static int write_image(const char *path, const uint8_t *px, int w, int h, int ch) { - if (ends_with(path, ".png")) return stbi_write_png(path, w, h, ch, px, w * ch); - if (ends_with(path, ".bmp")) return stbi_write_bmp(path, w, h, ch, px); - if (ends_with(path, ".tga")) return stbi_write_tga(path, w, h, ch, px); - if (ends_with(path, ".jpg") || ends_with(path, ".jpeg")) - return stbi_write_jpg(path, w, h, ch, px, 95); - fprintf(stderr, "yapf: unsupported output format: %s (use .yapf .png .bmp .tga .jpg)\n", path); - return 0; -} +static const struct { const char *name; uint8_t v; } fmt_names[] = { + { "bc1", YAPF_GPU_BC1 }, { "bc3", YAPF_GPU_BC3 }, { "bc4", YAPF_GPU_BC4 }, + { "bc5", YAPF_GPU_BC5 }, { "bc6h", YAPF_GPU_BC6H }, { "bc7", YAPF_GPU_BC7 } +}; static const char *gpu_name(int g) { - static const char *n[] = { "RGBA8", "RGB8", "RG8", "R8", "SRGB8_A8", "SRGB8" }; - return g >= 0 && g < 6 ? n[g] : "unknown"; + static const char *px[] = { "RGBA8", "RGB8", "RG8", "R8", "SRGB8_A8", "SRGB8", + "RGBA16", "RGB16", "RG16", "R16", "RGBA16F", "RGB16F", "RG16F", "R16F", + "RGBA32F", "RGB32F", "RG32F", "R32F" }; + static const char *bc[] = { "BC1", "BC2", "BC3", "BC4", "BC4_SNORM", "BC5", "BC5_SNORM", + "BC6H", "BC6H_SF", "BC7" }; + static const char *etc[] = { "ETC2_RGB8", "ETC2_RGB8A1", "ETC2_RGBA8", "EAC_R11", + "EAC_R11_SNORM", "EAC_RG11", "EAC_RG11_SNORM" }; + static const char *astc[] = { "ASTC_4x4", "ASTC_5x4", "ASTC_5x5", "ASTC_6x5", "ASTC_6x6", + "ASTC_8x5", "ASTC_8x6", "ASTC_8x8", "ASTC_10x5", "ASTC_10x6", "ASTC_10x8", + "ASTC_10x10", "ASTC_12x10", "ASTC_12x12" }; + if (g >= 0 && g < 18) return px[g]; + if (g >= 0x40 && g < 0x4A) return bc[g - 0x40]; + if (g >= 0x50 && g < 0x57) return etc[g - 0x50]; + if (g >= 0x60 && g < 0x6E) return astc[g - 0x60]; + return "unknown"; +} + +static uint8_t gpu_hint(int ch, int srgb, int sample) { + static const uint8_t u8[2][4] = { { YAPF_GPU_R8, YAPF_GPU_RG8, YAPF_GPU_RGB8, YAPF_GPU_RGBA8 }, + { YAPF_GPU_R8, YAPF_GPU_RG8, YAPF_GPU_SRGB8, YAPF_GPU_SRGB8_A8 } }; + static const uint8_t wide[3][4] = { + { YAPF_GPU_R16, YAPF_GPU_RG16, YAPF_GPU_RGB16, YAPF_GPU_RGBA16 }, + { YAPF_GPU_R16F, YAPF_GPU_RG16F, YAPF_GPU_RGB16F, YAPF_GPU_RGBA16F }, + { YAPF_GPU_R32F, YAPF_GPU_RG32F, YAPF_GPU_RGB32F, YAPF_GPU_RGBA32F } }; + if (sample == YAPF_SAMPLE_U8) return u8[srgb ? 1 : 0][ch - 1]; + return wide[sample - 1][ch - 1]; +} + +static int parse_map(const char *s, uint8_t *map) { + memset(map, 0, 4); + for (int c = 0; c < 4 && *s; c++) { + const char *e = strchr(s, ','); + size_t n = e ? (size_t)(e - s) : strlen(s); + size_t i; + for (i = 0; i < sizeof(ch_names) / sizeof(ch_names[0]); i++) + if (strlen(ch_names[i].name) == n && strncmp(ch_names[i].name, s, n) == 0) break; + if (i == sizeof(ch_names) / sizeof(ch_names[0])) return 0; + map[c] = ch_names[i].v; + s += n + (e ? 1 : 0); + } + return 1; +} + +/* ── Reading any input into a texture ────────────────────────────────── */ + +static yapf_texture_t *new_tex(uint32_t w, uint32_t h, int ch, int sample) { + yapf_texture_t d; + memset(&d, 0, sizeof(d)); + d.width = w; d.height = h; d.depth = 1; d.layers = 1; d.frames = 1; d.mip_levels = 1; + d.channels = (uint8_t)ch; d.sample = (uint8_t)sample; + return yapf_tex_create(&d); +} + +static yapf_texture_t *read_input(const char *path, const opts_t *o, int *from_yapf) { + *from_yapf = 0; + if (is_yapf(path)) { + int err = 0; + yapf_texture_t *t = yapf_tex_load(path, 0, 0, &err); + if (!t) fprintf(stderr, "yapf: cannot read %s (error %d)\n", path, err); + *from_yapf = 1; + return t; + } + int w, h, ch; + yapf_texture_t *t = NULL; + if (stbi_is_hdr(path)) { + float *f = stbi_loadf(path, &w, &h, &ch, 0); + if (!f) { fprintf(stderr, "yapf: cannot read %s: %s\n", path, stbi_failure_reason()); return NULL; } + int sample = o->float32 ? YAPF_SAMPLE_F32 : YAPF_SAMPLE_F16; + t = new_tex((uint32_t)w, (uint32_t)h, ch, sample); + if (t) { + size_t n = (size_t)w * h * ch; + if (sample == YAPF_SAMPLE_F32) memcpy(t->data[0], f, n * 4); + else for (size_t i = 0; i < n; i++) ((uint16_t *)t->data[0])[i] = yapf_float_to_half(f[i]); + t->transfer = YAPF_TRANSFER_LINEAR; + } + stbi_image_free(f); + } else if (stbi_is_16_bit(path) && !o->eight_bit) { + stbi_us *p = stbi_load_16(path, &w, &h, &ch, 0); + if (!p) { fprintf(stderr, "yapf: cannot read %s: %s\n", path, stbi_failure_reason()); return NULL; } + t = new_tex((uint32_t)w, (uint32_t)h, ch, YAPF_SAMPLE_U16); + if (t) memcpy(t->data[0], p, (size_t)w * h * ch * 2); + stbi_image_free(p); + } else { + stbi_uc *p = stbi_load(path, &w, &h, &ch, 0); + if (!p) { fprintf(stderr, "yapf: cannot read %s: %s\n", path, stbi_failure_reason()); return NULL; } + t = new_tex((uint32_t)w, (uint32_t)h, ch, YAPF_SAMPLE_U8); + if (t) memcpy(t->data[0], p, (size_t)w * h * ch); + stbi_image_free(p); + } + if (!t) fprintf(stderr, "yapf: %s: unsupported size or out of memory\n", path); + return t; +} + +/* Several same-sized inputs into one cube / array / volume / animation. */ +static yapf_texture_t *assemble(const char *kind, int n, char **paths, const opts_t *o) { + yapf_texture_t *first = NULL, *t = NULL; + for (int i = 0; i < n; i++) { + int y; + yapf_texture_t *in = read_input(paths[i], o, &y); + if (!in) { yapf_tex_free(t); yapf_tex_free(first); return NULL; } + if (!t) { + yapf_texture_t d = *in; + d.mip_levels = 1; + if (strcmp(kind, "cube") == 0) { + if (n % 6) { fprintf(stderr, "yapf: a cube needs 6 faces per layer (+X -X +Y -Y +Z -Z)\n"); yapf_tex_free(in); return NULL; } + d.shape = YAPF_SHAPE_CUBE; d.layers = (uint16_t)(n / 6); + } else if (strcmp(kind, "array") == 0) { + d.layers = (uint16_t)n; + } else if (strcmp(kind, "volume") == 0) { + d.shape = YAPF_SHAPE_VOLUME; d.depth = (uint32_t)n; + } else if (strcmp(kind, "anim") == 0) { + d.frames = (uint16_t)n; + } else { + fprintf(stderr, "yapf: pack kind must be cube, array, volume or anim\n"); + yapf_tex_free(in); + return NULL; + } + t = yapf_tex_create(&d); + if (!t) { fprintf(stderr, "yapf: invalid %s (cube faces must be square)\n", kind); yapf_tex_free(in); return NULL; } + first = in; + } else if (in->width != t->width || in->height != t->height || + in->channels != t->channels || in->sample != t->sample) { + fprintf(stderr, "yapf: %s does not match the size and channels of %s\n", paths[i], paths[0]); + yapf_tex_free(in); yapf_tex_free(t); yapf_tex_free(first); + return NULL; + } + size_t bytes = yapf_tex_surface_bytes(in, 0); + if (t->shape == YAPF_SHAPE_VOLUME) memcpy((uint8_t *)t->data[0] + bytes * i, in->data[0], bytes); + else if (t->frames > 1) memcpy(t->data[yapf_tex_surface(t, (uint32_t)i, 0, 0, 0)], in->data[0], bytes); + else if (t->shape == YAPF_SHAPE_CUBE) memcpy(t->data[yapf_tex_surface(t, 0, (uint32_t)i / 6, (uint32_t)i % 6, 0)], in->data[0], bytes); + else memcpy(t->data[yapf_tex_surface(t, 0, (uint32_t)i, 0, 0)], in->data[0], bytes); + if (in != first) yapf_tex_free(in); + } + if (t && t->frames > 1) + for (int f = 0; f < t->frames; f++) t->frame_us[f] = (uint32_t)(o->frame_ms * 1000.0 + 0.5); + yapf_tex_free(first); + return t; +} + +/* ── Writing .yapf ───────────────────────────────────────────────────── */ + +static int needs_v2(const yapf_texture_t *t, const opts_t *o) { + return o->v2 || o->entropy || o->format >= 0 || o->coverage || o->normal || o->have_map || + o->linear_alpha || o->primaries || o->transfer || o->author || o->license || + o->source || o->nnotes || t->sample != YAPF_SAMPLE_U8 || t->shape != YAPF_SHAPE_2D || + t->layers > 1 || t->frames > 1 || t->payload != YAPF_PAYLOAD_PIXELS || t->note_count; +} + +static int write_yapf(yapf_texture_t *t, const char *out, const opts_t *o, int from_yapf) { + if (!from_yapf) { + t->flags = (uint8_t)((o->srgb && t->sample == YAPF_SAMPLE_U8 && !o->normal ? YAPF_FLAG_SRGB : 0) | + (o->premult ? YAPF_FLAG_PREMULT_ALPHA : 0)); + t->format = gpu_hint(t->channels, t->flags & YAPF_FLAG_SRGB, t->sample); + } + if (o->normal) { + t->normal = (uint8_t)o->normal; + t->channel_map[0] = YAPF_CH_NORMAL_X; + t->channel_map[1] = YAPF_CH_NORMAL_Y; + if (t->channels >= 3) t->channel_map[2] = YAPF_CH_NORMAL_Z; + else t->normal |= YAPF_NORMAL_XY; + t->flags &= (uint8_t)~YAPF_FLAG_SRGB; + } + if (o->have_map) memcpy(t->channel_map, o->map, 4); + if (o->linear_alpha && (t->channels == 2 || t->channels == 4)) + t->linear_mask |= (uint8_t)(1u << (t->channels - 1)); + if (o->primaries) t->primaries = (uint8_t)o->primaries; + if (o->transfer) t->transfer = (uint8_t)o->transfer; + if (o->frame_ms > 0 && t->frames > 1) + for (int f = 0; f < t->frames; f++) t->frame_us[f] = (uint32_t)(o->frame_ms * 1000.0 + 0.5); + if (o->loops >= 0) t->loop_count = (uint16_t)o->loops; + + if (o->mips && t->payload == YAPF_PAYLOAD_PIXELS) { + int rc = yapf_tex_build_mips(t, 0, (uint8_t)o->coverage); + if (rc != YAPF_OK) { + fprintf(stderr, "yapf: cannot build mips%s\n", o->coverage ? " (--coverage needs an alpha channel)" : ""); + return 1; + } + } + int v2 = needs_v2(t, o); + if (o->format >= 0) { + yapf_texture_t *c = NULL; + int rc = yapf_tex_compress(t, (uint8_t)o->format, &c); + if (rc != YAPF_OK) { + fprintf(stderr, "yapf: cannot compress to %s: %s\n", gpu_name(o->format), + o->format == YAPF_GPU_BC6H ? "BC6H needs a 16-bit or HDR (float) image" + : "LDR block formats need an 8-bit image"); + return 1; + } + opts_t rest = *o; /* the blocks are made: write them */ + rest.format = -1; + rest.mips = 0; + rest.v2 = 1; + int r2 = write_yapf(c, out, &rest, 1); + yapf_tex_free(c); + return r2; + } + if (!v2) { + /* 8-bit 2D: version 1, readable by every YAPF decoder */ + yapf_image_t img; + memset(&img, 0, sizeof(img)); + img.width = t->width; img.height = t->height; img.channels = t->channels; + img.gpu_format = t->format; img.flags = t->flags; img.mip_levels = t->mip_levels; + img.mips = (uint8_t **)t->data; + img.pixels = (uint8_t *)t->data[0]; + if (yapf_save(out, &img) != YAPF_OK) { fprintf(stderr, "yapf: cannot write %s\n", out); return 1; } + return 0; + } + if (!yapf_tex_find_note(t, "TOOL")) yapf_tex_add_note(t, "TOOL", "yapf command-line tool", 22); + if (o->author) yapf_tex_add_note(t, "AUTH", o->author, (uint32_t)strlen(o->author)); + if (o->license) yapf_tex_add_note(t, "LICN", o->license, (uint32_t)strlen(o->license)); + if (o->source) yapf_tex_add_note(t, "SRCE", o->source, (uint32_t)strlen(o->source)); + for (int i = 0; i < o->nnotes; i++) { + const char *eq = strchr(o->notes[i], '='); + if (!eq || eq - o->notes[i] != 4) { fprintf(stderr, "yapf: --note wants TAG=text with a 4-letter tag\n"); return 1; } + yapf_tex_add_note(t, o->notes[i], eq + 1, (uint32_t)strlen(eq + 1)); + } + if (yapf_tex_save(out, t, o->entropy ? YAPF_ENC_ENTROPY : 0) != YAPF_OK) { + fprintf(stderr, "yapf: cannot write %s\n", out); + return 1; + } + return 0; +} + +/* ── Exporting a surface to a common format ──────────────────────────── */ + +/* One surface slice as 8-bit pixels (*hdr = 0) or float RGB(A) (*hdr = 1). */ +static void *surface_view(const yapf_texture_t *t, uint32_t s, uint32_t slice, + uint32_t *w, uint32_t *h, int *ch, int *hdr) +{ + uint32_t m = s % t->mip_levels, d; + yapf_tex_mip_size(t, m, w, h, &d); + if (slice >= d) return NULL; + size_t n = (size_t)*w * *h, slice_bytes = yapf_tex_surface_bytes(t, m) / d; + const uint8_t *src = (const uint8_t *)t->data[s] + slice_bytes * slice; + *hdr = 0; + if (t->payload == YAPF_PAYLOAD_BLOCKS) { + int is6 = t->format == YAPF_GPU_BC6H || t->format == YAPF_GPU_BC6H_SF; + void *out = malloc(n * (is6 ? 6 : 4)); + if (!out) return NULL; + if (yapf_blocks_decode(t->format, src, *w, *h, out) != YAPF_OK) { free(out); return NULL; } + if (!is6) { *ch = 4; return out; } + float *f = (float *)malloc(n * 3 * sizeof(float)); + for (size_t i = 0; f && i < n * 3; i++) f[i] = yapf_half_to_float(((uint16_t *)out)[i]); + free(out); + *ch = 3; *hdr = 1; + return f; + } + *ch = t->channels; + if (t->sample == YAPF_SAMPLE_U8) { + uint8_t *o = (uint8_t *)malloc(n * *ch); + if (o) memcpy(o, src, n * *ch); + return o; + } + if (t->sample == YAPF_SAMPLE_U16) { + uint8_t *o = (uint8_t *)malloc(n * *ch); + for (size_t i = 0; o && i < n * *ch; i++) o[i] = (uint8_t)((((const uint16_t *)src)[i] + 128) / 257); + return o; + } + float *f = (float *)malloc(n * *ch * sizeof(float)); + for (size_t i = 0; f && i < n * *ch; i++) + f[i] = t->sample == YAPF_SAMPLE_F16 ? yapf_half_to_float(((const uint16_t *)src)[i]) + : ((const float *)src)[i]; + *hdr = 1; + return f; +} + +static uint8_t *to8(const float *f, size_t n) { + uint8_t *o = (uint8_t *)malloc(n); + for (size_t i = 0; o && i < n; i++) { + float v = f[i] < 0 ? 0 : f[i] > 1 ? 1 : f[i]; + o[i] = (uint8_t)(v * 255.0f + 0.5f); + } + return o; +} + +static int write_image(const char *path, const void *px, int w, int h, int ch, int hdr) { + if (ends_with(path, ".hdr")) { + if (!hdr) { + float *f = (float *)malloc((size_t)w * h * ch * sizeof(float)); + for (size_t i = 0; f && i < (size_t)w * h * ch; i++) f[i] = ((const uint8_t *)px)[i] / 255.0f; + int ok = f && stbi_write_hdr(path, w, h, ch, f); + free(f); + return ok; + } + return stbi_write_hdr(path, w, h, ch, (const float *)px); + } + uint8_t *tmp = NULL; + if (hdr) { + tmp = to8((const float *)px, (size_t)w * h * ch); + if (!tmp) return 0; + fprintf(stderr, "yapf: note: float pixels clamped to 0..1 for %s (use .hdr to keep them)\n", path); + px = tmp; + } + int ok = 0; + if (ends_with(path, ".png")) ok = stbi_write_png(path, w, h, ch, px, w * ch); + else if (ends_with(path, ".bmp")) ok = stbi_write_bmp(path, w, h, ch, px); + else if (ends_with(path, ".tga")) ok = stbi_write_tga(path, w, h, ch, px); + else if (ends_with(path, ".jpg") || ends_with(path, ".jpeg")) ok = stbi_write_jpg(path, w, h, ch, px, 95); + else fprintf(stderr, "yapf: unsupported output format: %s (use .yapf .png .bmp .tga .jpg .hdr)\n", path); + free(tmp); + return ok; +} + +static int export_image(const yapf_texture_t *t, const char *in, const char *out, const opts_t *o) { + uint32_t faces = yapf_tex_faces(t); + if ((uint32_t)o->mip >= t->mip_levels || (uint32_t)o->layer >= t->layers || + (uint32_t)o->face >= faces || (uint32_t)o->frame >= t->frames) { + fprintf(stderr, "yapf: %s has %u mip(s), %u layer(s), %u face(s), %u frame(s)\n", + in, t->mip_levels, t->layers, faces, t->frames); + return 1; + } + uint32_t s = yapf_tex_surface(t, (uint32_t)o->frame, (uint32_t)o->layer, (uint32_t)o->face, (uint32_t)o->mip); + uint32_t w, h; + int ch, hdr; + void *px = surface_view(t, s, (uint32_t)o->slice, &w, &h, &ch, &hdr); + if (!px) { + fprintf(stderr, "yapf: cannot export %s: %s\n", in, + t->payload == YAPF_PAYLOAD_BLOCKS ? "no built-in decoder for this block format (ETC2 / ASTC)" + : "no such slice, or out of memory"); + return 1; + } + if (t->sample == YAPF_SAMPLE_U16 && !ends_with(out, ".hdr")) + fprintf(stderr, "yapf: note: 16-bit samples written as 8-bit\n"); + int ok = write_image(out, px, (int)w, (int)h, ch, hdr); + free(px); + if (!ok) { fprintf(stderr, "yapf: cannot write %s\n", out); return 1; } + return 0; +} + +/* ── info / verify / bench / thumbnail ───────────────────────────────── */ + +static uint8_t *read_all(const char *path, size_t *n) { + FILE *f = fopen(path, "rb"); + if (!f) return NULL; + fseek(f, 0, SEEK_END); + long sz = ftell(f); + rewind(f); + uint8_t *buf = sz >= 0 ? (uint8_t *)malloc((size_t)sz + 1) : NULL; + if (buf && fread(buf, 1, (size_t)sz, f) != (size_t)sz) { free(buf); buf = NULL; } + fclose(f); + *n = buf ? (size_t)sz : 0; + return buf; +} + +static const char *err_name(int e) { + switch (e) { + case YAPF_ERR_INVALID: return "invalid argument"; + case YAPF_ERR_IO: return "cannot read the file"; + case YAPF_ERR_OOM: return "out of memory"; + case YAPF_ERR_CORRUPT: return "corrupt or not a YAPF file"; + case YAPF_ERR_UNSUPPORTED: return "needs a feature this decoder does not have"; + case YAPF_ERR_CHECKSUM: return "checksum mismatch (damaged file)"; + case YAPF_ERR_TRUNCATED: return "file is truncated"; + default: return "error"; + } } static int cmd_info(const char *path) { - FILE *f = fopen(path, "rb"); - uint8_t hdr[20]; - if (!f || fread(hdr, 1, 20, f) != 20 || memcmp(hdr, "YAPF", 4) != 0) { - fprintf(stderr, "yapf: %s is not a YAPF file\n", path); - if (f) fclose(f); + size_t n; + uint8_t *buf = read_all(path, &n); + int err = 0; + yapf_texture_t *t = buf ? yapf_tex_open(buf, n, 0, &err) : NULL; + if (!t) { + fprintf(stderr, "yapf: %s: %s\n", path, buf ? err_name(err) : "cannot open"); + free(buf); return 1; } - fseek(f, 0, SEEK_END); - long size = ftell(f); - fclose(f); - unsigned w = hdr[12] | hdr[13] << 8 | (unsigned)hdr[14] << 16 | (unsigned)hdr[15] << 24; - unsigned h = hdr[16] | hdr[17] << 8 | (unsigned)hdr[18] << 16 | (unsigned)hdr[19] << 24; + static const char *shape[] = { "2D", "cube", "volume" }; + static const char *sample[] = { "8-bit", "16-bit", "half float", "float" }; + static const char *prim[] = { "unspecified", "BT.709 / sRGB", "Display P3", "BT.2020", + "ACES AP0", "ACEScg", "Adobe RGB" }; + static const char *tf[] = { "unspecified", "sRGB", "linear", "gamma 2.2", "PQ", "HLG", "BT.1886" }; + static const char *mf[] = { "unknown", "box", "tent", "Kaiser", "Lanczos", "Mitchell", "authored" }; + uint32_t w, h, d; + yapf_tex_mip_size(t, 0, &w, &h, &d); printf("%s\n", path); - printf(" format version : %u\n", hdr[4]); - printf(" size : %u x %u, %u channel(s)\n", w, h, hdr[5]); - printf(" mip levels : %u\n", hdr[7]); - printf(" gpu format : %s\n", gpu_name(hdr[6])); - printf(" flags : %s%s\n", hdr[8] & YAPF_FLAG_SRGB ? "sRGB " : "linear ", - hdr[8] & YAPF_FLAG_PREMULT_ALPHA ? "premultiplied-alpha" : ""); - printf(" file size : %ld bytes (%.1f%% of raw)\n", size, - 100.0 * (double)size / ((double)w * h * (hdr[5] ? hdr[5] : 1))); - yapf_image_t *img = yapf_load(path); - printf(" decodes : %s\n", img ? "ok" : "FAILED (corrupt or unsupported)"); - yapf_free(img); - return img ? 0 : 1; + printf(" format version : %u\n", buf[4]); + printf(" size : %u x %u%s", w, h, t->shape == YAPF_SHAPE_VOLUME ? "" : "\n"); + if (t->shape == YAPF_SHAPE_VOLUME) printf(" x %u\n", d); + printf(" shape : %s, %u layer(s), %u frame(s)\n", shape[t->shape], t->layers, t->frames); + printf(" mip levels : %u\n", t->mip_levels); + if (t->payload == YAPF_PAYLOAD_BLOCKS) + printf(" payload : GPU blocks, %s (%u channel(s))\n", gpu_name(t->format), t->channels); + else + printf(" payload : lossless pixels, %u channel(s), %s, GPU hint %s\n", + t->channels, sample[t->sample], gpu_name(t->format)); + printf(" flags : %s%s%s\n", t->flags & YAPF_FLAG_SRGB ? "sRGB" : "linear", + t->flags & YAPF_FLAG_PREMULT_ALPHA ? ", premultiplied alpha" : "", + t->flags & YAPF_FLAG_COVERAGE ? ", coverage-preserving mips" : ""); + if (buf[4] == 2) { + uint32_t needs = (uint32_t)buf[0x1C] | (uint32_t)buf[0x1D] << 8; + printf(" needs : %s%s%s%s%s%s\n", needs ? "" : "nothing beyond 8-bit 2D", + needs & YAPF_NEED_WIDE ? "wide " : "", needs & YAPF_NEED_BLOCKS ? "blocks " : "", + needs & YAPF_NEED_ENTROPY ? "entropy " : "", needs & YAPF_NEED_LAYOUT ? "layout " : "", + needs & YAPF_NEED_FRAMES ? "frames" : ""); + printf(" channels :"); + for (int c = 0; c < t->channels; c++) + printf(" %s%s", ch_name(t->channel_map[c]), t->linear_mask & (1u << c) ? "(linear)" : ""); + printf("\n"); + if (t->normal) + printf(" normal map : %s, %s\n", + (t->normal & 3) == YAPF_NORMAL_GL ? "OpenGL (+Y)" : (t->normal & 3) == YAPF_NORMAL_DX ? "DirectX (-Y)" : "no convention", + t->normal & YAPF_NORMAL_XY ? "XY (Z reconstructed)" : t->normal & YAPF_NORMAL_OCT ? "octahedral" : "XYZ"); + printf(" colour : primaries %s, transfer %s\n", + t->primaries < 7 ? prim[t->primaries] : "?", t->transfer < 7 ? tf[t->transfer] : "?"); + if (t->mip_levels > 1) + printf(" mip filter : %s%s%s\n", (t->mip_filter & 0x7F) < 7 ? mf[t->mip_filter & 0x7F] : "?", + t->mip_filter & YAPF_MIP_LINEAR_LIGHT ? ", linear light" : "", + t->flags & YAPF_FLAG_COVERAGE ? "" : ""); + if (t->flags & YAPF_FLAG_COVERAGE) printf(" alpha ref : %u\n", t->alpha_ref); + if (t->frames > 1) + printf(" animation : %u frames, first %.1f ms, loops %s\n", t->frames, + t->frame_us ? t->frame_us[0] / 1000.0 : 0.0, t->loop_count ? "" : "forever"); + for (uint32_t i = 0; i < t->note_count; i++) { + const yapf_note_t *nt = &t->notes[i]; + int text = 1; + for (uint32_t k = 0; k < nt->size; k++) if (nt->data[k] < 9) text = 0; + if (text && nt->size < 200) printf(" note %.4s : %.*s\n", nt->tag, (int)nt->size, nt->data); + else printf(" note %.4s : %u bytes\n", nt->tag, nt->size); + } + printf(" stream prefix :"); + for (uint32_t m = t->mip_levels; m-- > 0; ) { + uint32_t mw; + yapf_tex_mip_size(t, m, &mw, NULL, NULL); + printf(" %u px: %llu B%s", mw, (unsigned long long)yapf_tex_stream_bytes(t, m), m ? "," : "\n"); + } + } + size_t raw = 0; + for (uint32_t s = 0; s < yapf_tex_surface_count(t); s++) { + uint32_t mw, mh, md; + yapf_tex_mip_size(t, s % t->mip_levels, &mw, &mh, &md); + raw += (size_t)mw * mh * md * t->channels * (t->payload ? 1 : yapf_sample_bytes(t->sample)); + } + printf(" file size : %zu bytes (%.1f%% of raw)\n", n, 100.0 * (double)n / (double)raw); + yapf_tex_free(t); + t = yapf_tex_load_memory(buf, n, 0, 0, &err); + printf(" decodes : %s\n", t ? "ok (checksums verified)" : err_name(err)); + yapf_tex_free(t); + free(buf); + return t ? 0 : 1; +} + +static int cmd_verify(const char *path) { + size_t n; + uint8_t *buf = read_all(path, &n); + int err = YAPF_ERR_IO; + yapf_texture_t *t = buf ? yapf_tex_load_memory(buf, n, 0, 0, &err) : NULL; + if (t) printf("%s: ok%s\n", path, buf[4] == 2 ? " (header, tables and every tile checksum match)" + : " (version 1: decodes; no checksums in v1)"); + else printf("%s: FAILED — %s\n", path, err_name(err)); + yapf_tex_free(t); + free(buf); + return t ? 0 : 1; } static int cmd_bench(const char *path) { - FILE *f = fopen(path, "rb"); - if (!f) { fprintf(stderr, "yapf: cannot open %s\n", path); return 1; } - fseek(f, 0, SEEK_END); - long n = ftell(f); - rewind(f); - uint8_t *buf = (uint8_t *)malloc((size_t)n); - if (!buf || fread(buf, 1, (size_t)n, f) != (size_t)n) { fclose(f); free(buf); return 1; } - fclose(f); - yapf_image_t *img = yapf_load_memory(buf, (size_t)n); - if (!img) { fprintf(stderr, "yapf: %s does not decode\n", path); free(buf); return 1; } - double raw = (double)img->width * img->height * img->channels; - yapf_free(img); - for (int mt = 0; mt < 2; mt++) { + size_t n; + uint8_t *buf = read_all(path, &n); + if (!buf) { fprintf(stderr, "yapf: cannot open %s\n", path); return 1; } + int err; + yapf_texture_t *t = yapf_tex_load_memory(buf, n, 0, 1, &err); + if (!t) { fprintf(stderr, "yapf: %s: %s\n", path, err_name(err)); free(buf); return 1; } + double raw = 0; + for (uint32_t s = 0; s < yapf_tex_surface_count(t); s++) raw += (double)yapf_tex_surface_bytes(t, s % t->mip_levels); + yapf_tex_free(t); + for (int mode = 0; mode < 3; mode++) { + int threads = mode == 1 ? 0 : 1, flags = mode == 2 ? YAPF_LOAD_TRUST : 0; + if (mode == 2 && buf[4] != 2) break; /* Best of 15 rounds; each round repeats enough decodes to last * ~20 ms, so short images are measured warm and not timer-bound. */ double one = now_s(); - yapf_free(mt ? yapf_load_memory_mt(buf, (size_t)n, 0) : yapf_load_memory(buf, (size_t)n)); + yapf_tex_free(yapf_tex_load_memory(buf, n, flags, threads, NULL)); one = now_s() - one; int reps = (int)(0.02 / (one > 1e-6 ? one : 1e-6)) + 1; double best = 1e9; for (int r = 0; r < 15; r++) { - double t = now_s(); - for (int k = 0; k < reps; k++) - yapf_free(mt ? yapf_load_memory_mt(buf, (size_t)n, 0) - : yapf_load_memory(buf, (size_t)n)); - t = (now_s() - t) / reps; - if (t < best) best = t; + double tt = now_s(); + for (int k = 0; k < reps; k++) yapf_tex_free(yapf_tex_load_memory(buf, n, flags, threads, NULL)); + tt = (now_s() - tt) / reps; + if (tt < best) best = tt; } - printf(" %-12s %8.3f ms %6.2f GB/s\n", mt ? "all cores" : "1 thread", best * 1e3, raw / best / 1e9); + static const char *nm[] = { "1 thread", "all cores", "no checks" }; + printf(" %-12s %8.3f ms %6.2f GB/s\n", nm[mode], best * 1e3, raw / best / 1e9); } free(buf); return 0; @@ -210,23 +615,31 @@ static int cmd_bench(const char *path) { /* Preview no larger than size×size: start from the smallest mip that is * still big enough, then average boxes of source pixels. */ static int cmd_thumbnail(const char *in, const char *out, int size) { - yapf_image_t *img = yapf_load_mt(in, 0); - if (!img) { fprintf(stderr, "yapf: cannot read %s\n", in); return 1; } + int err; + yapf_texture_t *t = yapf_tex_load(in, 0, 0, &err); + if (!t) { fprintf(stderr, "yapf: cannot read %s: %s\n", in, err_name(err)); return 1; } if (size < 1) size = 256; int m = 0; - while (m + 1 < img->mip_levels && - (int)(img->width >> (m + 1)) >= size && (int)(img->height >> (m + 1)) >= size) + while (m + 1 < t->mip_levels && + (int)(t->width >> (m + 1)) >= size && (int)(t->height >> (m + 1)) >= size) m++; - uint32_t sw = img->width >> m ? img->width >> m : 1, sh = img->height >> m ? img->height >> m : 1; - const uint8_t *src = img->mips[m]; + uint32_t sw, sh; + int ch, hdr; + uint8_t *src = (uint8_t *)surface_view(t, (uint32_t)m, 0, &sw, &sh, &ch, &hdr); + yapf_tex_free(t); + if (!src) { fprintf(stderr, "yapf: cannot preview %s\n", in); return 1; } + if (hdr) { + uint8_t *b = to8((const float *)src, (size_t)sw * sh * ch); + free(src); + if (!(src = b)) return 1; + } double k = (double)size / (double)(sw > sh ? sw : sh); if (k > 1.0) k = 1.0; uint32_t dw = (uint32_t)(sw * k + 0.5), dh = (uint32_t)(sh * k + 0.5); if (dw < 1) dw = 1; if (dh < 1) dh = 1; - int ch = img->channels; uint8_t *dst = (uint8_t *)malloc((size_t)dw * dh * ch); - if (!dst) { yapf_free(img); return 1; } + if (!dst) { free(src); return 1; } for (uint32_t y = 0; y < dh; y++) { uint32_t y0 = (uint32_t)((uint64_t)y * sh / dh), y1 = (uint32_t)((uint64_t)(y + 1) * sh / dh); if (y1 <= y0) y1 = y0 + 1; @@ -238,14 +651,14 @@ static int cmd_thumbnail(const char *in, const char *out, int size) { for (uint32_t yy = y0; yy < y1; yy++) for (uint32_t xx = x0; xx < x1; xx++) sum += src[((size_t)yy * sw + xx) * ch + c]; - uint64_t n = (uint64_t)(y1 - y0) * (x1 - x0); - dst[((size_t)y * dw + x) * ch + c] = (uint8_t)((sum + n / 2) / n); + uint64_t nn = (uint64_t)(y1 - y0) * (x1 - x0); + dst[((size_t)y * dw + x) * ch + c] = (uint8_t)((sum + nn / 2) / nn); } } } int ok = stbi_write_png(out, (int)dw, (int)dh, ch, dst, (int)dw * ch); free(dst); - yapf_free(img); + free(src); if (!ok) fprintf(stderr, "yapf: cannot write %s\n", out); return ok ? 0 : 1; } @@ -285,7 +698,7 @@ static const char *ext_name(const char *path) { * and a lossless check. */ static int cmd_compare(int n, char **paths) { int rc = 0; - double tot_src = 0, tot_yapf = 0, tot_src_t = 0, tot_yapf_t = 0; + double tot_src = 0, tot_yapf = 0, tot_src_t = 0, tot_yapf_t = 0, tot_ent = 0; for (int i = 0; i < n; i++) { const char *path = paths[i]; if (is_yapf(path)) { @@ -310,7 +723,7 @@ static int cmd_compare(int n, char **paths) { memset(&img, 0, sizeof(img)); img.width = (uint32_t)w; img.height = (uint32_t)h; img.channels = (uint8_t)ch; img.mip_levels = 1; img.pixels = px; img.flags = YAPF_FLAG_SRGB; - img.gpu_format = gpu_hint(ch, 1); + img.gpu_format = gpu_hint(ch, 1, YAPF_SAMPLE_U8); void *enc = NULL; size_t esz = 0; @@ -339,6 +752,28 @@ static int cmd_compare(int n, char **paths) { printf(" %-5s %10s %5.1f%% of raw decode %8.3f ms %.1fx faster, %.0f%% of the %s size\n", "YAPF", fmt_kb((double)esz, b), 100.0 * esz / raw, t_one * 1e3, t_src / t_one, 100.0 * esz / fsz, ext_name(path)); + /* version 2 with the optional Huffman planes */ + { + yapf_texture_t v; + void *data[1] = { px }; + size_t esz2 = 0; + void *enc2 = NULL; + double t_e; + memset(&v, 0, sizeof(v)); + v.width = (uint32_t)w; v.height = (uint32_t)h; v.depth = 1; v.layers = 1; v.frames = 1; + v.mip_levels = 1; v.channels = (uint8_t)ch; v.flags = YAPF_FLAG_SRGB; v.data = data; + if (yapf_tex_encode(&v, YAPF_ENC_ENTROPY, &enc2, &esz2) == YAPF_OK) { + yapf_texture_t *c2 = yapf_tex_load_memory(enc2, esz2, 0, 1, NULL); + if (!c2 || memcmp(c2->data[0], px, (size_t)w * h * ch) != 0) lossless = 0; + yapf_tex_free(c2); + YAPF_TIME_BEST(t_e, yapf_tex_free(yapf_tex_load_memory(enc2, esz2, 0, 1, NULL))); + printf(" %-5s %10s %5.1f%% of raw decode %8.3f ms %.1fx faster, %.0f%% of the %s size\n", + "YAPF+E", fmt_kb((double)esz2, c), 100.0 * esz2 / raw, t_e * 1e3, + t_src / t_e, 100.0 * esz2 / fsz, ext_name(path)); + tot_ent += (double)esz2; + yapf_free_buffer(enc2); + } + } printf(" YAPF on all cores: decode %.3f ms (%.1f GB/s), encode %.0f ms, %s\n\n", t_all * 1e3, raw / t_all / 1e9, t_enc * 1e3, lossless ? "lossless: yes" : "LOSSLESS CHECK FAILED"); @@ -350,8 +785,10 @@ static int cmd_compare(int n, char **paths) { } if (n > 1 && tot_yapf_t > 0) { char a[32], b[32]; - printf("Total: originals %s, YAPF %s (%.0f%%); decoding %.1fx faster on one core\n", - fmt_kb(tot_src, a), fmt_kb(tot_yapf, b), 100.0 * tot_yapf / tot_src, tot_src_t / tot_yapf_t); + char e[32]; + printf("Total: originals %s, YAPF %s (%.0f%%), YAPF+E %s (%.0f%%); decoding %.1fx faster on one core\n", + fmt_kb(tot_src, a), fmt_kb(tot_yapf, b), 100.0 * tot_yapf / tot_src, + fmt_kb(tot_ent, e), 100.0 * tot_ent / tot_src, tot_src_t / tot_yapf_t); } return rc; } @@ -359,96 +796,131 @@ static int cmd_compare(int n, char **paths) { static void usage(void) { fprintf(stderr, "usage: yapf [options] convert (formats by extension)\n" - " yapf info show header information\n" + " yapf pack [options]\n" + " yapf info header, tables and notes\n" + " yapf verify check every checksum\n" " yapf bench time decoding\n" " yapf thumbnail [size] preview, default 256 px\n" " yapf compare [more...] size and speed vs the original format\n" "\n" - "reads: .yapf .png .jpg .bmp .tga .gif .psd .pnm .hdr\n" - "writes: .yapf .png .jpg .bmp .tga\n" + "reads: .yapf .png (8/16-bit) .jpg .bmp .tga .gif .psd .pnm .hdr\n" + "writes: .yapf .png .jpg .bmp .tga .hdr\n" "\n" "options when writing .yapf:\n" - " --mips store a full mip chain\n" - " --linear pixels are linear, not sRGB\n" - " --premult alpha is premultiplied\n" + " --mips store a full mip chain (sRGB averaged in linear light)\n" + " --coverage REF mips keep alpha-test coverage at alpha REF (0-255)\n" + " --format F GPU blocks: bc1 bc3 bc4 bc5 bc6h bc7\n" + " --entropy Huffman-coded planes: smaller, slower to decode\n" + " --linear pixels are linear, not sRGB\n" + " --premult alpha is premultiplied\n" + " --linear-alpha alpha is linear even in an sRGB image\n" + " --normal gl|dx normal map (OpenGL +Y / DirectX -Y); implies linear\n" + " --map a,b,c,d channel meanings: r g b a luma nx ny nz rough gloss metal\n" + " ao height spec emissive opacity mask cavity thickness id -\n" + " --primaries P bt709 p3 bt2020 aces acescg adobe\n" + " --transfer T srgb linear gamma22 pq hlg bt1886\n" + " --8bit read 16-bit images as 8-bit\n" + " --float32 keep .hdr input as 32-bit float (default half)\n" + " --frame-ms N animation frame time (pack anim), --loops N\n" + " --author S --license S --source S --note TAG=text\n" + " --v2 write version 2 even for a plain 8-bit image\n" "options when reading .yapf:\n" - " --mip N export mip level N\n"); + " --mip N --layer N --face N --frame N --slice N which image to export\n"); +} + +static int parse_opts(int argc, char **argv, int start, opts_t *o) { + static const char *prim[] = { "", "bt709", "p3", "bt2020", "aces", "acescg", "adobe" }; + static const char *tf[] = { "", "srgb", "linear", "gamma22", "pq", "hlg", "bt1886" }; + memset(o, 0, sizeof(*o)); + o->srgb = 1; + o->format = -1; + o->loops = -1; + for (int i = start; i < argc; i++) { + const char *a = argv[i], *v = i + 1 < argc ? argv[i + 1] : NULL; + if (strcmp(a, "--mips") == 0) o->mips = 1; + else if (strcmp(a, "--linear") == 0) o->srgb = 0; + else if (strcmp(a, "--premult") == 0) o->premult = 1; + else if (strcmp(a, "--entropy") == 0) o->entropy = 1; + else if (strcmp(a, "--v2") == 0) o->v2 = 1; + else if (strcmp(a, "--8bit") == 0) o->eight_bit = 1; + else if (strcmp(a, "--float32") == 0) o->float32 = 1; + else if (strcmp(a, "--linear-alpha") == 0) o->linear_alpha = 1; + else if (!v) return 0; + else if (strcmp(a, "--coverage") == 0) { o->coverage = atoi(v); o->mips = 1; i++; + if (o->coverage < 1 || o->coverage > 255) return 0; } + else if (strcmp(a, "--format") == 0) { + size_t k; + for (k = 0; k < sizeof(fmt_names) / sizeof(fmt_names[0]); k++) + if (strcmp(v, fmt_names[k].name) == 0) o->format = fmt_names[k].v; + if (o->format < 0) return 0; + i++; + } + else if (strcmp(a, "--normal") == 0) { + o->normal = strcmp(v, "gl") == 0 ? YAPF_NORMAL_GL : strcmp(v, "dx") == 0 ? YAPF_NORMAL_DX : 0; + if (!o->normal) return 0; + i++; + } + else if (strcmp(a, "--map") == 0) { if (!parse_map(v, o->map)) return 0; o->have_map = 1; i++; } + else if (strcmp(a, "--primaries") == 0 || strcmp(a, "--transfer") == 0) { + const char **names = a[2] == 'p' ? prim : tf; + int found = 0; + for (int k = 1; k < 7; k++) if (strcmp(v, names[k]) == 0) found = k; + if (!found) return 0; + if (a[2] == 'p') o->primaries = found; else o->transfer = found; + i++; + } + else if (strcmp(a, "--frame-ms") == 0) { o->frame_ms = atof(v); i++; } + else if (strcmp(a, "--loops") == 0) { o->loops = atoi(v); i++; } + else if (strcmp(a, "--author") == 0) { o->author = v; i++; } + else if (strcmp(a, "--license") == 0) { o->license = v; i++; } + else if (strcmp(a, "--source") == 0) { o->source = v; i++; } + else if (strcmp(a, "--note") == 0) { if (o->nnotes == 16) return 0; o->notes[o->nnotes++] = v; i++; } + else if (strcmp(a, "--mip") == 0) { o->mip = atoi(v); i++; } + else if (strcmp(a, "--layer") == 0) { o->layer = atoi(v); i++; } + else if (strcmp(a, "--face") == 0) { o->face = atoi(v); i++; } + else if (strcmp(a, "--frame") == 0) { o->frame = atoi(v); i++; } + else if (strcmp(a, "--slice") == 0) { o->slice = atoi(v); i++; } + else return 0; + } + if (o->mip < 0 || o->layer < 0 || o->face < 0 || o->frame < 0 || o->slice < 0) return 0; + return 1; } int main(int argc, char **argv) { - if (argc == 3 && strcmp(argv[1], "info") == 0) return cmd_info(argv[2]); - if (argc == 3 && strcmp(argv[1], "bench") == 0) return cmd_bench(argv[2]); + if (argc == 3 && strcmp(argv[1], "info") == 0) return cmd_info(argv[2]); + if (argc == 3 && strcmp(argv[1], "verify") == 0) return cmd_verify(argv[2]); + if (argc == 3 && strcmp(argv[1], "bench") == 0) return cmd_bench(argv[2]); if (argc >= 3 && strcmp(argv[1], "compare") == 0) return cmd_compare(argc - 2, argv + 2); if ((argc == 4 || argc == 5) && strcmp(argv[1], "thumbnail") == 0) return cmd_thumbnail(argv[2], argv[3], argc == 5 ? atoi(argv[4]) : 256); - if (argc < 3) { usage(); return 2; } + + opts_t o; + if (argc >= 5 && strcmp(argv[1], "pack") == 0) { + int first_opt = 3; + while (first_opt < argc && strncmp(argv[first_opt], "--", 2) != 0) first_opt++; + int n = first_opt - 3 - 1; /* inputs, then the output */ + if (n < 1 || !parse_opts(argc, argv, first_opt, &o)) { usage(); return 2; } + const char *out = argv[first_opt - 1]; + if (!is_yapf(out)) { fprintf(stderr, "yapf: pack writes a .yapf file\n"); return 2; } + if (o.frame_ms <= 0) o.frame_ms = 100; + yapf_texture_t *t = assemble(argv[2], n, argv + 3, &o); + if (!t) return 1; + int rc = write_yapf(t, out, &o, 0); + yapf_tex_free(t); + return rc; + } + if (argc < 3 || !parse_opts(argc, argv, 3, &o)) { usage(); return 2; } const char *in = argv[1], *out = argv[2]; - int mips = 0, srgb = 1, premult = 0, mip_out = 0; - for (int i = 3; i < argc; i++) { - if (strcmp(argv[i], "--mips") == 0) mips = 1; - else if (strcmp(argv[i], "--linear") == 0) srgb = 0; - else if (strcmp(argv[i], "--premult") == 0) premult = 1; - else if (strcmp(argv[i], "--mip") == 0 && i + 1 < argc) mip_out = atoi(argv[++i]); - else { usage(); return 2; } - } - - /* Read */ - yapf_image_t src; - yapf_image_t *loaded = NULL; - uint8_t *stb_px = NULL; - memset(&src, 0, sizeof(src)); - if (is_yapf(in)) { - loaded = yapf_load_mt(in, 0); - if (!loaded) { fprintf(stderr, "yapf: cannot read %s (missing, corrupt or not YAPF v1)\n", in); return 1; } - src = *loaded; - } else { - int w, h, ch; - if (stbi_is_16_bit(in)) - fprintf(stderr, "yapf: note: %s has 16-bit channels; YAPF stores 8 bits per channel\n", in); - stb_px = stbi_load(in, &w, &h, &ch, 0); - if (!stb_px) { fprintf(stderr, "yapf: cannot read %s: %s\n", in, stbi_failure_reason()); return 1; } - src.width = (uint32_t)w; src.height = (uint32_t)h; src.channels = (uint8_t)ch; - src.pixels = stb_px; src.mip_levels = 1; - src.flags = srgb ? YAPF_FLAG_SRGB : 0; - src.gpu_format = gpu_hint(ch, srgb); - } - - /* Write */ - int rc = 0; + int from_yapf; + yapf_texture_t *t = read_input(in, &o, &from_yapf); + if (!t) return 1; + int rc; if (is_yapf(out)) { - yapf_image_t dst = src; - if (!loaded) { - if (premult) dst.flags |= YAPF_FLAG_PREMULT_ALPHA; - } - dst.mips = NULL; - dst.mip_levels = 1; - if (mips && build_mips(&dst) != 0) { fprintf(stderr, "yapf: out of memory\n"); rc = 1; } - if (!rc && yapf_save(out, &dst) != YAPF_OK) { fprintf(stderr, "yapf: cannot write %s\n", out); rc = 1; } - if (dst.mips) { - for (int m = 1; m < dst.mip_levels; m++) free(dst.mips[m]); - free(dst.mips); - } + rc = write_yapf(t, out, &o, from_yapf); } else { - const uint8_t *px = src.pixels; - uint32_t w = src.width, h = src.height; - if (mip_out > 0) { - if (!loaded || mip_out >= src.mip_levels) { - fprintf(stderr, "yapf: %s has no mip level %d\n", in, mip_out); - rc = 1; - } else { - px = src.mips[mip_out]; - w = src.width >> mip_out ? src.width >> mip_out : 1; - h = src.height >> mip_out ? src.height >> mip_out : 1; - } - } - if (!rc && !write_image(out, px, (int)w, (int)h, src.channels)) { - fprintf(stderr, "yapf: cannot write %s\n", out); - rc = 1; - } + rc = export_image(t, in, out, &o); } - - yapf_free(loaded); - stbi_image_free(stb_px); + yapf_tex_free(t); return rc; } diff --git a/yapf.c b/yapf.c index d926109..5f49a19 100644 --- a/yapf.c +++ b/yapf.c @@ -396,7 +396,9 @@ YAPF__SIMD_FN void yapf__simd_inverse_row(uint8_t *d, const uint8_t *P0, } __m128i p2 = _mm_loadu_si128((const __m128i *)(const void *)(P2 + k)); __m128i r, g, b; - if (xf) { /* subtract-green */ + if (xf == 2) { /* none (v2) */ + r = p0; g = p1; b = p2; + } else if (xf) { /* subtract-green */ g = p0; r = _mm_add_epi8(_mm_sub_epi8(p1, bias), g); b = _mm_add_epi8(_mm_sub_epi8(p2, bias), g); @@ -520,7 +522,9 @@ YAPF__SIMD_FN void yapf__simd_inverse_row(uint8_t *d, const uint8_t *P0, continue; } uint8x16_t p2 = vld1q_u8(P2 + k), r, g, b; - if (xf) { + if (xf == 2) { + r = p0; g = p1; b = p2; + } else if (xf) { g = p0; r = vaddq_u8(vsubq_u8(p1, bias), g); b = vaddq_u8(vsubq_u8(p2, bias), g); @@ -854,6 +858,943 @@ static void yapf__unfilter_row(uint8_t f, uint8_t *R, const uint8_t *U, * stores the tile raw whenever coding would not make it smaller. */ #define YAPF__SUBGREEN 0x10 +#define YAPF__ENTROPY 0x20 /* v2: entropy plane mask follows */ +#define YAPF__NOXFORM 0x40 /* v2: planes are R, G, B (+ A) as stored */ + +/* ── yapf-sum ────────────────────────────────────────────────────────── */ +/* + * The version 2 checksum. Four independent 64-bit lanes take the 8-byte + * little-endian words of the input in turn (word j goes to lane j mod 4; + * the last word is zero padded): + * a = (a ^ w) * K; a ^= a >> 32; K = 0x9E3779B97F4A7C15 + * Lane i starts at (i + 1) * K ^ n, n = byte length. The result is + * h = a0 ^ rotl(a1, 16) ^ rotl(a2, 32) ^ rotl(a3, 48); + * h *= K; h ^= h >> 32; low 32 bits of h. + * The lanes keep four multiplies in flight, so it runs well above the + * decoder's own speed. + */ +#define YAPF__K 0x9E3779B97F4A7C15ull + +static uint64_t yapf__rotl(uint64_t v, int r) { return (v << r) | (v >> (64 - r)); } + +uint32_t yapf_checksum(const void *data, size_t size) { + const uint8_t *p = (const uint8_t *)data; + uint64_t n = (uint64_t)size; + uint64_t a[4]; + for (int i = 0; i < 4; i++) a[i] = ((uint64_t)(i + 1) * YAPF__K) ^ n; + size_t j = 0, words = size / 8; + for (; j + 4 <= words; j += 4) { + for (int i = 0; i < 4; i++) { + uint64_t v = (a[i] ^ yapf__r64le(p + (j + i) * 8)) * YAPF__K; + a[i] = v ^ (v >> 32); + } + } + for (; j < words; j++) { + uint64_t v = (a[j & 3] ^ yapf__r64le(p + j * 8)) * YAPF__K; + a[j & 3] = v ^ (v >> 32); + } + if (size & 7) { + uint8_t tail[8] = {0}; + memcpy(tail, p + words * 8, size & 7); + uint64_t v = (a[j & 3] ^ yapf__r64le(tail)) * YAPF__K; + a[j & 3] = v ^ (v >> 32); + } + uint64_t h = a[0] ^ yapf__rotl(a[1], 16) ^ yapf__rotl(a[2], 32) ^ yapf__rotl(a[3], 48); + h *= YAPF__K; + h ^= h >> 32; + return (uint32_t)h; +} + +static uint16_t yapf__r16le(const uint8_t *p) { + return (uint16_t)(p[0] | (p[1] << 8)); +} + +static void yapf__w16le(uint8_t *p, uint32_t v) { + p[0] = (uint8_t)v; + p[1] = (uint8_t)(v >> 8); +} + +static void yapf__w64le(uint8_t *p, uint64_t v) { + yapf__w32le(p, (uint32_t)v); + yapf__w32le(p + 4, (uint32_t)(v >> 32)); +} + +/* ── Entropy stage (version 2, optional) ─────────────────────────────── */ +/* + * A surface that uses it starts with a code book: Huffman tables keyed by + * (slot, context). Pixel planes: slot = transform · 16 + plane (transform + * 0 YCoCg-R, 1 subtract-green, 2 none), context = how busy the neighbours + * are (folded residuals left + above, in 7 classes). Block tiles: slot = + * byte lane of the block, context 0. Sharing the tables per surface + * keeps tiles independent and their headers tiny. + * + * Code table: 1 byte n − 1 (n = symbols 0..n−1 described), then n 4-bit + * code lengths (0 = unused, 1..11), low nibble first; an odd + * count is padded with a zero nibble. Kraft sum <= 1. + * Codes are canonical (shorter first, then by symbol, as in deflate) and + * written least significant bit first, so a decoder reads them with one + * 2048-entry table lookup. + */ +#define YAPF__HUFF_BITS 11 +#define YAPF__HUFF_SIZE (1u << YAPF__HUFF_BITS) + +static uint32_t yapf__bitrev(uint32_t c, int n) { + uint32_t r = 0; + for (int i = 0; i < n; i++) { r = (r << 1) | (c & 1); c >>= 1; } + return r; +} + +/* Parses a code table into lut (entry = length << 8 | symbol, 0 = no + * code). Returns the byte after the table, or NULL. */ +static const uint8_t *yapf__huff_table(const uint8_t *p, const uint8_t *end, + uint16_t *lut) +{ + if (p >= end) return NULL; + uint32_t n = (uint32_t)*p++ + 1, nbytes = (n + 1) / 2; + if ((size_t)(end - p) < nbytes) return NULL; + uint8_t len[256]; + uint32_t count[YAPF__HUFF_BITS + 1] = {0}, kraft = 0; + for (uint32_t i = 0; i < n; i++) { + uint8_t l = (uint8_t)((p[i >> 1] >> ((i & 1) * 4)) & 15); + if (l > YAPF__HUFF_BITS) return NULL; + len[i] = l; + count[l]++; + if (l) kraft += YAPF__HUFF_SIZE >> l; + } + if ((n & 1) && (p[n >> 1] >> 4)) return NULL; + if (kraft == 0 || kraft > YAPF__HUFF_SIZE) return NULL; + p += nbytes; + + uint32_t next[YAPF__HUFF_BITS + 2], code = 0; + count[0] = 0; + for (int l = 1; l <= YAPF__HUFF_BITS; l++) { + code = (code + count[l - 1]) << 1; + next[l] = code; + } + memset(lut, 0, YAPF__HUFF_SIZE * sizeof(uint16_t)); + for (uint32_t s = 0; s < n; s++) { + int l = len[s]; + if (!l) continue; + uint32_t r = yapf__bitrev(next[l]++, l); + for (uint32_t k = r; k < YAPF__HUFF_SIZE; k += 1u << l) + lut[k] = (uint16_t)((l << 8) | s); + } + return p; +} + +/* Bit reader for code streams; reading past the end yields zeros and is + * caught by yapf__br_done. */ +typedef struct { + const uint8_t *p; + size_t L, pos; + uint64_t buf, used; + int bits; +} yapf__br; + +static void yapf__br_init(yapf__br *b, const uint8_t *p, size_t L) { + b->p = p; b->L = L; b->pos = 0; b->buf = 0; b->used = 0; b->bits = 0; +} + +static int yapf__br_sym(yapf__br *b, const uint16_t *lut) { + if (b->bits < (int)YAPF__HUFF_BITS) { + if (b->pos + 8 <= b->L) { + b->buf |= yapf__r64le(b->p + b->pos) << b->bits; + b->pos += (size_t)((63 - b->bits) >> 3); + b->bits |= 56; + } else { + while (b->bits <= 56) { + b->buf |= (uint64_t)(b->pos < b->L ? b->p[b->pos] : 0) << b->bits; + b->pos++; + b->bits += 8; + } + } + } + uint16_t e = lut[b->buf & (YAPF__HUFF_SIZE - 1)]; + if (!e) return -1; + int l = e >> 8; + b->buf >>= l; + b->bits -= l; + b->used += (uint64_t)l; + return e & 0xFF; +} + +/* The stream must end in its last byte. */ +static int yapf__br_done(const yapf__br *b) { + return (b->used + 7) / 8 == b->L; +} + +#define YAPF__SLOTS 48 +#define YAPF__CTX 7 + +/* Context of a residual from its left and upper neighbours' folded + * residuals s = l + u: 0, 1–2, 3–6, 7–14, 15–30, 31–62, 63 and up + * (= min(6, floor(log2(s + 1)))). Branch free: the decoder computes it + * on the critical path of every sample. */ +static int yapf__ctx(uint32_t s) { + return (s > 0) + (s > 2) + (s > 6) + (s > 14) + (s > 30) + (s > 62); +} + +/* The same as table offsets for the decoder: ctx(min(s, 63)) << 11. */ +#define YAPF__C(c) (uint16_t)((c) << 11) +static const uint16_t yapf__ctx_off[64] = { /* index min(s, 63) */ + YAPF__C(0), YAPF__C(1), YAPF__C(1), YAPF__C(2), YAPF__C(2), YAPF__C(2), YAPF__C(2), + YAPF__C(3), YAPF__C(3), YAPF__C(3), YAPF__C(3), YAPF__C(3), YAPF__C(3), YAPF__C(3), YAPF__C(3), + YAPF__C(4), YAPF__C(4), YAPF__C(4), YAPF__C(4), YAPF__C(4), YAPF__C(4), YAPF__C(4), YAPF__C(4), + YAPF__C(4), YAPF__C(4), YAPF__C(4), YAPF__C(4), YAPF__C(4), YAPF__C(4), YAPF__C(4), YAPF__C(4), + YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), + YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), + YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), + YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), YAPF__C(5), + YAPF__C(6) +}; +#undef YAPF__C + +/* Decoder side of a code book: per slot one array of 7 × 2048 lookup + * entries (contexts without a table stay 0 = invalid), so context and + * bits form a single index. */ +typedef struct { + uint16_t *slot[YAPF__SLOTS]; + uint16_t *mem; +} yapf__dbook; + +/* Book: u16 table count, then per table: slot, context, code table. */ +static int yapf__book_read(const uint8_t *p, size_t len, yapf__dbook *bk) { + const uint8_t *end = p + len, *q; + uint8_t seen[YAPF__SLOTS][YAPF__CTX]; + uint32_t slots = 0; + memset(bk, 0, sizeof(*bk)); + memset(seen, 0, sizeof(seen)); + if (len < 2) return -1; + uint32_t n = yapf__r16le(p); + p += 2; + if (n == 0 || n > YAPF__SLOTS * YAPF__CTX) return -1; + /* first pass: validate the layout and count the slots in use */ + for (uint32_t i = 0, k; i < n; i++) { + if (end - p < 3) return -1; + uint8_t slot = p[0], ctx = p[1]; + if (slot >= YAPF__SLOTS || ctx >= YAPF__CTX || seen[slot][ctx]) return -1; + seen[slot][ctx] = 1; + for (k = 0; k < YAPF__CTX && (k == ctx || !seen[slot][k]); k++) {} + if (k == YAPF__CTX) slots++; /* first table of this slot */ + uint32_t ns = (uint32_t)p[2] + 1; + p += 3 + (ns + 1) / 2; + if (p > end) return -1; + } + if (p != end) return -1; + bk->mem = (uint16_t *)calloc((size_t)slots * YAPF__CTX * YAPF__HUFF_SIZE, sizeof(uint16_t)); + if (!bk->mem) return -1; + uint32_t used = 0; + for (q = end - len + 2; q < end; ) { + uint8_t slot = q[0], ctx = q[1]; + if (!bk->slot[slot]) bk->slot[slot] = bk->mem + (size_t)used++ * YAPF__CTX * YAPF__HUFF_SIZE; + q = yapf__huff_table(q + 2, end, bk->slot[slot] + (size_t)ctx * YAPF__HUFF_SIZE); + if (!q) return -1; + } + return 0; +} + +/* Entropy plane payload: filters, u16 stream length, the folded + * residuals in raster order, each with the table of its context. + * + * Every residual's context needs the one decoded just before it, so one + * stream is a chain of dependent steps. The planes of a tile are + * independent streams, though: the decoder runs up to four of them in + * the same loop and the CPU overlaps their chains. */ +typedef struct { + const uint8_t *p; + size_t L; + const uint16_t *lut; + uint8_t *z; +} yapf__hstream; + +/* Parses the header of an entropy plane into *s; returns the end of the + * payload or NULL. */ +static const uint8_t *yapf__huff_head(const uint8_t *p, const uint8_t *end, + const uint8_t **filt_out, uint32_t th, + const yapf__dbook *bk, int slot, + uint8_t *z, yapf__hstream *s) +{ + uint32_t nb = (th + YAPF__BAND - 1) / YAPF__BAND; + if (!bk || !bk->slot[slot] || (size_t)(end - p) < nb + 2) return NULL; + *filt_out = p; + for (uint32_t b = 0; b < nb; b++) + if (!yapf__filter_valid(p[b])) return NULL; + p += nb; + s->L = yapf__r16le(p); + p += 2; + if ((size_t)(end - p) < s->L) return NULL; + s->p = p; + s->lut = bk->slot[slot]; + s->z = z; + return p + s->L; +} + +#define YAPF__HSTATE(k) \ + uint64_t buf##k = 0; \ + int bits##k = 0; \ + size_t pos##k = 0; \ + uint32_t l##k = 0; \ + const uint8_t *P##k = s[k].p, *U##k = yapf__zero_row; \ + const size_t L##k = s[k].L; \ + const uint16_t *T##k = s[k].lut; \ + uint8_t *Z##k = s[k].z; + +#define YAPF__HROW(k) \ + U##k = y ? Z##k + i - tw : yapf__zero_row; \ + l##k = 0; + +#define YAPF__HSTEP(k) \ + { \ + if (pos##k + 8 <= L##k) { /* branch free while not near the end */ \ + buf##k |= yapf__r64le(P##k + pos##k) << bits##k; \ + pos##k += (size_t)((63 - bits##k) >> 3); \ + bits##k |= 56; \ + } else if (bits##k < (int)YAPF__HUFF_BITS) { \ + while (bits##k <= 56) { \ + buf##k |= (uint64_t)(pos##k < L##k ? P##k[pos##k] : 0) << bits##k; \ + pos##k++; \ + bits##k += 8; \ + } \ + } \ + uint32_t s_ = l##k + U##k[x]; \ + uint32_t e_ = T##k[yapf__ctx_off[s_ < 63 ? s_ : 63] | \ + (uint32_t)(buf##k & (YAPF__HUFF_SIZE - 1))]; \ + int n_ = (int)(e_ >> 8); \ + if (!e_) return -1; \ + buf##k >>= n_; \ + bits##k -= n_; \ + l##k = e_ & 0xFF; \ + Z##k[i] = (uint8_t)l##k; \ + } + +/* bits consumed = 8·pos − bits still buffered; the stream must end in + * its last byte */ +#define YAPF__HEND(k) if ((8 * (uint64_t)pos##k - (uint64_t)bits##k + 7) / 8 != L##k) return -1; + +static int yapf__hdec1(const yapf__hstream *s, uint32_t tw, uint32_t th) { + YAPF__HSTATE(0) + for (uint32_t y = 0, i = 0; y < th; y++) { + YAPF__HROW(0) + for (uint32_t x = 0; x < tw; x++, i++) YAPF__HSTEP(0) + } + YAPF__HEND(0) + return 0; +} + +static int yapf__hdec2(const yapf__hstream *s, uint32_t tw, uint32_t th) { + YAPF__HSTATE(0) YAPF__HSTATE(1) + for (uint32_t y = 0, i = 0; y < th; y++) { + YAPF__HROW(0) YAPF__HROW(1) + for (uint32_t x = 0; x < tw; x++, i++) { YAPF__HSTEP(0) YAPF__HSTEP(1) } + } + YAPF__HEND(0) YAPF__HEND(1) + return 0; +} + +static int yapf__hdec3(const yapf__hstream *s, uint32_t tw, uint32_t th) { + YAPF__HSTATE(0) YAPF__HSTATE(1) YAPF__HSTATE(2) + for (uint32_t y = 0, i = 0; y < th; y++) { + YAPF__HROW(0) YAPF__HROW(1) YAPF__HROW(2) + for (uint32_t x = 0; x < tw; x++, i++) { YAPF__HSTEP(0) YAPF__HSTEP(1) YAPF__HSTEP(2) } + } + YAPF__HEND(0) YAPF__HEND(1) YAPF__HEND(2) + return 0; +} + +static int yapf__hdec4(const yapf__hstream *s, uint32_t tw, uint32_t th) { + YAPF__HSTATE(0) YAPF__HSTATE(1) YAPF__HSTATE(2) YAPF__HSTATE(3) + for (uint32_t y = 0, i = 0; y < th; y++) { + YAPF__HROW(0) YAPF__HROW(1) YAPF__HROW(2) YAPF__HROW(3) + for (uint32_t x = 0; x < tw; x++, i++) { + YAPF__HSTEP(0) YAPF__HSTEP(1) YAPF__HSTEP(2) YAPF__HSTEP(3) + } + } + YAPF__HEND(0) YAPF__HEND(1) YAPF__HEND(2) YAPF__HEND(3) + return 0; +} + +/* Decodes n (1..) streams, four at a time; zeroes each tail group. */ +static int yapf__huff_run(const yapf__hstream *s, int n, uint32_t tw, uint32_t th) { + for (int k = 0; k < n; k += 4) { + int m = n - k < 4 ? n - k : 4, rc; + rc = m == 4 ? yapf__hdec4(s + k, tw, th) : m == 3 ? yapf__hdec3(s + k, tw, th) + : m == 2 ? yapf__hdec2(s + k, tw, th) : yapf__hdec1(s + k, tw, th); + if (rc != 0) return -1; + } + for (int k = 0; k < n; k++) memset(s[k].z + (size_t)tw * th, 0, YAPF__GROUP); + return 0; +} + +/* One entropy plane on its own. */ +static const uint8_t *yapf__huff_plane(const uint8_t *p, const uint8_t *end, + uint8_t *z, const uint8_t **filt_out, + uint32_t tw, uint32_t th, + const yapf__dbook *bk, int slot) +{ + yapf__hstream s; + p = yapf__huff_head(p, end, filt_out, th, bk, slot, z, &s); + if (!p || yapf__huff_run(&s, 1, tw, th) != 0) return NULL; + return p; +} + +/* Code lengths (<= 11 bits) for freq[0..n). Plain Huffman by the + * two-queue method; if a code comes out too long the counts are halved + * and it is built again. */ +static void yapf__huff_lengths(const uint32_t *freq_in, uint32_t n, uint8_t *len) { + uint32_t freq[256], sym[256], m = 0; + memcpy(freq, freq_in, n * sizeof(uint32_t)); + for (;;) { + m = 0; + for (uint32_t s = 0; s < n; s++) { len[s] = 0; if (freq[s]) sym[m++] = s; } + if (m == 0) return; + if (m == 1) { len[sym[0]] = 1; return; } + /* insertion sort by frequency (m <= 256) */ + for (uint32_t i = 1; i < m; i++) { + uint32_t v = sym[i], j = i; + while (j > 0 && freq[sym[j - 1]] > freq[v]) { sym[j] = sym[j - 1]; j--; } + sym[j] = v; + } + /* nodes 0..m-1 leaves (sorted), m..2m-2 internal in creation order */ + uint64_t w[512]; + uint32_t parent[512], li = 0, ii = m, nn = m; + for (uint32_t i = 0; i < m; i++) w[i] = freq[sym[i]]; + for (uint32_t k = 0; k + 1 < m; k++) { + uint32_t pick[2]; + for (int t = 0; t < 2; t++) { + if (li < m && (ii >= nn || w[li] <= w[ii])) pick[t] = li++; + else pick[t] = ii++; + } + w[nn] = w[pick[0]] + w[pick[1]]; + parent[pick[0]] = parent[pick[1]] = nn; + nn++; + } + uint8_t depth[512]; + depth[nn - 1] = 0; + int maxl = 0; + for (uint32_t k = nn - 1; k-- > 0; ) { + depth[k] = (uint8_t)(depth[parent[k]] + 1); + if (k < m && depth[k] > maxl) maxl = depth[k]; + } + if (maxl <= (int)YAPF__HUFF_BITS) { + for (uint32_t i = 0; i < m; i++) len[sym[i]] = depth[i]; + return; + } + for (uint32_t s = 0; s < n; s++) if (freq[s]) freq[s] = (freq[s] >> 1) | 1; + } +} + +typedef struct { + uint64_t acc; + int bits; + yapf__out *o; +} yapf__bitw; + +static void yapf__bitw_put(yapf__bitw *b, uint32_t code, int len) { + b->acc |= (uint64_t)code << b->bits; + b->bits += len; + while (b->bits >= 8) { + b->o->buf[b->o->len++] = (uint8_t)b->acc; + b->acc >>= 8; + b->bits -= 8; + } +} + +static void yapf__bitw_flush(yapf__bitw *b) { + if (b->bits > 0) b->o->buf[b->o->len++] = (uint8_t)b->acc; + b->acc = 0; + b->bits = 0; +} + +/* Canonical codes for lengths len[0..ns). */ +static void yapf__huff_codes(const uint8_t *len, uint32_t ns, uint16_t *code) { + uint32_t count[YAPF__HUFF_BITS + 1] = {0}, next[YAPF__HUFF_BITS + 2], c = 0; + for (uint32_t s = 0; s < ns; s++) if (len[s]) count[len[s]]++; + for (int l = 1; l <= (int)YAPF__HUFF_BITS; l++) { + c = (c + count[l - 1]) << 1; + next[l] = c; + } + for (uint32_t s = 0; s < ns; s++) + code[s] = len[s] ? (uint16_t)yapf__bitrev(next[len[s]]++, len[s]) : 0; +} + +typedef struct { /* encoder side of a code book */ + uint32_t freq[YAPF__SLOTS][YAPF__CTX][256]; + uint8_t len[YAPF__SLOTS][YAPF__CTX][256]; + uint16_t code[YAPF__SLOTS][YAPF__CTX][256]; +} yapf__ebook; + +/* Encoder modes for the tile writers. */ +#define YAPF__ENC_PLAIN 0 /* groups only */ +#define YAPF__ENC_STATS 1 /* groups, and count symbols for a book */ +#define YAPF__ENC_BOOK 2 /* entropy planes / packed blocks */ + +/* Builds every table that has symbols and writes the book. */ +static int yapf__book_write(yapf__ebook *eb, yapf__out *o) { + uint32_t n = 0; + if (yapf__reserve(o, 2) != 0) return -1; + size_t at = o->len; + o->len += 2; + memset(eb->len, 0, sizeof(eb->len)); + for (int s = 0; s < YAPF__SLOTS; s++) + for (int c = 0; c < YAPF__CTX; c++) { + const uint32_t *f = eb->freq[s][c]; + uint32_t ns = 256; + while (ns > 0 && !f[ns - 1]) ns--; + if (ns == 0) continue; + yapf__huff_lengths(f, ns, eb->len[s][c]); + yapf__huff_codes(eb->len[s][c], ns, eb->code[s][c]); + if (yapf__reserve(o, 3 + 128) != 0) return -1; + o->buf[o->len++] = (uint8_t)s; + o->buf[o->len++] = (uint8_t)c; + o->buf[o->len++] = (uint8_t)(ns - 1); + for (uint32_t k = 0; k < ns; k += 2) + o->buf[o->len++] = (uint8_t)(eb->len[s][c][k] | (k + 1 < ns ? eb->len[s][c][k + 1] << 4 : 0)); + n++; + } + yapf__w16le(o->buf + at, n); + return n ? 0 : 1; /* 1: nothing to code */ +} + +static void yapf__plane_stats(yapf__ebook *eb, int slot, const uint8_t *z, uint32_t tw, uint32_t th) { + for (uint32_t y = 0, i = 0; y < th; y++) + for (uint32_t x = 0; x < tw; x++, i++) { + uint32_t s = (x ? z[i - 1] : 0u) + (y ? z[i - tw] : 0u); + eb->freq[slot][yapf__ctx(s)][z[i]]++; + } +} + +/* Entropy plane payload (see yapf__huff_plane). Returns 1 when some + * symbol has no code in the book (the caller keeps the group form). */ +static int yapf__huff_emit_plane(yapf__out *o, const uint8_t *z, const uint8_t *filt, + uint32_t tw, uint32_t th, const yapf__ebook *eb, int slot) +{ + uint32_t nb = (th + YAPF__BAND - 1) / YAPF__BAND; + uint64_t bits = 0; + for (uint32_t y = 0, i = 0; y < th; y++) + for (uint32_t x = 0; x < tw; x++, i++) { + uint32_t s = (x ? z[i - 1] : 0u) + (y ? z[i - tw] : 0u); + uint8_t l = eb->len[slot][yapf__ctx(s)][z[i]]; + if (!l) return 1; + bits += l; + } + size_t L = (size_t)((bits + 7) / 8); + if (L > 0xFFFF) return 1; + if (yapf__reserve(o, nb + 2 + L + 8) != 0) return -1; + memcpy(o->buf + o->len, filt, nb); + o->len += nb; + yapf__w16le(o->buf + o->len, (uint32_t)L); + o->len += 2; + yapf__bitw bw = { 0, 0, o }; + for (uint32_t y = 0, i = 0; y < th; y++) + for (uint32_t x = 0; x < tw; x++, i++) { + uint32_t s = (x ? z[i - 1] : 0u) + (y ? z[i - tw] : 0u); + int c = yapf__ctx(s); + yapf__bitw_put(&bw, eb->code[slot][c][z[i]], eb->len[slot][c][z[i]]); + } + yapf__bitw_flush(&bw); + return 0; +} + +/* ── Plane set emission (version 2) ──────────────────────────────────── */ +/* + * Writes a coded v2 tile: header byte, (wide) u16 constant mask, entropy + * mask if any plane is entropy coded, constant values, payloads. With a + * book (mode BOOK) each plane is coded both ways and the entropy form is + * kept only if it is at least 1/32 smaller; mode STATS feeds the book. + */ +typedef struct { + int np; /* planes */ + int wide; /* masks are u16 (wide samples) */ + uint32_t cmask; /* constant planes */ + const uint8_t *cval; /* constant value per plane */ + const uint8_t *z[16]; /* folded residuals per plane */ + const uint8_t *filt[16]; +} yapf__planeset; + +static int yapf__emit_planeset(yapf__out *o, yapf__out *tmp, uint8_t hdr, + const yapf__planeset *ps, uint32_t tw, uint32_t th, + int mode, yapf__ebook *eb) +{ + size_t off[17]; + uint32_t emask = 0; + int tsel = (hdr & YAPF__SUBGREEN) ? 1 : (hdr & YAPF__NOXFORM) ? 2 : 0; + tmp->len = 0; + for (int c = 0; c < ps->np; c++) { + off[c] = tmp->len; + if (ps->cmask & (1u << c)) continue; + if (yapf__emit_plane(tmp, ps->z[c], ps->filt[c], tw, th) != 0) return -1; + if (mode == YAPF__ENC_STATS) yapf__plane_stats(eb, tsel * 16 + c, ps->z[c], tw, th); + if (mode != YAPF__ENC_BOOK) continue; + size_t gsz = tmp->len - off[c]; + int r = yapf__huff_emit_plane(tmp, ps->z[c], ps->filt[c], tw, th, eb, tsel * 16 + c); + if (r < 0) return -1; + size_t hsz = r ? (size_t)-1 : tmp->len - off[c] - gsz; + if (!r && hsz + gsz / 32 < gsz) { + memmove(tmp->buf + off[c], tmp->buf + off[c] + gsz, hsz); + tmp->len = off[c] + hsz; + emask |= 1u << c; + } else { + tmp->len = off[c] + gsz; + } + } + if (yapf__reserve(o, 5 + (size_t)ps->np + tmp->len) != 0) return -1; + o->buf[o->len++] = (uint8_t)(hdr | (emask ? YAPF__ENTROPY : 0)); + if (ps->wide) { yapf__w16le(o->buf + o->len, ps->cmask); o->len += 2; } + if (emask) { + if (ps->wide) { yapf__w16le(o->buf + o->len, emask); o->len += 2; } + else o->buf[o->len++] = (uint8_t)emask; + } + for (int c = 0; c < ps->np; c++) + if (ps->cmask & (1u << c)) o->buf[o->len++] = ps->cval[c]; + if (tmp->len) memcpy(o->buf + o->len, tmp->buf, tmp->len); + o->len += tmp->len; + return 0; +} + +/* ── Wide samples (version 2) ────────────────────────────────────────── */ +/* + * 16- and 32-bit samples are coded as byte planes with the 8-bit + * machinery. Floats first become order-preserving integers (sign set: + * all bits flipped; else the sign bit set), so nearby values stay nearby. + * The colour transforms are the 8-bit ones at the sample width (mod 2^16 + * or 2^32); a third choice keeps R, G, B as they are. Channel c gives + * planes c·B .. c·B + B − 1, most significant byte first, so the smooth + * high bytes and the noisy low bytes each get their own filters. + */ +#define YAPF__MAX_PLANES 16 +#define YAPF__ROW (YAPF_TILE_SIZE + 1) + +static size_t yapf__sbytes(int sample) { + return sample == YAPF_SAMPLE_U8 ? 1 : sample == YAPF_SAMPLE_F32 ? 4 : 2; +} + +static uint32_t yapf__vmask(int bits) { + return bits == 32 ? 0xFFFFFFFFu : (1u << bits) - 1; +} + +static uint32_t yapf__fmap(uint32_t v, int bits) { + uint32_t s = 1u << (bits - 1); + return (v & s) ? (~v & yapf__vmask(bits)) : (v | s); +} + +static uint32_t yapf__funmap(uint32_t v, int bits) { + uint32_t s = 1u << (bits - 1); + return (v & s) ? (v ^ s) : (~v & yapf__vmask(bits)); +} + +/* Arithmetic shift right by one of a bits-wide two's complement value. */ +static uint32_t yapf__sar1(uint32_t v, int bits) { + return ((v >> 1) | (v & (1u << (bits - 1)))) & yapf__vmask(bits); +} + +static uint32_t yapf__get_sample(const uint8_t *p, int sample) { + if (sample == YAPF_SAMPLE_U8) return *p; + if (sample == YAPF_SAMPLE_F32) { uint32_t v; memcpy(&v, p, 4); return v; } + uint16_t v; memcpy(&v, p, 2); return v; +} + +static void yapf__put_sample(uint8_t *p, int sample, uint32_t v) { + if (sample == YAPF_SAMPLE_U8) { *p = (uint8_t)v; return; } + if (sample == YAPF_SAMPLE_F32) { memcpy(p, &v, 4); return; } + uint16_t h = (uint16_t)v; + memcpy(p, &h, 2); +} + +typedef struct { + uint8_t plane[2][YAPF__MAX_PLANES][YAPF__TILE_PX]; + uint8_t z[2][YAPF__MAX_PLANES][YAPF__TILE_PX + YAPF__GROUP]; + uint8_t filt[2][YAPF__MAX_PLANES][YAPF_TILE_SIZE / YAPF__BAND]; + uint32_t val[4][YAPF__TILE_PX]; +} yapf__wscratch; + +static int yapf__encode_tile_wide( + yapf__out *o, yapf__out *tmp, yapf__wscratch *sc, + const uint8_t *pixels, uint32_t img_w, + uint32_t tx, uint32_t ty, uint32_t tw, uint32_t th, + int ch, int sample, int mode, yapf__ebook *eb) +{ + int sb = (int)yapf__sbytes(sample), bits = 8 * sb; + int isf = sample == YAPF_SAMPLE_F16 || sample == YAPF_SAMPLE_F32; + int np = ch * sb; + uint32_t total = tw * th, half = 1u << (bits - 1), m = yapf__vmask(bits); + size_t start = o->len, raw = (size_t)total * ch * sb; + int tries = ch >= 3 ? 3 : 1, cur = 0; + uint32_t best_est = UINT32_MAX, cmask[2] = { 0, 0 }; + uint8_t cval[2][YAPF__MAX_PLANES]; + uint8_t xfs[2] = { 0, 0 }; + + /* t = 0: YCoCg-R, 1: subtract-green, 2: none (only with 3+ channels; + * with 1–2 channels the planes are the channels). */ + for (int t = 0; t < tries; t++) { + for (uint32_t y = 0; y < th; y++) { + const uint8_t *row = pixels + ((size_t)(ty + y) * img_w + tx) * (size_t)ch * sb; + for (uint32_t x = 0; x < tw; x++) { + uint32_t v[4], i = y * tw + x; + for (int c = 0; c < ch; c++) { + v[c] = yapf__get_sample(row + ((size_t)x * ch + c) * sb, sample); + if (isf) v[c] = yapf__fmap(v[c], bits); + } + if (ch >= 3 && t == 0) { + uint32_t r = v[0], g = v[1], b = v[2]; + uint32_t o_ = (r - b) & m, t_ = (b + yapf__sar1(o_, bits)) & m; + uint32_t q = (g - t_) & m; + v[0] = (t_ + yapf__sar1(q, bits)) & m; + v[1] = (o_ + half) & m; + v[2] = (q + half) & m; + } else if (ch >= 3 && t == 1) { + uint32_t r = v[0], g = v[1], b = v[2]; + v[0] = g; + v[1] = (r - g + half) & m; + v[2] = (b - g + half) & m; + } + for (int c = 0; c < ch; c++) sc->val[c][i] = v[c]; + } + } + /* planes, constant mask, filters, size estimate */ + uint8_t (*plane)[YAPF__TILE_PX] = sc->plane[cur]; + uint32_t est = 0, cm = 0; + for (int c = 0; c < ch; c++) + for (int k = 0; k < sb; k++) { + int pl = c * sb + k, sh = 8 * (sb - 1 - k); + uint8_t *P = plane[pl]; + for (uint32_t i = 0; i < total; i++) P[i] = (uint8_t)(sc->val[c][i] >> sh); + uint32_t i = 1; + while (i < total && P[i] == P[0]) i++; + if (i == total) { cm |= 1u << pl; cval[cur][pl] = P[0]; est += 16; continue; } + est += yapf__choose_filters(P, tw, th, sc->z[cur][pl], sc->filt[cur][pl]); + } + if (est < best_est) { + best_est = est; + cmask[cur] = cm; + xfs[cur] = (uint8_t)t; + cur ^= 1; /* keep this set, work in the other one */ + } + } + int keep = cur ^ 1; + uint8_t hdr = ch < 3 ? 0 : xfs[keep] == 1 ? YAPF__SUBGREEN : xfs[keep] == 2 ? YAPF__NOXFORM : 0; + + yapf__planeset ps; + memset(&ps, 0, sizeof(ps)); + ps.np = np; + ps.wide = 1; + ps.cmask = cmask[keep]; + ps.cval = cval[keep]; + for (int c = 0; c < np; c++) { ps.z[c] = sc->z[keep][c]; ps.filt[c] = sc->filt[keep][c]; } + if (yapf__emit_planeset(o, tmp, hdr, &ps, tw, th, mode, eb) != 0) return -1; + + if (o->len - start > raw + 1) { /* stored: raw LE samples */ + o->len = start; + if (yapf__reserve(o, raw + 1) != 0) return -1; + o->buf[o->len++] = YAPF__STORED; + for (uint32_t y = 0; y < th; y++) { + const uint8_t *row = pixels + ((size_t)(ty + y) * img_w + tx) * (size_t)ch * sb; + for (uint32_t i = 0; i < tw * (uint32_t)ch; i++) { + uint32_t v = yapf__get_sample(row + (size_t)i * sb, sample); + for (int k = 0; k < sb; k++) o->buf[o->len++] = (uint8_t)(v >> (8 * k)); + } + } + } + return 0; +} + +typedef struct { + uint8_t plane[YAPF__MAX_PLANES][YAPF_TILE_SIZE * YAPF__ROW]; + uint8_t z[YAPF__TILE_PX + YAPF_TILE_SIZE]; +} yapf__wdec; + +static int yapf__decode_tile_wide( + const uint8_t *data, size_t len, + uint8_t *pixels, uint32_t img_w, + uint32_t tx, uint32_t ty, uint32_t tw, uint32_t th, + int ch, int sample, int simd, const yapf__dbook *bk, yapf__wdec *sc) +{ + int sb = (int)yapf__sbytes(sample), bits = 8 * sb, np = ch * sb; + int isf = sample == YAPF_SAMPLE_F16 || sample == YAPF_SAMPLE_F32; + uint32_t half = 1u << (bits - 1), m = yapf__vmask(bits); + const uint8_t *p = data, *end = data + len; + size_t rowb = (size_t)tw * ch * sb; + + if (len == 0) return -1; + uint8_t hdr = *p++; + if (hdr == YAPF__STORED) { + if (len != 1 + rowb * th) return -1; + for (uint32_t y = 0; y < th; y++) { + uint8_t *d = pixels + ((size_t)(ty + y) * img_w + tx) * (size_t)ch * sb; + for (uint32_t i = 0; i < tw * (uint32_t)ch; i++, p += sb) { + uint32_t v = 0; + for (int k = sb; k-- > 0; ) v = (v << 8) | p[k]; + yapf__put_sample(d + (size_t)i * sb, sample, v); + } + } + return 0; + } + if (hdr & 0x8F) return -1; + if ((hdr & YAPF__NOXFORM) && (hdr & YAPF__SUBGREEN)) return -1; + if (ch < 3 && (hdr & (YAPF__NOXFORM | YAPF__SUBGREEN))) return -1; + int xf = (hdr & YAPF__NOXFORM) ? 2 : (hdr & YAPF__SUBGREEN) ? 1 : 0; + if (ch < 3) xf = 2; + + if (end - p < 2) return -1; + uint32_t cmask = yapf__r16le(p), emask = 0; + p += 2; + if (cmask >> np) return -1; + if (hdr & YAPF__ENTROPY) { + if (end - p < 2 || !bk) return -1; + emask = yapf__r16le(p); + p += 2; + if ((emask >> np) || (emask & cmask)) return -1; + } + for (int pl = 0; pl < np; pl++) { + if (!(cmask & (1u << pl))) continue; + if (p >= end) return -1; + memset(sc->plane[pl], *p++, sizeof(sc->plane[pl])); + } + for (int pl = 0; pl < np; pl++) { + if (cmask & (1u << pl)) continue; + const uint8_t *filt = NULL; + memset(sc->z + (size_t)tw * th, 0, sizeof(sc->z) - (size_t)tw * th); + p = (emask & (1u << pl)) + ? yapf__huff_plane(p, end, sc->z, &filt, tw, th, bk, (ch < 3 ? 0 : xf) * 16 + pl) + : yapf__unpack_plane(p, end, sc->z, &filt, tw, th, simd); + if (!p) return -1; + if (tw != YAPF_TILE_SIZE) { + size_t used = ((size_t)tw * th + YAPF__GROUP - 1) / YAPF__GROUP * YAPF__GROUP; + memset(sc->z + used, 0, sizeof(sc->z) - used); + } + uint8_t *P = sc->plane[pl]; + for (uint32_t y = 0; y < th; y++) { + uint8_t f = filt[y / YAPF__BAND]; + uint32_t k = f >= YAPF__F_UP ? (uint32_t)(f - YAPF__F_UP + 1) : 1; + uint8_t *R = P + (size_t)y * YAPF__ROW; + const uint8_t *U = y ? R - YAPF__ROW : yapf__zero_row; + const uint8_t *K = y >= k ? R - (size_t)k * YAPF__ROW : yapf__zero_row; + const uint8_t *Z = sc->z + (size_t)y * tw; +#if YAPF__SIMD + if (simd) { + if (f == YAPF__F_LEFT || f == YAPF__F_GRAD) + yapf__simd_serial_row(f, R, U, Z, tw); + else if (f >= YAPF__F_UP) + yapf__simd_vertical_row(R, K, K, Z, tw); + else if (f == YAPF__F_UPAVG) + yapf__simd_vertical_row(R, U, U + 1, Z, tw); + else + yapf__simd_vertical_row(R, U + 1, U + 1, Z, tw); + R[tw] = R[tw - 1]; + continue; + } +#endif + yapf__unfilter_row(f, R, U, K, Z, tw); + } + } + if (p != end) return -1; + + for (uint32_t y = 0; y < th; y++) { + uint8_t *d = pixels + ((size_t)(ty + y) * img_w + tx) * (size_t)ch * sb; + size_t r = (size_t)y * YAPF__ROW; + for (uint32_t x = 0; x < tw; x++) { + uint32_t v[4]; + for (int c = 0; c < ch; c++) { + uint32_t a = 0; + for (int k = 0; k < sb; k++) a = (a << 8) | sc->plane[c * sb + k][r + x]; + v[c] = a; + } + if (xf == 0) { + uint32_t o_ = (v[1] - half) & m, q = (v[2] - half) & m; + uint32_t t_ = (v[0] - yapf__sar1(q, bits)) & m; + uint32_t g = (q + t_) & m; + uint32_t b = (t_ - yapf__sar1(o_, bits)) & m; + v[0] = (b + o_) & m; + v[1] = g; + v[2] = b; + } else if (xf == 1) { + uint32_t g = v[0]; + v[0] = (v[1] + g - half) & m; + v[2] = (v[2] + g - half) & m; + v[1] = g; + } + for (int c = 0; c < ch; c++) + yapf__put_sample(d + ((size_t)x * ch + c) * sb, sample, + isf ? yapf__funmap(v[c], bits) : v[c]); + } + } + return 0; +} + +/* ── GPU block tiles (version 2) ─────────────────────────────────────── */ +/* + * A block tile holds up to 16 × 16 blocks, rows of blocks top to bottom, + * exactly as the GPU takes them. Packed tiles (bit 31 of the tile size) + * code the bytes lane by lane — byte 0 of every block, then byte 1, … — + * each lane with its own table of the surface's code book (slot = lane): + * endpoint bytes and index bytes have very different statistics. The + * bit stream fills the tile exactly. + */ +static int yapf__decode_block_tile(const uint8_t *src, size_t len, int packed, + uint8_t *dst, size_t dst_row, + uint32_t tbw, uint32_t tbh, uint32_t S, + const yapf__dbook *bk) +{ + size_t n = (size_t)tbw * tbh, raw = n * S; + if (!packed) { + if (len != raw) return -1; + for (uint32_t r = 0; r < tbh; r++) + memcpy(dst + r * dst_row, src + (size_t)r * tbw * S, (size_t)tbw * S); + return 0; + } + if (!bk) return -1; + yapf__br br; + yapf__br_init(&br, src, len); + for (uint32_t k = 0; k < S; k++) { + const uint16_t *lut = bk->slot[k]; + if (!lut) return -1; + for (size_t i = 0; i < n; i++) { + int v = yapf__br_sym(&br, lut); + if (v < 0) return -1; + dst[(i / tbw) * dst_row + (i % tbw) * S + k] = (uint8_t)v; + } + } + return yapf__br_done(&br) ? 0 : -1; +} + +/* Writes one block tile; *packed is set when the packed form is used. */ +static int yapf__encode_block_tile(yapf__out *o, const uint8_t *src, size_t src_row, + uint32_t tbw, uint32_t tbh, uint32_t S, + int mode, yapf__ebook *eb, int *packed) +{ + size_t n = (size_t)tbw * tbh, raw = n * S; + *packed = 0; + if (mode == YAPF__ENC_STATS) + for (uint32_t r = 0; r < tbh; r++) + for (size_t i = 0; i < (size_t)tbw * S; i++) + eb->freq[i % S][0][src[r * src_row + i]]++; + if (mode == YAPF__ENC_BOOK) { + uint64_t bits = 0; + for (uint32_t r = 0; r < tbh && bits != UINT64_MAX; r++) + for (size_t i = 0; i < (size_t)tbw * S; i++) { + uint8_t l = eb->len[i % S][0][src[r * src_row + i]]; + if (!l) { bits = UINT64_MAX; break; } + bits += l; + } + if (bits != UINT64_MAX && (bits + 7) / 8 < raw) { + if (yapf__reserve(o, (size_t)(bits + 7) / 8 + 8) != 0) return -1; + yapf__bitw bw = { 0, 0, o }; + for (uint32_t k = 0; k < S; k++) /* lane by lane */ + for (size_t i = 0; i < n; i++) { + uint8_t v = src[(i / tbw) * src_row + (i % tbw) * S + k]; + yapf__bitw_put(&bw, eb->code[k][0][v], eb->len[k][0][v]); + } + yapf__bitw_flush(&bw); + *packed = 1; + return 0; + } + } + if (yapf__reserve(o, raw) != 0) return -1; + for (uint32_t r = 0; r < tbh; r++) { + memcpy(o->buf + o->len, src + r * src_row, (size_t)tbw * S); + o->len += (size_t)tbw * S; + } + return 0; +} /* Encoder scratch, heap allocated once per level (too big for small * thread stacks). */ @@ -867,7 +1808,7 @@ static int yapf__encode_tile( yapf__out *o, yapf__scratch *sc, const uint8_t *pixels, uint32_t img_w, uint32_t tx, uint32_t ty, uint32_t tw, uint32_t th, - int ch) + int ch, int mode, yapf__ebook *eb, yapf__out *tmp) { uint32_t total = tw * th; size_t start = o->len; @@ -909,6 +1850,21 @@ static int yapf__encode_tile( } int t = (tries == 2 && est[1] < est[0]) ? 1 : 0; + if (mode != YAPF__ENC_PLAIN) { /* v2: statistics / entropy planes */ + yapf__planeset ps; + uint8_t cval[4]; + memset(&ps, 0, sizeof(ps)); + ps.np = ch; + ps.cmask = mask[t]; + ps.cval = cval; + for (int c = 0; c < ch; c++) { + cval[c] = sc->plane[t][c][0]; + ps.z[c] = sc->z[t][c]; + ps.filt[c] = sc->filt[t][c]; + } + if (yapf__emit_planeset(o, tmp, (uint8_t)(mask[t] | (t ? YAPF__SUBGREEN : 0)), + &ps, tw, th, mode, eb) != 0) return -1; + } else { if (yapf__reserve(o, 1 + 4) != 0) return -1; o->buf[o->len++] = (uint8_t)(mask[t] | (t ? YAPF__SUBGREEN : 0)); for (int c = 0; c < ch; c++) @@ -916,6 +1872,7 @@ static int yapf__encode_tile( for (int c = 0; c < ch; c++) if (!(mask[t] & (1u << c)) && yapf__emit_plane(o, sc->z[t][c], sc->filt[t][c], tw, th) != 0) return -1; + } /* Fall back to a stored tile if coding did not pay off. */ if (o->len - start > raw + 1) { @@ -931,11 +1888,14 @@ static int yapf__encode_tile( return 0; } +/* v2 (version 2 files) additionally allows bit 5 (a mask byte follows: + * bit c = plane c is entropy coded with book bk) and bit 6 (no colour + * transform). */ static int yapf__decode_tile( const uint8_t *data, size_t len, uint8_t *pixels, uint32_t img_w, uint32_t tx, uint32_t ty, uint32_t tw, uint32_t th, - int ch, int simd) + int ch, int simd, int v2, const yapf__dbook *bk) { uint8_t rows[4][YAPF__RING][YAPF_TILE_SIZE + 1]; /* last rows, ring */ uint8_t z[4][YAPF__TILE_PX + YAPF__GROUP]; @@ -953,19 +1913,31 @@ static int yapf__decode_tile( p + rowb * y, rowb); return 0; } - if (hdr & 0xE0) return -1; /* reserved bits set */ - uint8_t xf = hdr & YAPF__SUBGREEN; + if (hdr & (v2 ? 0x80 : 0xE0)) return -1; /* reserved bits set */ + if ((hdr & YAPF__NOXFORM) && (hdr & YAPF__SUBGREEN)) return -1; + uint8_t xf = (hdr & YAPF__NOXFORM) ? 2 : (hdr & YAPF__SUBGREEN) ? 1 : 0; + uint8_t emask = 0; + int ent = hdr & YAPF__ENTROPY; hdr &= 0x0F; if ((hdr >> ch) || (xf && ch < 3)) return -1; /* unknown bits set */ + if (ent) { + if (p >= end || !bk) return -1; /* entropy needs the surface book */ + emask = *p++; + if ((emask >> ch) || (emask & hdr)) return -1; + } for (int c = 0; c < ch; c++) { if (!(hdr & (1u << c))) continue; if (p >= end) return -1; memset(rows[c], *p++, sizeof(rows[c])); } + yapf__hstream hs[4]; + int nh = 0; for (int c = 0; c < ch; c++) { if (hdr & (1u << c)) continue; - p = yapf__unpack_plane(p, end, z[c], &filt[c], tw, th, simd); + p = (emask & (1u << c)) + ? yapf__huff_head(p, end, &filt[c], th, bk, xf * 16 + c, z[c], &hs[nh++]) + : yapf__unpack_plane(p, end, z[c], &filt[c], tw, th, simd); if (!p) return -1; /* SIMD row kernels read whole 16-sample chunks; keep the unused * tail of narrow tiles defined. */ @@ -975,6 +1947,7 @@ static int yapf__decode_tile( } } if (p != end) return -1; + if (nh && yapf__huff_run(hs, nh, tw, th) != 0) return -1; /* Row by row: undo the filters of every plane (independent dependency * chains the CPU can overlap), then the colour transform while the @@ -1042,7 +2015,9 @@ static int yapf__decode_tile( if (ch == 3) { /* Planes → separate R, G, B rows (vectorises), then interleave. */ uint8_t rgb[3][YAPF_TILE_SIZE]; - if (xf) { + if (xf == 2) { + memcpy(rgb[0], P0, tw); memcpy(rgb[1], P1, tw); memcpy(rgb[2], P2, tw); + } else if (xf) { for (uint32_t x = 0; x < tw; x++) { uint8_t g = P0[x]; rgb[0][x] = (uint8_t)(P1[x] + g - 128); @@ -1063,6 +2038,10 @@ static int yapf__decode_tile( for (uint32_t x = 0; x < tw; x++, d += 3) { d[0] = rgb[0][x]; d[1] = rgb[1][x]; d[2] = rgb[2][x]; } + } else if (xf == 2) { + for (uint32_t x = 0; x < tw; x++, d += 4) { + d[0] = P0[x]; d[1] = P1[x]; d[2] = P2[x]; d[3] = P3[x]; + } } else if (xf) { YAPF__INVERSE_SG(4, d[3] = P3[x]) } else if (ch == 4) { @@ -1114,7 +2093,7 @@ static int yapf__encode_level( uint32_t tw = mw - x0 < YAPF_TILE_SIZE ? mw - x0 : YAPF_TILE_SIZE; uint32_t th = mh - y0 < YAPF_TILE_SIZE ? mh - y0 : YAPF_TILE_SIZE; size_t start = o->len; - rc = yapf__encode_tile(o, sc, pixels, mw, x0, y0, tw, th, ch); + rc = yapf__encode_tile(o, sc, pixels, mw, x0, y0, tw, th, ch, YAPF__ENC_PLAIN, NULL, NULL); if (rc == 0) yapf__w32le(o->buf + t * 4, (uint32_t)(o->len - start)); } } @@ -1122,17 +2101,56 @@ static int yapf__encode_level( return rc; } -/* A level whose tile table has been validated; tiles can then be decoded - * in any order, from any thread. */ +/* A level (surface) whose tile table has been validated; tiles can then + * be decoded in any order, from any thread. Version 1 levels are one + * 8-bit slice with 4-byte table entries; version 2 adds slices, wide + * samples, block tiles and 8-byte entries (size, yapf-sum). */ typedef struct { const uint8_t *data; size_t *offs; /* tcount + 1 byte offsets into data */ uint8_t *pixels; - uint32_t mw, mh, tcx, tcy; + uint32_t mw, mh, md, tcx, tcy; /* tcy: tile rows per slice */ int ch; + int sample; /* YAPF_SAMPLE_* (pixel payload) */ int simd; /* SIMD kernels usable on this CPU */ + int v2; + int verify; /* check tile yapf-sums */ + uint32_t blk; /* bytes per GPU block, 0 = pixel payload */ + uint32_t nbx, nby; /* blocks across / down */ + yapf__dbook *book; /* the surface's code book, or NULL */ } yapf__level; +static void yapf__level_free(yapf__level *lv) { + free(lv->offs); + lv->offs = NULL; + if (lv->book) { free(lv->book->mem); free(lv->book); lv->book = NULL; } +} + +/* Parses the tile table; lv must have data, the grid and v2 set. */ +static int yapf__level_table(yapf__level *lv, size_t len) { + uint64_t tc64 = (uint64_t)lv->tcx * lv->tcy * lv->md; + size_t ent = lv->v2 ? 8 : 4; + if ((uint64_t)(len / ent) < tc64) return -1; + size_t tcount = (size_t)tc64; + lv->offs = (size_t *)malloc((tcount + 1) * sizeof(size_t)); + if (!lv->offs) return -1; + + size_t pos = tcount * ent; + for (size_t t = 0; t < tcount; t++) { + uint32_t sz = yapf__r32le(lv->data + t * ent); + if (lv->v2) { + if ((sz >> 31) && (!lv->blk || !lv->book)) return -1; /* packed: blocks + book */ + sz &= 0x7FFFFFFFu; + } + if (sz > len - pos) return -1; + lv->offs[t] = pos; + pos += sz; + } + lv->offs[tcount] = pos; + if (lv->v2 && pos != len) return -1; + return 0; +} + static int yapf__level_init(yapf__level *lv, const uint8_t *data, size_t len, uint8_t *pixels, uint32_t mw, uint32_t mh, int ch) { @@ -1140,44 +2158,80 @@ static int yapf__level_init(yapf__level *lv, const uint8_t *data, size_t len, lv->pixels = pixels; lv->mw = mw; lv->mh = mh; + lv->md = 1; lv->tcx = yapf__tiles(mw); lv->tcy = yapf__tiles(mh); lv->ch = ch; - - size_t tcount = (size_t)lv->tcx * lv->tcy; - if (len / 4 < tcount) return -1; - lv->offs = (size_t *)malloc((tcount + 1) * sizeof(size_t)); - if (!lv->offs) return -1; - - size_t pos = tcount * 4; - for (size_t t = 0; t < tcount; t++) { - uint32_t sz = yapf__r32le(data + t * 4); - if (sz > len - pos) return -1; - lv->offs[t] = pos; - pos += sz; - } - lv->offs[tcount] = pos; - return 0; + lv->sample = YAPF_SAMPLE_U8; + return yapf__level_table(lv, len); } -/* Decode tile rows first_row, first_row + step, ... */ +/* Decodes tile (tx, row) of a level, row counting across all slices. + * dst == NULL writes into the level's pixels; otherwise into dst with + * dst_w texels (or blocks) per row, the tile at its top-left corner. + * Returns 0, -1 (corrupt) or YAPF_ERR_CHECKSUM. */ +static int yapf__level_tile(const yapf__level *lv, uint32_t tx, uint32_t row, + yapf__wdec *wsc, uint8_t *dst, size_t dst_w) +{ + size_t t = (size_t)row * lv->tcx + tx; + uint32_t z = row / lv->tcy, ty = row % lv->tcy; + const uint8_t *src = lv->data + lv->offs[t]; + size_t len = lv->offs[t + 1] - lv->offs[t]; + + if (lv->v2 && lv->verify && yapf_checksum(src, len) != yapf__r32le(lv->data + t * 8 + 4)) + return YAPF_ERR_CHECKSUM; + + if (lv->blk) { + uint32_t bx0 = tx * YAPF_BLOCK_TILE, by0 = ty * YAPF_BLOCK_TILE; + uint32_t tbw = lv->nbx - bx0 < YAPF_BLOCK_TILE ? lv->nbx - bx0 : YAPF_BLOCK_TILE; + uint32_t tbh = lv->nby - by0 < YAPF_BLOCK_TILE ? lv->nby - by0 : YAPF_BLOCK_TILE; + int packed = (int)(yapf__r32le(lv->data + t * 8) >> 31); + size_t rowb; + if (!dst) { + rowb = (size_t)lv->nbx * lv->blk; + dst = lv->pixels + (size_t)z * lv->nby * rowb + (size_t)by0 * rowb + (size_t)bx0 * lv->blk; + } else { + rowb = dst_w * lv->blk; + } + return yapf__decode_block_tile(src, len, packed, dst, rowb, tbw, tbh, lv->blk, lv->book); + } + + uint32_t x0 = tx * YAPF_TILE_SIZE, y0 = ty * YAPF_TILE_SIZE; + uint32_t tw = lv->mw - x0 < YAPF_TILE_SIZE ? lv->mw - x0 : YAPF_TILE_SIZE; + uint32_t th = lv->mh - y0 < YAPF_TILE_SIZE ? lv->mh - y0 : YAPF_TILE_SIZE; + size_t px = yapf__sbytes(lv->sample) * (size_t)lv->ch; + uint8_t *base; + uint32_t img_w; + if (!dst) { + base = lv->pixels + (size_t)z * lv->mw * lv->mh * px; + img_w = lv->mw; + } else { + base = dst; + img_w = (uint32_t)dst_w; + x0 = y0 = 0; + } + if (lv->sample == YAPF_SAMPLE_U8) + return yapf__decode_tile(src, len, base, img_w, x0, y0, tw, th, lv->ch, lv->simd, lv->v2, lv->book); + return yapf__decode_tile_wide(src, len, base, img_w, x0, y0, tw, th, lv->ch, + lv->sample, lv->simd, lv->book, wsc); +} + +/* Decode tile rows first_row, first_row + step, ... (over all slices) */ static int yapf__level_decode_rows(const yapf__level *lv, uint32_t first_row, uint32_t step) { - for (uint32_t ty = first_row; ty < lv->tcy; ty += step) { - for (uint32_t tx = 0; tx < lv->tcx; tx++) { - size_t t = (size_t)ty * lv->tcx + tx; - uint32_t x0 = tx * YAPF_TILE_SIZE, y0 = ty * YAPF_TILE_SIZE; - uint32_t tw = lv->mw - x0 < YAPF_TILE_SIZE ? lv->mw - x0 : YAPF_TILE_SIZE; - uint32_t th = lv->mh - y0 < YAPF_TILE_SIZE ? lv->mh - y0 : YAPF_TILE_SIZE; - if (yapf__decode_tile(lv->data + lv->offs[t], - lv->offs[t + 1] - lv->offs[t], - lv->pixels, lv->mw, x0, y0, tw, th, lv->ch, - lv->simd) != 0) - return -1; - } + uint32_t rows = lv->tcy * lv->md; + yapf__wdec *wsc = NULL; + int rc = 0; + if (!lv->blk && lv->sample != YAPF_SAMPLE_U8 && first_row < rows) { + wsc = (yapf__wdec *)malloc(sizeof(yapf__wdec)); + if (!wsc) return YAPF_ERR_OOM; } - return 0; + for (uint32_t ty = first_row; ty < rows && rc == 0; ty += step) + for (uint32_t tx = 0; tx < lv->tcx && rc == 0; tx++) + rc = yapf__level_tile(lv, tx, ty, wsc, NULL, 0); + free(wsc); + return rc; } /* ── Threads ─────────────────────────────────────────────────────────── */ @@ -1257,7 +2311,12 @@ static int yapf__decode_levels(const yapf__level *levels, int nlevels, yapf__thread th[YAPF__MAX_THREADS]; n = threads > 0 ? threads : yapf__cpu_count(); if (n > YAPF__MAX_THREADS) n = YAPF__MAX_THREADS; - if ((uint32_t)n > levels[0].tcy) n = (int)levels[0].tcy; + { + uint32_t rows = 0; + for (int m = 0; m < nlevels; m++) + if (levels[m].tcy * levels[m].md > rows) rows = levels[m].tcy * levels[m].md; + if ((uint32_t)n > rows) n = (int)rows; + } /* Starting a thread costs about as much as decoding a few tiles, so * give each one at least YAPF__TILES_PER_THREAD tiles of work. */ { @@ -1291,7 +2350,7 @@ static int yapf__decode_levels(const yapf__level *levels, int nlevels, #endif for (int i = 0; i < n; i++) - if (jobs[i].rc != 0) return -1; + if (jobs[i].rc != 0) return jobs[i].rc; return 0; } @@ -1299,9 +2358,12 @@ static int yapf__decode_levels(const yapf__level *levels, int nlevels, * yapf_load_memory / yapf_load * ======================================================================== */ +static yapf_image_t *yapf__load_v2_image(const uint8_t *buf, size_t size, int threads); + static yapf_image_t *yapf__load(const uint8_t *buf, size_t size, int threads) { if (!buf || size < YAPF__HDR_LEN) return NULL; if (memcmp(buf, YAPF__MAGIC, 4) != 0) return NULL; + if (buf[4] == 2) return yapf__load_v2_image(buf, size, threads); uint8_t version = buf[4]; uint8_t channels = buf[5]; @@ -1373,7 +2435,7 @@ yapf_image_t *yapf_load_memory_mt(const void *buffer, size_t size, int threads) return yapf__load((const uint8_t *)buffer, size, threads); } -static yapf_image_t *yapf__load_file(const char *filename, int threads) { +static uint8_t *yapf__read_file(const char *filename, size_t *out_len) { if (!filename) return NULL; FILE *f = fopen(filename, "rb"); if (!f) return NULL; @@ -1394,7 +2456,14 @@ static yapf_image_t *yapf__load_file(const char *filename, int threads) { fclose(f); if (!buf) return NULL; if (err) { free(buf); return NULL; } + *out_len = len; + return buf; +} +static yapf_image_t *yapf__load_file(const char *filename, int threads) { + size_t len = 0; + uint8_t *buf = yapf__read_file(filename, &len); + if (!buf) return NULL; yapf_image_t *img = yapf__load(buf, len, threads); free(buf); return img; @@ -1513,3 +2582,2098 @@ void yapf_free(yapf_image_t *img) { } free(img); } + +/* ======================================================================== + * Version 2: textures + * ======================================================================== */ + +#define YAPF__V2_HDR 64 +#define YAPF__V2_SURFACE 16 +#define YAPF__MAX_SURFACES (1u << 24) + +static const struct { uint8_t fmt, bw, bh, bytes; } yapf__block_formats[] = { + { YAPF_GPU_BC1, 4, 4, 8 }, { YAPF_GPU_BC2, 4, 4, 16 }, + { YAPF_GPU_BC3, 4, 4, 16 }, { YAPF_GPU_BC4, 4, 4, 8 }, + { YAPF_GPU_BC4_SNORM, 4, 4, 8 }, { YAPF_GPU_BC5, 4, 4, 16 }, + { YAPF_GPU_BC5_SNORM, 4, 4, 16 }, { YAPF_GPU_BC6H, 4, 4, 16 }, + { YAPF_GPU_BC6H_SF, 4, 4, 16 }, { YAPF_GPU_BC7, 4, 4, 16 }, + { YAPF_GPU_ETC2_RGB8, 4, 4, 8 }, { YAPF_GPU_ETC2_RGB8A1, 4, 4, 8 }, + { YAPF_GPU_ETC2_RGBA8, 4, 4, 16 }, { YAPF_GPU_EAC_R11, 4, 4, 8 }, + { YAPF_GPU_EAC_R11_SNORM, 4, 4, 8 }, { YAPF_GPU_EAC_RG11, 4, 4, 16 }, + { YAPF_GPU_EAC_RG11_SNORM, 4, 4, 16 }, + { YAPF_GPU_ASTC_4x4, 4, 4, 16 }, { YAPF_GPU_ASTC_5x4, 5, 4, 16 }, + { YAPF_GPU_ASTC_5x5, 5, 5, 16 }, { YAPF_GPU_ASTC_6x5, 6, 5, 16 }, + { YAPF_GPU_ASTC_6x6, 6, 6, 16 }, { YAPF_GPU_ASTC_8x5, 8, 5, 16 }, + { YAPF_GPU_ASTC_8x6, 8, 6, 16 }, { YAPF_GPU_ASTC_8x8, 8, 8, 16 }, + { YAPF_GPU_ASTC_10x5, 10, 5, 16 }, { YAPF_GPU_ASTC_10x6, 10, 6, 16 }, + { YAPF_GPU_ASTC_10x8, 10, 8, 16 }, { YAPF_GPU_ASTC_10x10, 10, 10, 16 }, + { YAPF_GPU_ASTC_12x10, 12, 10, 16 }, { YAPF_GPU_ASTC_12x12, 12, 12, 16 } +}; + +int yapf_block_info(uint8_t format, uint32_t *bw, uint32_t *bh, uint32_t *bytes) { + for (size_t i = 0; i < sizeof(yapf__block_formats) / sizeof(yapf__block_formats[0]); i++) + if (yapf__block_formats[i].fmt == format) { + if (bw) *bw = yapf__block_formats[i].bw; + if (bh) *bh = yapf__block_formats[i].bh; + if (bytes) *bytes = yapf__block_formats[i].bytes; + return 1; + } + return 0; +} + +size_t yapf_sample_bytes(uint8_t sample) { + return sample <= YAPF_SAMPLE_F32 ? yapf__sbytes(sample) : 0; +} + +/* Library-side state of a texture: the file tables of an opened file, + * and the marker that the library owns data, notes and frame times. */ +typedef struct { + int version; /* 1 or 2 (0: made by yapf_tex_create) */ + int flags; /* YAPF_LOAD_* */ + uint32_t needs; + uint64_t file_size, front; + uint64_t *off; + uint32_t *size, *tcheck; +} yapf__texi; + +static uint32_t yapf__faces(const yapf_texture_t *t) { + return t->shape == YAPF_SHAPE_CUBE ? 6u : 1u; +} + +uint32_t yapf_tex_faces(const yapf_texture_t *tex) { + return tex ? yapf__faces(tex) : 0; +} + +/* Checks a description; *count receives the number of surfaces. */ +static int yapf__tex_check(const yapf_texture_t *t, uint32_t *count) { + if (!t) return -1; + if (t->channels < 1 || t->channels > 4) return -1; + if (t->sample > YAPF_SAMPLE_F32 || t->payload > YAPF_PAYLOAD_BLOCKS) return -1; + if (t->shape > YAPF_SHAPE_VOLUME) return -1; + if (t->width == 0 || t->width > YAPF_MAX_DIM) return -1; + if (t->height == 0 || t->height > YAPF_MAX_DIM) return -1; + if (t->layers == 0 || t->frames == 0) return -1; + if (t->mip_levels < 1 || t->mip_levels > YAPF_MAX_MIPS) return -1; + if (t->shape == YAPF_SHAPE_VOLUME) { + if (t->depth == 0 || t->depth > YAPF_MAX_DIM || t->layers != 1) return -1; + } else if (t->depth > 1) { + return -1; + } + if (t->shape == YAPF_SHAPE_CUBE && t->width != t->height) return -1; + if (t->payload == YAPF_PAYLOAD_BLOCKS && !yapf_block_info(t->format, NULL, NULL, NULL)) + return -1; + uint64_t n = (uint64_t)t->frames * t->layers * yapf__faces(t) * t->mip_levels; + if (n > YAPF__MAX_SURFACES) return -1; + /* the largest surface must be addressable (32-bit builds) */ + uint64_t texels = (uint64_t)t->width * t->height * (t->shape == YAPF_SHAPE_VOLUME ? t->depth : 1); + if (texels * 4 * 4 > (uint64_t)(SIZE_MAX / 2)) return -1; + if (count) *count = (uint32_t)n; + return 0; +} + +uint32_t yapf_tex_surface_count(const yapf_texture_t *tex) { + uint32_t n = 0; + return yapf__tex_check(tex, &n) == 0 ? n : 0; +} + +uint32_t yapf_tex_surface(const yapf_texture_t *tex, uint32_t frame, + uint32_t layer, uint32_t face, uint32_t mip) +{ + return ((frame * tex->layers + layer) * yapf__faces(tex) + face) * tex->mip_levels + mip; +} + +void yapf_tex_mip_size(const yapf_texture_t *tex, uint32_t mip, + uint32_t *w, uint32_t *h, uint32_t *d) +{ + uint32_t a = tex->width >> mip, b = tex->height >> mip; + uint32_t c = tex->shape == YAPF_SHAPE_VOLUME ? tex->depth >> mip : 1; + if (w) *w = a ? a : 1; + if (h) *h = b ? b : 1; + if (d) *d = c ? c : 1; +} + +/* Blocks across and down for a mip of a block texture. */ +static void yapf__mip_blocks(const yapf_texture_t *t, uint32_t mip, + uint32_t *nbx, uint32_t *nby, uint32_t *S) +{ + uint32_t w, h, bw = 4, bh = 4, bytes = 16; + yapf_tex_mip_size(t, mip, &w, &h, NULL); + yapf_block_info(t->format, &bw, &bh, &bytes); + *nbx = (w + bw - 1) / bw; + *nby = (h + bh - 1) / bh; + *S = bytes; +} + +size_t yapf_tex_surface_bytes(const yapf_texture_t *tex, uint32_t mip) { + uint32_t w, h, d; + yapf_tex_mip_size(tex, mip, &w, &h, &d); + if (tex->payload == YAPF_PAYLOAD_BLOCKS) { + uint32_t nbx, nby, S; + yapf__mip_blocks(tex, mip, &nbx, &nby, &S); + return (size_t)nbx * nby * d * S; + } + return (size_t)w * h * d * tex->channels * yapf__sbytes(tex->sample); +} + +void yapf_tex_tile_grid(const yapf_texture_t *tex, uint32_t mip, + uint32_t *tiles_x, uint32_t *tiles_y) +{ + uint32_t tx, ty; + if (tex->payload == YAPF_PAYLOAD_BLOCKS) { + uint32_t nbx, nby, S; + yapf__mip_blocks(tex, mip, &nbx, &nby, &S); + tx = (nbx + YAPF_BLOCK_TILE - 1) / YAPF_BLOCK_TILE; + ty = (nby + YAPF_BLOCK_TILE - 1) / YAPF_BLOCK_TILE; + } else { + uint32_t w, h; + yapf_tex_mip_size(tex, mip, &w, &h, NULL); + tx = yapf__tiles(w); + ty = yapf__tiles(h); + } + if (tiles_x) *tiles_x = tx; + if (tiles_y) *tiles_y = ty; +} + +static yapf_texture_t *yapf__tex_new(void) { + yapf_texture_t *t = (yapf_texture_t *)calloc(1, sizeof(yapf_texture_t)); + if (!t) return NULL; + t->internal_ = calloc(1, sizeof(yapf__texi)); + if (!t->internal_) { free(t); return NULL; } + return t; +} + +void yapf_tex_free(yapf_texture_t *tex) { + if (!tex) return; + yapf__texi *ti = (yapf__texi *)tex->internal_; + uint32_t n = 0; + if (tex->data) { + n = (uint32_t)((uint64_t)tex->frames * tex->layers * yapf__faces(tex) * tex->mip_levels); + for (uint32_t s = 0; s < n; s++) free(tex->data[s]); + free(tex->data); + } + for (uint32_t i = 0; i < tex->note_count; i++) free(tex->notes[i].data); + free(tex->notes); + free(tex->frame_us); + if (ti) { free(ti->off); free(ti->size); free(ti->tcheck); free(ti); } + free(tex); +} + +/* Copies the description fields (everything up to loop_count). */ +static void yapf__tex_copy_desc(yapf_texture_t *d, const yapf_texture_t *s) { + d->width = s->width; d->height = s->height; + d->depth = s->shape == YAPF_SHAPE_VOLUME ? s->depth : 1; + d->layers = s->layers; d->frames = s->frames; d->shape = s->shape; + d->mip_levels = s->mip_levels; d->payload = s->payload; d->format = s->format; + d->sample = s->sample; d->channels = s->channels; d->flags = s->flags; + memcpy(d->channel_map, s->channel_map, 4); + d->linear_mask = s->linear_mask; d->normal = s->normal; + d->primaries = s->primaries; d->transfer = s->transfer; + d->mip_filter = s->mip_filter; d->alpha_ref = s->alpha_ref; + d->loop_count = s->loop_count; +} + +yapf_texture_t *yapf_tex_create(const yapf_texture_t *desc) { + yapf_texture_t probe; + uint32_t n; + if (!desc) return NULL; + probe = *desc; + if (probe.shape != YAPF_SHAPE_VOLUME && probe.depth == 0) probe.depth = 1; + if (yapf__tex_check(&probe, &n) != 0) return NULL; + yapf_texture_t *t = yapf__tex_new(); + if (!t) return NULL; + yapf__tex_copy_desc(t, &probe); + t->data = (void **)calloc(n, sizeof(void *)); + if (!t->data) { yapf_tex_free(t); return NULL; } + if (t->frames > 1) { + t->frame_us = (uint32_t *)calloc(t->frames, sizeof(uint32_t)); + if (!t->frame_us) { yapf_tex_free(t); return NULL; } + } + for (uint32_t s = 0; s < n; s++) { + t->data[s] = calloc(1, yapf_tex_surface_bytes(t, s % t->mip_levels)); + if (!t->data[s]) { yapf_tex_free(t); return NULL; } + } + return t; +} + +int yapf_tex_add_note(yapf_texture_t *tex, const char tag[4], + const void *data, uint32_t size) +{ + if (!tex || !tex->internal_ || !tag || (size && !data)) return YAPF_ERR_INVALID; + yapf_note_t *n = (yapf_note_t *)realloc(tex->notes, (tex->note_count + 1) * sizeof(yapf_note_t)); + if (!n) return YAPF_ERR_OOM; + tex->notes = n; + n += tex->note_count; + memcpy(n->tag, tag, 4); + n->size = size; + n->data = (uint8_t *)malloc(size ? size : 1); + if (!n->data) return YAPF_ERR_OOM; + if (size) memcpy(n->data, data, size); + tex->note_count++; + return YAPF_OK; +} + +const yapf_note_t *yapf_tex_find_note(const yapf_texture_t *tex, const char tag[4]) { + if (!tex || !tag) return NULL; + for (uint32_t i = 0; i < tex->note_count; i++) + if (memcmp(tex->notes[i].tag, tag, 4) == 0) return &tex->notes[i]; + return NULL; +} + +/* ── Writing ─────────────────────────────────────────────────────────── */ + +typedef struct { + yapf__scratch *sc; + yapf__wscratch *wsc; + yapf__ebook *eb; + yapf__out tmp; +} yapf__enc; + +/* u32 book size, the book, the tile table and the tiles of one surface. + * *checked receives the length the tile-table check covers. */ +static int yapf__encode_tiles(yapf__out *o, yapf__enc *e, const yapf_texture_t *t, + uint32_t mip, const uint8_t *src, int mode, + const yapf__out *book, size_t *checked) +{ + uint32_t w, h, d, tcx, tcy, nbx = 0, nby = 0, S = 0; + int blocks = t->payload == YAPF_PAYLOAD_BLOCKS; + int ch = t->channels, sb = (int)yapf__sbytes(t->sample); + yapf_tex_mip_size(t, mip, &w, &h, &d); + yapf_tex_tile_grid(t, mip, &tcx, &tcy); + if (blocks) yapf__mip_blocks(t, mip, &nbx, &nby, &S); + size_t n = (size_t)tcx * tcy * d, bl = book ? book->len : 0; + size_t slice = blocks ? (size_t)nbx * nby * S : (size_t)w * h * ch * sb; + + o->len = 0; + if (yapf__reserve(o, 4 + bl + n * 8) != 0) return -1; + yapf__w32le(o->buf, (uint32_t)bl); + if (bl) memcpy(o->buf + 4, book->buf, bl); + size_t table = 4 + bl; + memset(o->buf + table, 0, n * 8); + o->len = table + n * 8; + *checked = o->len; + + size_t t_i = 0; + for (uint32_t z = 0; z < d; z++) + for (uint32_t ty = 0; ty < tcy; ty++) + for (uint32_t tx = 0; tx < tcx; tx++, t_i++) { + size_t start = o->len; + int packed = 0, rc; + const uint8_t *sl = src + (size_t)z * slice; + if (blocks) { + uint32_t bx0 = tx * YAPF_BLOCK_TILE, by0 = ty * YAPF_BLOCK_TILE; + uint32_t tbw = nbx - bx0 < YAPF_BLOCK_TILE ? nbx - bx0 : YAPF_BLOCK_TILE; + uint32_t tbh = nby - by0 < YAPF_BLOCK_TILE ? nby - by0 : YAPF_BLOCK_TILE; + rc = yapf__encode_block_tile(o, sl + ((size_t)by0 * nbx + bx0) * S, + (size_t)nbx * S, tbw, tbh, S, mode, e->eb, &packed); + } else { + uint32_t x0 = tx * YAPF_TILE_SIZE, y0 = ty * YAPF_TILE_SIZE; + uint32_t tw = w - x0 < YAPF_TILE_SIZE ? w - x0 : YAPF_TILE_SIZE; + uint32_t th = h - y0 < YAPF_TILE_SIZE ? h - y0 : YAPF_TILE_SIZE; + rc = sb == 1 + ? yapf__encode_tile(o, e->sc, sl, w, x0, y0, tw, th, ch, mode, e->eb, &e->tmp) + : yapf__encode_tile_wide(o, &e->tmp, e->wsc, sl, w, x0, y0, tw, th, ch, + t->sample, mode, e->eb); + } + if (rc != 0) return -1; + size_t sz = o->len - start; + yapf__w32le(o->buf + table + t_i * 8, (uint32_t)sz | ((uint32_t)packed << 31)); + yapf__w32le(o->buf + table + t_i * 8 + 4, yapf_checksum(o->buf + start, sz)); + } + return 0; +} + +/* One surface; with entropy: a counting pass, the book, a coding pass, + * and the book version is kept only if it comes out smaller. */ +static int yapf__encode_surface(yapf__out *o, yapf__enc *e, const yapf_texture_t *t, + uint32_t mip, const uint8_t *src, int entropy, + size_t *checked, int *used_entropy) +{ + if (!entropy || !e->eb) + return yapf__encode_tiles(o, e, t, mip, src, YAPF__ENC_PLAIN, NULL, checked); + memset(e->eb->freq, 0, sizeof(e->eb->freq)); + if (yapf__encode_tiles(o, e, t, mip, src, YAPF__ENC_STATS, NULL, checked) != 0) return -1; + yapf__out book = { NULL, 0, 0 }, alt = { NULL, 0, 0 }; + size_t alt_checked = 0; + int rc = yapf__book_write(e->eb, &book); + if (rc == 0) rc = yapf__encode_tiles(&alt, e, t, mip, src, YAPF__ENC_BOOK, &book, &alt_checked); + if (rc == 0 && alt.len < o->len) { + yapf__out tmp = *o; + *o = alt; + alt = tmp; + *checked = alt_checked; + *used_entropy = 1; + } + free(book.buf); + free(alt.buf); + return rc < 0 ? -1 : 0; +} + +int yapf_tex_encode(const yapf_texture_t *tex, int flags, void **out_data, size_t *out_size) { + if (!out_data || !out_size) return YAPF_ERR_INVALID; + *out_data = NULL; + *out_size = 0; + uint32_t S; + yapf_texture_t t; + if (!tex) return YAPF_ERR_INVALID; + t = *tex; + if (t.shape != YAPF_SHAPE_VOLUME && t.depth == 0) t.depth = 1; + if (yapf__tex_check(&t, &S) != 0 || !t.data) return YAPF_ERR_INVALID; + for (uint32_t s = 0; s < S; s++) if (!t.data[s]) return YAPF_ERR_INVALID; + if (t.frames > 1 && !t.frame_us) return YAPF_ERR_INVALID; + uint64_t notes = 0; + for (uint32_t i = 0; i < t.note_count; i++) { + if (t.notes[i].size && !t.notes[i].data) return YAPF_ERR_INVALID; + notes += 8 + (uint64_t)t.notes[i].size; + } + if (notes > 0xFFFFFFFFull) return YAPF_ERR_INVALID; + + int err = YAPF_OK, used_ent = 0; + int entropy = (flags & YAPF_ENC_ENTROPY) != 0; + yapf__out *bufs = (yapf__out *)calloc(S, sizeof(yapf__out)); + uint32_t *tchk = (uint32_t *)calloc(S, sizeof(uint32_t)); + yapf__enc e; + memset(&e, 0, sizeof(e)); + if (!bufs || !tchk) err = YAPF_ERR_OOM; + if (err == YAPF_OK && t.payload == YAPF_PAYLOAD_PIXELS) { + if (t.sample == YAPF_SAMPLE_U8) e.sc = (yapf__scratch *)malloc(sizeof(yapf__scratch)); + else e.wsc = (yapf__wscratch *)malloc(sizeof(yapf__wscratch)); + if (!e.sc && !e.wsc) err = YAPF_ERR_OOM; + } + if (err == YAPF_OK && entropy && !(e.eb = (yapf__ebook *)malloc(sizeof(yapf__ebook)))) + err = YAPF_ERR_OOM; + for (uint32_t s = 0; s < S && err == YAPF_OK; s++) { + size_t n = 0; + if (yapf__encode_surface(&bufs[s], &e, &t, s % t.mip_levels, + (const uint8_t *)t.data[s], entropy, &n, &used_ent) != 0) + err = YAPF_ERR_OOM; + else + tchk[s] = yapf_checksum(bufs[s].buf, n); + } + + uint64_t front = YAPF__V2_HDR + (uint64_t)S * YAPF__V2_SURFACE + + (t.frames > 1 ? 4ull * t.frames : 0) + notes; + uint64_t total = front; + for (uint32_t s = 0; s < S && err == YAPF_OK; s++) total += bufs[s].len; + if (err == YAPF_OK && total > (uint64_t)SIZE_MAX) err = YAPF_ERR_INVALID; + + uint8_t *file = NULL; + if (err == YAPF_OK && !(file = (uint8_t *)malloc((size_t)total))) err = YAPF_ERR_OOM; + if (err == YAPF_OK) { + uint32_t needs = 0; + if (t.payload == YAPF_PAYLOAD_PIXELS && t.sample != YAPF_SAMPLE_U8) needs |= YAPF_NEED_WIDE; + if (t.payload == YAPF_PAYLOAD_BLOCKS) needs |= YAPF_NEED_BLOCKS; + if (used_ent) needs |= YAPF_NEED_ENTROPY; + if (t.shape != YAPF_SHAPE_2D || t.layers > 1) needs |= YAPF_NEED_LAYOUT; + if (t.frames > 1) needs |= YAPF_NEED_FRAMES; + + uint8_t *h = file; + memset(h, 0, YAPF__V2_HDR); + memcpy(h, YAPF__MAGIC, 4); + h[4] = 2; + h[5] = t.channels; h[6] = t.format; h[7] = t.mip_levels; + h[8] = t.flags; h[9] = t.payload; h[10] = t.sample; h[11] = t.shape; + yapf__w32le(h + 0x0C, t.width); + yapf__w32le(h + 0x10, t.height); + yapf__w32le(h + 0x14, t.depth); + yapf__w16le(h + 0x18, t.layers); + yapf__w16le(h + 0x1A, t.frames); + yapf__w32le(h + 0x1C, needs); + memcpy(h + 0x20, t.channel_map, 4); + h[0x24] = t.linear_mask; h[0x25] = t.normal; + h[0x26] = t.primaries; h[0x27] = t.transfer; + h[0x28] = t.mip_filter; h[0x29] = t.alpha_ref; + yapf__w16le(h + 0x2A, t.loop_count); + yapf__w32le(h + 0x2C, (uint32_t)notes); + yapf__w64le(h + 0x30, total); + + /* surface data, smallest mips first */ + uint64_t pos = front; + for (uint32_t m = t.mip_levels; m-- > 0; ) + for (uint32_t s = m; s < S; s += t.mip_levels) { + uint8_t *ent = file + YAPF__V2_HDR + (size_t)s * YAPF__V2_SURFACE; + yapf__w64le(ent, pos); + yapf__w32le(ent + 8, (uint32_t)bufs[s].len); + yapf__w32le(ent + 12, tchk[s]); + memcpy(file + pos, bufs[s].buf, bufs[s].len); + pos += bufs[s].len; + } + uint8_t *q = file + YAPF__V2_HDR + (size_t)S * YAPF__V2_SURFACE; + if (t.frames > 1) + for (uint32_t f = 0; f < t.frames; f++, q += 4) yapf__w32le(q, t.frame_us[f]); + for (uint32_t i = 0; i < t.note_count; i++) { + memcpy(q, t.notes[i].tag, 4); + yapf__w32le(q + 4, t.notes[i].size); + if (t.notes[i].size) memcpy(q + 8, t.notes[i].data, t.notes[i].size); + q += 8 + t.notes[i].size; + } + yapf__w32le(h + 0x38, yapf_checksum(file + YAPF__V2_HDR, (size_t)(front - YAPF__V2_HDR))); + yapf__w32le(h + 0x3C, yapf_checksum(file, 0x3C)); + *out_data = file; + *out_size = (size_t)total; + } + + if (bufs) for (uint32_t s = 0; s < S; s++) free(bufs[s].buf); + free(bufs); + free(tchk); + free(e.sc); + free(e.wsc); + free(e.eb); + free(e.tmp.buf); + return err; +} + +int yapf_tex_save(const char *filename, const yapf_texture_t *tex, int flags) { + if (!filename) return YAPF_ERR_INVALID; + void *data = NULL; + size_t size = 0; + int err = yapf_tex_encode(tex, flags, &data, &size); + if (err != YAPF_OK) return err; + FILE *f = fopen(filename, "wb"); + if (!f) err = YAPF_ERR_IO; + else { + if (fwrite(data, 1, size, f) != size) err = YAPF_ERR_IO; + if (fclose(f) != 0 && err == YAPF_OK) err = YAPF_ERR_IO; + } + yapf_free_buffer(data); + return err; +} + +/* ── Reading ─────────────────────────────────────────────────────────── */ + +static yapf_texture_t *yapf__open_fail(yapf_texture_t *t, int *err, int code) { + if (err) *err = code; + yapf_tex_free(t); + return NULL; +} + +/* Tiles in surface s (all slices). */ +static uint64_t yapf__surface_tiles(const yapf_texture_t *t, uint32_t mip) { + uint32_t tcx, tcy, d; + yapf_tex_tile_grid(t, mip, &tcx, &tcy); + yapf_tex_mip_size(t, mip, NULL, NULL, &d); + return (uint64_t)tcx * tcy * d; +} + +yapf_texture_t *yapf_tex_open(const void *buffer, size_t avail, int flags, int *err) { + const uint8_t *b = (const uint8_t *)buffer; + if (err) *err = YAPF_OK; + if (!b) return yapf__open_fail(NULL, err, YAPF_ERR_INVALID); + if (avail < 5) return yapf__open_fail(NULL, err, YAPF_ERR_TRUNCATED); + if (memcmp(b, YAPF__MAGIC, 4) != 0) return yapf__open_fail(NULL, err, YAPF_ERR_CORRUPT); + if (b[4] != 1 && b[4] != 2) return yapf__open_fail(NULL, err, YAPF_ERR_UNSUPPORTED); + + yapf_texture_t *t = yapf__tex_new(); + if (!t) return yapf__open_fail(NULL, err, YAPF_ERR_OOM); + yapf__texi *ti = (yapf__texi *)t->internal_; + ti->flags = flags; + uint32_t S; + + if (b[4] == 1) { + if (avail < YAPF__HDR_LEN) return yapf__open_fail(t, err, YAPF_ERR_TRUNCATED); + t->channels = b[5]; t->format = b[6]; t->mip_levels = b[7]; t->flags = b[8]; + t->width = yapf__r32le(b + 12); t->height = yapf__r32le(b + 16); + t->depth = 1; t->layers = 1; t->frames = 1; + ti->version = 1; + if (yapf__tex_check(t, &S) != 0) return yapf__open_fail(t, err, YAPF_ERR_CORRUPT); + ti->front = YAPF__HDR_LEN + (uint64_t)S * YAPF__MIP_ENTRY; + ti->file_size = avail; + if (avail < ti->front) return yapf__open_fail(t, err, YAPF_ERR_TRUNCATED); + } else { + if (avail < YAPF__V2_HDR) return yapf__open_fail(t, err, YAPF_ERR_TRUNCATED); + if (!(flags & YAPF_LOAD_TRUST) && yapf_checksum(b, 0x3C) != yapf__r32le(b + 0x3C)) + return yapf__open_fail(t, err, YAPF_ERR_CHECKSUM); + t->channels = b[5]; t->format = b[6]; t->mip_levels = b[7]; t->flags = b[8]; + t->payload = b[9]; t->sample = b[10]; t->shape = b[11]; + t->width = yapf__r32le(b + 0x0C); + t->height = yapf__r32le(b + 0x10); + t->depth = yapf__r32le(b + 0x14); + t->layers = yapf__r16le(b + 0x18); + t->frames = yapf__r16le(b + 0x1A); + ti->needs = yapf__r32le(b + 0x1C); + memcpy(t->channel_map, b + 0x20, 4); + t->linear_mask = b[0x24]; t->normal = b[0x25]; + t->primaries = b[0x26]; t->transfer = b[0x27]; + t->mip_filter = b[0x28]; t->alpha_ref = b[0x29]; + t->loop_count = yapf__r16le(b + 0x2A); + uint32_t notes = yapf__r32le(b + 0x2C); + ti->file_size = yapf__r64le(b + 0x30); + ti->version = 2; + + if (ti->needs & ~(uint32_t)YAPF_NEED_ALL) return yapf__open_fail(t, err, YAPF_ERR_UNSUPPORTED); + if (yapf__tex_check(t, &S) != 0) return yapf__open_fail(t, err, YAPF_ERR_CORRUPT); + /* every feature in use must be announced */ + if ((t->payload == YAPF_PAYLOAD_PIXELS && t->sample != YAPF_SAMPLE_U8 && !(ti->needs & YAPF_NEED_WIDE)) || + (t->payload == YAPF_PAYLOAD_BLOCKS && !(ti->needs & YAPF_NEED_BLOCKS)) || + ((t->shape != YAPF_SHAPE_2D || t->layers > 1) && !(ti->needs & YAPF_NEED_LAYOUT)) || + (t->frames > 1 && !(ti->needs & YAPF_NEED_FRAMES))) + return yapf__open_fail(t, err, YAPF_ERR_CORRUPT); + + ti->front = YAPF__V2_HDR + (uint64_t)S * YAPF__V2_SURFACE + + (t->frames > 1 ? 4ull * t->frames : 0) + notes; + if (ti->front > ti->file_size) return yapf__open_fail(t, err, YAPF_ERR_CORRUPT); + if (avail < ti->front) return yapf__open_fail(t, err, YAPF_ERR_TRUNCATED); + if (!(flags & YAPF_LOAD_TRUST) && + yapf_checksum(b + YAPF__V2_HDR, (size_t)(ti->front - YAPF__V2_HDR)) != yapf__r32le(b + 0x38)) + return yapf__open_fail(t, err, YAPF_ERR_CHECKSUM); + + const uint8_t *q = b + YAPF__V2_HDR + (size_t)S * YAPF__V2_SURFACE; + if (t->frames > 1) { + t->frame_us = (uint32_t *)malloc(t->frames * sizeof(uint32_t)); + if (!t->frame_us) return yapf__open_fail(t, err, YAPF_ERR_OOM); + for (uint32_t f = 0; f < t->frames; f++, q += 4) t->frame_us[f] = yapf__r32le(q); + } + const uint8_t *qe = q + notes; + while (q < qe) { + if (qe - q < 8) return yapf__open_fail(t, err, YAPF_ERR_CORRUPT); + uint32_t sz = yapf__r32le(q + 4); + if ((size_t)(qe - q - 8) < sz) return yapf__open_fail(t, err, YAPF_ERR_CORRUPT); + if (yapf_tex_add_note(t, (const char *)q, q + 8, sz) != YAPF_OK) + return yapf__open_fail(t, err, YAPF_ERR_OOM); + q += 8 + sz; + } + } + + t->data = (void **)calloc(S, sizeof(void *)); + ti->off = (uint64_t *)calloc(S, sizeof(uint64_t)); + ti->size = (uint32_t *)calloc(S, sizeof(uint32_t)); + ti->tcheck = (uint32_t *)calloc(S, sizeof(uint32_t)); + if (!t->data || !ti->off || !ti->size || !ti->tcheck) + return yapf__open_fail(t, err, YAPF_ERR_OOM); + for (uint32_t s = 0; s < S; s++) { + uint32_t mip = s % t->mip_levels; + if (ti->version == 1) { + const uint8_t *e = b + YAPF__HDR_LEN + (size_t)s * YAPF__MIP_ENTRY; + ti->off[s] = yapf__r32le(e); + ti->size[s] = yapf__r32le(e + 4); + } else { + const uint8_t *e = b + YAPF__V2_HDR + (size_t)s * YAPF__V2_SURFACE; + ti->off[s] = yapf__r64le(e); + ti->size[s] = yapf__r32le(e + 8); + ti->tcheck[s] = yapf__r32le(e + 12); + if (ti->off[s] > ti->file_size || ti->size[s] > ti->file_size - ti->off[s]) + return yapf__open_fail(t, err, YAPF_ERR_CORRUPT); + if (ti->size[s] < 4 || (ti->size[s] - 4) / 8 < yapf__surface_tiles(t, mip)) + return yapf__open_fail(t, err, YAPF_ERR_CORRUPT); + } + if (ti->off[s] < ti->front) return yapf__open_fail(t, err, YAPF_ERR_CORRUPT); + } + return t; +} + +uint64_t yapf_tex_stream_bytes(const yapf_texture_t *tex, uint32_t first_mip) { + if (!tex || !tex->internal_) return 0; + const yapf__texi *ti = (const yapf__texi *)tex->internal_; + if (!ti->off) return 0; + uint32_t S = yapf_tex_surface_count(tex); + uint64_t end = ti->front; + for (uint32_t s = 0; s < S; s++) + if (s % tex->mip_levels >= first_mip && ti->off[s] + ti->size[s] > end) + end = ti->off[s] + ti->size[s]; + return end; +} + +/* Prepares the level for surface s; alloc: allocate tex->data[s]. */ +static int yapf__surface_level(yapf_texture_t *t, const uint8_t *buf, size_t avail, + uint32_t s, yapf__level *lv, int alloc) +{ + const yapf__texi *ti = (const yapf__texi *)t->internal_; + uint32_t mip = s % t->mip_levels, w, h, d; + memset(lv, 0, sizeof(*lv)); + if (ti->off[s] + ti->size[s] > avail) return YAPF_ERR_TRUNCATED; + uint64_t tiles = yapf__surface_tiles(t, mip); + const uint8_t *sd = buf + ti->off[s]; + size_t ssz = ti->size[s], pre = 0; + if (ti->version == 2) { + /* u32 book size, book, tile table: covered by the tile-table check */ + uint32_t bl = yapf__r32le(sd); + if (ssz < 4 || bl > ssz - 4 || (ssz - 4 - bl) / 8 < tiles) return YAPF_ERR_CORRUPT; + pre = 4 + (size_t)bl; + if (!(ti->flags & YAPF_LOAD_TRUST) && yapf_checksum(sd, pre + (size_t)tiles * 8) != ti->tcheck[s]) + return YAPF_ERR_CHECKSUM; + if (bl) { + if (!(ti->needs & YAPF_NEED_ENTROPY)) return YAPF_ERR_CORRUPT; + lv->book = (yapf__dbook *)malloc(sizeof(yapf__dbook)); + if (!lv->book) return YAPF_ERR_OOM; + if (yapf__book_read(sd + 4, bl, lv->book) != 0) return YAPF_ERR_CORRUPT; + } + } + if (alloc && !t->data[s]) { + t->data[s] = malloc(yapf_tex_surface_bytes(t, mip)); + if (!t->data[s]) return YAPF_ERR_OOM; + } + yapf_tex_mip_size(t, mip, &w, &h, &d); + lv->data = sd + pre; + lv->pixels = (uint8_t *)t->data[s]; + lv->mw = w; lv->mh = h; lv->md = d; + lv->ch = t->channels; + lv->sample = t->sample; + lv->v2 = ti->version == 2; + lv->verify = !(ti->flags & YAPF_LOAD_TRUST); +#if YAPF__SIMD + lv->simd = yapf__simd_available(); +#endif + yapf_tex_tile_grid(t, mip, &lv->tcx, &lv->tcy); + if (t->payload == YAPF_PAYLOAD_BLOCKS) { + uint32_t S; + yapf__mip_blocks(t, mip, &lv->nbx, &lv->nby, &S); + lv->blk = S; + } + if (yapf__level_table(lv, ssz - pre) != 0) return YAPF_ERR_CORRUPT; + return YAPF_OK; +} + +static int yapf__map_rc(int rc) { + return rc == 0 ? YAPF_OK : rc == YAPF_ERR_CHECKSUM || rc == YAPF_ERR_OOM ? rc : YAPF_ERR_CORRUPT; +} + +int yapf_tex_decode_surface(yapf_texture_t *tex, const void *buffer, size_t avail, + uint32_t surface, int threads) +{ + if (!tex || !tex->internal_ || !buffer) return YAPF_ERR_INVALID; + if (!((yapf__texi *)tex->internal_)->off) return YAPF_ERR_INVALID; + if (surface >= yapf_tex_surface_count(tex)) return YAPF_ERR_INVALID; + yapf__level lv; + int rc = yapf__surface_level(tex, (const uint8_t *)buffer, avail, surface, &lv, 1); + if (rc == YAPF_OK) rc = yapf__map_rc(yapf__decode_levels(&lv, 1, threads)); + yapf__level_free(&lv); + return rc; +} + +int yapf_tex_decode_tile(const yapf_texture_t *tex, const void *buffer, size_t avail, + uint32_t surface, uint32_t tx, uint32_t ty, uint32_t slice, + void *out, size_t stride, + uint32_t *x, uint32_t *y, uint32_t *w, uint32_t *h) +{ + if (!tex || !tex->internal_ || !buffer || !out) return YAPF_ERR_INVALID; + if (!((const yapf__texi *)tex->internal_)->off) return YAPF_ERR_INVALID; + if (surface >= yapf_tex_surface_count(tex)) return YAPF_ERR_INVALID; + uint32_t mip = surface % tex->mip_levels, mw, mh, md, tcx, tcy; + yapf_tex_mip_size(tex, mip, &mw, &mh, &md); + yapf_tex_tile_grid(tex, mip, &tcx, &tcy); + if (tx >= tcx || ty >= tcy || slice >= md) return YAPF_ERR_INVALID; + + /* unit: bytes per texel (or block); the row must hold the tile */ + size_t unit; + uint32_t span, bw = 1, bh = 1, across; + if (tex->payload == YAPF_PAYLOAD_BLOCKS) { + uint32_t bytes = 16, nbx, nby, S; + yapf_block_info(tex->format, &bw, &bh, &bytes); + yapf__mip_blocks(tex, mip, &nbx, &nby, &S); + unit = bytes; + span = YAPF_BLOCK_TILE; + across = nbx; + } else { + unit = (size_t)tex->channels * yapf__sbytes(tex->sample); + span = YAPF_TILE_SIZE; + across = mw; + } + { + uint32_t need = across - tx * span < span ? across - tx * span : span; + if (stride % unit != 0 || stride / unit < need) return YAPF_ERR_INVALID; + } + yapf__level lv; + int rc = yapf__surface_level((yapf_texture_t *)tex, (const uint8_t *)buffer, avail, + surface, &lv, 0); + if (rc == YAPF_OK) { + yapf__wdec *wsc = NULL; + if (!lv.blk && lv.sample != YAPF_SAMPLE_U8) { + wsc = (yapf__wdec *)malloc(sizeof(yapf__wdec)); + if (!wsc) rc = YAPF_ERR_OOM; + } + if (rc == YAPF_OK) + rc = yapf__map_rc(yapf__level_tile(&lv, tx, slice * lv.tcy + ty, wsc, + (uint8_t *)out, stride / unit)); + free(wsc); + } + yapf__level_free(&lv); + if (rc == YAPF_OK) { + uint32_t px = tx * span * bw, py = ty * span * bh; + uint32_t pw = span * bw, ph = span * bh; + if (x) *x = px; + if (y) *y = py; + if (w) *w = mw - px < pw ? mw - px : pw; + if (h) *h = mh - py < ph ? mh - py : ph; + } + return rc; +} + +yapf_texture_t *yapf_tex_load_memory(const void *buffer, size_t size, + int flags, int threads, int *err) +{ + yapf_texture_t *t = yapf_tex_open(buffer, size, flags, err); + if (!t) return NULL; + yapf__texi *ti = (yapf__texi *)t->internal_; + if (ti->version == 2 && size < ti->file_size) + return yapf__open_fail(t, err, YAPF_ERR_TRUNCATED); + uint32_t S = yapf_tex_surface_count(t); + yapf__level *lv = (yapf__level *)calloc(S, sizeof(yapf__level)); + int rc = lv ? YAPF_OK : YAPF_ERR_OOM; + for (uint32_t s = 0; s < S && rc == YAPF_OK; s++) + rc = yapf__surface_level(t, (const uint8_t *)buffer, size, s, &lv[s], 1); + if (rc == YAPF_ERR_TRUNCATED) rc = YAPF_ERR_CORRUPT; + if (rc == YAPF_OK) rc = yapf__map_rc(yapf__decode_levels(lv, (int)S, threads)); + if (lv) for (uint32_t s = 0; s < S; s++) yapf__level_free(&lv[s]); + free(lv); + if (rc != YAPF_OK) return yapf__open_fail(t, err, rc); + return t; +} + +yapf_texture_t *yapf_tex_load(const char *filename, int flags, int threads, int *err) { + size_t len = 0; + uint8_t *buf = yapf__read_file(filename, &len); + if (!buf) { if (err) *err = YAPF_ERR_IO; return NULL; } + yapf_texture_t *t = yapf_tex_load_memory(buf, len, flags, threads, err); + free(buf); + return t; +} + +/* v1 API on a version 2 file: frame 0, layer 0, face 0 of an 8-bit + * pixel texture (not volumes), with its mips. */ +static yapf_image_t *yapf__load_v2_image(const uint8_t *buf, size_t size, int threads) { + yapf_texture_t *t = yapf_tex_open(buf, size, 0, NULL); + if (!t) return NULL; + yapf__texi *ti = (yapf__texi *)t->internal_; + if (t->payload != YAPF_PAYLOAD_PIXELS || t->sample != YAPF_SAMPLE_U8 || + t->shape == YAPF_SHAPE_VOLUME || size < ti->file_size) { + yapf_tex_free(t); + return NULL; + } + yapf__level lv[YAPF_MAX_MIPS]; + int rc = YAPF_OK; + memset(lv, 0, sizeof(lv)); + for (uint32_t m = 0; m < t->mip_levels && rc == YAPF_OK; m++) + rc = yapf__surface_level(t, buf, size, m, &lv[m], 1); + if (rc == YAPF_OK) rc = yapf__decode_levels(lv, t->mip_levels, threads); + for (uint32_t m = 0; m < YAPF_MAX_MIPS; m++) yapf__level_free(&lv[m]); + + yapf_image_t *img = NULL; + if (rc == 0) img = (yapf_image_t *)calloc(1, sizeof(yapf_image_t)); + if (img) img->mips = (uint8_t **)calloc(t->mip_levels, sizeof(uint8_t *)); + if (img && img->mips) { + img->width = t->width; img->height = t->height; img->channels = t->channels; + img->gpu_format = t->format; img->flags = t->flags; img->mip_levels = t->mip_levels; + for (uint32_t m = 0; m < t->mip_levels; m++) { + img->mips[m] = (uint8_t *)t->data[m]; /* surface m: frame 0, layer 0, face 0 */ + t->data[m] = NULL; + } + img->pixels = img->mips[0]; + } else if (img) { + free(img); + img = NULL; + } + yapf_tex_free(t); + return img; +} + +/* ======================================================================== + * Half floats + * ======================================================================== */ + +uint16_t yapf_float_to_half(float f) { + uint32_t x; + memcpy(&x, &f, 4); + uint32_t sign = (x >> 16) & 0x8000u, e = (x >> 23) & 0xFFu, m = x & 0x7FFFFFu; + if (e == 255) return (uint16_t)(sign | 0x7C00u | (m ? 0x200u | (m >> 13) : 0)); + int32_t ee = (int32_t)e - 127 + 15; + if (ee >= 31) return (uint16_t)(sign | 0x7C00u); + if (ee <= 0) { /* subnormal or zero */ + if (ee < -10) return (uint16_t)sign; + m |= 0x800000u; + int shift = 14 - ee; + uint32_t hm = m >> shift, rem = m & ((1u << shift) - 1), halfway = 1u << (shift - 1); + if (rem > halfway || (rem == halfway && (hm & 1))) hm++; + return (uint16_t)(sign | hm); + } + uint32_t h = sign | ((uint32_t)ee << 10) | (m >> 13), rem = m & 0x1FFFu; + if (rem > 0x1000u || (rem == 0x1000u && (h & 1))) h++; /* carries into the exponent */ + return (uint16_t)h; +} + +float yapf_half_to_float(uint16_t h) { + uint32_t s = (uint32_t)(h & 0x8000u) << 16, e = (h >> 10) & 31u, m = h & 0x3FFu, x; + if (e == 0) { + if (!m) x = s; + else { + e = 113; + while (!(m & 0x400u)) { m <<= 1; e--; } + x = s | (e << 23) | ((m & 0x3FFu) << 13); + } + } else if (e == 31) { + x = s | 0x7F800000u | (m << 13); + } else { + x = s | ((e + 112) << 23) | (m << 13); + } + float f; + memcpy(&f, &x, 4); + return f; +} + +/* ======================================================================== + * GPU block codecs (BC1–BC7) + * ======================================================================== */ + +/* BC6H / BC7 partitions. 2 subsets: bit i set = pixel i in subset 1. + * 3 subsets: 2 bits per pixel. Anchors: the pixel of each extra subset + * whose index drops its top bit. */ +static const uint16_t yapf__part2[64] = { + 0xCCCC, 0x8888, 0xEEEE, 0xECC8, 0xC880, 0xFEEC, 0xFEC8, 0xEC80, + 0xC800, 0xFFEC, 0xFE80, 0xE800, 0xFFE8, 0xFF00, 0xFFF0, 0xF000, + 0xF710, 0x008E, 0x7100, 0x08CE, 0x008C, 0x7310, 0x3100, 0x8CCE, + 0x088C, 0x3110, 0x6666, 0x366C, 0x17E8, 0x0FF0, 0x718E, 0x399C, + 0xAAAA, 0xF0F0, 0x5A5A, 0x33CC, 0x3C3C, 0x55AA, 0x9696, 0xA55A, + 0x73CE, 0x13C8, 0x324C, 0x3BDC, 0x6996, 0xC33C, 0x9966, 0x0660, + 0x0272, 0x04E4, 0x4E40, 0x2720, 0xC936, 0x936C, 0x39C6, 0x639C, + 0x9336, 0x9CC6, 0x817E, 0xE718, 0xCCF0, 0x0FCC, 0x7744, 0xEE22 +}; + +static const uint32_t yapf__part3[64] = { + 0xAA685050, 0x6A5A5040, 0x5A5A4200, 0x5450A0A8, 0xA5A50000, 0xA0A05050, 0x5555A0A0, 0x5A5A5050, + 0xAA550000, 0xAA555500, 0xAAAA5500, 0x90909090, 0x94949494, 0xA4A4A4A4, 0xA9A59450, 0x2A0A4250, + 0xA5945040, 0x0A425054, 0xA5A5A500, 0x55A0A0A0, 0xA8A85454, 0x6A6A4040, 0xA4A45000, 0x1A1A0500, + 0x0050A4A4, 0xAAA59090, 0x14696914, 0x69691400, 0xA08585A0, 0xAA821414, 0x50A4A450, 0x6A5A0200, + 0xA9A58000, 0x5090A0A8, 0xA8A09050, 0x24242424, 0x00AA5500, 0x24924924, 0x24499224, 0x50A50A50, + 0x500AA550, 0xAAAA4444, 0x66660000, 0xA5A0A5A0, 0x50A050A0, 0x69286928, 0x44AAAA44, 0x66666600, + 0xAA444444, 0x54A854A8, 0x95809580, 0x96969600, 0xA85454A8, 0x80959580, 0xAA141414, 0x96960000, + 0xAAAA1414, 0xA05050A0, 0xA0A5A5A0, 0x96000000, 0x40804080, 0xA9A8A9A8, 0xAAAAAA44, 0x2A4A5254 +}; + +static const uint8_t yapf__anchor2[64] = { + 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, + 15, 2, 8, 2, 2, 8, 8, 15, 2, 8, 2, 2, 8, 8, 2, 2, + 15, 15, 6, 8, 2, 8, 15, 15, 2, 8, 2, 2, 2, 15, 15, 6, + 6, 2, 6, 8, 15, 15, 2, 2, 15, 15, 15, 15, 15, 2, 2, 15 +}; +static const uint8_t yapf__anchor3a[64] = { + 3, 3, 15, 15, 8, 3, 15, 15, 8, 8, 6, 6, 6, 5, 3, 3, + 3, 3, 8, 15, 3, 3, 6, 10, 5, 8, 8, 6, 8, 5, 15, 15, + 8, 15, 3, 5, 6, 10, 8, 15, 15, 3, 15, 5, 15, 15, 15, 15, + 3, 15, 5, 5, 5, 8, 5, 10, 5, 10, 8, 13, 15, 12, 3, 3 +}; +static const uint8_t yapf__anchor3b[64] = { + 15, 8, 8, 3, 15, 15, 3, 8, 15, 15, 15, 15, 15, 15, 15, 8, + 15, 8, 15, 3, 15, 8, 15, 8, 3, 15, 6, 10, 15, 15, 10, 8, + 15, 3, 15, 10, 10, 8, 9, 10, 6, 15, 8, 15, 3, 6, 6, 8, + 15, 3, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 3, 15, 15, 8 +}; + +static const uint8_t yapf__w2[4] = { 0, 21, 43, 64 }; +static const uint8_t yapf__w3[8] = { 0, 9, 18, 27, 37, 46, 55, 64 }; +static const uint8_t yapf__w4[16] = { 0, 4, 9, 13, 17, 21, 26, 30, 34, 38, 43, 47, 51, 55, 60, 64 }; + +static const uint8_t *yapf__weights(int bits) { + return bits == 2 ? yapf__w2 : bits == 3 ? yapf__w3 : yapf__w4; +} + +static int yapf__bc_interp(int a, int b, int w) { + return (a * (64 - w) + b * w + 32) >> 6; +} + +/* 128-bit little-endian bit reader / writer */ +static uint32_t yapf__getbits(const uint8_t *b, unsigned *pos, unsigned n) { + uint32_t v = 0; + for (unsigned i = 0; i < n; i++, (*pos)++) + v |= (uint32_t)((b[*pos >> 3] >> (*pos & 7)) & 1u) << i; + return v; +} + +static void yapf__putbits(uint8_t *b, unsigned *pos, uint32_t v, unsigned n) { + for (unsigned i = 0; i < n; i++, (*pos)++) + if ((v >> i) & 1u) b[*pos >> 3] |= (uint8_t)(1u << (*pos & 7)); +} + +/* ── BC1–BC5 ─────────────────────────────────────────────────────────── */ + +static void yapf__565(uint16_t c, int *rgb) { + int r = (c >> 11) & 31, g = (c >> 5) & 63, b = c & 31; + rgb[0] = (r << 3) | (r >> 2); + rgb[1] = (g << 2) | (g >> 4); + rgb[2] = (b << 3) | (b >> 2); +} + +/* BC1 colour block; four_only: BC2 / BC3 always use four colours. */ +static void yapf__bc1_block(const uint8_t *s, uint8_t out[16][4], int four_only) { + uint16_t c0 = yapf__r16le(s), c1 = yapf__r16le(s + 2); + uint32_t idx = yapf__r32le(s + 4); + int pal[4][4]; + yapf__565(c0, pal[0]); + yapf__565(c1, pal[1]); + pal[0][3] = pal[1][3] = pal[2][3] = pal[3][3] = 255; + for (int k = 0; k < 3; k++) { + if (c0 > c1 || four_only) { + pal[2][k] = (2 * pal[0][k] + pal[1][k] + 1) / 3; + pal[3][k] = (pal[0][k] + 2 * pal[1][k] + 1) / 3; + } else { + pal[2][k] = (pal[0][k] + pal[1][k] + 1) / 2; + pal[3][k] = 0; + } + } + if (!(c0 > c1 || four_only)) pal[3][3] = 0; + for (int i = 0; i < 16; i++) + for (int k = 0; k < 4; k++) out[i][k] = (uint8_t)pal[(idx >> (2 * i)) & 3][k]; +} + +/* BC4 channel block into out[i][ch]; snorm values are shifted by +128. */ +static void yapf__bc4_block(const uint8_t *s, uint8_t out[16][4], int ch, int snorm) { + int pal[8]; + uint64_t idx = 0; + for (int k = 0; k < 6; k++) idx |= (uint64_t)s[2 + k] << (8 * k); + if (!snorm) { + int r0 = s[0], r1 = s[1]; + pal[0] = r0; pal[1] = r1; + if (r0 > r1) for (int i = 2; i < 8; i++) pal[i] = ((8 - i) * r0 + (i - 1) * r1 + 3) / 7; + else { + for (int i = 2; i < 6; i++) pal[i] = ((6 - i) * r0 + (i - 1) * r1 + 2) / 5; + pal[6] = 0; pal[7] = 255; + } + } else { + int r0 = (int8_t)s[0], r1 = (int8_t)s[1]; + if (r0 < -127) r0 = -127; + if (r1 < -127) r1 = -127; + pal[0] = r0; pal[1] = r1; + if (r0 > r1) { + for (int i = 2; i < 8; i++) { + int v = (8 - i) * r0 + (i - 1) * r1; + pal[i] = v >= 0 ? (v + 3) / 7 : -((-v + 3) / 7); + } + } else { + for (int i = 2; i < 6; i++) { + int v = (6 - i) * r0 + (i - 1) * r1; + pal[i] = v >= 0 ? (v + 2) / 5 : -((-v + 2) / 5); + } + pal[6] = -127; pal[7] = 127; + } + for (int i = 0; i < 8; i++) pal[i] += 128; + } + for (int i = 0; i < 16; i++) out[i][ch] = (uint8_t)pal[(idx >> (3 * i)) & 7]; +} + +/* ── BC7 ─────────────────────────────────────────────────────────────── */ + +static const struct { + uint8_t ns, pb, rb, isb, cb, ab, epb, spb, ib, ib2; +} yapf__bc7_modes[8] = { + { 3, 4, 0, 0, 4, 0, 1, 0, 3, 0 }, { 2, 6, 0, 0, 6, 0, 0, 1, 3, 0 }, + { 3, 6, 0, 0, 5, 0, 0, 0, 2, 0 }, { 2, 6, 0, 0, 7, 0, 1, 0, 2, 0 }, + { 1, 0, 2, 1, 5, 6, 0, 0, 2, 3 }, { 1, 0, 2, 0, 7, 8, 0, 0, 2, 2 }, + { 1, 0, 0, 0, 7, 7, 1, 0, 4, 0 }, { 2, 6, 0, 0, 5, 5, 1, 0, 2, 0 } +}; + +static int yapf__subset(int ns, int part, int i) { + if (ns == 2) return (yapf__part2[part] >> i) & 1; + if (ns == 3) return (int)((yapf__part3[part] >> (2 * i)) & 3); + return 0; +} + +static int yapf__is_anchor(int ns, int part, int i) { + if (i == 0) return 1; + if (ns == 2) return i == yapf__anchor2[part]; + if (ns == 3) return i == yapf__anchor3a[part] || i == yapf__anchor3b[part]; + return 0; +} + +static void yapf__bc7_block(const uint8_t *s, uint8_t out[16][4]) { + unsigned pos = 0; + int mode = 0; + while (mode < 8 && !yapf__getbits(s, &pos, 1)) mode++; + if (mode == 8) { memset(out, 0, 64); return; } + int ns = yapf__bc7_modes[mode].ns, cb = yapf__bc7_modes[mode].cb, ab = yapf__bc7_modes[mode].ab; + int part = (int)yapf__getbits(s, &pos, yapf__bc7_modes[mode].pb); + int rot = (int)yapf__getbits(s, &pos, yapf__bc7_modes[mode].rb); + int isb = (int)yapf__getbits(s, &pos, yapf__bc7_modes[mode].isb); + int ne = ns * 2, ep[6][4]; + for (int c = 0; c < 3; c++) + for (int e = 0; e < ne; e++) ep[e][c] = (int)yapf__getbits(s, &pos, cb); + for (int e = 0; e < ne; e++) ep[e][3] = ab ? (int)yapf__getbits(s, &pos, ab) : 255; + int pbit = yapf__bc7_modes[mode].epb | yapf__bc7_modes[mode].spb; + if (yapf__bc7_modes[mode].epb) { + for (int e = 0; e < ne; e++) { + int p = (int)yapf__getbits(s, &pos, 1); + for (int c = 0; c < 4; c++) if (c < 3 || ab) ep[e][c] = (ep[e][c] << 1) | p; + } + } else if (yapf__bc7_modes[mode].spb) { + for (int k = 0; k < ns; k++) { + int p = (int)yapf__getbits(s, &pos, 1); + for (int c = 0; c < 3; c++) { + ep[2 * k][c] = (ep[2 * k][c] << 1) | p; + ep[2 * k + 1][c] = (ep[2 * k + 1][c] << 1) | p; + } + } + } + for (int e = 0; e < ne; e++) + for (int c = 0; c < 4; c++) { + int bits = (c < 3 ? cb : ab) + pbit; + if (c == 3 && !ab) continue; + int v = ep[e][c] << (8 - bits); + ep[e][c] = v | (v >> bits); + } + int ib = yapf__bc7_modes[mode].ib, ib2 = yapf__bc7_modes[mode].ib2; + int idx[16], idx2[16]; + for (int i = 0; i < 16; i++) + idx[i] = (int)yapf__getbits(s, &pos, ib - yapf__is_anchor(ns, part, i)); + for (int i = 0; i < 16; i++) + idx2[i] = ib2 ? (int)yapf__getbits(s, &pos, ib2 - (i == 0)) : 0; + const uint8_t *w1 = yapf__weights(ib), *w2 = yapf__weights(ib2 ? ib2 : 2); + for (int i = 0; i < 16; i++) { + int k = yapf__subset(ns, part, i), px[4]; + const int *a = ep[2 * k], *b = ep[2 * k + 1]; + int cw, aw; + if (!ib2) { cw = w1[idx[i]]; aw = w1[idx[i]]; } + else if (!isb) { cw = w1[idx[i]]; aw = w2[idx2[i]]; } + else { cw = w2[idx2[i]]; aw = w1[idx[i]]; } + for (int c = 0; c < 3; c++) px[c] = yapf__bc_interp(a[c], b[c], cw); + px[3] = yapf__bc_interp(a[3], b[3], aw); + if (rot) { int t = px[3]; px[3] = px[rot - 1]; px[rot - 1] = t; } + for (int c = 0; c < 4; c++) out[i][c] = (uint8_t)px[c]; + } +} + +/* ── BC6H ────────────────────────────────────────────────────────────── */ +/* + * Header layouts as sequences of (field, first bit, bit count), in the + * order they appear after the mode bits. Fields: 0–3 red w, x, y, z; + * 4–7 green; 8–11 blue; 12 partition. The same table drives the reader + * and the encoder's writer. + */ +#define YAPF__F(f, lo, n) { f, lo, n } +typedef struct { uint8_t f, lo, n; } yapf__seg; +enum { RW, RX, RY, RZ, GW, GX, GY, GZ, BW, BX, BY, BZ, PD }; + +static const yapf__seg yapf__bc6_m0[] = { + {GY,4,1},{BY,4,1},{BZ,4,1},{RW,0,10},{GW,0,10},{BW,0,10},{RX,0,5},{GZ,4,1},{GY,0,4}, + {GX,0,5},{BZ,0,1},{GZ,0,4},{BX,0,5},{BZ,1,1},{BY,0,4},{RY,0,5},{BZ,2,1},{RZ,0,5},{BZ,3,1},{PD,0,5},{0,0,0} }; +static const yapf__seg yapf__bc6_m1[] = { + {GY,5,1},{GZ,4,1},{GZ,5,1},{RW,0,7},{BZ,0,1},{BZ,1,1},{BY,4,1},{GW,0,7},{BY,5,1},{BZ,2,1}, + {GY,4,1},{BW,0,7},{BZ,3,1},{BZ,5,1},{BZ,4,1},{RX,0,6},{GY,0,4},{GX,0,6},{GZ,0,4},{BX,0,6}, + {BY,0,4},{RY,0,6},{RZ,0,6},{PD,0,5},{0,0,0} }; +static const yapf__seg yapf__bc6_m2[] = { + {RW,0,10},{GW,0,10},{BW,0,10},{RX,0,5},{RW,10,1},{GY,0,4},{GX,0,4},{GW,10,1},{BZ,0,1},{GZ,0,4}, + {BX,0,4},{BW,10,1},{BZ,1,1},{BY,0,4},{RY,0,5},{BZ,2,1},{RZ,0,5},{BZ,3,1},{PD,0,5},{0,0,0} }; +static const yapf__seg yapf__bc6_m3[] = { + {RW,0,10},{GW,0,10},{BW,0,10},{RX,0,4},{RW,10,1},{GZ,4,1},{GY,0,4},{GX,0,5},{GW,10,1},{GZ,0,4}, + {BX,0,4},{BW,10,1},{BZ,1,1},{BY,0,4},{RY,0,4},{BZ,0,1},{BZ,2,1},{RZ,0,4},{GY,4,1},{BZ,3,1},{PD,0,5},{0,0,0} }; +static const yapf__seg yapf__bc6_m4[] = { + {RW,0,10},{GW,0,10},{BW,0,10},{RX,0,4},{RW,10,1},{BY,4,1},{GY,0,4},{GX,0,4},{GW,10,1},{BZ,0,1}, + {GZ,0,4},{BX,0,5},{BW,10,1},{BY,0,4},{RY,0,4},{BZ,1,1},{BZ,2,1},{RZ,0,4},{BZ,4,1},{BZ,3,1},{PD,0,5},{0,0,0} }; +static const yapf__seg yapf__bc6_m5[] = { + {RW,0,9},{BY,4,1},{GW,0,9},{GY,4,1},{BW,0,9},{BZ,4,1},{RX,0,5},{GZ,4,1},{GY,0,4},{GX,0,5}, + {BZ,0,1},{GZ,0,4},{BX,0,5},{BZ,1,1},{BY,0,4},{RY,0,5},{BZ,2,1},{RZ,0,5},{BZ,3,1},{PD,0,5},{0,0,0} }; +static const yapf__seg yapf__bc6_m6[] = { + {RW,0,8},{GZ,4,1},{BY,4,1},{GW,0,8},{BZ,2,1},{GY,4,1},{BW,0,8},{BZ,3,1},{BZ,4,1},{RX,0,6}, + {GY,0,4},{GX,0,5},{BZ,0,1},{GZ,0,4},{BX,0,5},{BZ,1,1},{BY,0,4},{RY,0,6},{RZ,0,6},{PD,0,5},{0,0,0} }; +static const yapf__seg yapf__bc6_m7[] = { + {RW,0,8},{BZ,0,1},{BY,4,1},{GW,0,8},{GY,5,1},{GY,4,1},{BW,0,8},{GZ,5,1},{BZ,4,1},{RX,0,5}, + {GZ,4,1},{GY,0,4},{GX,0,6},{GZ,0,4},{BX,0,5},{BZ,1,1},{BY,0,4},{RY,0,5},{BZ,2,1},{RZ,0,5},{BZ,3,1},{PD,0,5},{0,0,0} }; +static const yapf__seg yapf__bc6_m8[] = { + {RW,0,8},{BZ,1,1},{BY,4,1},{GW,0,8},{BY,5,1},{GY,4,1},{BW,0,8},{BZ,5,1},{BZ,4,1},{RX,0,5}, + {GZ,4,1},{GY,0,4},{GX,0,5},{BZ,0,1},{GZ,0,4},{BX,0,6},{BY,0,4},{RY,0,5},{BZ,2,1},{RZ,0,5},{BZ,3,1},{PD,0,5},{0,0,0} }; +static const yapf__seg yapf__bc6_m9[] = { + {RW,0,6},{GZ,4,1},{BZ,0,1},{BZ,1,1},{BY,4,1},{GW,0,6},{GY,5,1},{BY,5,1},{BZ,2,1},{GY,4,1}, + {BW,0,6},{GZ,5,1},{BZ,3,1},{BZ,5,1},{BZ,4,1},{RX,0,6},{GY,0,4},{GX,0,6},{GZ,0,4},{BX,0,6}, + {BY,0,4},{RY,0,6},{RZ,0,6},{PD,0,5},{0,0,0} }; +static const yapf__seg yapf__bc6_m10[] = { + {RW,0,10},{GW,0,10},{BW,0,10},{RX,0,10},{GX,0,10},{BX,0,10},{0,0,0} }; +static const yapf__seg yapf__bc6_m11[] = { + {RW,0,10},{GW,0,10},{BW,0,10},{RX,0,9},{RW,10,1},{GX,0,9},{GW,10,1},{BX,0,9},{BW,10,1},{0,0,0} }; +static const yapf__seg yapf__bc6_m12[] = { + {RW,0,10},{GW,0,10},{BW,0,10},{RX,0,8},{RW,11,1},{RW,10,1},{GX,0,8},{GW,11,1},{GW,10,1}, + {BX,0,8},{BW,11,1},{BW,10,1},{0,0,0} }; +static const yapf__seg yapf__bc6_m13[] = { + {RW,0,10},{GW,0,10},{BW,0,10},{RX,0,4},{RW,15,1},{RW,14,1},{RW,13,1},{RW,12,1},{RW,11,1},{RW,10,1}, + {GX,0,4},{GW,15,1},{GW,14,1},{GW,13,1},{GW,12,1},{GW,11,1},{GW,10,1}, + {BX,0,4},{BW,15,1},{BW,14,1},{BW,13,1},{BW,12,1},{BW,11,1},{BW,10,1},{0,0,0} }; + +static const struct { + uint8_t code, codebits, w, dr, dg, db; + const yapf__seg *seg; +} yapf__bc6_modes[14] = { + { 0x00, 2, 10, 5, 5, 5, yapf__bc6_m0 }, { 0x01, 2, 7, 6, 6, 6, yapf__bc6_m1 }, + { 0x02, 5, 11, 5, 4, 4, yapf__bc6_m2 }, { 0x06, 5, 11, 4, 5, 4, yapf__bc6_m3 }, + { 0x0A, 5, 11, 4, 4, 5, yapf__bc6_m4 }, { 0x0E, 5, 9, 5, 5, 5, yapf__bc6_m5 }, + { 0x12, 5, 8, 6, 5, 5, yapf__bc6_m6 }, { 0x16, 5, 8, 5, 6, 5, yapf__bc6_m7 }, + { 0x1A, 5, 8, 5, 5, 6, yapf__bc6_m8 }, { 0x1E, 5, 6, 6, 6, 6, yapf__bc6_m9 }, + { 0x03, 5, 10, 10, 10, 10, yapf__bc6_m10 }, { 0x07, 5, 11, 9, 9, 9, yapf__bc6_m11 }, + { 0x0B, 5, 12, 8, 8, 8, yapf__bc6_m12 }, { 0x0F, 5, 16, 4, 4, 4, yapf__bc6_m13 } +}; + +static int yapf__sext(int v, int bits) { + int m = 1 << (bits - 1); + v &= (1 << bits) - 1; + return (v ^ m) - m; +} + +static int yapf__bc6_unq(int v, int bits, int sgn) { + if (!sgn) { + if (bits >= 15) return v; + if (v == 0) return 0; + if (v == (1 << bits) - 1) return 0xFFFF; + return ((v << 16) + 0x8000) >> bits; + } + if (bits >= 16) return v; + int s = v < 0; + if (s) v = -v; + int u; + if (v == 0) u = 0; + else if (v >= (1 << (bits - 1)) - 1) u = 0x7FFF; + else u = ((v << 15) + 0x4000) >> (bits - 1); + return s ? -u : u; +} + +static uint16_t yapf__bc6_finish(int v, int sgn) { + if (!sgn) return (uint16_t)((v * 31) >> 6); + v = v < 0 ? -(((-v) * 31) >> 5) : (v * 31) >> 5; + return (uint16_t)(v < 0 ? 0x8000 | -v : v); +} + +static void yapf__bc6_block(const uint8_t *s, uint16_t out[16][3], int sgn) { + unsigned pos = 0; + int code = (int)yapf__getbits(s, &pos, 2), m = -1; + if (code > 1) code |= (int)yapf__getbits(s, &pos, 3) << 2; + for (int i = 0; i < 14; i++) + if (yapf__bc6_modes[i].code == code) { m = i; break; } + if (m < 0) { memset(out, 0, 96); return; } /* reserved modes */ + + int f[13] = {0}; + for (const yapf__seg *g = yapf__bc6_modes[m].seg; g->n; g++) + f[g->f] |= (int)yapf__getbits(s, &pos, g->n) << g->lo; + int W = yapf__bc6_modes[m].w, regions = m < 10 ? 2 : 1, ne = regions * 2; + int db[3] = { yapf__bc6_modes[m].dr, yapf__bc6_modes[m].dg, yapf__bc6_modes[m].db }; + int ep[4][3]; + for (int c = 0; c < 3; c++) { + for (int e = 0; e < ne; e++) ep[e][c] = f[c * 4 + e]; + if (sgn) ep[0][c] = yapf__sext(ep[0][c], W); + int delta = m != 9 && m != 10; + for (int e = 1; e < ne; e++) { + if (delta || sgn) ep[e][c] = yapf__sext(ep[e][c], db[c]); + if (delta) { + ep[e][c] = (ep[e][c] + ep[0][c]) & ((1 << W) - 1); + if (sgn) ep[e][c] = yapf__sext(ep[e][c], W); + } + } + for (int e = 0; e < ne; e++) ep[e][c] = yapf__bc6_unq(ep[e][c], W, sgn); + } + int part = f[PD], ib = regions == 2 ? 3 : 4; + const uint8_t *wt = yapf__weights(ib); + for (int i = 0; i < 16; i++) { + int k = regions == 2 ? (yapf__part2[part] >> i) & 1 : 0; + int anchor = i == 0 || (regions == 2 && i == yapf__anchor2[part]); + int idx = (int)yapf__getbits(s, &pos, ib - anchor); + for (int c = 0; c < 3; c++) + out[i][c] = yapf__bc6_finish(yapf__bc_interp(ep[2 * k][c], ep[2 * k + 1][c], wt[idx]), sgn); + } +} + +int yapf_blocks_decode(uint8_t format, const void *blocks, uint32_t w, uint32_t h, void *out) { + uint32_t bw, bh, S; + if (!blocks || !out || !w || !h) return YAPF_ERR_INVALID; + if (!yapf_block_info(format, &bw, &bh, &S)) return YAPF_ERR_INVALID; + if (format >= YAPF_GPU_ETC2_RGB8) return YAPF_ERR_UNSUPPORTED; + const uint8_t *b = (const uint8_t *)blocks; + uint32_t nbx = (w + 3) / 4, nby = (h + 3) / 4; + int hdr = format == YAPF_GPU_BC6H || format == YAPF_GPU_BC6H_SF; + for (uint32_t by = 0; by < nby; by++) + for (uint32_t bx = 0; bx < nbx; bx++, b += S) { + uint8_t px[16][4]; + uint16_t hx[16][3]; + switch (format) { + case YAPF_GPU_BC1: yapf__bc1_block(b, px, 0); break; + case YAPF_GPU_BC2: + yapf__bc1_block(b + 8, px, 1); + for (int i = 0; i < 16; i++) px[i][3] = (uint8_t)(((b[i / 2] >> (4 * (i & 1))) & 15) * 17); + break; + case YAPF_GPU_BC3: + yapf__bc1_block(b + 8, px, 1); + yapf__bc4_block(b, px, 3, 0); + break; + case YAPF_GPU_BC4: case YAPF_GPU_BC4_SNORM: + memset(px, 0, sizeof(px)); + yapf__bc4_block(b, px, 0, format == YAPF_GPU_BC4_SNORM); + for (int i = 0; i < 16; i++) px[i][3] = 255; + break; + case YAPF_GPU_BC5: case YAPF_GPU_BC5_SNORM: + memset(px, 0, sizeof(px)); + yapf__bc4_block(b, px, 0, format == YAPF_GPU_BC5_SNORM); + yapf__bc4_block(b + 8, px, 1, format == YAPF_GPU_BC5_SNORM); + for (int i = 0; i < 16; i++) px[i][3] = 255; + break; + case YAPF_GPU_BC7: yapf__bc7_block(b, px); break; + default: yapf__bc6_block(b, hx, format == YAPF_GPU_BC6H_SF); break; + } + for (uint32_t y = 0; y < 4 && by * 4 + y < h; y++) + for (uint32_t x = 0; x < 4 && bx * 4 + x < w; x++) { + size_t o = (size_t)(by * 4 + y) * w + bx * 4 + x; + if (hdr) memcpy((uint16_t *)out + o * 3, hx[y * 4 + x], 6); + else memcpy((uint8_t *)out + o * 4, px[y * 4 + x], 4); + } + } + return YAPF_OK; +} + +/* ── Encoders ────────────────────────────────────────────────────────── */ +/* + * Straightforward, decent-quality encoders so YAPF can make GPU textures + * on its own: principal axis endpoints with least-squares refinement. + * For the best BC7 / ASTC quality use a dedicated encoder and store its + * blocks (payload = BLOCKS). + */ + +/* Principal axis of n points of dim k (power iteration). */ +static void yapf__pca(const float (*p)[4], int n, int k, float *mean, float *axis) { + for (int c = 0; c < k; c++) { + mean[c] = 0; + for (int i = 0; i < n; i++) mean[c] += p[i][c]; + mean[c] /= (float)n; + } + float cov[4][4] = {{0}}; + for (int i = 0; i < n; i++) + for (int a = 0; a < k; a++) + for (int b = 0; b < k; b++) + cov[a][b] += (p[i][a] - mean[a]) * (p[i][b] - mean[b]); + float v[4] = { 1.0f, 0.7f, 0.4f, 0.2f }; + for (int it = 0; it < 12; it++) { + float nv[4] = {0}, len = 0; + for (int a = 0; a < k; a++) { + for (int b = 0; b < k; b++) nv[a] += cov[a][b] * v[b]; + len += nv[a] * nv[a]; + } + if (len < 1e-12f) break; + float mx = 0; /* scale by the largest component */ + for (int a = 0; a < k; a++) mx = nv[a] > mx ? nv[a] : -nv[a] > mx ? -nv[a] : mx; + for (int a = 0; a < k; a++) v[a] = nv[a] / mx; + } + for (int a = 0; a < k; a++) axis[a] = v[a]; +} + +/* Endpoints at the extremes of the projections onto the principal axis. */ +static void yapf__extents(const float (*p)[4], int n, int k, float *e0, float *e1) { + float mean[4], axis[4], lo = 1e30f, hi = -1e30f, len2 = 0; + yapf__pca(p, n, k, mean, axis); + for (int c = 0; c < k; c++) len2 += axis[c] * axis[c]; + if (len2 < 1e-20f) len2 = 1.0f; + for (int i = 0; i < n; i++) { + float t = 0; + for (int c = 0; c < k; c++) t += (p[i][c] - mean[c]) * axis[c]; + if (t < lo) lo = t; + if (t > hi) hi = t; + } + lo /= len2; /* the axis need not be unit length */ + hi /= len2; + for (int c = 0; c < k; c++) { e0[c] = mean[c] + axis[c] * lo; e1[c] = mean[c] + axis[c] * hi; } +} + +/* Least squares endpoints for given interpolation weights (0..1). */ +static int yapf__lsq(const float (*p)[4], const float *wt, int n, int k, float *e0, float *e1) { + float aa = 0, ab = 0, bb = 0, ax[4] = {0}, bx[4] = {0}; + for (int i = 0; i < n; i++) { + float b = wt[i], a = 1.0f - b; + aa += a * a; ab += a * b; bb += b * b; + for (int c = 0; c < k; c++) { ax[c] += a * p[i][c]; bx[c] += b * p[i][c]; } + } + float det = aa * bb - ab * ab; + if (det < 1e-8f && det > -1e-8f) return -1; + for (int c = 0; c < k; c++) { + e0[c] = (ax[c] * bb - bx[c] * ab) / det; + e1[c] = (bx[c] * aa - ax[c] * ab) / det; + } + return 0; +} + +static int yapf__clampi(float v, int lo, int hi) { + int i = (int)(v + 0.5f); + if (v < 0) i = (int)(v - 0.5f); + return i < lo ? lo : i > hi ? hi : i; +} + +static uint16_t yapf__to565(const float *c) { + return (uint16_t)((yapf__clampi(c[0] * 31.0f / 255.0f, 0, 31) << 11) | + (yapf__clampi(c[1] * 63.0f / 255.0f, 0, 63) << 5) | + yapf__clampi(c[2] * 31.0f / 255.0f, 0, 31)); +} + +/* BC1 colour block from 16 RGBA pixels. four: force 4-colour mode + * (BC2 / BC3); otherwise pixels with alpha < 128 become transparent. */ +static void yapf__bc1_encode(const uint8_t px[16][4], uint8_t *out, int four) { + float p[16][4]; + int transparent = 0, n = 0, map[16]; + for (int i = 0; i < 16; i++) { + if (!four && px[i][3] < 128) { transparent = 1; map[i] = -1; continue; } + map[i] = n; + for (int c = 0; c < 3; c++) p[n][c] = px[i][c]; + n++; + } + if (n == 0) { /* all transparent */ + yapf__w16le(out, 0); yapf__w16le(out + 2, 0); yapf__w32le(out + 4, 0xFFFFFFFFu); + return; + } + float e0[4], e1[4]; + uint32_t best_idx = 0, best_err = UINT32_MAX; + uint16_t best_c0 = 0, best_c1 = 0; + yapf__extents((const float (*)[4])p, n, 3, e0, e1); + for (int it = 0; it < 3; it++) { + uint16_t c0 = yapf__to565(e1), c1 = yapf__to565(e0); + int three = transparent; + if (three ? c0 > c1 : c0 < c1) { uint16_t t = c0; c0 = c1; c1 = t; } + if (!three && c0 == c1) { /* single colour */ + if (c1 > 0) c1--; else c0++; + if (c0 < c1) { uint16_t t = c0; c0 = c1; c1 = t; } + } + uint8_t blk[8], full[16][4]; + /* decode a block with indices 0, 1, 2, 3 to get the palette */ + yapf__w16le(blk, c0); yapf__w16le(blk + 2, c1); yapf__w32le(blk + 4, 0xE4u); + yapf__bc1_block(blk, full, four); + int nc = three ? 3 : 4; + uint32_t idx = 0, err = 0; + float wt[16]; + static const float wv4[4] = { 0.0f, 1.0f, 1.0f / 3, 2.0f / 3 }; + static const float wv3[3] = { 0.0f, 1.0f, 0.5f }; + for (int i = 0; i < 16; i++) { + if (map[i] < 0) { idx |= 3u << (2 * i); continue; } + uint32_t be = UINT32_MAX; + int bi = 0; + for (int j = 0; j < nc; j++) { + uint32_t e = 0; + for (int c = 0; c < 3; c++) { int d = (int)full[j][c] - px[i][c]; e += (uint32_t)(d * d); } + if (e < be) { be = e; bi = j; } + } + idx |= (uint32_t)bi << (2 * i); + err += be; + wt[map[i]] = three ? wv3[bi] : wv4[bi]; + } + if (err < best_err) { best_err = err; best_idx = idx; best_c0 = c0; best_c1 = c1; } + if (err == 0) break; + /* refine: endpoints in palette order (index 0 = c0) */ + float n0[4], n1[4]; + if (yapf__lsq((const float (*)[4])p, wt, n, 3, n0, n1) != 0) break; + for (int c = 0; c < 3; c++) { e1[c] = n0[c]; e0[c] = n1[c]; } + } + yapf__w16le(out, best_c0); + yapf__w16le(out + 2, best_c1); + yapf__w32le(out + 4, best_idx); +} + +/* BC4 block from 16 values. */ +static void yapf__bc4_encode(const uint8_t *v, uint8_t *out) { + int lo = 255, hi = 0; + for (int i = 0; i < 16; i++) { if (v[i] < lo) lo = v[i]; if (v[i] > hi) hi = v[i]; } + uint64_t best_bits = 0; + uint32_t best_err = UINT32_MAX; + int best0 = hi, best1 = lo; + for (int d0 = 0; d0 < 3; d0++) + for (int d1 = 0; d1 < 3; d1++) { + int r0 = hi - d0, r1 = lo + d1; + if (r0 < r1) continue; + if (r0 == r1) { if (d0 || d1) continue; } + uint8_t blk[8] = { (uint8_t)r0, (uint8_t)r1, 0, 0, 0, 0, 0, 0 }; + uint8_t pal[16][4]; + int palv[8]; + for (int j = 0; j < 8; j++) { + uint64_t ix = 0; + for (int i = 0; i < 16; i++) ix |= (uint64_t)j << (3 * i); + for (int k = 0; k < 6; k++) blk[2 + k] = (uint8_t)(ix >> (8 * k)); + yapf__bc4_block(blk, pal, 0, 0); + palv[j] = pal[0][0]; + } + uint64_t bits = 0; + uint32_t err = 0; + for (int i = 0; i < 16; i++) { + int bj = 0, be = 1 << 30; + for (int j = 0; j < 8; j++) { + int d = palv[j] - v[i]; + if (d * d < be) { be = d * d; bj = j; } + } + bits |= (uint64_t)bj << (3 * i); + err += (uint32_t)be; + } + if (err < best_err) { best_err = err; best_bits = bits; best0 = r0; best1 = r1; } + } + out[0] = (uint8_t)best0; + out[1] = (uint8_t)best1; + for (int k = 0; k < 6; k++) out[2 + k] = (uint8_t)(best_bits >> (8 * k)); +} + +/* BC7 mode 6 (one subset, RGBA 7.7.7.7 + p-bit per endpoint, 4-bit + * indices). Opaque blocks keep the p-bits at 1 so alpha stays 255. + * Returns the squared error. */ +static uint32_t yapf__bc7_mode6(const uint8_t px[16][4], uint8_t *out) { + float p[16][4], e0[4], e1[4]; + int opaque = 1; + for (int i = 0; i < 16; i++) { + for (int c = 0; c < 4; c++) p[i][c] = px[i][c]; + if (px[i][3] != 255) opaque = 0; + } + yapf__extents((const float (*)[4])p, 16, opaque ? 3 : 4, e0, e1); + if (opaque) e0[3] = e1[3] = 255; + + uint8_t best[16]; + uint32_t best_err = UINT32_MAX; + for (int it = 0; it < 3; it++) { + int q[2][4], pb[2]; + const float *e[2] = { e0, e1 }; + for (int k = 0; k < 2; k++) { + uint32_t bestp = UINT32_MAX; + for (int pbit = opaque ? 1 : 0; pbit < 2; pbit++) { + uint32_t er = 0; + int qq[4]; + for (int c = 0; c < 4; c++) { + qq[c] = yapf__clampi((e[k][c] - (float)pbit) / 2.0f, 0, 127); + int d = (2 * qq[c] + pbit) - (int)(e[k][c] + 0.5f); + er += (uint32_t)(d * d); + } + if (er < bestp) { bestp = er; pb[k] = pbit; memcpy(q[k], qq, sizeof(qq)); } + } + } + int v[2][4]; + for (int k = 0; k < 2; k++) for (int c = 0; c < 4; c++) v[k][c] = 2 * q[k][c] + pb[k]; + int idx[16]; + uint32_t err = 0; + float wt[16]; + for (int i = 0; i < 16; i++) { + uint32_t be = UINT32_MAX; + int bi = 0; + for (int j = 0; j < 16; j++) { + uint32_t er = 0; + for (int c = 0; c < 4; c++) { + int d = yapf__bc_interp(v[0][c], v[1][c], yapf__w4[j]) - px[i][c]; + er += (uint32_t)(d * d); + } + if (er < be) { be = er; bi = j; } + } + idx[i] = bi; + err += be; + wt[i] = yapf__w4[bi] / 64.0f; + } + if (err < best_err) { + best_err = err; + int swap = idx[0] >= 8; + int a = swap ? 1 : 0, b = swap ? 0 : 1; + memset(best, 0, 16); + unsigned pos = 0; + yapf__putbits(best, &pos, 1u << 6, 7); + for (int c = 0; c < 4; c++) { + yapf__putbits(best, &pos, (uint32_t)q[a][c], 7); + yapf__putbits(best, &pos, (uint32_t)q[b][c], 7); + } + yapf__putbits(best, &pos, (uint32_t)pb[a], 1); + yapf__putbits(best, &pos, (uint32_t)pb[b], 1); + for (int i = 0; i < 16; i++) { + int ix = swap ? 15 - idx[i] : idx[i]; + yapf__putbits(best, &pos, (uint32_t)ix, i == 0 ? 3 : 4); + } + } + if (err == 0) break; + float n0[4], n1[4]; + if (yapf__lsq((const float (*)[4])p, wt, 16, 4, n0, n1) != 0) break; + memcpy(e0, n0, sizeof(n0)); + memcpy(e1, n1, sizeof(n1)); + if (opaque) e0[3] = e1[3] = 255; + } + memcpy(out, best, 16); + return best_err; +} + +/* Fits indices of `bits` bits for the points of one subset against + * endpoints v0, v1 (k channels); returns the error, fills idx / wt. */ +static uint32_t yapf__bc7_fit(const uint8_t px[16][4], const int *sel, int n, + const int *v0, const int *v1, int c0, int k, int bits, + int *idx, float *wt) +{ + const uint8_t *w = yapf__weights(bits); + uint32_t err = 0; + for (int i = 0; i < n; i++) { + uint32_t be = UINT32_MAX; + int bi = 0; + for (int j = 0; j < (1 << bits); j++) { + uint32_t e = 0; + for (int c = c0; c < c0 + k; c++) { + int d = yapf__bc_interp(v0[c], v1[c], w[j]) - px[sel[i]][c]; + e += (uint32_t)(d * d); + } + if (e < be) { be = e; bi = j; } + } + idx[i] = bi; + wt[i] = w[bi] / 64.0f; + err += be; + } + return err; +} + +/* BC7 mode 5: RGB 7 bits and alpha 8 bits on separate 2-bit indices, so + * colour and alpha need not move together (cut-outs, decals). */ +static uint32_t yapf__bc7_mode5(const uint8_t px[16][4], uint8_t *out) { + int sel[16]; + float p[16][4], e0[4], e1[4]; + for (int i = 0; i < 16; i++) { sel[i] = i; for (int c = 0; c < 4; c++) p[i][c] = px[i][c]; } + yapf__extents((const float (*)[4])p, 16, 3, e0, e1); + + int bq[2][3] = {{0}}, bidx[16] = {0}; + uint32_t cerr = UINT32_MAX; + for (int it = 0; it < 3; it++) { + int q[2][3], v[2][4] = {{0}}, idx[16]; + float wt[16]; + for (int c = 0; c < 3; c++) { + q[0][c] = yapf__clampi(e0[c] * 127.0f / 255.0f, 0, 127); + q[1][c] = yapf__clampi(e1[c] * 127.0f / 255.0f, 0, 127); + v[0][c] = (q[0][c] << 1) | (q[0][c] >> 6); + v[1][c] = (q[1][c] << 1) | (q[1][c] >> 6); + } + uint32_t e = yapf__bc7_fit(px, sel, 16, v[0], v[1], 0, 3, 2, idx, wt); + if (e < cerr) { cerr = e; memcpy(bq, q, sizeof(q)); memcpy(bidx, idx, sizeof(idx)); } + float n0[4], n1[4]; + if (e == 0 || yapf__lsq((const float (*)[4])p, wt, 16, 3, n0, n1) != 0) break; + memcpy(e0, n0, sizeof(n0)); + memcpy(e1, n1, sizeof(n1)); + } + int lo = 255, hi = 0; + for (int i = 0; i < 16; i++) { if (px[i][3] < lo) lo = px[i][3]; if (px[i][3] > hi) hi = px[i][3]; } + int ba[2] = { lo, hi }, aidx[16] = {0}; + uint32_t aerr = UINT32_MAX; + for (int d0 = 0; d0 < 4; d0++) + for (int d1 = 0; d1 < 4; d1++) { + int a0 = lo + d0, a1 = hi - d1, v0[4], v1[4], idx[16]; + float wt[16]; + if (a0 > a1) continue; + v0[3] = a0; v1[3] = a1; + uint32_t e = yapf__bc7_fit(px, sel, 16, v0, v1, 3, 1, 2, idx, wt); + if (e < aerr) { aerr = e; ba[0] = a0; ba[1] = a1; memcpy(aidx, idx, sizeof(idx)); } + } + /* anchors: the first index of each set keeps its top bit clear */ + if (bidx[0] >= 2) { + for (int c = 0; c < 3; c++) { int t = bq[0][c]; bq[0][c] = bq[1][c]; bq[1][c] = t; } + for (int i = 0; i < 16; i++) bidx[i] = 3 - bidx[i]; + } + if (aidx[0] >= 2) { + int t = ba[0]; ba[0] = ba[1]; ba[1] = t; + for (int i = 0; i < 16; i++) aidx[i] = 3 - aidx[i]; + } + unsigned pos = 0; + memset(out, 0, 16); + yapf__putbits(out, &pos, 1u << 5, 6); + yapf__putbits(out, &pos, 0, 2); /* no rotation */ + for (int c = 0; c < 3; c++) { + yapf__putbits(out, &pos, (uint32_t)bq[0][c], 7); + yapf__putbits(out, &pos, (uint32_t)bq[1][c], 7); + } + yapf__putbits(out, &pos, (uint32_t)ba[0], 8); + yapf__putbits(out, &pos, (uint32_t)ba[1], 8); + for (int i = 0; i < 16; i++) yapf__putbits(out, &pos, (uint32_t)bidx[i], i ? 2 : 1); + for (int i = 0; i < 16; i++) yapf__putbits(out, &pos, (uint32_t)aidx[i], i ? 2 : 1); + return cerr + aerr; +} + +/* BC7 mode 1: two subsets, RGB 6 bits + a shared p-bit per subset, 3-bit + * indices; for opaque blocks with two colour regions. */ +static uint32_t yapf__bc7_mode1(const uint8_t px[16][4], int part, uint8_t *out) { + int q[2][2][3], pb[2], idx16[16]; + uint32_t total = 0; + for (int k = 0; k < 2; k++) { + int sel[16], n = 0; + float p[16][4], e0[4], e1[4]; + for (int i = 0; i < 16; i++) + if (yapf__subset(2, part, i) == k) { + sel[n] = i; + for (int c = 0; c < 3; c++) p[n][c] = px[i][c]; + n++; + } + yapf__extents((const float (*)[4])p, n, 3, e0, e1); + uint32_t best = UINT32_MAX; + for (int it = 0; it < 2; it++) { + for (int pbit = 0; pbit < 2; pbit++) { + int qq[2][3], v[2][4] = {{0}}, idx[16]; + float wt[16]; + for (int c = 0; c < 3; c++) { + const float *e[2] = { e0, e1 }; + for (int j = 0; j < 2; j++) { + qq[j][c] = yapf__clampi((e[j][c] * 127.0f / 255.0f - (float)pbit) / 2.0f, 0, 63); + int x = (qq[j][c] << 1) | pbit; + v[j][c] = (x << 1) | (x >> 6); + } + } + uint32_t er = yapf__bc7_fit(px, sel, n, v[0], v[1], 0, 3, 3, idx, wt); + if (er < best) { + best = er; + pb[k] = pbit; + memcpy(q[k], qq, sizeof(qq)); + for (int i = 0; i < n; i++) idx16[sel[i]] = idx[i]; + } + if (it == 0 && pbit == 1) { + float n0[4], n1[4]; + if (yapf__lsq((const float (*)[4])p, wt, n, 3, n0, n1) == 0) { + memcpy(e0, n0, sizeof(n0)); + memcpy(e1, n1, sizeof(n1)); + } + } + } + } + total += best; + int anchor = k == 0 ? 0 : yapf__anchor2[part]; + if (idx16[anchor] >= 4) { + for (int c = 0; c < 3; c++) { int t = q[k][0][c]; q[k][0][c] = q[k][1][c]; q[k][1][c] = t; } + for (int i = 0; i < 16; i++) if (yapf__subset(2, part, i) == k) idx16[i] = 7 - idx16[i]; + } + } + unsigned pos = 0; + memset(out, 0, 16); + yapf__putbits(out, &pos, 2, 2); + yapf__putbits(out, &pos, (uint32_t)part, 6); + for (int c = 0; c < 3; c++) + for (int k = 0; k < 2; k++) { + yapf__putbits(out, &pos, (uint32_t)q[k][0][c], 6); + yapf__putbits(out, &pos, (uint32_t)q[k][1][c], 6); + } + yapf__putbits(out, &pos, (uint32_t)pb[0], 1); + yapf__putbits(out, &pos, (uint32_t)pb[1], 1); + for (int i = 0; i < 16; i++) + yapf__putbits(out, &pos, (uint32_t)idx16[i], 3 - yapf__is_anchor(2, part, i)); + return total; +} + +/* BC7: mode 6, plus mode 5 when there is alpha, plus mode 1 on the two + * most promising partitions (least summed per-subset variance) for + * opaque blocks. The block with the smallest error wins. */ +static void yapf__bc7_encode(const uint8_t px[16][4], uint8_t *out) { + uint8_t cand[16]; + uint32_t best = yapf__bc7_mode6(px, out), e; + int opaque = 1; + for (int i = 0; i < 16; i++) if (px[i][3] != 255) opaque = 0; + if (best == 0) return; + if (!opaque) { + if ((e = yapf__bc7_mode5(px, cand)) < best) { best = e; memcpy(out, cand, 16); } + return; + } + float score[64]; + for (int part = 0; part < 64; part++) { + float s = 0; + for (int k = 0; k < 2; k++) { + float sum[3] = {0}, sq[3] = {0}; + int n = 0; + for (int i = 0; i < 16; i++) + if (yapf__subset(2, part, i) == k) { + n++; + for (int c = 0; c < 3; c++) { sum[c] += px[i][c]; sq[c] += (float)px[i][c] * px[i][c]; } + } + for (int c = 0; c < 3; c++) s += sq[c] - sum[c] * sum[c] / (float)n; + } + score[part] = s; + } + for (int t = 0; t < 2; t++) { + int bp = 0; + for (int part = 1; part < 64; part++) if (score[part] < score[bp]) bp = part; + score[bp] = 1e30f; + if ((e = yapf__bc7_mode1(px, bp, cand)) < best) { best = e; memcpy(out, cand, 16); } + } +} + +/* BC6H unsigned, one region: tries modes 11–14 (10.10, 11.9, 12.8, 16.4) + * and keeps the one with the smallest error in half-float bits. */ +static void yapf__bc6_encode(const uint16_t px[16][3], uint8_t *out) { + float p[16][4], e0[4], e1[4]; + int hv[16][3]; + for (int i = 0; i < 16; i++) + for (int c = 0; c < 3; c++) { + int h = px[i][c]; + if (h & 0x8000) h = 0; /* negative: unsigned format */ + if (h > 0x7BFF) h = 0x7BFF; /* inf / nan: largest finite */ + hv[i][c] = h; + p[i][c] = (float)h * 64.0f / 31.0f; /* interpolation domain */ + } + yapf__extents((const float (*)[4])p, 16, 3, e0, e1); + + uint8_t best[16]; + uint64_t best_err = UINT64_MAX; + memset(best, 0, 16); + for (int m = 10; m < 14; m++) { + int W = yapf__bc6_modes[m].w, d = yapf__bc6_modes[m].dr; + float f0[4], f1[4]; + memcpy(f0, e0, sizeof(f0)); + memcpy(f1, e1, sizeof(f1)); + for (int it = 0; it < 2; it++) { + int q[2][3], ok = 1; + for (int c = 0; c < 3; c++) { + const float *e[2] = { f0, f1 }; + for (int k = 0; k < 2; k++) { + float u = e[k][c]; + int v = W >= 15 ? yapf__clampi(u, 0, (1 << W) - 1) + : yapf__clampi((u - (float)(1 << (15 - W))) / (float)(1 << (16 - W)), + 0, (1 << W) - 1); + q[k][c] = v; + } + if (m != 10) { + int delta = q[1][c] - q[0][c]; + if (delta < -(1 << (d - 1)) || delta > (1 << (d - 1)) - 1) ok = 0; + } + } + if (!ok) break; + int u[2][3]; + for (int k = 0; k < 2; k++) for (int c = 0; c < 3; c++) u[k][c] = yapf__bc6_unq(q[k][c], W, 0); + int idx[16]; + uint64_t err = 0; + float wt[16]; + for (int i = 0; i < 16; i++) { + uint64_t be = UINT64_MAX; + int bi = 0; + for (int j = 0; j < 16; j++) { + uint64_t er = 0; + for (int c = 0; c < 3; c++) { + int64_t dd = (int64_t)yapf__bc6_finish(yapf__bc_interp(u[0][c], u[1][c], yapf__w4[j]), 0) - hv[i][c]; + er += (uint64_t)(dd * dd); + } + if (er < be) { be = er; bi = j; } + } + idx[i] = bi; + err += be; + wt[i] = yapf__w4[bi] / 64.0f; + } + if (err < best_err) { + best_err = err; + int swap = idx[0] >= 8, a = swap ? 1 : 0, b = swap ? 0 : 1; + int f[13] = {0}; + for (int c = 0; c < 3; c++) { + f[c * 4] = q[a][c]; + f[c * 4 + 1] = m == 10 ? q[b][c] : (q[b][c] - q[a][c]) & ((1 << d) - 1); + } + /* with swapped endpoints the delta must still fit */ + int fits = 1; + if (m != 10) + for (int c = 0; c < 3; c++) { + int delta = q[b][c] - q[a][c]; + if (delta < -(1 << (d - 1)) || delta > (1 << (d - 1)) - 1) fits = 0; + } + if (fits) { + memset(best, 0, 16); + unsigned pos = 0; + yapf__putbits(best, &pos, yapf__bc6_modes[m].code, 5); + for (const yapf__seg *g = yapf__bc6_modes[m].seg; g->n; g++) + yapf__putbits(best, &pos, (uint32_t)(f[g->f] >> g->lo), g->n); + for (int i = 0; i < 16; i++) { + int ix = swap ? 15 - idx[i] : idx[i]; + yapf__putbits(best, &pos, (uint32_t)ix, i == 0 ? 3 : 4); + } + } else { + best_err = UINT64_MAX; + } + } + float n0[4], n1[4]; + if (yapf__lsq((const float (*)[4])p, wt, 16, 3, n0, n1) != 0) break; + memcpy(f0, n0, sizeof(n0)); + memcpy(f1, n1, sizeof(n1)); + } + } + memcpy(out, best, 16); +} + +/* Reads texel (x, y) of a pixel surface slice as RGBA8 / RGB half. */ +static void yapf__texel_rgba8(const yapf_texture_t *t, const uint8_t *src, uint32_t w, + uint32_t h, uint32_t x, uint32_t y, uint8_t *o) +{ + if (x >= w) x = w - 1; /* edge blocks repeat the last texel */ + if (y >= h) y = h - 1; + const uint8_t *s = src + ((size_t)y * w + x) * t->channels; + switch (t->channels) { + case 1: o[0] = o[1] = o[2] = s[0]; o[3] = 255; break; + case 2: o[0] = o[1] = o[2] = s[0]; o[3] = s[1]; break; + case 3: o[0] = s[0]; o[1] = s[1]; o[2] = s[2]; o[3] = 255; break; + default: memcpy(o, s, 4); break; + } +} + +static void yapf__texel_half(const yapf_texture_t *t, const uint8_t *src, uint32_t w, + uint32_t h, uint32_t x, uint32_t y, uint16_t *o) +{ + if (x >= w) x = w - 1; + if (y >= h) y = h - 1; + size_t sb = yapf__sbytes(t->sample); + const uint8_t *s = src + ((size_t)y * w + x) * t->channels * sb; + for (int c = 0; c < 3; c++) { + int k = t->channels >= 3 ? c : 0; + uint32_t v = yapf__get_sample(s + (size_t)k * sb, t->sample); + if (t->sample == YAPF_SAMPLE_F16) { o[c] = (uint16_t)v; continue; } + float f; + if (t->sample == YAPF_SAMPLE_F32) memcpy(&f, &v, 4); + else f = t->sample == YAPF_SAMPLE_U16 ? (float)v / 65535.0f : (float)v / 255.0f; + o[c] = yapf_float_to_half(f); + } +} + +int yapf_tex_compress(const yapf_texture_t *src, uint8_t format, yapf_texture_t **out) { + uint32_t S; + if (!out) return YAPF_ERR_INVALID; + *out = NULL; + if (yapf__tex_check(src, &S) != 0 || src->payload != YAPF_PAYLOAD_PIXELS || !src->data) + return YAPF_ERR_INVALID; + int hdr = format == YAPF_GPU_BC6H; + if (format != YAPF_GPU_BC1 && format != YAPF_GPU_BC3 && format != YAPF_GPU_BC4 && + format != YAPF_GPU_BC5 && format != YAPF_GPU_BC7 && !hdr) + return YAPF_ERR_UNSUPPORTED; + if (hdr == (src->sample == YAPF_SAMPLE_U8)) return YAPF_ERR_INVALID; + + yapf_texture_t d = *src; + d.payload = YAPF_PAYLOAD_BLOCKS; + d.format = format; + d.sample = hdr ? YAPF_SAMPLE_F16 : YAPF_SAMPLE_U8; + d.channels = format == YAPF_GPU_BC4 ? 1 : format == YAPF_GPU_BC5 ? 2 : hdr ? 3 + : format == YAPF_GPU_BC3 ? 4 : (src->channels == 4 || src->channels == 2) ? 4 : 3; + for (int c = d.channels; c < 4; c++) d.channel_map[c] = YAPF_CH_DEFAULT; + d.linear_mask &= (uint8_t)((1u << d.channels) - 1); + if (d.normal && d.channels == 2) d.normal = (uint8_t)((d.normal & 3) | YAPF_NORMAL_XY); + yapf_texture_t *t = yapf_tex_create(&d); + if (!t) return YAPF_ERR_OOM; + for (uint32_t i = 0; i < src->note_count; i++) + if (yapf_tex_add_note(t, src->notes[i].tag, src->notes[i].data, src->notes[i].size) != YAPF_OK) { + yapf_tex_free(t); + return YAPF_ERR_OOM; + } + if (t->frames > 1 && src->frame_us) memcpy(t->frame_us, src->frame_us, t->frames * sizeof(uint32_t)); + + for (uint32_t s = 0; s < S; s++) { + uint32_t m = s % src->mip_levels, w, h, dd, nbx, nby, B; + yapf_tex_mip_size(src, m, &w, &h, &dd); + yapf__mip_blocks(t, m, &nbx, &nby, &B); + size_t slice = (size_t)w * h * src->channels * yapf__sbytes(src->sample); + uint8_t *o = (uint8_t *)t->data[s]; + for (uint32_t z = 0; z < dd; z++) { + const uint8_t *sl = (const uint8_t *)src->data[s] + z * slice; + for (uint32_t by = 0; by < nby; by++) + for (uint32_t bx = 0; bx < nbx; bx++, o += B) { + uint8_t px[16][4]; + uint16_t hx[16][3]; + for (int i = 0; i < 16; i++) { + uint32_t x = bx * 4 + (uint32_t)(i & 3), y = by * 4 + (uint32_t)(i >> 2); + if (hdr) yapf__texel_half(src, sl, w, h, x, y, hx[i]); + else yapf__texel_rgba8(src, sl, w, h, x, y, px[i]); + } + if (format == YAPF_GPU_BC4 || format == YAPF_GPU_BC5) { + /* raw channels, not the gray expansion */ + for (int k = 0; k < (format == YAPF_GPU_BC5 ? 2 : 1); k++) { + uint8_t v[16]; + for (int i = 0; i < 16; i++) { + uint32_t x = bx * 4 + (uint32_t)(i & 3), y = by * 4 + (uint32_t)(i >> 2); + if (x >= w) x = w - 1; + if (y >= h) y = h - 1; + int c = k < src->channels ? k : 0; + v[i] = sl[((size_t)y * w + x) * src->channels + c]; + } + yapf__bc4_encode(v, o + 8 * k); + } + } else if (format == YAPF_GPU_BC1) { + yapf__bc1_encode((const uint8_t (*)[4])px, o, 0); + } else if (format == YAPF_GPU_BC3) { + uint8_t a[16]; + for (int i = 0; i < 16; i++) a[i] = px[i][3]; + yapf__bc4_encode(a, o); + yapf__bc1_encode((const uint8_t (*)[4])px, o + 8, 1); + } else if (format == YAPF_GPU_BC7) { + yapf__bc7_encode((const uint8_t (*)[4])px, o); + } else { + yapf__bc6_encode((const uint16_t (*)[3])hx, o); + } + } + } + } + *out = t; + return YAPF_OK; +} + +/* ======================================================================== + * Mip generation + * ======================================================================== */ + +/* x^2.4 for x in [0, 1] without libm: x² · x^0.4, x^0.4 by Newton on + * y^5 = x². */ +static double yapf__pow24(double x) { + if (x <= 0) return 0; + double a = x * x, y = 1.0; + for (int i = 0; i < 60; i++) { + double y4 = y * y * y * y, ny = y - (y4 * y - a) / (5.0 * y4); + if (ny == y) break; + y = ny; + } + return x * x * y; +} + +static double yapf__srgb_to_lin(double v) { + return v <= 0.04045 ? v / 12.92 : yapf__pow24((v + 0.055) / 1.055); +} + +/* 8-bit sRGB: code → linear, and the linear thresholds halfway between + * codes, so linear → code rounds exactly as in sRGB space. */ +typedef struct { float lin[256], mid[255]; } yapf__srgb_tab; + +static void yapf__srgb_init(yapf__srgb_tab *t) { + for (int i = 0; i < 256; i++) t->lin[i] = (float)yapf__srgb_to_lin(i / 255.0); + for (int i = 0; i < 255; i++) t->mid[i] = (float)yapf__srgb_to_lin((i + 0.5) / 255.0); +} + +static float yapf__lin_to_srgb(const yapf__srgb_tab *t, float v) { + int lo = 0, hi = 255; /* first code whose mid >= v */ + while (lo < hi) { + int m = (lo + hi) / 2; + if (t->mid[m] < v) lo = m + 1; else hi = m; + } + return (float)lo / 255.0f; +} + +/* Index of the alpha channel, or -1. */ +static int yapf__alpha_channel(const yapf_texture_t *t) { + for (int c = 0; c < t->channels; c++) + if (t->channel_map[c] == YAPF_CH_ALPHA || t->channel_map[c] == YAPF_CH_OPACITY) return c; + if (t->channel_map[t->channels - 1] == YAPF_CH_DEFAULT && (t->channels == 2 || t->channels == 4)) + return t->channels - 1; + return -1; +} + +static float yapf__load_f(const yapf_texture_t *t, const uint8_t *p) { + uint32_t v = yapf__get_sample(p, t->sample); + switch (t->sample) { + case YAPF_SAMPLE_U8: return (float)v / 255.0f; + case YAPF_SAMPLE_U16: return (float)v / 65535.0f; + case YAPF_SAMPLE_F16: return yapf_half_to_float((uint16_t)v); + default: { float f; memcpy(&f, &v, 4); return f; } + } +} + +static void yapf__store_f(const yapf_texture_t *t, uint8_t *p, float f) { + uint32_t v; + switch (t->sample) { + case YAPF_SAMPLE_U8: v = (uint32_t)yapf__clampi(f * 255.0f, 0, 255); break; + case YAPF_SAMPLE_U16: v = (uint32_t)yapf__clampi(f * 65535.0f, 0, 65535); break; + case YAPF_SAMPLE_F16: v = yapf_float_to_half(f); break; + default: memcpy(&v, &f, 4); break; + } + yapf__put_sample(p, t->sample, v); +} + +/* Share of alpha values above ref after scaling by k, measured on the + * values as they will be stored (q = 255 or 65535; 0 for floats). */ +static double yapf__coverage(const float *a, size_t n, float ref, float k, float q) { + size_t c = 0; + for (size_t i = 0; i < n; i++) { + float v = a[i] * k; + if (q > 0) { + v = v > 1.0f ? q : (float)(int)(v * q + 0.5f); + if (v > ref * q) c++; + } else if (v > ref) { + c++; + } + } + return n ? (double)c / (double)n : 0.0; +} + +int yapf_tex_build_mips(yapf_texture_t *tex, uint32_t levels, uint8_t alpha_ref) { + uint32_t S; + if (!tex || !tex->internal_ || yapf__tex_check(tex, &S) != 0) return YAPF_ERR_INVALID; + if (tex->payload != YAPF_PAYLOAD_PIXELS) return YAPF_ERR_INVALID; + uint32_t full = 1; + while (full < YAPF_MAX_MIPS && + ((tex->width >> full) || (tex->height >> full) || + (tex->shape == YAPF_SHAPE_VOLUME && (tex->depth >> full)))) + full++; + if (levels == 0 || levels > full) levels = full; + + int ach = yapf__alpha_channel(tex); + if (alpha_ref && ach < 0) return YAPF_ERR_INVALID; + int srgb[4]; + for (int c = 0; c < tex->channels; c++) + srgb[c] = (tex->flags & YAPF_FLAG_SRGB) && tex->sample == YAPF_SAMPLE_U8 && c != ach && + !(tex->linear_mask & (1u << c)) && + (tex->transfer == YAPF_TRANSFER_UNSPECIFIED || tex->transfer == YAPF_TRANSFER_SRGB); + + yapf__srgb_tab *st = (yapf__srgb_tab *)malloc(sizeof(yapf__srgb_tab)); + if (!st) return YAPF_ERR_OOM; + yapf__srgb_init(st); + + yapf_texture_t d = *tex; + d.mip_levels = (uint8_t)levels; + uint32_t NS = yapf_tex_surface_count(&d); + void **data = (void **)calloc(NS, sizeof(void *)); + if (!data) { free(st); return YAPF_ERR_OOM; } + int rc = YAPF_OK; + size_t sb = yapf__sbytes(tex->sample), ch = tex->channels; + float qs = tex->sample == YAPF_SAMPLE_U8 ? 255.0f : tex->sample == YAPF_SAMPLE_U16 ? 65535.0f : 0.0f; + + for (uint32_t base = 0; base < NS && rc == YAPF_OK; base += levels) { + uint32_t old = base / levels * tex->mip_levels; /* mip 0 of this face in tex */ + data[base] = tex->data[old]; + tex->data[old] = NULL; + float *alpha0 = NULL; + double target = 0; + for (uint32_t m = 1; m < levels && rc == YAPF_OK; m++) { + uint32_t pw, ph, pd, w, h, dd; + yapf_tex_mip_size(&d, m - 1, &pw, &ph, &pd); + yapf_tex_mip_size(&d, m, &w, &h, &dd); + const uint8_t *src = (const uint8_t *)data[base + m - 1]; + uint8_t *dst = (uint8_t *)malloc(yapf_tex_surface_bytes(&d, m)); + float *alpha = alpha_ref ? (float *)malloc((size_t)w * h * dd * sizeof(float)) : NULL; + if (!dst || (alpha_ref && !alpha)) { free(dst); free(alpha); rc = YAPF_ERR_OOM; break; } + data[base + m] = dst; + if (alpha_ref && m == 1) { + size_t n0 = (size_t)pw * ph * pd; + alpha0 = (float *)malloc(n0 * sizeof(float)); + if (!alpha0) { free(alpha); rc = YAPF_ERR_OOM; break; } + for (size_t i = 0; i < n0; i++) alpha0[i] = yapf__load_f(tex, src + (i * ch + ach) * sb); + target = yapf__coverage(alpha0, n0, alpha_ref / 255.0f, 1.0f, qs); + } + for (uint32_t z = 0; z < dd; z++) + for (uint32_t y = 0; y < h; y++) + for (uint32_t x = 0; x < w; x++) { + uint32_t xs[2] = { 2 * x < pw ? 2 * x : pw - 1, 2 * x + 1 < pw ? 2 * x + 1 : pw - 1 }; + uint32_t ys[2] = { 2 * y < ph ? 2 * y : ph - 1, 2 * y + 1 < ph ? 2 * y + 1 : ph - 1 }; + uint32_t zs[2] = { 2 * z < pd ? 2 * z : pd - 1, 2 * z + 1 < pd ? 2 * z + 1 : pd - 1 }; + int nz = d.shape == YAPF_SHAPE_VOLUME ? 2 : 1; + size_t o = (((size_t)z * h + y) * w + x) * ch; + for (size_t c = 0; c < ch; c++) { + float sum = 0; + for (int k = 0; k < nz; k++) + for (int j = 0; j < 2; j++) + for (int i = 0; i < 2; i++) { + size_t q = (((size_t)zs[k] * ph + ys[j]) * pw + xs[i]) * ch + c; + sum += srgb[c] ? st->lin[src[q]] : yapf__load_f(tex, src + q * sb); + } + float v = sum / (float)(4 * nz); + if (srgb[c]) v = yapf__lin_to_srgb(st, v); + if (alpha && (int)c == ach) alpha[o / ch] = v; + yapf__store_f(tex, dst + (o + c) * sb, v); + } + } + if (alpha) { + /* scale alpha so coverage at alpha_ref matches mip 0 */ + size_t n = (size_t)w * h * dd; + float lo = 0.0f, hi = 8.0f, ref = alpha_ref / 255.0f; + for (int it = 0; it < 20; it++) { + float mid = 0.5f * (lo + hi); + if (yapf__coverage(alpha, n, ref, mid, qs) < target) lo = mid; else hi = mid; + } + double clo = yapf__coverage(alpha, n, ref, lo, qs), chi = yapf__coverage(alpha, n, ref, hi, qs); + float k = (target - clo <= chi - target) ? lo : hi; /* nearer bound */ + for (size_t i = 0; i < n; i++) { + float v = alpha[i] * k; + if (tex->sample != YAPF_SAMPLE_F16 && tex->sample != YAPF_SAMPLE_F32 && v > 1.0f) v = 1.0f; + yapf__store_f(tex, dst + (i * ch + ach) * sb, v); + } + free(alpha); + } + } + free(alpha0); + } + if (rc != YAPF_OK) { + for (uint32_t s = 0; s < NS; s++) { + if (s % levels == 0) { /* give mip 0 back */ + if (data[s]) tex->data[s / levels * tex->mip_levels] = data[s]; + } else { + free(data[s]); + } + } + free(data); + free(st); + return rc; + } + free(st); + for (uint32_t s = 0; s < S; s++) free(tex->data[s]); /* old mips (mip 0 moved) */ + free(tex->data); + tex->data = data; + tex->mip_levels = (uint8_t)levels; + int any_srgb = 0; + for (int c = 0; c < tex->channels; c++) any_srgb |= srgb[c]; + tex->mip_filter = (uint8_t)(YAPF_MIP_BOX | (any_srgb ? YAPF_MIP_LINEAR_LIGHT : 0)); + if (alpha_ref) { tex->flags |= YAPF_FLAG_COVERAGE; tex->alpha_ref = alpha_ref; } + else tex->flags &= (uint8_t)~YAPF_FLAG_COVERAGE; + return YAPF_OK; +} diff --git a/yapf.h b/yapf.h index 4e60939..5f57933 100644 --- a/yapf.h +++ b/yapf.h @@ -38,6 +38,13 @@ * loader can upload the decoded buffer straight to a texture without any * additional transcoding step. * + * Two versions exist. Version 1 (below) is a single 8-bit 2D image with + * optional mips. Version 2 is the texture container: GPU block payloads + * (BCn / ETC2 / ASTC), 16-bit and float samples, cubemaps, arrays and + * volumes, channel meanings, per-tile checksums, optional entropy-coded + * planes, notes (metadata) and animation frames. Its complete + * specification is docs/FORMAT.md; a summary follows the v1 spec. + * * ========================================================================= * FILE FORMAT (version 1) * ========================================================================= @@ -140,6 +147,37 @@ * ───────────────────────────────────────────────── * bit 0 PREMULT_ALPHA alpha is premultiplied into RGB * bit 1 SRGB pixels are in sRGB gamma (informational only) + * bit 2 COVERAGE (v2) mips keep alpha-test coverage at alpha_ref + * + * ========================================================================= + * FILE FORMAT (version 2) — summary; the full text is docs/FORMAT.md + * ========================================================================= + * + * HEADER (64 bytes) + * 0x00 "YAPF" 0x04 version 2 0x05 channels 0x06 format + * 0x07 mips 0x08 flags 0x09 payload 0x0A sample 0x0B shape + * 0x0C width u32 0x10 height u32 0x14 depth u32 + * 0x18 layers u16 0x1A frames u16 0x1C needs u32 + * 0x20 channel map[4] 0x24 linear mask 0x25 normal 0x26 primaries + * 0x27 transfer 0x28 mip filter 0x29 alpha ref 0x2A loop count u16 + * 0x2C notes size u32 0x30 file size u64 + * 0x38 tables check u32 (yapf-sum of bytes [0x40, data start)) + * 0x3C header check u32 (yapf-sum of bytes [0, 0x3C)) + * SURFACE TABLE 16 bytes per surface: offset u64, size u32, + * tile-table check u32. Surface s of (frame, layer, + * face, mip) is ((frame·layers + layer)·faces + face)·mips + mip + * FRAME TABLE u32 duration in µs per frame (only when frames > 1) + * NOTES tag[4], length u32, bytes — repeated + * SURFACE DATA smallest mips first, so a file prefix is a complete + * low-resolution texture. Per surface: u32 book size, + * code book (Huffman tables by slot and context, only + * with entropy coding), tile table (size u32 with bit 31 = + * packed, yapf-sum u32 per tile), then the tiles. Pixel + * tiles are the v1 tiles plus byte planes for wide samples + * and entropy-coded planes; block tiles hold 16 × 16 GPU + * blocks, raw or packed lane by lane. + * NEEDS a decoder must refuse a file with a needs bit it does + * not implement; everything else may be ignored safely. */ #ifndef YAPF_H @@ -156,11 +194,58 @@ #define YAPF_GPU_R8 0x03 #define YAPF_GPU_SRGB8_A8 0x04 #define YAPF_GPU_SRGB8 0x05 +/* v2 pixel hints for wide samples */ +#define YAPF_GPU_RGBA16 0x06 +#define YAPF_GPU_RGB16 0x07 +#define YAPF_GPU_RG16 0x08 +#define YAPF_GPU_R16 0x09 +#define YAPF_GPU_RGBA16F 0x0A +#define YAPF_GPU_RGB16F 0x0B +#define YAPF_GPU_RG16F 0x0C +#define YAPF_GPU_R16F 0x0D +#define YAPF_GPU_RGBA32F 0x0E +#define YAPF_GPU_RGB32F 0x0F +#define YAPF_GPU_RG32F 0x10 +#define YAPF_GPU_R32F 0x11 +/* v2 GPU block formats (payload = YAPF_PAYLOAD_BLOCKS). The sRGB flag + * selects the _SRGB variant where the API has one. */ +#define YAPF_GPU_BC1 0x40 /* RGB(A1), 8 bytes per 4×4 */ +#define YAPF_GPU_BC2 0x41 /* RGBA, explicit 4-bit alpha */ +#define YAPF_GPU_BC3 0x42 /* RGBA */ +#define YAPF_GPU_BC4 0x43 /* R */ +#define YAPF_GPU_BC4_SNORM 0x44 +#define YAPF_GPU_BC5 0x45 /* RG (normal maps) */ +#define YAPF_GPU_BC5_SNORM 0x46 +#define YAPF_GPU_BC6H 0x47 /* RGB half float, unsigned */ +#define YAPF_GPU_BC6H_SF 0x48 /* RGB half float, signed */ +#define YAPF_GPU_BC7 0x49 /* RGB(A) */ +#define YAPF_GPU_ETC2_RGB8 0x50 +#define YAPF_GPU_ETC2_RGB8A1 0x51 +#define YAPF_GPU_ETC2_RGBA8 0x52 +#define YAPF_GPU_EAC_R11 0x53 +#define YAPF_GPU_EAC_R11_SNORM 0x54 +#define YAPF_GPU_EAC_RG11 0x55 +#define YAPF_GPU_EAC_RG11_SNORM 0x56 +#define YAPF_GPU_ASTC_4x4 0x60 /* ASTC: 0x60 + footprint index */ +#define YAPF_GPU_ASTC_5x4 0x61 +#define YAPF_GPU_ASTC_5x5 0x62 +#define YAPF_GPU_ASTC_6x5 0x63 +#define YAPF_GPU_ASTC_6x6 0x64 +#define YAPF_GPU_ASTC_8x5 0x65 +#define YAPF_GPU_ASTC_8x6 0x66 +#define YAPF_GPU_ASTC_8x8 0x67 +#define YAPF_GPU_ASTC_10x5 0x68 +#define YAPF_GPU_ASTC_10x6 0x69 +#define YAPF_GPU_ASTC_10x8 0x6A +#define YAPF_GPU_ASTC_10x10 0x6B +#define YAPF_GPU_ASTC_12x10 0x6C +#define YAPF_GPU_ASTC_12x12 0x6D /* ── Flags ───────────────────────────────────────────────────────────── */ #define YAPF_FLAG_PREMULT_ALPHA 0x01 #define YAPF_FLAG_SRGB 0x02 +#define YAPF_FLAG_COVERAGE 0x04 /* v2: mips preserve alpha-test coverage */ /* ── Channel constants ───────────────────────────────────────────────── */ @@ -176,12 +261,16 @@ #define YAPF_ERR_IO -2 #define YAPF_ERR_OOM -3 #define YAPF_ERR_CORRUPT -4 +#define YAPF_ERR_UNSUPPORTED -5 /* valid file, feature this decoder lacks */ +#define YAPF_ERR_CHECKSUM -6 /* a yapf-sum does not match */ +#define YAPF_ERR_TRUNCATED -7 /* data lies beyond the bytes available yet */ /* ── Limits ──────────────────────────────────────────────────────────── */ #define YAPF_MAX_DIM 65535u #define YAPF_MAX_MIPS 16u #define YAPF_TILE_SIZE 64u +#define YAPF_BLOCK_TILE 16u /* v2 block tiles: 16 × 16 GPU blocks */ /* ── Image descriptor ────────────────────────────────────────────────── */ @@ -196,6 +285,150 @@ typedef struct { uint8_t **mips; /* mips[0]=base alias, mips[1..mip_levels-1] owned */ } yapf_image_t; +/* ── Version 2: texture description ──────────────────────────────────── */ + +/* payload */ +#define YAPF_PAYLOAD_PIXELS 0 /* lossless YAPF tiles */ +#define YAPF_PAYLOAD_BLOCKS 1 /* GPU blocks, uploaded as they are */ + +/* sample: how one channel value is stored (pixel payload); for block + * payloads, the type of the decoded texels (U8, or F16 for BC6H) */ +#define YAPF_SAMPLE_U8 0 +#define YAPF_SAMPLE_U16 1 +#define YAPF_SAMPLE_F16 2 /* IEEE half, as uint16_t bits */ +#define YAPF_SAMPLE_F32 3 + +/* shape */ +#define YAPF_SHAPE_2D 0 +#define YAPF_SHAPE_CUBE 1 /* 6 faces: +X −X +Y −Y +Z −Z; with layers + > 1 a cube array */ +#define YAPF_SHAPE_VOLUME 2 /* depth slices; mips halve depth too */ + +/* needs: features a decoder must implement to read the file */ +#define YAPF_NEED_WIDE 0x01 /* sample other than U8 (pixel payload) */ +#define YAPF_NEED_BLOCKS 0x02 /* GPU block payload */ +#define YAPF_NEED_ENTROPY 0x04 /* code books: entropy planes, packed blocks */ +#define YAPF_NEED_LAYOUT 0x08 /* cube, volume or layers > 1 */ +#define YAPF_NEED_FRAMES 0x10 /* frames > 1 */ +#define YAPF_NEED_ALL 0x1F /* everything this library implements */ + +/* channel meanings (channel_map[c]); 0 = default for the channel count */ +#define YAPF_CH_DEFAULT 0 +#define YAPF_CH_RED 1 +#define YAPF_CH_GREEN 2 +#define YAPF_CH_BLUE 3 +#define YAPF_CH_ALPHA 4 +#define YAPF_CH_LUMA 5 +#define YAPF_CH_NORMAL_X 6 +#define YAPF_CH_NORMAL_Y 7 +#define YAPF_CH_NORMAL_Z 8 +#define YAPF_CH_ROUGHNESS 9 +#define YAPF_CH_GLOSS 10 /* smoothness = 1 − roughness */ +#define YAPF_CH_METALNESS 11 +#define YAPF_CH_OCCLUSION 12 /* ambient occlusion */ +#define YAPF_CH_HEIGHT 13 +#define YAPF_CH_SPECULAR 14 +#define YAPF_CH_EMISSIVE 15 +#define YAPF_CH_OPACITY 16 /* alpha-test / cut-out mask */ +#define YAPF_CH_MASK 17 /* generic mask */ +#define YAPF_CH_CAVITY 18 +#define YAPF_CH_THICKNESS 19 +#define YAPF_CH_ID 20 /* material or object id */ +#define YAPF_CH_UNUSED 255 + +/* normal: bits 0–1 convention, bits 2–3 encoding */ +#define YAPF_NORMAL_NONE 0x00 +#define YAPF_NORMAL_GL 0x01 /* +Y up (OpenGL, Blender, Unity) */ +#define YAPF_NORMAL_DX 0x02 /* −Y (DirectX, Unreal) */ +#define YAPF_NORMAL_XYZ 0x00 +#define YAPF_NORMAL_XY 0x04 /* Z = sqrt(1 − x² − y²) is reconstructed */ +#define YAPF_NORMAL_OCT 0x08 /* octahedral */ + +/* primaries / transfer (0 = unspecified: the sRGB flag decides) */ +#define YAPF_PRIMARIES_UNSPECIFIED 0 +#define YAPF_PRIMARIES_BT709 1 /* = sRGB */ +#define YAPF_PRIMARIES_DISPLAY_P3 2 +#define YAPF_PRIMARIES_BT2020 3 +#define YAPF_PRIMARIES_ACES_AP0 4 +#define YAPF_PRIMARIES_ACES_AP1 5 /* ACEScg */ +#define YAPF_PRIMARIES_ADOBE_RGB 6 + +#define YAPF_TRANSFER_UNSPECIFIED 0 +#define YAPF_TRANSFER_SRGB 1 +#define YAPF_TRANSFER_LINEAR 2 +#define YAPF_TRANSFER_GAMMA22 3 +#define YAPF_TRANSFER_PQ 4 /* SMPTE ST 2084 */ +#define YAPF_TRANSFER_HLG 5 +#define YAPF_TRANSFER_BT1886 6 /* gamma 2.4 */ + +/* mip filter (bits 0–6) | YAPF_MIP_LINEAR_LIGHT */ +#define YAPF_MIP_UNKNOWN 0 +#define YAPF_MIP_BOX 1 +#define YAPF_MIP_TENT 2 +#define YAPF_MIP_KAISER 3 +#define YAPF_MIP_LANCZOS 4 +#define YAPF_MIP_MITCHELL 5 +#define YAPF_MIP_AUTHORED 6 /* made by hand / by an external tool */ +#define YAPF_MIP_LINEAR_LIGHT 0x80 /* sRGB channels were filtered in linear light */ + +/* load flags */ +#define YAPF_LOAD_TRUST 0x01 /* skip checksum verification */ + +/* encode flags */ +#define YAPF_ENC_ENTROPY 0x01 /* allow entropy-coded planes and packed + blocks where they are smaller: ~10 % + smaller photos, ~3× smaller GPU blocks + on disk; slower to decode */ + +/* A note (metadata entry). Well-known tags: "AUTH" author, "LICN" + * licence, "SRCE" source, "TOOL" creating tool, "TITL" title, "DESC" + * description, "DATE" ISO 8601 date (all UTF-8 text); "ICCP" ICC profile; + * "ORNT" orientation (1 byte); "DPI " 2 × u32 milli-dots per inch; + * "CLL " 2 × u16 max content / max frame-average light level (cd/m²). + * Tags that start with a lowercase letter belong to applications. */ +typedef struct { + char tag[4]; + uint32_t size; + uint8_t *data; +} yapf_note_t; + +/* + * A version 2 texture. Surfaces are numbered + * s = ((frame × layers + layer) × faces + face) × mip_levels + mip + * (faces = 6 for cubes, else 1) and data[s] holds the whole surface: + * pixels: width_m × height_m × depth_m × channels samples, rows top to + * bottom, slices in order; samples in the native byte order + * (uint8_t, uint16_t, half bits as uint16_t, float); + * blocks: ceil(width_m / bw) × ceil(height_m / bh) blocks per slice, + * row-major — exactly what glCompressedTexImage / Vulkan take. + * width_m = max(1, width >> m), likewise height and (volumes) depth. + */ +typedef struct { + uint32_t width, height, depth; /* depth: volumes only, else 1 */ + uint16_t layers; /* array layers, >= 1 */ + uint16_t frames; /* animation frames, >= 1 */ + uint8_t shape; /* YAPF_SHAPE_* */ + uint8_t mip_levels; /* 1–16 */ + uint8_t payload; /* YAPF_PAYLOAD_* */ + uint8_t format; /* YAPF_GPU_*: block format, or hint */ + uint8_t sample; /* YAPF_SAMPLE_* */ + uint8_t channels; /* 1–4 */ + uint8_t flags; /* YAPF_FLAG_* */ + uint8_t channel_map[4]; /* YAPF_CH_* per channel */ + uint8_t linear_mask; /* bit c: channel c is linear even in + an sRGB image (e.g. alpha, masks) */ + uint8_t normal; /* YAPF_NORMAL_* */ + uint8_t primaries, transfer; /* YAPF_PRIMARIES_*, YAPF_TRANSFER_* */ + uint8_t mip_filter; /* YAPF_MIP_* */ + uint8_t alpha_ref; /* coverage reference (with COVERAGE) */ + uint16_t loop_count; /* animation loops, 0 = forever */ + uint32_t *frame_us; /* frames durations in µs, or NULL */ + uint32_t note_count; + yapf_note_t *notes; + void **data; /* one pointer per surface */ + void *internal_; /* library use: file tables, ownership */ +} yapf_texture_t; + /* ── Public API ──────────────────────────────────────────────────────── */ #ifdef __cplusplus @@ -249,6 +482,132 @@ void yapf_free_buffer(void *data); */ void yapf_free(yapf_image_t *img); +/* + * The v1 functions above also read version 2 files that hold 8-bit pixels + * of a 2D, cube or array texture: they return frame 0, layer 0, face 0 + * with all its mips. yapf_save / yapf_encode always write version 1. + */ + +/* ── Version 2: textures ─────────────────────────────────────────────── */ + +/* + * yapf_tex_create — a library-owned texture with the description of + * `desc` (every field up to loop_count; notes, frame times and data are + * not copied) and zeroed storage for every surface. frame_us is + * allocated (zeroed) when frames > 1. NULL on invalid description / OOM. + */ +yapf_texture_t *yapf_tex_create(const yapf_texture_t *desc); + +/* yapf_tex_free — release a texture made by the library. NULL-safe. */ +void yapf_tex_free(yapf_texture_t *tex); + +/* Surface helpers. */ +uint32_t yapf_tex_faces(const yapf_texture_t *tex); /* 6 or 1 */ +uint32_t yapf_tex_surface_count(const yapf_texture_t *tex); +uint32_t yapf_tex_surface(const yapf_texture_t *tex, uint32_t frame, + uint32_t layer, uint32_t face, uint32_t mip); +void yapf_tex_mip_size(const yapf_texture_t *tex, uint32_t mip, + uint32_t *w, uint32_t *h, uint32_t *d); +size_t yapf_tex_surface_bytes(const yapf_texture_t *tex, uint32_t mip); +size_t yapf_sample_bytes(uint8_t sample); /* 1, 2 or 4 */ + +/* Notes: add (library-owned textures only; data is copied) and find. */ +int yapf_tex_add_note(yapf_texture_t *tex, const char tag[4], + const void *data, uint32_t size); +const yapf_note_t *yapf_tex_find_note(const yapf_texture_t *tex, const char tag[4]); + +/* + * yapf_tex_encode / yapf_tex_save — write a version 2 file. `flags` is + * YAPF_ENC_*. Works on library-owned and caller-built textures alike + * (nothing is taken over). Returns YAPF_OK or a YAPF_ERR_* code. + */ +int yapf_tex_encode(const yapf_texture_t *tex, int flags, + void **out_data, size_t *out_size); +int yapf_tex_save(const char *filename, const yapf_texture_t *tex, int flags); + +/* + * yapf_tex_load / yapf_tex_load_memory — read a version 1 or 2 file and + * decode every surface on up to `threads` threads (0 = one per core). + * `flags` is YAPF_LOAD_*. On failure NULL is returned and *err (if not + * NULL) receives the YAPF_ERR_* reason. + */ +yapf_texture_t *yapf_tex_load(const char *filename, int flags, int threads, int *err); +yapf_texture_t *yapf_tex_load_memory(const void *buffer, size_t size, + int flags, int threads, int *err); + +/* + * Streaming and partial decoding. + * + * yapf_tex_open — read only the header, tables and notes. `avail` may be + * less than the whole file (a download in progress): it must cover the + * front matter, else YAPF_ERR_TRUNCATED. All data[] pointers are NULL. + * + * yapf_tex_stream_bytes — how many leading bytes of the file hold every + * surface of mip `first_mip` and smaller (the file stores small mips + * first, so these are a usable texture on their own). + * + * yapf_tex_decode_surface — decode surface s into tex->data[s] + * (allocated). `buffer`/`avail` is the file or a prefix of it; returns + * YAPF_ERR_TRUNCATED when the surface is not in it yet. + * + * yapf_tex_tile_grid / yapf_tex_decode_tile — virtual texturing: the + * surfaces of mip m are cut into tiles (64 × 64 pixels, or 16 × 16 + * blocks), tx across, ty down, per slice. decode_tile writes one tile + * into `out`, rows `stride` bytes apart, and reports its pixel rectangle. + */ +yapf_texture_t *yapf_tex_open(const void *buffer, size_t avail, int flags, int *err); +uint64_t yapf_tex_stream_bytes(const yapf_texture_t *tex, uint32_t first_mip); +int yapf_tex_decode_surface(yapf_texture_t *tex, const void *buffer, size_t avail, + uint32_t surface, int threads); +void yapf_tex_tile_grid(const yapf_texture_t *tex, uint32_t mip, + uint32_t *tiles_x, uint32_t *tiles_y); +int yapf_tex_decode_tile(const yapf_texture_t *tex, const void *buffer, size_t avail, + uint32_t surface, uint32_t tx, uint32_t ty, uint32_t slice, + void *out, size_t stride, + uint32_t *x, uint32_t *y, uint32_t *w, uint32_t *h); + +/* + * yapf_tex_build_mips — replace the mips of a library-owned pixel texture + * with a chain made from mip 0: `levels` levels (0 = down to 1 × 1). + * 2 × 2 (× 2 for volumes) box filter; sRGB channels are averaged in + * linear light. alpha_ref > 0 rescales alpha on every mip so the share + * of texels above alpha_ref stays that of mip 0 (fences and foliage do + * not thin out); it sets YAPF_FLAG_COVERAGE. Records the filter used. + */ +int yapf_tex_build_mips(yapf_texture_t *tex, uint32_t levels, uint8_t alpha_ref); + +/* ── GPU blocks ──────────────────────────────────────────────────────── */ + +/* Block footprint and size of a YAPF_GPU_* block format; 0 if not one. */ +int yapf_block_info(uint8_t format, uint32_t *block_w, uint32_t *block_h, + uint32_t *block_bytes); + +/* + * yapf_tex_compress — a new library-owned texture holding `src` (pixel + * payload) as GPU blocks. Built-in encoders: BC1, BC3, BC4, BC5, BC7 and + * BC6H (unsigned). LDR formats take 8-bit pixels; BC6H takes 16-bit, + * half or float pixels. For other formats bring blocks from your own + * encoder: a payload = BLOCKS texture with data[] filled. + */ +int yapf_tex_compress(const yapf_texture_t *src, uint8_t format, + yapf_texture_t **out); + +/* + * yapf_blocks_decode — decode a w × h image of BC1–BC7 blocks (previews, + * tools). LDR formats write RGBA8 (BC4 / BC5: R / RG, then 0, 255; + * snorm values shifted by +128); BC6H writes RGB half floats. Returns + * YAPF_ERR_UNSUPPORTED for ETC2 / EAC / ASTC. + */ +int yapf_blocks_decode(uint8_t format, const void *blocks, uint32_t w, uint32_t h, + void *out); + +/* yapf-sum, the checksum used by version 2 files. */ +uint32_t yapf_checksum(const void *data, size_t size); + +/* Half float conversion helpers. */ +uint16_t yapf_float_to_half(float f); +float yapf_half_to_float(uint16_t h); + #ifdef __cplusplus } #endif