hrx-loom

Loom: Model optimization is no longer locked by the engine.
Experimental dev preview.

hrx-loom is a new end-to-end stack (compiler, runtime and serving tools) that lets anyone write, optimize, serve and share kernels and models without modifying the inference engine or the server.

Kernels are written in .loom files, a vector programming model reminiscent of MLIR that is easy for agents. To show what the stack can do early on, we ported 60+ models as an experiment, and you can fetch and serve them today.

  1. Step 1Install

    Install hrx-loom from PyPI.

    $ pip install "hrx-loom[hf]==0.1.0"
  2. Step 2Fetch

    Pull a Loom model from Hugging Face.

    $ loom-fetch https://huggingface.co/loom-fun/unsloth-Qwen3-0.6B-loom/resolve/main/model.json --output=qwen3-0.6b
  3. Step 3Serve

    Start a local OpenAI-compatible server.

    $ loom-serve --device=amdgpu --model=qwen3-0.6b/model.json
Want more?Port
Yours

Install loom-skills and ask your agent to port and optimize any GGUF.

> /loom Port, validate and optimize <hf-gguf>.

See our porting guide for more info.

01 / try one

Models you can fetch and serve today.

Every model here is a model package on Hugging Face, tuned and tested on Strix Halo. Pick one, open it for its loom-fetch and loom-serve commands and the --memory_bytes it needs, and you're serving it in minutes. They're ranked by speedup over llama.cpp ROCm on the same weights, as a geometric mean across decode, prefill and 4-stream workloads. In the profile column, bars point up where Loom is faster and down where it's slower.

tok/s: loom over llama.cpp per-workload speedup, ±2× full scale gate pass · fail · no evidence T tools · V vision · R thinking · S speculative · P sampling every model, with detail →
hover any mark to trace a model across every chart · click to open it
02 / by workload

Early experiments, workload by workload.

These are first results, not a ceiling. Each model was ported and tuned by an agent using loom-skills, so the spread reflects how far each port has been optimized so far. One dot per model for each workload, on a log scale centred on parity with llama.cpp. Hollow dots are workloads where llama.cpp is still faster. The tick on each row is the median.

speedup · loom ÷ llama.cpp
03 / absolute

Tokens per second against weights size.

Ratios hide scale, so this is the raw rate. Each model is a pair joined by a line: a square for Loom and a circle for llama.cpp. A line that runs down from the square means Loom is slower on that workload. Both axes are logarithmic.

tok/s vs GB
04 / context

How the gap moves with context length.

One line per model, with the median drawn in bold. The shaded area is where llama.cpp is faster.

prefill speedup · document length1k → 32k
decode speedup · context lengthshort → 32k
05 / set free

Model optimization has been set free.

Historically, supporting a new model meant changing the inference engine first. Every new architecture or kernel had to wait on engine work, which slowed everyone down and left the community dependent on a small group of maintainers. With Loom, models and kernels live outside the engine: anyone can port, optimize and share a model without touching the runtime or the server.

A small SDK

The compiler, runtimes and devtools have no dependencies. Download them as a wheel or link them into your own application; they build from source in about a minute and would still fit on a few floppy disks.

Many targets

One stack targets x86, AMD GPU, AMD NPU and generic Vulkan devices.

Standalone or embedded

Run it as an inference server with loom-serve, or plug it into existing engines, like the experimental HRX backend for llama.cpp.

Tell us what you need

We're still learning how the community will use and deploy new inference stacks, so we'd love your feedback. Open an issue on hrx-system or loom-skills.