decision.js

So far

• 34 ms for 3 decisions on a support ticket with d1-omni (Apple M5 Pro, Chrome).
• About 2× faster than Google's LiteRT.js on the same model.
• 6 ms per ticket with EmbeddingGemma 2 typed decisions, which embeds the questions once and then only reads each new state; faster than Google's LiteRT-LM on the same architecture.
• Answers that match the models' official PyTorch code, and nothing leaves the page.

const model = await Decision.load(url);
const { answers } = await model.systemOne(ticket, {
  topic: { type: "choice", instructions: "What is the topic?",
           criteria: { billing: null, delivery: null } },
  churn: { type: "noul",   instructions: "Is the customer about to leave?" },
});
// answers.churn → { noul: 0.83 }

How it works

1 · Every question in one pass, no padding

Each question becomes its own prompt. decision.js packs them end to end on one token axis and runs the whole batch as a single GPU command buffer; per-token metadata tells the attention which tokens belong together. Engines with static shapes pad every prompt to a fixed length instead.

2 · Weights converted once, then straight to the matrix units

The weights ship as 8-bit (half the download). At load time a GPU kernel turns them into f16 laid out in 8×8 blocks, the shape the GPU's matrix units read, so the matrix multiplication loads them directly with no dequantization in the hot loop (the design of Google's ML Drift).

3 · Where the time goes

Once the matrix multiplications run near the hardware's limit, everything else is small, so it is fused: each d3-lite DeltaNet layer (convolution, norms, gates, the recurrence and the gated norm) is a single kernel, and the MLP's activation runs inside its matmul.

4 · Only the options are scored

A chat model would compute a score for every word of its vocabulary at every step. A decision needs one step and a few candidates: decision.js reads the hidden state at the answer position and scores only the options' codes.