An Inference NPU Trained a 325M Model to B200-Like Loss. What Does That Prove?
A 325M model trained from scratch on inference NPUs nearly matched its B200 twin’s loss, and both sets of weights are public. What can another team actually reproduce from the release?
Can hardware sold for inference also learn an entire language model? VIDRAFT's October 26 report offers a concrete result: a small model trained from scratch on an inference NPU reached almost the same next-byte prediction loss as its B200-trained counterpart. The useful finding concerns the quality of the resulting weights. It does not yet tell an infrastructure team how many GPUs it could replace, or what a supported training deployment would cost. VIDRAFT's announcement
What makes this an inference-hardware experiment
The developer identified the hardware as Rebellions ATOM-Max in a GeekNews post, while the model card specifies 32 NPU chips against one B200 GPU. Both details come from VIDRAFT’s account of the experiment. Developer post
Rebellions' own developer documentation supplies the relevant product context. It lists ATOM-Max support, describes its SDK as intended for inference, and says fine-tuning is not currently supported. Its normal workflow compiles an existing model into an executable representation and uses a runtime to produce outputs. Support for importing a PyTorch model therefore does not automatically mean support for a PyTorch training loop. Rebellions developer documentation, SDK quick start
During inference, fixed weights transform inputs into predictions. Training adds a loss calculation, a backward pass that determines how each trainable weight affects that loss, and an optimizer update. Those operations require additional intermediate values and state. Repeating them for thousands of steps also exposes numerical problems that may never appear in a short forward-pass demonstration. PyTorch's automatic differentiation documentation describes the gradient calculation underlying this distinction; it does not document VIDRAFT's implementation. PyTorch autograd
A separate deployment account from Red Hat shows what the commercial ecosystem currently looks like. Its May guide connects ATOM devices to OpenShift AI and serves models through a certified vLLM container. This is useful evidence of an actual inference deployment path, written by a team outside VIDRAFT. Red Hat is a Rebellions partner, however, and the guide neither reproduces nor validates VKORE training. Red Hat's deployment guide
VIDRAFT's additional claim is that it built a full training path on this hardware. It reports that early long runs diverged after initially working, and that stabilizing those runs was a major engineering task. The final NPU checkpoint reached step 76,292 after about four days. The method and training loop remain private while a patent application is in progress. An ATOM owner cannot currently reproduce that run simply by following the public vendor quick start.
A controlled model comparison, with unequal device counts
The experiment compares two versions of the same model rather than unrelated checkpoints that happen to have similar parameter counts. According to the report, both started with identical random weights and used the same training corpus, batch size, optimizer and learning-rate schedule. The order of randomly sampled text windows differed between runs, so their input sequences were not identical at every step.
| Property | NPU run | GPU reference |
|---|---|---|
| Hardware | 32 inference NPU chips | One B200 GPU |
| Model | 325.6M parameters, 20 layers, matched initial weights | |
| Corpus | 6.94 GB of UTF-8 bytes from FineWeb-Edu sample/10BT shards 000 and 001 | |
| Training budget | About 5 billion byte tokens; 65,536 byte tokens per step | |
| Final checkpoint | Step 76,292 | Step 76,293 |
| Matched speed, energy and cost | Not published | |
The NPU run used 32 chips for about four days; no matching B200 elapsed time was published. Similar final loss therefore leaves unanswered how much training each setup can complete in the same period. Device counts alone cannot fill that gap because compute, memory and communication differ between the systems.
For a buyer, those missing measurements affect the decision directly. A training job that reaches the target quality at a different time changes how many experiments a team can run. Server power, hardware pricing and access to the required software then determine the operating cost. The public evidence answers a model-quality question, but it cannot yet support a GPU replacement budget.
The corrected evaluation is stronger than the original one
The model card originally described 256 fixed windows from the end of the training file as held out. Training actually sampled random windows across that whole file. Those evaluation bytes could therefore have been seen during training. They could measure performance on that corpus, but they were not a clean test on unseen material.
VIDRAFT corrected the description and added an evaluation on shard 013, which was not included in the training file. Both checkpoints were tested on the same 1,024 windows. The public script measures next-byte cross entropy over 256 predictions per window: a lower value means the model assigned more probability to the correct next bytes. Revised model card
| Metric | NPU-trained | B200-trained |
|---|---|---|
| Unseen-shard loss, 1,024 windows | 0.7993 | 0.7994 |
| Unseen-shard bits per byte | 1.1531 | 1.1533 |
| Training-corpus loss, 256 windows | 0.8067 | 0.8070 |
The paired mean difference was -0.0001, with a standard error of 0.0007, and the NPU checkpoint had lower loss on 50.1% of windows. These numbers support a narrow reading: the two checkpoints performed almost identically on this test. They do not establish that the NPU produces a better model. The standard error comes from differences across evaluation windows, not from repeating training with many random seeds; there was one training seed per hardware setup.
Byte tokens and short context define the result's reach
VKORE uses a vocabulary of 256 byte values. Its 256-token context therefore means 256 byte tokens, rather than 256 words or the subword tokens used by many deployed language models. Basic ASCII characters use one byte each, while many characters in other writing systems occupy several UTF-8 bytes. A token count alone cannot express the same amount of language across these representations. Model configuration
This design makes the hardware comparison easier to interpret because both models consume the same bytes without a separately trained tokenizer. It also means the five-billion-token training budget should not be directly compared with another model's subword-token budget. Likewise, strong next-byte prediction over these short windows does not demonstrate long-document summarization, extended conversations or retrieval across a large context.
The files use the standard LlamaForCausalLM implementation supported by Transformers. That describes a loadable model format; it does not mean the experiment started with Meta's pretrained Llama weights. VKORE is a base model trained from random initialization, without an instruction-tuning stage that would make it a general assistant. Its own published generation samples contain factual errors. A team choosing a chatbot should evaluate the capabilities it needs rather than treating this hardware experiment as an assistant benchmark.
What the release lets another team reproduce
The repository places the NPU checkpoint at its root and the GPU checkpoint in the b200_reference subfolder. The evaluate_unseen.py script downloads shard 013, samples windows with seed 2026 and computes both sets of losses and their paired differences. It selects CUDA when available and CPU otherwise. The evaluation therefore does not require access to an ATOM device. Evaluation script
That makes the behavior of the published weights inspectable. It does not reconstruct the training operation that produced them. Running the script would test checkpoint loss; it would not establish which hardware executed each backward operation or reveal the private stabilization technique.
The immediately available work is to reevaluate the two checkpoints on the same text. Training a new model on ATOM would additionally require the private training software and its support arrangements. For teams hoping to reuse existing inference hardware, access to that software remains the practical condition separating this result from a training deployment.