Extropic opens the first transformer family built for a probabilistic chip: 294 nanojoules per token against the H100's 40.9 microjoules

AI hardware company Extropic has introduced Z1T, its first family of transformer-like models designed for its probabilistic chips, in a post dated 4 September. The headline claim: running on sparse probabilistic chips, these models show "over 100x energy efficiency gains vs GPUs", along with a corresponding scaling law.

Where the number comes from

The measurement the company gives is direct: pushing one token through a four block stack costs 294.52 nanojoules; the same work on an H100 costs 40.9 microjoules. The ratio is roughly 139x. Of those 294.52 nanojoules, 285.78 go to the FPGA and only 8.74 to Z1 sampling.

Z1 is Extropic's probabilistic sub-threshold CMOS chip. Each pbit has 16 tunable couplings, and that sparsity is a hard constraint of the hardware as well as the source of its efficiency. The model was therefore built around sparse neural network operations.

The limits the company states itself

The most notable thing about the post is that the caveats are written by the company:

• The work is described as a "preliminary investigation".
• The model used in the comparison is small relative to an H100; kernel launch times weigh on the measurement, and the model runs at 0.006% MFU.
• Extropic writes plainly: "If we were to batch on the GPU, H100s would be substantially more efficient."
• The advice follows from that: use the Z1 plus co-processor setup for decode; prefill is better done on GPU for now.
• The energy estimate does not include data movement between Z1 and the FPGA.

One more detail: reaching the same loss as a GPT-2 model with the sparser Z1T takes about an order of magnitude more FLOPs. Because the operations on Z1 gain about three orders of magnitude in efficiency over GPUs, the net result lands at two orders of magnitude.

A signal about the next step

Extropic notes that the FPGA consumes more than 95% of the energy. Their conclusion: if more of the operations move onto the sub-threshold CMOS substrate, efficiency up to 1000x over GPUs could be possible. That is not a measurement but, in their own words, a potential.

The weights are open

The company has opened the training recipe on GitHub for the sparse transformers behind its scaling law, and the weights of one of the larger training runs on Hugging Face. The post notes this is the company's first open source model.

Why it matters

AI's energy bill is now an infrastructure problem. What Extropic has done is rebuild the transformer not around the GPU's dense matrix multiply but around sparse, probabilistic hardware. The result is not a product yet but a preliminary study; still, adding connectivity as a new variable in the scaling law shows the path has become measurable.