By Dr. Jonathan Tapson, Chief Development Officer

Edge AI needs more than faster compute. Memory traffic and repeated multiplication still strain the power budget.

As models move to 1–4 bit weights, the arithmetic gets cheaper — but each weight-activation pair still triggers its own multiplication, and low-bit weight storage does not automatically eliminate per-element multiplication.

We’ve filed a patent application for a different approach that uses add before multiply.

Conventional inference decodes every weight and multiplies it by its activation, one pair at a time. Our method turns that sequence around:

• The weight code itself acts as an address offset that selects one of a small set of accumulators — 2^k of them for k-bit codes.
• Each activation is added into the bin its weight code points to.
• At the end, each bin’s weight code is decoded, the resulting weight is multiplied by that bin’s accumulated activation sum, and the bin results are added together.

A length-n dot product with k-bit weight codes takes n + 2^k additions and at most 2^k multiplications, down from n. In the paper’s basic formulation, a 1,000-term dot product with 2-bit weight codes requires at most four multiplications rather than 1,000. Just as important: the high-precision decoded weight vector is never created at all.

BrainChip’s business is doing more AI within a tight power and memory budget. This method attacks both sides of that budget:

• Fewer multiplication operations create an opportunity to share hardware and reduce switching activity — area savings depending on the implementation.
• Avoiding the decoded weight vector cuts local weight memory by about 7× for 16-bit decoded weights represented by 2-bit codes.
• It fits where edge models are heading: low-bit quantization, with integer or floating-point activations, and with arbitrary — including nonlinear — weight decoders.
• The proposed scatter-accumulate processing element is designed for a weight-stationary systolic-array architecture, replacing its conventional multiply-accumulate unit.

We’d rather tell you exactly where the wins land than hand you a headline number.

The additions remain: the cost model in the paper predicts a smaller, bounded improvement in arithmetic energy than the multiplication-count reduction suggests. And in the reported FPGA synthesis comparison against a matched MAC array, an unshared scatter-accumulate processing element was larger. Savings appeared when the reduction multiplier was shared across processing elements.

Large models — LLMs included — are being pushed to the edge, and the efficient low-bit inference stack is one place to address power constraints. We have filed a patent application seeking protection for the proposed method and the processing-element architecture, to strengthen BrainChip’s position in that stack and the IP story we bring to licensing partners and customers alongside our existing neuromorphic work.

Software tests confirmed exact integer equivalence and found floating-point error no worse than the conventional pipeline in the tested cases, and we have sized the design in RTL. Next up: ASIC-level area and power numbers, and end-to-end accuracy on real quantized models. We’re excited to see where it goes.

Low-bit inference needs more than smaller weights. It needs a data path that avoids unnecessary work. Add-before-multiply offers a path worth testing — with its benefits measured at the implementation level.

A full description of the method, “Decode Late, Multiply Once: Rethinking the Dot Product for Quantized Weights,” is available at arXiv:2610.31913.

If you’re building low-bit inference hardware and want to talk licensing, or if you’d like to follow this work as ASIC results land, get in touch.

Stay tuned to the next in the series: Akida in High-Frequency Trading: The Smart Signal Layer posting next Wednesday.