One Akida Chip Is Efficient. Eight Are Something Else.

What We Learned Scaling Edge AI Across a Multi-Chip Akida System

Multi-chip edge AI: eight AKD1500 M.2 chips on one PCIe switch and one host deliver 14.7x throughput on a model too big for one chip, 99% scaling efficiency when the model fits, 15.7 million I/Q samples per second for wide-band RF, and zero accuracy lost to model splitting.

By Amir Naderi, Machine Learning Engineer

Edge AI has always come with a trade-off: run a model small enough to fit on one low-power chip, or run a bigger model somewhere that isn’t low power. We’ve been testing a third option. Use more chips.

We put eight Akida 1.0 (AKD1500) M.2 cards in a single Intel Core i7 mini-PC and tried several ways of spreading work across them. Here’s what we found.

Big models gain more than the chip count

AkidaNet is BrainChip’s image-classification network built for Akida. Version 1.0 is the full-width network and 0.5 is the half-width one. “@224” means it takes 224×224-pixel images.

AkidaNet 1.0 @224 needs about 3.7 chips’ worth of resources. On a single chip, it has to keep reloading layers, and it reached 7.7 frames per second (FPS).

We split it into stages across eight chips and added extra copies of the slowest stages. The results:

  • Throughput: 7.7 FPS to 113 FPS, a 14.7x gain from eight chips
  • Latency: 131 ms down to 84 ms

The gain beats 8x because each chip loads its stage once and stays busy, instead of swapping layers in and out. A planning tool picks the split automatically, and its speed predictions came within about 2% of what we measured.

Small models scale almost linearly

AkidaNet 0.5 @224 went from 41 FPS on one chip to 326 FPS on eight, about 99% scaling efficiency. That’s a simple way to serve several camera streams from one box.

Throughput speed-up from eight Akida 1.0 chips versus one chip: 8.0x for AkidaNet 0.5 @224 (41 to 326 FPS) and 14.7x for AkidaNet 1.0 @224 (7.7 to 113 FPS), against 8x for linear scaling.

Throughput speed-up from eight Akida 1.0 chips versus one chip: 8.0x for AkidaNet 0.5 @224 (41 to 326 FPS) and 14.7x for AkidaNet 1.0 @224 (7.7 to 113 FPS), against 8x for linear scaling.

Wide-band RF scales too

For RF, each chip can take the frames from its own slice of the band, so the bandwidth covered grows with the number of cards.

We ran a modulation classifier on the same mini-PC, with all eight AKD1500 M.2 cards behind a PEX8749 PCIe switch. Each chip ran the same 4-bit SWaP-AMC network independently on its own stream of 1,024-sample frames, with no data passing between chips. SWaP-AMC is automatic modulation classification built for low size, weight, and power.

At batch 64:

  • Per chip: about 1,920 frames per second
  • All eight chips: 15,356 frames per second, about 15.7 million I/Q samples per second
  • Accuracy: every chip’s predictions matched a CPU reference, run on the mini-PC’s own processor, frame for frame

When we split 2,600 recorded frames into eight slices, one per chip, the reassembled result was identical to a single-chip pass.

The host is the real design choice

The hard lessons came from the host, not the chips.

  • One container per chip: the Akida runtime lets only one process open the chips and runs them one at a time within that process. Reaching the full rate took one container per chip.
  • One CPU core per chip: the runtime keeps a core busy for each chip, so the host’s processor-platform power rose from 12 W idle to about 60 W.
  • Where the energy goes: that’s about 3.9 mJ per frame at the host, against an estimated 0.14 mJ on the chips themselves.

The chips scale. The host is what you have to choose carefully for a low-power multi-chip system, which is why we recommend microcontroller- or FPGA-class hosts rather than a PC.

Splitting a model doesn’t change its output

We cut the network at all 14 possible split points, and the results were identical every time. Splitting adds speed without costing any accuracy.

Other things we tried

  • Four models at once: four different models on four chips, about 1,200 inferences per second in total
  • High-resolution tiling: on a 672×672 scene built from a 3×3 mosaic of photos, tiling across chips found 86% of the objects, versus 8% when the scene was shrunk to the model’s 224×224 input. We counted an object as found when its class appeared in the model’s top-5 predictions, judged against the model’s own answer on each photo alone. With 10 distinct photos, this shows a direction rather than a benchmark.
  • Cross-generation detection: a CenterNet detector designed for Akida 2.0 runs on Akida 1.0 hardware. It’s feasible, but not yet accurate enough to deploy.

The bottom line

More chips don’t just add capacity. For models too big for one chip, they remove the reloading penalty and deliver more than linear gains. For models that fit, they scale almost perfectly. Splitting costs nothing in accuracy. And the biggest power lever in a multi-chip system is the host you pair it with.

Tell us what you’d run on multiple chips

Building multi-camera, high-resolution, or wide-band RF systems at the edge? Tell us what workloads you’d want to try.

Get the hardware: order AKD1500 M.2 cards from the BrainChip Shop.

Start building: find the tools, models, and documentation on the BrainChip Developer Hub.