Compact computers can now run surprisingly large language models locally. But fitting a model into memory is not the same as running it quickly, and advertised NPU TOPS are a particularly poor substitute for measured LLM throughput.

This comparison focuses on single-user local inference and coding …