How to run the benchmark
Install NanoLLM and get the default model (the installer fetches it, or
nanollm models pull).Run the benchmark. The default is the quick set:
autoand two representative presets;--fullruns them all:nanollm bench nanollm bench --full
Each preset runs in a process of its own: a warm-up, three repetitions, the median and the spread. A preset that does not fit the machine is marked "refused" — that is a result, not an error. Close heavy programs and plug the laptop in while it runs.
Afterwards NanoLLM shows a summary and the exact JSON it would send, and asks. Nothing is sent without your "y". For scripts:
--submit.Offline, the result stays in a file; send it later:
nanollm bench submit bench/results/bench-DATE-standard.json
Withdraw a submission (with the installation key that signed it):
nanollm bench withdraw ID
Anonymous mode (
bench.anonymous: true) signs with a throwaway key: the submission cannot be withdrawn or linked to your others.
What is sent
Preset metrics, the NanoLLM and llama.cpp versions, the model hashes and a hardware profile: CPU, memory, GPU, computer maker and model, OS, power source, disk type. Never host or user names, serial numbers, UUIDs or MAC addresses; of the model's answers, only a hash. See the privacy policy.
How the numbers are made
Each group (model, suite, preset, system, CPU, GPU, OS) shows the median and p25–p75. Each installation and each machine (a coarse hardware fingerprint) counts once however many times it submits. Results of different suite versions never mix. Generation speed is checked against memory bandwidth for plausibility; outliers go to moderation.