Nano*
ru

How to run the benchmark

  1. Install NanoLLM and get the default model (the installer fetches it, or nanollm models pull).

  2. Run the benchmark. The default is the quick set: auto and two representative presets; --full runs them all:

    nanollm bench
    nanollm bench --full

    Each preset runs in a process of its own: a warm-up, three repetitions, the median and the spread. A preset that does not fit the machine is marked "refused" — that is a result, not an error. Close heavy programs and plug the laptop in while it runs.

  3. Afterwards NanoLLM shows a summary and the exact JSON it would send, and asks. Nothing is sent without your "y". For scripts: --submit.

  4. Offline, the result stays in a file; send it later:

    nanollm bench submit bench/results/bench-DATE-standard.json
  5. Withdraw a submission (with the installation key that signed it):

    nanollm bench withdraw ID

    Anonymous mode (bench.anonymous: true) signs with a throwaway key: the submission cannot be withdrawn or linked to your others.

What is sent

Preset metrics, the NanoLLM and llama.cpp versions, the model hashes and a hardware profile: CPU, memory, GPU, computer maker and model, OS, power source, disk type. Never host or user names, serial numbers, UUIDs or MAC addresses; of the model's answers, only a hash. See the privacy policy.

How the numbers are made

Each group (model, suite, preset, system, CPU, GPU, OS) shows the median and p25–p75. Each installation and each machine (a coarse hardware fingerprint) counts once however many times it submits. Results of different suite versions never mix. Generation speed is checked against memory bandwidth for plausibility; outliers go to moderation.