How long do you wait for the first token, and how fast do the next ones arrive? Rockchip’s RKNN-LLM v1.3.1 benchmark lists both measurements for Spark-X2.5 1.7B.

Board Quantization First token Generation Memory
RK3576 W4A16 768.66 ms 13.57 tokens/s 1,028.58 MB
RK3588 W8A8 386.23 ms 15.61 tokens/s 1,830.32 MB

Rockchip benchmark measurements

Rockchip used 128 input tokens and requested 64 new tokens, with the CPU and NPU running at their respective maximum frequencies. The two rows use different quantization formats. For a board you plan to deploy, start with its listed format and test your own prompt length. Record time to first token separately from generation speed.

Rockchip’s September 29 benchmark table