[watcher] 14:15:14 等待下载完成... [watcher] 14:15:14 PTQ1_0=6% (358MB) mmproj=42% (255MB) [watcher] 14:16:14 PTQ1_0=9% (555MB) mmproj=76% (456MB) [watcher] 14:17:14 PTQ1_0=13% (756MB) mmproj=100% (600MB) [watcher] 14:18:14 PTQ1_0=16% (954MB) mmproj=100% (600MB) [watcher] 14:19:14 PTQ1_0=20% (1167MB) mmproj=100% (600MB) [watcher] 14:20:14 PTQ1_0=24% (1362MB) mmproj=100% (600MB) [watcher] 14:21:14 PTQ1_0=27% (1566MB) mmproj=100% (600MB) [watcher] 14:22:14 PTQ1_0=30% (1712MB) mmproj=100% (600MB) [watcher] 14:23:14 PTQ1_0=33% (1875MB) mmproj=100% (600MB) [watcher] 14:24:14 PTQ1_0=36% (2067MB) mmproj=100% (600MB) [watcher] 14:25:14 PTQ1_0=39% (2260MB) mmproj=100% (600MB) [watcher] 14:26:14 PTQ1_0=41% (2340MB) mmproj=100% (600MB) [watcher] 14:27:14 PTQ1_0=41% (2340MB) mmproj=100% (600MB) [watcher] 14:28:14 PTQ1_0=41% (2340MB) mmproj=100% (600MB) [watcher] 14:29:14 PTQ1_0=41% (2340MB) mmproj=100% (600MB) [watcher] 14:30:14 PTQ1_0=41% (2340MB) mmproj=100% (600MB) [watcher] 14:31:14 PTQ1_0=41% (2340MB) mmproj=100% (600MB) [watcher] 14:32:14 PTQ1_0=41% (2340MB) mmproj=100% (600MB) [watcher] 14:33:14 PTQ1_0=41% (2340MB) mmproj=100% (600MB) [watcher] 14:34:14 PTQ1_0=41% (2340MB) mmproj=100% (600MB) [watcher] 14:35:14 PTQ1_0=41% (2340MB) mmproj=100% (600MB) [watcher] 14:36:14 PTQ1_0=41% (2340MB) mmproj=100% (600MB) [watcher] 14:37:14 PTQ1_0=41% (2340MB) mmproj=100% (600MB) [watcher] 14:38:14 PTQ1_0=41% (2340MB) mmproj=100% (600MB) [watcher] 14:39:14 PTQ1_0=41% (2340MB) mmproj=100% (600MB) [watcher] 14:40:14 PTQ1_0=41% (2340MB) mmproj=100% (600MB) [watcher] 14:41:14 PTQ1_0=41% (2340MB) mmproj=100% (600MB) [watcher] 14:42:14 PTQ1_0=41% (2340MB) mmproj=100% (600MB) [watcher] 14:43:14 PTQ1_0=100% (5671MB) mmproj=100% (600MB) [watcher] ✅ 两个文件字节数完全匹配，开始基准测试 ================ PTQ1_0 ================ --- llama-bench: tg128 / pp512 / pp4096 --- ggml_cuda_init: found 1 CUDA devices (Total VRAM: 15866 MiB): Device 0: NVIDIA GeForce RTX 5060 Ti, compute capability 12.0, VMM: yes, VRAM: 15866 MiB | model | size | params | backend | ngl | fa | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | --: | --------------: | -------------------: | | qwen35 27B PTQ1_0 - 1.75 bpw ternary (group 128) | 5.53 GiB | 26.90 B | CUDA | 99 | 1 | pp512 | 464.19 ± 5.24 | | qwen35 27B PTQ1_0 - 1.75 bpw ternary (group 128) | 5.53 GiB | 26.90 B | CUDA | 99 | 1 | pp4096 | 464.16 ± 0.01 | | qwen35 27B PTQ1_0 - 1.75 bpw ternary (group 128) | 5.53 GiB | 26.90 B | CUDA | 99 | 1 | tg128 | 45.12 ± 0.10 | build: 9a9394a89 (10709) --- 启动 llama-server: ctx=262144, KV=q4_0, fa=on, port=8081 --- server pid=30513 服务就绪（等了 4s） --- 加载后显存 --- 12658 MiB, 16311 MiB --- 发送一个长需求以扩充 KV，然后测显存峰值 --- OK completion_tokens= 51 prompt_tokens= 20025 timings= {"prompt_ms": 45016.354, "prompt_per_second": 444.8383358634508, "predicted_per_second": 36.24202131900662} --- 长请求后显存峰值 --- 12668 MiB, 16311 MiB --- /props --- {'n_ctx': None, 'total_slots': 1, 'model_path': '/home/zyw/bonsai2/models/PTQ1_0.gguf'} --- 质量对比提示词（端口 8081）--- [OK] A-中文创作 6.6s completion=258 43.7 tok/s [OK] B-文档改写 2.2s completion=73 43.6 tok/s [OK] C-数学推理 18.4s completion=782 43.6 tok/s [OK] D-指令跟随 0.8s completion=18 42.7 tok/s [OK] E-数学推理-带思考 9.9s completion=409 43.7 tok/s --- 关闭服务 --- ================ PTQ1_0 完成 ================ [watcher] 14:45:37 === PTQ1_0 全部完成 ===