> Second, besides noise (bars are Wilson 95% confidence intervals, very conservative for run-to-run noise), there is little difference down to 4-bit; only the 2-bit scores a bit lower.
the 2nd derivate of shannon covariance of noise begs to differ
jnwatson 30 minutes ago [-]
Mind blown. The more I read about statistics, the less I know.
purpleflame1257 51 minutes ago [-]
There's a real hole here at Q3. A critical breakpoint here is sub 16-GB cards, which covers the 5080, 5070 Ti, 5060ti, and several other cards from this generation and the last. It would be instructive to see where the quality knee is.
civvv 22 minutes ago [-]
Running Q3 on my AMD RX 9070XT. 32k context and 32/TPS. Apart from the context window preventing it from doing any large tasks, this thing is seriously powerful. I could probably push it to 64k context. Local open models are the future, and I am definitely getting a more powerful card. Very fun!
slim 11 minutes ago [-]
Running Q3 on 5060ti with 64k context. It runs great
jadbox 28 minutes ago [-]
Q3 XL and Q3 XS are the two I'm trying to decide on
Farmadupe 7 minutes ago [-]
hmm, assuming that this article is part written by claude and part human-written, can anyone help me find a rule of thumb for "how to know if the article is worth reading"?
Because on the one hand, the prose and the presentation is painful (narrating irrelevant points, nonlinear X-axes, ambiguous chart labels, etc etc),
But on the other hand, the result that I'm assuming the author means ("on these evals, generation quality seems fairly good") is worthwhile?
Because I really struggle with this question at the moment. Am I allowed to draw an adverse inference that "if the writeup presents irrelevant text side by side with the data, then this may be a sign that the author does not understand the task that they are attempting to write up"?
clircle 3 minutes ago [-]
[delayed]
36 minutes ago [-]
48 minutes ago [-]
dvh 14 minutes ago [-]
Could this be used to estimate how many fingers LLM have?
quietraster 25 minutes ago [-]
the 4-bit matching bf16 on terminal-bench is a useful data
bellowsgulch 46 minutes ago [-]
Qwen3.8 27B seems like it was clearly supposed to be a high-end consumer open-weights model, but the t/s is so low for me on my old M1 Max 64GB that I hope others are getting use out of it.
Unfortunately, the calculus has changed and it seems cheaper to me to just use MiMo V2.5 for pennies or DeepSeek V4 Flash instead of using Qwen anymore unless I need a local model specifically for doing reverse engineering work that gets otherwise rejected.
> Qwen3.8 27B seems like it was clearly supposed to be a high-end consumer open-weights model, but the t/s is so low for me on my old M1 Max 64GB that I hope others are getting use out of it.
Have you tried it with MTPLX? I get around 30 tok/s with it, also on an M1 Max with 64GB.
SwellJoe 24 minutes ago [-]
Even at 30 t/s, 3.8 thinks so long, even on medium, it still takes 3x or more longer than any cloud model, in my testing.
Xeoncross 32 minutes ago [-]
Nice, which model quantization is this? Is it on huggingface?
Xeoncross 33 minutes ago [-]
I leave it running at night. No danger of burning my token subscriptions and it has hours and hours to run slowly with a manager like: github.com/kunchenguid/gnhf
ThrowawayTestr 10 minutes ago [-]
I treat it like image gen. Send a prompt then come back in 40 minutes.
zrail 13 minutes ago [-]
[dead]
InvectusXIV 27 minutes ago [-]
[flagged]
Rendered at 16:32:54 GMT+0000 (UTC) with Wasmer Edge.
Confidence intervals have nothing to do with run-to-run variation. They have little to do with anything people usually ascribe to them (https://link.springer.com/article/10.3758/s13423-015-0947-8 ), but even less with run-to-run variation (https://link.springer.com/article/10.1007/s10654-016-0149-3 misconception 22).
Because on the one hand, the prose and the presentation is painful (narrating irrelevant points, nonlinear X-axes, ambiguous chart labels, etc etc),
But on the other hand, the result that I'm assuming the author means ("on these evals, generation quality seems fairly good") is worthwhile?
Because I really struggle with this question at the moment. Am I allowed to draw an adverse inference that "if the writeup presents irrelevant text side by side with the data, then this may be a sign that the author does not understand the task that they are attempting to write up"?
Unfortunately, the calculus has changed and it seems cheaper to me to just use MiMo V2.5 for pennies or DeepSeek V4 Flash instead of using Qwen anymore unless I need a local model specifically for doing reverse engineering work that gets otherwise rejected.
Have you tried it with MTPLX? I get around 30 tok/s with it, also on an M1 Max with 64GB.