maldie

llama.cpp to get 115 tok/s on RTX 4090 with Qwen3.6-27B. For example in Windows the latest CUDA variant llama-b9678-bin-win-cuda-13.3-x64.zip and Unsloth UD-Q4_K_XL MTP gguf:

llama-server.exe --host 0.0.0.0 --alias "Qwen3.6-27B-MTP" -m "F:\Qwen3.6-27B-UD-Q4_K_XL-MTP.gguf" -c 75000 -ngl 99 --metrics --temp 0.6 --top-p 0.95 --min-p 0.00 --top-k 20 --presence-penalty 0.0 --no-mmap -t 16 --spec-type draft-mtp --spec-draft-n-max 3 --reasoning on -fa on --parallel 1 -lv 4

Note that this does not use kv cache quants as in my case quants offload to CPU and tanks performance. Also keep in mind this almost maxes VRAM usage so any additional browsers or other programs that use VRAM should be closed. For chat go to http://localhost:8080/ and minimize the window to maximize perf as the web page UI draw itself consumes a lot of GPU perf via constant context switching.

Can try bigger than -c 75000 until perf gets lower than 100 tok/s - that means something is off as windows starts paging out memory or other issues. -c 50000 seems sweetspot if running browsers and stuff that consume 2GB VRAM. If wanting more than -c 140000 then likely need to use a bit smaller model quant.

CPU usage should be near zero, maybe 1 core load. If you see 8+ core load then settings are off and something is offloaded to CPU (for example kv cache). GPU load should be about 100%, meaning it utilizes work optimally in this case.

-t 16 can be omitted or set to the amount of physical cores, not important in this dense model that is 100% in GPU.

Can be pushed to 125 tok/s with that model if using --spec-draft-n-max 4 but VRAM usage also increases, so context needs to be smaller.

If speed is not important and want max context length then remove the draft-mtp parameters and also might need to use k and v quants like --cache-type-v q8_0, leave k f16 if possible to keep quality.

Yes, disassembling and tinkering with microwave is almost the only easily available thing I would not recommend at all because it seems so ordinary but actual danger is way too high and hidden. Even if you know exactly the dangers, it takes only one distracted move, just being tired or too complacent to end really badly. Like even some medium time disconnected and expectedly safe units can have real nasty shock in them when discharge resistor is faulty, that failure mode does not usually kill but not recommended.

For sure seems they are likely deliberately riding on the fame of the movie. I too instantly thought it is some kind of Flow movie animation collaboration similarily like Flow is represented in Blender 4.4 splash screen or is even their mascot.

Interesting what power do microwaves in US deliver. In EU the regular micros are about 900W range give or take 200W (that is 1500W from wall because microwaves are not efficient machines) but kettles have usually range 2200..3000. So it is obvious why everyone uses kettles. I myselft have 3kW one and it heats 0.35 litres to boiling in 51 seconds. To have similar microwave it would have to be 3000W micro that is 4500W from wall and have not seen that kind of monsters, also regular 1 phase EU socket delivers max 3600W so that limits it also.

This is complete insanity that vertical task bar is still not implemented, pretty much what OS has to do - manage memory, manage processes/threads, manage windows to enable smooth UX with multiple programs. It more and more fails on the last one. Seems the policies have changed to drop support for everything that is used for example less than 1% of users, (just neglecting 10 million users, costing likely billions $ time and sweat for those users, nbd). Even if it would cost MS like tow part time developers time/money.

It is such a neccessary multitasking functionality on work environments that I'm quite lost when Win 10 support ends and the alternative taskbar tools do not work reliably then. Small hope that Windows 12 or whatever comes next would be the better one.

Agreed, expectations are everything or at least very important to keep sanity in this area.

Reality is that big majority of startup-s do fail to "make it" 90% of time to even be self sustaining. Financially meaning that you pay from your pocket overall and the resources have to come somewhere, day job or whatever, making it very much overwhelming and can break even a strong man. Only small percentage out of those 10% actually make it to sustainable enough to live by.

Percentages can begin to rise on your favour after learning from failures but there is of course no guarantees, some domains are inherently much less forgiving or have fierce competition. It is important to resarch and understand the landscape you're going into.

If you realise your odds then mindset starts to get clearer, you know what you risk losing and accept that loss is likely, you still put enough effort to win but don't set your mind so deep that everything collapses with the project.

It might be contrary to some suggestions that you have to put everything into startup to have a chance but that is also correct imo, you do have to be able to invest all your available time on the project. Keeping in mind that the project is not absolute everything in life and being able to walk away at right time as to not sunk endless resources to unfruitful endeavour keep you healthier.