HN user
zhisbug
live demo is here: https://fastwan.fastvideo.org/
Pokémon is increasingly used to evaluate modern large language models, but current practices lack standardization, and depend heavily on game-specific harness. The Pokémon Red involves three major tasks—navigation, combat control and training a competitive Pokémon team. We find they come with limitations: navigation tasks are too hard, combat control is too simple, and Pokémon team training is too expensive. We address these issues in Lmgame Bench, a new framework offering standardized evaluations and initial results across diverse games.
where other models tops out in a few moves
More details here: https://x.com/haoailab/status/1909712259326394519
We find that spatial perception and spatial reasoning remain very difficult even for the strongest models like o3 or Claude 3.7
gaming agent code here: https://github.com/lmgame-org/GamingAgent/tree/main
Sliding tile attention accelerates Hunyuan video generation by 3x with no quality drop and no need for training
Try our demo and let us know
This is pretty clever and seems to have high potential, but it still relies on humans. What if some day all humans cannot outsmart AI?
please try and give us feedback!
We hope to redefine ai evaluation via our gamified AI evaluation platform: game arena!
We study how to approximate the famous shortest-job-first scheduling in LLM inference!
Hugginface model and data link: https://huggingface.co/FastVideo
New work from the vLLM team that disaggregates prefill and decoding to maximize goodput (throughput subject to latency constraints) in LLM serving
New parallel decoding algorithm that trades flops for latency reduction
oops typo
yep, it is agonistic to 4-bit. You can deploy a 4-bit model and still use vllm + pagedattention to double or even triple your serving throughput.
This really depends on what GPUs you use. If you GPUs has very small amount of memory, vLLM will help more.
vLLM addresses the memory bottleneck for saving KV caches and hence increases the throughput.
could you provide any reference? Is it a variant of ELO?
Released by Lmsys.org who made Vicuna
And I think the problem of taking the roles of users in vicuna is caused by this bug: https://github.com/lm-sys/FastChat/commit/1bb234265d16bdfd50...
which has been fixed recently.
Lmsys are launching new training jobs after this patch, please stay tuned.
My point is that I am not aware of any official 4-bit quantization version (delta or weights) by lmsys so it might too early to draw your conclusion that vicuna finetuned llamas degenerates a lot of performance at 4 bit but others are fine.
it is really a matter of having faith on pytorch (or JAX) or on third-party cross-platform supports like llama-cpp. Apparently pytorch reduces a lot of complexity and grows extremely faster on cross-platform supports.
And, PyTorch does so well on GPUs!
no, the requirement on a particular HF commit has been fixed. It is no longer needed.
Lmsys hasn't released any official 4-bit version. It might be a better idea to wait for the official 4-bit version. But it is interesting to learn that the third-party 4bit version has performance degeneration.
No llama.cpp nor any compilation complexity. Run with two Python commands!