HN user
I presume this works (will work) also for JupyterLite that is based on Pyodide ? Would be great if it helped getting the latest OpenCV-python version [0] and it's dnn goodies being available on a zero-install client side Notebook !
OpenCV being in the list of Pyodide modules [0] was the biggest boon for my online teaching experience because remotely dealing with install woes (corporate proxies & cie) was a show stopper for regular Python. I'm hoping that they will package this new version and that it will bring the new neural networks engines goodies to the no-install crowd !
[0] https://pyodide.org/en/latest/usage/packages-in-pyodide.html
Nice ! My most pressing request for VSS would be efficient binary vectors : is this on the table ?
Only those who don't care/know about prompt processing speed are buying Macs for LLM inference.
For LLM inference, I don't think the PCIe bandwidth matters much and a GPU could improve greatly the prompt processing speed.
Indeed, recent Flash Attention is a pain point for non CUDA.
The idea is presumably that you would "sell" at an artificially low price.
The Soviets had the […]first woman,[…]
That is quite the claim !
You forgot the "/s", or do you actually believe that it's capitalism's fault is a mother taking care of her children is "unpaid labor" ?
"is hard" ≠ "sucks"
Most interesting ! Would you mind sharing the prompt and the resulting CLAUDE.md file ?
Thx !
IMO, it would be more interesting to have a 3-way comparison of price/performance between DeepSeek 671b running on :
1. M3 Ultra 512 2. AMD Epyc (which Gen ? AVX512 and DDR5 might make a difference in both performance and cost , Gen 4 or Gen 5 have 8 or 9 t/s https://github.com/ggml-org/llama.cpp/discussions/11733 ) 2. AMD Epyc + 4090 or 5090 running KTransformers (over 10 t/s decode ? https://github.com/kvcache-ai/ktransformers/blob/main/doc/en...)
DeepSeek is not a model.Which model did you use (v3 ? R1 ? a distillation ?) at which quantization ?
Nice ! Is it possible to connect to an in browser DB like WASM DuckDB https://duckdb.org/docs/api/wasm/overview.html or https://github.com/babycommando/entity-db ?
That would be most useful imho !
It seems that this aims to refutes claims for inaction with facts about spending money. However, the high speed rail project or homelessness management seem to show that in California, $$$ spent doesn't always imply that the problem is actually tackled in a meaningful way.
How do we know that this extension can be trusted ?
«Unfortunately, I have only seen 3 models, 3B or over, handle RAG.»
I would love to know which are these 3 models, especially if they can perform grounded RAG. If you have models (and their grounded RAG prompt formats) to share, I'm very interested !
Thx.
Just played a bit with it. Were you working with ASCII ? This example didn't work for you ? https://github.com/jfalcou/eve/blob/a141ba93048bb2916c2157a9...
Counterpoint : your message is not synthetic data and will contribute to lots of LLMs saying the same. Many such cases ?
(It seems to me obvious that a fgrep would sanitize synthetic data obtained from competitors.)
DeepSeek v3 can run on CPU & RAM :
https://www.reddit.com/r/LocalLLaMA/comments/1hqidbs/deepsee...
Epyc Gen4 and 12 memory channels of DDR5 @4800 should give you 7 to 9 t/s.
I don't think that Python would be the right language for such low-level performance maxxing endeavor. I would have picked C++ but t was eye opening for me to see how rust enabled such low level optimization, so I'm grateful for the choice.
Amazingly thorough ! I love how the author leaves no stone unturned. I had no idea you could do the kind of low level efficiency shaving in Rust. I wonder how a C++ implementation with https://github.com/jfalcou/eve would compare.
Most interesting ! Amazing job at optimizing various parts of the task. It seems that being an MoE with 'only' 37B active params per token would put it within the reach of CPU & RAM inference for the lucky hobbyist with an Epyc homelab and 8 or 16 memory channels on a second hand single or dual Gen2 mobo (around $2500 used). Any idea of how hard it would be (will?) support the new architecture for llama.cpp ?
I must confess that my interest in LLMs is grounded RAG as I consider any intrinsic knowledge of the LLL to be unreliable overfitting. Is DeepkSeek able to perform grounded RAG like Command R and Nous-Hermes 3 for instance ?
Thx for this amazing model and all the insights in your report !
For training, doesn't checkpoint saving make high reliability a moot point ? Why pay for 99.99999? uptime when you can restart your training from last/best model ?
About words embeddings, the №1 example is the famous King - Man + Women = Queen This works nicely in the vector space but fails to make a visual impression when projected on 2 dimensions. Neither with ACP, nor MDS ot t-SNE in my experience : https://bhugueney.gitlab.io/test-notebooks-org-publish/jupyt...
(← JupyterLite Notebook doing words embedding in the browser : don't try to run this on a smartphone !)
Does anyone know how to nicely visualize the poster child of words embeddings ?
I really wish there was a name (that I knew of) for this logical fallacy of using precise quantities of meaninglessly confused categories. What do you think "farmland" is ? Our ancestors who knew how to raise cattle and farm cereals for millennia were surviving on the verge of starvation : why do you think that was ? Were they so dumb they'd rather die of hunger than convert some of their grazing pastures to cereal agriculture ? Or some "farmland" can be used for grazing but not for agriculture, maybe ? Same for proteins : all amino acids are not equal, a a pound of soy protein won't have the same effect on your body as a pound of meat proteins. And don't get me started on nutrients and their bio availability (e.g. iron and B12). As far as CO₂ emissions from cattle, where do your think the C comes from ? The grass/food that cattle eat, what would happen to it if let to decompose ?
The dirty secret of education is that no school outperform nor underperform much other schools when adjusting for differences in student populations :
https://freddiedeboer.substack.com/p/why-selection-bias-is-t...
If I may hijack a bit this thread, I am currently looking for an optimization solver for a specific goal : I wish to constraint the values of variables so that tuples of variables are unique within sets of tuples.
Cf. https://stackoverflow.com/questions/71878354/constraint-prog...
For now, I could not find anything satisfactory, so any recommendation would be most welcome. Otherwise, I guess I'll have to generate a variable for each tuple and use and all_different (e.g. https://cpmpy.readthedocs.io/en/latest/api/expressions/globa... ) on those.
I'm sure most people would want that, but I don't think that we know how to "make people" change. Much in the same way that we don't know how to have school "make children" more academic, we don't know how to make inmates less violent / impulsive / thrill-seeking (except by the inexorable passing of time). I'd be very happy to be corrected, tho.
OTOH, giving proper (i.e. often involuntary) treatment for violent mental illnesses would for sure improve the situation.
I would have expected the Clojure versions of types to inherit from the Java versions. Any reason not to do that ?