xlayn

Let's make the whole working/colaborating environment a training input for the model, we have to replace each and every link of human everywhere, think about the stock prices... yes it will have cathastropic consequences to succeed but then think again when you vest in 40 years you will be rich!

and then the first reply is someone saying that if not someone else will... and then another saying that the whole idea of business is to make money... and then another that the guy from valve is a jerk...

get your eyesight check, at some point in life I was having difficulties staying focus and even started having headaches... and I needed prescription glasses...

Also very important, try to sleep the best you can and do exercise... I remember reading here about exercise in some cases exceeding the benefits of ADHD medications...

If you are overdosing things... don't... caffeine can be overdosed and the effects are nasty and lingering... caffeine is meant to be a zero sum game, where it gets you speed in the morning and when it wears off then you go to sleep... if you over dose it then you are making your body over exert and at some point the body will communicate the results via extreme tiredness, bad humor, etc...

Don't be hard on yourself, from everyone we are the most hard on ourselves when we should be the most kind of them all.

I would like to argue that trying to provide a free service is non achievable, most of the time it will drill down to ads, people are already paying electricity and time in ads. If we pay say 3 secs of compute time of monero, and everyone pay the same... you remove the ads from the internet, people will start gettind paid without gate keepers for content they generate, and you can charge the AI machine for ingesting your content.

X402... I was not aware, I had this idea of making HTTP connection depend on a monero transaction, the monero transaction should take around 3 secs of the average computer/cellphone... once you have paid that you can access the resource. You wanna crawl the whole internet non stop, you pay non stop, 3 secs is probably the same as we pay in ads for those without adblockers and then content generators can start getting paid for the resources they generate.

because we need quadratic energy increase to increase speed linearly, that's why a 200hp car is not twice as fast as a 100hp one. Gee let me elaborate a bit more... if the 100hp car has a top speed of 100mph, a 200hp car will have maybe 130mph of top speed (assuming all other things the same) because you are fighting friction of an inmense amount of things that want to stop that movement. Anecdote... this famous US plane made with this titanium allow, I remember reading that at the speeds it was able to reach, the pressure of the air hitting the surfaces is so much that it will cause other metals to fail. Imagine the amount of energy you have to spend to heat the whole surface of the plane while traveling through cool air!

I want to rush to git clone, but as things are, the odds are extremely high that this kind of things that are too good to be real are honeypots and something there will compromise your machine or make your llm start working for someone else...

Another perspective, if you compare it to two years ago, how much more expensive is it and how much better? we are paying the sAIm Taxltman. Just see, you could buy the steam deck for 250 refurbished 2 years ago, now it's what 700$? Try to buy 2 64GB dims of ram.

I created a patch for llama.cpp to store on disk instead of deleting the kv cache as well as the checkpoints... there is this bug on llama.cpp if you have more than one instance going on of chats... and that causes the kv cache to be lost between changes of chat... And I can tell you, using Qwen3.627B after one day of use you can have 120-200Gb of chats on disk. And yes it's way way faster, even if you get it from a spinning disk it's still faster than re-computing the whole thing...

I guess for a 300B parameter or more and couple million users with the price of storage increasing as part of ramagedon this is also not viable...

I hope someone at jetbrains with enough power read what we are saying in this thread.... changes for the sake of changes are bad... when they did the change to the "new UI" I kind of stomach it.. because the option to go to the other one was there... but I don't know... how much benefit did everyone derive from it? But when the AI crazyness started it was f downhill... Jetbrains need to do a one thing... expose a connector so the ai can connect to the thing and do what it needs to do... so the IDE amplifies the model... go to definition... give me back all the errors... without having to play with bash, make the thing go fast... and please don't put the stuff we want to do behind a gated thing that only you can give us and charge us for...

I stopped paying and stayed in 2024.2, every once in a while I see if there is anything meaningful worth it there... nop

Ohhhh geee!!! I just applied the patch to my local git copy. You need to use the model on the PR that he submitted, the model is particular because it has extra information that allows the MTP to happen. I have two amd gpus, and qwen3.6 27B qk6 does around 20t/s generation... If I run it only on one I get like 35t/s.

But with this patch I saw 46t/s with qwen3.6 27B q8... this is insane, it's 250% faster than the original speed, there was no gpu I could upgrade to get that kind of boost, amazing!

If anthropic is doing this as a result of "optimizations" they need to stop doing that and raise the price. The other thing, there should be a way to test a model and validate that the model is answering exactly the same each time. I have experienced twice... when a new model is going to come out... the quality of the top dog one starts going down... and bam.. the new model is so good.... like the previous one 3 months ago.

The other thing, when anthropic turns on lazy claude... (I want to coin here the term Claudez for the version of claude that's lazy.. Claude zzZZzz = Claudez) that thing is terrible... you ask the model for something... and it's like... oh yes, that will probably depend on memory bandwith... do you want me to search that?...

YES... DO IT... FRICKING MACHINE..

I had a 6950 on my pc from when I built the thing... and then bought the 7900 for $5xx, that allows me to run more models, and then I saw the "Radeon AI PRO" and after a couple of frustrating talks with certain LLM to try to get an idea on what the speed of the card is I decided to go, buy it and test it to check what's the actual speed.

I updated the results, with just the Devstral part, but ran the full suite for it, and posted all the results file as well as a script to re-run the process.

The results are more spectacular...

The model pointed way better in gsm8k, but lost a bit on the other categories.

Fair point on the writing style, I used Claude extensively on this project, including drafting. The experiments and ideas are mine though.

On the prior art: you're right that layer duplication has been explored before. What I think is new here is the systematic sweep toolkit + validation on standard benchmarks (lm-eval BBH, GSM8K, MBPP) showing exactly which 3 layers matter for which model. The Devstral logical deduction result (0.22→0.76) was a surprise to me.

If there are ComfyUI nodes that do this for image models, I'd love links, the "cognitive modes" finding (different duplication patterns that leads to different capability profiles from the same weights) might be even more interesting for diffusion models.

You can check here the results for Devstral, speed limits me, but these are the results for the first 50 tests of the command

  # Run lm-evaluation-harness
  lm_eval --model local-chat-completions \
      --model_args model=test,base_url=http://localhost:8089/v1/chat/completions,num_concurrent=1,max_retries=3,tokenized_requests=False \
      --tasks gsm8k_cot,ifeval,mbpp,bbh_cot_fewshot_logical_deduction_five_objects,mbpp \
      --apply_chat_template --limit 50 \
      --output_path ./eval_results

I explored that, again with Devstral, but the execution with 4 times the same circuit lead to less score on the tests.

I chat with the model to see if the thing was still working and seemed coherent to me, I didn't notice anything off.

I need to automate testing like that, where you pick the local maxima and then iterate over that picking layers to see if it's actually better, and then leave the thing running overnight

The other interesting point is that right now I'm copy pasting the layers, but a patch in llama.cpp can make the same model now behave better by a fact of simply following a different "flow" without needing more vram...

if this is validated enough it can eventually lead to ship some kind of "mix" architecture with layers executed to fit some "vibe?"

Devstral was the first one I tried and optimize for math/eq, but that din't result in any better model, then I added the reason part, and that resulted in "better" model

I used the devstral with the vibe.cli and it look sharp to me, thing didn't fail, I also used the chat to "vibe" check it and look ok to me.

The other thing is that I pick a particular circuit and that was "good" but I don't know if it was a local maxima, I think I ran just like 10 sets of the "fast test harness" and pick the config that gave the most score... once I have that I use that model and run it against the llm_eval limited to only 50 tests... again for sake of speed, I didn't want to wait a week to discover the config was bad

I published the results for devstral... results folder of the github https://github.com/alainnothere/llm-circuit-finder/tree/main...

I'm using the following configuration --tasks gsm8k_cot,ifeval,mbpp,bbh_cot_fewshot_logical_deduction_five_objects,mbpp I did also try humaneval but something in the harness is missing and failed...

notice that I'm running 50 tests for each task, mostly because of time limitation as it takes like two hours to validate the run for the base model and the modified one.

I'll also try to publish the results of the small tests harness when I'm testing the multiple layers configurations, for reference this is phi-4-Q6_K.gguf, still running, I'm now giving more importance to the Reason factor, the reason factor comes from running a small subset of all the problems in the task config above

Initially I tried the approach of the highest math/eq but in resulted in models that were less capable overall with the exception of math, and math like in the original research is basically how good was the model at giving you the answer of a really though question, say the cubic root of some really large number... but that didn't translate to the model being better at other tasks...

  Config  | Lyr | Math   | EQ    | Reas   | Math Δ  | EQ Δ  | Reas Δ  | Comb Δ
  --------|-----|--------|-------|--------|---------|-------|---------|-------
  BASE    |   0 | 0.7405 | 94.49 | 94.12% |     --- |   --- |     --- |    ---
  (6,9)   |   3 | 0.7806 | 95.70 | 94.12% | +0.0401 | +1.21 |  +0.00% |  +1.21
  (9,12)  |   3 | 0.7247 | 95.04 | 94.12% | -0.0158 | +0.55 |  +0.00% |  +0.55
  (12,15) |   3 | 0.7258 | 94.14 | 88.24% | -0.0147 | -0.35 |  -5.88% |  -6.23
  (15,18) |   3 | 0.7493 | 95.74 | 88.24% | +0.0088 | +1.25 |  -5.88% |  -4.63
  (18,21) |   3 | 0.7204 | 93.40 | 94.12% | -0.0201 | -1.09 |  +0.00% |  -1.09
  (21,24) |   3 | 0.7107 | 92.97 | 88.24% | -0.0298 | -1.52 |  -5.88% |  -7.41
  (24,27) |   3 | 0.6487 | 95.27 | 88.24% | -0.0918 | +0.78 |  -5.88% |  -5.10
  (27,30) |   3 | 0.7180 | 94.65 | 88.24% | -0.0225 | +0.16 |  -5.88% |  -5.73
  (30,33) |   3 | 0.7139 | 94.02 | 94.12% | -0.0266 | -0.47 |  +0.00% |  -0.47
  (33,36) |   3 | 0.7104 | 94.53 | 94.12% | -0.0301 | +0.04 |  +0.00% |  +0.04
  (36,39) |   3 | 0.7017 | 94.69 | 94.12% | -0.0388 | +0.20 |  +0.00% |  +0.20
  (6,10)  |   4 | 0.8125 | 96.37 | 88.24% | +0.0720 | +1.88 |  -5.88% |  -4.01
  (9,13)  |   4 | 0.7598 | 95.08 | 94.12% | +0.0193 | +0.59 |  +0.00% |  +0.59
  (12,16) |   4 | 0.7482 | 93.71 | 88.24% | +0.0076 | -0.78 |  -5.88% |  -6.66
  (15,19) |   4 | 0.7617 | 95.16 | 82.35% | +0.0212 | +0.66 | -11.76% | -11.10
  (18,22) |   4 | 0.6902 | 92.27 | 88.24% | -0.0504 | -2.23 |  -5.88% |  -8.11
  (21,25) |   4 | 0.7288 | 94.10 | 88.24% | -0.0117 | -0.39 |  -5.88% |  -6.27
  (24,28) |   4 | 0.6823 | 94.57 | 88.24% | -0.0583 | +0.08 |  -5.88% |  -5.80
  (27,31) |   4 | 0.7224 | 94.41 | 82.35% | -0.0181 | -0.08 | -11.76% | -11.84
  (30,34) |   4 | 0.7070 | 94.73 | 94.12% | -0.0335 | +0.23 |  +0.00% |  +0.23
  (33,37) |   4 | 0.7009 | 94.38 |100.00% | -0.0396 | -0.12 |  +5.88% |  +5.77
  (36,40) |   4 | 0.7057 | 94.84 | 88.24% | -0.0348 | +0.35 |  -5.88% |  -5.53
  (6,11)  |   5 | 0.8168 | 95.62 |100.00% | +0.0762 | +1.13 |  +5.88% |  +7.02
  (9,14)  |   5 | 0.7245 | 95.23 | 88.24% | -0.0160 | +0.74 |  -5.88% |  -5.14
  (12,17) |   5 | 0.7825 | 94.88 | 88.24% | +0.0420 | +0.39 |  -5.88% |  -5.49
  (15,20) |   5 | 0.7832 | 95.86 | 88.24% | +0.0427 | +1.37 |  -5.88% |  -4.52
  (18,23) |   5 | 0.7208 | 92.42 | 88.24% | -0.0197 | -2.07 |  -5.88% |  -7.95
  (21,26) |   5 | 0.7055 | 92.89 | 88.24% | -0.0350 | -1.60 |  -5.88% |  -7.48
  (24,29) |   5 | 0.5825 | 95.04 | 94.12% | -0.1580 | +0.55 |  +0.00% |  +0.55
  (27,32) |   5 | 0.7088 | 94.18 | 88.24% | -0.0317 | -0.31 |  -5.88% |  -6.19
  (30,35) |   5 | 0.6787 | 94.69 | 88.24% | -0.0618 | +0.20 |  -5.88% |  -5.69
  (33,38) |   5 | 0.6650 | 94.96 | 88.24% | -0.0755 | +0.47 |  -5.88% |  -5.41
  (6,12)  |   6 | 0.7692 | 95.39 | 94.12% | +0.0287 | +0.90 |  +0.00% |  +0.90
  (9,15)  |   6 | 0.7405 | 94.65 | 94.12% | -0.0000 | +0.16 |  +0.00% |  +0.16
  (12,18) |   6 | 0.7582 | 94.57 | 88.24% | +0.0177 | +0.08 |  -5.88% |  -5.80
  (15,21) |   6 | 0.7828 | 93.52 | 88.24% | +0.0423 | -0.98 |  -5.88% |  -6.86
  (18,24) |   6 | 0.7308 | 92.93 | 94.12% | -0.0097 | -1.56 |  +0.00% |  -1.56
  (21,27) |   6 | 0.6791 | 92.54 | 82.35% | -0.0615 | -1.95 | -11.76% | -13.72

There is a performance improvement as per [0][1] the memory speed went up from 5500MT/s to 6400.

[0] https://www.steamdeck.com/en/tech [1] https://www.steamdeck.com/en/tech/deck

Sennheiser PC31, I'm not sure if the speakers on those are shared with the PX headphones, which sound by the way very nice for music.

https://www.amazon.com/Sennheiser-31-II-Binaural-Headset-Mic...

and in case your laptop/desk doesn't have mic and headphone jack you can use

https://www.amazon.com/Sabrent-External-Adapter-Windows-AU-M...

that works with Linux, not sure about windows/mac.

for the cellphone plantronics voyager legend, which is expensive but works every time very well.

HP Mac Mini. This is actually interesting, when Apple switched to max 16 Gb of ram on their PRO line, HP throws something with PRO graphics and Xeon.

Nice catch, there are two parts in trying to support my theory, first included in my post:

As we get older more and more energy would be used on preservation (read it as fix damage and less efficient process as result of age), therefore shrinking/eliminating everything not being used it's necessary.

The second is an entry on how our bodies are machines oriented to try to avoid wasting energy... or better said preserving it... for that part my canonical reference would be the Algernon argument:

http://www.gwern.net/Drug%20heuristics

To me seems logical: As we get older more and more energy would be used on preservation (read it as fix damage and less efficient process as result of age), therefore shrinking/eliminating everything not being used it's necessary.

Edit: TL;DR: From the PDF conslussions:

it is found that working hours up to 25–30 hours per week have a positive impact on cognition for males depending on the measure and up to 22–27 hours for females. After that, working hours have a negative impact on cognitive functioning.

If you have to pull an army of cores to fight two the really impressive arch belongs to Apple.

OTOH I see this as the optimization of hardware to fix the cap that software, paradigms and devs mean.

Another fab should mean options and with that price decrease. I wonder how much a given chip price can be decreased with this move.

Another way of reading this is that making more use of a given tech should pay the initial cost faster thus making possible more research and improved processes.

Sadly another way of reading this is that Intel doesn't have any tech that would capitalize in better chips anymore.

Yes, I gave a read to it but if I remember I think there was a piece that didn't perform up to the speed of the thunderbolt link making the whole solution run 1/4 speed.

The solution approach I think goes as back as when using the express card port on elitebooks

https://www.youtube.com/watch?v=IDiizICogMQ

It would be incredible on the other hand to get this solutions as official products from vendors like nvidia or asus without the DIY (not because I'm against it but to push making use of this solutions officially will improve the state of the solutions).

Another good reason for dropping the M is in my opinion all related to how good another company got at creating gpus... that's Intel.

Intel first eliminated the whole aftermarket entry level gpu industry and will probably eliminate the middle tier also.

As Pluma states

"If you're going with a dedicated graphics card in your laptop, battery life is already out of the window, so you might as well get as much processing power as the thing can handle"

I had to disable every security measure just to get to a "we need flash" notice.

Anyone with a youtube link?

Thanks for the reply; "Prolog-style solving of logical expressions in JS" sounds like a better title.

Why is such complication introduced and what are the benefits of it?

      firstVar || (secondVar=='Bob' && thirdVar==41)

On the other hand JavaScript has always had logic operators... therefore has always been able to perform logic programming.....

From http://www.ecmascript.org/es4/spec/overview.pdf page 18

  “&&=” and ”||=” operators
     ES4 introduces assignment operators for the logical   
     “&&” and “||” operators. These assignment operators
     are short-circuiting; if the value of the left-hand- 
     side determines the value of the result then the right-     
     handside is not evaluated.