linuxhansl

Yes... And, perhaps the political comments here are a reasonable reaction to the misguided political influence that shaped grok.

How can we trust grok after this? At least it will take a while.

I need a model that can answer questions in an unbiased way and do what it is told. If I need a specific political opinion I can find that myself - thank you very much.

Not much one can do I fear...

Install f-droid and all kinds of 3rd part apps now.

Install GrapheneOS. (I'm guilty of not having that done that,yet :( )

Sign the petition (https://keepandroidopen.org/).

What Google is doing is shameful. One of the promises of Android was being more open than the restrictive Apple ecosystem.

Now that they reached penetration they do the switch - under the guise of security.

Just let me do with my hardware what I want to do it. Let it be my responsibility to install whatever I want (and stop calling it "side-loading", as if I am doing something shady from the "side").

We need to resist this! Alas, from the broader response it seems that most people just do not care.

How qwen3.6:27b compare to qwen3.6:35b-a3b (MoE) in your experience (if you tried). I find the dense models are way too slow on my H/W.

I soooo wish that to be true. Alas, in my experience it is not... Yet.

What is true is that it gets easier and faster to run local models. With QAT (quantization aware training), turboquant (or similar) K/V compression; what used to be impossible to run is now fairly easy.

I can run gemma4:26b-a4b-qat on my laptop with 20-30 tokens/s with a 256k context window. That was unthinkable just 6 months ago.

So the local models are "OK" for small'ish projects.

But it does not at all(!) compare to the frontier models. For a large project Claude's Opus 4.6+ just work, whereas local gemma tangles itself up, makes weird mistakes, and just can't handle it (for those cases it is faster if I do it myself).

If the trends continues, with 1.58bit QAT models, even better K/V compression, faster multi-token prediction et al, maybe soon it will be comparable.

This is really awesome. I tried my test-task (generate a Python wrapper for a fairly complex C-interface) with gemma4:26b-a4b-it-qat. And for the first time it would just do it, without prodding and without errors. (Of course Claude just did it a heartbeat, too.)

My optimal local setup now is gemmma4-qat and Q8_0 K/V cache quantization with 256k context windows. And that runs fine with 12GB VRAM and another 10GB in RAM.

Previously I tried with gemma4:26b-a4b-it-q4_K_M and qwen3.6:35b-a3b-q4_K_M, and they both would tie themselves into knots (especially qwen3.6 can take forever with incessant "but wait..." thinking loops.) More often than not, they would not finish the task.

It seems true these 4b QAT models are as precise as Q8_0 quantization (which is supposedly indistinguishable from bf16).

I am really excited about the prospect of local LLM inference.

Thank god some sanity prevails.

Adding these to the index immediately would force passive index funds (multiple trillions of $) to buy this stock, and thus not allow the market to make performance based decisions.

It's truly a shame that the NASDAQ caved and I will definitely reduce my position in such index funds (I have less trust in it now).

I use Claude every day. Often for multiple hours a day. Basically doing my job not worrying how many tokens I spend (as in too many or too few). This is a pretty complex code base (database optimizer and related).

Just looked at spent for the past 30 day, didn't even come to $600. 95% of my tokens are from cache. If I were to reach even $1500 I have to let claude run unsupervised over night (and with the amount of mistakes it still makes and guidance it needs, I do not believe we are there yet.)

I am using Opus 4.x at work, and these "smaller" (20-80bn, 3-4bn active) models at home. Unfortunately there is no comparison, yet (IMHO anyway).

With Opus I can work, trust its designs, architecture suggestions, and code changes, even in a complex code base.

The smaller models seem to "try". They work for smaller tasks, but for more complex task it's often more work than doing it myself.

I wish it were different, and maybe in a year or two it will be.

Please, not the SAT!

My son is prepping for the SAT and I am helping him. I studied physics and computer science, and was a advanced math A+ student...

IMHO: The SAT is useless, solving equations under artificial time constraints is something that only happens in these kind of tests. The focus is on solving problems fast and getting a good score, and nobody really cares if you understand the math behind it.

So, please, if you go back to testing, find something more useful than the SAT.

Good.

Just naming things differently does not work in other countries.

If it quacks like a duck, swims like a duck, and looks like a duck, then it probably is a duck.

Other countries paying $10,000's to educate people who then want to apply this knowledge in the US. US reaction: "Nah." Besides, we are talking about legal immigration here.

I don't get it.

Are we purposefully dumbing down the country now?

Sure, some things are trade-secrets or national security issues or whatnot, but those are already not shared.

More than militarily the US has always led with soft-power (science, culture, etc). We are throwing all this soft-power away.

Perhaps there is a legitimate reason, but like so many things in this administration this feels like a knee-jerk reaction to... Something.

Wrong the way it would be wrong to predict that if you set your kitchen on fire, the result will be a renovation.

This might be favorite metaphor ever, and one I'll quoting in the future! :)

I think the author conflates social media with other inventions like a portable GPS device, an electronic map, a music player, or indeed a cell phone.

As far as social media goes the author is (IMHO) spot on. You do not have to look far to see how that is at least harming democracy around the globe. For democracy to flourish you need reflective voters who can entertain multiple viewpoints and make informed decisions. That is what social media - in its most common current form - discourages and rather optimizes for attention-time (which is money).

And of course (some) anonymity paired with global reach would not bring out the best in people. Anger and flames spread faster than conciliatory messages and get you more dopamine posting those.

Just my $0.02.

I guess it all depends on what you use it for.

I work on database optimizers and other database related stuff, and I can assure Claude Code - with all the highest settings - does make mistakes. It will generate a test that does not actually test what it "thinks" it tests. It will confidently break stuff.

Do not get me wrong. It is still awesome! It takes much of grunt work off me. It can game out designs decisions even when that needs to refactor a lot of code. If you point out a mistake more often than not it can fix it itself.

It's just for a critical project I would never ship it without understanding every line of code - with the exception perhaps of some of the test code. Maybe in a year or two that will be different.

The size of the KV cache (context stored) is proportional to the number of layers of the model and number of "hidden dimensions". For a 400B model it could be 30-60GB for just an 8K context window (depends on the model, etc, just a ballpark).

So shrinking that by 6x (from fp16), would be big win for larger models. True, while TurboQuant can also be applied to model weights, it won't save size over q4 compression, but will have better accuracy.

Edits: Better context

I am fascinated by this and similar research (RotorQuant, etc). It seem by next year we will be able to run this year's largest models on last year's hardware. :)

Maybe we won't need as many data centers and as much power as we thought. Maybe we can run more powerful models locally.

Hmm... Sure, if you do not need a database then do not use a database.

Don't use a sports-car to haul furniture or a garbage truck as an ambulance. For the use case and scale mentioned in the article it's obvious not to use a database.

Am I missing something? I guess many people are the using the tools they are familiar with and rarely question whether they are really applicable. Is that the message?

I think a more interesting question is whether you will need a single source of truth. If you don't you can scale on many small data sets without a database.

I will say this before I shut up with my rant: If you start with a design that scales you will have an easier to scale when it is time without re-engineering your stack. Whether you think you will need to scale depends on your projected growth and the nature of your problem (do you need a single source of truth, etc.)

Edits: Spelling

So Cal.com favors security through obscurity.

Open Source was always open to "many eyes" in theory exposing itself to zero-day vulnerabilities. But the "many eyes" go for the good and the bad actors.

As far as I am concerned... Way to go Cal.com, and a good reminder to never use your services.

Good. Now leave TikTok and Facebook as well. People who care will find out what you are up to, and people who don't won't see you on social media anyway.

I left Twitter, Facebook, et al about a decade ago. And I can assure you: You will never miss any important development.

The notion that we need to plugged into Twitter, X, whatever, to stay up to date is simply false.

How much many more wars over gas or oil do we need to finally just take the energy that (for the most part) is available locally and renewable?!

The petrol era is coming to an end. Our current administration might desperately want to remain a petrol state (for reasons that escape me), but it will only delay the inevitable. The EU is not much better either. The writing has been on the wall, and even since the Russian invasion into Ukraine not much has happened.

What is going on? Are we all insane, or is it just intense lobbying of yesterday's petrol industry?

That's not entirely unreasonable. As long as there is a way to enable this in perpetuity for my device(s) and it works for all Android devices it's a compromise I could live with.

Again, can we, please, stop call it side-loading. I'm not sliding in anything "from the side" on the sly, I am simply installing an app of my choice on my damn phone.

Hats off to Anthropic for not wavering here.

Supply-chain risks means "the potential for adversaries to sabotage, subvert, or disrupt the integrity and delivery of defense systems, including software, hardware, and services, to degrade national security".

So now Anthropic is an adversary, because it does not want "fully autonomous weapons" or automated mass surveillance? Sure thing, DoD. Go use Grok or whatever, I'm sure that will go great.

Fry's was mentioned in Douglas Coupland's "Microserfs". I loved that book and thoughts Fry's was cool even before I set foot in one.

Years later I entered a store, and somehow it was already nostalgic then :)

So weird, I haven't thought Fry's for the past 20 years.

There was something about wandering the aisles and seeing the merchandise and getting ideas

That. Exactly.

Congressional Republicans let him do it.

Congressional Republicans confirmed every single one of his cabinet members.

So perhaps he demands it, but does not give congress a pass. At all. They chose to let him.

Android sideloading should be the same

In fact we should not even call it "sideloading", as if we are sneaking anything in "from the side". It is simply installing something I like on a device that I own.

My device can warn me about security consequences and let me be the one who decides what to do (with my device).

+1000

I donated a few $100's to the petition.

With 23,623 (as of today) signatures I doubt anybody really cares, and we'd all rather be cheeple doing the tech companies' bidding as long as we can flop on our couches and consume.

Clearly Google wants to make money off their monopoly (created in part from initial openness) and they are disguising it as some security/safety enhancement bullsh*t. Shameful!

My main question: I chose Android over Apple because of the extra freedoms it affords me. When that goes away, what reason do I have continuing with Android?

In this case it was distributed without our data centers (10k's machines or so at that time spread around the planet), but the code to generate ids was 100% under our control. Rather than inventing some distributed generation (or collision detection), a stateless approach with random numbers just seemed the right choice.

Finally some sanity. The administration has use laws about "national security" and other so call "emergencies" to impose tariffs. If everything is an emergency then nothing is, and that was clearly not congress' intention with those laws.

The power to impose tariffs rests with the legislator, not the executive. Of course our congress is effectively useless - we can thank decades of Mitch McConnell's (and others) "not giving the other side anything" thinking for that.

Heh. I once had to make an argument that 256 bit randomly assigned identifiers are good enough without explicit collision checking. People wanted me to add complex and expensive collision checks.

My argument was the 2^256 actually approaches the number of atom in the observable universe (within 1 to 3 orders of magnitude), and that collisions are so unlikely that we'll have millions of datacenter meltdowns first (all assuming we have a good source of random numbers, of course). In the end I convinced everybody that even 128 bits are good enough, without any collision checking required.

I thought my arguments was clever, but this is so much better. :)