nik736

Gemma 4 31B is underrated. It surprises me a lot.

I have put together an internal benchmark on 1000s of business documents with weird tables, structure, etc. that I run on every relevant model release. Opus 4.8 performs very very well. But it is obviously overkill for the task (and expensive at doing so). I just wanted to respond to the OP.

Opus is very good at OCR. Way better than the small 1-4B VLMs. If Opus failed, most likely those smaller models will fail as well.

Why would you need the 3rd run if you pick the "one in the middle"?

(I originally was going to say a computer that plays chess, but computers play chess with no intuition or instinct--they just search a gigantic solution space very quickly.)

Isn't that how LLM models are trained right now? Trying to predict the next word within a "gigantic solution space". Interesting.

In Germany we have several accounting software solutions like that for 5-10+ years that integrate with bank accounts, paypal, etc. - automatically suggests booking accounts and exports it via a REST API to the software accountants use. Accountants have access to it. Is this basically the same as your solution?

No lightmode?

Well, we have to "register" every new IP or new mail server with them as well. It's annoying and a weird system, but they respond quickly and it's just one todo we have to think about.

Meaning that the technology was there and ready to make an experience that was truly excellent

In general I would agree, but Siri is honestly still so bad.

What I am missing with Gnome is the global menu I have with macOS. It's just my preferred way of working. This is also what I liked about Unity. Gnome seems to follow the same direction as Windows.

Additionally miller columns in Finder are just awesome and I don't have them with Nautilus. Those two things are honestly dealbreakers for me.

Yes, but only because of data privacy concerns.

GitLab is very very heavy with a lot of bloat and sadly still a bad UI/UX. I prefer Gitea for its simplicity. Gitea Actions are similar to Github Actions and they work great.

Which models will this be able to run at an acceptable token/s rate?

The problem with ONCE is that software is never finished. This is why most ONCE software that is still available today is charging a one off licensing fee + update fee (e.g. charge yearly for major updates or 10% of the one off fee per year). This is sustainable, but your model isn't. You will notice down the road in 2-4 years that it's no fun to work for free for users that expect an update because it requires patching or there are breaking changes.

It's only on-die ECC not real ECC

It's an interesting article, thanks for that.

What people forget about the OVH or Hetzner comparison is that for those entry servers they are known for, think the Advance line with OVH or AX with Hetzner. Those boxes come with some drawbacks.

The OVH Advance line for example comes without ECC memory, in a server, that might host databases. It's a disaster waiting to happen. There is no option to add ECC memory with the Advance line, so you have to use Scale or High Grade servers, which are far from "affordable".

Hetzner per default comes with a single PSU, a single uplink. Yes, if nothing happens this is probably fine, but if you need a reliable private network or 10G this will cost extra.

The most annoying thing for me currently is that when connecting to local smb shares with Finder and adding favorites (directories on shares), after a reboot they are still there under favorites, but it won't connect to them when clicking on them. So I have to manually reconnect to the server every time. This wasn't happening before.

Other than that, of course some things got slower, but overall it's an OK release. For example he new system settings were bad before, now they got a bit worse, but macOS 26 didn't introduce those changes.

They changed their license to AGPL, removed features (Web UI, etc.) and now they don't provide docker images/binaries. It's their project but; what's next?

Is there a fork already?

Twilio seems to be affected as well

They limit them to 7500 IOPS, as stated in their docs. It also doesn't scale with size, the limit is there for every volume of any size.

Thanks for asking! The infrastructure is actually the less interesting part for us, since our platform is written to be completely portable. The USP is the platform itself, that means the managed aspect. You can run your workload on our platform without having to worry too much about servers or infrastructure. Deployment can be done either through Git Integrations or our 1 click deployment option for popular open source services. You can scale your application up and down in seconds and additionally we try to offer every important component as an all in one solution inhouse. Think managed dbs, transactional emails, etc., those are all products we offer today.

We actually started out in 2019 by colocating our own network/server equipment in a colo facility in Frankfurt (InterXion). We are bootstrapped and since our USP is the software, not the infrastructure, we decided to partner with a reliable infrastructure company that is available globally, flexible enough and for data privacy reasons is an EU based company (there are not many!) so we could roll out our platform faster. Since we still own our own IP space and are a RIPE member, we can migrate to our own infrastructure down the road.

In this business the focus shouldn't be to own the whole "supply chain", but to deliver a reliable solution to customers. Everyone is a reseller of someone.

Nodion -> Leaseweb. Leaseweb -> Iron Mountain. Iron Mountain -> electricity companies, dark fiber, etc.

Heroku/Vercel -> AWS. AWS also uses colo facilities like Equinix, InterXion, NTT or e-shelter in Frankfurt.

There is also https://www.hetzner.com/ (IaaS), https://www.leaseweb.com/en/ (IaaS) and https://www.nodion.com/en/ (PaaS).

Disclaimer: I am the founder of Nodion.

If you have enough memory to load a model, but not enough bandwidth to handle it, you will get a very low token/s output.

This is only the base model, no upgrades yet for the Pro/Max version. The memory bandwidth is 153GB/s which is not enough to run viable open source LLM models properly.

https://arstechnica.com/security/2025/06/meta-and-yandex-are...

Ingress is free with most providers. Yes, of course it adds a bit pressure on the cpu, but yours would add latency, might add peering issues, additional costs, etc. and I have no control.

For me it's not a big deal, my app autoscales and rate limit details are stored in Redis, so it's super fast. For x100 the traffic it wouldn't even need to scale since it's simply hitting Redis.

Rails 8 introduced built in rate limiting, smilar to what you described. Since it's built in already I have no use for your service, but good luck!

More friendly, less usable