bradfa

I can understand that the AI labs might care about other labs distilling their models as it can eat into their competitive advantage, but do consumers care at all? Aren't consumers benefiting from this practice by getting better cheaper models as a result?

There are plenty of 3rd party providers hosting deepseek models, if you don't want to use the 1st party API. 3rd party providers are generally slightly more expensive, but still quite cheap compared to other models of similar vintage and size.

Finally found it: https://dev.meta.ai/legal/

Paid prompts are not used for training. Data retention is not entirely clear, they will keep it if they think you're breaking the rules, but it's not clear how long they keep it if they don't think you're breaking the rules.

Where is the data retention policy information for paid API per-token uses? Every other provider has one and makes it clear how they handle your data. A quick look doesn’t show one for this new offering.

To get memory mapping across systems you'd want to look into infiniband (https://en.wikipedia.org/wiki/InfiniBand) but the adapters generally aren't cheap. Common to find in super computer clusters.

If you have a desktop CPU with 2 memory channels and 2 DIMMs per channel, then the 2 DIMMs on each single channel share all of the address and data lines, there's just a chip select difference to pick which DIMM you're addressing. There is some extra loading on the lines, since you have 2 DIMMs, hence the supported speeds are usually slightly lower, but you only add 1-2 more actual PCB traces to have 2 DIMMs per channel vs 1 DIMM per channel. In contrast, adding another memory controller would add upwards of almost 100 additional PCB traces from the CPU.

Each memory controller interface is a not-insignificant number of PCB traces. Increasing the number of memory controllers may dramatically increase the number of PCB layers (or may not, it really depends on the CPU pinout) but it definitely will increase the number of pins on the CPU socket.

This is one of the main reasons (the other is the number of PCIe lanes) why high end desktop and server CPUs have like double the number of pins and so much bigger sockets as compared to consumer desktop CPUs.

Many newer Chinese lab models are releasing with int4 native weights. Latest NVIDIA generation GPUs have a hard time with this and can actually be slower than previous generations. This may make Blackwell depreciate faster than other recent generations.

I’m also watching Intel Celestial with 160GB of LPDDR. Noticed lower memory throughput than AMD or NVIDIA, but potentially significantly lower cost per card. Two of them would likely run deepseek-v4-flash sized models pretty decently.

Trace antennas are very specifically designed shapes given the PCB material and other nearby components and structures. It’s never “just a trace”.

You’re right that traces can emit and receive RF, hence why things like FCC limits and testing exist even for unintentional emitters, but even failing devices which greatly exceed the limits have traces which are many orders of magnitude worse at being antennas than a typical trace antenna.

Right, but they're smart enough to know how to manipulate the market short term with such "research" and then take advantage of it, all without breaking the law.

With MoE models like Deepseek’s and with multiple Crescent Island accelerators, the aggregate memory throughput actually doesn’t look that bad. Two Crescent Island gets roughly 1400GB/s and Deepseek-v4-flash with 13B parameters active nets roughly 100t/s which is decent for a small team or great for a single user.

More Crescent Island scale up, although not likely entirely linearly.

But all GPU inference work like this, it’s not specific to Intel. Just Intel promises more affordable cards with big memory so they’re attractive.

How easy is it to swap the fan bar out for a failed fan? It looks like a single unit holding all the fans. Can the sled be pulled but retained in the rack and then fan bars removed and reinstalled without fully removing the sled and without tools?

Curious how robust the (what looks like PCIe edge connector slots) connection to the drives is in practice. Obviously converting from the horizontal mainboard to a vertical drive requires such a connection, making it a plug-in card at least allows for replacing the card if it breaks/wears/etc, and mounting the front of the adapter card to a bulkhead should prevent much shifting of the card in the slot. Neat design and reuse of a cheap high speed connector.

Are the Qualcomm Dragonfly chips not considered high end?

I wish you luck! The pain points you identified are definitely real and solving them would be valuable.

The workflow for user space can definitely improve some of this pain but I feel like a large portion of any embedded Linux development effort still ends up in the weeds for boot related items (secure boot, proper updates, nuanced kernel patches, bootloaders, device trees, and supporting machine variants, etc). Solving those to make them easy is a hard problem for sure.

Just be careful doing this if you have cats as pets.

Other comments also hint at this idea, a distributed training solution is currently an open research problem. Solving it is not easy, yet. But 10 years ago what we have today for LLMs would have looked similarly impossible, so have hope, and apply yourself to the problem if you find it interesting!

In states in the USA which perform emissions testing, many of them did not mandate it for diesel cars. For example, I owned a VW Jetta TDI and in New York (which has yearly emissions testing where an OBDII computer is mandated to be connected to gasoline powered cars in order to pass the yearly emissions inspection) and I was exempt from the emissions testing entirely.

A 3rd party sensor would be incredibly expensive for inspection stations to purchase as it would need to meter the air and fuel which enter the engine (assuming we aren't going to trust the car's computer which already knows these figures) as well as to measure the emissions out of the tail pipe. This is economically unrealistic to implement without a dramatic price increase in the cost of regular emissions testing.

Trusting the computer is the economical and realistically widely implementable solution. But yes, it has it's blind spots.

So the natural extension of this would be plugins which have curated open source allow-lists? Similar to how I trust uBlock Origin's default ad filtering block-lists, I would similarly trust a curated open source allow-list for email domains, and then I would add my own from the "to screen" folder?

No shoving sensors required, the data is all in the ECU accessible over OBDII interface. The car knows if it’s compliant in real time using the sensors it already has.

It was. We had sealed beam headlights for a while till we didn’t. There were common rules for aiming and it worked. The lights weren’t all that bright and the styling was not stellar, however.

They hired a good number of smart people who know how to do open source. So they’re trying. We shall see if it works.

Qualcomm is a “fool me once, shame on you, fool me twice, you don’t fool me twice” kind of situation. So many horrible experiences in the past that people are going to be hesitant.

Qualcomm are trying harder now it seems. But it will take time to repair their reputation in the PC market.

Xerox had (maybe still has?) some printers with 5 or 6 color toner capacity. CMYK plus you could order special color toner in stock or fully custom mixes (minimum order sizes apply for full custom) but it was great for companies who had logos which could not be exactly represented by CMYK half toning as the spot color toner could be their exact logo color.

I’m sure the same kind of thing would be possible using Prusa’s documented methods with a little extra work.

Q8 quant is very minimal fall off in terms of KLD against the lab 16 bit. If you have the memory for BF16 KV-cache (which is usually easier to stomach) then the Q8 is very close. But even Q8 quant model with Q8 KV-cache is very close.

Smaller quants for the model start to fall off but more importantly, smaller KV-cache quants fall off much faster so avoid less than Q8 there.

Normal secure boot does not use the TPM. Secure boot is the proactive process of ensuring only allowed code loads and executes.

The TPM is used for measured boot, the post process to understand what actually was booted and if the right set of things were booted then to allow unlocking of specific items like keys.

Both are important but they are not the same thing.

The article you link to explains how to defeat the sniffing with TPM 2.0. But also, there’s no reason a physical TPM has to be a separate IC package.

Would love to see actual security focused hardware/software features, like full OP-TEE, fTPM (or a more ideally a real physical TPM), and similar. For example, so that the OTP isn't the only way to store a disk encryption unlock key.

The existing secure boot mechanisms aren't bad, but allowing for more than one public key hash in OTP would be nice, too.

These kinds of things are expected to be on modern embedded SOCs and SOMs now.

It's a pretty normal thing to do for small LCD screens. Linux has had SPI framebuffer support via fbtft subsystem (in staging tree now, previously was out of tree) for well over a decade. It works quite well.

Many silicon vendors, when providing said binary blobs to a device OEM or even just documentation or source code for the binary blobs, will make companies agree to a license or other legal terms which prohibits reverse engineering. Often the direct recipient of the binary blobs (the OEM of the device) cannot legally let their employees nor contractors perform the reverse engineering.

Generally, unless a similar license or legal terms are required to be agreed to by the end user, nothing stops the end user from reversing said binary blobs. But before you attempt this, be sure you fully understand every legal document which was presented to you by the device vendor. Click-through EULAs included.