HN user

ycui7

57 karma
Posts1
Comments30
View on HN

discounted competitor could cheat. they can offer subpar model response and sell it as deepseek-v4. it is uneconomical to prove inference providers are cheating, so they get away with it. cheating inference provider does not care if their customers stay.

Qwen 3.8 3 days ago

it takes extra effort to open source a model even if you had it running internally. even traditional software takes extra effort to get released as open source.

Qwen 3.8 3 days ago

every cloud provider trains on your data, regardless of what they promise. real user interaction is the best reinforcement-learning trace.

24-bit was created because microphone want to record large dynamic range without gain switching circuit.

96kHz was created to better reproduce 20kHz high frequency, so the digital noise shaping filter does not need to be super sharp right at the Nyquist frequency.

Both were introduced for a sound technical reason. beyond that, most are marketing non-sense to cheat consumers.

This is not surprising at all. The biggest benefit of cloud model in terms of energy efficiency is that when running more than 1 requests, the power consumption of said GPU roughly stayed the same. The more concurrency requests the server can handle, the less power each request consume. The server GPU is already likely more energy efficient than local GPU, concurrency make the cost structure unbeatable by local hardware. It is generally assumed the local hardware only run 1 request, but if the local engine is meant to serve a small business with meaningful concurrency, the economy might still work out.

Every vendor defines their audio jack connector serial port differently. It is very dangerous to use 3.5mm jack. There is no pinout standard of using 3.5mm.

Even as pure audio jack, the 3.5mm connector has two standards, with the difference on ground and mic.

B70 idles at 30W, while RTX PRO 4500 idles at 9W (measured to be 5W at wall).

B70 runs at 1/3 token output rate of RTX PRO 4500 and consume 3X idle power when do nothing.

When you get 4 of these, the idle power alone is 120W. That is a lot of electricity if left on 24/7.

At that power consumption, you also end up being more expensive than API calls and many times slower. It starts to feel very stupid to run local interference.

If the client is very keen on privacy, then they can pay for the NVIDIA.

I end up returning my B70s, and bought RTX PRO 6000.

You can get 120TPS (144 peak) with Qwen3.6-27B on RTX PRO 6000 with autoround when MTP enabled. It runs faster than sonnet api calls.

5090 gets maybe 100TPS with MTP

Problem is the more B70 you have, the slower the inference it gets(due to terrible software atm). A single B70 is almost barely faster than CPU inference. If you have 4 B70, you might as well run interference on CPU and be faster with cheaper DDR5 instead of GDDR6.

Intel Arc B70 when released, can only produce 1/3 of the token of RTX PRO 4500. Well, it also cost 1/3 of RTX PRO 4500.

It lacked software support the for the primary target application, running LLM. The officially supported vllm fork is 6 version behind mainline. It did not run the latest hot new open models on huggingface. Parallel two of B70 reduce token rate, not improve it. So, the software behind B70 is basically so far behind.

It is weird that the reviewer does not mention RTX PRO 6000 96GB, but mentioned RTX PRO 5000 72GB. 72GB RTX PRO 5000 is a special order, and much less people are aware of it. RTX PRO 6000 is known by mostly everyone in the LLM world.

I cannot understand why would a tech reviewer do that.