HN user
Laziness and weak connections makes sense as differentiators.
However I'm not sure Excel is such a great illustration in that case, as it's neither lazy nor weakly connected; at least at the surface.
One thing I've never fully grokked is how this differs from an observable pattern where one can publish new values to inputs, propagate that through the computation, and push newly computed values to listeners.
I guess there's probably optimizations around change detection and stopping the propagation if there's no change (though observables can do that as well). The stabilize command also makes things interesting as a way to batch changes together before recomputing (but again, doable with observables too).
Is the delta primarily coming from introspection and automatically building the compute graph? Or is there something more fundamental that I'm missing?
I'm curious to hear what bottlenecks you encountered in the traditional path. Of all the compute and data shuffling involved in LLM inference, I would have thought shuffling the raw input/output around would have been a trivial part of the overall cost, and thus not a big optimization target?
As someone who uses gradle largely out of inertia, I'm curious what you would pick as a better alternative.
I've found swearing at a model to be quite effective in getting it to rethink and correct its mistakes. This seems to apply across Codex, Claude, Qwen, and Gemma/Gemini.
I don't know if the model is picking up on a "need to lock in and be more rigorous" signal, or if the model providers are routing to smarter models if they detect a frustrated user. But if a model keeps making the same mistakes, swearing at it often helped kick it out of a glut and onto the right track.
Or it could just be catharsis.
context with 2.1 bits of entropy per token
Can you elaborate on this? I'm seen estimates of ~1.5bit per English letter, and tokens encode a lot more than that - sometimes full words, with multimodal even more. If KV cache embedding are storing more than just simple tokens but entire concepts with context and nuance, that'll bump the entropy up quite quickly.
VPS comes at the cost of potential for oversubscription - even from more reputable vendors. You never really know if you're actually getting what you're paying for.
One annoyance (I don't know if they've since fixed it) was that Docker Hub would count pulls that don't contain an update towards the rate limit. That ultimately prompted me to switch to alternate repositories.
How well do we understand the tokenization for Claude? I'd posit that the exact human-representation of this markup is likely irrelevant if it's all being converted into a single token.
Code length will itself become a problem. The instruction cache is limited in size and often quite small. Bloating instruction counts with lots of duplicated code will eventually have a negative effect on performance.
Ultimately, there's too many factors to predetermine which approach is faster. Write clean code, and let a profiler guide optimizations when needed.
Additionally, so long as we can be sure the human's output is not actively adversarial, we can xor it into the entropy pool. Entropy can only increase this way.
I'm reminded of a diagram from the pitch doc for the original Diablo [0] that made its rounds across the web recently. The dungeon/town split was particularly sharp back then, but the broad design has stuck with modern ARPG design, either in the form of safe zones around town or explicit town zones.
A lot of this seems to be due to modern multiplayer design, with shared town instances and (usually) private dungeon/outside instances.
[0] https://www.gamedeveloper.com/business/here-s-a-look-at-the-... (scroll down)
Good catch; it was somewhat ambiguous in the report.
I see a lot of discussion in this thread stemming from some confusion+not reading the actual report[0].
Some key points:
1. The Camera+Card was encased in a separate enclosure made of titanium+sapphire, and did not seem to be exposed to extreme pressures.
2. The encryption was done via a variant of LUKS/dm-crypt, with the key stored on the NVRAM of a chip (Edited; not in TrustZone).
3. The recovery was done by transplanting the original chip onto a new working board. No manufacturer backdoors or other hidden mechanisms were used.
4. Interestingly, the camera vendor didn't seem to realize there was any encryption at all.
[0] https://data.ntsb.gov/Docket/Document/docBLOB?ID=18741602&Fi...
Getting on the public suffix list is easier said than done [1].
Can you elaborate on this? I didn't see anything in either link that would indicate unreasonable challenges. The PSL naturally has a a series of validation requirements, but I haven't heard of any undue shenanigans.
Is it great that such vital infrastructure is held together by a ragtag band of unpaid volunteers? No; but that's hardly unique in this space.
My 2c is that it is worthwhile to train on AI generated content that has obtained some level of human approval or interest, as a form of extended RLHF loop.
It reminds me of the early days of Typescript rollout, which similarly focused on a smooth on-boarding path for existing large projects.
More restrictive requirements (ie `noImplicitAny`) could be turned on one at a time before eventually flipping the `strict` switch to opt in to all the checks.
The way the question was framed, it was ambiguous whether "draw again" only applied to B, or whether A would draw again as well. I'm assuming the 'infinity' answer applies only to the former case?
IEEE754 is not great for pure maths, however, it is fine for real life.
Partially. It can be fine for pretty much any real-life use case. But many naive implementations of formulae involve some gnarly intermediates despite having fairly mundane inputs and outputs.
The entire frontpage was just 468KB, 91KB compressed, and most of that was the search index. Very neat!
You mean $2/customer? That doesn't actually sound that bad, and is in line with many other marketing efforts.
Yup. I was referring to the 1.58B quant which seemed to be performing alright and would be the smallest real-DeepSeek model. That requires ~140GB, which is just barely doable on a 128GB RAM + 24GB VRAM setup + a lot of patience. Others have made it work at 64GB RAM + a fast SSD.
The true minimally-quantized DeepSeek experience will need one or possibly two 8xH100 nodes, so well upwards of $100K in CapEx.
The only 32B distill I'm aware of is `DeepSeek-R1-Distill-Qwen-32B`, which would be a base model of `Qwen-32B` distilled (further trained) on outputs from the full R1 model.
Reminder: DeepSeek distilled models are better thought of as fine-tunes of Qwen/Llama using DeepSeek output, and are not the same as actual DeepSeek v3 or R1.
This unfortunate naming has sown plenty of confusion around DeepSeek's quality and resource requirements. Actual DeepSeek v3/R1 continues to require at least ~100GB of VRAM/Mem/SSD, and this does not change that.
On the other hand, 3B1B's audience tends to heavily bias towards the tech crowd. I'm at a FAANG, and a decent number of our senior engineers know the channel.
I agree it's not enough to directly push policy, but the impact is certainly larger than what the subscriber count might otherwise suggest.
Depending on how you count, the ratio might not be that small. A lot of hot code are written in hand-coded inline assembly, so in terms of CPU cycles run it's probably non-negligible.
i.e. take a look at the glibc implementation of 'strcmp` [0]
[0] https://github.com/bminor/glibc/blob/master/sysdeps/x86_64/m...
"What kind of hooker takes credit cards?"
"A rich one!"
A couple of cities have been doing this as well. Vancouver Canada has lidar point clouds covering the entire city, going back to 2013.
[0] https://opendata.vancouver.ca/explore/dataset/lidar-2022/inf...
Potentially dumb question: wouldn't the existence of a global magnetic field already imply the presence of a global electric field? Or is this discovery a separate unrelated electric field?
I've no plans for it myself, but that sounds like an interesting project! It should be fairly easy to hack together. The actual logic itself is just a few dozen lines of glue code around a single HTTP POST request.