radq

I'm disappointed with the commentary here. "GPU bubble" is an industry standard term, and literally how I would describe this to my colleagues in the industry. Look for example at the second slide here https://media.steampowered.com/apps/valve/2015/Alex_Vlachos_...

This is what people in the field call it. I'm sorry you're offended.

Appreciate you saying the blog was nice. Not sure what you mean by "CODEX fingerprints", but I'll engage with the other points. We work on small models, and our customers want real-time inference on modern GPUs. The sub-title says "near-realtime VLM inference". 20-30ms forward passes are a non-starter for these workloads.

If you scroll down to the section titled "A cost model for the bubble", you will find both benchmark results and us saying, "you get back anywhere from a few percent to a third; more the faster your accelerator/model is".

Thank you for the kind words. We will write and share more of these.

The 'point' skill is trained on a ton of UI data; we've heard of a lot of people using it in combination with a bigger driver model for UI automation. We are also planning on post-training it to work end-to-end for this in an agentic setting before the final release -- this was one of the main reasons we increased the model's context length.

Re: chart understanding, there are a lot of different types of charts out there but it does fairly well! We posted benchmarks for ChartQA in the blog but it's on par with GPT5* and slightly better than Gemini 2.5 Flash.

* To be fair to GPT5, it's going to work well on many more types of charts/graphs than Moondream. To be fair to Moondream, GPT5 isn't really well suited to deploy in a lot of vision AI applications due to cost/latency.

Thanks! If you could shoot me a note at vik@m87.ai with any examples of the precision/recall issues you saw I'd appreciate it a ton.

Cool project! The codebase is simple and well documented, a good starting point for anyone interested in how to implement a high-performance inference engine. The prefix sharing is very relevant for anyone running batch inference to generate RL rollouts.

Hello folks, I work on moondream. Posted a demo video on twitter for this release: https://x.com/vikhyatk/status/1864727630093934818

Happy to answer any questions!

Not true, H100s cost $2-3/GPU/hr on the open market.

Have you considered sponsoring an open-source project? ;)

1/3rd "activated parameters", while also requiring 2x the VRAM.

The training technique used here (fitting something similar to a NeRF to different views of the same image) is pretty similar to this paper which uses a similar technique to denoise (instead of upscale) output features: https://arxiv.org/abs/2401.02957

I'm confused - you posted in the "who wants to be hired" thread, and then got an email from this company asking if you'd be interested?

Do outlier features emerge in sub-100M parameter models? I haven't seen any research discuss it below the 124M scale (bert-base). At that scale training a model takes ~4 days on an 8xA100 node.

The plugin is supposed to ask for confirmation, according to OpenAI's documentation at least.

When a user asks a relevant question, the model may choose to invoke an API call from your plugin if it seems relevant; for POST requests, we require that developers build a user confirmation flow to avoid destruction actions.

https://platform.openai.com/docs/plugins/introduction

Literally happened to me, though it was during COVID lockdown so it's possible things have gotten better.

Looks like you're right, there was a recommendation to increase it but I don't see anything about it taking effect.

I don't expect a lot of people to take this up. Express Entry was already an option for these folks and you get permanent residency from the start under that program.

As someone that's used to live in Canada and is currently in the USA, this is not true. Six month wait time to get an x-ray for a hairline fracture, people dying because of long wait times for cancer screenings etc. In the USA things are expensive but you at least get access when you need it.

You can switch jobs on an H1-B, and if you get fired/laid off you currently get 180 days to find a new job (though that used to be 60 days previously).

More like the engineers and scientists built it, but legal and PR think it's a bad idea to release. Which they're probably right about considering the reception Galactica got.

Andrej Karpathy's "zero to hero" series [1] was how I learned the fundamentals of this stuff. It's especially useful because he explains the why and provides intuitive explanations instead of just talking about the what and how. Would recommend it if you haven't checked it out already.

[1] https://www.youtube.com/watch?v=VMj-3S1tku0&list=PLAqhIrjkxb...

Yeah, that's pretty close. It might be more precise to say they trained one big model that includes 8 "expert networks" and a mechanism to route between them, since everything is trained together.

There isn't a lot of public interpretability work on mixture-of-expert transformer models, but I'd suspect the way they specialize in tasks is going to be pretty alien to us. I would be surprised if we find that one of the expert networks is used for math, another for programming, another for poetry etc. It's more likely we'll see a lot of overlap between the networks going off of Anthropic's work on superposition [1], but who really knows?

[1] https://transformer-circuits.pub/2022/toy_model/index.html

If it's similar to the switch transformer architecture [1], which I suspect it is, then the models are all trained on the same corpus and the routing model learns automatically which experts to route to.

It's orthogonal to beam search - the benefit of the architecture is that it allows sparse inference.

[1] https://arxiv.org/pdf/2101.03961.pdf

Looks like Adobe makes about 26% of their revenue from EMEA, so it's clearly worth it for them to continue operating there and deal with the regulatory overhead. But is it possible for them to have a different corporate structure that mitigates some of these challenges? For example, could they license their technology to an EU-based distributor instead of having direct operations in the EU?

I have some reservations about the specification - it looks like there are a lot of caveats to how it can be used. For example, it's allowed in for and for-of loops, but not in for-in loops. Also looks like destructuring will not be allowed.

    using res = getResource(); 
    const { x, y } = res; // ok
    
    using { x, y } = getResource(); // error

We were missing two architecture patterns that were needed to get deeper nets to converge: residual nets [1] which solved gradient propagation, and batch normalization [2] which solved initialization.

[1] Residual nets (2015): https://arxiv.org/abs/1512.03385

[2] Batch normalization (2015): https://arxiv.org/abs/1502.03167

I have a similar setup and decided to use Tailscale. It's been working seamlessly so far.

As someone that rides a bicycle around human-driven vehicles, I will take my chances with autonomous cars. The autonomous car isn't going to drive drunk, use a phone, or drive erratically because the driver is feeling frustrated.

I've been working on training models for this problem. Haven't shipped as a product yet because the inference cost is too high (takes 10-15 minutes to transcribe each song). Would love to try it out on the music you're trying to learn and get your feedback. Email address is in my profile.