yeldarb

See the url in my comment (search for the term rfdetr-2xlarge). 2XL does indeed go up to 800x800 and has PML1.0 license instead of apache 2.0.

All of the models, including the Apache 2.0 ones, can be configured to go higher than 800x800. The difference between the ones with the PML license and the Apache 2.0 ones is the backbone, not the resolution.

I'd suggest you read the ICLR paper[1] which shows clearly the difference between the backbones at various latencies in Figure 1.

For many domain-specific (often less common and odd dimensioned) objects, downscaling will severely reduce recall.

We released an entire paper[2] at Neurips about the long-tail transferability of models across a multitude of domains and benchmarked RF-DETR against that benchmark. The Apache 2.0 model is pareto optimal over the larger PML model at latencies less than the XL size.

(I'm one of the co-founders of Roboflow and worked on RF-DETR and RF100-VL.)

[1] https://arxiv.org/abs/2511.09554 [2] https://arxiv.org/abs/2505.20612

That may be true for legacy CNNs but very few production use-cases require such a large resolution with DETRs. The latency scales quadratically with the resolution.

Regardless, you can do whatever resolution you want with the Apache 2.0 model. Just change the config at runtime; it was trained to be resolution agnostic.

You are correct that we also released larger models with a larger backbone under a different, non open-source license.

Suspect choice for the paper to only include a single DETR from 2022 in the headline pareto chart and claim to have "the strongest AP–latency trade-off"... Clearly the authors were aware of models that exceed theirs given they even mentioned some of them in the introduction.

In parallel, DETR [5] cast detection as end-to-end set prediction, and its real-time descendants (RT-DETR [98], D-FINE [55], DEIM [21], RF-DETR [62]) have narrowed the accuracy gap with CNN based detectors on standard benchmarks.

It’s a big improvement if you’re already paying them but, given their aggressive approach to licensing, I can’t imagine why anyone would choose to use an Ultralytics model on a new project in 2026. You’re just asking to be shaken down and have to pay off a large bill down the line.

RF-DETR is both faster and more accurate and truly open source with an Apache 2.0 license: https://github.com/roboflow/rf-detr

Full disclosure: I’m one of the co-founders of Roboflow (we made RF-DETR, wrote this blog post, and are a sub-licensor of Ultralytics’ models.)

Yes, it should.

We have a JS SDK that supports RF-DETR: https://docs.roboflow.com/deploy/sdks/web-browser

We used DINOv2 as the backbone of our RF-DETR model, which is SOTA on realtime object detection and segmentation: https://github.com/roboflow/rf-detr

It makes a great target to distill SAM3 to.

We (Roboflow) have had early access to this model for the past few weeks. It's really, really good. This feels like a seminal moment for computer vision. I think there's a real possibility this launch goes down in history as "the GPT Moment" for vision. The two areas I think this model is going to be transformative in the immediate term are for rapid prototyping and distillation.

Two years ago we released autodistill[1], an open source framework that uses large foundation models to create training data for training small realtime models. I'm convinced the idea was right, but too early; there wasn't a big model good enough to be worth distilling from back then. SAM3 is finally that model (and will be available in Autodistill today).

We are also taking a big bet on SAM3 and have built it into Roboflow as an integral part of the entire build and deploy pipeline[2], including a brand new product called Rapid[3], which reimagines the computer vision pipeline in a SAM3 world. It feels really magical to go from an unlabeled video to a fine-tuned realtime segmentation model with minimal human intervention in just a few minutes (and we rushed the release of our new SOTA realtime segmentation model[4] last week because it's the perfect lightweight complement to the large & powerful SAM3).

We also have a playground[5] up where you can play with the model and compare it to other VLMs.

[1] https://github.com/autodistill/autodistill

[2] https://blog.roboflow.com/sam3/

[3] https://rapid.roboflow.com

[4] https://github.com/roboflow/rf-detr

[5] https://playground.roboflow.com

We (Roboflow) have had early access to this model for the past few weeks. It's really, really good. This feels like a seminal moment for computer vision. I think there's a real possibility this launch goes down in history as "the GPT Moment" for vision.

The two areas I think this model is going to be transformative in the immediate term are for rapid prototyping and distillation.

Two years ago we released autodistill[1], an open source framework that uses large foundation models to create training data for training small realtime models. I'm convinced the idea was right, but too early; there wasn't a big model good enough to be worth distilling from back then. SAM3 is finally that model (and will be available in Autodistill today).

We are also taking a big bet on SAM3 and have built it into Roboflow as an integral part of the entire build and deploy pipeline[2], including a brand new product called Rapid[3], which reimagines the computer vision pipeline in a SAM3 world. It feels really magical to go from an unlabeled video to a fine-tuned realtime segmentation model with minimal human intervention in just a few minutes (and we rushed the release of our new SOTA realtime segmentation model[4] last week because it's the perfect lightweight complement to the large & powerful SAM3).

We also have a playground[5] up where you can play with the model and compare it to other VLMs.

[1] https://github.com/autodistill/autodistill

[2] https://blog.roboflow.com/sam3/

[3] https://rapid.roboflow.com

[4] https://github.com/roboflow/rf-detr

[5] https://playground.roboflow.com

Wonder if you can subtract these vectors to get the opposite effect and what that ends up being for things like sycophancy or hallucination.

I also wonder what other personality vectors exist.. would be cool to find an “intelligence” vector we could boost to get better outputs from the same model. Seems like this is likely to exist given how prompting it to cosplay as a really smart person can elicit better outputs.

Is this a new product or a marketing page tying together a bunch of the existing MediaPipe stuff into a narrative?

Got really excited then realized I couldn’t figure out what “Google AI Edge” actually _is_.

Edit: I think it’s largely a rebrand of this from a couple years ago: https://developers.googleblog.com/en/introducing-mediapipe-s...

I've been reflecting a bit on this and remembering what it used to be like when I did hackathons regularly a decade or so ago. This project seems on-par with the type of 48 hour hackathon project I used to do (assuming CLIP had existed), but now I was able to do it in 2 hours instead of 48.

I can't imagine someone non-technical building something like this with prompting. The success of the project was highly dependent on my direction of the model to do what I wanted it to do (even though I gave it leeway in exactly how to do it). It did feel a bit like managing another engineer to do something vs doing it myself.

I don't use agents like this in my day to day work yet (I experimented with OpenHands a couple of months ago but it was frustrating, expensive, and took just as long as doing the task myself). But I'm thinking I probably will be a year from now.

A few times when the model got stuck I copy/pasted some stuff into o1 and pasted its response back into v0 (felt kind of like "escalating" to a more senior engineer) and that helped it get unstuck. Future models will be even more capable than o1. I imagine there will likely need to be a UI for "bringing in the big guns" of a smarter model in the future even if the grunt-work is done by a fast+cheap base model.

There's probably also something to letting the model "speak its native tongue". I don't know next.js but letting the model work with patterns it's been trained on probably helped it be more effective (compared to having OpenHands work in my own codebase using a structure it's unfamiliar with).

Hey all, sharing a project we made in 2 hours at the Vercel+NVIDIA hackathon last week.

While the app is cool, the thing that blew my mind is that the entire app was coded by Vercel's v0 agent. In other words: I did not write a single line of code to create the app (though my teammate did write the backend scraper & DB filler by hand).

[1] Writeup: https://blog.roboflow.com/nycerebro/

[2] Repo (including the generated code + initial meaty prompts): https://github.com/yeldarby/nycerebro

[3] v0 session: https://v0.dev/chat/nyc-erebro-app-RwzRUEMGveH?b=b_6AuWalvG7...

Is there any Docker alternative on Mac that can utilize the MPS device in a container? ML stuff is many times slower in a container on my Mac than running outside

More context from Jeremy Howard (fast.ai): https://x.com/jeremyphoward/status/1857765905188651456

It’s sad that only Google can (and honestly a bit surprising that Google hasn’t) use multimodal video models to index the semantic contents & transcripts of these videos for search. Huge long tail of unique content.

If you do it naively your video frames will buffer waiting to be consumed causing a memory leak and eventual crash (or quick crash if you’re running on a device with constrained resources).

You really need to have a thread consuming the frames and feeding them to a worker that can run on its own clock.

We’ve got an open source pipeline as part of inference[1] that handles the nuances (multithreading, batching, syncing, reconnecting) of running multiple real time streams (pass in an array of RTSP urls) for CV models like YOLO: https://blog.roboflow.com/vision-models-multiple-streams/

[1] https://github.com/roboflow/inference

Tried it a few weeks ago for a task (had a few dozen files in an open source repo I wanted to write tests for in a similar way to each other).

I gave it one example and then asked it to do the work for the other files.

It was able to do about half the files correctly. But it ended up taking an hour, costing >$50 in OpenAI credits, and took me longer to debug, fix, and verify the work than it would have to do the work manually.

My take: good glimpse of the future after a few more Moore’s Law doublings and model improvement cycles make it 10x better, 10x faster, and 10x cheaper. But probably not yet worth trying to use for real work vs playing with it for curiosity, learning, and understanding.

Edit: writing the tests in this PR given the code + one test as an example was the task: https://github.com/roboflow/inference/pull/533

This commit was the manual example: https://github.com/roboflow/inference/pull/533/commits/93165...

This commit adds the partially OpenDevin written ones: https://github.com/roboflow/inference/pull/533/commits/65f51...

Is this an editorialized list? Or is there some sort of programmatic filter applied?

Would have expected to see our repo, roboflow/sports, here given 1211 new stars (4.3x) this week.

https://star-history.com/#roboflow/sports&Date

Anyone know if there’s a way to get access to devices in a container on MacOS yet? (Eg USB webcam, MPS or CUDA accelerator)

FWIW you can use roboflow models on-device as well. detect.roboflow.com is just a hosted version of our inference server (if you run the docker somewhere you can swap out that URL for localhost or wherever your self-hosted one is running). Behind the scenes it’s an http interface for our inference[1] Python package which you can run natively if your app is in Python as well.

Pi inference is pretty slow (probably ~1 fps without an accelerator). Usually folks are using CUDA acceleration with a Jetson for these types of projects if they want to run faster locally.

Some benefits are that there are over 100k pre-trained models others have already published to Roboflow Universe[2] you can start from, supports many of the latest SOTA models (with an extensive library[3] of custom training notebooks), tight integration with the dataset/annotation tools that are at the core of Roboflow for creating custom models, and good support for common downstream tasks via supervision[4].

[1] https://github.com/roboflow/inference

[2] https://universe.roboflow.com

[3] https://github.com/roboflow/notebooks

[4] https://github.com/roboflow/supervision

Don't see a license listed in the repo; presumably needs to be the same as Meta's Llama 3 license?

It’s very good. And the cool thing is it’s made for fine tuning also. Excited to see how fine-tuned OCR models do.

How it does this is really cool. It’s got a VAE decoder. Reminds me a lot of how SAM works.

Love the concept.

I've used vast.ai (similar "Airbnb for GPUs" pitch) for years to spin up cheap test machines with GPUs you can't really find in the cloud (and especially consumer-grade GPUs like 4090s). Any insight into how this is different/better?

Found this really cool; I didn’t even know Deepmind was working on Fusion research https://www.wired.com/story/deepmind-ai-nuclear-fusion/

Also if this sort of data optimization stuff actually does add new laws of physics, that seems weird, right? I guess maybe we’re just a game. But if the super-alien things are trying to study something, then these extra laws of physics would seem to be defects.

Unless the creators of the simulation have also derived that they, too, are in a simulation and so this “new” law of physics also holds in the one level up simulation.

While Transformers have enabled tremendous progress in various application settings, such architectures still lag behind traditional symbolic planners for solving complex decision making tasks. In this work, we demonstrate how to train Transformers to solve complex planning tasks and present Searchformer, a Transformer model that optimally solves previously unseen Sokoban puzzles 93.7% of the time, while using up to 26.8% fewer search steps than standard A∗ search. Searchformer is an encoder-decoder Transformer model trained to predict the search dynamics of A∗. This model is then fine-tuned via expert iterations to perform fewer search steps than A∗ search while still generating an optimal plan. In our training method, A∗'s search dynamics are expressed as a token sequence outlining when task states are added and removed into the search tree during symbolic planning. In our ablation studies on maze navigation, we find that Searchformer significantly outperforms baselines that predict the optimal plan directly with a 5-10× smaller model size and a 10× smaller training dataset. We also demonstrate how Searchformer scales to larger and more complex decision making tasks like Sokoban with improved percentage of solved tasks and shortened search dynamics.

Neat; TIL about Sokoban puzzles. I remember playing Chip's Challenge on Windows 3.1 when I was a kid which had a lot of levels like that.

Not using transformers, but we do object detection in the browser with small quantized yolo models that are about 7mb and run at 30+ fps on modern laptops via tensorflow.js and onnxruntime-web.

Lots of cool demos and real world applications you can build with it. Eg we powered an AR card ID feature for Magic: The Gathering, built a scavenger hunt for SXSW, a test proctoring assistant (to warn you if you’re likely to get DQ’d for eg wearing headphones), and a pill counter for pharmacists. Really powerful for distribution to not make users install an app or need anything other than their smartphone.