HN user
computer-vision, ml and all things meta.
I think we all ought to look at the ZDR fine-print here.
I get that in principle that there's no retention, but these are powerful models that can comprehend, paraphrase and summarize your logs for the sake of "product" improvement. Who knows what's collected here.
The ICP question was more around the model itself. Are they looking to license it to robotics companies? Do they imagine that devs at robotics companies would be willing to deploy these models as a black box?
It’s unclear to me what their desired outcome for a blog post like this. If you’ve ever worked in a robotics setting, 80% implies that 20% of your autonomous actions are incorrect. Imagine if this were the case for autonomous driving where your car misbehaves 1 in every 5 actions it takes.
Posts like this just reminds me of the end to end demos AV companies built in the early days using a single camera - only to realize that it’s harder than it looks years later into development.
Frontier labs are realizing that software/models themselves don’t have real moats and move to embodied ai.
SOTA 80% means a practically useless robot. What are they really imagining their ICP to be here?
It's live now, https://github.com/vlm-run/mm.
We were planning to open-source this soon, but jumped the gun and posted about the video encoders here since it seemed relevant.
In either case, here you go, it's public now: https://github.com/vlm-run/mm.
Exactly! We experimented with a whole bunch of video encoding techniques for LLMs here: https://vlm-run.github.io/mm/encoders/#video
Pretty terribly expensive way to watch a video with Claude.
Use Gemini or some local VLM to do this way more efficiently. We spent quite a bit of time on video understanding, and Claude will just burn tokens.
Check out this library: https://vlm-run.github.io/mm/
You can swap models and try out different encoding methods for videos (https://vlm-run.github.io/mm/encoders/#video)
This is neat. I'd love to figure out a sequence of emojis that triggers the LLM in ways that puzzles a human.
That’s a pretty large binary for simply loading images.
In all honesty, opencv has stood the test of time and I’m certain newer LLMs will likely not attempt to rewrite it from scratch.
P.S. I’ve been a user since the IplImage days, circa 2007, and I’d still consider using it over most CV libraries today.
I can’t help but think that there are so many astroturfed comments in here.
Seems like a concerted and distributed effort from the entire Anthropic team every time to get this on top of HN.
Kind of crazy how many bespoke python sandbox implementations have popped up in the past few months.
I’d love to see if we can get GPU access within these runtimes, that’d be awesome.
What’s your experience with Monty? Been looking at it for one of our environments and it seems very promising.
VLM Run (https://vlm.run) | 1x Product + 1x ML Staff Engineer | Santa Clara, CA (HQ)
We're building the inference and orchestration layer for production Vision-Language Models. We care deeply about fast and ergonomic visual inference, reliable structured outputs, and the observability to iterate on them.
A few things we've shipped recently you can poke at:
1. Orion: our visual agent that reasons and acts over images, video, and documents. Chat at https://chat.vlm.run.
2. mm-ctx: a Unix-style multimodal CLI (find, cat, grep, wc) that gives coding agents real context over images, video, and PDFs. Rust core, Python devex.
3. vlmbench: single-file CLI for benchmarking VLM inference (TTFT, TPOT, throughput) across vLLM, Ollama, and SGLang.
Email hiring "at" vlm.run with your GitHub + a couple recent projects.
[2] https://pypi.org/project/mm-ctx | https://www.vlm.run/open-source/mm
[3] https://github.com/vlm-run/vlmbench | https://www.vlm.run/open-source/vlmbench
The recent claude code leak also revealed that they're poisoning their competitors via anti-distillation policies baked in claude code CLI (fake tool calls, adding noise etc).
VLM Run (https://vlm.run) | 1x Infrastructure Engineer + 2x AI/ML Engineer | Santa Clara, CA (HQ)
VLM Run is building infrastructure for production Vision-Language Model (VLM) systems — fast inference, tool-use + orchestration, reliable structured outputs, and the observability to iterate quickly. We’re a deeply technical team of veteran AI / computer-vision engineers (20+ years combined, MIT/CMU PhDs) who’ve shipped production ML infrastructure across autonomous driving and LLMs.
Open roles:
1. Infrastructure Engineer (Full-time, ONSITE): $150K–$220K + 1–3% equity https://app.dover.com/apply/VLM%20Run/8d4fa3b1-5b38-42e1-927...
2. AI/ML Engineer (Full-time, ONSITE): $150K–$220K + 0.5–3% equity https://app.dover.com/apply/VLM%20Run/1a490851-1ea1-4f12-a0f...
Email hiring "at" vlm.run with your GitHub + a couple recent projects.
P.S. We recently launched Orion, our visual agent that can reason and act over images, videos and documents. You can chat with Orion at https://chat.vlm.run and see capabilities at https://docs.vlm.run.
Real-time or continuous learning is great on paper, but to get this to work without extremely expensive regression testing and catastrophic forgetting is a real challenge.
Credit to the team for taking this on, but I’d be skeptical of announcements like this without at least 3–6 months of proven production deployments. Definitely curious how this plays out.
What do you think actually happened here in the past week?
They used Kimi, failed to acknowledge it in the original Composer announcement. Kimi team probably reached out and asked WTF? Their only recourse was to publicly disclose their whitepaper with Kimi mentioned to win brownie points about being open about their training pipeline, while placating the Kimi team.
VLM Run (https://vlm.run) | 1x Infrastructure Engineer + 2x AI/ML Engineer | Santa Clara, CA (HQ)
VLM Run is building infrastructure for production Vision-Language Model (VLM) systems — fast inference, tool-use + orchestration, reliable structured outputs, and the observability to iterate quickly. We’re a deeply technical team of veteran AI / computer-vision engineers (20+ years combined, MIT/CMU PhDs) who’ve shipped production ML infrastructure across autonomous driving and LLMs.
Open roles:
1. Infrastructure Engineer (Full-time, ONSITE): $150K–$220K + 0.5–3% equity https://app.dover.com/apply/VLM%20Run/8d4fa3b1-5b38-42e1-927...
2. AI/ML Engineer (Full-time, ONSITE): $150K–$220K + 0.5–3% equity https://app.dover.com/apply/VLM%20Run/1a490851-1ea1-4f12-a0f...
Email hiring "at" vlm.run with your GitHub + a couple recent projects.
P.S. We recently launched Orion, our visual agent that can reason and act over images, videos and documents. You can chat with Orion at https://chat.vlm.run and see capabilities at https://docs.vlm.run.
AI allows you to accelerate the initial build process, but I think engineering is all about craftsmanship. Today most LLMs have poor taste and chipping away the cruft matters more than ever.
uvx probably is the way to go here (fully self-contained environment for each skill), and use stdout as the I/O bridge between skills.
The cold-boot time on this model can hardly be called “serverless”
ELO scores for OCR don't really make much sense - it's trying to reduce accuracy to a single voting score without any real quality-control on the reviewer/judge.
I think a more accurate reflection of the current state of comparisons would be a real-world benchmark with messy/complex docs across industries, languages.
Apple OCR even on the Mac is insanely good, in fact way better than AWS textract/GCP cloud vision OCR.
Any idea what model is being used?
VLM Run (https://vlm.run) | Infrastructure Engineer + DevRel + AI/ML Engineer | Santa Clara, CA (HQ)
VLM Run is building infrastructure for production Vision-Language Model (VLM) systems — fast inference, tool-use + orchestration, reliable structured outputs, and the observability to iterate quickly. We’re a deeply technical team of veteran AI / computer-vision engineers (20+ years combined) who’ve shipped production ML infrastructure across autonomous driving and LLMs.
Open roles:
1. Infrastructure Engineer (Full-time, ONSITE): $150K–$220K + 0.5–3% equity https://app.dover.com/apply/VLM%20Run/8d4fa3b1-5b38-42e1-927...
2. Founding DevRel (Full-time, ONSITE/REMOTE): $90K–$140K + 0.5–3% equity https://app.dover.com/apply/VLM%20Run/de84c63e-fd0a-418b-929...
3. AI/ML Engineer (Full-time, ONSITE): $150K–$220K + 0.5–3% equity https://app.dover.com/apply/VLM%20Run/1a490851-1ea1-4f12-a0f...
Email hiring "at" vlm.run with your GitHub + a couple recent projects.
P.S. We recently launched *Orion*, our visual agent that can reason and act over images, videos and documents. You can chat with Orion at https://chat.vlm.run and see capabilities at https://docs.vlm.run.
It's like going to the grocery store and buying tabloids, pretending they're scientific journals.
This is pure gold. I've always found this approach of evals on a moving-target via consensus broken.
I'd love to see Claude Code remove more lines than it added TBH.
There's a ton of cruft in code that humans are less inclined to remove because it just works, but imagine having LLM doing the clean up work instead of the generation work.
Here's a short cookbook exploring an agentic approach to vision–language tasks: detection, segmentation, OCR, generation, and combining classical CV tools with VLM reasoning.
Happy to run examples if you leave a comment.
[1] IPython notebook: https://github.com/vlm-run/vlmrun-cookbook/blob/main/noteboo...
[2] Colab: https://colab.research.google.com/github/vlm-run/vlmrun-cook...
What is photopea built on?
This is why arenas are generally a bad idea for assessing correctness in visual tasks.