HN user

antves

69 karma
Posts3
Comments51
View on HN

I think one main point is that not all "computer use" is the same, the harness and agentic experience matters a lot. A poorly designed API experience can actually be _less_ efficient than a well designed browser or computer use experience

In particular, the vision-based approach used in the evaluation has clear limitations with regard to efficiency due to its nature (small observation window, heterogeneous modality)

At Smooth we use an hybrid DOM/vision approach and we index very strongly on small models. An interesting fact is that UIs are generally designed to minimize ambiguity and supply all and only the necessary context as token-efficient as possible, and the UX is cabled up to abstract the APIs in well-understood interface patterns e.g. dropdowns or autocompletes. This makes navigation easier and that's why small models can do it, which is another dimension that must be considered

We typically recommend using APIs/MCP where available and well designed, but it's genuinely surprising how token-efficient agentic browser navigation can actually be

we benchmarked on webvoyager and have a few more benchmarks coming up (see https://docs.smooth.sh/performance), we’ll be publishing the full results shortly

we run headful browsers with a fingerprinting that is as stealth as possible and on top of that we can use your ip

anecdotally, making all requests originate from your own residential address has been a major success compared to other cloud-only solutions

it will be interesting to see how this will play out, having to “hide the agent” feels like a temporary work around until society accepts that agents actually do exist

it's interesting to see how things will play out, but I really believe that doing Claude Code (maybe with Opus 4.6) + click tool + move_mouse tool + snapshot page tool + another 114 more tools is definitely not the best approach

the main issue with this interface is that the commands are too low-level and that there is no way of controlling the context over time

once a snapshot is added to the context those tokens will take up very precious context window space, leading to context rot, higher cost, and higher latency

that's why agents need to use very large models for these kind of systems to work and, unfortunately, even then they're very slow, expensive, and less reliable than using a purpose-made system

I wonder if a standardized interface will organically emerge over time. At the moment SKILL.md + CLI seem to be the most broadly adopted interface - even more than MCP maybe

the Claude --chrome command has a few limitations:

1. it exposes low-level tools which make your agent interact directly with the browser which is extremely slow, VERY expensive, and less effective as the agent ends up dealing with UI mechanics instead of thinking about the higher-level goal/intents

2. it makes Claude operate the browser via screenshots and coordinates-based interaction, which does not work for tasks like data extraction where it needs to be able to attend to the whole page - the agent needs to repeatedly scroll and read one little screenshot at the time and it often misses critical context outside of the viewport. It also makes the task more difficult as the model has to figure out both what to do and how to do it, which means that you need to use larger models to make this paradigm actually work

3. because it uses your local browser, it also means that it has full access to your authenticated accounts by default which might not be ideal in a world where prompt-injections are only getting started

if you actively use the --chrome command we'd love to hear your experience!

have you tried any other AI browser automation tools? we would be curious to hear about your use cases because the use cases we have been working on with our customers involve scenarios where traditional playwright automations are not viable, e.g. they operate on net new websites and net new tasks for each execution

There are pros and cons to running the browser on your own machine

For example, with remote browsers you get to give your swarm of agents unlimited and always-on browsers that they can use concurrently without being bottlenecked by your device resources

I think we tend to default to thinking in terms of one agent and one browser scenarios because we anthropomorphize them a lot, but really there is no ceiling to how parallel these workflows can become once you unlock autonomous behavior

We love these tools but they were designed for testing, not for automation. They are too low-level to be used as they are by AI.

For example, the playwright MCP is very unreliable and inefficient to use. To mention a few issues, it does not correctly pierce through the different frames and does not handle the variety of edge cases that exist on the web. This means that it can't click on the button it needs to click on. Also, because it lacks control over the context design, it cannot optimize for contextual operations and your LLM trace gets polluted with incredible amount of useless tokens. This increases cost, task complexity for the LLM, and latency

On top of that, these tools rely on the accessibility tree, which is just not a viable approach for a huge number of websites

Our approach is actually very cost-effective compared to alternatives. Our browser uses a token-efficient LLM-friendly representation of the webpage that keeps context size low, while also allowing small and efficient models to handle the low-level navigation. This means agents like Claude can work at a higher abstraction level rather than burning tokens on every click and scroll, which would be far more expensive

It's amazing how agents like Claude Code become very much more autonomous when they have the ability to verify their work. That's part of the reason why they work much better for unit-testable work.

I think this paradigm was very visible in yesterday's blog post from Anthropic (https://www.anthropic.com/engineering/building-c-compiler) when they mentioned that giving the agents the ability to verify against GCC was the key to unlock further progress

Giving a browser to these agents is a no brainer, especially if one works in QA or develops web-based services

Thanks for bringing this point up!

We take security very seriously and one of the main advantages of using Smooth over running things on your personal device is that your agent gets a browser in a sandboxed machine with no credentials or permissions by default. This means that the agent will be able to see only what you allow it to see. We also have some degree of guard-railing which we will continue to mature over time. For example, you can control which URLs the agent is allowed to view and which are off limits.

Until we'll be able to run everything locally on device, there must be a level of trust in the organizations that control the technology stack, passing from the LLM all the way to the infrastructure providers. And this applies to every personal information you disclose at any touch point to any AI company.

I believe that this trust is something that we and every other company in the space will need to fundamentally continue to grow and mature with our community and our users.

Totally agree! The web for agents is evolving very fast and it's still unclear what it will look like

Our take is that, while that happens, agents today need to be able to access all the web resources that we can access as humans

Also, browsers are a really special piece of software because they provide access to almost every other kind of software. This makes them arguably the single most important tool for AI agents, and that’s why we believe that a browser might be all agents need to suddenly become ten times more useful than they already are

Thanks for asking! There are a few core differences: 1. we expose a higher level interface which allows the agent to think about what to do as opposed to what to do 2. we developed a token-efficient representation of the webpages that combines both visual and textual elements, heavily optimized for what LLMs are good at. 3. because we control the agentic loop, it also means that we can do fancy things on contextual injections, compressions, asynchronous manipulations, etc which are impossible to achieve when exposing the navigation interface 4. we use a coding agent under the hood, meaning that it can express complex actions efficiently and effectively compared to the CLI interface that agent-browser exposes 5. because we control the agent, we can use small and efficient LLMs which make the system much faster, cheaper, and more reliable

Also, our service comes with batteries included: the agent can use browsers in our cloud with auto-captcha solvers, stealth mode, we can proxy your own ip, etc

I agree, these mitigations alone can't be sufficient, but they are all necessary within a wider framework.

The only way to make this kind of agents safe is to work on every layer. Part of it is teaching the underlying model to see the dangers, part of it is building stronger critics, and part of it is hardening the systems they connect to. These aren’t alternatives, we need all of them.