HN user
In the last weeks I experimented with Claude Computer Use to automate some daily tasks (via its Ui.Vision chat integration, see https://forum.ui.vision/t/v9-5-0-brings-computer-use-ai-chat... ) - and the results are mixed. Claude gets things wrong way too often to be useful.
Has anyone done any comparison Claude Computer Use vs OpenAI Operator? Is it signifcantly better?
100% this. I am using the open source Ui.vision to automate some business tasks. Works well, but only 10% of the work is for automating the main workflow, 90% of the work goes into error and edge case handling (e. g. Internet down, website (to scrape data from) down, some input data has typos or the wrong date format, etc).
A human can work around all these error cases once she encounters them. Current RPA tools like Uipath or ui.vision need explicit programming for every potential situation. And I see no indication that Claude is doing any better than this.
For starters, for visual automation to work reliably the OCR quality needs to improve further and be 100% reliable. Even in that very basic "AI" area, Claude, ChatGPT, Gemini are good, but not good enough yet.
"there’s not much left to innovate"
???
Totally disagree. Just one example: Tesla has only two global mass market models (3+Y). There is plenty of room to innovate on this topic alone. Model 2? Model whatever. Nothing new in years.
Wow, what a click-baity misleading headline. Key point is that this number is going down:
"The big news is that more and more people are opting to trade their Teslas for an EV from a legacy automaker"
This might be me in a few years. Tesla stopped being innovative years ago. No surprise, its CEO is high on drugs most of the time and spends his days (and nights) posting childish memes.
(1) AI:
What you said is all correct, but there are some positive steps as well, for example ELLIS: https://ellis.eu/
ELLIS is a great way to compete with the top US universities. Much more is needed, especially access to computing power, just as the CERN for AI proposal suggests.
(2) EU research:
EU research funding itself is a success:
https://erc.europa.eu/news-events/news/statement-erc-scienti...
If browser extensions could access the local LLMs - Now THAT would be a very cool feature.
Imagine what (test) automation extensions like Selenium IDE or Ui.vision could do then. Or translation extensions. Or.... so many cool options.
and debug if it errors or isn't what you wanted.
That is a potential problem Because at this point the user will still need good HTML/Selenium/coding knowledge to debug. We ran into the same issue using chatgpt generated scripts for ui.vision. Once a QA person has to figure out why some generated code does not work, it becomes a hassle and removes any initial advantage over the classic record & replay approach.
And your source for this statement is which Telegram group?
What has a "German copyright cease and desist" to do with privacy, or the lack of privacy? And who is "them"? A few sources and details instead of just ranting would be useful.
Ui.Vision dekstop automation is cross-platform works on Linux and macOS (and Windows). It is not exactly the same, though.
Wow, Twitter is collapsing much faster than I expected. With PG and some other high-profile accounts gone, many will loose interest in their Twitter feed fast. Rinse and repeat.
I could charge at home, but don’t have a garage
No problem (why should it?)
BEVs are improving quickly so what I buy might be superseded soon
That WAS my concern, but EV resale value is amazingly stable, better than most ICE in my area (EU). After all, EVs are still cars, so innovation cycles are long (many years).
Why? FSD even in its current state will save lives. Think of it as driver-assist on steroids.
I'd rather pay with Bitcoin/ETH/xx than using Mastercard. Currently 1-2 big US companies have ALL my payment data, they know everything about my life. Is this good?
Privacy is the killer feature for crypto. Or is there another non-crypto solution to this problem?
So, what other car (that I can buy now) gives me the same range?
Yes! Recording xpath works fine in many cases, but once something on the website changes you are left with this auto-generated Xpath that you do not know how to change. Maintaining such a test is a nightmare and a waste of brain-cycles.
I hate XPaths with a passion. They are powerful but a waste of time. The easiest solution is to use visual automation tools like UI Vision (web browser) or Sikuli (desktop). There you write something like "click: login" - and the word login is found not by parsing the DOM, but by using OCR.
Obviously such an automation runs much slower than any Selenium based automation, but it its sooo much easier to create and maintain, especially on complex websites with tons of Javascript and frameworks. I believe this is the future of causal (test) automation.
they're appointed by national governments.
...which are elected.
So if you want e. g. a right-wing EU commissioner just vote for your local right-wing party. It is that easy.
"Not being able to code" is not black and white. Many can follow basic steps, as long as it gets not too complex. Thus "Low code" tools like Powerautomate,selenium ide, ahk or UIvision are great for people to do test automation. They are a quick way to create an automation. And many users learn/start to code this way.
There is an easy alternative: Use Firefox
Nazis only care about skin color. You could be from Mars - as long as you are white, all is ok.
This is big problem. Without (more) PoC living in this area, nothing improves.
But would I recommend a PoC friend to live in Dresden? Not really, I would rather advise him to look for a job in Berlin or Frankfurt.
I use both!
UI Vision is unique because it supports visual automation with image recognition and OCR. A bit like Sikuli but inside the browser. This avoids dealing with CSS selectors and XPaths.
The Mac built-in tool does not work on Youtube videos, which is one of my use cases for the Copyfish OCR tool. This, and the integrated translation.
Copyfish is a good alternative to Project Naptha and works in Firefox:
1) OCR'ing of a PDF is difficult. The only good service is Google
OCRspace is OK, too, and easier to use. You can just send the PDF. It is free for PDFs with 3 or less pages.
2) People want searchable OCR'd PDFs where you can highlight the text, even when it's a bitmap underneath.
OCRspace can also create searchable PDFs: https://ocr.space/searchablepdf
"positions in EU parliament are chosen and voted for by politicians only"
100% wrong.
Elections to the European Parliament take place every five years by universal adult suffrage, and with more than 400 million people eligible to vote, they are considered the second largest democratic elections in the world after India's.
Yes, using CV/OCR for automation is the future.
My key point was that there are newer alternatives (the OP asked for that). PyAutoGUI would be another newer alternative. So are Applitools and UIPath, but they are not open-source.
The challenge with Tesseract is that its OCR quality is (too) low for screenshots because screenshots are usually text on colored background. Tesseract is optimized for classical text document OCR. Of course, you can switch OCR engines with Sikuli (I did that before), but this requires touching the source code.
Newer: UI.Vision is newer and I find it easier to use. A key difference is that UI.Vision supports text reading (via OCR) out of the box: https://ui.vision/rpa/x/desktop-automation