News for hackers. Stuff that matters.

eastendguy

Re:Introducing Operator on Friday January 24, 2025 @03:27PM (#42816606)
In the last weeks I experimented with Claude Computer Use to automate some daily tasks (via its Ui.Vision chat integration, see https://forum.ui.vision/t/v9-5-0-brings-computer-use-ai-chat... ) - and the results are mixed. Claude gets things wrong way too often to be useful.

Has anyone done any comparison Claude Computer Use vs OpenAI Operator? Is it signifcantly better?

Re:Computer use, a new Claude 3.5 Sonnet, and Claude 3.5 Haiku on Wednesday October 23, 2024 @04:59AM (#41923181)
100% this. I am using the open source Ui.vision to automate some business tasks. Works well, but only 10% of the work is for automating the main workflow, 90% of the work goes into error and edge case handling (e. g. Internet down, website (to scrape data from) down, some input data has typos or the wrong date format, etc).

A human can work around all these error cases once she encounters them. Current RPA tools like Uipath or ui.vision need explicit programming for every potential situation. And I see no indication that Claude is doing any better than this.

For starters, for visual automation to work reliably the OCR quality needs to improve further and be 100% reliable. Even in that very basic "AI" area, Claude, ChatGPT, Gemini are good, but not good enough yet.

"there’s not much left to innovate"

???

Totally disagree. Just one example: Tesla has only two global mass market models (3+Y). There is plenty of room to innovate on this topic alone. Model 2? Model whatever. Nothing new in years.

Wow, what a click-baity misleading headline. Key point is that this number is going down:

"The big news is that more and more people are opting to trade their Teslas for an EV from a legacy automaker"

This might be me in a few years. Tesla stopped being innovative years ago. No surprise, its CEO is high on drugs most of the time and spends his days (and nights) posting childish memes.

(1) AI:

What you said is all correct, but there are some positive steps as well, for example ELLIS: https://ellis.eu/

ELLIS is a great way to compete with the top US universities. Much more is needed, especially access to computing power, just as the CERN for AI proposal suggests.

(2) EU research:

EU research funding itself is a success:

https://erc.europa.eu/news-events/news/statement-erc-scienti...

If browser extensions could access the local LLMs - Now THAT would be a very cool feature.

Imagine what (test) automation extensions like Selenium IDE or Ui.vision could do then. Or translation extensions. Or.... so many cool options.

Re:Show HN: Autotab – Boring AI Agents for real world tasks on Wednesday October 25, 2023 @09:01AM (#38012319)
> and debug if it errors or isn't what you wanted.

That is a potential problem Because at this point the user will still need good HTML/Selenium/coding knowledge to debug. We ran into the same issue using chatgpt generated scripts for ui.vision. Once a QA person has to figure out why some generated code does not work, it becomes a hassle and removes any initial advantage over the classic record & replay approach.

Re:Germany opposes client-side scanning planned by EU on Thursday March 30, 2023 @10:09AM (#35372937)
And your source for this statement is which Telegram group?
Re:Germany opposes client-side scanning planned by EU on Thursday March 30, 2023 @10:00AM (#35372832)
What has a "German copyright cease and desist" to do with privacy, or the lack of privacy? And who is "them"? A few sources and details instead of just ranting would be useful.
Re:AutoHotkey v2 Official Release Announcement on Monday January 30, 2023 @03:49AM (#34577353)
Ui.Vision dekstop automation is cross-platform works on Linux and macOS (and Windows). It is not exactly the same, though.
Re:Paul Graham is leaving Twitter on Sunday December 18, 2022 @02:54PM (#34042181)
Wow, Twitter is collapsing much faster than I expected. With PG and some other high-profile accounts gone, many will loose interest in their Twitter feed fast. Rinse and repeat.
Re:Winter & Cold Weather EV Range Loss in 7,000 Cars on Sunday December 18, 2022 @12:05PM (#34039673)
> I could charge at home, but don’t have a garage

No problem (why should it?)

> BEVs are improving quickly so what I buy might be superseded soon

That WAS my concern, but EV resale value is amazingly stable, better than most ICE in my area (EU). After all, EVs are still cars, so innovation cycles are long (many years).

Why? FSD even in its current state will save lives. Think of it as driver-assist on steroids.
Re:How crypto goes to zero on Thursday November 24, 2022 @06:58AM (#33730316)
I'd rather pay with Bitcoin/ETH/xx than using Mastercard. Currently 1-2 big US companies have ALL my payment data, they know everything about my life. Is this good?

Privacy is the killer feature for crypto. Or is there another non-crypto solution to this problem?

So, what other car (that I can buy now) gives me the same range?
Re:Web Automation: Don't Use Selenium, Use Playwright on Friday November 11, 2022 @07:46AM (#33560145)
Yes! Recording xpath works fine in many cases, but once something on the website changes you are left with this auto-generated Xpath that you do not know how to change. Maintaining such a test is a nightmare and a waste of brain-cycles.
Re:Web Automation: Don't Use Selenium, Use Playwright on Friday November 11, 2022 @07:42AM (#33560110)
I hate XPaths with a passion. They are powerful but a waste of time. The easiest solution is to use visual automation tools like UI Vision (web browser) or Sikuli (desktop). There you write something like "click: login" - and the word login is found not by parsing the DOM, but by using OCR.

Obviously such an automation runs much slower than any Selenium based automation, but it its sooo much easier to create and maintain, especially on complex websites with tons of Javascript and frameworks. I believe this is the future of causal (test) automation.

Re:In Europe, the bird will fly by our EU rules on Friday October 28, 2022 @09:03AM (#33370862)
> they're appointed by national governments.

...which are elected.

So if you want e. g. a right-wing EU commissioner just vote for your local right-wing party. It is that easy.

Re:Robot Framework, generic open source automation framework on Thursday October 27, 2022 @12:54PM (#33360476)
"Not being able to code" is not black and white. Many can follow basic steps, as long as it gets not too complex. Thus "Low code" tools like Powerautomate,selenium ide, ahk or UIvision are great for people to do test automation. They are a quick way to create an automation. And many users learn/start to code this way.
Re:Proxy Chrome extensions are not going to be usable in MV3 on Monday September 19, 2022 @02:58PM (#32902417)
There is an easy alternative: Use Firefox
Nazis only care about skin color. You could be from Mars - as long as you are white, all is ok.
This is big problem. Without (more) PoC living in this area, nothing improves.

But would I recommend a PoC friend to live in Dresden? Not really, I would rather advise him to look for a job in Berlin or Frankfurt.

Re:Show HN: I created a browser automation tool on Saturday August 06, 2022 @06:58AM (#32366937)
I use both!

UI Vision is unique because it supports visual automation with image recognition and OCR. A bit like Sikuli but inside the browser. This avoids dealing with CSS selectors and XPaths.

Re:Project Naptha on Monday July 11, 2022 @09:40AM (#32055976)
The Mac built-in tool does not work on Youtube videos, which is one of my use cases for the Copyfish OCR tool. This, and the integrated translation.
Re:Project Naptha on Monday July 11, 2022 @04:07AM (#32053494)
Copyfish is a good alternative to Project Naptha and works in Firefox:

https://ocr.space/copyfish

> 1) OCR'ing of a PDF is difficult. The only good service is Google

OCRspace is OK, too, and easier to use. You can just send the PDF. It is free for PDFs with 3 or less pages.

> 2) People want searchable OCR'd PDFs where you can highlight the text, even when it's a bitmap underneath.

OCRspace can also create searchable PDFs: https://ocr.space/searchablepdf

"positions in EU parliament are chosen and voted for by politicians only"

100% wrong.

Elections to the European Parliament take place every five years by universal adult suffrage, and with more than 400 million people eligible to vote, they are considered the second largest democratic elections in the world after India's.

Re:SikuliX – Automate what you see on a computer monitor on Sunday February 13, 2022 @07:27AM (#30321262)
Yes, using CV/OCR for automation is the future.
Re:SikuliX – Automate what you see on a computer monitor on Sunday February 13, 2022 @07:21AM (#30321231)
My key point was that there are newer alternatives (the OP asked for that). PyAutoGUI would be another newer alternative. So are Applitools and UIPath, but they are not open-source.

The challenge with Tesseract is that its OCR quality is (too) low for screenshots because screenshots are usually text on colored background. Tesseract is optimized for classical text document OCR. Of course, you can switch OCR engines with Sikuli (I did that before), but this requires touching the source code.

Re:SikuliX – Automate what you see on a computer monitor on Sunday February 13, 2022 @05:16AM (#30320705)
Newer: UI.Vision is newer and I find it easier to use. A key difference is that UI.Vision supports text reading (via OCR) out of the box: https://ui.vision/rpa/x/desktop-automation

Older »