News for hackers. Stuff that matters.
Re:R1 Computer Use by mountainriver on Thursday February 06, 2025 @04:39PM (#42966765)
Perception is just 1-2 screenshots. A number of recent VLM models have a lot more pretraining data on GUI interactions, which helps.
Re:R1 Computer Use by iiJDSii on Thursday February 06, 2025 @04:47PM (#42966820)
Such as? Are they able to recognize arbitrary GUI elements from various desktop programs, web browsers, etc?
Re:R1 Computer Use by mountainriver on Thursday February 06, 2025 @05:01PM (#42966940)
Qwen2.5-vl seems to be the best right now by our tests.

UI-TARS by bytedance also has a good amount of pretraining.

Molmo is also very good at coordinates.