Re:R1 Computer Use
by mountainriver on Thursday February 06, 2025 @04:39PM
(#42966765)
Perception is just 1-2 screenshots. A number of recent VLM models have a lot more pretraining data on GUI interactions, which helps.
[ Parent
| on HN ]
Such as? Are they able to recognize arbitrary GUI elements from various desktop programs, web browsers, etc?
Re:R1 Computer Use
by mountainriver on Thursday February 06, 2025 @05:01PM
(#42966940)
Qwen2.5-vl seems to be the best right now by our tests.
UI-TARS by bytedance also has a good amount of pretraining.
Molmo is also very good at coordinates.