News for hackers. Stuff that matters.

A comment on Show HN: Simplex: Automate browser workflows using code and natural language. Parent comment

Re:Show HN: Simplex: Automate browser workflows using code a... by marcon680 on Wednesday January 15, 2025 @12:12AM (#42707559)
We finetune our own VLMs for this -- unfortunately prefer not to share which ones we use specifically! ClickClickClick looks awesome, have you heard of FerretUI (https://arxiv.org/pdf/2404.05719)? Pretty similar idea.
[ Parent | on HN ]
Re:Show HN: Simplex: Automate browser workflows using code a... by mkagenius on Wednesday January 15, 2025 @12:58AM (#42707801)
Yes, I tried a similar one called "omniparser" - where the issue was it was missing annotating some UI elements sometimes. Moreover, Gemini and Molmo worked right out of the box without needing any fine tune.
[ Parent | on HN ]