News for hackers. Stuff that matters.
Re:Show HN: Tarsier – Vision utilities for web interaction a... by abrichr on Wednesday May 15, 2024 @02:49PM (#40370833)
Congratulations on shipping!

In https://github.com/OpenAdaptAI/OpenAdapt/blob/main/openadapt... we use FastSAM to first segment the UI elements, then have the LLM describe each segment individually. This seems to work quite well; see https://twitter.com/OpenAdaptAI/status/1789430587314336212 for a demo.

More coming soon!

Re:Show HN: Tarsier – Vision utilities for web interaction a... by jackienotchan on Wednesday May 15, 2024 @04:01PM (#40371651)
Looking at OpenAdapt, I'm wondering why they didn't integrate Tarsier into AgentGPT, which is their flagship github repo but doesn't seem to be under active development anymore.
Re:Show HN: Tarsier – Vision utilities for web interaction a... by KhoomeiK on Wednesday May 15, 2024 @04:15PM (#40371832)
We have a lot more powerful use-cases for Tarsier in web data extraction at the moment. Stay tuned for a broader launch soon!