News for hackers. Stuff that matters.
Re:Show HN: Tarsier – Vision utilities for web interaction a... by pk19238 on Wednesday May 15, 2024 @12:59PM (#40369513)
this is such a creative solution. reminds me of how a team rendered wolfenstein into ASCII characters and fine tuned mistral to successfully play it.
Re:Show HN: Tarsier – Vision utilities for web interaction a... by KhoomeiK on Wednesday May 15, 2024 @01:04PM (#40369569)
Thanks! Yeah, it seems like a lot can be done with just text while we wait for multimodal models to catch up. The recent Platonic Representation Hypothesis [1] also suggests that different models, regardless of modality, build the same internal representations of the world.

[1] https://arxiv.org/abs/2405.07987