Re:Implementation of vision language model in a single file ...
by avisoori1x on Monday April 22, 2024 @08:40AM
(#40113791)
I implemented a vision language model consisting of an image encoder, a multimodal projection module and a decoder language model in pure PyTorch. Think of this as a simplified version of what you see in GPT-4 or Claude 3 in terms of vision capabilities demonstrated by a language model. The name ‘seemore’ is my way of paying homage to Andrej Karpathy’s ‘makemore’ because here I use a character level autoregressive language model much like in his basic transformer implementation. seemore.py in the repo has the single file with everything in it.
[ Parent
| on HN ]