Re:Multi-modal prompt injection image attacks against GPT-4V
by jcims on Saturday October 14, 2023 @10:48AM
(#37880879)
Ah! Do you think this is like an adjunct or fine-tuning kind of thing or do they retrain GPT 4+ images? That seems like a ginormous task.
[ Parent
| on HN ]
Re:Multi-modal prompt injection image attacks against GPT-4V
by simonw on Saturday October 14, 2023 @11:03AM
(#37881008)
When they launched GPT-4 back in March they talked about its image capabilities - I think they trained it into the model from the start, but then spent several extra months on additional safety research, as described in their paper: https://cdn.openai.com/papers/GPTV_System_Card.pdf
Re:Multi-modal prompt injection image attacks against GPT-4V
by GaggiX on Saturday October 14, 2023 @12:38PM
(#37881941)
I don't think they have trained the model from scratch on images too, but more like Flamingo or similar methods, and the LLM model is at most finetuned but not trained from scratch.
If the model was trained from scratch then I would expect it to be able to develop new emergent behavior like being much better at ascii art, SVG, OpenSCAD scripting at least in the case the model was trained on the usual masked objective but not only on text tokens but image tokens too.