Re:Big Tech's underground race to buy AI training data
by dleink on Friday April 05, 2024 @03:20PM
(#39946052)
> I don't think [things created by humans] would contain anything novel.
That's quite a proposition.
Re:Big Tech's underground race to buy AI training data
by passion__desire on Friday April 05, 2024 @04:51PM
(#39947061)
Not every essay is created equal. Plus I don't understand what would a new way of combining same words, given llms already have seen trillions of tokens, would achieve. llms could inpaint to arrive at similar texts.
Re:Big Tech's underground race to buy AI training data
by nicklecompte on Friday April 05, 2024 @08:41PM
(#39948961)
I agree with your overall point that an AI which can learn about the world directly won't need eleventy billion documents to learn language generation. Just two comments:
1) Based on how pre-verbal children learn, one nitpick is that I strongly suspect we need to give AI touch and a sense of space in order to truly understand quantity, causality, object permanence, etc.
2) Something that is not a nitpick: even a superhuman multimodal AI wouldn't have direct access to human emotions, sexuality, ideas of natural beauty, etc. I don't think humans have run out of interesting things to say about these ideas.
(In particular, I don't think a superhuman AI is capable of understanding music unless it is directly emulating the biological processes by which humans understand music. The issue is not "logical" - melodies don't actually make sense analytically.)
I don't think college essays, etc would contain anything novel. Future techniques could smoothly interpolate better creating ever-anew wordmud.