It's been a minute since I've looked at embeddings in this context but isn't Johnson-Lindenstrauss going to be applicable here so that you can get away with 1024-long (or shorter) vectors?