Which is what I used to do and the problems were Signal keeps two copies of the file by default and puts them on your device memory. So your device ends up having 3 copies of the same data, on the same device.

This is rather impractical if you have a lot of content in Signal, which I do because my family use it extensively to send photos and videos (which is the sort of thing you generally really want to backup).