I had some fun trying to get this to work with prompts based on specific brands etc. It felt like it had a pretty good attempt at a Fender Jazz Bass and a Moog Mother, but struggled with a Juno 6. Is that pure coincidence or would the model be able to understand the semantics of specific instruments and things as well as more generic terms?

Good question. In my experience combining generic descriptors is what works best. This is probably due to the text captions used during training mostly consist of generic instrument names, genre names and adjectives.