News for hackers. Stuff that matters.

A comment on Generating classical music with LSTM neural networks.

Re:Generating classical music with LSTM neural networks by citnaj on Wednesday November 28, 2018 @03:06PM (#18555130)
>"More recently, there is a shift towards using a Transformer architecture, and right now I’m experimenting with that as well."

I'm really curious- any early results to share on that? Attention really does make a big difference on a lot of things (including work I've done so I know first hand). It should improve the coherence of the entire music piece in theory at least, right?

[ Parent | on HN ]
Re:Generating classical music with LSTM neural networks by kasrahbar on Wednesday November 28, 2018 @05:37PM (#18556336)
Check out Music Transformer that was recently published https://arxiv.org/abs/1809.04281

Some generated samples: https://storage.googleapis.com/music-transformer/index.html

[ Parent | on HN ]
Re:Generating classical music with LSTM neural networks by bcheung on Wednesday November 28, 2018 @08:14PM (#18557319)
The accompaniment examples are cool. That would be a very nice tool to have. Just play a melody and it auto-generates an accompaniment.
[ Parent | on HN ]
Re:Generating classical music with LSTM neural networks by mcleavey on Wednesday November 28, 2018 @03:12PM (#18555173)
Transformer is working really well- I'm very excited. I'll probably be sharing results soon. Yes, the attention makes a huge difference & the pieces are both more creative and more coherent.
[ Parent | on HN ]
Re:Generating classical music with LSTM neural networks by gwern on Wednesday November 28, 2018 @07:34PM (#18557109)
Have you considered using 'learning from human preferences' as the loss function in addition to the Transformers? That was another OpenAI project, and it seems tailor-made for music generation: what is more 'I know it when I hear it' than music quality?
[ Parent | on HN ]
Re:Generating classical music with LSTM neural networks by citnaj on Wednesday November 28, 2018 @04:00PM (#18555648)
That is exciting! I'll be watching on Twitter :)
[ Parent | on HN ]
Re:Generating classical music with LSTM neural networks by minimaxir on Wednesday November 28, 2018 @04:27PM (#18555836)
textgenrnn (https://github.com/minimaxir/textgenrnn) uses a simple Attention Weighted Average at the end of the model for text generation, which in my testing allows the model to learn much better.
[ Parent | on HN ]