I do not have much experience with flask, but I recently told a coworker: if you are fighting the Django framework to do something, atleast question why you are doing that because there is a good chance you are going down the wrong path. The caveat is, yes there are some times when that is necessary, but it is also necessary to question what you are doing, and why it isn't possible.
Django isn't particular bad at this (it has some escape hatches to allow you to e.g. write custom SQL), but it is overal still quite a big framework and it's certainly harder to bend to your will than Flask.
So, you need to know the extra things it gives you and figure out if they will be helpful for your website.
Loosely, Django is: routing + HTML request parameter handling + HTML rendering + templating + user sessions + lots of database read/write help, assuming simple CRUD access patterns.
Flask is: routing + HTML request parameter handling + HTML rendering. Then, if you want any of the other things, you go and find a library for them.