As far as my extremely poor understanding of the law goes: this depends on where you live but generally you are not allowed to download a digital copy of a physical book you own, but you are allowed to create your own [1].

It may also be worth noting that most jurisdictions are only interested in distribution, not downloading, so the chances of prosecution are slim. A small company you may have heard of called Meta is currently using a similar argument in US court [2].

[1] https://ebooks.stackexchange.com/questions/1111/i-have-a-pri...

[2] https://news.ycombinator.com/item?id=43125840