It's like learning Spanish. If you memorize a bunch of words you won't be able to speak Spanish. But if you have a bunch of Spanish words memorized than you'll get a lot more out of reading/listening/ trying to speak Spanish than if you don't.
And memorization is usually a far less taxing activity that you can do on your phone when waiting in line or using the bathroom.
edit: Also, speaking as someone who mostly taught themselves physics from the books in college and didn't go to class. (I was not a great student, I did _okay_, and probably got lucky).
What I do: Fact based stuff that I want to just recall quickly go in one deck. I do this deck daily - 5-10 minutes max while having breakfast.
The more complicated cards go in the second deck, which I do only occasionally (e.g. weekends when I have time).
When I initially had them all in one deck, there was no way to do a daily review in a few minutes, and so I would often do it less frequently which messes up the whole point of SRS.
Say you want to learn about eigenvectors and eigenvalues. Instead of writing the definition, you might have a card which asks the following:
"Create a matrix whose eigenvectors or eigenvalues are always equal."
This will force you to do two things. First, you will learn from creating the card since you're forced to synthesize the information rather than copy. Second, your recall will force you to work through this newly created problem.
Another general strategy is to just pick a problem from a book that uses the concept. If you ever get the problem wrong, you oblige yourself to solving an additional related problem from the same book (i.e. find the basis of one of the following matrices).
I guarantee you will learn 10x more effectively.
Also beware of falling victim to spending more time on building tools than learning... I've found that pencil and paper go a lot farther than people give credit.