Sure, eating better has a huge impact. But you miss the main argument why physical training helps in weight loss: Physical training increaes your resting metabolic rate by ~10%. https://www.ncbi.nlm.nih.gov/pubmed/11283427

Since you are not doing sports 95% of the time, this has a lot of leverage. And you can build muscle with 30min of training per day easily.

Also, you please check whether you mean kilocalories when you say calories.