I drink a lot of coffe, but with sugar. I've tried slowly quitting sugar in coffee, but that only made my stomach ache and drinking it was very unpleasant. Without coffee, I would have to make some other caffeine providing solution, without it it's too hard to focus on coding for me (I can live without coffee just fine, I go cold-turkey for two weeks every year, but it's holdiays and I'm away from computer, so no need for coffee).
Coffee is best with a meal; it raises the stomach acid, which improves digestion of the meal. Plus you don't get the cortisol spike when it's taken with a substantial meal.
Definitely wean yourself away from adding sugar to coffee. I have it without any sugar, but a bit of cream, and nible a pastry along side drinking the coffee. So if you can nibble on something sweet, absolutely skip the added sugar in the coffee.
I definitely do think that the best diet is the one you can stick to. I've tried a bunch, but I've learned that the thing that works the best for me is learning how to eat to manage hunger rather than managing calories. Managing calories while being gnawingly hungry is just miserable, as you've attested to.