What kind of scanner are you using? This sounds like a great idea.

I'm doing something similar and I'm using the Fujitsu Scansnap S1300 which comes integrated with Evernote support.

Wow I wonder how many are this organized/anal about tracking stuff. My bills sit in a pile on the table until the first of the month, I pay them, and throw them in a box, never to be seen again.

What value is there in scanning your electric bill and archiving it digitally? When have you EVER needed to refer to something like that after it's been paid. For me, never.

I found it surprisingly satisfying to get rid of paper completely. As a self-employed person, I need to keep most receipts including house expenses (for my home office). It's a lot easier to just scan everything as it comes in as opposed to thinking about whether it's worth saving, putting it in a box or file folder, then sometime later think about whether or not it's time to dispose of it. And you never know, maybe some day my grandchildren will be amazed to learn how cheap it was for grandpa to heat his house back in the early 21st century.

When I'm doing my taxes.

Does it allow me to keep a budget and track expenses/info/receipts against that budget?

Unfortunately no. I use Mint for tracking income and expenses (at least the ones I pay with cards or online) but it doesn't integrate the receipts.