There are many layers of HIPAA outside of of just electronic data security. You must train your staff on security protocols (e.g not sniffing for celebrity records, etc).

I am pretty sure even if your office is all paper charts you still fall under certain HIPAA guidelines such as notifying patients if there was a breach (e.g physically stealing records from the office). This happened in Rocklin, CA a few years ago.

http://www.cda.org/news-events/burglary-leads-to-lengthy-hip...

Yes. Thats the role of regulation, to entrench the incumbent and stifle acceptable innovation.

Plus add to what you mentioned that you need strong disk encryption and some physical security. Else an external key logger and its over.