I actually built something exactly like this, using the google calendar API and FUSE. It wasn't very difficult at all, I think it took half a day. I'll try to find the code, it's been about a year since I used it since it was just a toy and I didn't want to maintain it.
https://github.com/dengste/org-caldav does just that
I've been looking for something like this that will let me edit in plaintext but also sync both ways with gCal for the benefits of plaintext but also the caveats that you mentioned.