You can rent and stay in Rosslyn Castle, very close to the chapel. I stayed there once, and it was one of the neatest places I've been. The only shower is on the way to the dungeon.
http://www.landmarktrust.org.uk/search-and-book/properties/r...
http://en.wikipedia.org/wiki/Rosslyn_Chapel
Rosslyn Chapel is also famous for apparently housing the Ark of the Covenant and/or the Holy Grail... :-)