It doesn't work with the Colemak input method that comes with OS X though. I don't think there's any way to change caps lock to delete in a keylayout file, but you can always use PCKeyboardHack.