'directly on the external interface of my home router' is my ISP. Do I trust my ISP/government/whoever-bugged-in-at-the-curb enough not to add some wide route and scan 10/8,192.168/16, then jump into my LAN? In fact I've done precisely the inverse: on the outward side of a previous ISP of mine lies both a globally routable IP address and a bunch of networks of theirs on some 10/8 range, with... interesting properties. Notably that 10/8 can SSH/telnet/web-admin as root into my ISP-provided router/modem. Maybe even with some additional trick I could reach to other subscribers, hack their modem and break into their LAN.
Nothing prevents you to drop outward packets whose source don't match your fixed (or DHCPv6) IP layout (e.g 2a01:whatever/126), since auto-configuration has stuff like ff:fe in the middle. Or you could leverage autoconf and drop packets from IPs not matching your list of MAC addresses. Also you can not send router advertisements nor respond to router sollicitations, or even filter out RA/RS packets, so that no autoconfiguration take place.
In fact, you should do this to prevent spoofing attacks from being sourced from inside your network.
That's true if by 'outside' you mean 'directly on the external interface of my home router'. Because if it can indeed route my 10.something/192.168.something internal network, all the routers in between most likely can't. Maybe that's why because think NAT is safer: firewall holes are, in practice, much harder to exploit in IPv4.
(Well, that, plus the firewall rules are much simpler. A few fixed ranges to drop, and be done with it. In IPv6, all has to be dynamic, and potentially configurable)