In the absence of an IOMMU configured to block GPU accesses to non-GPU memory regions, yes - however, drivers should block any request to do so.