Seems that git status nowadays does manage to avoid running the smudge filter, unless the file's stat has changed. This overhead does still exist for other operations, like git checkout.