The server does not delay sending until the advertising frame is acknowledged. So, for a large push the client might cancel the push, but for small pushes the client will already have received the data before the cancel can reach the server.

In general, if you are only pushing something that is both small and required that isn't a problem, but when using push for things like MPEG-DASH this becomes a point to think about.