No. An X11 connection forwarded with -Y can receive all keyboard events, capture all screen pixels, inject mouse/keyboard events and read/write the clipboard. A malicious X11 client app can totally pwn you, forwarded or not.