A lot of these features are normally handled by other services. So for videos, images, and gifs, these can be represented by a URL that's pasted in the channel that you can view with a web browser.
The advantage of doing it this way actually makes it easier to scroll back in the chat history when you're searching for something that was posted.
- messages sent to you while logged out are lost
- no support for videos, images and GIFs
- (usually) not great support for emoji
- no support for file attachments
- no useful mobile clients unless with support from the server
Yes, those can be considered frivolous features, but they are used by people to express themselves (which is, in essence, what a chat protocol is used for), they are expected because IRC's competition has them, and some of them are an objective usability improvement that I'm not willing to go back on.