I’ll be exploring a combination of pre-processing such as stemming in order to also reduce the token length while preserving as much important information.

Thanks for the feedback.