spflueger

Exactly. I also feel like being able to choose a model for the use case could be worth an idea. So instead of trying to squeeze all kinds of knowledge into a single model, even if it's moe, just focus models on use cases. I bet you only need double digit billion parameter models for that with same or even better performance

Quite some years ago there was some data backed evidence of this I would say. https://www.dkriesel.com/blog/2019/1229_video_und_folien_mei...

Here is the link to his talk directly https://youtu.be/0rb9CfOvojk?si=7EImZU9x4zFb6LSf

Its true that the network is quite dense and used by also cargo trains, but there is no denying that things got worse and worse. I constantly experience delay to do some stuff not working. I forgot even the minute threshold when a train is still punctual according to DB. I believe it's 10min by now, which can be deadly if you need to switch trains :)

Looks interesting. The 12GB minimum RAM requirements seem quite steep though. Why so much?