Ragel (or more like finite state machines) excel at parsing streams with all sorts of nuances while still being rigid and bailing on invalid input. Considering the ken_all.csv isn't really standard csv anymore, it's better to write tailored parser for it, than to approach it as a typical csv file.