String tokenization in C

Re:String tokenization in C by Someone on Saturday December 15, 2018 @10:35AM (#18688706)

> A few strtok() calls can easily parse lines like:

     keyword=value1, value2, value3

The challenge with parsing isn’t parsing correct inputs; it’s generating useful error messages and recovering on incorrect inputs such as

     keyword=,,value1, value2, value3,,,,

or even

     =keyword=,,value1, value2, value3,,,,

strtok isn’t the best tool for doing that.

(Yes, those could be valid inputs, but if they are, chances are they should be parsed differently)

[ Parent | on HN ]

Re:String tokenization in C by yason on Sunday December 16, 2018 @06:33AM (#18692681)

That's what I meant: strtok() is well-defined in what it does. The man page is really short. I don't understand people who complain it doesn't do something it's not supposed to do. Yet it's very useful in what it does.

Your mileage may vary but it's also a common issue having to filter out empty items out of something like ",foo,bar ,,,,baz,,xyzzy," where you only really want those four words. You will especially encounter this in parsing user input which might have extra whitespace, or badly formed lists of items.

Use strtok() to split C strings into tokens delimited by the given set of delimiters. If you want to catch each comma but skip over any whitespace and split at the first '=' only, then use something else.

Let's say if I'm going to need a configuration file for my program I'd most certainly start with something I can parse with strtok(). I would really need very specific needs to warrant a more complex format that would require a more sophisticated parser in which case using one would be a no-brainer.

[ Parent | on HN ]