Just as map and reduce in Python is "unpythonic", interfaces that mint new temp strings to process what should be a simple charstar are un-c-like, and should be avoided.
That's not what I'm doing at all, though. I'm just giving char *'s and size_t's for portions of the original character array, not duplicating subsections of the array at all.
You are, in other words, reinventing strspn?
strspn takes a set of characters to match against and tells you how long to go until you reach a non-matching character. My tokenizer, like strtok, takes a set of delimiter characters. So same thing except with complement sets, I guess.
It's also part of a larger library that consistently uses pointer/length pairs to designate strings as opposed to null termination, which admittedly is quite non-C. No clue whether it's actually a good idea.
EDIT: If anyone's actually curious, I have it on github: http://github.com/philwelch/string
Just as map and reduce in Python is "unpythonic", interfaces that mint new temp strings to process what should be a simple charstar are un-c-like, and should be avoided.