Re:Date parsing performance on iOS (NSDateformatter vs sqlite)
by 0x0 on Saturday September 07, 2013 @07:28AM
(#6344746)
I wonder if this could be improved by just using the standard C library strftime(3) instead of going through sqlite?
[ Parent
| on HN ]
Re:Date parsing performance on iOS (NSDateformatter vs sqlite)
by jurre on Saturday September 07, 2013 @09:58AM
(#6345083)
I was wondering the same thing since that's also what apple recommends in situations like these. This is what I got on the same hardware:
strptime_l took 58.803 seconds
NSDateFormatter took 107.570 seconds
sqlite3 took 7.022 seconds
And with MishraAnurag's suggestion of using timegm instead of mktime:
strptime_l took 21.656 seconds
NSDateFormatter took 108.163 seconds
sqlite3 took 7.096 seconds
Re:Date parsing performance on iOS (NSDateformatter vs sqlite)
by coldcode on Saturday September 07, 2013 @10:03AM
(#6345100)
Why not see what sqlite is doing and do something in C yourself that solves the actual problem. It's not surprising that a general purpose Obj-C (or any language) class isn't terribly fast at one specific thing.
Re:Date parsing performance on iOS (NSDateformatter vs sqlite)
by jurre on Saturday September 07, 2013 @10:19AM
(#6345144)
Yeah that would probably be the way to go ultimately if you're doing a lot of date parsing, I agree!
Re:Date parsing performance on iOS (NSDateformatter vs sqlite)
by MishraAnurag on Saturday September 07, 2013 @10:23AM
(#6345156)
Are you using mktime to get the unix timestamp? That might be the slower part as opposed to strptime.
Re:Date parsing performance on iOS (NSDateformatter vs sqlite)
by jurre on Saturday September 07, 2013 @10:26AM
(#6345167)
I am, source is here: https://gist.github.com/jurre/6475263
My c is quite poor so if you have any suggestions on how to improve I'd love to hear them!
Re:Date parsing performance on iOS (NSDateformatter vs sqlite)
by MishraAnurag on Saturday September 07, 2013 @10:59AM
(#6345242)
I'd suggest using timegm instead of mktime, or set the TZ environment variable to UTC to ensure all implementations return an identical date. I ran the same tests and found that the strptime was quite fast, but gmtime was taking most of the time. To speed that up, you could borrow SQLite's implementation. Checkout the computeJD function from SQLite's date.c - http://www.sqlite.org/src/doc/trunk/src/date.c
Re:Date parsing performance on iOS (NSDateformatter vs sqlite)
by jurre on Saturday September 07, 2013 @11:14AM
(#6345286)
timegm actually already makes a huge difference, thanks! Might be useful to make a small fast date parsing library based on the sqlite source code.
Re:Date parsing performance on iOS (NSDateformatter vs sqlite)
by mjn on Saturday September 07, 2013 @08:21AM
(#6344849)
If date formatting is a bottleneck for me (it is surprisingly often, because it's very slow in some languages) I typically just run it through the command-line program 'convdate' [1] from crush-tools, which is more or less just a wrapper around strptime+strftime.
[1] https://code.google.com/p/crush-tools/wiki/ConvdateUserDocs
Re:Date parsing performance on iOS (NSDateformatter vs sqlite)
by justincormack on Saturday September 07, 2013 @09:29AM
(#6345019)
If it is a bottleneck shelling out is not a great solution...
Re:Date parsing performance on iOS (NSDateformatter vs sqlite)
by pestaa on Saturday September 07, 2013 @09:42AM
(#6345048)
Shelling out for each piece of data is indeed not great.
Shelling out for batch-processing loads of data is on the other hand great.
Re:Date parsing performance on iOS (NSDateformatter vs sqlite)
by mjn on Saturday September 07, 2013 @10:09AM
(#6345113)
Yes, and that's the workload convdate is intended for: it batch-converts an entire column of a tab-delimited file. The larger crush-tools suite is intended for unix-style batch processing of tabular data, but fills in some functionality that the classic set of POSIX tools (cut, sort, paste, join, etc.) didn't cover.