Re:Ask HN: I need to categorize 150,000 products
by caw on Saturday August 18, 2012 @11:56AM
(#4401427)
You're more familiar with your data, but the last word doesn't need to be a hard and fast rule. The last word there looks like a product code. However, the first word looks like a brand.
If this is more of an exception, I'm sure you can generate a pretty small list of what falls into that. If you regex for some number of volts with some mAH, it's probably a battery or a motor. That at least reduces the number of things you have to manually go through later to clean up the data.
Re:Ask HN: I need to categorize 150,000 products
by 1123581321 on Saturday August 18, 2012 @11:55AM
(#4401425)
Glad it's helpful. In cases like that one I suppose you look for clues and have another script to assign categories. For example, if most batteries have voltage in the title you can regex for \d+V\w (or whatever) and assign battery to matches. Could become complex quickly, though.
OUTRAGE 5C NRG35 3S1P 11.1V 800mAH 35C NRG355C-8003
That's a LiPo battery, and should be classified under "Batteries", but nothing in the title explicitly states that fact. You can tell it's a battery because of the parametrics (11.1 volts, 35C discharge, 3S, etc) but none of that would show up as a categorical classification.
Then again, I probably want sub-categories in batteries for those parametrics, so perhaps I just need to allow categories to be nested under other categories and manually assign tiers later.
Thanks for the suggestion, I'm going to play around with this!