What the farmer in your example cares about is the legal classification of tomatoes in context of subsidies and taxation. That's an entirely different thing that biological, or culinary classification, and needs to be explicitly separated. If it isn't it leads to idiocy.
Real life examples of such inane reasoning can be seen with some very popular complaints about "stupid EU bureaucracy" - for example, that EU "decided that snails are fish" (they aren't, they just get bucketed with fish for some trade deal). People really do use it as a serious argument.
And sure, instead of legal classifications, one can discuss ethics related examples. Such as what makes for the good life? Something as simple as saying, the good life is based on some common sense notion of good or the good life is different for everyone and dependent on unique personal circumstances, already assigns numerous categories implicitly. That doesn't mean that both of those notions of the good life cannot be right. It does mean that we need categories to discuss and reconcile their differences, if we wish to have a pluralistic and liberal [in the 19th century sense of the word] society.
Even something as minor as the category of a tomato, would matter very much to a tomato farmer when, hypothetically, all vegetable farms qualify for a higher level of government subsidies than fruit farms. Or to a retailer when, again hypothetically, fruits and vegetables are charged different sales tax rates.