Here is how to connect the dots: holding risk requires finding out information about the risky thing, in order to quantify the risk or take action to avoid it. The reason farmers aren’t the best holders, boils down to the fact that farmers are busy.
The farmers in Lincolnshire do not have time to do weather modelling for the next eight months in Oceania, or monitor the regulatory capture of the Australian Wheat Board, or analyse the litigation around irrigation in the Murray-Darling Basin, or to make submissions to the New South Wales parliament on climate credit schemes for land use change, or analyse the impact of new regulation of certain kinds of pesticides in NSW.
Human activity is very complex. A lot of factors go into the supply and demand for wheat, and the information needs to come from all corners of the Earth. If you want that information in the hands of farmers in Lincolnshire, to be able to best choose what crop to plant and when, you need a transmission mechanism for it. Price is a very effective mechanism, because all sorts of people who don’t know anything about wheat are incentivised to participate in sending the information onwards. Yes, some of these people are pure speculators, but they do actually serve a function. A healthy price transmission mechanism kinda needs these people in there, even if it’s distasteful. If you don’t want them involved, you basically need another information transmission mechanism, because it doesn’t work without them. If you think you can do better without them, good luck with the famines, because many, many people have tried and that’s how it tends to go.
Where do you draw the line between (useful) arbitrage and "pure speculation"?
Much of what is commonly known as speculation is actually an important mechanism for price quality or liquidity.
Obviously there are limits, and there are ample opportunities for making a one-sided profit without regulations, but people often seem to miss the value that arbitrageurs tangibly provide to them: Being able to exchange foreign currency at very tight spreads almost 24/7; being able to buy and sell even not commonly traded stocks etc. are often a function of that.
Are you familiar with the arguments of (more popularly) Aaron Brown and (transitively) Jeffrey Williams?
Essentially, the idea that a farmer would be an active participant in a futures market is quaint, but the vast majority of activity is speculation. This is not a contradiction of your point, but an elaboration of a counter-intuitive part of it.
One might look at a futures market and see that well over 98 % of the activity is buying and selling by people who never have any reason to care about wheat other than for the possibility of its price going up or down. But this large-scale speculation is precisely the thing that makes it possible for a farmer to hedge (by providing liquidity and a motive for the counterpart of the hedge) or, as Williams' points out, perhaps more commonly "take out loans in commodities" for their convenience yield.
Essentially, the Lincolnshire farmer can lock in a price with a plain forward contract. However, that does take a double coincidence of demands (or whatever the phrase is) and the standardised nature of futures contracts help avoid that problem.
But! The most common use of futures contracts (aside from speculation) is not (or at least was not, when Williams wrote his book) hedging, but effectively borrowing and lending in commodities.