You also seem to be overstating the importance of exact dates when it comes to agriculture: my experience is that +-13 days does not make a practical difference, especially if it slowly accumulates (it's not like you would suddenly have to do the harvest 13 days later from one year to the next — you actuslly had to do it 10-13 days "early" once the calendar was switched, and it didn't make a difference even then).
If human civilization continued using the Julian calendar, we wouldn't have been any worse off: nothing points at it that we would have been. I am not saying that Gregorian calendar is "worse" at all (though those born on Feb 29 might beg to disagree when they go 8 years between birthdays 2096-2104 :)), just that where it's better does not matter much.
Earth's rotation is — arguably — better measured against the stars, so a sidereal day is more "correct", yet it would not map to our daily routine at all.
So I believe the question is not what is more "correct" (we've long established that we are dealing with approximations at best, Julian calendar included: it already had leap years), but what is more useful? And that's what has driven adoption of the Gregorian calendar in most of the world, but I am fine if somewhere it's more useful not to have to worry about did some days just disappear at some point in the past.