The US didn't experience its dramatic economic growth in the 1800s by assembling a large and disciplined army; instead, it disassembled large and disciplined slave plantations, moving the agricultural center of the country to smaller, more productive farms in the North. (Farm productivity per acre continued to vary inversely with farm size, worldwide, until the 1970s.) I think it was its network of what we would now consider small industrial companies, coupled with its relatively low level of violence (due to geographic isolation), that enabled it to outstrip many of the Great Powers in economic growth.
By contrast, during the Edo period, Japan was an extremely large and disciplined organization --- which produced some remarkable art but also made it militarily and economically weak.
So I think the picture is much more complex than, "The success of a society was proportionate to its ability to assemble large and disciplined organizations." I suspect Drucker and Coase's insights may help to provide a more nuanced view.
Edo Japan is a pretty unique case. A famous historical curiosity, in fact. And indeed the power latent in Japanese society became visible in its unprecedentedly rapid industrialization in the second half of the 19th century.
In your essay you predict that "large organizations will probably never again play the leading role they did up till the last quarter of the twentieth century". Are you suggesting that average company size will revert to a level lower than even 1900? Or are you saying that average company size may remain roughly the same, it's just that economic dynamism will come from startups, rather than corporate laboratories?
When you say that traditionally, bigger organizations were better, do you mean they were better:
a) militarily b) in developing new technologies that make lives better c) in producing goods like trains and cars d) better at generating revenues/profits e) better at attracting the best and the brightest f) producing achievements that are remembered by historians g) all of the above
It is important to clarify this, because some of these measures of success oppose each other. For instance, the Egyptian governments were great at f) but at the expense of b).
I at first took your essay to define success as b) improving technology. But if so, the thesis is obviously wrong. Most of the great inventions of the industrial revolution were created by tinkerers and small enterprises, not large organizations. Just look at the development of the textile technology http://en.wikipedia.org/wiki/Timeline_of_clothing_and_textil... . Almost every single invention from 1600 to 1900 was developed by a startup (and many of these startups were funded by outside investors, Kleiners Perkins has nothing on the Boston Associates http://en.wikipedia.org/wiki/The_Boston_Associates ).
Is there a particular book you read that discusses this idea? I'd be interested in reading it.
It seems that you have a different idea than Coase about what limited the size of companies; he believed that companies stopped getting bigger because as they got bigger, they started becoming less "successful", in the sense of "profitable". What's your alternative explanation?
How are you defining success? How well the company did per-capita (per employee)?