News for hackers. Stuff that matters.
Re:Joel on Software: Anecdotes by pg on Tuesday November 18, 2008 @11:46PM (#369141)
A lot of essays are extrapolated anecdotes. It's not an intrinsically evil form. You just have to do it right.
Re:Joel on Software: Anecdotes by spolsky on Wednesday November 19, 2008 @12:15AM (#369167)
If you have real data, and used the anecdote to illustrate it, you're doing it right.

If you give us the anecdote, but don't extrapolate to wild grand theories, you're doing it right... simultaneously telling an interesting story and giving people more data points to build up their own understanding of how the world works.

If you give us an anecdote, then tag on some theory so that the anecdote isn't mere storytelling, trying to make the anecdote more than it really is, it's deception (at best), or, at the very least, talking about something you know nothing about.

And all these anecdotes and know-nothing books, blog posts, and articles are crowding out real, genuine stories and useful knowledge.

Re:Joel on Software: Anecdotes by DaniFong on Wednesday November 19, 2008 @12:27AM (#369185)
There are many theories that Gladwell's book ties together, and there's some pretty serious data on each of them. I know three, and all of this book I've read are excerpts. Each of his books includes a reading list and citations, and I don't think this will be any different. In particular, page 10 of Flynn's book is worth reading. It is not merely Gladwell's hypothesis.

Terman et. al's studies of gifted children (http://books.google.com/books?id=MXwVAAAAIAAJ&printsec=f...)

Sherar et. al's study of physical maturity in hockey player development (http://www.informaworld.com/smpp/content~content=a778078133~...)

Flynn's book "On Intelligence" and review of the growing IQ of the worldwide population and ethnographic studies of Chinese. (http://books.google.com/books?hl=en&lr=&id=qvBipuypY...)

I don't think one should claim an insufficient justification of his hypotheses without actually examining the justification thoroughly.

Oftentimes, for Gladwell in particular, people claim that he's being unrigorous, imprecise, or incomplete. He's not being incomplete. He's being concise. If he weren't concise, almost nobody would pay any attention. Almost nobody would learn anything from him. And almost nobody would, wanting to learn more, have the opportunity to follow the reference list that he carefully puts out. Which I guess few people do, anyway.

Re:Joel on Software: Anecdotes by jimbokun on Wednesday November 19, 2008 @04:35PM (#369958)
I believe in one of his interviews (the New Yorker?) Gladwell kind of sheepishly admitted that people think he's a genius while really he's just packaging up other people's work with a more engaging presentation.
Re:Joel on Software: Anecdotes by andreyf on Thursday November 20, 2008 @06:58PM (#371225)
he's just packaging up other people's work with a more engaging presentation

Since it's pretty so rare to find someone who both (1) spends the time to find and understands that work, and (2) has the literary ability to engagingly present it, by some definitions, that can be considered genius.

Re:Joel on Software: Anecdotes by calambrac on Wednesday November 19, 2008 @03:09PM (#369848)
You expected any kind of depth, even in criticizing others for a lack of depth, from Spolsky?
Re:Joel on Software: Anecdotes by run4yourlives on Wednesday November 19, 2008 @12:38AM (#369197)
I think you're setting an unattainable goal that would make for an extremely boring read. You might enjoy statistics that much, but most people don't.

Every single one of us is a collection of anecdotes, personal experiences and gathered data. Every single viewpoint ever held by every single person is a theory based on such a collection.

You aren't honestly suggesting that those opinions that are based more on the first two than the latter are completely invalid, thus unworthy of being shared?

Sure, books like the ones you describe - and in fact I'd posit at least 80% of your own blog posts, if not more - are more heavily invested in anecdote than they are proper data. They, and the conclusions they present are nonetheless valid pieces of information and help build a balanced view that is continually evolving.

The world is simply too complex a place to gather hard data on every possible subject matter before forming an opinion or articulating a theory. Some of the most outlandish theories in history - evolution, string theory, climate change all come to mind - start from anecdote and observation prior to being supported by hard facts and abundant data.

Re:Joel on Software: Anecdotes by spolsky on Wednesday November 19, 2008 @12:47AM (#369203)
Probably true. I'm probably suffering from some kind of temporary airport-bookstore-induced dementia which is making me particularly intolerant to this particular genre.
Re:Joel on Software: Anecdotes by run4yourlives on Wednesday November 19, 2008 @12:52AM (#369212)
Well, I'll give you the recent explosion of opinions on how our world works, that's for sure. It's getting people to do what they should have been doing in high-school though, which is think about why things are the way they are.

There are different stages of the journey, I suppose.

Re:Joel on Software: Anecdotes by bokonist on Wednesday November 19, 2008 @12:34AM (#369192)
Most non-fiction books and essays are collections of anecdotes plus a guiding interpretation. Data (as in statistics) is hard to come by and rarely reliable. And without the interpretation, you simply have a bunch of mindless facts. The world has an infinite number of facts. Selecting facts and arranging them in a coherent interpretation is what most writing is about. Even the driest history textbook is mostly a work of interpretation.

Still, something about both Friedman and Gladwell really bothers me. I think it is simply that I believe their theories are mostly wrong or useless. It's not the style or methodology that is the problem, but the results.

Re:Joel on Software: Anecdotes by timr on Wednesday November 19, 2008 @12:50AM (#369207)
Essays aren't science, and shouldn't be held to the same intellectual standard. They don't have to be self-contained packets of truth, because we are reading them to be entertained, and are supposed to be discerning consumers of information, capable of filtering out others' over-generalizations and strained metaphors.

Obviously, it's best if an essay is both truthful and entertaining, but if an author decides to trade scientific precision to achieve a more vibrant story, that's fine with me. The world is far too big to stick to the facts.

Re:Joel on Software: Anecdotes by bpyne on Wednesday November 19, 2008 @10:56AM (#369543)
I'm not trying to be a wise guy but can you define "real" data?
Re:Joel on Software: Anecdotes by mechanical_fish on Wednesday November 19, 2008 @11:42AM (#369611)
In this case I think it's a cargo cult phenomenon. Sometime soon after World War II our culture decided that wearing a long white coat, having a Ph.D., and publishing "real data" was the touchstone of truth in every field. As if demanding "real data" from an essayist wasn't like asking your fiancée to take a battery of psychological tests before you'll marry her.

Gladwell isn't a scientist, as he has said many times. He is inspired by scientists (among other people), but what he actually writes are book-length essays. You're welcome to argue against an essay -- that's what the form is designed to provoke -- and you can do so in several ways: You can write an essay of your own, or you can do a scientific study, or you can even perform a stunt:

After we came out of the church, we stood talking for some time together of Bishop Berkeley's ingenious sophistry to prove the nonexistence of matter, and that every thing in the universe is merely ideal. I observed, that though we are satisfied his doctrine is not true, it is impossible to refute it. I never shall forget the alacrity with which Johnson answered, striking his foot with mighty force against a large stone, till he rebounded from it -- "I refute it thus."

But the modern fashion is to respond to an essay that you don't like by demanding that it dress itself up like a Nature manuscript and then criticizing the flaws in the costume. Which is silly. The more elegant thing to do is to argue with the essay. Write an essay refuting Gladwell. Quote some scientists of your own in the process if it helps. Obviously, debating a guy with the writing chops of Gladwell is a tricky task, but it's better to try (or to remain politely silent and wait for others to try) than to flail around ineffectually.

Or, if you prefer scientific debate to essay-writing, I'm told that Gladwell seems to be trying to provide more footnotes to the science that inspires him. Follow one of those footnotes to the scene of battle, grab a handful of related journal articles, and join the fray. Don't be surprised to find every other scientist on earth alongside you, though. Scientists have a tendency to swarm fields which have been popularized.

Re:Joel on Software: Anecdotes by apu on Wednesday November 19, 2008 @12:08AM (#369160)
But it seems that to "do it right," the writer must be able to pick the right anecdotes and draw the right conclusions from them. This is probably close to impossible if he's not an expert in the area. And if the writer's an expert, then he should be able to provide other (non-anecdotal) evidence to support his claims as well.

I think that's why many of these books are so frustrating to read -- they seem to rely almost completely on anecdotes, without any sort of underlying data to look at long-term/overarching trends/themes. Thus, instead of saying, "here are various data, including these representative anecdotes about some of them, and so we can see that my theory is true," these writers usually say "this is my theory, and look how well these carefully chosen anecdotes match it!"

Re:Joel on Software: Anecdotes by ntoshev on Wednesday November 19, 2008 @05:45AM (#369364)
A lot of expertise is not based on scientific statistics, but rather on the sample of problems the expert encounters and his intuition.
Re:Joel on Software: Anecdotes by jackchristopher on Wednesday November 19, 2008 @01:03PM (#369713)
It's implicit in the form. Mediums can bend but they break. The hard part is figuring out where.

Essays make you generalize; they're short. But generalizations need interpretation.

And if you really want to know something, you read a technical paper or do it yourself.

I could go on about this in comment but won't; Comments want to be witty.

Re:Joel on Software: Anecdotes by Agathos on Wednesday November 19, 2008 @03:21PM (#369864)
You mean, "I was on a bus, complaining about Thomas Friedman, when the driver looked back and said, 'A lot of essays are extrapolated anecdotes. It's not an intrinsically evil form.' And I realized it was true. You just have to do it right."