"Was this ever a problem, or was it overcome?"

The problems Schneier is talking about have very different characteristics than the ones in this article. So the problems Schneier is talking about are still very real. But solving little pieces of the problem is definitely possible, especially with a good amount of human mediation.

Human mediatiation is key. Skilled intelligence analysis made by well versed operators. even then it is very hard.

As one intelligence analyst once told me " it is looking for a needle in stack of needles"

Ahh, so difference in data mining is that it implies there's no human in the loop.

Whereas palantir situates itself as a tool for intelligence, rather than as an oracle.

Most Palantir installations don't automatically generate or analyze the graphs. It's installed as essentially a very large, hand curated, semantic graph store and retrieval system with a nice client interface. Most of the work done with Palantir is manual.

If you watch some demos on their site you'll see that the workflow is almost entirely manual save for the moments when they retrieve sections of the semantic graph from the datastore.

Also notice that they never talk about the excruciatingly long process of how that data got jammed into the system. Imagine the target goal for an entire day's work for one person is to import 10 news articles into the system and you'll understand why they aren't doing the kinds of things Schneider was talking about (even if their brilliant marketing and sales departments sell it otherwise).

edit: word on the street is that they are still cashflow negative which is very worrisome considering all the fundraising they've been doing + they've almost saturated the government market

Finding out "Who is a terrorist, and what are they going to bomb?" is a much tougher problem than, "What information can I glean from Facebook, MySpace and Twitter to indicate this person might be a higher credit risk?"

For instance, I imagine a high number of updates on a Facebook profile containing the word "drunk" might lead to a higher risk profile for auto insurance.