To protect the information it in case one invididual gets compromised.
> I bet that even at "Top Secret" levels, the narrowness of view is stultifying.
Yes and that often makes work seem pointless and unrewarding. Invididuals migth get to view a very thin slice of a large project. So they might be designing an algorithm or a widget without knowing how and where it fits in the big picture.
I am guessing for some projects this compartmentalization leads to the dilution of guilt. "Hey I only build detonators", "I only build the shell", "I build the remote control", "I only leave the device in a certain place without know what it is". But all these people could be making something that would hurt or kill someone.
So, while there are implications of need to know they are IMO less significant than capable of understanding.
The pyramid structure that seems to permeate all things //system// has the following salient feature. Say a system pawn is a node at level $L$ and his/her boss is at level $L+1$ and the boss' boss is at level $L+2$. Depending on social skills and intelligence, the pawn might have information about "what is going on" at level $L+1$ by talking to the boss, but for sure he/she has //no// information about what is going on at $L+2$. In a way secrecy (i.e. information non-awareness) is the //essence// of the pyramid.
Assange has this paper [1], which talks about information being the perfect way to choke the system.
[1] cryptome.org/0002/ja-conspiracies.pdf [ abstract: Consider the Gonspiracy G = (V,E) where V is the set of conspirators and E is the set of inside information links. The conspiracy G is embedded in a larger graph, society, S. Let |G| be the power of the Gonspiracy. The 'good guys'[2] want |G| to be small, the bag guys want |G| to be large. Assange defines the total conspiratory power as
|G| = sum information flows in e for all e in E
So to fight G, one must either cut edges e, or generally
reduce the flow of information flowing on edges, by
scaring the nodes {v \in V} that they might be found out.
]Seems like that would inevitably lead to everyone in the organization unable to determine whether or not their contribution was useful. (Short of them actually saving a life at the bottom level.)
With no hope of finding meaning in your job, it would make a 40-year career to retirement a long career indeed.
I can also think of a situation where a contractor might hire a mid-level manager's husband, wife or child. The mid-level manager would be able to crank up his or her headcount to get the relative a sizeable Christmas "bonus". I've heard tell that this sort of thing actually happens.
There are, however, members of the intelligence community who have the breadth of view to make good judgment calls. Just not enough of them, and no accountability.
But why would you divide up information into 15 or 20 categories? I bet that even at "Top Secret" levels, the narrowness of view is stultifying.