Tuesday, August 10, 2004

I'm reading James Surowiecki's The Wisdom of Crowds, which is a generally useful addition to the broad genre of popular treatments of complexity, emergence, networks and the like. His particular focus is on the decision-making process, as opposed to the emergence of order in systems as discussed in Steven Johnson's Emergence. Setting aside the rather bizarre conception of "rational behavior" as narrow self-interest that provides the straw man for many of his observations about social behavior, there is much of interest for anyone involved in committees, collectives, organizations, etc.

One of the general challenges of the "network form" is how to ensure that the organization maintains some coherence and common purpose via a decentralized structure. Surowiecki addresses this question in chapter 4 of his book.

Decentralization's great strength is that it encourages independence and specialization on the one hand while still allowing people to coordinate their activities and solve difficult problems on the other. Decentralization's great weakness is that there's no guarantee that valuable information which is uncovered in one part of the the system will find its way through the rest of the system. Sometimes valuable information never gets disseminated, making it less useful that it otherwise would be. What you;d like is a way for individuals to specialize and to acquire local knowledge -- which increases the total amount of information available in the system -- while also being able to aggregate that local knowledge and private information into a collective whole. (p 71-2)

That is, the problem is not "decentralization" per se, but data aggregation in the system (that is, in the network). "[A] decentralized system can only produce generally intelligent results if there's a means of aggregating the information of everyone in the system." (74)

"Information aggregation" is a network function in that the structure of the network -- the efficiency of the links, the degree of filtering or friction or consolidation at the nodes -- determines how well information is aggregated. If links are missing, or nodes collect but do not forward information, aggregation will suffer.

"Aggregation -- which could be seen as a curious form of centralization -- is therefore paradoxically important to the success of decentralization". (75) The "centralization" is really the "collectivization" of information, transforming it from local to global information. "It's possible and desirable, to have collective decisions made by decentralized agents."

Decentralization is hard, and teeters on the edge of disorganization. The crucial difference is the integrity of the network in keeping the nodes in contact, and providing the mechanism for information aggregration.

Citing the Iraq army during the 2003 Iraq war, the destruction of the Iraqi command and control system left pockets of resistance -- "all tactics and no strategy" per one British commander. The U.S. Army, on the other hand, is cited as the "true decentralized military" in that war. Individual units were given wide latitude for decision-making, but maintained a functioning communication system that allowed the local experience/information to be shared, aggregated and become collective information.

The chain of command remains essential to the way the military works, and all battlefield action takes place within a framework defined by what's known as the Commander's Intent, which essentially lays out a campaign's objectives. But increasingly, successul campaigns may depend as much on the fast aggregation of information from the field as on preexisting, top-down strategies. (p 77)