One of the more interesting people at the Wikimania conference was Robert David Steele at http://www.oss.net. He is a former case officer for the CIA, and was an very interesting person to talk to. One of his points was that the intellegence community is too obsessed with “secrets” and is missing a lot of the intellegence that is available in the open. I agree. When the Secretary of Defense says that he has no idea what China’s intentions are, when the answer is in any Chinese history book, and has been the same as it has been for that last 170 years, you know something is wrong somewhere. People are so obsessed with intercepting PLA generals discussing golf that they don’t read the newspapers.
Anyhow, I like listening to professionals because you learn how they think. The thing that I learned from Mr. Steele was that intelligence is not about secrets but about the systematic organization and processing of information. What he was concerned mainly about was data presentation. It was also obvious in listening to him that intellegence is extremely politicized, and getting someone who agrees with you in the right bureaucratic positions is important.
The thing that struck me about his description of the national intelligence process is that it seemed remarkable similar to the business decision making process, or for that matter the geological oil finding business. Business decision making is all about sorting through and analyzing data, and there are benefits in doing it in an organized systematic way. Right now I’m not terribly interested in spying on Iran, but I am interested in finding out what the business conditions are like in a NYC investment bank, projecting the future of my company, or the process the Chinese government used to decide to block wikipedia. Figuring that out seems like it would benefit from reading Mr. Steele’s books.
This brings back some more memories. I once worked for a particularly incompetent oil company, and had a cool idea. There are lots of charts and diagrams that people use to look for oil. Why not put all of those charts and diagrams in a single web page. What’s more the charts and diagrams could have been drawn by existing software which means that they would be interactive. The whole point was to create a “consensus view of reality” from all of the experts working on an oil field. In the oil world, there is a nice expression “ground truth” which is the truth in the ground (i.e. is there really oil there or not).
But like all new and original ideas, this one was crushed into the ground by the powers that be. This also has some relevance to Lawerence Lessig’s talk on Read-only culture. Now that I’m no longer working for that oil company, all of the code and data that could be used to generate this new oil discovery system is now just rotting away useless. I could do something really useful with it, but I don’t have the copyrights. This waste of code, would not happen with open source.