“Half of the meaningful things philosophy has said about artificial intelligence have already been said by Turing 50 years ago.” I do not remember who said this, and it is probably an overstatement, but it is not far from the truth. Even the AI textbook by Russell and Norvig claims that Turing’s paper Computing Machinery and Intelligence contains “virtually all objections [against the possibility of thinking machines] that have been raised in the half century since his paper appeared.”

Here are the slides for the presentation I held in Tuesday’s philosophy class, in the hope that they may be of some use, even if part of it is incomprehensible for anyone who did not read the paper or listen to the talk:

The problem with informal speculation is that it is easy to be unclear in your writing, and being unclear in your writing usually results from being unclear in your thinking.

Take, for example, my last post, where I was speculating about the properties of recursively improving systems. When I wrote that a system cannot predict a system of greater algorithmic complexity, I did not make clear whether I meant that not all systems of a certain greater complexity can be predicted (which is true) or that no system of a certain greater complexity can be predicted (which is false). For a system to improve recursively with respect to some goal in a way that increases its complexity, there is no need for it to be able to predict all systems of a certain greater complexity. Thus, the whole argument breaks down.

The problem with formal argumentation is that, even if you resort only to the most basic rules of logic, what you prove might not be what you intended to prove.

Take, for example, the paragraph above. The claim that some systems can learn to predict systems of higher algorithmic complexity can be proven formally. You define what you mean by “system”, “complexity” and “learn to predict” in mathematical terms, show an example of two systems, one with lower, one with higher algorithmic complexity, and how the former can learn to predict the latter. From now on, you are free to proclaim that there are simple systems that can learn to predict complex systems. Impressive!

Caveat: Do not mention that you were using the standard definition of “learning to predict” which says that a system learns to predict another system if, after a finite number of observations, the system knows all following outputs of the other system. And, please, stay quiet about the fact that the system that was predicted in your proof did not output anything but zeros after a finite time of complex behavior. Otherwise, people might think that what you have shown has little relation to what is usually meant when we talk about “learning to predict” behavior. And, more destroyingly, they would be right.

As soon as the context changes just a little, as soon your assumptions differ just a little, the value of a formal argument immediately becomes negative. Not only does such an argument say nothing about whether a conclusion is true or false, it will also let you sleep soundly, with the security that there is no need to further think about what you know — it is proven.

Informal arguments, on the other side, cannot provide security in the first place. The truth of informal arguments depends on what you mean by the words you use. Different people associate different meanings with different words, and what was once a discussion soon becomes a game for idle linguists — a fact that is painfully clear if you are doing philosophy. When rational people disagree, even after prolonged discussion, you can almost always trace it back to words being used in slightly different ways.

Your blurry, informal argument based on theorems used out of scope might well convince me. At that point, arguing has long stopped being our joint search for truth. Why bother?