If one imagines the robot in the example as if it was a perfect functional copy of the human brain plus the human sensory organs, so that the only difference between conceived robot and real human being is the SUBSTANCE it is build up, then I guess Searle would agree that this perfect simulation has in fact “intentional states”. I think, Searle is no substance chauvinist in the sense that he would ascribe intentionality to perfect simulations, if and only if they are made up from the same stuff than brains are mad up. He would rather argue that, if it is possible to build such a android robot, one day, then this robot will be build out of the same substance like brains, just because the brain molecules will turn out to be the most efficient material to build such supercomputers. This is of course and as always no argument against A.I.

I guess Searle would admit, that, in principle it might be possible to produce (not to simulate) with computer chips the very same causal powers than that, possessed by neurons, and even to arrange them in the same functional way as neurons are organized in the brain. But such a fancy robot, which basic hardware components share the exactly same causal powers than neurons do, is no help, because it would be trivially true that such a robot has intentionality (and of course any other property of the brain), since it has the same causal powers as the brain.

In my opinion it is important to specify the used concept of intentionality more detailed, using the notion of “derived” – and “original”-intentionality. What is important, is not solely the fact the computer programs have states that are about something in the world, but that this being about something in the world originates in the computers themselves. In the first sense, any Chess computer can be described as exhibiting intentionality (Dennett’s intentional stance), but only the latter kind is the intentionality in question. How can a machine (how can we?) “produce” intentionality? The problem with Searle is, in my opinion, that he uses a kind of original intentionality that is ways too strong without any reasons. As long as we can explain how brains work, without this strong reading of original-intetionality, that is by only ascribe or impute intentionality to the brain, this should also be enough for computer programs. There is no need to adopt an intentionality-realist point of view.

Before that let me point out that it seems that all philosophical debate can not step beyond a point, and most of the time circles around issues it simply can not resolve. The reason for that is actually very simple, namely as long as we use natural language as the language for our discourse we struggle for clarity. Secondly the notion of an essence of something will usually prevent understanding and many people with no knowledge of the history of philosophy fail to clarify the topic. One can easily find out if somebody is such an naive essentialists, e.g. if he says: ‘X is hard to define’, with X being something like intelligence, life or conciousness. Every concpept is hard to define percisely for the reason of it’s use.

Now, let me come to the main argument of Searle which runs in absolute parallel to the foundation of Computer Science after Turing. The essence is the mechanical rule. To emphasise the argument Searle uses additional words, such as ‘just’, ‘pure’, ‘syntactical’. And of course the chinese room thought experiment is only a conundrum, only a koan if I might say in the tradition of Hofstadter, a paradox.

The crucial thing I want to say is that a the model of computation after Turing makes a very similar assumption, namely that there are atomic rules of computation. And of course his work has led to tremendous progress in computer science, after Shannon’s theory of information showed how one can create perfect information out of imperfect physical systems. A discovery yet to be spelled out is the problems that arise in Artificial Intelligence even today after 50 years of work in connection with the assumption of an atomistic mechanical rule.

143 123=266. This computation takes only a second, but which steps are really necessary for it? This question is very hard to answer, because if I want to know in all detail how my brain produced the output I have to go all the way from perception to cognition, from a brain to neurons, etc. If I type in 143 123 in my calculator the complexity of the operation is not as high but still stagering. For me the so called mind body problem should be considered as Popper has put it. There seem to be different kinds of worlds. He called it World-1 (the physical), World-2 (the concious) and World-3 (the ideal). The seperation of the world into the actual and the ideal goes back to Platon. I think today we have at least some intuition where difficulties come from. They come from the scope of observation.

So how should one understand atomic rules? In reality everthing is physically grounded. I can’t do something which doesn’t obey the laws of physics, whatever in detail they might be (one can never be to sure, e.g. a physicist has recently stopped light). So my the operations of my mind, my brain, are subject to the laws of physics and ultimately only be understood within the physical world. Quantum computation suggests, that NP problems are perhaps computable in polynomial time. After all we don’t want to compute NP hard problems, but do what humans to, because – as you correctly pointed out -, we know it is possible, as long as one assumes humans obey the laws of physics and the input-output mapping is reproducable (which Penrose disputes). If we one takes this stance it is very productive to think about the computation of bodies (Morphological computation). There is now much work in embodied AI which shows that the intelligence of many human actions come from a body-coupling, like walking and even social behavior. I am convinced that higher cognitive capacities are to some extent only reproducable from lower capacities (sensory-motor coordinations and morphological computation in general). For this thesis, there probably can not be a formal proof, but also not really falsified. One form of proof is the proof of concept of engineering, that is if we can build it, we can claim to have understood the underlying processes.

Perhaps I can clarify my point with the following.
Searle’s arguments seem to me more than confused, especially here: ‘a simulated rainstorm does not make you wet, a simulated fire does not burn down anything.’ Well, thinking about flying and flying is not the same either, but one can think about walking and walk (if ‘the I’ has the capacity to do so). So if an AI wants to do something it has to have the capacity to do so. If it is physically static it can not move around and gain different perspectives of objects. If it has no sensors it can not feel and if it has no actuators it can not do anything. Most of our human capacity stems exactly from this interaction with the environment. Understanding of music, comes from the physical interaction with an instrument, understanding of objects in general from our early childhood manipulations. Our communication comes from the interaction with other people. A child alone in his room doesn’t learn anyhting, although once an adult it can study by itself. Our capacity to move comes from the structure of the body in its entirety, not just the brain! Our capacity to make tools comes mainly from the structure of our hands, not just the brain and it’s size. There are many other examples which are now being studied and have been studied by biologists. I belive it is crucial to understand that a human is not isolated, it’s capacities develop from the interaction with others, with the environment and the structure of his body, and is not a brain in the vat.