Relating the Sciences: A Compression Theory of Interscientific Reduction
8 comments 
In our project of understanding the world, we have created physics, biology, psychology, and a number of other disciplines. Now we want to turn our project into a rational one, a science that does not only find good hypotheses about the world, but that does so effectively. This requires that we first understand science itself: How do different sciences relate to each other? Why are there different sciences in the first place? And, within a single science, why does it look like we can distinguish ordinary science from scientific revolutions?
One way to ask how biology, psychology and elementary physics relate is the following: Given enough time and space to write, could we translate each biological statement into a statement in terms of physics that is true if and only if the biological statement is true? Likewise, can we translate psychological statements into statements about the physical states and processes of a system? Can psychological statements be translated into statements about biology?
Here is the gist of my thoughts:
The laws of a correct theory of elementary physics must be able to compress complete descriptions of system without loss. A complete description is a description that, in principle, would allow you to recreate the system exactly. It contains all the information there is in the system.
In contrast, inexact physics and special sciences like biology and psychology are lossy compressors of a system’s complete description. Given a lossy description, you might be able to restore certain features of the system, but never recover it completely (except for some degenerate systems).
To make an analogy:
A .png file compresses an image without loss — given the file, you can recreate the original image on the screen perfectly. The price you pay for this ability is that your file is relatively large.
In contrast, lossy image formats like .gif and .jpg create smaller files and they allow you recreate certain features of the original image, but usually do not recover it completely. For example, .jpg is more faithful to the original colors, .gif preserves edges and structural details better, but neither can restore the initial image completely.
Consequently, taking a .jpg image and saving it to .gif will result in loss both of the information that .jpg does not preserve and of the information that .gif does not preserve.
As may already be clear from the analogy, there are implications of the compression view for the relation between the sciences:
Statements from the special sciences and approximate physics can be translated into (possibly very long, disjunctive) statements about elementary physics without additional loss. However, such a translation will not make the statements more exact — information that is not there in a statement from one of the special sciences won’t be there in its translation, so what you get might be something like “it looks like physical situation A, or like physical situation B, or like physical situation C, or …”.
Statements from the special sciences can be translated into each other, but this will result in additional loss of information. In effect, what we need to do here is to first translate into exact physics (without additional loss) and then recompress into the target special science (with loss). Since different special sciences usually keep different structural details of the state intact, such a translation will usually throw away information. The more different the features that the two special sciences keep, the more loss we suffer.
I say usually since it is conceivable that statements formulated within a certain special science contain strictly more information than the translations within another science, just like statements in a correct elementary physics contain strictly more information than any of the special sciences, and just like a .gif format with 8 bits of color information (256 colors) contains strictly more information than a .gif format with 4 bits of information (16 colors). If you’re a philosopher, you might say that the latter, more coarse theory/format supervenes on the former, and if you’re a daring philosopher of mind, you might hypothesize that the relation between biology and psychology is just like this.
Why are there different special sciences? Why do we need special sciences at all?
The analogous question can be asked about image, audio and movie compression algorithms, and here the answer is clear: We don’t have enough space for lossless compression and in the end all we care about are certain features (and in different situations, we care about different features). In the case of audio compression, we only care about sounds within the range of 20 Hz to 20,000 Hz since everything else isn’t perceivable by the human ear.
Similarly, when we want to describe a phenomenon using scientific theories, we cannot use elementary physics as it takes too much time and space (although that would give us the most accurate predictions) and in the end, we do not care about all aspects of the phenomenon equally anyway. In thinking about how the brain works, neuroscientists do not look at the brain as an arbitrary physical system whose behavior is to be predicted, but instead it is certain aspects of this system that they try to explain. The aspects we care about tell us how to compress our observations into theories, and since we are not always interested in the same aspects, we need different ways of compression: different special sciences.
The compression view also gives us a way to think of the difference between ordinary science that discovers new facts within a framework and scientific revolutions that bring conceptual change:
Ordinary science is the process of finding out how the compressed version of interesting situations look like and how lossy the compression is when we apply existing compression algorithms — theories — to different situations. We smooth out small bugs in the compression algorithm, but fundamentally, we don’t change our framework: we use the existing compression algorithm.
Scientific revolutions change how we compress our observations: Every reasonable revolution either improves how strongly we can compress (e.g. by showing that what we thought of as different phenomena can be explained by the same principle) or makes our compressions less lossy (e.g. by replacing a black box term like elan vital with a structured theory). Since compression and prediction are two sides of the same coin, another interpretation of scientific revolutions is that they change the prediction algorithm whereas ordinary science mainly makes and checks predictions.
Now you be the judge how lossy this view on science really is.
Posts
Interesting view. However, I would like to object to two premises: Firstly, the speculation that there is such a thing as a “lossless” (complete) description. My (laughably superficial) understanding of quantum mechanics and its inherently probabilistic nature makes me doubt that we can have useful and meaningful descriptions at the downmost level. (Anyone enough of a physicist around here?) Secondly and relatedly, the implicit conjecture that there are no such things as emergent phenomena. The relevant information we’re looking for is not always even present at lower levels. (Anyone enough of an information theorist?) And I think this is one of the hallmarks of scientific revolutions: Finding the appropriate levels of description—which is not always a process of abstraction (or “compression”).
(Yeay vague speculation :))
I suggest a third figure to illustrate some consequences of your idea in the context of memory limitations.
I am not at all sure about the fourth item. Maybe I should have inverted the colors for good measure. ;)
Hendrik: If you think that it is not only beliefs, but the world itself that can be uncertain, then your most fundamental description would only allow you to restore a state up to whatever uncertainty there is (which is not a problem for the compression view). If you’re suspicious of the notion that the world itself can be uncertain, then the “many worlds” view of quantum mechanics may be for you.
Regarding emergence: Reality only has one level and if this level does not contain certain information, no redescription will take care of that. Different people use the term “emergence” to denote different things, but I have yet to see a situation that is both real and such that viewing it as compression is not an improvement over opaquely calling it “emergence”.
Johannes: That’s a great illustration! I inlined it to get more people to look at it.
I’d love to discuss this over a couple of blueberry common senses… (Although I shiver with fear of the elementary physics description of my gustatory sensations)
(That’s because we are all still confused about qualia and consequently cannot properly integrate them into our model of the world.)
Here’s a question/comment motivated by my current main occupation:
How do you think language relates to all that? I know, linguistics is probably not what you mean when talking about science. But if statements in the language of elementary physics should compress all information about a human being without loss, then also language with all its complexities should be included in this “image”.
But how do you express in a physical statement the fact that this and that word refers to this and that entity, or even worse – that this and that sentence refers to this and that state of affairs. And how do you express all the relations that hold between the different words themselves. The multitude of languages in the world, and the seemingly infitine amount of meaningful forms that can be constructed in them are certainly difficult to pin down in a single (no matter how long!) physical statement.
In my opinion, this is not a problem that tedious work in the hard sciences could solve but rather a fundamental shortcoming with the approach you suggested. Meaning is something virtual, it does no exist in the physical world as such, but is actively created by living beings – it is a relation that holds between a living being and something in its environment. But how do you describe an arbitrary relation, be it as simple as the relation between the word “cat” and the animal cat, in physical statement? Actually, includeing the latter two would not be enough – you also should include the interpreter to whom this particular word has this particular meaning… But if the theory of elementary physics cannot describe this tripartite relation, it cannot compress the “images” of living beings (especially humans) without loss.
I am very curous about your feedback;)
Human language is part of our world, and in as far as linguistics is an empirical science, the compression view certainly applies.
To me, your question is part of the larger program of explaining meaning, morality, consciousness, and other seemingly nonmechanistic things within a mechanistic framework without losing them in the process. For one such attempt, see Gary Drescher’s book “Good and Real” (which does not get everything right, but is an interesting read nonetheless).
The caricature version of my view on how language and reference can be explained within a mechanistic framework goes like this:
As you mentioned, the meaning of a word is not a property of that word, but of an interpreter. The same word may have different meanings for different interpreters. To be an interpreter, you need to have a model of the world that is constructed from concepts and these concepts need to have names.
A concept is not unlike an entity in a formal scientific theory — its computational structure mirrors certain structural features of the real thing such that you can derive useful statements about the real thing by performing syntactic operations on the concept in your head. By combining your concepts in a way that mirrors relevant structure of the world, you build a model of the world in your head that you can use to reason about the world by purely syntactic means.
When you want to say what the word “cat” means to an interpreter without using intentional vocabulary, you remember that, when you learned the word “cat”, what you learned was to associate this string of letters with your prelinguistic concept of a cat (added: or possibly it was the co-occurrence of the string of letters and real cats that enabled you to learn the prelinguistic concept and the word at the same time). Now, whenever you see the string “cat”, this concept gets invoked, allows you to pose queries about its properties and to use it in reasoning. Using your continuously updated model of the world, you can even determine whether there is a cat in any given situation by checking whether your concept of a cat is part of your model of that situation or not.
The same holds for references to states of affairs, although here you may additionally have the compositional structure of your input string that gets mirrored into a compositionally constructed model of the situation in your head.
If I want to express what the word “cat” refers to for a certain interpreter without using intentional vocabulary, what I need to do is to completely describe the conceptual structure and world model of the interpreter and denote that the meaning of “cat” is every physical structure that, via perceptual input and subsequent situation model, invokes the prelinguistic concept cat (which itself is a computational structure). This is just one option; other interpretations of what is meant by “meaning” exist, but they can be explained similarly. From the fact that I can give such a nonintentional description and that this description does not contain any information that is not part of a complete description of the physical state of the interpreter, it follows that all the information that is required to determine what “cat” means to this interpreter is already there in the interpreter’s description in elementary physics.
(This sketch obviously contains simplifications and not all details are worked out yet.)
Regarding the character of the conceptual representation of cats, this might be interesting to some (it doesn’t get very linguistic though):
http://psychology.emory.edu/cognition/barsalou/papers/Barsalou_chap_2005_situated_conceptualization.pdf
(This paper obviously contains simplifications and not all details are worked out yet. Compression galore)