Mysterious Answers to Mysterious Questions by Eliezer Yudkowsky

This classic from Yud is worth reading again if you haven't recently. Relevant to some of our recent consciousness discussions, representing the meta-principle behind the rationalist physicalist position (consciousness is a nonmysterious physical phenomenon viewed from "inside"). Basically this position is implicitly a total rejection of mysterious answers and the supernatural as incoherent anti-epistemology. But Yud takes this rejection beyond mere opinion to be a technical discipline you can practice to subjectively *notice* when you are engaged in irrationalism, and stop it. That's two very important ideas in this article.

Sorry, not a fan. Yud's essay is just as mystery-bound and incoherent as what it's fighting.

If you take simply it as, "Don't believe random woo," well sure, I agree with that, but that's also trivial.

But if you try to make Yud's claim nontrivial and rigorous, it falls apart.

The whole issue is that we don't know in advance what the boundaries of science yet to be discovered are, or what future science will look like. From the perspective of the best Newtonian physics of 1800, the fundamental physics we have today (quantum field theory and general relativity) would seem like bizarre magic, even though they're not.

We may yet discover much more about consciousness. If we do, what we discover, once systematized, might seem as rigorous as fundamental physics does now, and no more (or less!) "mysterious." But that's an ex post judgment. Ex ante, there can still be a lot of we-know-not-what.

Yud is using the word "mysterious" in an entire connotative way. For him, it basically means "yucky." He is adding nothing rigorous to the conversation, one way or the other.

Things we don't understand can legitimately be described as mysterious! There's absolutely nothing wrong with that. Later, we may come to understand them better, and they will become less mysterious. We could say that about electric fields, to give an example from a century or two ago. It's no big deal. Yud is not clarifying anything with a polemic against the mysterious.

>>2300
Did you read the post? He gives a definition of what he means by mysterious answers. First of all it means explanations that don't have content to them but just wallow in the mystery. Second of all it means a psychological pattern where the subject clings to the idea that some particular phenomenon is beyond the reach of rational or physical explanation to protect certain other beliefs from empirical refutation. Yud is explicitly not arguing against mysterious questions, as you are implying. There are all kinds of mysteries in the world. He's arguing against mysterious answers, where we "answer" our curiosity with sacredness and thought-stoppage rather than technical explanation.

Electricity or quantum or all of that were mighty mysterious when first discovered, but the explanations are now tight equations and very constrained ontologies, and we didn't get there by wallowing in mystery. From the perspective of 1800, our current ontologies can be distinguished from the random magical thinking around electricity etc by the precise nature of the claims and the psychological neutrality. In the course of real science there is a lot of woo generated as a byproduct of having an open mind, as there was with electricity, but this is not on the critical path and not something you should actually accept in yourself. Technical thinking about mystery has a different psychological flavor from magical thinking about mystery, and it can be distinguished at the time, not just in retrospect.

Consciousness or intelligence or agency is of course a mysterious question. We don't have rigorous technical answers. Nothing wrong with that, as you say. But all the time people take this lack of constraint as an opportunity for injecting magic into their worldviews for various psychological and religious reasons. If the "magic" was tightly constrained predictive hypotheses, it wouldn't be magic, but it's actually more like a taboo on thinking. The point isn't to actually predict the phenomenon, but to protect certain beliefs from empirical refutation. One of the tells is that the mysterious answer is often morally significant and taboo to question. It's a "god of the gaps" thing, where you find the mysterious gaps in our knowledge of the world, and fill those gaps with all the magic and fairies you need to believe in, and call that an answer. It's a sacrifice of being able to actually answer the question for the sake of psychological or religious satisfaction. I see this all the time.

When the unknown is not just a lack of knowledge, but becomes the sacred abode of the gods or moral truth, you fucked up. The unknown is for exploring, ignoring, or betting on, not worshipping.

>>2301
Yeah, sure, Yud gives his own definition of "mysterious." That's kind of bullshit. It's an English word that already has a meaning.

> Yud is explicitly not arguing against mysterious questions, as you are implying. There are all kinds of mysteries in the world. He's arguing against mysterious answers ... people take this lack of constraint as an opportunity for injecting magic into their worldviews ...

That's the problem with Yud's treatment, right there. The issue, it turns out, isn't "mystery" at all. It's giving magical answers. I called that woo, and I stated up front that I think it's bad.

If someone wrote an essay explaining why magical/woo answers are bad (e.g., they lack testability, or when tested they fail), I would have no disagreement (although I would consider it a bit obvious and trivial).

But that's not what Yud wrote. He targeted the "mysterious," while equivocating with the real problem being magic. Not a good essay.

> Meeting the great dragon Unknown, the vitalists did not draw their swords to do battle, but bowed their necks in submission.

This is an obviously retarded accusation to lodge against Lord Kelvin and Henri Bergson. The term elan vital was coined by Bergson (decades after Wöhler’s synthesis of urea) to indicate the creative impetus of man, i.e. the exact attitude Yud is claiming the vitalists don't believe in.

Vitalism has a clear advantage over materialism in that it contains the actual joy and play that is so clearly in physis. In contrast the only thing Yud's writing contains is his own bulbous ego.

>>2303
I'm not familiar with Bergson's work, but I can never figure out why vitalism is always contrasted to materialism. The elan vital I've seen actually argued seems to a higher level description of the nature of life, spirit and consicousness, or even a philosophical disposition to celebrate life, not a claim about non-material substances or forces. Yudkowsky et al seem to take it to be a material substance, which is retarded.

Can someone explain what vitalism is actually claiming?

>>2304
> Can someone explain what vitalism is actually claiming?

It's hard to say because there wasn't a uniform vitalist theory or hypothesis in the late 19th or early 20th century.

I agree that it's retarded to construe vitalism as hypothesizing a undiscovered substance that would account for life.

If I were to work on something like vitalism, I would take thermodynamics as my starting point. The key observation is that the laws of physics have the nonobvious property that they give rise to systems that locally decrease entropy while, of course, increasing it elsewhere –– and this takes place at ascending levels of organization. [Reference: Every Life is On Fire, by Jeremy England]

Reductionist will angrily insist "but this isn't vitalism, it's all reducible to low-level physics!" Because reductionists are retarded. The point is that the reduction isn't that interesting. Most interesting things (including science: chemistry, biology; but also higher human things) take place at the higher levels.

>>2305
>I would take thermodynamics as my starting point ... systems that locally decrease entropy
the existence and physical uniqueness of life, broadly construed is your key observation here. It's not just like normal matter undergoing normal forward evolution. It's a special regime where relative to normal thermodynamics, it's almost like time and entropy are reversed. Deviations don't compound, they diminish. Things don't happen because of what happened in the past, but what will be needed in the future. The thing the materialists are always on about is that this is all ultimately made of arrangement of lower level physical stuff eg atoms and molecules etc and not magic, but that's not really the interesting thing. The interesting thing is that it's possible to build out of atoms these entropy-reversed regions that use a larger flow of energy along the usual entropy gradient to locally reverse it in a self-replicating way. There is some not fully understood vital principle at work here.

This also covers intelligence, which also displays this strange reversal of causality (reasoning backwards from ends) as its core phenomenon. I believe intelligence (or spirit, consciousness) and life are this same phenomenon. Part of what defeats reductionism here is actually the same principle again: by inverting the usual order of causality, life actually inverts the usual order of reduction. Instead of building up from atoms and molecules to materials and objects, it goes the other way: you have to start with the vital principle of life, which takes form in a particular organism, which needs particular organs and cells, which need to be made of particular atoms and molecules. Of course it bottoms out in and "runs on" the usual physics, but that's not really interesting because causality and reduction are locally reversed.

A productive vitalism would be trying to really figure this local causality/entropy/time/reduction inversion business out. That's the key phenomenon, and the non-reductionists are right that shoehorning it into a reductionist framework misses the point. I believe it could be simulated on computer (non-physicalists do not agree), but I don't believe it ever actually has been. That would be a useful thing to do for example. We don't understand it well enough to build a minimal example in simulation. All our cellular automata and physics sims and AI etc are not actually "alive" in this way. We're getting close to something maybe with AI and all that, but not in a principled way where we actually understand the principle in play.

>>2306
I agree with most of this, but I think we should be careful with language on this topic.

> It's not just like normal matter undergoing normal forward evolution. It's a special regime where relative to normal thermodynamics, it's almost like time and entropy are reversed.

You keep saying "normal" as if we already understood how matter, thermodynamics, and evolution work, when part of the problem is that we don't (yet). What you're calling "normal," I would roughly call "Newtonian." On the one hand, I agree that these things will not turn out to be Newtonian. On the other hand, we already knew that Newtonian physics is far from a final theory, so we shouldn't be thinking of its conceptions as "normal."

The phenomenon of local entropy-reversal doesn't happen just with life. It's already present (in simpler form) with proteins. Maybe other molecules too. In fact, maybe the formation of atoms is (in simpler form) already an example.

So, rather than think in terms of a "special regime," I'd emphasize that the laws of physics themselves give rise to these phenomena by their normal (there's that word again) operation. Yes, the higher forms of the phenomena (like plants and animals) look less and less Newtonian and more and more like intelligence. But that's just how things are, not a "special regime."

As for time reversal, I'd just note that general relativity is time-invariant. It has no predefined forward or backward. So if something like that turns out be involved in thermodynamics, it wouldn't shock me.

Let's not get over-reliant on physicalist epistemology for things physicalism wasn't built for. Don't want to get potentially existing meta levels mixed around. Physicalism helps us explain how the brain responded to environmental pressures and was forged into a remarkable cpu. Doesn't tell us why there are people inside of them and not Roombas.
(https://en.wikipedia.org/wiki/Philosophical_zombie).

Leibniz breaks it down this way: If we imagine that there is a "machine whose structure makes it think, sense, and have perceptions, we could conceive it enlarged, keeping the same proportions, so that we could enter into it, as one enters a mill. Assuming
that, when inspecting its interior, we will find only parts that push one
another, and we will never find anything to explain a perception. And so, one should seek perception in the simple substance and not in the composite or in the machine.... we must confess that perception, and what depends upon it, is inexplicable in terms of mechanical reasons, that is through shapes and motions.

See also Kant:

"That he [man] is not wholly and purely corporeal may be strictly proven, if
this appearance is considered as a thing in itself, from the fact that the unity
of consciousness, which must be met with in all cognition (including that of
oneself) makes it impossible that representations divided among various
subjects could constitute a unified thought; therefore materialism can never
be used as a principle for explaining the nature of the soul."

Consciousness really shouldn't exist, indeed *doesn't* exist in important physical respects.

>>2310
> Let's not get over-reliant on physicalist epistemology for things physicalism wasn't built for.

You seem to be taking "physicalism" as roughly synonymous with "materialism." I explicitly don't do that. When I speak of the "physical," I'm thinking of Aristotle's phusis, or nature. It's just how things are, which I don't presume to understand in advance. I certainly don't presume that things are only material. On the contrary, it seems obvious to me, with Aristotle, that form is also a fundamental principle of things. That's how we can tell a dog from a cat, or a hydrogen atom from a helium atom.

Does form explain consciousness? Not all by itself. But it sure does seem that knowledge at all levels has to do with the communication of form.

>>2310
If consciousness doesnt exist in some physical sense, what causes you to talk about it? The p-zombie concept is incoherent as far as i can tell. Whatever interior magic makes the difference between zombies and conscious humans has no bearing on observable reality. Whatever process you are observing that has you physically speaking about consciousness must also be shared by a p-zombie. To speak coherently about its own conscious experience, the purely physical p-zombie would have to introspect on its own physical thought process in the same way you do. It would also “experience” this interiority the same way you do. If you can assert contrary to its own internal experience that it actually doesnt have some magical additional interiority that constitutes real consciousness, then i can assert the same about myself and all of you, and invoke occams razor against your complications.

This consciousness stuff seems to be an ideological hallucination as far as i can tell, though i cant quite imagine what the ideology is. If you’ll humor me for a rude question i’ll ask this: why is it important to you that consciousness be nonphysical? What bad thing would be true if it was not?

>>2307
You are right its not some kind of hard metaphysical break into two different physical regimes. Its more like heuristic classes of behavior like solid vs liquid vs gas. Would you accept that there is a difference in behavior between teleological and nonteleological phenomena relating to the “direction” of causality? For example an atom’s internal state is somewhat teleological in that it “wants” to return to settled states by spitting out photons etc. but the motion of that atom in a complex field is much more “causal” rather than teleological. Again these arent super hard distinctions, but they are significant. Life as a complex teleological phenomenon is not unphysical, but it is different from our usual heuristics for causality. This difference is i think what “vitalism” is gesturing at, and the source of much confusion and magical thinking.

>>2313
> Life as a complex teleological phenomenon is not unphysical, but it is different from our usual heuristics for causality.

I agree, but I think this is more a fact about our usual heuristics than a fact about life.

Even simple systems, like particles interacting or an atom being perturbed, can be understood as teleological when analyzed using Lagrangian mechanics rather than high-school Newtonian mechanics. In principle, more complex systems, such as living systems, are "just" systems with more complex Lagrangians. In practice, more complex systems quickly exceed our ability to write down and analyze the relevant Lagrangian.

I take vitalism to be the charge to take seriously the dynamics that we as humans should and do care about, without being mind-nuked by reductionism. There's no magic, but higher-level dynamics are perfectly real.

>>2313

> Whatever interior magic makes the difference between zombies and conscious humans has no bearing on observable reality.

This is exactly the problem. Gnon didn't need to create consciousness.

You know or at least experience the fact of 2 + 2 = 4, but an abacus simply performs an operation, as does a calculator, as does an AI, as does a p-zombie — they don't *know* or *experience* anything.

> why is it important to you that consciousness be nonphysical?

It has more to do with the concern that consciousness is the gate through which we observe the physical. We then use the physical, guided by consciousness, to backwards-interrogate consciousness. Fair enough — but we should be aware of consciousness's definitional primacy in our sense-making.

>> 2311

I'd be interested to see you expand on this, maybe more long-form elsewhere. I don't really track the argument as it applies to consciousness. Your tentative explanation from the other post ("reflective" interior processing) struck me as basically neurological, so I did make the materialist leap.

>>2320
>> Whatever interior magic makes the difference between zombies and conscious humans has no bearing on observable reality.

> This is exactly the problem. Gnon didn't need to create consciousness.

You don't know that. It could be that Gnon did need to create consciousness. It could be that not only do p-zombies not exist, but they could not exist. It could be that whenever you have a being that does all the things humans do, down to a fine-grained level, that being just will be conscious, due to the way nature works, for reasons we don't yet understand.

I personally believe that the above things not only could be, but are in fact the case. I think thought experiments about p-zombies are misleading, that they trade upon our ignorance regarding the causal sources of consciousness to reach false conclusions about it.

Well Gnon did make us after its likeness.

>>2322
I agree whole heartedly with this anon. "Gnon created consciousness" because the kind of animals we are and needed to be to fill the open niches in the world needed to be conscious in an entirely natural and straightforward way. Consciousness is necessary to and identical with intelligence of a sufficiently general and reflective kind, which you apparently need to act as a coherent social agent. P-zombies are not actually possible.

That said, while I agree in a strict sense that we don't understand this (we haven't built it yet for one), I don't think it's one of the grand mysteries that some others seem to think it is. I am basically satisfied by the claim that our internal conscious experience is the inside of a representational reasoning system. I don't see how you could produce an intelligent reasoning system capable of reflection on its internal states and not have it report the same things about its internal experience that we do. And it seems quite possible to build one on the same old types of computers we use for everything else.

The mystery is just the puzzle of how exactly to architect this, similar to the mystery of how to create a fusion reactor or something.