Right, but note what you are saying: reason going wrong.
I don't think I even need to add anything there. Your phrasing already reveals that there is "a rightness ... by which it may be judged" in reason that there is not in faith.
> But, in rationalist theory, by positing itself as the means and also the end (we use reason to get closer to reason; we have faith in reason), the term forms the sort of self-sustaining loop that Zizek (or really Lacan) calls a "master signifier", and is also the essence of all theology.
I agree that reason is ultimately circular. I even agree that religion can also be circular. (I disagree strongly that all religion is circular, or that religion is inherently circular, or even that religion inherently hinges on faith.) Nonetheless, I think that where religion is circular it is so in a different way from reason. When we reject the circularity of religion, we do so not because it is inherently wrong to posit a God that justifies everything, but because it is not useful. The very self-supporting structure of the faith, the retreat to an insulated core element, prevents its adherent from doing any cognitive or predictive work with it. Rather, I would argue the circularity of religion is a consequence of reason, a retreat into self-justifying tautology prompted by the failure of religion to compete with reason on its own grounds.
In other words, reason closes on itself in a way that shapes itself to a particular inherent standard ("that seems to be associated in some way with success in interacting with the natural world", my reason throws in, giving yet another reason why it is judged valuable under itself, argumentatively achieving nothing yet undeterred.) However, religious faith does not even have any such constraint.
> bland evolutionary ethical theorizing, like "minimizing suffering"
I don't know why you are talking about suffering. I was not talking about suffering. I have no idea where you are going with that.
Evolved animals don't seek to minimize suffering. They don't even seek to maximize propagation; they're just the result of a process of relentless selection for things that propagate.
> "hard truths" are "hard" because they involve noticing objective aspects of ourselves
I'm sorry, you were previously accusing rationality of putting too much faith in reason - and now we're supposed to just notice objective aspects of ourselves? With what absolute source of truth, exactly, are we supposed to do this?
> The recognition often comes in a flash, with a sense of relief, rather than through some sort of laborious process
Now, I myself am as much of an addict to this sort of sudden endorphin-supported flash of insight as the next nerd. But - though it is an indicator of a simple theory that supports the facts observed, that does not in itself suffice to establish truth, let alone "objective truth"!
I have a new theory: lots of rationalists are programmers because programmers are used to having the sudden flash of insight be wrong.
> "how do I know that true things are true?".
I mean - rationality in particular does not engage in any attempt to establish truth, so I feel this charge somewhat misses its mark. Rather, rationality seeks to improve the quality of one's beliefs by engaging in processes that interact with the presumption of an ordered natural world in order to come to beliefs that are, given that assumption, more likely to be true.
That's rather why it's called "Less Wrong" rather than "Objectively Right".
> but how can you achieve intersubjective agreement in affairs which are only observable to a single subject, such as "my emotions"
I have a rejoinder to this called "you are not a unique and beautiful snowflake" and also emotions are totally observable to others, like, it's usually not even hard. Doesn't even take blood tests or brain autopsy. When some dude beats my face in with a bottle cause I fucked his wife, I do not have a particular hard time parsing his emotional state - or the evolutionary reasons behind it! Emotions are interesting because they have effects, on yourself (the brain is part of the natural world!) but also on others through you. Honestly, the idea that emotions are primarily individual is just so weird to me. They seem one of the most shared aspects of humanity there is.
> It seems better to ask the question of "what do I want that I'm having difficulty achieving? And what factual observations about myself can I make, which demonstrate patterns that prevent me from achieving this?"
From what you're saying, I take it you have observed rationalists failing at this. But I mean, going just by the text and the people I know, this just seems like solid rational thinking. I don't see any contradiction here.
I brought this up because the EA community seems very closely associated with rationalist ideas. Although I'm sure there are other rationalist conceptions of "the good" out there, I've yet to see them clearly articulated.
> ...emotions...
Perhaps the word I should have used is "desire", as emotion springs from desire, and although specific emotional states are easily observable, the underlying desire is what often remains hidden, subject to the other's fantasies. But it's an aside at this point.
> That's rather why it's called "Less Wrong" rather than "Objectively Right".
I'm glad you brought this up, because it raises what I see as the key question, the crux of the matter: "Less Wrong" relative to whom? And for what end?
The EA community aren't even unified in minimizing suffering. Most of the formulations I've seen are about maximizing good life, though there are certainly people who care about suffering in the community. (I think it's mostly Tomasik.) I haven't seen any of them try to justify their preferences with evpsych though.
> I'm glad you brought this up, because it raises what I see as the key question, the crux of the matter: "Less Wrong" relative to whom? And for what end?
Once more: Less Wrong relative to a presumed-ordered natural world. To the end of shaping this world to be more in line with one's preferences.
The "natural world" doesn't have beliefs, the world itself cannot be right or wrong, it just is. So the question again is, Less Wrong relative to whom?
> To the end of shaping this world to be more in line with one's preferences.
And what if one is unsure of one's preferences?
What?
Okay, let me try really small steps.
We assume the natural world is ordered.
We assume that the brain holds beliefs, which are compressive/predictive patterns, about the observations of the natural world.
Inasmuch as these are predictive, and inasmuch as we can take multiple actions, then a belief can be qualitatively better or worse in selecting actions that make the world be more good by one's judgment.
Beliefs can fail to be appropriate to select actions on (at least) two metrics: they can be "right or wrong", or they can be "more or less useful."
An example of a wrong belief: if you think it is raining, and so you bring an umbrella, but it is not raining (the belief is wrong), then you have failed to improve your position: you are dry anyway, but now you are also carrying an umbrella, which is annoying.
An example of a useless belief: you can believe that your shirt has more fibers than the average shirt of its kind. This may be true, but even if it is true, there is no decision that you would make where the belief is instrumental in deciding which of the possible actions leads to a world that better matches your preference.
(Unless you have very specific preferences about clothing.)
The goal of becoming Less Wrong is for your brain to contain fewer beliefs that are wrong, and more beliefs that are right; and also (but less importantly) fewer beliefs that are useless, and more beliefs that are useful. The term "Less Wrong" does not refer to a comparison to some other existing human, but rather to the counterfactual outcome of failing to reflect on your thoughts - Less Wrong is a collection of techniques whose goal is, when applied, to make your brain have less wrong beliefs and less useless beliefs, in order to improve your ability to shape the world to your preferences. (Whether they actually achieve this is a hot topic of debate - but that is the standard which they set.)
> And what if one is unsure of one's preferences?
That depends on your preferences for your preferences. :)
In any case, setting your preferences is not the purpose of rationality. (Though as your preferences are part of the ordered natural world, rationality also aims to improve your ability to shape your preferences, this does not tend to be a central consideration.)
But in essence, yes.
(Though of course, again, the ultimate goal is to be better able to achieve preferred outcomes in the world, and if thinking only the word "blue" on repeat produced outcomes perfectly in line with your preferences, then rationality would espouse blue blue blue blue. But at that point, most of the techniques gathered under the label would be useless, because yadda yadda ordered world etc.)
Basically, I am personally of the belief that any "wrong" or "useless" beliefs that one holds are always actually useful, as all belief is instrumental. Ignorance is maintained because it serves a purpose.
I am also of the belief that the only way to determine why one holds a wrong belief is to understand the "emotional" or desire-based reason that underlies or supports it. Hence why I spent a lot of time above discussing emotions. We have seen in recent times the repeated failure of individuals to adjust their beliefs even in the face of overwhelming evidence. Why is this? My stance is that their "wrong" beliefs serve deep emotional needs for them. Most people have stronger desires than the desire to arrive at truth in their evaluation of the world.
The difference in emphasis here is shifting from "the brain as a prediction-making device" to "the human body including the brain is a system that acts to relieve tensions placed upon it", and that prediction-making is one biological strategy for doing so. Of course, better predictions can and will relieve more tension than worse predictions, but my stance is that being able to conceptualize the causes and nature of tension directly will lead to better outcomes, as it permits stronger meta-cognition.
I see the term "preference" doing a lot of work in rationalist thought, and I generally taboo it. The term comes from economics and fails IMO to adequately describe the nature of desire. But if I were to use the term in this instance, I would claim that disentangling one's preferences from internal conflict is the first step one should take prior to tackling the problem of rationality (because we need to know what we even want to get done before we figure out how to do it). This involves interrogating our beliefs about what our preferences are and why, which is something that the rationalist focus on the "world" seems uniquely unequipped for dealing with.
I hope this all makes sense!
(I'm having a lot of fun with this debate too, it's really forced me to firm up my understanding of faith and reason.)
> But if I were to use the term in this instance, I would claim that disentangling one's preferences from internal conflict is the first step one should take prior to tackling the problem of rationality (because we need to know what we even want to get done before we figure out how to do it). This involves interrogating our beliefs about what our preferences are and why, which is something that the rationalist focus on the "world" seems uniquely unequipped for dealing with.
I agree with this take! Though I think it's less that this is something rationality cannot do as that, well, Eliezer when he was writing the Sequences, had an overriding belief that put all his other preferences at risk. This tends to focus one's attention on strategy. :)
The most illuminating post on this, for me, is https://www.lesswrong.com/posts/SGR4GxFK7KmW7ckCB/something-... . You see this notion of "rationality is a tool to come to conclusions when the conclusions really really matter" all over his writing. I especially like the phrasing in https://www.lesswrong.com/posts/wustx45CPL5rZenuo/no-safe-de... :
> No one begins to truly search for the Way until their parents have failed them, their gods are dead, and their tools have shattered in their hand.
Which implies that to most people, the pursuit of individual rationality, possibly even above mental wellbeing, is counteradaptive.
(Man, does Eliezer have a way with words though.)
(To me, personally, the idea of holding wrong beliefs is like a persistent itch or annoyance. I do not have a driving need, but I also have few attachments. And, of course, I am a nerd, and this is all nerd cocaine. All that sums up to create an ideological space in which I am comfortable.)
This part particularly resonated with me:
> Your trust will not break, until you apply all that you have learned here and from other books, and take it as far as you can go, and find that this too fails you—that you have still been a fool, and no one warned you against it—that all the most important parts were left out of the guidance you received—that some of the most precious ideals you followed, steered you in the wrong direction—
> —and if you still have something to protect, so that you must keep going, and cannot resign and wisely acknowledge the limitations of rationality—
> —then you will be ready to start your journey as a rationalist.
My trust broke years ago, and I found the rationalists, and then I realized, unlike Yudkowsky (I guess?), that I didn't have something to protect. So, my personal goals are in that sense very different.
In my experience with the rationalist community as well, I found that most people I met didn't have a specific personal motivation beside their beliefs surrounding x-risk, and I personally find the set of axioms required to accept rationalist beliefs on x-risk a bit... difficult to swallow.
(Also, AI X-Risk makes a lot more sense if you're already habituated to thinking of the mind in terms of algorithm. 40% programmers: not a coincidence.)
I'm also a programmer and am very habituated in terms of thinking in algorithms, but when I had my "break", I realized that my beliefs were not working on a very deep level, and I needed something a lot stronger/deeper than rationality to push me out of whatever local extremum I was trapped in.
(I think we've run out of disagreements. Good talk?)
(Woe! A deeper rift than even incompatible epistemologies: incompatible chat clients!)
There are plenty of cases of reason misleading, going wrong, perhaps as many or more as reason going right. But of course, judgment of right and wrong depends on your criteria, and reason will never help you achieve that which is prior to reason: faith, in the most basic form.
But, in rationalist theory, by positing itself as the means and also the end (we use reason to get closer to reason; we have faith in reason), the term forms the sort of self-sustaining loop that Zizek (or really Lacan) calls a "master signifier", and is also the essence of all theology. All other considerations are shoved under the bus, or else dictated by a sort of bland evolutionary ethical theorizing, like "minimizing suffering", without really interrogating what that even means (if they looked into older philosophy, they would see that thought itself could be considered a form of suffering). It is not necessarily bad to hold to a master signifier as such, as all faith exists in this form, but the rationalists seem especially unaware or unable to interrogate their beliefs to this level.
> I am very leery of the notion of "hard truths", because it imbues an argument with value based on the emotional work performed in reasoning it out.
This is a misunderstanding of what I meant to express, and I apologize for using that term. In my experience (and in the psychoanalytic literature), "hard truths" are "hard" because they involve noticing objective aspects of ourselves that we were previously blind to. The recognition often comes in a flash, with a sense of relief, rather than through some sort of laborious process (of course, the preparation for that flash might well be a laborious process of re-learning how to look!). The "hard truth" is being able to admit something like "actually, I'm not just out to get the truth. I want social recognition after all."
> Something does not become more or less true by being hard to admit; an untruth can be as hard to admit as a truth
Such a focus on truth in relation to psychological affairs feels, to me, wrong-headed, in that all truth is contingent, grounded in some act of judgment. Don't read this the wrong way: contingent != relative. It just means that we need to address the question of "how do I know that true things are true?".
The cultural impact of the enlightenment, especially of Kant, was to collapse "truth" into an intersubjective agreement based on sensory observation ("science"), but how can you achieve intersubjective agreement in affairs which are only observable to a single subject, such as "my emotions" (i.e. "how is psychology possible?")? This is where psychoanalysis had its real epistemological innovation: free association as a technique is a way of generating shared knowledge about what was formerly private. Christian "confession" functioned similarly. Modern behavioral psychology is several steps behind. But with or without such a technique, one can never really achieve the absolute (gnostic) truth about oneself, at least not without a strong frame that answers the question "under what circumstances is a given statement about myself true?"
It seems better to ask the question of "what do I want that I'm having difficulty achieving? And what factual observations about myself can I make, which demonstrate patterns that prevent me from achieving this?" Notice how this moves the direction of thought (of reason, even!) away from "finding the truth" and toward a more environmentally-oriented mode of "understanding", closer to what Plato meant by that term, with its connections to "wisdom". The "hard truth" here as I mean it, is noticing sites of competing desires, as Spinoza and Freud do in their discussions of "ambivalence", and then being willing to take a stance or a risk that attempts to resolve the tensions.
With that in mind...
> I don't think psychotherapy needs particularly to be correct about the causes it posits in order to function as therapy. Whereas if you are mistaken about the evolutionary cause of an emotion, you will completely misconstrue it and probably behave badly.
This is a feature, not a bug, because as I said earlier, the goal isn't to discover some absolute truth about yourself, but to solve problems. Evopsych explanations claim to provide truth, but actually serve to solve problems, through some intellectual sleight-of-hand.
Similarly, the use of reason in thought serves to solve problems, meet individual needs, even if framed as "uncovering the truth". This + my deep involvement in the rationalist world for several years is why I feel comfortable making statements like this one from a few days ago: https://news.ycombinator.com/item?id=28198926