I am trying to work through how I missed my brother being a Neo-Nazi. I know I didn't apply the toolkits because I had an emotional attachment to my brother. In my first post about my brother I blamed my emotions.

But I think I got it wrong. I mean, he is a neo-Nazi, but blaming emotions for missing that is a bit of a cop-out — I need to just blame myself. Ford was warning me about my brother for years, just from little things I said. I kept defaulting to the status-quo. Why didn't I listen? Why didn't I see things clearly?

The hypothesis I would like to explore is that our true enemy is our sub-conscious. Emotions are our only way to consciously peek into our sub-conscious, so they are needed tools.

In short, I was using the guardrail toolkit, unlike my brother. But I was not using the advanced toolkit, just like my brother. I didn't fall for any of his conspiracy theories, but I did trick myself into believing my brother was someone he wasn't. Think of this post as an early preview of what the full toolkit aims to help people with: their emotions.

Emotions make us feel like we are thinking rationally. In order to use emotions as tools, we should not always take them at face value. To be effective, we must observe our emotions and see what they are telling us about ourselves. This is the premise of the advanced toolkits (not yet published).

So I would like to go back to how I failed with my brother. In one instance he said a remark that is most easily interpreted as racist. He stated it like a fact. I don’t remember the exact wording, but it was something along the lines that black people have a lower IQ. But I couldn't square that with what I knew of him: I had never seen him treat anyone of any race differently. Before this comment we had several conversations about how the world is rigged in favor of those who are born into fortunate circumstances. So I put these two thoughts together. He must have meant, this race of people have been historically persecuted and denied education. The racist interpretation also didn’t track with how everyone in the family thought of my brother. I concluded "he doesn't really believe what he says”. I then felt relief. Not just from this instance, but many others over the last several years. I kept rationalizing until I got to 'relief'.

Observing my relief, I would have seen the contradiction: relief from what? What am I being relieved from? Relieved that he is good? That doesn't make sense. I should have felt proud or happy if I thought he was a good person.

I might feel relief because...

• I don't need to change my behavior

• I don't need to put in additional effort

• I don't need to look at the other half of the superposition

• I don't need to confront my brother

• I don't need to jeopardize the relationship

That is me trying to preserve the status quo. Relief, in my situation, is clearly self preservation. The emotion is a hint, a shortcut, to help me consider alternative 'selves'. It is a direct line to my status quo bias.

First of all, that was a really quick way to identify my potential problems. I have never gone to therapy, but this sure feels like what you are suppose to gain from it — clarity. I can see the outline of the other superposition in this observation.

But how do I have the outline of something I haven't consciously observed yet? This is the position I was denying existed for so many years. Yet somehow, I knew the upshot of what it would entail.

I might need to:

• change my behavior

• put in additional effort

• look at the other half of the superposition

• confront my brother

• jeopardize the relationship

The emotion of relief can only stem from my subconscious. I don't have any other explanation. If I consciously would have dealt with the ramifications... well I would have seen things clearly. But I didn't. My subconscious threw me some 'relief' to keep the status quo. It told me what I wanted to hear: "You're right, no need to change, no need to expend any more effort".

I am literally my own worst enemy.

Observing my emotions I can clearly see my self-sabotage.

Why does my subconscious do this to me?

The human brain has evolved powerful mechanisms to maintain stability and avoid unnecessary risks. This manifests as status quo bias — the tendency to prefer current states over change.

We have 'models' of the world in our head. These help us predict what we think might happen, and thus affect how we make decisions. These models balance what psychologists call the explore-exploit tradeoff: when to stick with proven strategies versus trying new ones. Our status quo bias is just trying to 'play it safe'. We have a bias to use models that seem to be working instead of crafting new and uncertain models. To some degree I rationally knew all this. But that knowledge didn't help me.

... Models that seem to be working. This is where rationalization comes in. We can keep a model working much longer if we just throw out or 'adjust' some evidence. Smooth over the rough edges. Our subconscious prefers existing models so much, that it makes reality malleable.

Our perception of reality can be messy. Rationalizations are a sort of shortcut to get to a 'working model'. A basic theory of how something works. The problem is that we continue to use rationalizations after we have a 'basic model'. Rationalizations at this stage prevent refinement of a basic theory. In other words, there is an anchoring effect to our initial model.

Our subconscious wants to minimize uncertainty. Rationalizations just shift any 'potential' uncertainty to the future. An error in observation, may be just that, an error. But when it isn't an error, we are throwing out evidence.

For my brother, I shifted all the uncertainty down the road. Being close to my brother made my rationalizations really strong. When he finally said something that I couldn't rationalize, I was hit with all the 'ignored' evidence at once. Uncertainty tried to collapse. I still kept trying to rationalize it. I wanted to ignore it. I was in denial. Now that I have confronted him, I have been able to adjust my model.

If rationalizations are fueled by emotions and an anchoring effect, we really only need to look at our emotions.

So what are emotions, in their entirety? Uncertainty can be part of it, but I think emotions can stem from many subconscious causes.

• Prediction errors (uncertainty)

• Reward/punishment signals

• Social context

• Physiological state

• Past experiences

I think the inclusion of past experiences is particularly interesting (in a dangerous way). That is the mechanism that builds one-way bridges. If we start accepting 'answers' that are false, they become part of our past experiences. We start trying to predict off of these 'answers', and if our predictions fail, we will start to rationalize. We can then use these rationalized 'answers' to keep our model alive. Add in some reinforcing social interactions and it is actually pretty easy to see how someone like my brother, or anyone with good intentions as their guide, can end up getting stuck believing wrong things. For my brother, Jews were the 'answer'. Twitter was (at least part of) his social interactions.

For me, I used my past experiences with my brother to rationalize the words he was saying, to rationalize all the red flags that now seem so obvious. With the exception of two people, all of our mutual acquaintances described him as 'having a good heart'. Those two people were right about my brother. But I looked for consensus instead of truth. Those two people must be wrong, otherwise... everyone else would be. And — that would be bad.

My emotions were a tool I had, but chose to ignore. It is the loose thread I could have pulled on to unravel my delusions about my brother.

I don't think my experience is that different from others’. I have no direct evidence to support this — I can't inhabit other peoples experiences directly. But if I'm wrong, and this doesn't apply to you, you can stop reading here.

If we assume most people don't really analyze most of their decisions and instead use the literal interpretation of their emotions, then this also sheds light on why our current information hellscape is so effective at tearing society apart. Social media algorithms ratchet up the social context of those who will reinforce any 'answers' we already have. We also anchor to new 'answers' from the same spheres that we agreed with in the past. At the same time we hear so many conflicting 'answers' from other bubbles of the media ecosystem. We rationalize those away, driving us harder to 'what we already know'. When we interact with people from these other spheres we use any flaws in their arguments to rationalize why we shouldn't listen to anything they have to say in future. This is what Ford calls 'a runaway bad information environment' (we will do a post on this shortly). At no point does 'truth' enter into this. We just have 'answers'.

Let me use my new clarity, and my position as an apprentice to Ford, to offer advice to anyone who really cares about the truth and thinking clearly: If you've read our guardrail toolkit, you probably thought "this makes sense", "I already know this", "I already do this". But you probably don't. The simplicity of the toolkit disguises how hard it is to categorically follow. Our biggest obstacle is ourselves. I thought I knew my brother, so I could skip using the toolkit with him. But isn't that the point of the toolkit? "Don't get stuck believing wrong things". This was exactly what I did with my brother. He is stuck believing the Jews are the problem, and until now I was stuck believing that he wasn't a person who wants violent solutions.

Ford has a favorite fallacy, and we use it almost like a meme now:

But, that would be bad.

If you ever think about something like this — if your only reason to deny a possibility is "but that would be bad" — it should be a 5-alarm fire that makes you stop in your tracks. The amount of uncertainty behind that statement is immense. LOOK AT IT. Every fiber in your being will be trying to make you ignore it, to seriously entertain only the rationalized possibility. It is so immense that it is starting to bleed into your conscious thoughts. The truth is hiding in plain sight. Denial is your response, maybe 'confusion' is your only emotion.

If either of these thoughts ever cross your mind in response to something you know could be significant:

• I don't need to change my behavior

• I don't need to put in additional effort

Please stop. Look at your emotions. Write down what you are feeling. Read it back to yourself. How can you interpret them? Try to list as many interpretations as you can, then negate them (or remove the negation). What are you telling yourself? What do you already know about yourself and the situation?

Of course, everyone is different, and maybe Ford and I are unique but..

Share

Subscribe now

Turns out, my brother is a full blown neo-Nazi. I'm just trying to figure out where he lands on the spectrum between a LARPer and Timothy McVeigh.

I wrote about my rationality fail earlier, and I was on the right track, but I probably could have figured this out 4 or 5 years ago...

In fairness to myself, we really cut down contact around then. He got busy with a startup and was doing a ton of travelling. We went from seeing each other weekly down to a few times a year.

They say humans are bad at imagining exponential rates of change, and I think my brother walked off a cliff during this time. I was only figuring things out at a linear rate. I have been online long enough to have read and seen other people take this path, and so while I'm surprised it happened to my brother, I do understand a bit about the nature of what happened. The challenge has been trying to convey it to friends and family.

My parents in particular are struggling with what they feel is a contradiction: my brother is not an evil/hateful/crazy person, so how can he hold such evil/hateful/crazy beliefs?

My parents don't see my brother as an evil skinhead. He doesn't fit the mold of what they would expect a neo-Nazi to look and act like.

This is why I struggled as well. My brother and I have a long history together. We are so alike in so many ways, I couldn't imagine any way that I could believe those things. To this day, I think we both sincerely try to understand the world, and dig in to the 'hard problems'. We hold many of the same premises about the world and agree on a lot of facts, but we came to very different positions on what to conclude about them. For example, when we would agree to observations like "the system is rigged against those less fortunate" and "capitalism is unfair to those without capital", the thought of Jews never entered my mind. We were having conversations like this for a very long time. But at some point, my brother decided that it is the Jews.

All our common language and previous discussions became 'evidence' to cement his new belief. So how long has he been thinking that the Jews account for all these observations in our previous conversations? How many discussions have I innocently had that helped cement these beliefs?

How come I didn't end up believing in something equally crazy? How do I know what I believe is sane? How do I make sure I remain sane?

What is NOT an answer to this question: good intentions and lots of effort.

You can end up believing all sorts of crazy things using good intentions and lots of effort. Both history and the current day is filled with it. Someone like Alex Jones cares deeply about protecting the country and is afraid the government is corrupt. I think lots of people would agree with that statement, if they didn't associate it with Jones. Jones has good intentions and puts in a lot of effort to make others aware. He would be admirable if he had a process that kept him tethered to reality. But he doesn't.

It's not just Jones -- most people don't have a process. They rely on good intentions: their own and those of people they trust. People driven solely by good intentions mean well. But without a process the good intentions become fuel for rationalizations. You end up using good intentions to draw conclusions, and then put in effort to justify them. It is confirmation bias on steroids.

Once you start justifying baseless conclusions, you can end up vilifying whole races of people. For my brother, I truly believe he still wants to make the world better -- but his solutions at this point are violent and racist, and based on nothing real.

Good intentions can get you stuck believing wrong things. And by 'stuck', I mean your beliefs can only develop in one direction. This is the process of 'radicalization'.

"Words build bridges to unexplored regions" - Hitler

Without a process, you might build a one-way bridge to any sort of reality-denying belief. Your brain will become toast. Coming back across one of these bridges will only happen by pure chance. You will have lost all agency -- the beliefs will drive all of your behavior and taint all information that you take in. All meaningful thought ceases. You just have conclusions that you cannot stop believing. Your brain can't do anything but halt and catch fire.

So how can process beat good intentions and effort?

We have the "guardrail" toolkit posted and that is the basic process. (It is pretty dense and requires some explanation to help it make sense. We will provide full commentary in the coming weeks.) The reason the process works, is that all the rules are categorical. There is no situation in life you can't use them. Since the 'tools' are categorical, the toolkit is rote. There is no 'decision' besides listening to yourself and how much you 'care' about any given topic. In that sense, Ford's process uses our natural inclinations to our advantage. You just have to reflexively always use the toolkits. That is the only failure point. I made a single exception and you can see the results. With enough exceptions (or lack of any process), you can end up believing things like my brother, and thinking you're the good guy for it.

One of the issues I feel most of us have, is that we are afraid to think of others as full and complete human beings. My parents don't want to think of their son as 'evil', because that erases all the good things they know about him. But that misses the point. They don't have to call him evil (he doesn't think of himself that way). All the good things he's done are still true. It's just also true that he now believes really bad things.

A related issue is that we are afraid to take evil ideologies as arguments. I understand why this happens: it is revolting to do. Why should we take this hateful crap seriously?

But Hitler had a sympathetic crowd when he spoke. They built bridges using his words. They all had good intentions: a better life for the German people. How did that turn into war and genocide?

Humanizing is a tool we must employ to better understand the problem. How are these one-way bridges being built? What words are building them? How are the words changing how people think? How do 'good' people end up believing 'evil' things? We need to humanize 'evil' if we want to learn how to prevent it. We also need to humanize 'evil' in order to recognize when it shows up in real life, even if it's coming from people we love.

Millions of one-way bridges are being built right now, this very second. This problem has been documented all over our modern world. We all worry about fake news and misinformation. We already know that the truth is important.

But truth isn't just 'important'. It is the ONLY thing that matters. And process is the ONLY thing that gets us to the truth.

...Do you have a process? What types of bridges are you building?

Subscribe now

Homework for concerned citizens: I would recommend watching Imperium (2016). Do any of the characters think of themselves as 'evil'? Both the FBI and the neo-Nazis are trying to 'get rid of evil' in the world. Why should we presume the FBI is 'good'? Try using a reality vs fiction framing instead of good vs evil. Does it get easier to believe 'nice' people would do horrible things?

The full, original version of the toolkit… is very long. And exacting. It was designed for use by someone who wants to hold themselves to the highest possible standard.

Think of what follows as the speedrun version: it is stripped down to essentials, and focused on the most approachable parts.

We will be talking about this introductory version of the toolkit extensively over the next few weeks.

Subscribe now

INTRODUCTION

There is one universal starter goal: Don't get stuck believing wrong things.

Why this goal?

If you care about something (anything at all), then supporting it with your actions ALWAYS requires two things:

1. Keep your actions aligned with reality

2. Keep your actions aligned with your goal

If you don't do these things... then when you act, your action may not have the 'cause → effect' you are hoping for. You may fail to support the outcome you care about. You may even work AGAINST what you care about, unknowingly.

So to avoid self-sabotage: 1) we need to always understand 'real' reality, and 2) we need to clearly understand how our actions relate to our goal, so that we can keep them aligned with each other.

But — we have an information problem. Sometimes we may make mistakes. Others may make mistakes, and we may not be in a position to notice. People may lie to us. 'Base facts' may not be confirmable. On and on and on. There are infinity-and-counting ways to mess up, even when we mean well and try hard. So how can we always be aligned with reality, and keep our actions consistent with our goal, when we full-stop can't know everything with certainty all the time?

Well...there is nothing you can do or say or think that will ALWAYS prevent you from making an error. What's worse: what you do to prevent getting fooled in one scenario, may make you susceptible to getting fooled the next time. Circumstances and luck are just part of the picture.

So let's get off total prevention. We can instead replace our two goals, with one goal that has equivalent intention.

Don't get stuck believing wrong things.

This replacement goal allows room for the idea that we're going to get things wrong sometimes, in little ways or big ones, and says that our goal is just to not get STUCK like that. If we're drifting from reality, or if we're getting inconsistent, then we want to be able to see any warning signs as soon as possible, so that we can correct course.

This, at least, is doable in theory. And it gives us our best chance of acting in a way that actually supports the things we care about — since "know everything and act perfectly all the time" is off the table.

...But this goal, as-is, is not very practical. 'Don't get stuck believing wrong things'. I can say this, and agree with it, but I don't really know how to do it — at least not in a moment-to-moment sense. What should I actually do, what specific actions should I take, if I want to achieve this goal?

That's what the following 'toolkit' is for.

Working off our one goal, 'Don't get stuck believing wrong things', we add smaller, more specific categorical goals that all point in that direction. 'Categorical' meaning that, if done, achieving these goals would ALWAYS, INEVITABLY support that main goal. Then we repeat the process, breaking each new goal down into smaller, more specific goals still. Until we end up with goals that we can actually see how to achieve, all of which would categorically support the goal we want.

These smaller, more specific goals are 'the rules'. Just apply them, in all circumstances, as often as you can remember to do so. No worries about exceptions, because there are none.

You should inspect the following rules to make sure that they do indeed seem to be categorically bent towards fulfillment of the main goal. They should, in fact, seem necessary for it.

If, on inspection, you agree that following all the rules would be needed for our main goal... and if you agree that 'don't get stuck believing wrong things' is a goal that applies in all circumstances... then these are rules to live by.

PREVIEW OF MAIN TECHNIQUES:

• Atomize your judgments

• Default to "I don't know"

• Bin every claim you hear into either 1) a prompt for a question or 2) emotional white noise

• Note explicitly when your interest in a claim or question surges, and when it drops off

• Be specific about what question you're answering (especially if/as it shifts)

• Map thoroughly every time you want to answer a question

• Look for and favor good mapping in your casual sources for info

1 — ATOMIZE YOUR JUDGMENT

1-1 — Atomize your judgment. Handle everything in the smallest units possible. For example, multi-part claims/arguments should be broken down into individual claims.

2 — ON HEARING A NEW CLAIM, WHAT YOU SHOULD DO INTERNALLY:

2-1 — Default to saying/thinking "I don't know".

2-2 — Acknowledge that if you want to change 'ignorance' to 'knowledge', or verify knowledge you think you already have, it will take further particular steps. If you are not willing to consider further steps, back to 1.

Re-upped rule: 'Atomize your judgment' can apply to "I don't know". You can separate a claim into details you do know for certain, and elements you say "I don't know" to.

3 — ON HEARING A NEW CLAIM, WHAT YOU SHOULD DO WITH THE CLAIM:

3-1 'Bin' the claim into one of two bins: it’s either a prompt for a question, or 'emotional white noise'.

• Keep in mind 1-1. You may need to break one claim you've heard into multiple components to be considered.

3-2 If you find a prompt, name the question you have (as specifically as possible).

• If you want to turn this prompt/question into 'knowledge', it will take further particular steps. Unless and until you have done that work, you only have a question.

3-3 If you end up with 'emotional white noise', observe and name the emotion(s) involved.

• Otherwise, discard the inputs. Keep in mind 2-1, and 3-4.

3-4 Reassign bins whenever appropriate.

Re-upped rule: 'Atomize your judgment' should apply to binning. You can separate a claim into elements you're interested in turning into a question, and elements that are 'emotional white noise'.

CONCEPT INTRO: ‘MAPPING’

When you want to move off "I don't know" and get some answers, the critical concept is that 'good information' = 'complete mapping'. 'Complete mapping' means putting ALL possible answers to the question in the same field of vision: you will need to make the strongest possible version of each answer, analyze each individually, and compare them to each other, in order to be able to see the 'map of answers'.

Complete mapping inevitably involves LOTS of research in order to find the 'raw facts' that form the base of each argument, as well as how each argument strings together in the eyes of its supporters. Training in research skills, as well as formal logic, will help at this stage. (Both are very learnable, and we can get into them if you ever get stuck or want to supercharge your efforts.) A good general knowledge base is also useful (eg learning the 'jargon' around a topic, and as much relevant history as possible), so that you can see the fullest possible context and provide additional facts/arguments for the map.

You 'win' mapping by having the most complete map possible, handled in the most rigorous way. Each argument should be made with high specificity. A person supporting any argument should be able to look at the map, see their position represented, and say "yes, that".

Answers, if any, just shake out from a carefully made map, so the map itself is what needs to be prioritized. Note as well that maps are ALWAYS conditional: if any relevant facts/arguments are discovered or overturned, the map (and any tentative answers) will always have to be revisited.

The good news:

• Just caring whether the map is complete, and whether arguments are represented fairly, will go a long, long way towards being good at this.

• Mapping allows collaborative research, even among people who disagree. Hypothetically, any people who value accuracy and fairness will ALL CONVERGE ON THE SAME MAP. So the burdens of research can be shared.

• Often, the basic facts are NOT in dispute between different positions. So making a map is often a shorter process than it may seem upfront.

To summarize, and really stress: MAPPING. IS. CRITICAL. Prioritize the completeness and the process of the map. Good mapping is doable, and it's the only way to get usable 'information'.

4 — WHEN TO MAP

When you are ready to consider escalating a prompt/specific question into a mapping effort:

4-1 Ask yourself whether you care enough about your prompt/question to put a lot of effort into thoroughly mapping it.

4-2 If you don't care enough to do this:

• State to yourself that you don't really care about this.

• Return to IDK for your own stance.

• Re-bin the claim that prompted your question as 'emotional white noise'. Name the emotion(s) involved. (What was the 'real issue' that made you care about this claim?)

4-3 If yes, acknowledge that unless and until you've actually put in all the work needed to map the question, you still only have a question.

5 — GOOD MAPPING IN PRACTICE

5-1 Specify your question until it is 'researchable'. Always center on ONE specific question.

• As you map one question, you are likely to generate more as you go, or have to make a sub-map in order to proceed. It's no problem to shift your question as you go, but always work on ONE question at a time, and be explicit about what that question is at any given moment.

• If you move off a question because something else is easier or more urgent to answer, just make a note of the question you want to come back to. Keep a running list/connection web, so that you can map all related questions over time.

5-2 MAP THOROUGHLY.

• Mapping recap: A 'map' is all the possible answers to a question, held in the same field of vision. Make the strongest possible version of each possible answer, gather all available evidence, analyze each argument in light of the evidence, and compare them to each other. A person supporting any possible answer should be able to look at the map, see their position represented, and say "yes, that". The map should remain open to all new inputs in future, and should be revisited every time something changes about the relevant information.

To re-stress: MAPPING. IS. CRITICAL. Prioritize the completeness and the process of the map. (CARE whether you have a good map, and that will go a long long way towards having one.)

PS: Since mapping is the crux of the whole 'real information' problem, it's the most 'academic' part. It's not that hard to do once you get the idea, but also — even if it always seems like a slog, it's important to not take shortcuts. If you're inclined to say SNOOZE, and NERD, and OH MY GOD THIS IS ABSURD while doing this part...yes. But push on through.

STEPS FOR MAPPING A QUESTION:

1) Specify your question until it is 'researchable'. Write out your specific question.

• For ease, start with the most specific, binary, verifiable-claim question you can that speaks to your core concern.

2) Dump all possible answers to that question onto the field.

• This includes answers currently being proposed, and hypothetically-possible answers you arrive at yourself.

3) Working 'answer' by 'answer':

• Strip each answer for its 'arguments' — explicate the logic chain as fully as possible.

• Assess which facts are used to support each argument. Highlight these factual claims.

4) Survey the full set of 'facts' (verifiable claims) now on the map.

• Which facts are NOT disputed between all answer sets?

• Which are disputed?

5) Sub-map each disputed fact/claim.

• If clear evidence overturns a 'fact', strike it from the field (no penalty on the argument that provided it).

  • NOTE: You can be extra stringent, if wanted, for overturning a claim. But you SHOULD overturn in cases where eg an eyewitness's report is the totality of the 'input', and they themselves retract it.

• If, after research, multiple factual interpretations are still possible, highlight and annotate these claims accordingly.

  • Think about what deciding evidence would look like, if any.

  • If more evidence is ever relevant to deciding the question, return to update the map.

6) Survey the arguments on the map in light of the shared/disputed fact set:

• Has the assessment of facts changed the strength of any of the arguments?

  • Did any argument rely totally on facts that are now overturned?

    • If possibly redeeming evidence, make note of "If - then”

    • If no possible redeeming evidence, strike that 'answer’

• If an argument relied partially on facts that are now overturned, re-explicate it in the strongest form possible

  • If other evidence possible available, find, repeat steps from top to here

• What is the individual assessment of each argument's logical chain?

  • If broken irrevocably, strike

  • If rewritable in a stronger form, do so

7) Survey the arguments on the map relative to each other, going argument by argument:

• Which do all of the following: 1) account for all known verified facts; 2) speak directly to main logic/concerns of other positions; 3) are logically strong and self-consistent themselves? Make note of these favorably.

• Which do ANY of the following: 1) fail to account for some of the known facts, 2) rely on facts that were overturned, 3) do not have verified facts (only unverified, if any) 4) do not speak to the main logic/concerns of other positions; 5) are logically poor (weak inductive, invalid deductive, or inconsistent internally).

WARNINGS FOR THE HARDEST PARTS OF THIS PROCESS (from my perspective): separating specific questions; explicating full argument chains AS THEY WERE INTENDED; chasing down input data to make strongest version of each argument.

6 — CURATING SOURCES FOR EMOTIONAL WHITE NOISE (a mapping shortcut)

6-1 Spot-check your most frequently used sources for good/bad mapping.

• Make note of which sources achieve complete mapping, and on which topics.

• Make note of which sources present faulty/incomplete maps (any topic).

Re-upped rule: 'Atomize your judgment' should apply to sources. For example: you can consider a network → an individual reporter → an individual reporter on one topic → one article that person's done regarding a particular topic → different claims within that article (on a section, paragraph, sentence, or word level)...

RECAP OF MAJOR POINTS

• Atomize your judgments

• Default to "I don't know"

• Bin every claim you hear into either 1) a prompt for a question or 2) emotional white noise

• Note explicitly when your interest in a claim or question surges, and when it drops off

• Be specific about what question you're answering (especially if/as it shifts)

• Map thoroughly every time you want to answer a question

• Look for and favor good mapping in your casual sources for info

BONUS CONCEPT: ‘ARATIONAL CORES’

Way, way down, at the very bottom of every 'argument' (ie each potential 'answer' provided to a map) is an arational belief. This is a belief that can't be justified any further by logic/factual support, and is simply asserted.

When you hit these beliefs, a rational approach to analyzing them is no longer appropriate. (This is why we use the word 'arational', instead of 'irrational': 'irrational' implies that rationality is the right framework, but is ‘being done wrong'.) These beliefs could be thought of as our deepest values, and/or our deepest aspirations for both ourselves and the world: logic genuinely can't be used to support them or tear them down. These are the beliefs we simply have, and will act on. If given arguments about them, we will ignore these.

Again, EVERY argument terminates in an arational belief. In a given argument, we call this the 'arational core' of that argument.

Like many people, I am getting to the breakpoint with a family member. Specifically my younger brother. He is saying... problematic... things. “Hitler had the right idea” kind of things. Things said, that I cannot help but wonder: "Did I really just hear that?"

The problem is that these espoused positions should logically lead to violence, if he truly believes them. If nothing else, the espoused positions are correlated with violent people.

My brother is not a violent person. At least, I have never seen him be violent, or even directly mean to others. That is to say, if he has a dark side I have never witnessed it. In fact it is quite the contrary. To any given individual, I have only seen him express kindness.

All of the friends and family that we share a relationship with usually describe him along the lines of "having a good heart".

My struggle is holding both of these positions in my head at once. They seem so contradictory that I can only do one thing: push out the violence-indicating rhetoric.

But I shouldn't.

If he was someone I didn't know, and I heard these words, I would not associate with the person who said them. But he is my brother: I cannot imagine him being violent. My actions at this point seem to mirror my mind. I should cut him out, but I feel that it is an overreaction. He is not a violent person, so he doesn’t really mean what he says. My action wants to be — inaction. I am realizing, now that the position is so clear, that I have been here before with him. I have chosen inaction before on less offensive positions. What causes this paralysis?

It feels like I'm having a quantum thought and my brother is in a superposition — who I think he must be, and who I see him becoming. It cannot be both. But every time I try to observe the thought I get one result: "I can't imagine him being violent." This isn't how superpositions work. Where is the other position? Why can I not explore that one?

If I pause, and try to watch myself observe the thought, I can see this other position, as if out of the corner of my eye. Then my eye settles on the only thing I feel can be true — the only position I can seem to grasp.

I am fooling myself. I know this other position exists. I can imagine anyone being violent. It is just imagination. I can imagine Gandhi or Jesus being violent if I want to. Surely I can do the same with my brother? I don't want to imagine it. Like most people I am conflict avoidant. If I imagine him being violent, this somehow registers as "if I imagine this, it will bring me closer to conflict."

But I am already in conflict with him. It is just in my head. Always present, in the background, eating up precious brain cycles, making me anxious. The superposition exists. I can see it. I must engage with it.

What if he isn't violent yet? What if he is entertaining ideas that will lead him to violence? What if he is at a fork on his journey in life, and starting to head down a very dark path?

Is this really why I get so anxious? Is this why I am so angry when he expresses these hateful views? Yes the views can generate anger, but what if my anger really exists because I think I see the path he is on and he doesn’t?

Am I angry because on some level I recognize I am struggling to see the other half of the superposition in my own head? Can I not see it because it implies bad things? Does the anger exist because, subconsciously, I know I am fooling myself?

Am I angry because I resent him for putting this burden on me? If he ever becomes violent and I clung to my belief that he could never be, if I never confronted him, is that on me?

In this case, I am my brother's keeper. I also have a societal responsibility to respond to the things I’ve heard him say. I have a responsibility to try with anyone close enough to me that I might be able to reach yet.

But what does trying look like?

For starters, I must hold the contradiction in my head. The contradiction is everything. If I truly believe he is actually a good person, who is misguided, then I must also evenly weight the possibility he fully believes what he says. I must imagine him becoming violent. I must confront him if I want to rid myself of this contradiction, and the anxiety it produces. I must set my anger to the side for the moment. I must interrogate the other half of the superposition.

I must have him acknowledge to me the path he is going to choose.

I don't think he is too far gone yet. Even if he was, I must confront him. But I think there is still a chance I can reach him.

He seems to be in the 'trying it on phase' of these positions. He has not expressed these positions to all of the family members yet. He is sort of polling the audience to see if there are others with these views, but more importantly, how those closest to him respond. Clearly you can find community for anything on the internet, but there is something special about close family members. How many forum members or casual acquaintances am I worth to him? I don't think this is any explicit calculation he is running, but there is mental math happening. If this is true, how I react may be pivotal in the path he chooses. This is a high stakes and critical moment, where I might have an effect. Yet my first instinct was inaction...

So the two questions I need to ask myself:

1. Do I believe there is a contradiction between who I think he is versus where I think he is heading?

2. Am I close enough to my brother that he will value our relationship above holding these 'new' positions?

The first is clearly a yes. The second in some ways is just a matter of tone. If I think he is 'too far gone', then I should still try to confront him, but it will be very different than if I think he still cares greatly about me and how I think of him.

Which brings me back to the quantum thought. This time I need to imagine him being the worst possible version of himself that does not go against my evidence. This is what interrogating the other half of the superposition entails.

I should view all those comments about his "good heart" in a different light. People could have easily described him as a good person, but they often chose to focus on his "heart". In some ways this subtle choice of language could be interpreted that most people see a contradiction in the things he says and the actions they have observed from him. I generally interpret "they are a good person" to mean they are a wholly good person. Their words and actions align closely.

So far as I know, no mutual acquaintance has seen bad actions. If anything they have similar evidence: good actions, but says lots of incongruent things. On the other hand… most of our mutual acquaintances are family and friends of family. We are adults and so that shared phase of our life ended some time ago. I don't know all the people he has interacted with since, all the non-mutual acquaintances. What do they think of him? Have they witnessed him being violent? This is where "I don't know" should keep things even odds for me. Of course I want to assume anyone would give a similar description of "good heart". But I can't know that. That is me always wanting to believe the best in him. But this is exactly where I shouldn't. What if he has spent too much time around the wrong sort of people? What if he sought out the wrong sort of people, on his own initiative? What if he has done violent things in a setting where it was more "socially normalized"?

So largely he has two 'groups' of people. He has all the people we mutually grew up with, who he keeps up appearances with (consistency bias). And a group of 'adult acquaintances' where I assume these positions are more normal. These are the two communities he is doing the mental math with.

The fact that the latter community's ideas are entering the former's sphere is a sign he is heading down a different fork. He is trying to be a whole and consistent person everywhere. This is worrying, as it means he believes the positions enough, and values the community that holds them enough, to attempt to start updating his ideological direction change to those who knew him differently.

So what tone should I strike when I confront him?

First, I must seriously and firmly state the contradiction I see, the superposition. This is what is generating my anger and anxiety. I need resolution on this. I should not dwell on the implications of his espoused positions any more than is necessary to state my contradiction. So this would be my issue with him being/becoming a hateful and violent person. That is all I really need from his positions. If I want the 'brother I know' to return, I should also focus on the aspects that I admire in him. I should repeat my perception of him. I should state that I refuse to believe he is a violent person. This makes him required to tell me that he is a violent person and I am wrong. This would mean that even if he isn't a violent person yet, he believes in his new social connections enough to overcome his long held persona.

In this scenario, consistency bias (from growing up) was the only thing keeping him from adopting these positions. If my assessment is correct, then I should expect him to become violent. In this scenario I cannot reach him. I will never know if I could have (unlikely), but I will know I at least tried. If this is the case I should close the relationship, but make it clear that I am always available to talk if he is ever willing to shed these beliefs. I do not want to burn the bridge with the brother I used to have. I only want to burn the bridge with who he (apparently) is now. I must acknowledge to myself that I am not the one who can reach him. Perhaps there is someone else who can, otherwise he is the only one who can extract himself from these hateful beliefs.

Another scenario is that he will argue that his positions do not lead to violence. If this is the case then there is a chance that I can reach him. This would seem to indicate that he also doesn't see himself as a violent person, or he secretly is a violent person but doesn't want to be (needs help). This will be the longest, most arduous route to help my brother. We will need to first come to agreement on how to discuss anything. This is where having a clear process for how to have hard talks is essential. (If this scenario comes to pass, Ford has agreed to help me navigate it, as we probably won't have all the toolkits published yet. To be clear: I’m not going to try to argue directly against his conspiracy theories. But if we can start talking, we need a way to talk about him without him falling into bad patterns.)

The ideal scenario, of course, is that he acknowledges that I am right and that he has lost his way and wants to discuss why he went down this path. This now seems really unlikely. I think he is too far gone for this scenario. I will not confront him in this tone.

So in order to respond to him I need to make sure I am doing so with the two most likely outcomes in mind: He is too far gone OR There is a disagreement in perceptions and/or conclusion. Notice something? My initial position is not included. The 'ideal condition' is what I kept wanting to believe. If it were true, he would probably figure it out without me confronting him. But nothing supports this with clear-eyed assessment. Only the opposite scenario seems likely...

Share

Besides writing up my personal struggles for the internet, I want to take a moment to explicitly state the meta-commentary this post implies.

Here I am, trying to write a blog about how to think better and be a truth-seeker, and I still have trouble holding certain contradictions in my head, interrogating both states in the superposition. The emotions are particularly potent in my current situation. Emotions clearly are not short-circuiting my rational brain. I have the superposition, it is there and I 'know' both positions. My emotions are short-circuiting my observation of the superposition.

Our challenge is that emotions make us feel like we already have the rational position, when in reality we have a rationalized position. This leaves the other half of the superposition ignored. We have found our answer. We know it is the answer. So why explore other possibilities? In other words, emotions simply try to blind us to the rational result. This is good in some ways, it means we have the result. We just need to get better at correcting the effects of emotions, like how glasses correct faults in our eyes.

Unfortunately emotional reactions release chemicals (drug-like substances) in the brain, and the (truly) rational thoughts... are just not emotional. Our emotions have an unfair advantage, and we need to be aware of this. The more we care about something, the stronger the corrective lens we must use to assess it.

I think the world has a drug problem, and it isn't letting us see the answers we already have.

Subscribe now

My parents and I started out with a close and confiding relationship. No subject was off-limits, and we talked freely about life, religion, and politics. When disagreements happened, they were taken in stride: it was understood that we were all engaging in good faith, and 'on the same team' at the end of it all.

Then, around 2016...something happened. Or had been happening slowly, and started to happen faster.

At some point, I noticed that when we talked, the occasional factual mistake wasn't handled like it used to be. They'd say something that didn't sound right, I'd find proof and bring up how it affected their argument...and then they'd just dismiss the problem. Seemingly frustrated at me for pushing back, and acting like it didn't change anything. I really didn't know what to do with this. My parents had always stressed honesty and accuracy, and I wasn't even trying to disagree with their argument yet. I just wanted to hear the version that didn't have errors.

In other words, I really wasn't trying to "disagree" at all — I just wanted to get our shared facts straight. But 'getting the facts straight' was treated more and more like an unresolvable "difference of opinion" — "let's just agree to disagree". I was still willing to be careful with my arguments, and to take objections seriously. Why were they suddenly not? Add into this: my parents were always very kind people. Why were they unwilling to take care, even when their positions had dangerous implications for others?

I didn't like the growing sense of contradiction — how kind and thoughtful they remained about personal matters, while becoming increasingly callous and careless on "the issues". Our talks became prickly, then heated, then unsustainable. We came to a silent understanding to talk less about politics.

But some things were unavoidable. There were topics that directly affected me, or people that we all cared about. I thought these, at least, would give them pause enough to have real conversations again.

That didn't happen. These were people who’d said they’d do anything for me — but now they wouldn’t even listen, or follow up on any of my concerns themselves. Faced with me telling them I was afraid, and could be directly harmed by things they were supporting, they continued to treat it like a game of differing opinions. I was also, at this point, accused of "disrespect" and insulted with names, which was new. So we agreed explicitly to put politics off-limits.

We continued to go through the same motions of seeing each other and calling, but now I dreaded spending time with them. We 'didn't talk politics', but they would cross the line here and there, in little ways or big ones. They seemed to enjoy my upset sometimes.

Then came the conspiracy theory and extremist talking points.

I could watch a blatantly made-up piece of rhetoric be born in some dark corner of the internet — and then hear it eventually come out of my parents' mouths. (While we 'weren't talking politics'.) This started to happen with such depressing regularity that I could make a game out of guessing which ones, and how long it would take. These were dangerous, and I pushed back. As before, it didn't matter at all. Most times, I couldn't even get a foot in the door to make a case; when I did, they'd listen, say nothing, and then act as if it never happened.

Years passed like this. Things got worse. Everything got tangled, and the more politics affected my life, the less I could talk about my life with them. Which seemed alright to me — it had been ages since I'd gotten comfort from talking with them anyway. I tried, still, to show up for the relationship, and to respect the aspects of them that I still loved and admired. But it got harder and harder to handle the perpetually-escalating division, that contradiction between who they clearly saw themselves as — kind, smart people and devoted parents, same as they'd always been — and who I saw them as. More and more, they seemed like the worst kind of two-faced hypocrites: cheerfully piling on to problems, and then ready to cry crocodile tears when they saw the effects. The entirely predictable effects.

Contempt is the relationship killer. Eventually I hit a point of crisis, and reviewed what I knew.

I had taken care to tell them repeatedly how their beliefs/actions affected me. They routinely dismissed my concerns about anything and everything, even when those concerns stopped being hypothetical and turned into real, past-tense events. What rational conclusion should I take away from all this, except that I mattered less to them than beliefs they refused to question? What else was I supposed to think when they doubled-down on certain things, except that our core values must be fundamentally different? And if so...why was I trying so hard? My life had taken a turn for the difficult at this point, and was only going to get more so — why not just put down all this painful effort, and be done with it?

Well. There was one big problem.

The same contradiction that I hated about them, was also something I couldn't fully explain away. They were constantly promoting horrible things and defending the indefensible — but they were also shirt-off-their-back caring people in most instances. I despised them for their refusal to examine beliefs that would hurt me — in part because I still had so many reasons to believe that they would never hurt me on purpose. Or anyone else, for that matter.

I couldn't keep going on as we had been. But it just didn't make sense.

I'd been stewing on their behavior for years. Now I gave one last shot to totally reframing it — this time, deliberately plugging in the premise that they DO CARE about the truth, and DO CARE whether their beliefs/actions are based in fact. What if all the dismissals of evidence, and thoughtlessly repeated falsehoods, and insistence that certain falsehoods were right while refusing to check (and and and, ad nauseam)...were NOT evidence that they were rejecting the value of truth full-stop. What else would account for this behavior?

The hypothesis for this could be nicknamed 'big truth, little truth'. What if they believe so strongly in some big underlying TRUTH, that the little truths are shoved aside as basically just confusing distractions — NOT BECAUSE THEY DON'T CARE whether these things are true, but because it became so incredibly difficult to know whether any individual thing IS true, AND WHETHER ANY GIVEN THING SHOULD CHANGE THE ARGUMENT? What if it started to feel impossible to chase down every little back-and-forth piece of "evidence" in an increasingly chaotic and rage-soaked information landscape...so they just stuck to what they "knew"? The 'big truths'?

In short, what if their problem was the problem I'd always had — trying to keep a sense of self and reality while reality itself feels so slippy and in-process?

I was still depressed, and exhausted, and furious at them. No matter what explanation I could give, they had still belittled my very real concerns, they had still refused to even hear me out on things that had huge consequences for my life. So it took me about a week to work through my own emotions enough to even care whether our relationship continued. After that, I put together an approach that FULLY allowed room for this hypothesis. It involved a specific type of start, and if the start was successful, specific ways to proceed from there.

So far...it's working.

We're having our first political talks in almost a decade. It feels a lot more like it used to. I'm trying. They're trying. It's high effort, it took a lot of personalization, and it will be ongoing for a long time to come...but it's a good faith effort, and we're 'on the same team'.

It is too early to know how this is all going to play out. We've been at it for weeks now, but it's slow going, and much more time will have to pass before we get real indication of permanent change. Then, success or failure, I'll need some time to think about what happened. So...all of this is to say that a write-up with personal particulars will be a long time coming.

What we will be talking about immediately are some of the background framings used to inform my approach — especially what I tried to believe, and convey, in order to get further than I've ever gotten with this. So subscribe if you want those.

Subscribe now

In the meantime, if you have your own holiday gathering coming up with family, and have at least one previously-close relationship you no longer feel alright in —here are my two cents.

• If you still care about them or the relationship in some way...don't give up. And don't condemn yourself to rolling over forever.

• Use the in-person opportunity to observe your relationship, with as much calm and distance as possible. 1) Try to think about what, at the root of it all, is causing you to feel hurt, or anger, or contempt. What, specifically, is at the heart of your trouble with them? Are there any ways this core issue comes through in their behavior towards you? 2) Look at what you still love and admire about the person, and watch how those things come out in their behavior. Are their good and bad qualities related? Or does what you love/admire feel like a real contradiction with what you hate?

• Buy time, and distance, however you can. Don't start anything, and keep redirecting focus to the family gathering if someone else picks a fight. (Keep in mind: you can always follow up later, in many ways, but it's going to be better for you if that response is considered and on your own terms.)

Again, we hope to be able to offer more concrete advice for restarting these conversations as soon as possible. Or, if you think it may help, feel free to reply to this email — no one at On-Kilter is any sort of expert, but if you have something to share or ask, we’re happy to listen.

What I had was a goal -- an aspiration to be a good thinker. Ford had a process. The difference in our approaches made me feel really dumb: I realized that I had named the biases, but I couldn’t consistently avoid all of them all of the time. Ford wasn't any smarter than me, they just followed some rules -- their 'process'. The rules seemed obvious after they were explained. Once I was armed with the rules, I got a lot better. Over the years we stayed in contact and kept discussing...well everything: Philosophy, Religion, Politics.

Time passed. As the information ecosystem rapidly adapted to the internet, the amount of new claims and assertions skyrocketed. I had improved my thinking, but which sources should I trust? What information was worth considering? It all felt very overwhelming. Ford, like before, was ready for this: the 'process' already accounted for it. They helped me improve how to handle information from any source. As things evolved our discussions turned to why 'the process' was so effective even against the wild west of the internet.

The story would have ended there. But then the worst parts of the internet metastasized and caused our entire information ecosystem to degrade. Bad ideas multiplied. Carelessness with truth became standard. The worst consequence: truth nihilism. People want to believe that truth matters, but they get overwhelmed when they try to engage. Eventually they give up and become careless themselves. We all know this is a problem. We all agree it's dire. There have been countless articles, blogs, vlogs, etc. talking about it in great detail, and explaining why it happens. So why does nothing ever seem to get better? Why do people keep getting overwhelmed and giving up? It dawned on me: everyone has this goal -- an aspiration to make the truth matter again, so we can get back to real discussion. But so far no one has put forth an effective and actionable process. People keep pointing out the problem, but knowing the problem doesn't tell you what to do.

So I asked Ford if they would help me translate all the bits of their process to a shareable format. We had developed our own terms and references for ideas. We had years of discussions compressed down to a short list of rules. We're now going to share these as a series of toolkits. The advice here can be used no matter how little effort you can spare. And it will help the truth to matter again. It will help people feel less overwhelmed. It helped me.

We will probably drop the 'core' toolkit documents all at once (at some point in the coming weeks). The weekly posts will cover examples on how to use the toolkits, how various aspects were developed, analysis of the information ecosystem both past and present, and spontaneous contemplation on various topics that intersect with the broader project.

Subscribe now

P.S. -- If you're like me and read lots of rationalist blogs and books, all of this may sound like other stuff you've heard. But for me, so far all that other stuff boils down to: "Just do a better job thinking. Keep an open mind. Figure out who to listen to. Be scientific. Make predictions and check them." This is what I was trying to do before I met Ford. Ford’s ‘process’ (toolkits we will share) is the only thing I've found that is ACTUALLY focused on practical action. It is a categorical algorithm, that you can learn by rote and apply every time.

Plus, most of that other stuff is paywalled. ...So there's that.

Our core toolkit has six tiers, divided by the amount of effort needed to use the advice. At the lowest tiers, learn how to be low-engagement without ever being part of the problem. At the middle tiers, learn to confidently navigate an environment of information, misinformation, and missing information. At the highest levels, learn successfully used methods to start up hard conversations with people who seem to inhabit a totally different reality.

We will be publishing the core toolkit over the next few months, along with lots of practice examples. Up to and around that, we will be publishing commentary from our team: this will range from background on how the toolkit came to be and how it was battle-tested, to more 'thinking out loud' conversations about information processing.

The goal is to provide a calm, confidence-building space. If we succeed in lowering your stress around taking in information, that is fantastic. If we can make you into a roaming mental fortress fit to handle the roughest of information hellscapes, even better.

Subscribe now

Who we are:

On-Kilter is a small team, offering advice that MUST self-justify in order to be useful. So we will be writing under obvious pseudonyms, and withholding any information that might speak to credentials. (If it helps you to imagine that we have extensive educational backgrounds, many certifications, and long careers in relevant fields...go right ahead.)

Fortitude Dangersmith (aka Ford) writes the main Substack. Promise Rightway runs the social accounts, and manages infrastructure. Verity Linktracer will provide all citations.