Вы здесь

Новости LessWrong.com

Подписка на Лента Новости LessWrong.com Новости LessWrong.com
A community blog devoted to refining the art of rationality
Обновлено: 14 минут 45 секунд назад

Book review: Rethinking Consciousness

10 января, 2020 - 23:41
Published on January 10, 2020 8:41 PM UTC

Princeton neuroscientist Michael Graziano wrote the book Rethinking Consciousness (2019) to explain his "Attention Schema" theory of consciousness (endorsed by Dan Dennett![1]). If you don't want to read the whole book, you can get the short version in this 2015 article.

I'm particularly interested in this topic because, if we build AGIs, we ought to figure out whether they are conscious, and/or whether that question matters morally. (As if we didn't already have our hands full thinking about the human impacts of AGI!) This book is nice and concrete and computational, and I think it at least offers a start to answering the first part of that question.

What is attention schema theory?

There are two ingredients.

For the first ingredient, you should read Kaj Sotala's excellent review of Consciousness and the Brain by Stan Dehaene (or read the actual book!) To summarize, there is a process in the brain whereby certain information gets promoted up to a "Global Neuronal Workspace" (GNW), a special richly-connected high-level subnetwork of the brain. Only information in the GNW can be remembered and described—i.e., this is the information of which we are "aware". For example, if someting flashes in our field of view too quickly for us to "notice", it doesn't enter the GNW. So it does get processed to some extent, and can cause local brain activity that persists for a couple seconds, but will not cascade to a large, widespread signal with long-lasting effects.

Every second of every day, information is getting promoted to the GNW, and the GNW is processing it and pushing information into other parts of the brain. This process does not constitute all of cognition, but it's an important part.

The second ingredient is that the brain likes to build predictive models of things—Graziano calls them "schemas" or "internal models". If you know what an apple is, your brain has an "apple model", that describes apples' properties, behavior, affordances, etc. Likewise, we all have a "body schema", a deeply-rooted model that tracks where our body is, what it's doing, and how it works. If you have a phantom limb, that means your body schema has a limb where your actual body does not. As the phantom limb example illustrates, these schemas are deeply rooted, and not particularly subject to deliberate control.

Now put the two together, and you get an "attention schema", an internal model of the activity of the GNW, which he calls attention.[2] The attention schema is supposedly key to the mystery of consciousness.

Why does the brain build an attention schema? Graziano offers two reasons, and I'll add a third.

  • First, it's important that we control attention (it being central to cognition), and control theory says it's impossible to properly control something unless you're modeling it. Graziano offers an example of trying to ignore a distraction. Experiments show that, other things equal, this is easier if we are aware of the distraction. That's counter-intuitive, and supports his claim.

  • Second, the attention schema can also be used to model other people's attention, which is helpful for interacting with them, understanding them, deceiving them, etc.

  • Third (I would add), the brain is a thing that by default builds internal models of everything it encounters. The workings of the GNW obviously has a giant impact on the signals going everywhere in the brain, so of course the brain is going to try to build a predictive model of it! I mention this partly because of my blank-slate-ish sympathies, but I think it's an important possibility to keep in mind, because it would mean that even if we desperately want to build a human-cognition-like AGI without an attention schema (if we want AGIs to be unconscious for ethical reasons; more on which below), it might be essentially impossible.

To be clear, if GNW is "consciousness" (as Dehaene describes it), then the attention schema is "how we think about consciousness". So this seems to be at the wrong level! This is a book about consciousness; shouldn't we be talking directly about the nature of consciousness itself?? I was confused about this for a while. But it turns out, he wants to be one level up! He thinks that's where the answers are, in the "the meta-problem of consciousness". See below.

When people talk about consciousness, they're introspecting about their attention schema

Let's go through some examples.

Naive description: I have a consciousness, and I can be aware of things, like right now I'm aware of this apple.

...and corresponding sophisticated description: One of my internal models is an attention schema. According to that schema, attention has a particular behavior wherein attention kinda "takes possession" of a different internal model, e.g. a model of a particular apple. Objectively, we would say that this happens when the apple model becomes active in the GNW.

Naive description: My consciousness is not a physical thing with color, shape, texture. So it's sorta metaphysical, although I guess it's roughly located in my head.

...and corresponding sophisticated description: Just as my internal model of "multiplication" has no property of "saltiness", by the same token, my attention schema describes attention as having no color, shape, or texture.

Naive desciption: I have special access to my own consciousness. I alone can truly experience my experiences.

...and corresponding sophisticated description: The real GNW does not directly interact with other people; it only interacts with the world by affecting my own actions. Reflecting that fact, my attention schema describes attention as a thing to which I have privileged access.

Naive description An intimate part of my consciousness is its tie to long-term memory. If you show me a video of me going scuba diving this morning, and I absolutely have no memory whatsoever of it, and you can prove that the video is real, well I mean, I don't know what to say, I must have been unconscious or something!

...and corresponding sophisticated description: Essentially everything that enters the GNW leaves at least a slight trace in long-term memory. Thus, one aspect of my attention schema is that it describes attention and memory as inextricably linked. According to my internal models, when attention "takes possession" of some piece of information, it leaves a trace in long-term memory, and conversely, nothing can get into long-term memory unless attention first takes possession of it.

Naive description: Hey, hey, what are you going on about "internal models" and "attention schema"? I don't know anything about that. I know what my consciousness is, I can feel it. It's not a model, it's not a computation, it's not a physical thing. (And don't call me naive!)

...and corresponding sophisticated description: All my internal models are simplified entities, containing their essential behavior and properties, but not usually capturing the nuts-and-bolts of how they work in the real world. (In a programming analogy, you could say that we're modeling the GNW's API & documentation, not its implementation.) Thus, my attention schema does not involve neurons or synapses or GNWs or anything like that, even if, in reality, that's what it's modeling.

The meta-problem of consciousness

The "hard problem of consciousness" is "why is there an experience of consciousness; why does information processing feel like anything at all?"

The "meta-problem of consciousness" is "why do people believe that there's a hard problem of consciousness?"

The meta-problem has the advantage of having obvious and non-confusing methods of attack: the belief that there's a hard problem of consciousness is an observable output of the brain, and can be studied by normal cognitive neuroscience.

But the real head-scratcher is: If we have a complete explanation of the meta-problem, is there anything left to explain regarding the hard problem? Graziano's answer seems to be a resounding "No!", and we end up with conversations like these:

Normal Person: What about qualia?

Person Who Has Solved The Meta-Problem Of Consciousness: Let me explain why the brain, as an information processing system, would ask the question "What about qualia"...

NP: What about subjective experience?

PWHSTMPOC: Let me explain why the brain, as an information processing system, would ask the question "What about subjective experience"...

NP: You're not answering my questions!

PWHSTMPOC: Let me explain why the brain, as an information processing system, would say "You're not answering my questions"...

...

The book goes through this type of discussion several times. I feel a bit torn. One side of me says: obviously Graziano's answers are correct, and obviously no other answer is possible. The other side of me says: No no no, he did not actually answer these questions!!

On reflection, I have to side with "Obviously Graziano's are correct, and no other answer is possible." But I still find it annoying and deeply unsatisfying.

Illusionism

Graziano says that his theory is within the philosophical school if thought called "Illusionism". But he thinks that term is misleading. He says it's not "illusion as in mirage", but "illusion as in mental construction", like how everything we see is an "illusion" rather than raw perceptual data.

Emulations

He has a fun chapter on brain uploading, which is not particularly related to the rest of the book. He discusses some fascinating neuroscience aspects of brain-scanning, like the mystery of whether glial cells do computations, but spends most of the time speculating about the bizarre implications for society.

Implications for AGI safety

He suggests that, since humans are generally pro-social, and part of that comes from modeling each other using attention schemas, perhaps the cause of AGI Safety could be advanced by deliberately building conscious AGIs with attention schemas (and, I presume, other human-like emotions). Now, he's not a particular expert on AGI Safety, but I think this is not an unreasonable idea; in fact it's one that I'm very interested in myself. (We don't have to blindly copy human emotions ... we can turn off jealousy etc.)

Implications for morality

One issue where Graziano is largely silent is the implications for moral philosophy.

For example, someday we'll have to decide: When we build AGIs, should we assign them moral weight? Is it OK to turn them off? Are our AGIs suffering? How would we know? Should we care? If humans go extinct but conscious AGIs have rich experiences as they colonize the universe, do we think of them as our children/successors? Or as our hated conquerers?

I definitely share the common intuition is that we should care about the suffering of things that are conscious (and/or sentient, I'm not sure what the difference is). However, in attention schema theory, there does not seem to be a sharp dividing line between "things with an attention schema" and "things without an attention schema", especially in the wide space of all possible computations. There are (presumably) computations that arguably involve something like an "attention schema" but with radically alien properties. There doesn't seem to be any good reason that, out of all the possible computational processes in the universe, we should care only and exactly about computations involving an attention schema. Instead, the picture I get is more like we're taking an ad-hoc abstract internal model and thoughtlessly reifying it. It's like if somebody worshipped the concept of pure whiteness, and went searching the universe for things that match that template, only to discover that white is a mixture of colors, and thus pure whiteness—when taken to be a literal description of a real-world phenomenon—simply doesn't exist. What then?

It's a mess.

So, as usual when I start thinking too hard about philosophy, I wind up back at Dentin's Prayer of the Altruistic Nihilist:

Why do I exist? Because the universe happens to be set up this way. Why do I care (about anything or everything)? Simply because my genetics, atoms, molecules, and processing architecture are set up in a way that happens to care.

So, where does that leave us? Well, I definitely care about people. If I met an AGI that was pretty much exactly like a nice person, inside and out, I would care about it too (for direct emotional reasons), and I would feel that caring about it is the right thing to do (for intellectual consistency reasons). For AGIs running more alien types of algorithms—man, I just have no idea.

  1. More specifically, I went to a seminar where Graziano explained his theory, and then Dan Dennett spoke and said that he had essentially nothing to disagree with concerning what Graziano had said. I consider that more-or-less an "endorsement", but I may be putting words in his mouth. ↩︎

  2. I found his discussion of "attention" vs "awareness" confusing. I'm rounding to the nearest theory that makes sense to me, which might or might not be exactly what he was trying to describe. ↩︎



Discuss

Example: Markov Chain

10 января, 2020 - 23:19
Published on January 10, 2020 8:19 PM UTC

The previous post in this sequence discussed how to throw away information in causal DAGs. This post provides a detailed example using a Markov chain.

Suppose we have an n-state Markov chain (CS people: picture a finite state machine with n states and random state transitions at each timestep). A matrix of state transition probabilities .mjx-chtml {display: inline-block; line-height: 0; text-indent: 0; text-align: left; text-transform: none; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; word-wrap: normal; word-spacing: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0; min-height: 0; border: 0; margin: 0; padding: 1px 0} .MJXc-display {display: block; text-align: center; margin: 1em 0; padding: 0} .mjx-chtml[tabindex]:focus, body :focus .mjx-chtml[tabindex] {display: inline-table} .mjx-full-width {text-align: center; display: table-cell!important; width: 10000em} .mjx-math {display: inline-block; border-collapse: separate; border-spacing: 0} .mjx-math * {display: inline-block; -webkit-box-sizing: content-box!important; -moz-box-sizing: content-box!important; box-sizing: content-box!important; text-align: left} .mjx-numerator {display: block; text-align: center} .mjx-denominator {display: block; text-align: center} .MJXc-stacked {height: 0; position: relative} .MJXc-stacked > * {position: absolute} .MJXc-bevelled > * {display: inline-block} .mjx-stack {display: inline-block} .mjx-op {display: block} .mjx-under {display: table-cell} .mjx-over {display: block} .mjx-over > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-under > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-stack > .mjx-sup {display: block} .mjx-stack > .mjx-sub {display: block} .mjx-prestack > .mjx-presup {display: block} .mjx-prestack > .mjx-presub {display: block} .mjx-delim-h > .mjx-char {display: inline-block} .mjx-surd {vertical-align: top} .mjx-mphantom * {visibility: hidden} .mjx-merror {background-color: #FFFF88; color: #CC0000; border: 1px solid #CC0000; padding: 2px 3px; font-style: normal; font-size: 90%} .mjx-annotation-xml {line-height: normal} .mjx-menclose > svg {fill: none; stroke: currentColor} .mjx-mtr {display: table-row} .mjx-mlabeledtr {display: table-row} .mjx-mtd {display: table-cell; text-align: center} .mjx-label {display: table-row} .mjx-box {display: inline-block} .mjx-block {display: block} .mjx-span {display: inline} .mjx-char {display: block; white-space: pre} .mjx-itable {display: inline-table; width: auto} .mjx-row {display: table-row} .mjx-cell {display: table-cell} .mjx-table {display: table; width: 100%} .mjx-line {display: block; height: 0} .mjx-strut {width: 0; padding-top: 1em} .mjx-vsize {width: 0} .MJXc-space1 {margin-left: .167em} .MJXc-space2 {margin-left: .222em} .MJXc-space3 {margin-left: .278em} .mjx-test.mjx-test-display {display: table!important} .mjx-test.mjx-test-inline {display: inline!important; margin-right: -1px} .mjx-test.mjx-test-default {display: block!important; clear: both} .mjx-ex-box {display: inline-block!important; position: absolute; overflow: hidden; min-height: 0; max-height: none; padding: 0; border: 0; margin: 0; width: 1px; height: 60ex} .mjx-test-inline .mjx-left-box {display: inline-block; width: 0; float: left} .mjx-test-inline .mjx-right-box {display: inline-block; width: 0; float: right} .mjx-test-display .mjx-right-box {display: table-cell!important; width: 10000em!important; min-width: 0; max-width: none; padding: 0; border: 0; margin: 0} .MJXc-TeX-unknown-R {font-family: monospace; font-style: normal; font-weight: normal} .MJXc-TeX-unknown-I {font-family: monospace; font-style: italic; font-weight: normal} .MJXc-TeX-unknown-B {font-family: monospace; font-style: normal; font-weight: bold} .MJXc-TeX-unknown-BI {font-family: monospace; font-style: italic; font-weight: bold} .MJXc-TeX-ams-R {font-family: MJXc-TeX-ams-R,MJXc-TeX-ams-Rw} .MJXc-TeX-cal-B {font-family: MJXc-TeX-cal-B,MJXc-TeX-cal-Bx,MJXc-TeX-cal-Bw} .MJXc-TeX-frak-R {font-family: MJXc-TeX-frak-R,MJXc-TeX-frak-Rw} .MJXc-TeX-frak-B {font-family: MJXc-TeX-frak-B,MJXc-TeX-frak-Bx,MJXc-TeX-frak-Bw} .MJXc-TeX-math-BI {font-family: MJXc-TeX-math-BI,MJXc-TeX-math-BIx,MJXc-TeX-math-BIw} .MJXc-TeX-sans-R {font-family: MJXc-TeX-sans-R,MJXc-TeX-sans-Rw} .MJXc-TeX-sans-B {font-family: MJXc-TeX-sans-B,MJXc-TeX-sans-Bx,MJXc-TeX-sans-Bw} .MJXc-TeX-sans-I {font-family: MJXc-TeX-sans-I,MJXc-TeX-sans-Ix,MJXc-TeX-sans-Iw} .MJXc-TeX-script-R {font-family: MJXc-TeX-script-R,MJXc-TeX-script-Rw} .MJXc-TeX-type-R {font-family: MJXc-TeX-type-R,MJXc-TeX-type-Rw} .MJXc-TeX-cal-R {font-family: MJXc-TeX-cal-R,MJXc-TeX-cal-Rw} .MJXc-TeX-main-B {font-family: MJXc-TeX-main-B,MJXc-TeX-main-Bx,MJXc-TeX-main-Bw} .MJXc-TeX-main-I {font-family: MJXc-TeX-main-I,MJXc-TeX-main-Ix,MJXc-TeX-main-Iw} .MJXc-TeX-main-R {font-family: MJXc-TeX-main-R,MJXc-TeX-main-Rw} .MJXc-TeX-math-I {font-family: MJXc-TeX-math-I,MJXc-TeX-math-Ix,MJXc-TeX-math-Iw} .MJXc-TeX-size1-R {font-family: MJXc-TeX-size1-R,MJXc-TeX-size1-Rw} .MJXc-TeX-size2-R {font-family: MJXc-TeX-size2-R,MJXc-TeX-size2-Rw} .MJXc-TeX-size3-R {font-family: MJXc-TeX-size3-R,MJXc-TeX-size3-Rw} .MJXc-TeX-size4-R {font-family: MJXc-TeX-size4-R,MJXc-TeX-size4-Rw} .MJXc-TeX-vec-R {font-family: MJXc-TeX-vec-R,MJXc-TeX-vec-Rw} .MJXc-TeX-vec-B {font-family: MJXc-TeX-vec-B,MJXc-TeX-vec-Bx,MJXc-TeX-vec-Bw} @font-face {font-family: MJXc-TeX-ams-R; src: local('MathJax_AMS'), local('MathJax_AMS-Regular')} @font-face {font-family: MJXc-TeX-ams-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_AMS-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_AMS-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_AMS-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-B; src: local('MathJax_Caligraphic Bold'), local('MathJax_Caligraphic-Bold')} @font-face {font-family: MJXc-TeX-cal-Bx; src: local('MathJax_Caligraphic'); font-weight: bold} @font-face {font-family: MJXc-TeX-cal-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-R; src: local('MathJax_Fraktur'), local('MathJax_Fraktur-Regular')} @font-face {font-family: MJXc-TeX-frak-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-B; src: local('MathJax_Fraktur Bold'), local('MathJax_Fraktur-Bold')} @font-face {font-family: MJXc-TeX-frak-Bx; src: local('MathJax_Fraktur'); font-weight: bold} @font-face {font-family: MJXc-TeX-frak-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-BI; src: local('MathJax_Math BoldItalic'), local('MathJax_Math-BoldItalic')} @font-face {font-family: MJXc-TeX-math-BIx; src: local('MathJax_Math'); font-weight: bold; font-style: italic} @font-face {font-family: MJXc-TeX-math-BIw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-BoldItalic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-BoldItalic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-BoldItalic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-R; src: local('MathJax_SansSerif'), local('MathJax_SansSerif-Regular')} @font-face {font-family: MJXc-TeX-sans-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-B; src: local('MathJax_SansSerif Bold'), local('MathJax_SansSerif-Bold')} @font-face {font-family: MJXc-TeX-sans-Bx; src: local('MathJax_SansSerif'); font-weight: bold} @font-face {font-family: MJXc-TeX-sans-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-I; src: local('MathJax_SansSerif Italic'), local('MathJax_SansSerif-Italic')} @font-face {font-family: MJXc-TeX-sans-Ix; src: local('MathJax_SansSerif'); font-style: italic} @font-face {font-family: MJXc-TeX-sans-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-script-R; src: local('MathJax_Script'), local('MathJax_Script-Regular')} @font-face {font-family: MJXc-TeX-script-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Script-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Script-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Script-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-type-R; src: local('MathJax_Typewriter'), local('MathJax_Typewriter-Regular')} @font-face {font-family: MJXc-TeX-type-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Typewriter-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Typewriter-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Typewriter-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-R; src: local('MathJax_Caligraphic'), local('MathJax_Caligraphic-Regular')} @font-face {font-family: MJXc-TeX-cal-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-B; src: local('MathJax_Main Bold'), local('MathJax_Main-Bold')} @font-face {font-family: MJXc-TeX-main-Bx; src: local('MathJax_Main'); font-weight: bold} @font-face {font-family: MJXc-TeX-main-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-I; src: local('MathJax_Main Italic'), local('MathJax_Main-Italic')} @font-face {font-family: MJXc-TeX-main-Ix; src: local('MathJax_Main'); font-style: italic} @font-face {font-family: MJXc-TeX-main-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-R; src: local('MathJax_Main'), local('MathJax_Main-Regular')} @font-face {font-family: MJXc-TeX-main-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-I; src: local('MathJax_Math Italic'), local('MathJax_Math-Italic')} @font-face {font-family: MJXc-TeX-math-Ix; src: local('MathJax_Math'); font-style: italic} @font-face {font-family: MJXc-TeX-math-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size1-R; src: local('MathJax_Size1'), local('MathJax_Size1-Regular')} @font-face {font-family: MJXc-TeX-size1-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size1-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size1-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size1-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size2-R; src: local('MathJax_Size2'), local('MathJax_Size2-Regular')} @font-face {font-family: MJXc-TeX-size2-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size2-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size2-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size2-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size3-R; src: local('MathJax_Size3'), local('MathJax_Size3-Regular')} @font-face {font-family: MJXc-TeX-size3-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size3-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size3-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size3-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size4-R; src: local('MathJax_Size4'), local('MathJax_Size4-Regular')} @font-face {font-family: MJXc-TeX-size4-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size4-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size4-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size4-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-R; src: local('MathJax_Vector'), local('MathJax_Vector-Regular')} @font-face {font-family: MJXc-TeX-vec-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-B; src: local('MathJax_Vector Bold'), local('MathJax_Vector-Bold')} @font-face {font-family: MJXc-TeX-vec-Bx; src: local('MathJax_Vector'); font-weight: bold} @font-face {font-family: MJXc-TeX-vec-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Bold.otf') format('opentype')} Tji gives the probability of transitioning to state j when the system starts the timestep in state i. Writing the state at time t as Xt, we have P[Xt+1=j|Xt=i]=Tji. More generally, P[Xt+m=j|Xt=i]=(Tm)ji, where Tm denotes a matrix power (i.e. T matrix-multiplied by itself m times). To complete the specification, we’ll assume that the system starts in a random state X0 at time 0, with the initial distribution P[X0] given.

As a causal DAG, this system is just a chain: the state at time t depends only on the state at time t−1:

X0→X1→X2→X3→...

People typically draw basic Markov chains the same way we draw finite state machines: a graph with one node for each state, and arcs indicating transitions. Unlike an FSM, where the next arc is chosen by a symbol from some input stream, here the next arc is chosen randomly - so each arc has a probability associated with it. An example:

This is NOT a causal diagram, it is a state transition diagram. It says that, if the system is in state 1, then at the next timestep it will randomly transition to state 1, 2, or 5. (I haven’t included the probabilities on each arc; all that matters for our purposes is that each arc shown has nonzero probability.) Since we have two graph representations of the system (the state transition diagram and the causal DAG), I will generally refer to vertices in the state transition diagram “states” and vertices Xt in the causal diagram as “nodes”.

What happens if we throw away long-term-irrelevant information from a node in this Markov chain?

Here’s the idea:

  • Pick the node Xt
  • Pick the set of nodes from X0 to Xt+m−1 for some large-ish m (we’ll denote this set X<t+m)
  • Throw away all info from Xt which is not relevant to nodes outside X<t+m: replace Xt with X′t, a representation of the function x→P[Xt+m=x|Xt].

Let’s think about what that last piece looks like. Xt could be any of the states 1 through 6; X′t must assign different values to any two states with different distributions P[Xt+m|Xt]. But for large m, many of the states will have (approximately) the same long-run distribution P[Xt+m|Xt] - this is the foundational idea of ergodicity. In the example above, nodes 5 & 6 will have the same long-run distribution, and nodes 2, 3, 4 will have the same long-run distribution.

To see why, imagine what happens if we start in state 5, assuming that the 5 -> 6 transition is much more likely than the 5 -> 5 transition. Well, since 5 -> 6 is much more likely than 5 -> 5, we’ll probably jump to state 6 next. And state 6 always jumps back to 5, so in two steps we’ll be back to 5. And so forth - back and forth, alternating between state 5 and 6 every timestep. But every one in awhile, we’ll jump from 5 -> 5, throwing the back-and-forth oscillation out of sync. If we imagine two copies of this chain running side-by-side, they’d start out oscillating in sync, but eventually drift out of sync. If we walk away for a while and look back at the chain much later, we’d expect that it’s roughly equally likely to be in state 5 or 6, regardless of which it started in.

That’s the key: if the chain started in state 5 or 6, with 5 -> 6 much more likely than 5 -> 5, than after a while, it would be roughly equally likely to be in state 5 or state 6. P[Xt+m|Xt=5]≈P[Xt+m|Xt=6] for large m. Even if 5 -> 6 is not much more likely than 5 -> 5, the two long-run distributions will still be the same - the long-run probabilities of 5 and 6 just won’t be roughly equal (we’ll stay in state 5 somewhat more often than 6).

A more general criteria:

  • View the state transition diagram as a directed graph, and ask which states are connected in both directions - i.e. a set of states in which we can reach any state from any other by following the arrows
  • Some arrows “knock oscillations out of sync” - read up on reducibility and ergodicity in Markov chains for technical details (I first saw this stuff in an operations research class)

If both of these criteria are met for some set of states, then each of those states i implies the same long-run behavior P[Xt+m|Xt=i].

Getting back to our abstraction: X′t doesn’t need to distinguish between states 5 and 6, or between states 2, 3, 4. Our states are grouped like this:

… and X′t is A, B, or C. Our causal diagram looks exactly like before, with X′t in place of Xt:

X0→...Xt−1→X′t→Xt+1…

We need to choose representative Xt-values for each of A, B, C, so we’ll pick A→1,B→6,C→3. So, if Xt−1=6, then X′t is B with probability 1 (since Xt is 5 or 6, both of which map to B). Xt+1 is then chosen as though Xt were 6, since 6 is our representative value for B.

Our abstract model no longer supports short-range queries around X′t. To see what goes wrong, consider P[Xt+1=6|Xt−1=6], assuming once again that 5 -> 6 is much more likely than 5 -> 5. In the original model, this gave rise to oscillation between states 5 and 6, so if the system was in state 6 at time t−1, then it would most likely be in state 6 again at time t+1. But in the new model, X′t throws away information distinguishing states 5 and 6 - both are just “B”. If Xt−1 = 6, then X′t = B, and Xt+1 behaves as though Xt were the representative value 6 - implying that Xt+1 is 5, rather than 6. No match :(.

Yet this does not impact the validity of long-range queries at all! Because both Xt=5 and Xt=6 imply the same long-run predictions, the model does support long-range queries, like P[Xt+m+4|Xt−1].

Finally, we can imagine cleaning up the model a bit by abstracting the whole chain, rather than just one node. Using the same info-throw-away transformation on every node, the abstraction looks like this:

Intuitively, not only do we have a Markov chain on the high-level variables X′t, we also have enough information in the high-level model to predict correlations between low-level Xt, as long as the Xt's we query are at least m timesteps apart. That's the property which makes this abstraction "natural" - more on that later.



Discuss

Offer of co-authorship

10 января, 2020 - 20:44
Published on January 10, 2020 5:44 PM UTC

I am offering interested persons to become a co-author on (a revised version of) my paper "forecasting using incomplete models". The paper was rejected from the Electronic Journal of Statistics with possibility to resubmit. The main criticism of the reviewers was: not enough citations and discussion of relation to other work. The job of the collaborator would be to revise the paper in order to (i) address this concern (ii) improve the overall presentation and make it more accessible. After that, the paper will be resubmitted to EJS (and/or other publications venues, according to what we will decide together).

Naturally, I will work closely with the collaborator and review the changes ey make, but ey are expected to do the lion share of the work in the revision.

The requirements for a candidate collaborator are:

  • Must: The knowledge required to understand the paper in full, including all the technical details. It is fine if ey need to refresh eir memory or look up particular theorems / definitions, but ey need to have enough background for that to be efficient. To see what sort of knowledge is necessary, one is advised to skim the paper. Mostly it involves measure theory, probability theory and functional analysis.

  • Advantage: Strong background in statistics, familiarity with the relevant literature

  • Advantage: Experience with doing literature surveys on technical subjects

  • Advantage: Experience with writing good explanations of technical subjects

  • Advantage: Experience with peer-reviewed academic publications

In order to apply, email me at rot13 of inarffn.xbfbl@vagryyvtrapr.bet. In your email, please provide me with (i) your name (ii) your background (iii) the extent to which you are comptaible with the above requirements and (iv) your motivation for applying.



Discuss

What is Success in an Immoral Maze?

10 января, 2020 - 16:20
Published on January 10, 2020 1:20 PM UTC

Previously in Sequence: Moloch Hasn’t WonPerfect CompetitionImperfect CompetitionDoes Big Business Hate Your Family?What is Life in an Immoral Maze?Stripping Away the Protections

Immoral Mazes are terrible places to be. Much worse than they naively appear. They promise the rewards and trappings of success. Do not be fooled. 

If there is one takeaway I want everyone to get from the whole discussion of Moral Mazes, it is this:

Being in an immoral maze is not worth it. They couldn’t pay you enough. Even if they could, they definitely don’t. If you end up CEO, you still lose. These lives are not worth it. Do not be a middle manager at a major corporation or other organization that works like this. Do not sell your soul.

When one works for an immoral maze, what is one hoping for? What is success?

Suppose you persevere. You make the sacrifices. Become the person you need to become. Put in the work day after day. Fortune smiles on you and you win out against all the others doing the same thing. You succeed.

What is success? What do you get in exchange?

For some managers, the drive for success is a quest for the generous financial rewards that high corporate position brings. For others, success means the freedom to define one’s work role with some latitude, to “get out from under the thumb of others.” For still others, it means the chance to gain power and to exert one’s will, to “call the shots,” to “do it my way,” or to know the curiously exhilarating pleasure of controlling other people’s fates. For still others, the quest for success expresses a deep hunger for the recognition and accolades of one’s peers. (Location 955, Quote 118)

This a warning. “Success,” in context, does not mean happiness. It does not make you healthy. It does not improve your reproductive fitness. It does not reflect or spread the values that you (one would hope) had when you stared down that road.

It gives you money. But in terms of actual meaningful personal consumption, you can’t really do much with it beyond status competitions. If you had plans to do something good with the money, by the time the day arrives, it is highly unlikely you’ll do it. You have changed yourself to succeed on your journey.

Even after you ‘succeed’ you probably keep putting tons of hours into the job in ways no amount of money can compensate for, once you already had basically enough.

What was the point? What are you even doing?

Note that failure is indeed much worse than success. You still paid all the sunk costs, including everything you are. You’ve invested a ton in and become very invested in local status hierarchies, and in the quest to climb them, which you have failed. Being ‘under the thumb’ of others who succeeded where you failed is deeply unpleasant – and is the most likely outcome, since the math says most who try will fail.

This is a song with some explicit content about what happens when one disregards this warning, and chooses poorly, although it misses perhaps the most important questions. Who am I? What have I become?

https://www.youtube.com/watch?v=5IsSpAOD6K8

Lyrics here.

Remember that maze conditions are not unique to corporations.

All of this holds true in any sufficiently large organization, to an extent that increases with its size, and will have the same effects if you seek to ascend the hierarchy within.

Size matters, but size is far from the only thing that matters. Some very small organizations effectively have very high maze levels. Some large organizations have relatively low maze levels.

Avoiding mazes is easier said than done. The first step is identifying them, where I will offer some heuristics in the next post. 

The second step after that is what to do about it, especially in difficult circumstances. Many of us believe we need the support of mazes in order to survive. At least for the moment, not all of us are wrong. 



Discuss

On Being Robust

10 января, 2020 - 06:51
Published on January 10, 2020 3:51 AM UTC

.mjx-chtml {display: inline-block; line-height: 0; text-indent: 0; text-align: left; text-transform: none; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; word-wrap: normal; word-spacing: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0; min-height: 0; border: 0; margin: 0; padding: 1px 0} .MJXc-display {display: block; text-align: center; margin: 1em 0; padding: 0} .mjx-chtml[tabindex]:focus, body :focus .mjx-chtml[tabindex] {display: inline-table} .mjx-full-width {text-align: center; display: table-cell!important; width: 10000em} .mjx-math {display: inline-block; border-collapse: separate; border-spacing: 0} .mjx-math * {display: inline-block; -webkit-box-sizing: content-box!important; -moz-box-sizing: content-box!important; box-sizing: content-box!important; text-align: left} .mjx-numerator {display: block; text-align: center} .mjx-denominator {display: block; text-align: center} .MJXc-stacked {height: 0; position: relative} .MJXc-stacked > * {position: absolute} .MJXc-bevelled > * {display: inline-block} .mjx-stack {display: inline-block} .mjx-op {display: block} .mjx-under {display: table-cell} .mjx-over {display: block} .mjx-over > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-under > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-stack > .mjx-sup {display: block} .mjx-stack > .mjx-sub {display: block} .mjx-prestack > .mjx-presup {display: block} .mjx-prestack > .mjx-presub {display: block} .mjx-delim-h > .mjx-char {display: inline-block} .mjx-surd {vertical-align: top} .mjx-mphantom * {visibility: hidden} .mjx-merror {background-color: #FFFF88; color: #CC0000; border: 1px solid #CC0000; padding: 2px 3px; font-style: normal; font-size: 90%} .mjx-annotation-xml {line-height: normal} .mjx-menclose > svg {fill: none; stroke: currentColor} .mjx-mtr {display: table-row} .mjx-mlabeledtr {display: table-row} .mjx-mtd {display: table-cell; text-align: center} .mjx-label {display: table-row} .mjx-box {display: inline-block} .mjx-block {display: block} .mjx-span {display: inline} .mjx-char {display: block; white-space: pre} .mjx-itable {display: inline-table; width: auto} .mjx-row {display: table-row} .mjx-cell {display: table-cell} .mjx-table {display: table; width: 100%} .mjx-line {display: block; height: 0} .mjx-strut {width: 0; padding-top: 1em} .mjx-vsize {width: 0} .MJXc-space1 {margin-left: .167em} .MJXc-space2 {margin-left: .222em} .MJXc-space3 {margin-left: .278em} .mjx-test.mjx-test-display {display: table!important} .mjx-test.mjx-test-inline {display: inline!important; margin-right: -1px} .mjx-test.mjx-test-default {display: block!important; clear: both} .mjx-ex-box {display: inline-block!important; position: absolute; overflow: hidden; min-height: 0; max-height: none; padding: 0; border: 0; margin: 0; width: 1px; height: 60ex} .mjx-test-inline .mjx-left-box {display: inline-block; width: 0; float: left} .mjx-test-inline .mjx-right-box {display: inline-block; width: 0; float: right} .mjx-test-display .mjx-right-box {display: table-cell!important; width: 10000em!important; min-width: 0; max-width: none; padding: 0; border: 0; margin: 0} .MJXc-TeX-unknown-R {font-family: monospace; font-style: normal; font-weight: normal} .MJXc-TeX-unknown-I {font-family: monospace; font-style: italic; font-weight: normal} .MJXc-TeX-unknown-B {font-family: monospace; font-style: normal; font-weight: bold} .MJXc-TeX-unknown-BI {font-family: monospace; font-style: italic; font-weight: bold} .MJXc-TeX-ams-R {font-family: MJXc-TeX-ams-R,MJXc-TeX-ams-Rw} .MJXc-TeX-cal-B {font-family: MJXc-TeX-cal-B,MJXc-TeX-cal-Bx,MJXc-TeX-cal-Bw} .MJXc-TeX-frak-R {font-family: MJXc-TeX-frak-R,MJXc-TeX-frak-Rw} .MJXc-TeX-frak-B {font-family: MJXc-TeX-frak-B,MJXc-TeX-frak-Bx,MJXc-TeX-frak-Bw} .MJXc-TeX-math-BI {font-family: MJXc-TeX-math-BI,MJXc-TeX-math-BIx,MJXc-TeX-math-BIw} .MJXc-TeX-sans-R {font-family: MJXc-TeX-sans-R,MJXc-TeX-sans-Rw} .MJXc-TeX-sans-B {font-family: MJXc-TeX-sans-B,MJXc-TeX-sans-Bx,MJXc-TeX-sans-Bw} .MJXc-TeX-sans-I {font-family: MJXc-TeX-sans-I,MJXc-TeX-sans-Ix,MJXc-TeX-sans-Iw} .MJXc-TeX-script-R {font-family: MJXc-TeX-script-R,MJXc-TeX-script-Rw} .MJXc-TeX-type-R {font-family: MJXc-TeX-type-R,MJXc-TeX-type-Rw} .MJXc-TeX-cal-R {font-family: MJXc-TeX-cal-R,MJXc-TeX-cal-Rw} .MJXc-TeX-main-B {font-family: MJXc-TeX-main-B,MJXc-TeX-main-Bx,MJXc-TeX-main-Bw} .MJXc-TeX-main-I {font-family: MJXc-TeX-main-I,MJXc-TeX-main-Ix,MJXc-TeX-main-Iw} .MJXc-TeX-main-R {font-family: MJXc-TeX-main-R,MJXc-TeX-main-Rw} .MJXc-TeX-math-I {font-family: MJXc-TeX-math-I,MJXc-TeX-math-Ix,MJXc-TeX-math-Iw} .MJXc-TeX-size1-R {font-family: MJXc-TeX-size1-R,MJXc-TeX-size1-Rw} .MJXc-TeX-size2-R {font-family: MJXc-TeX-size2-R,MJXc-TeX-size2-Rw} .MJXc-TeX-size3-R {font-family: MJXc-TeX-size3-R,MJXc-TeX-size3-Rw} .MJXc-TeX-size4-R {font-family: MJXc-TeX-size4-R,MJXc-TeX-size4-Rw} .MJXc-TeX-vec-R {font-family: MJXc-TeX-vec-R,MJXc-TeX-vec-Rw} .MJXc-TeX-vec-B {font-family: MJXc-TeX-vec-B,MJXc-TeX-vec-Bx,MJXc-TeX-vec-Bw} @font-face {font-family: MJXc-TeX-ams-R; src: local('MathJax_AMS'), local('MathJax_AMS-Regular')} @font-face {font-family: MJXc-TeX-ams-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_AMS-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_AMS-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_AMS-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-B; src: local('MathJax_Caligraphic Bold'), local('MathJax_Caligraphic-Bold')} @font-face {font-family: MJXc-TeX-cal-Bx; src: local('MathJax_Caligraphic'); font-weight: bold} @font-face {font-family: MJXc-TeX-cal-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-R; src: local('MathJax_Fraktur'), local('MathJax_Fraktur-Regular')} @font-face {font-family: MJXc-TeX-frak-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-B; src: local('MathJax_Fraktur Bold'), local('MathJax_Fraktur-Bold')} @font-face {font-family: MJXc-TeX-frak-Bx; src: local('MathJax_Fraktur'); font-weight: bold} @font-face {font-family: MJXc-TeX-frak-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-BI; src: local('MathJax_Math BoldItalic'), local('MathJax_Math-BoldItalic')} @font-face {font-family: MJXc-TeX-math-BIx; src: local('MathJax_Math'); font-weight: bold; font-style: italic} @font-face {font-family: MJXc-TeX-math-BIw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-BoldItalic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-BoldItalic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-BoldItalic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-R; src: local('MathJax_SansSerif'), local('MathJax_SansSerif-Regular')} @font-face {font-family: MJXc-TeX-sans-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-B; src: local('MathJax_SansSerif Bold'), local('MathJax_SansSerif-Bold')} @font-face {font-family: MJXc-TeX-sans-Bx; src: local('MathJax_SansSerif'); font-weight: bold} @font-face {font-family: MJXc-TeX-sans-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-I; src: local('MathJax_SansSerif Italic'), local('MathJax_SansSerif-Italic')} @font-face {font-family: MJXc-TeX-sans-Ix; src: local('MathJax_SansSerif'); font-style: italic} @font-face {font-family: MJXc-TeX-sans-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-script-R; src: local('MathJax_Script'), local('MathJax_Script-Regular')} @font-face {font-family: MJXc-TeX-script-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Script-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Script-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Script-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-type-R; src: local('MathJax_Typewriter'), local('MathJax_Typewriter-Regular')} @font-face {font-family: MJXc-TeX-type-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Typewriter-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Typewriter-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Typewriter-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-R; src: local('MathJax_Caligraphic'), local('MathJax_Caligraphic-Regular')} @font-face {font-family: MJXc-TeX-cal-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-B; src: local('MathJax_Main Bold'), local('MathJax_Main-Bold')} @font-face {font-family: MJXc-TeX-main-Bx; src: local('MathJax_Main'); font-weight: bold} @font-face {font-family: MJXc-TeX-main-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-I; src: local('MathJax_Main Italic'), local('MathJax_Main-Italic')} @font-face {font-family: MJXc-TeX-main-Ix; src: local('MathJax_Main'); font-style: italic} @font-face {font-family: MJXc-TeX-main-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-R; src: local('MathJax_Main'), local('MathJax_Main-Regular')} @font-face {font-family: MJXc-TeX-main-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-I; src: local('MathJax_Math Italic'), local('MathJax_Math-Italic')} @font-face {font-family: MJXc-TeX-math-Ix; src: local('MathJax_Math'); font-style: italic} @font-face {font-family: MJXc-TeX-math-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size1-R; src: local('MathJax_Size1'), local('MathJax_Size1-Regular')} @font-face {font-family: MJXc-TeX-size1-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size1-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size1-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size1-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size2-R; src: local('MathJax_Size2'), local('MathJax_Size2-Regular')} @font-face {font-family: MJXc-TeX-size2-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size2-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size2-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size2-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size3-R; src: local('MathJax_Size3'), local('MathJax_Size3-Regular')} @font-face {font-family: MJXc-TeX-size3-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size3-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size3-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size3-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size4-R; src: local('MathJax_Size4'), local('MathJax_Size4-Regular')} @font-face {font-family: MJXc-TeX-size4-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size4-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size4-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size4-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-R; src: local('MathJax_Vector'), local('MathJax_Vector-Regular')} @font-face {font-family: MJXc-TeX-vec-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-B; src: local('MathJax_Vector Bold'), local('MathJax_Vector-Bold')} @font-face {font-family: MJXc-TeX-vec-Bx; src: local('MathJax_Vector'); font-weight: bold} @font-face {font-family: MJXc-TeX-vec-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Bold.otf') format('opentype')}

Inspired in part by Being a Robust Agent. Flipside of Half-assing it with everything you've got.

Do you ever feel... fake? Like, at any minute, Scooby Doo and the gang might roll up and unmask you as a freeloading fraud impostor in front of everyone?

There are a lot of things to say about the impostor syndrome on a psychological basis (the fears are often unrealistic / unmerited, etc). But I'd like to take another angle. For a few years, I've tried to just make a habit of being un-unmaskable. Although this is a useful frame for me, your mileage may vary.

My point isn't going to just be "do the things you know you should". I think we're often bad at judging when corners are okay to cut, so you probably do better just by having the policy of not cutting corners, unless it's extremely obviously alright to do so. That is, generally err against using scissors when confronted with corners, even if it makes sense in the moment.

Concrete examples
  • Making insights truly a part of you. This doesn't mean one should freak out about the Math Gestapo checking whether you've memorized what Jordan normal form is. Rather... when I was just beginning to learn formal proof-based math, I worried "I'm about to go work with some of the smartest people in the world, and they'll instantly see I'm a fake who just picked up shallow knowledge". The internal response was "just get good enough that in no conceivable world could you be a fake who secretly can't do formal math".
  • Working out regularly, taking care of the small things, building the key good habits. Having your shit together.
  • Learning a lot of related areas, just in case they have key insights.
  • Regularly and automatically backing up your files, in multiple locations.
  • Using a password manager to generate and store strong passwords, automatically syncing your database over Dropbox, etc.
  • Rather than having embarrassing things on Facebook which you hope people won't find, just use a tool to search-and-delete incriminating cringey material from your past.
  • Keep your exhaustive resume up-to-date, using a slick LATEX template like you know you should.
  • Following best practices (e.g. when writing code, so there isn't a secret layer of gross code underneath the most prominent functions; when dealing with git repos, so future collaboration / merging works out okay).
  • Responding to emails after reading them. Not leaving people on read by mistake (I'm bad at this, actually).
  • Using spellcheck on your documents.[1]
  • Scheduling meetings and showing up on time by leaving a lot earlier. Avoiding the planning fallacy. Setting multiple alarms before flights.
  • Having enough slack.
The general philosophy

This robustness is a kind of epistemic humility - it's the kind of reasoning that robustly avoids the planning fallacy, only generalized. It's the kind of reasoning that double-checks answers before turning in the test. It's best practices, but for your own life.

I try to live my mental life such that, if people could read my thoughts, they would think I'm doing things right. That doesn't mean I'm always being polite to people in my mind, but it means that I'm not being deceitful, or unfair, or secretly cutting corners on work I'm doing for them.[2]

Again, the point isn't "have good habits and be happy". The point is that I think we often cut too many corners, and so I recommend a policy which leans towards not cutting corners (even when it locally makes sense). The benefits for me have been twofold: getting better results, and feeling more secure about myself while getting those results.

  1. Ironically, the first draft of this spelled "impostor" as "imposter". ↩︎

  2. Naturally, I probably fail anyways sometimes, because I'm somewhat biased / unable to achieve full transparency for my thoughts. ↩︎



Discuss

Halifax SSC Meetup -- Saturday 11/1/20

10 января, 2020 - 06:35
Published on January 10, 2020 3:35 AM UTC

Come drink coffee/tea with us and discuss topics wide-ranging: Can metaphysics be reduced to neuroscience? How will Brexit affect the scientific prospects of the UK? What is the import of rise in popularity of cat GIFs?

All new attendees will be given a free cookie.

(NOTE: We are meeting at the Humani-T cafe on South street, not young street)



Discuss

Farewell to the Entertainer

10 января, 2020 - 05:30
Published on January 10, 2020 2:30 AM UTC

For decades EV TAPCO 100M "Entertainer" systems supported contra dances everywhere:

It's a 10-channel mixer, with 8 XLR inputs and two high-impedance 1/4" inputs. It has two powered output channels, one for mains and one for monitors. Three-band parametric EQ, eight-band graphic EQ on the mains and monitors, and phantom power. It came out in the late 1970s or early 1980s, and it's impressive how far ahead of its time it is.

Forty years later it seems to me like the last few dances that were using Entertainers are moving on. I wanted I wanted to know more about their history, so I called Walter Lenk. Walter has been working in audio professionally since the early 1970s, building custom audio equipment, designing systems, and running sound. He had a lot to tell me!

Before about 1980 there weren't good options for this sort of sound reinforcement gig. Most groups were running 6-channel mono mixer-amplifiers like the Shure Vocal Master:

With only a single output channel, you can't run a separate monitor mix, and what is helpful to for the band to hear is often very different than what's helpful to the people in the hall.

The Entertainer came out in 1981, and was a huge improvement. It could run two output channels, mains and monitors, each piece was relatively light, and while it wasn't very powerful it had automatic limiters on the amplifier that would slightly turn down the gain as needed.


Ad in Billboard, 1981-07-11

Walter remembers seeing one at a trade show in New York in Fall 1981 and buying one as soon as he got back to Boston. He was very impressed with it, bought several more, and convinced a lot of Boston-area folks to get them. He also remembers writing a piece for the CDSS News where he talked about systems and recommended the Entertainer.

One remarkable thing about the Entertainer system is that it remained a reasonable choice for most groups for decades, with many groups buying used ones after EV stopped selling them. It was ahead of its time, but the combination of a separate mix for monitors and a relatively low maximum volume left it a bit niche. It was a good fit for churches and folk dances, but rock bands generally wanted something louder and didn't care as much about having separate monitors. Still, they were a good fit for us, and we got an impressive amount of service out of them!

Comment via: facebook



Discuss

Rationalist Scriptures?

10 января, 2020 - 05:10
Published on January 9, 2020 4:59 PM UTC

I understand that the title sounds like an oxymoron, however, we it seems like we do have scriptures. The Codex, HPMOR, Rationality A-Z, etc are basically required reading so that we can have a community with a fixed starting point, however they don't seem to be well compiled.

HPMOR is a complete story. The Codex and Rationality A-Z are two series of blog posts so they retread a lot of territory - has anyone tried to edit them into more coherent books?

Related, has anyone compiled a list of "Rationalist Wisdom"? Like a bunch of sayings that distill Rationalism down that we can point newbs to? I ask because I was looking for a list and couldn't find one, and pointing curious people to 1000+ page books is daunting.




Discuss

Outer alignment and imitative amplification

10 января, 2020 - 03:26
Published on January 10, 2020 12:26 AM UTC

.mjx-chtml {display: inline-block; line-height: 0; text-indent: 0; text-align: left; text-transform: none; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; word-wrap: normal; word-spacing: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0; min-height: 0; border: 0; margin: 0; padding: 1px 0} .MJXc-display {display: block; text-align: center; margin: 1em 0; padding: 0} .mjx-chtml[tabindex]:focus, body :focus .mjx-chtml[tabindex] {display: inline-table} .mjx-full-width {text-align: center; display: table-cell!important; width: 10000em} .mjx-math {display: inline-block; border-collapse: separate; border-spacing: 0} .mjx-math * {display: inline-block; -webkit-box-sizing: content-box!important; -moz-box-sizing: content-box!important; box-sizing: content-box!important; text-align: left} .mjx-numerator {display: block; text-align: center} .mjx-denominator {display: block; text-align: center} .MJXc-stacked {height: 0; position: relative} .MJXc-stacked > * {position: absolute} .MJXc-bevelled > * {display: inline-block} .mjx-stack {display: inline-block} .mjx-op {display: block} .mjx-under {display: table-cell} .mjx-over {display: block} .mjx-over > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-under > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-stack > .mjx-sup {display: block} .mjx-stack > .mjx-sub {display: block} .mjx-prestack > .mjx-presup {display: block} .mjx-prestack > .mjx-presub {display: block} .mjx-delim-h > .mjx-char {display: inline-block} .mjx-surd {vertical-align: top} .mjx-mphantom * {visibility: hidden} .mjx-merror {background-color: #FFFF88; color: #CC0000; border: 1px solid #CC0000; padding: 2px 3px; font-style: normal; font-size: 90%} .mjx-annotation-xml {line-height: normal} .mjx-menclose > svg {fill: none; stroke: currentColor} .mjx-mtr {display: table-row} .mjx-mlabeledtr {display: table-row} .mjx-mtd {display: table-cell; text-align: center} .mjx-label {display: table-row} .mjx-box {display: inline-block} .mjx-block {display: block} .mjx-span {display: inline} .mjx-char {display: block; white-space: pre} .mjx-itable {display: inline-table; width: auto} .mjx-row {display: table-row} .mjx-cell {display: table-cell} .mjx-table {display: table; width: 100%} .mjx-line {display: block; height: 0} .mjx-strut {width: 0; padding-top: 1em} .mjx-vsize {width: 0} .MJXc-space1 {margin-left: .167em} .MJXc-space2 {margin-left: .222em} .MJXc-space3 {margin-left: .278em} .mjx-test.mjx-test-display {display: table!important} .mjx-test.mjx-test-inline {display: inline!important; margin-right: -1px} .mjx-test.mjx-test-default {display: block!important; clear: both} .mjx-ex-box {display: inline-block!important; position: absolute; overflow: hidden; min-height: 0; max-height: none; padding: 0; border: 0; margin: 0; width: 1px; height: 60ex} .mjx-test-inline .mjx-left-box {display: inline-block; width: 0; float: left} .mjx-test-inline .mjx-right-box {display: inline-block; width: 0; float: right} .mjx-test-display .mjx-right-box {display: table-cell!important; width: 10000em!important; min-width: 0; max-width: none; padding: 0; border: 0; margin: 0} .MJXc-TeX-unknown-R {font-family: monospace; font-style: normal; font-weight: normal} .MJXc-TeX-unknown-I {font-family: monospace; font-style: italic; font-weight: normal} .MJXc-TeX-unknown-B {font-family: monospace; font-style: normal; font-weight: bold} .MJXc-TeX-unknown-BI {font-family: monospace; font-style: italic; font-weight: bold} .MJXc-TeX-ams-R {font-family: MJXc-TeX-ams-R,MJXc-TeX-ams-Rw} .MJXc-TeX-cal-B {font-family: MJXc-TeX-cal-B,MJXc-TeX-cal-Bx,MJXc-TeX-cal-Bw} .MJXc-TeX-frak-R {font-family: MJXc-TeX-frak-R,MJXc-TeX-frak-Rw} .MJXc-TeX-frak-B {font-family: MJXc-TeX-frak-B,MJXc-TeX-frak-Bx,MJXc-TeX-frak-Bw} .MJXc-TeX-math-BI {font-family: MJXc-TeX-math-BI,MJXc-TeX-math-BIx,MJXc-TeX-math-BIw} .MJXc-TeX-sans-R {font-family: MJXc-TeX-sans-R,MJXc-TeX-sans-Rw} .MJXc-TeX-sans-B {font-family: MJXc-TeX-sans-B,MJXc-TeX-sans-Bx,MJXc-TeX-sans-Bw} .MJXc-TeX-sans-I {font-family: MJXc-TeX-sans-I,MJXc-TeX-sans-Ix,MJXc-TeX-sans-Iw} .MJXc-TeX-script-R {font-family: MJXc-TeX-script-R,MJXc-TeX-script-Rw} .MJXc-TeX-type-R {font-family: MJXc-TeX-type-R,MJXc-TeX-type-Rw} .MJXc-TeX-cal-R {font-family: MJXc-TeX-cal-R,MJXc-TeX-cal-Rw} .MJXc-TeX-main-B {font-family: MJXc-TeX-main-B,MJXc-TeX-main-Bx,MJXc-TeX-main-Bw} .MJXc-TeX-main-I {font-family: MJXc-TeX-main-I,MJXc-TeX-main-Ix,MJXc-TeX-main-Iw} .MJXc-TeX-main-R {font-family: MJXc-TeX-main-R,MJXc-TeX-main-Rw} .MJXc-TeX-math-I {font-family: MJXc-TeX-math-I,MJXc-TeX-math-Ix,MJXc-TeX-math-Iw} .MJXc-TeX-size1-R {font-family: MJXc-TeX-size1-R,MJXc-TeX-size1-Rw} .MJXc-TeX-size2-R {font-family: MJXc-TeX-size2-R,MJXc-TeX-size2-Rw} .MJXc-TeX-size3-R {font-family: MJXc-TeX-size3-R,MJXc-TeX-size3-Rw} .MJXc-TeX-size4-R {font-family: MJXc-TeX-size4-R,MJXc-TeX-size4-Rw} .MJXc-TeX-vec-R {font-family: MJXc-TeX-vec-R,MJXc-TeX-vec-Rw} .MJXc-TeX-vec-B {font-family: MJXc-TeX-vec-B,MJXc-TeX-vec-Bx,MJXc-TeX-vec-Bw} @font-face {font-family: MJXc-TeX-ams-R; src: local('MathJax_AMS'), local('MathJax_AMS-Regular')} @font-face {font-family: MJXc-TeX-ams-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_AMS-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_AMS-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_AMS-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-B; src: local('MathJax_Caligraphic Bold'), local('MathJax_Caligraphic-Bold')} @font-face {font-family: MJXc-TeX-cal-Bx; src: local('MathJax_Caligraphic'); font-weight: bold} @font-face {font-family: MJXc-TeX-cal-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-R; src: local('MathJax_Fraktur'), local('MathJax_Fraktur-Regular')} @font-face {font-family: MJXc-TeX-frak-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-B; src: local('MathJax_Fraktur Bold'), local('MathJax_Fraktur-Bold')} @font-face {font-family: MJXc-TeX-frak-Bx; src: local('MathJax_Fraktur'); font-weight: bold} @font-face {font-family: MJXc-TeX-frak-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-BI; src: local('MathJax_Math BoldItalic'), local('MathJax_Math-BoldItalic')} @font-face {font-family: MJXc-TeX-math-BIx; src: local('MathJax_Math'); font-weight: bold; font-style: italic} @font-face {font-family: MJXc-TeX-math-BIw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-BoldItalic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-BoldItalic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-BoldItalic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-R; src: local('MathJax_SansSerif'), local('MathJax_SansSerif-Regular')} @font-face {font-family: MJXc-TeX-sans-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-B; src: local('MathJax_SansSerif Bold'), local('MathJax_SansSerif-Bold')} @font-face {font-family: MJXc-TeX-sans-Bx; src: local('MathJax_SansSerif'); font-weight: bold} @font-face {font-family: MJXc-TeX-sans-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-I; src: local('MathJax_SansSerif Italic'), local('MathJax_SansSerif-Italic')} @font-face {font-family: MJXc-TeX-sans-Ix; src: local('MathJax_SansSerif'); font-style: italic} @font-face {font-family: MJXc-TeX-sans-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-script-R; src: local('MathJax_Script'), local('MathJax_Script-Regular')} @font-face {font-family: MJXc-TeX-script-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Script-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Script-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Script-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-type-R; src: local('MathJax_Typewriter'), local('MathJax_Typewriter-Regular')} @font-face {font-family: MJXc-TeX-type-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Typewriter-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Typewriter-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Typewriter-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-R; src: local('MathJax_Caligraphic'), local('MathJax_Caligraphic-Regular')} @font-face {font-family: MJXc-TeX-cal-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-B; src: local('MathJax_Main Bold'), local('MathJax_Main-Bold')} @font-face {font-family: MJXc-TeX-main-Bx; src: local('MathJax_Main'); font-weight: bold} @font-face {font-family: MJXc-TeX-main-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-I; src: local('MathJax_Main Italic'), local('MathJax_Main-Italic')} @font-face {font-family: MJXc-TeX-main-Ix; src: local('MathJax_Main'); font-style: italic} @font-face {font-family: MJXc-TeX-main-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-R; src: local('MathJax_Main'), local('MathJax_Main-Regular')} @font-face {font-family: MJXc-TeX-main-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-I; src: local('MathJax_Math Italic'), local('MathJax_Math-Italic')} @font-face {font-family: MJXc-TeX-math-Ix; src: local('MathJax_Math'); font-style: italic} @font-face {font-family: MJXc-TeX-math-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size1-R; src: local('MathJax_Size1'), local('MathJax_Size1-Regular')} @font-face {font-family: MJXc-TeX-size1-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size1-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size1-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size1-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size2-R; src: local('MathJax_Size2'), local('MathJax_Size2-Regular')} @font-face {font-family: MJXc-TeX-size2-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size2-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size2-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size2-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size3-R; src: local('MathJax_Size3'), local('MathJax_Size3-Regular')} @font-face {font-family: MJXc-TeX-size3-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size3-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size3-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size3-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size4-R; src: local('MathJax_Size4'), local('MathJax_Size4-Regular')} @font-face {font-family: MJXc-TeX-size4-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size4-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size4-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size4-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-R; src: local('MathJax_Vector'), local('MathJax_Vector-Regular')} @font-face {font-family: MJXc-TeX-vec-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-B; src: local('MathJax_Vector Bold'), local('MathJax_Vector-Bold')} @font-face {font-family: MJXc-TeX-vec-Bx; src: local('MathJax_Vector'); font-weight: bold} @font-face {font-family: MJXc-TeX-vec-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Bold.otf') format('opentype')} Understanding the outer alignment problem

What really is outer alignment? In “Risks from Learned Optimization,” we defined outer alignment in the context of machine learning as “aligning the specified loss function with the intended goal.” But that's not a perfectly well-defined statement—what does it mean for a loss function to be “aligned” with the intended goal? If the goal we care about is maximizing U, do we need exactly L=−aU+b for constants a,b? That's a pretty high bar.

Well, what exactly do we want outer alignment for? At the end of the day, we care about whether the model that pops out the other end of our training procedure will be safe, which is a complicated question involving the loss function, the architecture, the implicit inductive biases, and so on. In what sense, then, is it even reasonable to look at just the lost function in isolation and ask whether it's aligned or not?

I think the strongest case for outer alignment being a meaningful problem in isolation comes from the argument that loss functions seem to scale pretty well with generic machine learning progress. If, as a silly example, your outer alignment scheme is to “train image classification models,” that's something that ML has progressively gotten better at over time. Compare that to the silly inner alignment scheme of “train a straightforward CNN”—that's something that ML has passed by pretty rapidly in favor of architectural improvements like residual connections even just for the task of image classification. Of course, outer alignment alone does not an aligned AGI make, so you still have to have some notion of how you're going to do inner alignment in mind—but loss functions scaling better is still a perfectly valid reason for focusing on outer alignment.[1]

Thus, it does seem quite reasonable to me for effort to go into finding “aligned” loss functions. But that still brings us back to the question of what exactly makes a loss function “aligned.” In the context of a specific training/inner alignment scheme, we can say that a loss function is aligned if, when plugged into that training scheme, it produces models which are aligned with our goals. But in the absence of any specific training scheme, what does it mean to say that a loss function is aligned in isolation? We can of course ask for L=−aU+b as I stated previously, though in my opinion I think achieving something like that is likely to be nearly impossible.

Outer alignment at optimum

I think there is another version of “outer aligned in isolation,” however, which is both meaningful and (at least somewhat) achievable which I will call outer aligned at optimum. Intuitively, I will say that a loss function is outer aligned at optimum if all the possible models that perform optimally according that loss function are aligned with our goals—that is, they are at least trying to do what we want. More precisely, let M=X→A and L=(X→A)→R=M→R. For a given loss function L∈L, let l∗=minM∈M L(M). Then, L is outer aligned at optimum if, for all M∗∈M such that L(M∗)=l∗, M∗ is trying to do what we want.

That's the definition—now why should we care? In basically any practical setting we're never going to reach perfect loss, so why should it matter if those functions which do have perfect loss are aligned or not? In my opinion, I think there is a strong argument for loss functions which are aligned at optimum being significantly less susceptible to Goodhart's Law as we scale up ML capabilities. Suppose you know that a loss function L is aligned for current ML capabilities. When you then scale up those capabilities and push harder on minimizing L, you immediately run into all the issues of Goodhart's Law where L can quickly cease to be a good proxy for alignment as you push harder on it. If you have a guarantee that L is aligned at optimum, however, then, while still quite possible, it's a lot harder for Goodhart's Law to bite you. In particular, if you think about the Goodhart taxonomy, alignment at optimum almost entirely rules out both Causal and Extremal Goodhart—since you know the relationship is valid and doesn't break down at the extremes—and ensures that Regressional and Adversarial Goodhart won't show up in the limit, though you could still see them before that point. Though this obviously doesn't just give you an alignment guarantee—before you get to the true optimum, you can still get Regressional Goodhart biting you through proxy alignment or Adversarial Goodhart biting you through deceptive alignment, for example—I think it is nevertheless still a very nice thing to have.

The case for imitative amplification

With all of that being said, I can get to the reason that I want to talk about all of this: I think that specifically what I will call imitative amplification—in contrast to other amplification-based approaches or debate-based approaches—has a strong claim to being outer aligned at optimum.[2] Specifically, when I say imitative amplification, I mean the class of training procedures which are attempting to produce models which approximate HCH as closely as possible. As a concrete example, consider the scheme where you train a model to minimize the difference between its output and the output of a human consulting that model. I want to contrast this with approval-based amplification, by which I mean the class of training procedures where the loss is generated using an approval signal from an amplified overseer. As a concrete example, consider the scheme where you train a model to maximize the extent to which a human consulting that model would approve of that model's output.[3]

So, why does imitative amplification have a stronger case for being outer aligned at optimum than approval-based amplification or debate? Well, precisely because we know what the optimum of imitative amplification is—it's HCH—whereas we really don't know what perfect approval-based amplification or perfect debate look like.[4] Though some challenges have been raised regarding whether HCH is actually aligned or not, I tend to be fairly skeptical of these challenges—HCH is just a bunch of humans after all and if you can instruct them not to do stupid things like instantiate arbitrary Turing machines, then I think a bunch of humans put together has a strong case for being aligned.[5] That being said, the same argument does not at all apply to approval-based amplification or debate.

First, let's consider approval-based amplification.[6] We know what the optimum of imitative amplification looks like—but what is the optimum of approval-based amplification? At first glance, one might imagine that the optimum of approval-based amplification looks like a model whose output is selected to be maximally approved of by HCH. That's very much not the case for the approval-based scheme I described earlier, however. If each step of training is done via maximizing an approval signal, then instead of a tree of humans you get a tree of models trying to maximize the approval that their parents in the tree would assign to their answers. And if you think that human approval can be gamed—which seems extremely likely in my opinion given that we see exactly that sort of gaming happening in our world already all the time—then this is very much not a safe tree. Now, one could make the argument that approval-based amplification can just become imitative amplification if the humans determine their approval by computing a distance function between what they would have said and what the model produced as its output. For example, you could ask your humans to come up with their answers first, then show them the model's answer and ask them to rate how close it was. I'm pretty skeptical of this approach, however—it doesn't seem at all clear to me that this gets around the approval-gaming problem, since the humans still get to see the model's answer and doing so could significantly change how they're thinking about the rating problem.[7]

Now, second, let's consider debate with a human judge. In many ways, debate was designed as an approach meant to fix the problems of approval-based reward signals. With a standard approval-based reward signal, the argument goes, it's easy to be tricked by a bad argument that you don't fully understand. In a debate setup, however, you get the benefit of having two competing systems trying to point out flaws in each other's arguments, which hopefully should prevent you from being tricked by bad arguments and thus fix the approval-gaming problem. I'm not convinced, though—false things can be significantly easier to argue for than true things just because there are fewer ways to attack them, they're more rhetorically powerful, or any other number of possible ways in which an argument can be subtly wrong yet still persuasive.[8] Regardless, however, I think the more fundamental objection is just that we really have no way of knowing what optimal play in debate looks like, which makes it very difficult to ever know whether it is outer aligned at optimum or not. With HCH, we know that it just looks like a tree of humans, which at least means we can reason about the parts and how they interact. With optimal debate, however, we have to somehow analyze, understand, and be confident in the alignment of superhuman play on a game involving humans assessing arbitrary strings of characters, which is something that in my opinion seems extremely difficult to do.

Addressing competitiveness concerns

All of that is an argument for why we should prefer imitative amplification from an alignment standpoint. However, there's also the problem of imitative amplification just not being competitive in terms of capabilities with other approaches. First of all, I think it's important to remember the importance of putting safety first—if something isn't safe, then we shouldn't build it. Of course, arms race dynamics could end up pushing one's hand into going with a best available current option in order to beat some other team which one believes will produce an AI which is even less likely to be safe, though I think it's important to remember that that's a last resort, not the default option. Furthermore, even in such a situation, it's still probably fine to eat an overhead cost that is just something like a constant factor worse.

With that being said, I still think there are strong arguments to be made for why imitative amplification can be done competitively. First, like the silly outer alignment scheme of “just train an image classification model” from earlier, imitative amplification gets to piggy-back off of generic ML progress. Imitative amplification is just a language modeling problem, which means generic progress on language modeling should generally be transferable to imitative amplification. Second, I think there is a strong case for language being sufficiently rich as a dataset for training an AGI, at least for the sorts of tasks which I think you will want to use your first AGI for.[9] For example, if the primary/most important purpose of your first AGI is to help you build your second AGI by helping you improve your AGI design, that's the sort of highly cognitive task which I think language is sufficient for. Certainly, if you needed your first AGI to be able to do fine motor control to be competitive, then imitative amplification probably won't get you there—but it seems pretty unlikely to me that ability to do fine motor control will be an important desiderata. Third, a common criticism of imitative amplification is that because imitation treats all data points the same, it won't be able to focus on the most important ones. However, that's not something that's fundamental to the task of imitation. For example, you could use active learning to select the most important data points rather than just using a fixed curriculum. Or, you could even weight different data points in your imitation loss using some outside importance criterion while still maintaining the guarantee of perfect imitation at optimum.

Regardless, I think the case for imitative amplification's safety is a strong argument in favor of at least focusing on figuring out whether it works and is safe first, before we give up and move to other approaches. Furthermore, even if imitative amplification on its own isn't competitive, I don't think that means we have to abandon it completely—there are modifications to imitative amplification that can be made to help improve competitiveness without sacrificing all of its benefits. For example, you could do reward-modeling-based distillation (e.g. RL + IRL as the distillation step) instead of imitation-based distillation, which, while not imitative (as the optimum isn't HCH anymore), also isn't based on human approval, which could be a nice property. Alternatively, you could first train an HCH model, and then use that model as the judge to train a debate model, which could have significant benefits over just using a human judge. While I don't think we should be focusing on those sorts of things now, the fact that such options exist makes it more likely that imitative amplification work can transfer to future approaches even if imitative amplification itself ends up not being competitive. In any event, I think the case for focusing on imitative amplification right now both from an outer alignment perspective as well as from a competitiveness perspective is quite strong.[10]

  1. There is still lots of potential for outer alignment work to be outdated by machine learning progress, however—see, for example, Daniel Kokotajlo's “A dilemma for prosaic AI alignment.” ↩︎

  2. I mentioned this previously a bit in this comment. ↩︎

  3. Note that the two categories of “imitative” and “approval-based” amplification do not cover the entire space of possible amplification-based approaches—there are other possible schemes in this domain as well. For example, you could use imitative amplification to train an HCH approximator, then do RL to produce a model which maximizes that model's approval—or even use your HCH model as a judge in a debate. Alternatively, you could do imitative amplification but instead of using standard imitation learning you could do IRL + RL instead. All of these different approaches have different alignment properties—I have singled out imitative amplification, approval-based amplification, and debate with a human judge because they are the approaches I'm most interested in talking about there, though they are far from the only ones. ↩︎

  4. Note that for the optimum of imitative amplification to be precisely HCH, you need it to be the case that you progressively enlarge your training data as you go along. The fact that you don't get good guarantees for finite datasets is certainly a problem, though it's one that you basically have to solve via inner alignment techniques and thus not one I want to focus on right now. ↩︎

  5. The question of whether theoretical HCH is aligned or not is a pretty complicated question that I don't really want to go into in full detail right now, so if you strongly disagree just take it as a given for this post. ↩︎

  6. Though there was a previous claim by William Saunders that RL amplification and imitative amplification are equivalent, I think that both of William's proposals there fall into my approval-based category, not my imitative category. See Rohin Shah's and Wei Dai's comments on William's post to that effect. ↩︎

  7. In particular, this breaks the analogy to counterfactual oracles. ↩︎

  8. I have a lot more to say on this point regarding reasons why false arguments can be more persuasive than true ones, though that's not something I want to go into in too much detail right now. ↩︎

  9. Much of my thinking here owes a debt to Geoffrey Irving. I also talked about the case for language being all you need a bit previously in this comment. ↩︎

  10. I also think imitative amplification has some nice inner alignment properties as well, since it gives you an amplified overseer to use for transparency/relaxed adversarial training. ↩︎



Discuss

Criticism as Entertainment

10 января, 2020 - 01:20
Published on January 9, 2020 10:20 PM UTC

Media Reviews

There is a popular genre of video that consist of shitting on other people’s work without any generative content. Let me provide some examples.

First, Cinema Sins. This is the first video I selected when looking for a movie I’d seen with a Cinema Sins I hadn’t (i.e. it’s not random, but it wasn’t selected for being particularly good or bad).

The first ten sins are:

  1. Use of a consistent brand for props in the movie they’d have to have anyway, unobtrusively enough that I never noticed until Cinema Sins pointed it out.
  2. A character being mildly unreasonable to provoke exposition.
  3. The logo
  4. Exposition that wasn’t perfectly justified in-story
  5. A convenience about what was shown on screen
  6. A font choice (from an entity that in-universe would plausibly make bad font choices)
  7. An omission that will nag at you if you think about it long enough or expect the MCU to be a completely different thing, with some information about why it happened.
  8. In-character choices that would be concerning in the real world and I would argue are treated appropriately by the movie, although reasonable people could disagree
  9. Error by character that was extremely obviously intentional on the part of the film makers. There is no reasonable disagreement on this point.
  10. An error perfectly in keeping with what we know about the character.

Of those, three to four could even plausibly be called sins of the movie- and if those bother you, maybe the MCU is not for you. The rest are deliberate choices by filmmakers to have characters do imperfect things. Everyone gets to draw their own line on characters being dumb- mine is after this movie but before 90s sitcoms running on miscommunication- but that’s irrelevant to this post because Cinema Sins is not helping you determine where a particular movie is relative to your line. Every video makes the movie sound exactly as bad as the last, regardless of the quality of the underlying movie. It’s like they analyze the dialogue sentence by sentence and look to see if there’s anything that could be criticized about it.

Pitch Meeting is roughly as useful, but instead of reacting to sentences, it’s reading the plot summary in a sarcastic tone of voice.

Pitch Meeting is at least bringing up actual problems with Game of Thrones season 8. But I dare you to tell if early Game of Thrones was better or worse than season 8, based on the Pitch Meeting.

I keep harping on “You can’t judge movie quality by the review”, but I don’t actually think that’s the biggest problem. Or rather, it’s a subset of the problem, which is you don’t learn anything from the review: not whether the reviewer considered the movie “good” or not, and not what could be changed to do make it better. Contrast with Zero Punctuation, a video game review series notorious for being criticism-as-entertainment, that nonetheless occasionally likes things, and at least once per episode displays a deep understanding of the problems of a game and what might be done to fix it.

Why Are You Talking About This?

It’s really, really easy to make something look bad, and the short-term rewards to doing so are high. You never risk looking stupid or having to issue a correction. It’s easier to make criticism funny. You get to feel superior. Not to mention the sheer joy in punishing bad things. But it’s corrosive. I’ve already covered (harped on) how useless shitting-on videos are for learning or improvement, but it goes deeper than that. Going in with those intentions changes how you watch the movie. It makes flaws more salient and good parts less so. You become literally less able to enjoy or learn from the original work.

Maybe this isn’t universal, but for me there is definitely a trade off between “groking the author’s concepts” and “interrogating the author’s concepts and evidence”. Groking is a good word here: it mostly means understand, but includes playing with the idea and applying it what I know.  That’s very difficult to do while simultaneously looking for flaws.

Should it be, though? Generating testable hypotheses should lead to greater understanding and trust or less trust, depending on the correctness of the book. So at least one of my investigation or groking procedures are wrong.

 



Discuss

South Bay Meetup, Saturday 1/25

9 января, 2020 - 23:03
Published on January 9, 2020 8:03 PM UTC



Discuss

[Link]Post-mortem on the Center for Long-Term Cybersecurity forecasts

9 января, 2020 - 22:38
Published on January 9, 2020 7:38 PM UTC

A collection of scenarios the CLTC predicted in 2015 about the 2020 cybersecurity landscape.

https://medium.com/cltc-bulletin/post-mortem-2020-looking-back-on-cltcs-scenarios-from-2015-3ad09fb6f1e7



Discuss

Preference synthesis illustrated: Star Wars

9 января, 2020 - 19:47
Published on January 9, 2020 4:47 PM UTC

This is my ordering of the core Star Wars movies:

  • The Empire Strikes Back > Return of the Jedi >= The Last Jedi > Rogue One > A New Hope = The Rise of Skywalker > The Force Awakens > Revenge of the Sith >> Attack of the Clones >>>>> The Phantom Menace

I'm sure that nobody will disagree with me or find these rankings in any way controversial ^_^

But I'm really posting these here to illustrate how preferences can be synthesised from partial preferences.

Because that order didn't exist in my head, I had to synthesise it, fitting various arguments and orderings together. Had I done it at a different time, or in a different mood, I would have had a substantially different ordering.

A few things were probably invariant, though. The quality of "The Empire Strikes Back" was clear in my head, as was the terribleness of "The Phantom Menace". "A New Hope" and "The Force Awakens" were easy to compare, being essentially the same movie.

But how did I compare "The Last Jedi", which I found to be an inconsistent sprawling mess with some of the most beautiful scenes in all of Star Wars (and the "old hero retired" trope, which I really enjoy), with "A New Hope", a tightly scripted elegant adventure story with little depth? And what about "Return of the Jedi", which has a great ending, a decent beginning, but a muddelish middle?

Well, Scott has defined ambijectivity, which allows one to say that comparisons like "Mozart’s music is better than Beethoven’s" is subjective, while "Mozart’s music is better than the music of the three-year old girl who lives upstairs from me and bangs on her toy piano sometimes" is more objective.

The idea is that there are hundreds of different ways in which we can compare things, and some of these are simple enough that they have an easy objective or subjective answer. And Mozart and Beethoven are tend to beat each other roughly equally on the more objective criteria, and can go either way on the subjective ones.

But they both tend to beat the three-year old on almost all criteria.

How I ordered the movies

So, in comparing the Star Wars movies, I stated with a few clear orderings, and then compared movies based on criteria I felt were relevant. For example, "Return of the Jedi" and "The Rise of Skywalker" are reasonably similar, and though I found Kylo Ren an interesting character, Vader is... Vader. Iconic characters and emotional arcs were used here.

"Rogue one" is tricky, as it's great on many criteria, but isn't maybe as enjoyable to watch as some of the other moves (in most ways more realistic, more tragic). I could have put it in many different locations, depending on what I emphasised (emotional arcs would have put it lower; exciting story would have put it higher; showing the cost of war would have put it higher, and so on).

I was aware that I would be presenting the ordering to the public, and this made me more likely to use arguments that I could defend. For example, I personally really liked "The Rise of Skywalker", but felt I couldn't justify (to myself or others) putting above "A New Hope". And do the original movies get a bonus for their originality? Maybe. In different moods, I could prefer different movies, quite easily.

After filling out a few orders, I then tried to make everything transitive, comparing a few of the movies one with another, and trying to collapse my circular preferences. There are two places where equal signs snuck in; note that these are pairs that are particularly hard to compare, and they gained the equal (or >=) sign mainly because I sorted out their relative ranking with the other movies and then really couldn't compare them well with each other.

I was also expecting to put "Revenge of the Sith" higher than it ultimately was. Unlike the other two prequels, I feel it was actually a decent movie, and wanted to reflect that by putting it higher; but, in any one-to-one comparisons with the other non-prequels, it ended up the loser. So it was a sort of Condorcet loser.

Analysis of my synthesis

So, how would I compare my own synthesis efforts with what I described in my research agenda?

There are some similarities, to be sure. Partial preferences, and some things that might be defined as meta preferences (not wanting circular preferences, preferring defensible criteria, my ultimately frustrated desire to show that "Revenge of the Sith" was not a bad movie).

Maybe the most important point is that my preference ordering of Star Wars movies did not exist until I synthesised it, and that the synthesis process could have ended up being different. So this is evidence for the core theses of the research agenda: that making preference orderings is a constructive, synthesis process, and that it doesn't have a single clear outcome.

All in all, my efforts were pretty sloppy. Many movies were compared on subjective feelings, the criteria I used were generally the first or second that sprung to mind, I made no effort to be systematic, or to consider at different times and with different moods or aims.

In these areas, I expect that an AI could do better, could correctly compute my "one-step hypothetical preferences", and get a more invariant representation of my preferences. So, in that way, a preference learning AI would be like me, just better.

But the "meta-preferences" are a bit more worrying. Are they genuine meta-preferences? Especially since the second one is one that was more subconscious, and the third one looks more like a standard preference than a meta-preference. If the category of meta-preference is not clear, then that part of the research agenda needs to be improved.

Anyway, this is my tour through Star Wars and synthesising preferences, two burning issues of different importance.



Discuss

[Personal Experiment] Training YouTube's Algorithm

9 января, 2020 - 12:04
Published on January 9, 2020 9:04 AM UTC

I created a new Google account almost a month ago. I've watched nothing but music and dancing on this account. Whenever it recommends a non-music non-dancing video I immediately click "Don't recommend channel"[1]. I listen to YouTube music every day.

If you you opened my YouTube account right now you'd see 509 recommendations on the home page. 487 would be for music and dancing. 22 would be things I'm not interested in. Below are YouTube's 22 non-musical recommendations:

  • Unboxing Violin and Testing
  • 不要随便让老师唱歌跳舞,搞不好他们是被教育事业耽误的明星!
  • 如何保護總統生命安全?前美國特勤局特務:「我們的任務就是站在威脅與目標之間」|科普小知識|GQ Taiwan
  • L'équipe du Togo - 60 minutes avec Kheiron
  • sony's lead graphic designer coming up with the PS5 logo
  • The Engoodening of No Man's Sky
  • What is an Exoplanet Documentary - Planets Beyond our Solar System
  • Rockman 3 – Dr. Wily no Saigo!? (FC)
  • NLE Choppa - Cottonwood (Official Trailer)
  • YOU is a weird show... (Alex Meyers channel)
  • Safiya & Tyler's Wedding Highlight Film
  • Martin Lawrence Finally Responds To Tisha Campbell Accusations: None Of That Was True - CH News
  • True Facts: Mating Dance of the Ostrich
  • 5 Sikretong Video Ng NASA Na Nabunyag
  • Nanatsu no Taizai 3ª Temporada EPISÓDIO 13 LEGENDADO PT BR
  • Weathering With You [Official English Dub Trailer - GKIDS] - January 15
  • 【Stanley】海克斯閃現福利熊熊!各種閃現外翻!5/0/10鬼神一般的戰績!原來是上路福利熊啊我還以為是輔助呢!改名以後戳仔大幅下降?成為了勇於認錯的韓服Nice 玩家 +1 [This is a Let's Play]
  • Star Wars: The Last Jedi - Training for the Last Jedi
  • r/TIHI | TeeHee
  • 韓國女生的台灣網咖旅館一日求生挑戰
  • The Hobbit with Lightsabers
  • When an Imperial Major Didn't Recognize Darth Vader(Canon) - Star Wars Comics Explained

Most of these are understandable errors at personalization. The violin unboxing makes sense because I listen to lots of violin music. The Star Wars and anime stuff makes sense because I watch "Star Wars Anime Opening" videos. "不要随便让老师唱歌跳舞..." is dancing-related. I think the movie trailers come from the fact that some of the music videos I watch take the form of movie clips. I was surprised by the science recommendations until I realized it's not difficult to infer that someone who listens to AIVA-generated music and "「only my railgun」歌ってみた【*なみりん】" over-and-over again might be interested in science. The wedding highlight film could be related to the wedding dance videos I enjoy.

The videos in Chinese, Spanish and Japanese make sense because I watch music in these languages. When I first created my account it recommended a lot of news, videogames and comedy (among other things). The foreign language news, videogames and comedy are probably just generic recommendations for Chinese speakers and Spanish speakers. If I click "Don't recommend channel" enough times they'll probably go away. I don't know where the French and Filipino videos came from[2] but I expect the same goes for them too.

This leaves 5 truly impersonal recommendations[3].

  • sony's lead graphic designer coming up with the PS5 logo
  • The Engoodening of No Man's Sky
  • YOU is a weird show...
  • Martin Lawrence Finally Responds To Tisha Campbell Accusations: None Of That Was True - CH News
  • r/TIHI | TeeHee

To summarize:

  • 97% of what YouTube shows me is under my control.
  • 2% is reasonable but incorrect generalizations from my behavior.
  • 1% is generic and completely out of my control.
  1. There are two ways to train YouTube to recommend videos: you can "like" them and you can watch them. I use these buttons as normal, clicking "like" only on exceptional videos I want to promote. There are several ways to train YouTube to not recommend videos: you can not watch them, you can "dislike" them, you can click "Not interested", you can click "Don't recommend channel" and you can click "Report". At first I used YouTube's "Not interested" button but quickly switched to "Don't recommend channel" because it resulted in better recommendations. I don't like clicking "dislike" and "Report" for channels I don't want to see because most videos I don't want to watch do produce quality content. ↩︎

  2. I regularly get recommendations like this in languages I don't speak. This could be because many of the Chinese instrumental music videos I watch have half-Chinese half-something-else titles. It could be that once you regularly watch videos in four different languages the algorithm decreases the importance of "language" as a feature. YouTube could be performing a Bayesian hyperparameter search and this is part of its exploration phase. Maybe YouTube recommends a small number of foreign language videos to everyone. ↩︎

  3. They could be true shots in the dark by YouTube's algorithm or stem from some deep correlation like as demographic profiling. For the sake of this analysis, all that matters is they're not effectively personalized. ↩︎



Discuss

Old Airports

9 января, 2020 - 06:30
Published on January 9, 2020 3:30 AM UTC

One thing I enjoy about aerial imagery is that you can see traces of how cities used to be. Tracing the spaces left behind by old train lines is a lot of fun, but recently I've been looking at some airports:

Old Saint-Pierre Airport (link), now with housing:

Old Jijiga Airport (link), also with housing, mostly recognizable as a piece of the street grid that doesn't match:

Old Guangzhou Airport (link), barely visible:

Old Quito Airport (link), now a large park and very visible. I think this the airport we used when we visited Ecuador in 2012:

Old Rio Branco Airport (link) is now a road:

Old Austin Airport (link) is very hard to recognize. All the ones above were single-runway, but Mueller airport had two large runways at an angle. Single-runway ones stand out a lot more, but this is still visible as a section of the city that doesn't really fit:

Old Lima Airport (link) closed sixty years ago and has been completely built over. But the runways survive as a pair of crossing roads:

There's a list on Wikipedia but my favorites are ones where I've been looking at an aerial photo and thought "there has to have been an airport there!"

Comment via: facebook



Discuss

Are "superforecasters" a real phenomenon?

9 января, 2020 - 05:55
Published on January 9, 2020 1:23 AM UTC

In https://slatestarcodex.com/2016/02/04/book-review-superforecasting/, Scott writes:

…okay, now we’re getting to a part I don’t understand. When I read Tetlock’s paper, all he says is that he took the top sixty forecasters, declared them superforecasters, and then studied them intensively. That’s fine; I’d love to know what puts someone in the top 2% of forecasters. But it’s important not to phrase this as “Philip Tetlock discovered that 2% of people are superforecasters”. This suggests a discontinuity, a natural division into two groups. But unless I’m missing something, there’s no evidence for this. Two percent of forecasters were in the top two percent. Then Tetlock named them “superforecasters”. We can discuss what skills help people make it this high, but we probably shouldn’t think of it as a specific phenomenon.

But in this article https://www.vox.com/future-perfect/2020/1/7/21051910/predictions-trump-brexit-recession-2019-2020, Kelsey Piper and Dylan Matthews write:

Tetlock and his collaborators have run studies involving tens of thousands of participants and have discovered that prediction follows a power law distribution. That is, most people are pretty bad at it, but a few (Tetlock, in a Gladwellian twist, calls them “superforecasters”) appear to be systematically better than most at predicting world events.

seeming to disagree. I'm curious who's right.

So there's the question of "is superforecaster a natural category" and I'm operationalizing that into "do the performances of GJP participants follow a power-law distribution, such that the best 2% are significantly better than the rest"?

Does anyone know the answer to that question? (And/or does anyone want to argue with that operationalization?)



Discuss

Subscripting Typographic Convention For Citations/Dates/Sources/Evidentials: A Proposal

9 января, 2020 - 01:20
Published on January 8, 2020 10:20 PM UTC

.mjx-chtml {display: inline-block; line-height: 0; text-indent: 0; text-align: left; text-transform: none; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; word-wrap: normal; word-spacing: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0; min-height: 0; border: 0; margin: 0; padding: 1px 0} .MJXc-display {display: block; text-align: center; margin: 1em 0; padding: 0} .mjx-chtml[tabindex]:focus, body :focus .mjx-chtml[tabindex] {display: inline-table} .mjx-full-width {text-align: center; display: table-cell!important; width: 10000em} .mjx-math {display: inline-block; border-collapse: separate; border-spacing: 0} .mjx-math * {display: inline-block; -webkit-box-sizing: content-box!important; -moz-box-sizing: content-box!important; box-sizing: content-box!important; text-align: left} .mjx-numerator {display: block; text-align: center} .mjx-denominator {display: block; text-align: center} .MJXc-stacked {height: 0; position: relative} .MJXc-stacked > * {position: absolute} .MJXc-bevelled > * {display: inline-block} .mjx-stack {display: inline-block} .mjx-op {display: block} .mjx-under {display: table-cell} .mjx-over {display: block} .mjx-over > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-under > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-stack > .mjx-sup {display: block} .mjx-stack > .mjx-sub {display: block} .mjx-prestack > .mjx-presup {display: block} .mjx-prestack > .mjx-presub {display: block} .mjx-delim-h > .mjx-char {display: inline-block} .mjx-surd {vertical-align: top} .mjx-mphantom * {visibility: hidden} .mjx-merror {background-color: #FFFF88; color: #CC0000; border: 1px solid #CC0000; padding: 2px 3px; font-style: normal; font-size: 90%} .mjx-annotation-xml {line-height: normal} .mjx-menclose > svg {fill: none; stroke: currentColor} .mjx-mtr {display: table-row} .mjx-mlabeledtr {display: table-row} .mjx-mtd {display: table-cell; text-align: center} .mjx-label {display: table-row} .mjx-box {display: inline-block} .mjx-block {display: block} .mjx-span {display: inline} .mjx-char {display: block; white-space: pre} .mjx-itable {display: inline-table; width: auto} .mjx-row {display: table-row} .mjx-cell {display: table-cell} .mjx-table {display: table; width: 100%} .mjx-line {display: block; height: 0} .mjx-strut {width: 0; padding-top: 1em} .mjx-vsize {width: 0} .MJXc-space1 {margin-left: .167em} .MJXc-space2 {margin-left: .222em} .MJXc-space3 {margin-left: .278em} .mjx-test.mjx-test-display {display: table!important} .mjx-test.mjx-test-inline {display: inline!important; margin-right: -1px} .mjx-test.mjx-test-default {display: block!important; clear: both} .mjx-ex-box {display: inline-block!important; position: absolute; overflow: hidden; min-height: 0; max-height: none; padding: 0; border: 0; margin: 0; width: 1px; height: 60ex} .mjx-test-inline .mjx-left-box {display: inline-block; width: 0; float: left} .mjx-test-inline .mjx-right-box {display: inline-block; width: 0; float: right} .mjx-test-display .mjx-right-box {display: table-cell!important; width: 10000em!important; min-width: 0; max-width: none; padding: 0; border: 0; margin: 0} .MJXc-TeX-unknown-R {font-family: monospace; font-style: normal; font-weight: normal} .MJXc-TeX-unknown-I {font-family: monospace; font-style: italic; font-weight: normal} .MJXc-TeX-unknown-B {font-family: monospace; font-style: normal; font-weight: bold} .MJXc-TeX-unknown-BI {font-family: monospace; font-style: italic; font-weight: bold} .MJXc-TeX-ams-R {font-family: MJXc-TeX-ams-R,MJXc-TeX-ams-Rw} .MJXc-TeX-cal-B {font-family: MJXc-TeX-cal-B,MJXc-TeX-cal-Bx,MJXc-TeX-cal-Bw} .MJXc-TeX-frak-R {font-family: MJXc-TeX-frak-R,MJXc-TeX-frak-Rw} .MJXc-TeX-frak-B {font-family: MJXc-TeX-frak-B,MJXc-TeX-frak-Bx,MJXc-TeX-frak-Bw} .MJXc-TeX-math-BI {font-family: MJXc-TeX-math-BI,MJXc-TeX-math-BIx,MJXc-TeX-math-BIw} .MJXc-TeX-sans-R {font-family: MJXc-TeX-sans-R,MJXc-TeX-sans-Rw} .MJXc-TeX-sans-B {font-family: MJXc-TeX-sans-B,MJXc-TeX-sans-Bx,MJXc-TeX-sans-Bw} .MJXc-TeX-sans-I {font-family: MJXc-TeX-sans-I,MJXc-TeX-sans-Ix,MJXc-TeX-sans-Iw} .MJXc-TeX-script-R {font-family: MJXc-TeX-script-R,MJXc-TeX-script-Rw} .MJXc-TeX-type-R {font-family: MJXc-TeX-type-R,MJXc-TeX-type-Rw} .MJXc-TeX-cal-R {font-family: MJXc-TeX-cal-R,MJXc-TeX-cal-Rw} .MJXc-TeX-main-B {font-family: MJXc-TeX-main-B,MJXc-TeX-main-Bx,MJXc-TeX-main-Bw} .MJXc-TeX-main-I {font-family: MJXc-TeX-main-I,MJXc-TeX-main-Ix,MJXc-TeX-main-Iw} .MJXc-TeX-main-R {font-family: MJXc-TeX-main-R,MJXc-TeX-main-Rw} .MJXc-TeX-math-I {font-family: MJXc-TeX-math-I,MJXc-TeX-math-Ix,MJXc-TeX-math-Iw} .MJXc-TeX-size1-R {font-family: MJXc-TeX-size1-R,MJXc-TeX-size1-Rw} .MJXc-TeX-size2-R {font-family: MJXc-TeX-size2-R,MJXc-TeX-size2-Rw} .MJXc-TeX-size3-R {font-family: MJXc-TeX-size3-R,MJXc-TeX-size3-Rw} .MJXc-TeX-size4-R {font-family: MJXc-TeX-size4-R,MJXc-TeX-size4-Rw} .MJXc-TeX-vec-R {font-family: MJXc-TeX-vec-R,MJXc-TeX-vec-Rw} .MJXc-TeX-vec-B {font-family: MJXc-TeX-vec-B,MJXc-TeX-vec-Bx,MJXc-TeX-vec-Bw} @font-face {font-family: MJXc-TeX-ams-R; src: local('MathJax_AMS'), local('MathJax_AMS-Regular')} @font-face {font-family: MJXc-TeX-ams-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_AMS-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_AMS-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_AMS-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-B; src: local('MathJax_Caligraphic Bold'), local('MathJax_Caligraphic-Bold')} @font-face {font-family: MJXc-TeX-cal-Bx; src: local('MathJax_Caligraphic'); font-weight: bold} @font-face {font-family: MJXc-TeX-cal-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-R; src: local('MathJax_Fraktur'), local('MathJax_Fraktur-Regular')} @font-face {font-family: MJXc-TeX-frak-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-B; src: local('MathJax_Fraktur Bold'), local('MathJax_Fraktur-Bold')} @font-face {font-family: MJXc-TeX-frak-Bx; src: local('MathJax_Fraktur'); font-weight: bold} @font-face {font-family: MJXc-TeX-frak-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-BI; src: local('MathJax_Math BoldItalic'), local('MathJax_Math-BoldItalic')} @font-face {font-family: MJXc-TeX-math-BIx; src: local('MathJax_Math'); font-weight: bold; font-style: italic} @font-face {font-family: MJXc-TeX-math-BIw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-BoldItalic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-BoldItalic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-BoldItalic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-R; src: local('MathJax_SansSerif'), local('MathJax_SansSerif-Regular')} @font-face {font-family: MJXc-TeX-sans-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-B; src: local('MathJax_SansSerif Bold'), local('MathJax_SansSerif-Bold')} @font-face {font-family: MJXc-TeX-sans-Bx; src: local('MathJax_SansSerif'); font-weight: bold} @font-face {font-family: MJXc-TeX-sans-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-I; src: local('MathJax_SansSerif Italic'), local('MathJax_SansSerif-Italic')} @font-face {font-family: MJXc-TeX-sans-Ix; src: local('MathJax_SansSerif'); font-style: italic} @font-face {font-family: MJXc-TeX-sans-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-script-R; src: local('MathJax_Script'), local('MathJax_Script-Regular')} @font-face {font-family: MJXc-TeX-script-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Script-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Script-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Script-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-type-R; src: local('MathJax_Typewriter'), local('MathJax_Typewriter-Regular')} @font-face {font-family: MJXc-TeX-type-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Typewriter-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Typewriter-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Typewriter-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-R; src: local('MathJax_Caligraphic'), local('MathJax_Caligraphic-Regular')} @font-face {font-family: MJXc-TeX-cal-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-B; src: local('MathJax_Main Bold'), local('MathJax_Main-Bold')} @font-face {font-family: MJXc-TeX-main-Bx; src: local('MathJax_Main'); font-weight: bold} @font-face {font-family: MJXc-TeX-main-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-I; src: local('MathJax_Main Italic'), local('MathJax_Main-Italic')} @font-face {font-family: MJXc-TeX-main-Ix; src: local('MathJax_Main'); font-style: italic} @font-face {font-family: MJXc-TeX-main-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-R; src: local('MathJax_Main'), local('MathJax_Main-Regular')} @font-face {font-family: MJXc-TeX-main-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-I; src: local('MathJax_Math Italic'), local('MathJax_Math-Italic')} @font-face {font-family: MJXc-TeX-math-Ix; src: local('MathJax_Math'); font-style: italic} @font-face {font-family: MJXc-TeX-math-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size1-R; src: local('MathJax_Size1'), local('MathJax_Size1-Regular')} @font-face {font-family: MJXc-TeX-size1-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size1-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size1-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size1-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size2-R; src: local('MathJax_Size2'), local('MathJax_Size2-Regular')} @font-face {font-family: MJXc-TeX-size2-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size2-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size2-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size2-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size3-R; src: local('MathJax_Size3'), local('MathJax_Size3-Regular')} @font-face {font-family: MJXc-TeX-size3-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size3-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size3-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size3-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size4-R; src: local('MathJax_Size4'), local('MathJax_Size4-Regular')} @font-face {font-family: MJXc-TeX-size4-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size4-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size4-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size4-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-R; src: local('MathJax_Vector'), local('MathJax_Vector-Regular')} @font-face {font-family: MJXc-TeX-vec-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-B; src: local('MathJax_Vector Bold'), local('MathJax_Vector-Bold')} @font-face {font-family: MJXc-TeX-vec-Bx; src: local('MathJax_Vector'); font-weight: bold} @font-face {font-family: MJXc-TeX-vec-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Bold.otf') format('opentype')}

Reviving an old GS proposal: borrowing from scientific notation and using subscripts like 'Gwern2020' for denoting sources (like citation, timing, or medium) might be a useful trick for clearer writing, compared to omitting such information or using standard cumbersome circumlocutions.

I don't believe the Sapir-Whorf hypothesis so beloved of 20th century thinkers & SF, or that we can make ourselves much more rational by One Weird Linguistic Trick. There is no far transfer, and the benefits of improved vocabulary/notation are inherently domain-specific. You think the same thoughts in English as you do in Chinese.

But, like good typography, good linguistic conventions may be worth, say, even as much as 5% of whatever one values, and that's not nothing. It's definitely worthwhile to do things like spellcheck your writings, after all, even though no amount of spellcheck can rescue a bad idea.

I already use a few unusual conventions, like attempting to use the Kesselman Estimative words to be more systematic about the strength of my claims or always linking fulltext in citations (currently upgrading to 'popups' which do not just link fulltext but present the abstract/excerpts/summary as well) or quote syntax highlighting (to distinguish literal quotes from things like paraphrases or dialogue or rhetorical questions), and I employ a few more domain-specific tricks like avoiding use of the word 'significance' in statistics contexts, automatically inflation-adjusting currencies (to avoid the trivial inconvenience of doing it by hand & so not doing it at all), or using research-specific checklists. Without straying into conlang territory or attempting to do everything in formal logic or serious eccentricity, what else could be done?

One idea for more precise English writing which I think could be usefully revived is broader use of subscripts.

The subscripting idea is derived from General Semantics*, which itself borrows it from standard scientific notation, like physics/statistics/mathematics/chemistry/programming: a superscript/subscript is an index distinguishing multiple versions of something, such as quantity, location, or time, eg xt vs xt+1. They're typically not seen outside of STEM contexts, aside from a few obscure uses like ruby/furigana glosses.

* I am considerably less impressed by other GS linguistic suggestions like E-Prime, but subscripting seems like it may be worth rescuing.

However, there are many places we could use subscripting to be clearer & more compact about which version we are referring to, using them as evidentials, and because it's clearer & more compact, we can afford to use it more places without it wasting space/effort/patience. Citations are a good use case. Why write "Friedenbach (2012)" if we can write "Friedenbach2012"? The latter is shorter, easier to read, less ambiguous (especially if we use it in parentheticals, see Friedenbach (2012)), and doesn't come in a dozen different slightly-varying house styles. And why restrict it to formal publications or written documents? Apply it to any quote, statement, or opinion where variables like time might be relevant. It is a single unified notation: regardless of whether something was thought, spoken, or written by me in 2020, it gets the same notation---"Gwern2020". The evidential can be expanded as necessary: if it's a paper or essay, the '2020' can be a hyperlink, or if it's a 'personal communication', then there can be a bibliography entry stating as much, or if it's the author about their own beliefs/actions/statements in 2020, no further information is necessary (and it avoids awkward custom phraseology like "As I thought back in 2020 or so...."). In contrast, normal citation style cumbersomely uses a different format for each, or provides no guidance: how do you gracefully cite a paper written one year but whose author changed their mind 5 years later based on new results and who told you so 10 years after that?

Because it's already used so much in technical writing, subscripting is reasonably familiar to anyone who took highschool chemistry and can be quickly figured out from context for those who've forgotten, and it's well-supported by fonts and markup languages: it's x~t~ in Pandoc Markdown (but not Reddit/LW?), x<sub>t</sub> in HTML, x<subscript>t</subscript> in DocBook, x_t in TeX/LaTeX, x\ :sub: \t in reStructuredText, etc. So subscripting can be used almost everywhere immediately, without needing to be a universal convention.

Example: here are 3 versions of a text; one stripped of citations and evidentials, one with them in long form, and one with subscripts:

  1. I went to Istanbul for a trip, and saw all the friendly street cats there, just as I'd read about in Abdul Bey; he quotes the local Hakim Abdul saying that the cats even look different from cats elsewhere (but after further thought, I'm not sure I agree with that there). I and my wife had a wonderful trip, although while she clearly enjoyed the trip to the city, she claimed the traffic was terribly oppressive and ruined the trip. (Oh really?)
  2. In 2010, I went to Istanbul for a trip, and saw all the friendly street cats there, just as I'd read about in Abdul Bey's 2000 Street Cats of Istanbul; he quotes the local Hakim Abdul in 1970 saying that the cats even look different from cats elsewhere (but after further thought as I write this now in 2020, I'm not sure I agree with Bey (2000)). I and my wife had a wonderful trip, although while she clearly enjoyed the trip to the city, on Facebook she claimed the traffic was terribly oppressive and ruined the trip. (Oh really?)
  3. I2010 went to Istanbul for a trip, and saw all the friendly street cats there, just as I'd read about in Abdul Bey2000 (Street Cats of Istanbul); he quotes the local Hakim Abdul1970 saying that the cats even look different from cats elsewhere (but after further thought, I'm not sure I2020 agree with Bey2000). I and my wife had a wonderful trip, although while she clearly enjoyed the trip to the city, she claimedFB the traffic was terribly oppressive and ruined the trip. (Oh really?)

In the first version, suppressing the metadata leads to a confusing passage. What did Bey write? We don't learn when Abdul expressed his opinion---which is important because Istanbul, as a large fast-growing metropolis, may have changed greatly over the 40 years from quote to visit. When did the speaker become skeptical of the claim Istanbul cats both act & look different? What might explain the wife's inconsistency, and which version should we put more weight on?

The second version answers all these questions, but at the cost of considerable prolixity, jamming in comma phrases to specify date or source. Few people would want to either write or read such a passage, and the fussiness has a distinctly pseudo-academic air. Unsurprisingly, few people will bother with this---any more than they will bother providing inflation-adjusted dollar amounts of something from a decade ago (even though that's misleading by a good 15% or so, and compounding), or they'd want to check a paywalled paper, or redo calculations in Roman numerals.

The third version may look a little alien because of the subscripts, but it provides all the information of the second version plus a little more (by making explicit the implicit '2020'), in considerably less space (as we can delete the circumlocutions in favor of a single consistent subscript), and reads more pleasantly (the metadata is literally out of the way until we decide we need it).

Compare and contrast this easily-understood & compact subscripting approach with another possible notation for disambiguating, the "X!Y" notation (derived ultimately from UUCP bang notation, AFAICT), which is associated with online fandoms & fanfiction, and gives notation like "2020!gwern". This notation puts the metadata first, which is confusing yodaspeak (what does the '2020' refer to? it dangles until you read on); it makes it inline & full-sized, and then tacks on an additional character just to take up even more space; it's confusing and unusual to anyone who isn't familiar with it from online fanfiction already, and to those who are familiar, it is low-status and has bad connotations.

The major downside, of course, is that it is novel and weird. It at least is not associated with fanfics like "!", and is associated with science & technology, but I'm sure it will deter readers anyway. Does it do enough good to be worth using despite the considerable hit to weirdness points? That I don't know.



Discuss

[AN #81]: Universality as a potential solution to conceptual difficulties in intent alignment

8 января, 2020 - 21:00
Published on January 8, 2020 6:00 PM UTC

[AN #81]: Universality as a potential solution to conceptual difficulties in intent alignment View this email in your browser Find all Alignment Newsletter resources here. In particular, you can sign up, or look through this spreadsheet of all summaries that have ever been in the newsletter. I'm always happy to hear feedback; you can send it to me by replying to this email.

Audio version here (may not be up yet).

Published a year ago, this sequence of five posts introduced the idea of ascription universality. I didn't really get it on a first reading, and only recently read it in enough detail that I think I understand the main ideas. This entire newsletter will focus on ascription universality; treat all of it as a "Highlight".

The key idea of these posts is that of universality: when we can say that some agent "knows everything that any other agent could know". Of course, there will always be some agent with arbitrarily complex beliefs, but we could hope to have agents that know everything that is known by any agent of some complexity class (e.g. agents that run in polynomial time). However, there are still simple programs that can have complex true beliefs: we could make agents that believe that P != NP and that P = NP, and one of those agents will fall into this class. However, these agents are somewhat degenerate -- they don't have a good reason to trust that their belief is true, and so we don't want to worry about them. Since we want to talk about justified belief, let's consider a property based on which of the two agents we would trust.

So, for now let's adopt an informal definition: a program A[C] is universal with respect to some class of programs C if we would trust any beliefs reported by A[C], no matter what beliefs we hear reported by programs in C. (I'll talk about the specific notion of "ascription universality" later.) Notably, we need to trust A[C] even over our own beliefs, that is, if A[C] believes something, we discard our position and adopt A[C]'s belief.

Note that this is still a very strong criterion. Consider the question of whether AI will lead to x-risk: if one agent thinks about fixed-point theorems, while another studies neural nets, and a third tries to predict the response of democratic institutions, the agent A[C] must compete with everything that these agents (justifiably) conclude. For now, let's set aside the question of how to build a universal agent, and instead understand what it enables us to do.

Informed oversight (revisited) (Paul Christiano) (summarized by Rohin): Introduced in this older post, we consider a setting in which an overseer is training an agent, and the overseer is smarter than the agent. Intuitively, this should work out well, because if the agent ever tries to do something bad, the overseer should notice this (since it is smarter), and penalize it. However, the problem is that the agent might in the course of computing an answer come up with considerations that the overseer can't easily find. For example, an agent tasked with creating novel literature might end up plagiarizing from one book; the overseer may not be able to search through all the books to detect the plagiarism. However, if the overseer is universal, then it should also know that the agent's output is plagiarized from the book (because the agent knows that), and it can penalize it appropriately.

We can even construct a semi-formal argument (see the post for the math). Let us assume that there is some true utility function U, and when the agent takes some action a, the overseer provides it with reward r = E[U(a)], where the expectation is relative to the overseer's beliefs. Let us assume away problems of distribution shift: in particular, we'll assume that when choosing between actions a1 and a2, the agent always chooses the action with higher reward. Now suppose that the agent chooses action a1. We can conclude that a1 has higher reward than a2. This means that the observer believes that a1 has higher true utility than a2. But by universality, we always trust the observer over the agent and ourselves, and so we must also believe that a1 has higher true utility than a2, which means that we believe the agent is choosing the right action.

Rohin's opinion: This application of universality is the one I believe the most; even intuitively it seems like the entire informed oversight problem is that the overseer doesn't know everything that the agent knows, and universality straightforwardly fixes that. So I see this as the motivating example for caring about universality at all.

Worst-case guarantees (Revisited) (Paul Christiano) (summarized by Rohin): Let's start considering distribution shift again: we imagine that the agent has already been trained, and there is no overseer any more. When the agent goes off distribution, it may not choose the action that would have gotten the highest reward, and in fact it could be chosen maliciously. As a silly motivating example, perhaps the agent learned the policy "behave well until I see that RSA-2048 (a product of two very large primes) has been factored"; such a policy will behave perfectly during training but will probably commit a treacherous turn eventually.

In this setting, we would like to have a worst-case guarantee, that the model will never behave unacceptably. However, we need to enforce this guarantee at training time, before we know what situations will arise at test time. Universality of the overseer isn't enough to guarantee this, because universality only means that the overseer knows what the agent currently knows; the agent can't really be said to "know" that it will deceive us in the future.

Nonetheless, it seems like the solution is very related to universality, in that we would like to "peer inside" the agent to figure out how it works, to make sure there aren't situations in which it would behave unacceptably. We could use adversarial training for this purpose: train an adversary to find inputs on which the model behaves unacceptably, and then train the model on those inputs to not do that. This will require interpretability techniques, so that the adversary actually stands a chance of finding the relevant inputs. In order to handle cases like RSA-2048 above, the adversary should also be able to provide "pseudo-inputs" like "the world in which the agent sees that RSA-2048 is factored" (otherwise the adversary would have to factor RSA-2048 itself to demonstrate an input on which the agent would fail, which is far too difficult), while not being allowed to provide pseudo-inputs like "the world in which the agent kills all humans" (which no agent can behave acceptably on). Ultimately, we need to make sure the adversary will always be able to find a bad pseudo-input if one exists, without making it too difficult for the agent to behave acceptably on all pseudo-inputs. See also relaxed adversarial training (AN #70).

Rohin's opinion: I was skeptical of the previous iteration of this post (AN #43), which proposed adversarial training, verification, and interpretability, because it seemed like none of them would allow us to get a guarantee on performance. I'm significantly more optimistic after reading this post, mainly because a couple of previous confusions have been resolved:

1. The point of verification is not that we can prove a theorem saying "this agent is beneficial"; the point is that by making relaxations (pseudo-inputs), a technique commonly used in formal verification, we can reduce the burden on the other methods being used (such as adversarial training).

2. Similarly, the point of interpretability is not to help us understand what the agent is doing or will do, it's to help the overseer (or adversary in adversarial training) understand that. Unlike us, the overseer / adversary can scale up along with the agent itself.

I still think that it would be hard to get a guarantee with adversarial training, given that adversarial training has to eliminate all vulnerabilities. On the other hand, it only has to find all of the settings where the agent is maliciously optimizing against us, which you might hope is a more natural category that is easier to identify without looking too much at particular inputs. This seems like an empirical question on which we'll hopefully get data, though even if it works in all cases that we see, that doesn't rule out the possibility that we failed to notice some issue that will only be triggered in the future (as in the RSA-2048 example).

Universality and model-based RL (Paul Christiano) (summarized by Rohin): So far, we've been talking about the model-free setting, where the overseer provides the incentives. What about model-based RL? Here, we might want to learn separate distributions over models and utility functions using iterated amplification or HCH, and then plan using any off-the-shelf algorithm, such as MCTS. The first new problem that arises is that our distribution over models may be malicious (see benign induction). Using universality, we can solve this problem in two different ways. First, we could use HCH to filter out malicious models. By universality, since a malicious model knows what its goals are, so does HCH, and thus HCH will detect it. Second, we could ask HCH to give us a good prior over models. By universality, this prior over models must be better than the prior used by the malicious models, and so the malicious models' predictions will be worse than that of HCH as a whole, and so the malicious models will not be able to affect the computation much.

The second new problem in the model-based setting is that it isn't clear how our utility function can extract all the relevant information from the model. For example, if the model is a detailed simulation of physics, how do you find the "humans" and evaluate how "happy" they are? Universality is at least necessary for this: if there are facts the model "knows", those facts may be relevant to the utility calculation, and so the utility function also needs to know them. We might hope that universality is sufficient: in the previous case, we might think that the physics simulation "knows" whether humans are happy. But this depends on details of what "know" means that haven't yet been fleshed out, and it depends on what "kind" of universality we can achieve in practice.

Universality and consequentialism within HCH (Paul Christiano) (summarized by Rohin): In all the settings discussed so far, we hope to use HCH (AN #34). What if HCH itself contains malicious optimization? Specifically, recall that HCH involves an arbitrarily large tree of humans breaking questions down into subquestions. Memetic selection on 10^100 humans should be much stronger than the selection done by evolution to produce us, and so could be quite dangerous. We would like to use HCH to check whether its own computations are malicious.

So, we'll define a pair of mutually recursive functions, Filter and Filtered-HCH. Filter takes as input a transcript from HCH, and using filtered-HCH it finds the best argument suggesting that the transcript is problematic. It then uses filtered-HCH to ask whether, in light of this argument, the transcript should be treated as problematic. This check is repeated thrice for robustness. (Note that the original HCH treats the transcript as trusted computations, while the checker treats it as an untrusted input.) Filtered-HCH has the same type signature as HCH (i.e. it produces answers to questions). Just like HCH, it computes an answer to the question, with the help of recursive calls to filtered-HCH to get answers to subquestions. Instead of returning the answer directly, it calls Filter to check that the computation done was problematic, and returns the special symbol ⟂ instead of the answer if so.

The basic hope is that if there's some malicious subcomputation implemented "on top of" the base agents in HCH, and it tries to break HCH, then it "knows" why that attempt could work, and so if filtered-HCH is universal with respect to computations that can be instantiated by (smaller versions of) HCH, then filtered-HCH will also "know" that the computation could break HCH, and will censor it as a result. Consider the threat model from Reliability amplification (AN #44), where the chance of any particular HCH computation with budget n becoming malicious is ε, independent of the input or any past HCH calls. Then, intuitively, filtered-HCH (which needs a higher budget, say kn) has a kε chance of becoming malicious, but since the check is repeated three times and a majority vote is taken, the chance of a malicious computation getting through is k^2ε^2, which will be much smaller than the original ε if k^2 ε << 1. (This is significantly better than the scheme in Reliability amplification (AN #44), because the question "is there a problem?" is a binary yes/no question, and so majority vote works a lot better.) The harder case is when specific inputs cause very high probabilities of failure (which I believe is the threat model from Security amplification (AN #45)). I particularly don't understand this section, but I believe the hope is that since the malicious computation is implemented "on top of" the (aligned) humans, only a small "part" of the computation can be malicious, and this can be detected by a small instance of Filtered-HCH on which the malicious computation cannot be implemented.

Rohin's opinion: I don't understand this post that well, so I'll refrain from opining on it. I think I would benefit from a more concrete example of how malicious optimization could be implemented "on top of" HCH, even when the humans are restricted to a small set of inputs on which they are "secure".

Towards formalizing universality (Paul Christiano) (summarized by Rohin): We've now seen a bunch of applications of universality. So what is ascription universality? It's a specific formalization of the concept of universality. Recall our informal definition so far: a program A[C] is universal with respect to some class of programs C if we would trust any beliefs reported by A[C], no matter what beliefs we hear reported by programs in C. But how can a program "report beliefs"? For HCH, to find out its belief for some question, we simply ask HCH the question and take its answer as its "belief". However, we can't do this for arbitrary computations: in particular, if our agent is deliberately hiding a belief from us (and so wouldn't report it if asked), we shouldn't simply say that it doesn't have that belief. This is where ascriptions come in: an ascription is any way of ascribing beliefs to the agent. Simply asking the agent and trusting its answer is one possible ascription. Another class of ascriptions would look at the agent's code and memory, and make inferences about the agent's beliefs from that. We want to say that HCH will dominate any "reasonable" ascription of beliefs to the agent, where an ascription procedure is "reasonable" if it isn't "doing the hard work"; for example, if an agent always says "Yes", it is unreasonable to ascribe to it the belief that Fermat's Last Theorem is true (even though when given the question "Is Fermat's Last Theorem true" it answers yes).

Putting it all together, A[C] is ascription universal with respect to some class of programs C if, for every "reasonable" ascription procedure and program c in C, A[C]'s beliefs (which we find by asking it) epistemically dominate the beliefs ascribed to c. A set of beliefs B1 epistemically dominates another set of beliefs B2 from our perspective if, given arbitrary information about B1 and B2, for every question our best guess at the answer is what we expect B1 would answer.

So far we've been talking about A[C] dominating an entire class of programs C. We might call this idealized ascription universality, and hope that it applies to systems with perfect optimization, such as debate (AN #5) under optimal play. In practice, we'll never get perfect optimization, and our overseer isn't going to be able to think of everything that any agent could be thinking about. Instead, we could aim for practical ascription universality, in which A[c] dominates a particular computation c, and it gets to look at the computation c as a hint. This would allow an overseer to train an agent, while epistemically dominating that agent. Effectively we're asking two questions: first, would our procedure be universal if the optimization was performed perfectly? Second, is weight sharing / interpretability strong enough to solve the optimization problems needed for universality?

The blog post also has some informal thoughts on why we should expect that a careful Bayesian should be able to epistemically dominate computations found via machine learning (which is why we might hope that HCH could be ascription universal).

Rohin's opinion: I was initially a bit confused by the decomposition into the idealized and the practical variant, but I think it makes more sense now: the assumption of perfect optimization gives you a lot of mileage. For example, A[C] might choose a particular computation c in C, and then use its beliefs; if that choice of computation were already "perfectly optimized" for convincing us, then we would always be at least as convinced of A[C]'s answer as the answer of any other c in C. Debate under optimal play is pretty similar: under optimal play, the agent only tells us about the most important considerations, and only brings up other considerations if its opponent asks about them; you would hope that this is like selecting the best c in C and reporting its reasons for its beliefs.

What's happening here is that the perfect optimization allows us to select the best c in C, or the best argument, etc. "for free" (i.e. without using any computation), giving us huge savings; everything else just has to present the results properly. Consider again the question of whether AI will lead to x-risk: while there are different agents thinking about fixed-point theorems, neural nets, and democratic institutions, perfect optimization lets us "pick" the agent whose arguments are most important. Then, we should only need about as much computation as the agent itself.

However, "presenting the results properly" is non-trivial: A[C] needs to report the beliefs in plain English, even if the computation c in C has its beliefs "encoded" (as in the case of a detailed physics simulation that can be said to "know" that humans are happy, but doesn't say that in plain English). It's not clear to me whether this will work out.

You might very reasonably be worried that perfect optimization of "the most convincing argument" will get you something that is convincing for the wrong reasons, and is not actually correlated with the truth. The definition of universality doesn't preclude this: it simply says that we trust A[C]'s beliefs over our own beliefs and any other agent's beliefs; this does hold for convincing but invalid arguments. I believe (but am not sure) that the hope is to deal with this by restricting inputs to the human to a small secure space, and by the techniques discussed above in Universality and consequentialism within HCH.

Nuances with ascription universality (evhub) (summarized by Rohin): Consider a formalization of honesty / transparency related to ascription universality: a system is honest, if relative to our expectations, its explicitly stated beliefs give us just as much information as any other way of ascribing beliefs to the system. A system that searches for ways to commit a treacherous turn would not be honest, since it would not state that it was performing this search. However, what if this search was "distilled" into the weights of a neural net (akin to a lookup table)? In that case, since we wouldn't ascribe beliefs to the lookup table, the system could be deemed honest by this criterion. So, what we actually need is for the entire training process to be honest. (A similar critique applies directly to ascription universality.)

Rohin's opinion: This is a good point; it shows that you need your overseer to be ascription universal with respect to the entire training process, not just the final agent.

Copyright © 2020 Rohin Shah, All rights reserved.


Want to change how you receive these emails?
You can update your preferences or unsubscribe from this list.



Discuss

Stripping Away the Protections

8 января, 2020 - 16:10
Published on January 8, 2020 1:10 PM UTC

Previously in sequence: Moloch Hasn’t WonPerfect CompetitionImperfect CompetitionDoes Big Business Hate Your Family?What is Life in an Immoral Maze?

The previous post painted a bleak picture of life as a middle manager in an Immoral Maze. Not every middle manager faces the high maze levels described in Moral Mazes, but I am confident that many do.

How did things get so bad?

We previously discussed Meditations on Moloch. That has a lot of gear-level analysis of how easy it is for undifferentiated high stakes competition along a single variable to destroy everything of value, and how much worse it is to do all those things at once than would be the sum of their parts.

Perfect Competition summarized and fleshed out that model so we could work with it, and defined Super-Perfect Competition as perfect competition without free exit, which when that restriction is meaningful results in less than zero economic profits.

Imperfect Competition illustrated how different idealized perfect competition is from the conditions we typically experience. Even when we use markets that seem highly competitive. Even in a market used as a canonical example of perfect competition. 

The difference between the situations discussed there, and the situation in Moral Mazes, is that the conditions of super-perfect competition really do apply. The protections against this happening have been stripped away.

An immoral maze is not as extreme as the imagined possible future em world where any simulated human caught doing something suboptimal is erased and replaced. But it is closer than one would first think. In a maze, people can’t yet be copied, but there are lots of aspirants to choose from, and any opportunity to rule one out is pounced upon.

Sources of positive differentiation between managers are systematically destroyed. As we saw above, managers do not believe they exist. So they don’t matter. This leaves only negative differentiation and selection, plus politics. 

Sources of meaningful object-level concerns and detail, and the relevance of any long term consequences of actions, are stripped away as well. Again, managers do not believe they exist. So they don’t matter.

This leaves a homogenous product that managers must become, the production of which is subject to perfect competition. 

Then the resulting politics runs amok. The system self-reinforces the reinforcement of conformity with these ideals.

Let’s go over the details that keep the rest of the world safe from such conditions. The bold terms are copied from the previous post, Imperfect Competition. 

No boss. Not only is there a boss. There are bosses and underlings as far as the eye can see. Everything is reduced to simple metrics and impressions that can be run up or down the chain. Everyone is judged on everything. Based on how they look and cause others to look, and what it says about the person being judged. There are only better and worse judgments, and ways to get them. Avoiding standing out in a negative way (e.g. ‘a thousand atta boys are wiped out by one oops’) is all that matters. The richness of real situations falls away. 

Skin in the game. Immoral mazes actively fight against anyone having skin in the object-level or long term game. They don’t track or remember anything, deliberately destroying or avoiding records to avoid future scapegoating. That leaves no skin in the real game, only skin in the game of short term appearances and politics, which is what skin in the game is supposed to defend against. Even if the system wanted to preserve skin in the game and distribute it usefully, rather than destroy it, there would be a severe shortage of it. Major corporations are too large and consequences too diffuse. There is not enough potential skin in the game to distribute to dozens of levels of management, even in theory under the best of circumstances, even if the company is fully privately held. Skin in the object level game is restricted to the very top and bottom of the hierarchy, if they use it at all. The CEO and board can own stock or options, and the people directly ‘on the line’ can perhaps be judged on individual performance.

Products and producers are not homogenized, and information about them is costly. This is the big one that is easiest to miss or be confused about. It’s hard not to notice that there’s a lot of bosses and a lack of skin in the game. Noticing that managers are effectively homogenized is hard, because they aren’t actually homogenized at all – rather it is the perception that they are, which causes them to act and be treated as if they were. Throughout the process of reaching middle management, aspiring managers become homogenized products, and learn to view themselves and their competition in this fashion, and that within managerial circles this is common knowledge. Any deviations are career suicide and severely punished. The idea that one could be better doesn’t parse – see skills differ, below. There’s also the double-think that if you were somehow meaningfully better, that (through what seems to them like some instinctive but unknown mechanism) likely makes you a threat and means you need to be taken out, or that if you’re better that means they are worse and being worse is death, so again, take you out. 

Reputation and experience matter a lot. Reputation inside a maze boils down to whether someone has negative marks that doom them, and how much momentum their career has, and what kind of political allies they’ve made. Your commitment and time spent are also tracked, but that rapidly ceases to differentiate between any of the survivors. None of that is the kind of reputation that does the work of keeping competition imperfect. Experience beyond generic management experience is not only not considered relevant, it is a liability. It means that you lack career momentum.

Market information is costly for consumers. Consumers are the bosses and fellow managers, who already understand the market in question all too well. They have no choice but to learn, even if the cost of doing so is high. The protection from expensive information comes from consumers choosing not to purchase all of it. 

Market information is costly for producers. Producers are also the bosses and fellow managers, since no one is close to the object level. So again, they already understand the market in question all too well. No one who matters lacks market information.

Skills differ. Complete skill stacks are rare. It is presumed that once a certain level is reached, everyone has a complete managerial skills stack. Everyone’s skill levels are assumed to be equal, aside from skill in politics. The conventional wisdom claims there is common knowledge that there is a generic ‘ability to run a thing’ skill stack. Everyone who has made it this far has that stack. Skills that aren’t generic or politics are considered to effectively not exist, or to be things you should be delegating regardless. Having them yourself is, as noted above, a liability rather than an asset. So there’s no reason to pick particular managers with a specific set of skills for particular jobs that require those skills. There’s no way to positively differentiate yourself using your skills. 

Unknown unknown information exists and matters. Managers are presumed to all be the same. There aren’t meaningful unknown unknowns. The products for which they coordinate production have unknown unknowns that will eventually matter to the company. They also have aspects that are hard to measure, or hard for someone outside the department (or anyone other than the manager themselves) to notice at all. But that is all a long term problem, the same way we were concerned with long term nutritional effects or the long term reliability of car models. No one cares if a manager was producing something with such long term issues. By the time they come to light, all association between that manager and the product has been long forgotten. Memories don’t last long enough. If one did notice, a middle manager would primarily consider this praiseworthy, because the person successfully got away with something, unless they had an opportunity to somehow use it against them. Management nirvana is described in Moral Mazes as getting promoted and then blaming your successor for your own “mistakes.” Which, of course, are not mistakes from your perspective. The manager is responsible for making sure things don’t go to hell in an observable way while they are still in charge, which would be really bad and likely end one’s career.

Fixed costs exist. Fixed costs matter for proper decision making only when they can be avoided (including the option of avoiding them by avoiding the whole enterprise). Otherwise, the ship has sailed, so they are sunk costs, which don’t count and don’t matter, except insofar as sunk cost fallacy is thing in context. If they try to pass those costs along, they just dig the hole deeper. Managers have paid huge fixed costs to become managers. At each step they have invested everything they have in their career trajectory. At many steps, they may regret having started down this path, but don’t realize all they are doing is digging a deeper hole for themselves. Or they realize it, but can’t figure out a way to stop. 

Production costs are asymmetric. There are massive asymmetric sunk costs. Sunk costs don’t count. Everyone has already given everything to be able to produce. Everyone is producing the same thing as a result, with the same marginal costs. Anyone not willing to visibly bear those costs is wiped out for that alone. Effectively, everyone has the same production costs aside from political battles. 

Economies of scale exist. Each manager is only one person. While the companies themselves have economies of scale, the competition between employees has no such concept.

Producer preferences differ. Producer preferences (aka employee preferences) in theory still exist, but letting them noticeably impact decisions is fatal. Doing so in secret still means sacrificing position in the game for whatever else you care about. Successful managers learn not to do that.

Location matters. Location is expected not to be a limiting factor. If a manager is asked to go manage a plant or office in another state or even abroad, refusing to do is a sign that they are insufficiently committed. They might try to steer decisions a non-zero amount to avoid the costs involved, since those costs do interfere with one’s ability to do the job, but this mostly isn’t a thing. This prevents differentiation based on location, which was how this made the original list, and it also wrecks the lives of the managers involved since they cannot choose where to live or predict where they will be.  

Consumers care about the individual producers. It is considered a grave sin in middle management culture to care about your underlings, also known as your producers. You protect them while they are loyal and competent and not the scapegoat at this time, because that is part of the job. You need to be seen as the type of person who does that. But actually sacrificing value for them would be seen as terribly weak. As you do not care about them, the people above you do not care about you either. Loyalty until such loyalty is no longer useful is a thing, but that is very much not the same thing nor does it fill the same role.  

Everyone is judged on a combination of politics, and whether they can properly produce the homogenized product ‘successful middle manager.’ This centrally includes associating only with those also producing such products, rewarding such products, and punishing any deviations from such products. You are responsible for everyone and everything below you, so you must demand fealty from those below you, so they’ll serve whatever agenda is most helpful to you. You must also ensure that they are producing the ‘successful middle manager’ product.   

The Opt-Out Clause

The good news for managers is that once one reaches a certain level, as long as one continues to play by the rules and make others comfortable (comfortable is a key concept we don’t have space to discuss here, for more see section B of Quotes from Moral Mazes), one can typically ride things out while carving out some amount of outside value for yourself. One does this by visibly opting out of the game, and no longer being seen as a threat. Everyone knows that some people must stay in place to keep the wheels mostly turning. Without such people the whole system doesn’t collapse too fast and take current management down. Thus, there is an option for graceful retirement from the climb up the ladder.

More generally, those who accept immobility are unwilling to sacrifice family life or free-time activities to put in the extraordinarily long hours at the office required in the upper circles of their corporations. Or they have made a realistic assessment of the age structure, career paths, and power relationships above them and conclude that there is no longer real opportunity for them. They may see that there is an irreparable mismatch between their own personal styles and the kinds of social skills being cultivated in well-entrenched higher circles. In many cases, they decide that they do not wish to put up with the great stress of higher management work that they have witnessed. (Location 962, Quote 119, Moral Mazes)

Producing ‘successful middle manager’ is largely a pass/fail operation. You can’t produce it by halves. Thus, once you realize you are unable or unwilling to produce it properly, either you move somewhere where you can do so, you quit entirely, or you stop trying to produce ‘successful middle manager’ and instead produce ‘contented middle manager.’ Those are the level one damage mitigation strategies. 

Later sections will deal with such damage mitigation options more broadly.  

Is This Missing Something?

On one level, this robustly answers the question “how did things get so bad?”

On another level, this begs that question entirely. It does not explain why everyone involved ended up with such crazy beliefs, or allowed things to get so bad. It merely explains the bind the players are in within the game as it currently exists.

There is the implication that the whole thing is wrapped in conspiracy and malice that is left implicit, its motives and modes of operation unclear. Without that element, the explanation feels incomplete at best.

This gap is because the model only implicitly mentions Moloch’s Army. I still haven’t quite figured out how to take that on, slash we’re not ready yet. Hopefully you now have a better idea of what this represents.

Mazes Everywhere

These dynamics don’t only exist in major corporations. By default, at least right here and right now, they arise in any sufficiently large organization. Barring strong optimization pressure holding them back, things over time in any given organization with multiple levels of management will become increasingly maze-like.

Governments are presumed to be mazes. Armies. Political parties. Unions. Academia. Sufficiently large non-profits are almost certainly mazes – the profit motive is if anything fighting against the property of being a maze. Getting rid of it would only speed things along.

This is doubly true because mazes do not only support themselves being mazes. They tend to support and encourage maze behaviors in other places and organizations, favoring mazes over non-mazes. Our major corporations have largely been mazes for decades.

A maze need not even be a formal organization. You can get similar effective behaviors whenever there is an effective multi-level hierarchy.

Consider Paul Graham and Sam Altman’s description of the ecosystem surrounding Y-Combinator, as analyzed recently by Ben Hoffman. Or the model of the same system in my older post In a world… of venture capital, where each round of funding can be considered the boss of the investors in the previous round.

The Next Questions

There now exist three categories of next questions. 

The first category (taken from this comment I made on Less Wrong) are the big global questions. To what extent are these dynamics the inevitable result of large organizations? If so, to what extent should we avoid creating large organizations? Has this dynamic ever been different in the past in other places and times, and if so why and can we duplicate those causes?

The second category are the big personal questions. Given the existence of these immoral mazes, what do I do? 

The third category is the one I’ve been struggling with, which is to finally get a good written model of the dynamics of anti-epistemic anti-virtue.



Discuss

Predictably Predictable Futures Talk: Using Expected Loss & Prediction Innovation for Long Term Benefits

8 января, 2020 - 15:51
Published on January 8, 2020 12:51 PM UTC

I recently gave a talk at the 3rd Oxford Workshop on Global Priorities Research

This covered a few topics:

  1. How we can think about and use Expected Loss to understand and optimize judgemental forecasting setups.
  2. Some thoughts around how (1) applies to our decisions about long-term forecasting.
  3. A few slides on Foretold and how similar systems could be used to help with (1) and (2).

I recently recorded a quick version of this talk on YouTube. As always, I'd be curious to get feedback on these ideas.

Original Talk Abstract

The last several decades have brought a large number of contributions to statistical and judgemental forecasting endeavors. The next several decades may bring significantly more.

As things get more advanced, not only will our understanding of how to predict distant events improve, but our understanding of the accuracy of these forecasts will improve as well. This should help us to resolve questions regarding when and how much to trust these forecasts.

This talk will present an overview of what we can expect if things go well and how thinking about this can inform our current actions when optimizing for the long term future.

Related Links:

Foretold

Expected Loss Graphs

Blog posts on "Amplifying generalist research via forecasting":

https://www.lesswrong.com/posts/cLtdcxu9E4noRSons/part-1-amplifying-generalist-research-via-forecasting-models

https://www.lesswrong.com/posts/FeE9nR7RPZrLtsYzD/part-2-amplifying-generalist-research-via-forecasting



Discuss

Страницы