Сборщик RSS-лент

[This is the tenth piece in my series on the Parametric Process Assumption (PPA), a new model of anthropics. You should start here if you are just seeing this for the first time.

The previous piece is "Putting it All Together."

This piece starts the more technical portion of this series.

TLDR:

• We make formal the repetition argument from the intro this series, showing that repeated sampling of the universe specified by some parameter creates an SIA instance nested inside an SSA instance.
• The argument from repetition cannot eliminate parameters that often lead to empty universes, so we add an "existential term" and call the result the PPA.
• The PPA can be expressed concisely as:
  
  mjx-mtable { display: inline-block; text-align: center; vertical-align: .25em; position: relative; box-sizing: border-box; border-spacing: 0; border-collapse: collapse; } mjx-mstyle[size="s"] mjx-mtable { vertical-align: .354em; } mjx-labels { position: absolute; left: 0; top: 0; } mjx-table { display: inline-block; vertical-align: -.5ex; box-sizing: border-box; } mjx-table > mjx-itable { vertical-align: middle; text-align: left; box-sizing: border-box; } mjx-labels > mjx-itable { position: absolute; top: 0; } mjx-mtable[justify="left"] { text-align: left; } mjx-mtable[justify="right"] { text-align: right; } mjx-mtable[justify="left"][side="left"] { padding-right: 0 ! important; } mjx-mtable[justify="left"][side="right"] { padding-left: 0 ! important; } mjx-mtable[justify="right"][side="left"] { padding-right: 0 ! important; } mjx-mtable[justify="right"][side="right"] { padding-left: 0 ! important; } mjx-mtable[align] { vertical-align: baseline; } mjx-mtable[align="top"] > mjx-table { vertical-align: top; } mjx-mtable[align="bottom"] > mjx-table { vertical-align: bottom; } mjx-mtable[side="right"] mjx-labels { min-width: 100%; } mjx-mtr { display: table-row; text-align: left; } mjx-mtr[rowalign="top"] > mjx-mtd { vertical-align: top; } mjx-mtr[rowalign="center"] > mjx-mtd { vertical-align: middle; } mjx-mtr[rowalign="bottom"] > mjx-mtd { vertical-align: bottom; } mjx-mtr[rowalign="baseline"] > mjx-mtd { vertical-align: baseline; } mjx-mtr[rowalign="axis"] > mjx-mtd { vertical-align: .25em; } mjx-mtd { display: table-cell; text-align: center; padding: .215em .4em; } mjx-mtd:first-child { padding-left: 0; } mjx-mtd:last-child { padding-right: 0; } mjx-mtable > * > mjx-itable > *:first-child > mjx-mtd { padding-top: 0; } mjx-mtable > * > mjx-itable > *:last-child > mjx-mtd { padding-bottom: 0; } mjx-tstrut { display: inline-block; height: 1em; vertical-align: -.25em; } mjx-labels[align="left"] > mjx-mtr > mjx-mtd { text-align: left; } mjx-labels[align="right"] > mjx-mtr > mjx-mtd { text-align: right; } mjx-mtd[extra] { padding: 0; } mjx-mtd[rowalign="top"] { vertical-align: top; } mjx-mtd[rowalign="center"] { vertical-align: middle; } mjx-mtd[rowalign="bottom"] { vertical-align: bottom; } mjx-mtd[rowalign="baseline"] { vertical-align: baseline; } mjx-mtd[rowalign="axis"] { vertical-align: .25em; } mjx-mfrac { display: inline-block; text-align: left; } mjx-frac { display: inline-block; vertical-align: 0.17em; padding: 0 .22em; } mjx-frac[type="d"] { vertical-align: .04em; } mjx-frac[delims] { padding: 0 .1em; } mjx-frac[atop] { padding: 0 .12em; } mjx-frac[atop][delims] { padding: 0; } mjx-dtable { display: inline-table; width: 100%; } mjx-dtable > * { font-size: 2000%; } mjx-dbox { display: block; font-size: 5%; } mjx-num { display: block; text-align: center; } mjx-den { display: block; text-align: center; } mjx-mfrac[bevelled] > mjx-num { display: inline-block; } mjx-mfrac[bevelled] > mjx-den { display: inline-block; } mjx-den[align="right"], mjx-num[align="right"] { text-align: right; } mjx-den[align="left"], mjx-num[align="left"] { text-align: left; } mjx-nstrut { display: inline-block; height: .054em; width: 0; vertical-align: -.054em; } mjx-nstrut[type="d"] { height: .217em; vertical-align: -.217em; } mjx-dstrut { display: inline-block; height: .505em; width: 0; } mjx-dstrut[type="d"] { height: .726em; } mjx-line { display: block; box-sizing: border-box; min-height: 1px; height: .06em; border-top: .06em solid; margin: .06em -.1em; overflow: hidden; } mjx-line[type="d"] { margin: .18em -.1em; } mjx-mrow { display: inline-block; text-align: left; } mjx-msubsup { display: inline-block; text-align: left; } mjx-script { display: inline-block; padding-right: .05em; padding-left: .033em; } mjx-script > mjx-spacer { display: block; } mjx-munder { display: inline-block; text-align: left; } mjx-over { text-align: left; } mjx-munder:not([limits="false"]) { display: inline-table; } mjx-munder > mjx-row { text-align: left; } mjx-under { padding-bottom: .1em; } mjx-c.mjx-c1D443.TEX-I::before { padding: 0.683em 0.751em 0 0; content: "P"; } mjx-c.mjx-c1D434.TEX-I::before { padding: 0.716em 0.75em 0 0; content: "A"; } mjx-c.mjx-c28::before { padding: 0.75em 0.389em 0.25em 0; content: "("; } mjx-c.mjx-c2C::before { padding: 0.121em 0.278em 0.194em 0; content: ","; } mjx-c.mjx-c1D45C.TEX-I::before { padding: 0.441em 0.485em 0.011em 0; content: "o"; } mjx-c.mjx-c29::before { padding: 0.75em 0.389em 0.25em 0; content: ")"; } mjx-c.mjx-c221D::before { padding: 0.442em 0.778em 0.011em 0; content: "\221D"; } mjx-c.mjx-c1D448.TEX-I::before { padding: 0.683em 0.767em 0.022em 0; content: "U"; } mjx-c.mjx-c3E::before { padding: 0.54em 0.778em 0.04em 0; content: ">"; } mjx-c.mjx-c2208::before { padding: 0.54em 0.667em 0.04em 0; content: "\2208"; } mjx-c.mjx-c4F.TEX-C::before { padding: 0.705em 0.796em 0.022em 0; content: "O"; } mjx-c.mjx-c1D465.TEX-I::before { padding: 0.442em 0.572em 0.011em 0; content: "x"; } mjx-c.mjx-c1D446.TEX-I::before { padding: 0.705em 0.645em 0.022em 0; content: "S"; } mjx-c.mjx-c1D43C.TEX-I::before { padding: 0.683em 0.504em 0 0; content: "I"; } mjx-c.mjx-c3D::before { padding: 0.583em 0.778em 0.082em 0; content: "="; } mjx-c.mjx-c7C::before { padding: 0.75em 0.278em 0.249em 0; content: "|"; } mjx-c.mjx-c1D439.TEX-I::before { padding: 0.68em 0.749em 0 0; content: "F"; } mjx-c.mjx-c1D445.TEX-I::before { padding: 0.683em 0.759em 0.021em 0; content: "R"; } mjx-c.mjx-c1D441.TEX-I::before { padding: 0.683em 0.888em 0 0; content: "N"; } mjx-c.mjx-c1D456.TEX-I::before { padding: 0.661em 0.345em 0.011em 0; content: "i"; } mjx-c.mjx-c2294::before { padding: 0.598em 0.667em 0 0; content: "\2294"; } mjx-c.mjx-c7B::before { padding: 0.75em 0.5em 0.25em 0; content: "{"; } mjx-c.mjx-c7D::before { padding: 0.75em 0.5em 0.25em 0; content: "}"; } mjx-c.mjx-cD7::before { padding: 0.491em 0.778em 0 0; content: "\D7"; } mjx-c.mjx-c1D43B.TEX-I::before { padding: 0.683em 0.888em 0 0; content: "H"; } mjx-c.mjx-c1D45A.TEX-I::before { padding: 0.442em 0.878em 0.011em 0; content: "m"; } mjx-c.mjx-c1D447.TEX-I::before { padding: 0.677em 0.704em 0 0; content: "T"; } mjx-c.mjx-c33::before { padding: 0.665em 0.5em 0.022em 0; content: "3"; } mjx-c.mjx-c1D461.TEX-I::before { padding: 0.626em 0.361em 0.011em 0; content: "t"; } mjx-c.mjx-c2192::before { padding: 0.511em 1em 0.011em 0; content: "\2192"; } mjx-c.mjx-c1D452.TEX-I::before { padding: 0.442em 0.466em 0.011em 0; content: "e"; } mjx-c.mjx-c1D463.TEX-I::before { padding: 0.443em 0.485em 0.011em 0; content: "v"; } mjx-c.mjx-c2E::before { padding: 0.12em 0.278em 0 0; content: "."; } mjx-c.mjx-c35::before { padding: 0.666em 0.5em 0.022em 0; content: "5"; } mjx-c.mjx-c22C5::before { padding: 0.31em 0.278em 0 0; content: "\22C5"; } mjx-c.mjx-c2211.TEX-S1::before { padding: 0.75em 1.056em 0.25em 0; content: "\2211"; } mjx-c.mjx-c2032::before { padding: 0.56em 0.275em 0 0; content: "\2032"; } mjx-c.mjx-c31::before { padding: 0.666em 0.5em 0 0; content: "1"; } mjx-c.mjx-c30::before { padding: 0.666em 0.5em 0.022em 0; content: "0"; } mjx-c.mjx-c39::before { padding: 0.666em 0.5em 0.022em 0; content: "9"; } mjx-TeXAtom { display: inline-block; text-align: left; } mjx-msub { display: inline-block; text-align: left; } mjx-c.mjx-c221E::before { padding: 0.442em 1em 0.011em 0; content: "\221E"; } mjx-c.mjx-c1D53C.TEX-A::before { padding: 0.683em 0.667em 0 0; content: "E"; } mjx-c.mjx-c5B::before { padding: 0.75em 0.278em 0.25em 0; content: "["; } mjx-c.mjx-c1D45B.TEX-I::before { padding: 0.442em 0.6em 0.011em 0; content: "n"; } mjx-c.mjx-c1D462.TEX-I::before { padding: 0.442em 0.572em 0.011em 0; content: "u"; } mjx-c.mjx-c2223::before { padding: 0.75em 0.278em 0.249em 0; content: "\2223"; } mjx-c.mjx-c1D703.TEX-I::before { padding: 0.705em 0.469em 0.01em 0; content: "\3B8"; } mjx-c.mjx-c5D::before { padding: 0.75em 0.278em 0.25em 0; content: "]"; } mjx-math { display: inline-block; text-align: left; line-height: 0; text-indent: 0; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; border-collapse: collapse; word-wrap: normal; word-spacing: normal; white-space: nowrap; direction: ltr; padding: 1px 0; } mjx-container[jax="CHTML"][display="true"] { display: block; text-align: center; margin: 1em 0; } mjx-container[jax="CHTML"][display="true"][width="full"] { display: flex; } mjx-container[jax="CHTML"][display="true"] mjx-math { padding: 0; } mjx-container[jax="CHTML"][justify="left"] { text-align: left; } mjx-container[jax="CHTML"][justify="right"] { text-align: right; } mjx-mi { display: inline-block; text-align: left; } mjx-c { display: inline-block; } mjx-utext { display: inline-block; padding: .75em 0 .2em 0; } mjx-mn { display: inline-block; text-align: left; } mjx-mo { display: inline-block; text-align: left; } mjx-stretchy-h { display: inline-table; width: 100%; } mjx-stretchy-h > * { display: table-cell; width: 0; } mjx-stretchy-h > * > mjx-c { display: inline-block; transform: scalex(1.0000001); } mjx-stretchy-h > * > mjx-c::before { display: inline-block; width: initial; } mjx-stretchy-h > mjx-ext { /* IE */ overflow: hidden; /* others */ overflow: clip visible; width: 100%; } mjx-stretchy-h > mjx-ext > mjx-c::before { transform: scalex(500); } mjx-stretchy-h > mjx-ext > mjx-c { width: 0; } mjx-stretchy-h > mjx-beg > mjx-c { margin-right: -.1em; } mjx-stretchy-h > mjx-end > mjx-c { margin-left: -.1em; } mjx-stretchy-v { display: inline-block; } mjx-stretchy-v > * { display: block; } mjx-stretchy-v > mjx-beg { height: 0; } mjx-stretchy-v > mjx-end > mjx-c { display: block; } mjx-stretchy-v > * > mjx-c { transform: scaley(1.0000001); transform-origin: left center; overflow: hidden; } mjx-stretchy-v > mjx-ext { display: block; height: 100%; box-sizing: border-box; border: 0px solid transparent; /* IE */ overflow: hidden; /* others */ overflow: visible clip; } mjx-stretchy-v > mjx-ext > mjx-c::before { width: initial; box-sizing: border-box; } mjx-stretchy-v > mjx-ext > mjx-c { transform: scaleY(500) translateY(.075em); overflow: visible; } mjx-mark { display: inline-block; height: 0px; } mjx-msup { display: inline-block; text-align: left; } mjx-c.mjx-c1D458.TEX-I::before { padding: 0.694em 0.521em 0.011em 0; content: "k"; } mjx-c.mjx-c32::before { padding: 0.666em 0.5em 0 0; content: "2"; } mjx-c.mjx-c2212::before { padding: 0.583em 0.778em 0.082em 0; content: "\2212"; } mjx-container[jax="CHTML"] { line-height: 0; } mjx-container [space="1"] { margin-left: .111em; } mjx-container [space="2"] { margin-left: .167em; } mjx-container [space="3"] { margin-left: .222em; } mjx-container [space="4"] { margin-left: .278em; } mjx-container [space="5"] { margin-left: .333em; } mjx-container [rspace="1"] { margin-right: .111em; } mjx-container [rspace="2"] { margin-right: .167em; } mjx-container [rspace="3"] { margin-right: .222em; } mjx-container [rspace="4"] { margin-right: .278em; } mjx-container [rspace="5"] { margin-right: .333em; } mjx-container [size="s"] { font-size: 70.7%; } mjx-container [size="ss"] { font-size: 50%; } mjx-container [size="Tn"] { font-size: 60%; } mjx-container [size="sm"] { font-size: 85%; } mjx-container [size="lg"] { font-size: 120%; } mjx-container [size="Lg"] { font-size: 144%; } mjx-container [size="LG"] { font-size: 173%; } mjx-container [size="hg"] { font-size: 207%; } mjx-container [size="HG"] { font-size: 249%; } mjx-container [width="full"] { width: 100%; } mjx-box { display: inline-block; } mjx-block { display: block; } mjx-itable { display: inline-table; } mjx-row { display: table-row; } mjx-row > * { display: table-cell; } mjx-mtext { display: inline-block; } mjx-mstyle { display: inline-block; } mjx-merror { display: inline-block; color: red; background-color: yellow; } mjx-mphantom { visibility: hidden; } _::-webkit-full-page-media, _:future, :root mjx-container { will-change: opacity; } mjx-c::before { display: block; width: 0; } .MJX-TEX { font-family: MJXZERO, MJXTEX; } .TEX-B { font-family: MJXZERO, MJXTEX-B; } .TEX-I { font-family: MJXZERO, MJXTEX-I; } .TEX-MI { font-family: MJXZERO, MJXTEX-MI; } .TEX-BI { font-family: MJXZERO, MJXTEX-BI; } .TEX-S1 { font-family: MJXZERO, MJXTEX-S1; } .TEX-S2 { font-family: MJXZERO, MJXTEX-S2; } .TEX-S3 { font-family: MJXZERO, MJXTEX-S3; } .TEX-S4 { font-family: MJXZERO, MJXTEX-S4; } .TEX-A { font-family: MJXZERO, MJXTEX-A; } .TEX-C { font-family: MJXZERO, MJXTEX-C; } .TEX-CB { font-family: MJXZERO, MJXTEX-CB; } .TEX-FR { font-family: MJXZERO, MJXTEX-FR; } .TEX-FRB { font-family: MJXZERO, MJXTEX-FRB; } .TEX-SS { font-family: MJXZERO, MJXTEX-SS; } .TEX-SSB { font-family: MJXZERO, MJXTEX-SSB; } .TEX-SSI { font-family: MJXZERO, MJXTEX-SSI; } .TEX-SC { font-family: MJXZERO, MJXTEX-SC; } .TEX-T { font-family: MJXZERO, MJXTEX-T; } .TEX-V { font-family: MJXZERO, MJXTEX-V; } .TEX-VB { font-family: MJXZERO, MJXTEX-VB; } mjx-stretchy-v mjx-c, mjx-stretchy-h mjx-c { font-family: MJXZERO, MJXTEX-S1, MJXTEX-S4, MJXTEX, MJXTEX-A ! important; } @font-face /* 0 */ { font-family: MJXZERO; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Zero.woff") format("woff"); } @font-face /* 1 */ { font-family: MJXTEX; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Main-Regular.woff") format("woff"); } @font-face /* 2 */ { font-family: MJXTEX-B; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Main-Bold.woff") format("woff"); } @font-face /* 3 */ { font-family: MJXTEX-I; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Math-Italic.woff") format("woff"); } @font-face /* 4 */ { font-family: MJXTEX-MI; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Main-Italic.woff") format("woff"); } @font-face /* 5 */ { font-family: MJXTEX-BI; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Math-BoldItalic.woff") format("woff"); } @font-face /* 6 */ { font-family: MJXTEX-S1; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Size1-Regular.woff") format("woff"); } @font-face /* 7 */ { font-family: MJXTEX-S2; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Size2-Regular.woff") format("woff"); } @font-face /* 8 */ { font-family: MJXTEX-S3; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Size3-Regular.woff") format("woff"); } @font-face /* 9 */ { font-family: MJXTEX-S4; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Size4-Regular.woff") format("woff"); } @font-face /* 10 */ { font-family: MJXTEX-A; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_AMS-Regular.woff") format("woff"); } @font-face /* 11 */ { font-family: MJXTEX-C; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Calligraphic-Regular.woff") format("woff"); } @font-face /* 12 */ { font-family: MJXTEX-CB; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Calligraphic-Bold.woff") format("woff"); } @font-face /* 13 */ { font-family: MJXTEX-FR; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Fraktur-Regular.woff") format("woff"); } @font-face /* 14 */ { font-family: MJXTEX-FRB; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Fraktur-Bold.woff") format("woff"); } @font-face /* 15 */ { font-family: MJXTEX-SS; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_SansSerif-Regular.woff") format("woff"); } @font-face /* 16 */ { font-family: MJXTEX-SSB; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_SansSerif-Bold.woff") format("woff"); } @font-face /* 17 */ { font-family: MJXTEX-SSI; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_SansSerif-Italic.woff") format("woff"); } @font-face /* 18 */ { font-family: MJXTEX-SC; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Script-Regular.woff") format("woff"); } @font-face /* 19 */ { font-family: MJXTEX-T; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Typewriter-Regular.woff") format("woff"); } @font-face /* 20 */ { font-family: MJXTEX-V; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Vector-Regular.woff") format("woff"); } @font-face /* 21 */ { font-family: MJXTEX-VB; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Vector-Bold.woff") format("woff"); } 0]}\end{split}" role="math"> when and 0 otherwise.

The next piece is "The Basic Model."]

The Argument from Repetition

In a previous piece, we discussed how repeating species draws under the SSA causes it to converge to something a bit like the SIA.

Here we’re going to formalize that. We’ll start with some universe prior  over universes , and then make that indexical by adding in another set of observer assignment variables , such that assigning to value corresponds to the logical predicate “I am observer ” We then define a joint distribution over and . Each universe is also associated with a set of observers, corresponding to the observers who actually exist in that universe.

The two most common indexical distributions are:

• The SSA:  if , and 0 otherwise
• The SIA:  if , and 0 otherwise

where is the number of observers in universe .

To represent what happens when universes are redrawn, we will add an additional parameter , from which we can draw universes . Our universe prior will be extended into a joint distribution over both and . We will sometimes refer to as “epistemic” and as “aleatoric” or the “process variable.”

Then we can define the “argument from repetition”: 

Argument from Repetition (AFR): Define to be the indexical distribution formed from the following sampling procedure:

• Sample from  
• Sample aleatoric universes from
• Select uniformly from the disjoint union

This is just describing the basic procedure we previously used when talking about parametric models and large-universe SSA: for each possible value of , imagine arbitrarily many samples, put all the observers from these samples in a pile, and then select one at random. If we applied this to the Sleeping Beauty problem, for instance, we would flip a bunch of coins and get a sequence (say, H H T) and then select uniformly from an associated set of universe-observer pairs (e.g. , just as we did in the intro to this series. (In the Sleeping Beauty problem, is trivially a singleton since we don't have any epistemic unknowns in the problem setup).

Since for a fixed each universe is sampled in proportion to its likelihood but takes up space in proportion to its size, we have that, in the limit,

.

In other words, AFR converges to the SIA over the conditional universe prior . We’ll write this as and say that, as N approaches infinity, 

.[1]

Existential Consistency

The one problem with the AFR approach is that it doesn’t do well for which have a high chance of producing no observers. Fortunately, from a model-design perspective, this problem is easy to fix. 

Consider the following situation, which we’ll call God’s Calculation:

To decide if he will create life in the universe, God computes the googolth digit of pi. If it is even, then he creates Adam and Eve. If it is odd, he flips ten trillion coins, and if every single one comes up tails, he creates Adam and Eve. If a single one comes up heads, he creates nothing.

You wake up and discover you are Adam.

Both the SSA and SIA agree in this situation: you should assume that the googolth digit of pi is probably even. But AFR does not: it confidently declares that the digit has . 

The problem is that because we assume the aleatoric parts of the process repeat endlessly, it doesn’t really matter how many flips it takes for God to create life; we have to assume he eventually would.

I don’t have a prefect philosophical answer for this, but I’m leaving that for future work.  For now, though, the problem is very easy to fix mathematically. What we can do is add an adjustment factor that scales each SIA instance by the likelihood that any observers would even exist in that universe, multiplying the probability of each parameter by 0 | \theta)" role="math">. This effectively separates each epistemic region of our probability space into a “life exists” and “life doesn’t exist,” then eliminates the “life doesn’t exist” portion.

The one thing I will say to justify this decision is that it is in fact equivalent what we would get if we replaced our prior with the prior 0)" role="math"> before we built our indexical distribution. In other words, the existential adjustment is equivalent to non-indexically conditioning on at least one observer existing, and only then building the AFR out of that distribution. Formally:

0} \rightarrow P^{PPA}" role="math">.

Whether or not that counts as philosophical justification I leave to the reader.[2]

Either way, when we reconsider God’s Calculation with this new adjustment, the “odd digit” universe gets scaled by , effectively eliminating it from the running exactly as in the SSA and SIA. The implications of this are mostly related to fine-tuning - thanks to this adjustment, the PPA can reproduce the arguments we discussed here.

The Parametric Process Assumption

This, at last, gives us the PPA:

Parametric Process Assumption (PPA): The PPA is the indexical distribution defined as 

0 \mid \theta)  \cdot P^{SIA | \theta}(u, o)" role="math">

when , and 0 otherwise.

We can think of this as having three layers:

• Epistemic: - What are the laws of physics?
• Existential: 0 \mid \theta)" role="math"> - How likely is life to exist under these laws of physics?
• Aleatoric: - Who are you likely to be under these laws of physics?

We can make this more self-contained by observing that the SIA is defined (for ) as 

Thus, the PPA can be written as:

0]}\end{split}" role="math">

when , and 0 otherwise.[3]

Finally, we note that the PPA can actually be reproduce both the SSA or the SIA with the right prior. If you consider all unknowns to be aleatoric, then can only take on a single value, so the entire distribution reduces to the SIA. Similarly, if you consider all unknowns to be epistemic (so that the aleatoric randomness is always fixed), the SIA portion is just a uniform distribution over observers in , so the distribution reduces to the SSA. We saw this same effect in the intro to this series, where modifying the Sleeping Beauty problem to select exclusively between epistemic and aleatoric probabilities caused us to switch between the SSA and SIA.

The next piece is "The Basic Model"

1. ^

   Convergence here is convergence in total variation. Claude pointed this out on a proofread

2. ^

   Also, I’m salty about this observation because I didn’t make it. Opus provided it unsolicited even though I didn’t ask for original insights, just proofreading. Now I have to add this stupid footnote. The exact identity written here is mine though.

3. ^

   This also relies on the simplification that 0 \mid \theta)}{\mathbb{E}[n_U \mid \theta]} = \frac{1}{\mathbb{E}[n_U \mid \theta, n_U > 0]}" role="math">, which I also did not notice until it was pointed out by Claude during proofreading.

Discuss

[This is the ninth piece in my series on the Parametric Process Assumption (PPA), a new model of anthropics.

You should start here if you only want to read one piece on the PPA, but if you only want to read two pieces, this is the second one to read.

The previous piece is "The Fermi Paradox, Fine-Tuning, and the SSA."

TLDR:

• The SSA's bias for Doomsday only applies if your reference class only includes your own species. The more more species in your universe/reference class, the less Doomsday effect you see
• The Basic Model of anthropics models the universe as a process progressing randomly until certain relevant subprocesses emerge spontaneously. An intelligent species like humans would be an example of one such process. It has 3 components:
  • (Epistemic) Propensity for Life: How many times does an observer-creating process emerge, on average?
  • (Aleatoric) Longevity: For each emergence, how many observers does the process produce before it terminates? 
  • Diversity: Everything else. When a process starts, what do the observers experience? What do they look like? etc.
• The previous arguments in this series tell us many things about the Basic Model
  • Diversity has no effect on the SSA, SIA or PPA
  • The SIA acts as an improper prior over longevity but not propensity
  • The SIA gives an undue bias towards large propensity
• The PPA treats propensity as an epistemic parameter and longevity as aleatoric, avoiding both Doomsday and presumptuousness

The next piece, "Deriving the PPA," begins the more technical portion of this series.]

SSA biases only affect small universes

One frustration for me when I started thinking harder about anthropic philosophy was the slow discovery that many of the worst conclusions of various anthropic models, at least as I understood them, were actually greatly exaggerated. Not all of them, of course, but there were quite a few cases where things treated like foregone conclusions (like the SSA implying Doomsday) often disappeared with certain choices of prior. 

As I've references in this series before, I believe a lot of this comes from the unfortunate tendency to discuss anthropics in terms of small thought experiments, almost always forcing us to choose between exactly one of two universes. This by itself is not a problem - I employed the same tools in several parts of this series - but it runs the real risk of oversimplifying past the point where we are actually saying something applicable.  

Consider, for instance, the Doomsday Argument. Typically, this is presented as a choice between some large universe mjx-msub { display: inline-block; text-align: left; } mjx-c.mjx-c1D43F.TEX-I::before { padding: 0.683em 0.681em 0 0; content: "L"; } mjx-c.mjx-c1D446.TEX-I::before { padding: 0.705em 0.645em 0.022em 0; content: "S"; } mjx-c.mjx-c1D45B.TEX-I::before { padding: 0.442em 0.6em 0.011em 0; content: "n"; } mjx-container[jax="CHTML"] { line-height: 0; } mjx-container [space="1"] { margin-left: .111em; } mjx-container [space="2"] { margin-left: .167em; } mjx-container [space="3"] { margin-left: .222em; } mjx-container [space="4"] { margin-left: .278em; } mjx-container [space="5"] { margin-left: .333em; } mjx-container [rspace="1"] { margin-right: .111em; } mjx-container [rspace="2"] { margin-right: .167em; } mjx-container [rspace="3"] { margin-right: .222em; } mjx-container [rspace="4"] { margin-right: .278em; } mjx-container [rspace="5"] { margin-right: .333em; } mjx-container [size="s"] { font-size: 70.7%; } mjx-container [size="ss"] { font-size: 50%; } mjx-container [size="Tn"] { font-size: 60%; } mjx-container [size="sm"] { font-size: 85%; } mjx-container [size="lg"] { font-size: 120%; } mjx-container [size="Lg"] { font-size: 144%; } mjx-container [size="LG"] { font-size: 173%; } mjx-container [size="hg"] { font-size: 207%; } mjx-container [size="HG"] { font-size: 249%; } mjx-container [width="full"] { width: 100%; } mjx-box { display: inline-block; } mjx-block { display: block; } mjx-itable { display: inline-table; } mjx-row { display: table-row; } mjx-row > * { display: table-cell; } mjx-mtext { display: inline-block; } mjx-mstyle { display: inline-block; } mjx-merror { display: inline-block; color: red; background-color: yellow; } mjx-mphantom { visibility: hidden; } _::-webkit-full-page-media, _:future, :root mjx-container { will-change: opacity; } mjx-math { display: inline-block; text-align: left; line-height: 0; text-indent: 0; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; border-collapse: collapse; word-wrap: normal; word-spacing: normal; white-space: nowrap; direction: ltr; padding: 1px 0; } mjx-container[jax="CHTML"][display="true"] { display: block; text-align: center; margin: 1em 0; } mjx-container[jax="CHTML"][display="true"][width="full"] { display: flex; } mjx-container[jax="CHTML"][display="true"] mjx-math { padding: 0; } mjx-container[jax="CHTML"][justify="left"] { text-align: left; } mjx-container[jax="CHTML"][justify="right"] { text-align: right; } mjx-mi { display: inline-block; text-align: left; } mjx-c { display: inline-block; } mjx-utext { display: inline-block; padding: .75em 0 .2em 0; } mjx-c::before { display: block; width: 0; } .MJX-TEX { font-family: MJXZERO, MJXTEX; } .TEX-B { font-family: MJXZERO, MJXTEX-B; } .TEX-I { font-family: MJXZERO, MJXTEX-I; } .TEX-MI { font-family: MJXZERO, MJXTEX-MI; } .TEX-BI { font-family: MJXZERO, MJXTEX-BI; } .TEX-S1 { font-family: MJXZERO, MJXTEX-S1; } .TEX-S2 { font-family: MJXZERO, MJXTEX-S2; } .TEX-S3 { font-family: MJXZERO, MJXTEX-S3; } .TEX-S4 { font-family: MJXZERO, MJXTEX-S4; } .TEX-A { font-family: MJXZERO, MJXTEX-A; } .TEX-C { font-family: MJXZERO, MJXTEX-C; } .TEX-CB { font-family: MJXZERO, MJXTEX-CB; } .TEX-FR { font-family: MJXZERO, MJXTEX-FR; } .TEX-FRB { font-family: MJXZERO, MJXTEX-FRB; } .TEX-SS { font-family: MJXZERO, MJXTEX-SS; } .TEX-SSB { font-family: MJXZERO, MJXTEX-SSB; } .TEX-SSI { font-family: MJXZERO, MJXTEX-SSI; } .TEX-SC { font-family: MJXZERO, MJXTEX-SC; } .TEX-T { font-family: MJXZERO, MJXTEX-T; } .TEX-V { font-family: MJXZERO, MJXTEX-V; } .TEX-VB { font-family: MJXZERO, MJXTEX-VB; } mjx-stretchy-v mjx-c, mjx-stretchy-h mjx-c { font-family: MJXZERO, MJXTEX-S1, MJXTEX-S4, MJXTEX, MJXTEX-A ! important; } @font-face /* 0 */ { font-family: MJXZERO; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Zero.woff") format("woff"); } @font-face /* 1 */ { font-family: MJXTEX; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Main-Regular.woff") format("woff"); } @font-face /* 2 */ { font-family: MJXTEX-B; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Main-Bold.woff") format("woff"); } @font-face /* 3 */ { font-family: MJXTEX-I; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Math-Italic.woff") format("woff"); } @font-face /* 4 */ { font-family: MJXTEX-MI; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Main-Italic.woff") format("woff"); } @font-face /* 5 */ { font-family: MJXTEX-BI; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Math-BoldItalic.woff") format("woff"); } @font-face /* 6 */ { font-family: MJXTEX-S1; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Size1-Regular.woff") format("woff"); } @font-face /* 7 */ { font-family: MJXTEX-S2; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Size2-Regular.woff") format("woff"); } @font-face /* 8 */ { font-family: MJXTEX-S3; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Size3-Regular.woff") format("woff"); } @font-face /* 9 */ { font-family: MJXTEX-S4; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Size4-Regular.woff") format("woff"); } @font-face /* 10 */ { font-family: MJXTEX-A; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_AMS-Regular.woff") format("woff"); } @font-face /* 11 */ { font-family: MJXTEX-C; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Calligraphic-Regular.woff") format("woff"); } @font-face /* 12 */ { font-family: MJXTEX-CB; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Calligraphic-Bold.woff") format("woff"); } @font-face /* 13 */ { font-family: MJXTEX-FR; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Fraktur-Regular.woff") format("woff"); } @font-face /* 14 */ { font-family: MJXTEX-FRB; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Fraktur-Bold.woff") format("woff"); } @font-face /* 15 */ { font-family: MJXTEX-SS; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_SansSerif-Regular.woff") format("woff"); } @font-face /* 16 */ { font-family: MJXTEX-SSB; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_SansSerif-Bold.woff") format("woff"); } @font-face /* 17 */ { font-family: MJXTEX-SSI; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_SansSerif-Italic.woff") format("woff"); } @font-face /* 18 */ { font-family: MJXTEX-SC; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Script-Regular.woff") format("woff"); } @font-face /* 19 */ { font-family: MJXTEX-T; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Typewriter-Regular.woff") format("woff"); } @font-face /* 20 */ { font-family: MJXTEX-V; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Vector-Regular.woff") format("woff"); } @font-face /* 21 */ { font-family: MJXTEX-VB; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Vector-Bold.woff") format("woff"); } mjx-c.mjx-c1D45D.TEX-I::before { padding: 0.442em 0.503em 0.194em 0; content: "p"; } and some small universe , each with a 50% chance a priori. Every observer, through their birth order, is assigned some number (like your brith rank) and, upon learning your own number, the SSA has you update towards a universe that is more likely in proportion to the ratio between and . For instance, if is 10 and is 20, then finding out we are individual 4 means we are twice as likely to exist in universe :

(If you're coming here straight from the intro, you may wish to take a quick peak at this earlier piece on mosaic plots.)

This is famously countered by the SIA, where L starts out twice as likely and therefore produces no bias when you learn your rank:

The problem with this entire setup, though, is that it assumes our current species is the only one that exists. 

Suppose instead that there were two species within our universe (and our reference class) - one blue and one red - and each was independently assigned to be either “long” or “short”. Our distribution under the SSA would look like this:

There are four scenarios corresponding to four different combinations of “long” and “short” species: SS, SL, LS, LL. 

Now look what happens when you learn you are the 4th member of the blue species:

Pay attention to the ratio of light blue vs dark blue after conditioning. It much closer to even than it was in the standard Doomsday example.

The ratio of short to long has evened out considerably from 2:1 to about 1.43:1. Add in an orange species, and the ratio becomes even closer to even at about 1.27:1:

In the limit, as you expand your universe and/or reference class to include more individuals, the shortness bias disappears.

This effect, by the way, does not rely on learning your color in the previous example. Simply knowing that there were more species in consideration is enough to asymptotically eliminate Doomsday bias from the SSA. By the time you get to the kinds of scales that seem likely by our current understanding of physics (see this previous piece), the SSA’s main problematic features - Doomsday and temporal inconsistency - begin to asymptotically vanish.[1] 

The Basic Model

Let’s re-introduce some terminology before we continue. As in the series intro, we’ll call the length of your species - whether it is “long” or “short” - its “longevity,” and we’ll use terms like “shortness bias” to refer to biases in longevity. When we talk about the “size” of the universe, using terms like “largeness bias,” we will instead be talking about the number of species that emerge. 

The previous section showed us that as our lower bound for the size of universe increases, the SSA’s shortness bias decreases with it. The SIA, once you learn your rank in the species, has neither a shortness nor longness bias, but it does have a largeness bias - however big your evidence suggests the universe must be, the SIA insists it must be much much large. In particular, if “infinitely large universe” has even the tiniest bit of positive probability in your prior, then you must discount all other possibilities.

Back when we introduced these terms int he intro, we talked about putting both of them into a single model, which we called the “Basic Model” of anthropics. 

Each species emerges somewhere in the universe, exists and grows for some time, and then goes extinct. Each color represents the different physiology/conditions under which the species exist, controlled by the diversity variable.

The idea was to imagine the universe as a giant random process, with each independently-emerging species as a subprocess. The exact number of subprocesses (the actual size of the universe) would also be random, but the expected number of subprocesses - the general tendency of the universe to independently create life - would not be random, but rather an epistemic unknown which we called the universe’s “propensity for life.” We illustrated with a toy thought experiment involving a grid of unknown size and spontaneously emerging chains of blue pixels.

The the subprocesses themselves are modeled as completely random, since by the time you get to the granularity of determining a species’ longevity, what you are measuring isn’t really the product of unknown physical laws, so much as randomness within the laws that are already there. The specifics of quantum mechanics or the speed of light, for instance, may have affected the likelihood of humans existing, but probably don’t play largely into how long humanity lasts once it’s here.[2]

Then, we throw in an extra variable to cover all the other facts about our species not related to longevity or propensity, and we get a model with three independent components:

• (Epistemic) Propensity for Life: How many times does an observer-creating process emerge, on average?
• (Aleatoric) Longevity: For each emergence, how many observers does the process produce before it terminates? 
• Diversity[3]: Everything else. When a process starts, what do the observers experience? What do they look like? etc.

Some Observations

There are a few things you should know about the Basic Model and how it behaves under the SSA and SIA. These are things we cover in more detail in the technical portions of this series that follow.

The first is that, as we previously discussed, the diversity term is basically only there to be thrown out. So long as diversity has no a priori effect on longevity (which, in our simplified model, it doesn’t), neither the SSA nor the SIA induce any additional diversity bias one way or the other. It can be safely ignored.

The second is that the SIA cannot really handle infinite propensity - having it as a hypothesis either forces you to insist the universe is infinitely large, or, more likely, just gives you an invalid distribution; this is part of the presumptuous philosopher effect. In the case of infinite longevity, however, the SIA can function as an improper prior, starting essentially undefined but snapping into a proper distribution as soon as we gather some (possibly incomplete) evidence about our rank in our species.

The reason an improper prior only helps with longevity but not propensity is that, at least at the moment, we have no way to learn the propensity of the entire multiverse, which means that we can never update out of the improper regime into something that actually lets us make predictions. Nor, as we discussed, could we ever expect to make such an update. With longevity, we have a well-defined concept like birth rank which can be defined a priori for all observers before our universe is actually sampled; to get an equivalent metric for species index, you would need to acquire some sort of global information about all species in the universe like "ranked distance to largest star" or even just "Nth species ever created," and even that would not localize you within the multiverse.

As for the SSA, we already mentioned that large propensity means that it loses its shortness bias, and in the limit of infinite propensity the SSA’s longevity distribution looks exactly like the SIA’s. But the SSA only handles longevity well if you already have reason to believe your universe's minimum propensity is high. While I did argue that high minimum propensity is probably accurate, if you don’t agree, then under the Basic Model the SSA will have both a smallness bias and a shortness bias. The SIA’s biases are not correlated in this fashion, so learning your rank only eliminates longevity biases without touching the largeness. 

Neither approach, therefore, can simultaneously handle all longevity and propensity values to our satisfaction. But the previous results do suggest a way forward: what we need is a distribution that treats longevity just like the SIA/infinite-propensity SSA, but doesn’t unduly bias us towards one propensity or the other. 

Which, you presumably already know, the PPA does handle both approaches.

The PPA, In Summary

Since introducing the PPA way back in the series intro, we haven’t actually spoken about it much. If you recall, the PPA operated by dividing the variables in our universe into two components: an aleatoric “process” portion and an epistemic “parameter” portion. In later pieces, we connected this to the concept of “parametric models” in Bayesian statistics.

The motivation for this dichotomy was the “argument from repetition,” where we showed that if you took, say, the Sleeping Beauty problem and sampled it over and over again, then whether you wound up deriving the SIA or the SSA depended, respectively, on whether your probability was aleatoric or epistemic. To create an epistemic unknown, we imagined the scientists in the Sleeping Beauty problem calculating the googolth digit of pi instead of using a coin. This was a value that would never change over repetitions, but was nonetheless unknown to us.

The PPA, we said, was just what happens when you applied this reasoning to a more complex universe model. The Basic Model is that more complex model, where we treat propensity like the digits of pi and longevity like a coin flip. The universe (or multiverse) gets sampled, in a virtual sense, over and over again, such that all possible species longevities occur at precisely the expected prior rate; longevity predictions converge the same way they did in large-universe SSA. As for propensity, even though the actual number of species in each of these resamples varies, the average across all of them stays fixed. So propensity remains SSAish, even as longevity becomes SIAish.[4]

And that’s about it, at least until we get more technical in the next couple pieces. There’s an extra term I’ll discuss there to correct for very low-propensity universes, and we'll need to do some work with limits to make the infinite cases work how we want them to. Aside from that, though, we've covered most of the key ideas. You can think of the PPA, as I originally did, as nesting the SIA inside the SSA, or you can simply think of it as just taking the SSA to infinite propensity and then backfilling that structure to finite propensity. I think both approaches take you to the same place.

The next piece, "Deriving the PPA," begins the more technical portion of this series.

1. ^

   There is an important requirement here, which is that you must assume all species longevities are drawn from the same distribution. Or, to be precise, that they are drawn from distributions with the same mean.

2. ^

   That said, one might still wonder if it makes more sense to think of the average longevity of all species as a parameter fixed by the laws of physics. I will discuss this in a piece I plan to publish within the coming weeks. This can be added to the model, and it does have a Doomsdayish effect, but I also believe there are good reasons not to make any part of longevity a parameter.

3. ^

   Diversity has no epistemic/aleatoric label because it does not effect our model which one you assign it

4. ^

   The PPA actually does have a real but almost negligible presumptuousness from two effects:

   1. The existential term: The PPA's down-weighting of universes where observers are unlikely to exist (as in the previous piece) causes a small asymptotically disappearing pull towards larger universes
   2. Since randomness within the universe is handled in an SIAish way, there is a tiny increase in presumptuous round this. In the model we'll use in the next sections, this will add exactly 1 additional species.

Discuss

[This is the eighth piece in my series on the Parametric Process Assumption (PPA), a new model of anthropics. You should start here if you are just seeing this for the first time.

The previous piece is "Temporal Consistency and Texas Sharpshooting."

TLDR:

• The Fermi Paradox is easily resolved by realizing the universe is gigantic. This means that there are a huge number of possible worlds where life is very very far apart, despite the fact that it also abundant throughout the universe.
• If all you know is that you exist and that the universe is big and random, then the SSA, contrary to popular belief, is sufficient to conclude that life in the universe is abundant. The SIA, on the other hand, adds an additional bias on top.

The next piece is "Putting it all Together."]

Big and Sparse

I confess that I’ve never really understood the Fermi Paradox. Or at least, I never understood why everyone always seemed so surprised by it.

If you haven’t heard, the Fermi Paradox is the apparent contradiction that, due to the vast age of the universe and the (seemingly) high probability of alien life, we should be observing signs of intelligent alien life all around the universe, and yet we don’t. If near-lightspeed travel were possible, for instance, it would presumably only take an intelligent species a few million years to colonize the entire Milky Way. For a galaxy that’s been around more than 13 billion years, the argument goes, you’d think someone would have gotten to that by now, right?

I do personally believe our existence implies other life is likely to exist somewhere, but the problem with the Fermi Paradox is that it’s not just assuming that life is likely to exist, but that life is likely to be dense. Maybe an intelligent species could theoretically populate the entire Milky Way in a few million years, but why do we expect there to be more than one intelligent species in a given galaxy? Why do we expect there to be more than one intelligent species in the closest thousand galaxies?

I’m not the first to make this point, and I won’t be the last, but if we’re going to attempt a coherent theory of anthropics, the point needs to be made here: the universe, it turns out, is really really big. 

The observable universe - just the observable part, which may or may not make up only an infinitesimal percentage - has a radius of 46.5 billion light-years, literally millions of times more than the range at which we can supposedly find life.[1]

Lower estimates tell us the number of galaxies in the observable universe measures a few hundred billion; let’s say for the sake of argument that the number is 200 billion. This means that the chance of intelligent life existing in any particular galaxy could be 1 in a million, and there would still be 200 thousand different planets out there just teeming with intelligent life, each one hundreds of millions of light-years away from its closest life-bearing neighbor. 

Pretend each pixel here is 200 million light years - this image is about the size of a 2D slice of the observable universe. Each pixel has a 1 in 1000 chance of “generating life,” which we illustrate with a circle centered on that pixel. Life emerges independently 185 times, each about 3.5 billion light years to their closest neighbor on average. The closest pair is 440 million light-years apart. 

For reference, our galaxy is less than 200 thousand light-years wide, or 1/1000 of a pixel. The chance that this procedure would have produced life at least once is mjx-c.mjx-c31::before { padding: 0.666em 0.5em 0 0; content: "1"; } mjx-c.mjx-c30::before { padding: 0.666em 0.5em 0.022em 0; content: "0"; } mjx-c.mjx-c39::before { padding: 0.666em 0.5em 0.022em 0; content: "9"; } mjx-TeXAtom { display: inline-block; text-align: left; } mjx-msub { display: inline-block; text-align: left; } mjx-c.mjx-c221E::before { padding: 0.442em 1em 0.011em 0; content: "\221E"; } mjx-c.mjx-c1D53C.TEX-A::before { padding: 0.683em 0.667em 0 0; content: "E"; } mjx-c.mjx-c5B::before { padding: 0.75em 0.278em 0.25em 0; content: "["; } mjx-c.mjx-c1D45B.TEX-I::before { padding: 0.442em 0.6em 0.011em 0; content: "n"; } mjx-c.mjx-c1D462.TEX-I::before { padding: 0.442em 0.572em 0.011em 0; content: "u"; } mjx-c.mjx-c2223::before { padding: 0.75em 0.278em 0.249em 0; content: "\2223"; } mjx-c.mjx-c1D703.TEX-I::before { padding: 0.705em 0.469em 0.01em 0; content: "\3B8"; } mjx-c.mjx-c5D::before { padding: 0.75em 0.278em 0.25em 0; content: "]"; } mjx-math { display: inline-block; text-align: left; line-height: 0; text-indent: 0; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; border-collapse: collapse; word-wrap: normal; word-spacing: normal; white-space: nowrap; direction: ltr; padding: 1px 0; } mjx-container[jax="CHTML"][display="true"] { display: block; text-align: center; margin: 1em 0; } mjx-container[jax="CHTML"][display="true"][width="full"] { display: flex; } mjx-container[jax="CHTML"][display="true"] mjx-math { padding: 0; } mjx-container[jax="CHTML"][justify="left"] { text-align: left; } mjx-container[jax="CHTML"][justify="right"] { text-align: right; } mjx-mi { display: inline-block; text-align: left; } mjx-c { display: inline-block; } mjx-utext { display: inline-block; padding: .75em 0 .2em 0; } mjx-mn { display: inline-block; text-align: left; } mjx-mo { display: inline-block; text-align: left; } mjx-stretchy-h { display: inline-table; width: 100%; } mjx-stretchy-h > * { display: table-cell; width: 0; } mjx-stretchy-h > * > mjx-c { display: inline-block; transform: scalex(1.0000001); } mjx-stretchy-h > * > mjx-c::before { display: inline-block; width: initial; } mjx-stretchy-h > mjx-ext { /* IE */ overflow: hidden; /* others */ overflow: clip visible; width: 100%; } mjx-stretchy-h > mjx-ext > mjx-c::before { transform: scalex(500); } mjx-stretchy-h > mjx-ext > mjx-c { width: 0; } mjx-stretchy-h > mjx-beg > mjx-c { margin-right: -.1em; } mjx-stretchy-h > mjx-end > mjx-c { margin-left: -.1em; } mjx-stretchy-v { display: inline-block; } mjx-stretchy-v > * { display: block; } mjx-stretchy-v > mjx-beg { height: 0; } mjx-stretchy-v > mjx-end > mjx-c { display: block; } mjx-stretchy-v > * > mjx-c { transform: scaley(1.0000001); transform-origin: left center; overflow: hidden; } mjx-stretchy-v > mjx-ext { display: block; height: 100%; box-sizing: border-box; border: 0px solid transparent; /* IE */ overflow: hidden; /* others */ overflow: visible clip; } mjx-stretchy-v > mjx-ext > mjx-c::before { width: initial; box-sizing: border-box; } mjx-stretchy-v > mjx-ext > mjx-c { transform: scaleY(500) translateY(.075em); overflow: visible; } mjx-mark { display: inline-block; height: 0px; } mjx-msup { display: inline-block; text-align: left; } mjx-c.mjx-c1D458.TEX-I::before { padding: 0.694em 0.521em 0.011em 0; content: "k"; } mjx-c.mjx-c32::before { padding: 0.666em 0.5em 0 0; content: "2"; } mjx-c.mjx-c2212::before { padding: 0.583em 0.778em 0.082em 0; content: "\2212"; } mjx-container[jax="CHTML"] { line-height: 0; } mjx-container [space="1"] { margin-left: .111em; } mjx-container [space="2"] { margin-left: .167em; } mjx-container [space="3"] { margin-left: .222em; } mjx-container [space="4"] { margin-left: .278em; } mjx-container [space="5"] { margin-left: .333em; } mjx-container [rspace="1"] { margin-right: .111em; } mjx-container [rspace="2"] { margin-right: .167em; } mjx-container [rspace="3"] { margin-right: .222em; } mjx-container [rspace="4"] { margin-right: .278em; } mjx-container [rspace="5"] { margin-right: .333em; } mjx-container [size="s"] { font-size: 70.7%; } mjx-container [size="ss"] { font-size: 50%; } mjx-container [size="Tn"] { font-size: 60%; } mjx-container [size="sm"] { font-size: 85%; } mjx-container [size="lg"] { font-size: 120%; } mjx-container [size="Lg"] { font-size: 144%; } mjx-container [size="LG"] { font-size: 173%; } mjx-container [size="hg"] { font-size: 207%; } mjx-container [size="HG"] { font-size: 249%; } mjx-container [width="full"] { width: 100%; } mjx-box { display: inline-block; } mjx-block { display: block; } mjx-itable { display: inline-table; } mjx-row { display: table-row; } mjx-row > * { display: table-cell; } mjx-mtext { display: inline-block; } mjx-mstyle { display: inline-block; } mjx-merror { display: inline-block; color: red; background-color: yellow; } mjx-mphantom { visibility: hidden; } _::-webkit-full-page-media, _:future, :root mjx-container { will-change: opacity; } mjx-c::before { display: block; width: 0; } .MJX-TEX { font-family: MJXZERO, MJXTEX; } .TEX-B { font-family: MJXZERO, MJXTEX-B; } .TEX-I { font-family: MJXZERO, MJXTEX-I; } .TEX-MI { font-family: MJXZERO, MJXTEX-MI; } .TEX-BI { font-family: MJXZERO, MJXTEX-BI; } .TEX-S1 { font-family: MJXZERO, MJXTEX-S1; } .TEX-S2 { font-family: MJXZERO, MJXTEX-S2; } .TEX-S3 { font-family: MJXZERO, MJXTEX-S3; } .TEX-S4 { font-family: MJXZERO, MJXTEX-S4; } .TEX-A { font-family: MJXZERO, MJXTEX-A; } .TEX-C { font-family: MJXZERO, MJXTEX-C; } .TEX-CB { font-family: MJXZERO, MJXTEX-CB; } .TEX-FR { font-family: MJXZERO, MJXTEX-FR; } .TEX-FRB { font-family: MJXZERO, MJXTEX-FRB; } .TEX-SS { font-family: MJXZERO, MJXTEX-SS; } .TEX-SSB { font-family: MJXZERO, MJXTEX-SSB; } .TEX-SSI { font-family: MJXZERO, MJXTEX-SSI; } .TEX-SC { font-family: MJXZERO, MJXTEX-SC; } .TEX-T { font-family: MJXZERO, MJXTEX-T; } .TEX-V { font-family: MJXZERO, MJXTEX-V; } .TEX-VB { font-family: MJXZERO, MJXTEX-VB; } mjx-stretchy-v mjx-c, mjx-stretchy-h mjx-c { font-family: MJXZERO, MJXTEX-S1, MJXTEX-S4, MJXTEX, MJXTEX-A ! important; } @font-face /* 0 */ { font-family: MJXZERO; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Zero.woff") format("woff"); } @font-face /* 1 */ { font-family: MJXTEX; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Main-Regular.woff") format("woff"); } @font-face /* 2 */ { font-family: MJXTEX-B; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Main-Bold.woff") format("woff"); } @font-face /* 3 */ { font-family: MJXTEX-I; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Math-Italic.woff") format("woff"); } @font-face /* 4 */ { font-family: MJXTEX-MI; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Main-Italic.woff") format("woff"); } @font-face /* 5 */ { font-family: MJXTEX-BI; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Math-BoldItalic.woff") format("woff"); } @font-face /* 6 */ { font-family: MJXTEX-S1; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Size1-Regular.woff") format("woff"); } @font-face /* 7 */ { font-family: MJXTEX-S2; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Size2-Regular.woff") format("woff"); } @font-face /* 8 */ { font-family: MJXTEX-S3; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Size3-Regular.woff") format("woff"); } @font-face /* 9 */ { font-family: MJXTEX-S4; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Size4-Regular.woff") format("woff"); } @font-face /* 10 */ { font-family: MJXTEX-A; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_AMS-Regular.woff") format("woff"); } @font-face /* 11 */ { font-family: MJXTEX-C; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Calligraphic-Regular.woff") format("woff"); } @font-face /* 12 */ { font-family: MJXTEX-CB; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Calligraphic-Bold.woff") format("woff"); } @font-face /* 13 */ { font-family: MJXTEX-FR; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Fraktur-Regular.woff") format("woff"); } @font-face /* 14 */ { font-family: MJXTEX-FRB; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Fraktur-Bold.woff") format("woff"); } @font-face /* 15 */ { font-family: MJXTEX-SS; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_SansSerif-Regular.woff") format("woff"); } @font-face /* 16 */ { font-family: MJXTEX-SSB; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_SansSerif-Bold.woff") format("woff"); } @font-face /* 17 */ { font-family: MJXTEX-SSI; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_SansSerif-Italic.woff") format("woff"); } @font-face /* 18 */ { font-family: MJXTEX-SC; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Script-Regular.woff") format("woff"); } @font-face /* 19 */ { font-family: MJXTEX-T; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Typewriter-Regular.woff") format("woff"); } @font-face /* 20 */ { font-family: MJXTEX-V; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Vector-Regular.woff") format("woff"); } @font-face /* 21 */ { font-family: MJXTEX-VB; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Vector-Bold.woff") format("woff"); } , that’s 99.9999…% with ninety 9s. The full observable universe would look like 460 of this image stacked on top of each other, bringing the total to about 85,000 life emergences and dropping the average distance to neighbabout a 1 billion light years, which is still too high to expect interaction.

I know this is the “low math” track of this series, but I cannot resist the temptation to drive this point with just a tiny bit of math. Let’s label the physical extent of the universe X, where X is basically any measure of size, such as "number of galaxies" in our last example. Then let’s declare another variable Y, which we’ll call the “life density” of the universe. Y tells us, for one unit of size, how many times we expect life to emerge in that region. So if the universe is 100 universe units, and Y is 1/10, then on average we would expect to see life spontaneously emerge in the universe 10 times - X times Y. This value, X times Y, is what we’ll call the universe’s “life propensity.”

We can guess at lower bounds for X, but we don’t know Y at all. In the example above, X was “number of galaxies”, and I suggested the density Y could be maybe 1/1,000,000. But it could reasonably be orders of magnitude lower than that. If the actual density was 1 in a billion, there would still be, in expectation, 200 intelligent species within the observable universe; the chance of life existing somewhere, anywhere, would still be effectively 100%. 

This point is very hard to overstate: if X is large, the range of possible values for Y is massive. If we selected a value for Y “at random,”[2] then we could wind up with virtually anything. To get a feel for it, even if we were to assume that literally nothing existed (or even could exist) outside of the observable universe, then the range of possible densities would look something like this gigantic thing:

Near the lower half of this range, even traveling many times the speed of light would not get aliens in range of humanity, even if they somehow knew which direction to go in.

And, I must remind you, this is assuming the observable universe is literally the only thing that exists. If anything exists outside the observable universe, and especially if there’s a multiverse, then the red part of this chart would go on basically forever. If you could go many times the speed of light, and even if you know which direction to go in, you might never reach another civilizaiton.

Just ask the SSA 

Nothing above is really all that new. These ideas are related to things like the Anthropic Principle, and they come up all the time, not just when talking about the Fermi paradox but also in discussions about cosmic fine-tuning. Across all kinds of contexts, the question “why do we exist at all” is very easily answered by “because the universe is big enough and random enough that life has a high chance of emerging at least somewhere.”

Now, I never know how much to claim “most people” believe something, but at least in many cases I’ve seen, the default assumption seems to be that if you want to believe in a big universe, then you really ought to adopt the SIA. But that’s not actually true. In fact, I would (and will) argue that the conclusions I drew above are actually more aligned with the SSA.

I think the disconnect comes from the fact that when people talk about the SSA or SIA, they typically focus on universes where at least one observer always exists. Sleeping Beauty must choose between one version of herself or two, the presumptuous philosopher must select between a trillion humans or a trillion trillion. For possible universes where no one might exist, however, the SSA and SIA basically take the same attitude, which is to say “that universe isn’t possible.” If there was a 20% chance Sleeping Beauty wouldn’t wake up at all, both the SSA and SIA would give that possibility 0% probability. The SSA’s famous smallness bias does not extend to the situations where nothing actually exists.

Let’s go back to playing dice. Suppose your friend Jim rolls some number of ten-sided dice, say N dice. You don’t know N, but after he’s finished rolling, he comes to you and tells you whether any of the dice came up 10. Not which die, not how many dice there were, just “yes, at least one of them was 10,” or “no, none of them were 10s.”

If getting a 10 corresponds to the observation that life exists, then 1/10 is the life density (because of the ten-sided die)[3], and N is the size of the universe. The propensity is the expected number of times he gets a ten, and that’s N/10. Since we don’t know which roll got the 10, this is equivalent to us knowing we exist, but not our relation to other species which may also exist.

What we want to know is, once we learn that Jim has rolled at least one 10, what credence should we give to each possible value of N? As with all anthropic problems, we’d have to start with a prior over universes (like the 50% chance of heads or tails in the Sleeping Beauty problem) and then see how our anthropic assumptions distort that prior. For the sake of argument, let’s say that N can be, with equal likelihood, any value between 0 and 100:

Now, having learned that Jim has rolled a 10, let’s go through each possible value of N and update our predictions accordingly by removing the percentage of rolls which would not get 10. The first thing we do is eliminate N = 0  - no dice means no chance of rolling 10. N = 1 reduces by exactly 90%, to 1/10 of its original value, since that’s the chance of a single die succeeding. With a little calculation, we find that N = 2 is reduced by 81% of its original value, to just under 20% of where it started. At N = 10, we’re down by about 35% of our original value, and by the time we reach N = 100, while rolling a 10 still isn’t technically guaranteed, our original prior is only adjusted by 0.003%. For all intents and purposes, if you roll a hundred 10-sided dice, you can be pretty damn sure you’ll get at least one 10. Hell, even if you roll only 50 10-sided dice, you can still be pretty sure at least one will come up 10.

However, since this process didn’t actually tell you which die came up 10 (by anaolgy, which observer you are), the SSA makes no further adjustments beyond removing these impossible universes. The SSA therefore transforms the above distribution of universes into something that looks like this:

The result is that universes with low N are more or less eliminated, but high N universes - universes where the size outstrips the density by even a small margin - are left basically unaffected. The flip side of this is that if you believe and/or observe that the density of life is low (e.g. that the dice are many-sided), you must conclude that N is high.[4]

And remember that this is the SSA, the assumption that presumably forces us to be deeply solipsistic. Even there, low density means the universe must be vast. This happens by default, simply because of the SSA’s distortions to the prior.

What about the SIA? In my opinion, it fares worse. The conventional narrative is that the SIA, by virtue of predicting larger universes, is the one that champions hugeness, and that our observations of the universe’s hugeness therefore confirm its accuracy. But what’s actually happening is that the SSA is the one which supports conventional hugeness arguments, and the SIA just throws in some weird, extra bias on top. Specifically, the SIA multiplies the likelihood of each N by about N, creating a completely unjustified assumption that N must be, a priori, quite high:

Ignore anthropics for a second and focus solely on Jim’s dice rolls: how, exactly, are we supposed to motivate a distribution of this shape in that problem? All we know is that there was at least one roll that came up 10. Past maybe 40 rolls, this event is pretty much guaranteed no matter how far above 40 you go. We must, in that situation at least, favor the SSA.

The next piece is "Putting it all Together"

1. ^

   From Wikipedia, the range is apparently a few tens of thousands of light-years, assuming a radio signal directed straight at us.

2. ^

   Which would obviously require a measure we don’t have.

3. ^

   Technicality: I switched from a rate to probability here. I did this because the example is discrete. In the continuous case we’ll look at later (with equations), we’ll obviously use rates again

4. ^

   Or I guess that God made humanity anyway.

5. ^

   Actually, what happens is our prior becomes impossible to define until we gain enough evidence to condition on a finite universe.

6. ^

   That is not, in my opinion, the weirdest or worst part of the AA for God, by the way. The weirdest part is the implication that the existence of infinity implies God. As I’ve been arguing, “infinity + randomness” seems a lot easier to swallow than “infinity + an intelligent entity that can nonetheless manipulate infinity at will.”

7. ^

   It should technically be “life mass,” since it’s a density times a volume. But for some reason I find that phrase off-putting.

8. ^

   To be a bit more precise,you can always assign arbitrary numbers to the dice and say "dice 3 rolls a 10." But what might be harder to do is come up with a numbering system that someone else would be able to identify if they didn't already have access to the dice. Even then, with dice you could

Discuss

[This is the seventh piece in my series on the Parametric Process Assumption (PPA), a new model of anthropics. You should start here if you are just seeing this for the first time.

The previous piece is "Indexical Models Don't Have Diversity Biases."

TLDR:

• SSA is not "temporally consistent" - learning your brith rank causes an indexical update which can, for instance, change the probabilities of a fair coin that has not yet been tossed
• In order to perform an indexical update - the kind which causes distortions in the SSA and removes presumptuousness from the SIA - you need to learn your rank relative to some absolute property defined for all observers.
  • Simply using a metric like "closest to the sun" would not help because all observers using such a procedure (picking the nearest star) would always find themselves the closest
  • An actual update would require something like "ranked distance from largest star in the universe" or "life-bearing universe in multiverse with Nth largest gravitational constant," information we could not possibly hope to acquire

The next piece is "The Fermi Paradox, Fine-Tuning, and the SSA."]

One well-established failure mode of the SSA is that it is not "temporally consistent." I believe Nick Bostrom first pointed this out and Joe Carlsmith has an excellent write up on it here. 

The simplest example of inconsistency in the SSA is the Late-Flip version of the Sleeping Beauty Problem where the coin is flipped only after Sleeping Beauty is woken on Monday. In this scenario, P(Heads | Monday) seems as if it must be 0.5, since the coin hasn’t even been flipped yet. The SSA, however, assigns 2/3 credence to P(Heads | Monday).

In that sense, what we mean when we say “temporal consistency” is that for some event X with fixed and known probability, like the flip of a known-fair coin, learning our identity should not alter our credence in X if our identity was set before X happened. But there are two catches to this:

1. This only applies if the future event has fixed and known probability. If you don’t know whether or not the coin is fair, but then learn that it is 75% biased towards heads, then this will change your credence in the coin going heads even though the event has not happened yet.
2. When learning something about your identity, there is nothing inherently paradoxical about indexical information changing the credences of past events.

Point 1, I hope, is non-controversial. Point 2 is a bit thornier, and I wouldn’t blame you for arguing we don’t want indexical updates to change the past either. Indeed, in order for temporal consistency to hold in all cases, the only credences it would actually be permissible to distort (as in, alter beyond a normal update on the universe prior) are those on events from before the very first observer was created. 

And it's also true that, upon learning your full identity, the SIA induces no distortions on either the past or the future, so point 2 is unnecessary in that case.

The situation for the PPA is not much worse, though. The PPA still does not distort any events on the same timeline as the universe you are in; the only thing it actually distorts are the credences you give to the parameters of your model, since those are the only things that behave SSAishly. The idea behind parameters however is that they represent fixed laws of math and physics, so - if they can be said to be temporal at all - they are unambiguously “sampled” before the universe or any observers existed. If you buy even the weakest form of point 2, then there is nothing inherently paradoxical about this.

In fact, the way we're using parameters in the PPA (as the variable for expected count of intelligent species) means that it is virtually impossible for us to make an update that causes an anthropic distortion. It would not be enough simply to learn the size of the universe, or that other forms of life existed on other planets. You would have to specifically learn what we're going to call a "nonindexical label" related to your specific place in the universe, which, as we're about to see, can only be done if you already have significant global knowledge of the entire universe's state.

Nonindexical Labeling

Let’s take a step back and talk about what it takes to even trigger the kind of indexical updates that make this sort of discussion necessary. As we saw in the piece on diversity, there is a lot of information about ourselves or the universe for which our credences are unperturbed by the use of indexicals.

Similarly, not every piece of information we learn about ourself will trigger a significant indexical update. To take a simple example, suppose we are in a Sleeping Beauty-like scenario where we are woken up either one day, two, three or four days in a row, starting on Monday. Learning that it was, say, Tuesday, would indeed cause a large indexical update, but merely learning that there had been one other wake-up would only cause an update in the universe prior.

Learning that the day is Tuesday under the SSA. The update is indexical, in that some of the rows are narrowed.

Learning that there are at least 2 wake-ups under the SSA. The update is only in the universe prior, in that some rows are removed but the widths do not change.

We can see that the first update is indexical because some of the rows are narrowed, whereas the second update is only on the prior because rows are removed but not narrowed. We can also have much smaller indexical updates by only learning, for instance, that there was a wake-up the day before, which instead of narrowing each row to a single observer, merely removes one from each row:

The prior is perturbed a little because we know we are not the first person, but the effect is smaller than a normal SSA update and shrinks as the number of observers increases.

In order to get strong indexical effects it’s not enough to place yourself relative to others; you need to learn your absolute position relative to other observers. This is easy enough to do (in an approximate sense) in the temporal case where your species/experiment has a start time and a well-defined notion of order, but in many cases learning absolute information is almost impossible. In particular, when we’re wondering about things like “how many other intelligent species exist in the universe?” it will be almost impossible to gather information which allows you to do meaningful updates.

Take another look at the 4 wake-up problem from a moment ago, but now imagine that each of these represents a possible intelligent species somewhere in the universe you might have been born into. If you learned you were only species #2, for instance, that would induce the same updates we had upon learning it was Tuesday in the last scenario. Everything would look the same.The question is how we would actually learn we were species #2.

The answer is that we would need some sort of “species rank.” One way to do this would be to do the same thing we did with birth rank, and just order temporally by species emergence in the universe. But how would we ever learn that information? In order to have any confidence that we were actually, say, the 2nd species in the universe, we would first have to map out the entire universe, identify every species in it, and then study their history until we were sure they emerged at a certain time. Only then would we get something like an indexical update.

Except that would only apply for our universe. What if we were in a multiverse? Would the concept of temporal ordering even exist between multiverses? Presumably not.

But let’s ignore that for a moment and keep the focus on our own universe. We’ve established that it’s difficult to use temporal ordering to learn species rank, but one might wonder if there are other markers we could use to rank species.

In fact, virtually any arbitrary metric could be used to label species. For instance, we might decide to use distance from the sun, in which case we can say that humans, being the only intelligence on Earth[1] get the label of “first.”

But this won’t give us an indexical update. If you think it would, you’re making a mistake known as the "Texas sharpshooter fallacy." The canonical metaphor for the fallacy is a man shooting randomly at his barn, hitting a spot completely at random, and then drawing a target right over where the bullet hit. Then everyone gathers round and thinks to themselves "Wow! What were the chances he would hit that exact target 3.231 feet from the left and 8.654 feet off the ground?"

The lesson for us is that when you condition on a certain event happening (for instance, the bullet hitting some extremely specific spot on the barn, or being closest to the sun), you also need to condition on the process that selected that event. It's not "what is the probability that you would hit a bullseye at coordinate (3.231, 8.654)?" it's "what is the probability that you would hit a bullseye drawn right where the bullet hit?" The answer to the second question is obviously "100%."

In the anthropic case, if every species were to assign a ranking by looking around, finding the closest star/large celestial landmark, and choosing to rank based on that, then they would all rank themselves number 1. Thus, the probability of being rank 1 using this procedure is 100% in all universes, and no update is possible.

Instead, what you would need to do is rank yourself according to some kind of universal anchor that would be identified identically by all species in the universe/multiverse, wherever they are. In a single universe, this could be time since the Big Bang, but that would mostly only effect your prior on the typical number of species within a single universe;[2] if you believed there might be a multiverse you wouldn’t be able to do any serious indexical updating because you wouldn’t know how many other life-bearing universes there were. 

And, as we mentioned, indexing those would be basically impossible. Perhaps you could rank multiverses by “largest gravitational constant” or something similar, but updating would still require you to know how much life existed in each and every multiverse. The point is, without significant global information about the entirety of existence, you’re not going to be able to perform indexical updates about your species. If you’re using the SSA, you’ll have trouble moving beyond your prior for species-level information[3] and if you’re using the SIA, you’ll be locked into a “presumptuous” view of the universe where you’re unduly biased towards a multitude of species.

Now, the fact that you need global information to perform indexical species updates is a practical objection, not a theoretical one. You can’t, as far as I can tell, use this fact as a first principle from which to design an anthropic assumption. But it’s still something to keep in mind when we explore the PPA, since it is only these sorts of fully-anchored species-level updates which can actually cause indexical SSAish updates to your model’s parameters. For all intents and purposes, you will never be able to learn the kind of information that would cause the PPA to perform an unusual update.

The next piece is "The Fermi Paradox, Fine-Tuning, and the SSA"

1. ^

   Well, until recently anyway

2. ^

   In the next piece, we’ll discuss the concept of “life density.” In that framing, learning you were the first species in the universe you’re in (or the observable part of it) would just cause you to update on life being low-density across the multiverse, but under a reasonable prior it would not be sufficient to tell you anything about the total amount of life.

3. ^

   This is only asymptotically true for large universes, see the next piece.

4. ^

   Another way to think about it: the SSA and SIA are functions that take a universe prior distribution and transforms it into an indexical distribution. In order for this process to be cogent, it needs to produce the same indexical distribution for all observers. That means the universe prior has to be the only input to the function. If you need to use indexical information to produce indexical labels, you can't deterministically produce a single output distribution.

5. ^

   Our universe's location in the entire multiverse, if it existed, would still be exchangeable.

6. ^

   What about temporal species information? To be nonexchangeable, you must learn not when your species exists int he universe, but your species' rank. In other words, you must learn that you are, e.g. the fourth intelligent species to emerge in the entire universe.

7. The universe, at least classically, is a Markov Chain.

8. ^

   To be clear: our proximity to the sun is still information that allows us to update our knowledge of the universe. But because that fact is exchangeable with respect other our universe prior, learning it alone cannot cause an indexical update about the number of other species in the universe, only an update accoridng on our prior.

Discuss

[This is the sixth piece in my series on the Parametric Process Assumption (PPA), a new model of anthropics. You should start here if you are just seeing this for the first time.

The previous piece is "Improper Priors and the Infinite SIA."

TLDR:

• Neither the SSA nor SIA have any bias towards observer attributes which aren't directly correlated with the number of observers in the universe
• For the Basic Model, which we discuss more at the end of the series, this will apply to any properties of various species (physiology, local universe observations, etc.) which don't a priori tell us something about how long our species will survive.

The next piece is "Temporal Consistency and Texas Sharpshooting."]

Another misconception I had at one point was that the SIA produced a great bias for diversity than the SSA.

By diversity, I mean basically any measure of observers having different experiences from my own. That could mean the existence of a person having a different childhood from mine, or it could mean four-armed green-skinned aliens from the Andromeda Galaxy. Literally anything that isn’t directly correlated with the number of observers in the universe counts.[1]

Both the SIA and SSA update on this type of information identically. To see why, let's start with a simple example where, with 50% probability, only humans exist, and with 50% probability we have a more diverse universe where in addition to the humans we also have an equal number of aliens. Unconditionally, the SIA does say the alien universe is more likely, but once we learn we are human, we revert to the universe prior and the bias disappears:

After learning you are human, the "Human + Alien" universe has the same mass as the "Human Only" universe

So, in practice, the conditional SIA is agnostic about observers different from us in ways we can observe, which includes all of our experiences up to this point.

What about the SSA? It is true that if you run the same scenario through the SSA, you will predictably find that conditioning on your humanity makes the aliens less likely to exist:

After learning you are human, the "Human + Alien" universe has half the mass of the "Human Only" universe

These examples were what originally convinced me that the SSA had a bias for less diversity, but they're actually focusing on the wrong things entirely.[2] The second universe in this example isn't just more diverse, it's also contains twice as many observers, and we already know that the SIA prefers bigger universes. We could just as easily image a situation where the smaller universe was more diverse, and then it would be the SSA which updates to a bigger universe.

Consider for instance a universe prior where with 50% probability there are 4N humans, and with 50% probability there are N humans and N aliens. Putting both side-by-side (with the SSA on top and the SIA on bottom) it's immediately clear that the SSA puts twice as much relative weight onto the "Human + Alien" universe.

That's because anthropic biases only concern themselves with universe size. If you use a model where size and diversity aren't correlated, then anthropic biases won't play into it.

More broadly, the reason the first "Humans or Humans + Aliens" model is wrong is because it assumes that humans must exist no matter what, and the only real question is what else exists besides. If we don't privilege humans in this way, but rather consider them just one type of life that might have existed among many, then we'll find it a lot easier to construct examples where diversity doesn't matter.

For instance, suppose for the sake of argument we knew that the universe would independently produce exactly two different species, and this meant there were exactly 3 equally likely outcomes:

• Both were human
• One was human and one was alien
• Both were alien

In this case, regardless of whether we used the SSA or the SIA, both the unconditional and conditional distributions would be identical:

This sort of effect is the same we'll see in the Basic Model later in this series. There, the diversity variable will apply at a species-level, referencing any of an observer's various properties (physiology, distance to closest star, etc.) which don't a priori tell them something about how long their species will survive. Thanks to the results above, we will be able to safely ignore those variables in our analysis.

The next piece is "Temporal Consistency in Anthropics"

1. ^

   Specifically, anything that isn’t correlated with the longevity of your species and/or the universe size within your prior. In reality, any number of things could be unexpectedly correlated with longevity, but if you don’t have any a priori reason to model them as correlated, the SSA and SIA won’t have anything special to say about it.

2. ^

   This example can also be used to argue that the SSA is too sensitive to reference classes, since whether or not you include the aliens in your reference class changes the SSA's prediction. See this piece by Joe Carlsmith.

   That's a very valid critique of the SSA, but it's not the point where making here. We're assuming our reference class contains every possible observer.

Discuss

[This is the fifth piece in my series on the Parametric Process Assumption (PPA), a new model of anthropics. You should start here if you are just seeing this for the first time.

The previous piece is "Parametric Modeling and Types of Uncertainty."

TLDR:

• Improper priors allow you to use a prior that is not a true distribution.
  • They can be updated and, with enough new information, be made proper
  • Until there is enough information to become proper, improper priors cannot be used to make predictions
• The SIA over universes with potentially infinite observers can be thought of as an improper prior that updates to proper once you learn enough about your identity

The next piece is "Indexical Models Don't Have Diversity Biases."]

Not all priors make predictions

[If you already know what an improper prior is and why the SIA functions as one, then you may wish to skip to the "SIA as an improper prior” section]

One thing that everyone knows is that it is not possible to make a uniform selection over the set of all integers. If you tried to make a uniform distribution over this range, you would have to assign each number a probability of 1/ mjx-TeXAtom { display: inline-block; text-align: left; } mjx-msub { display: inline-block; text-align: left; } mjx-c.mjx-c221E::before { padding: 0.442em 1em 0.011em 0; content: "\221E"; } mjx-c.mjx-c1D53C.TEX-A::before { padding: 0.683em 0.667em 0 0; content: "E"; } mjx-c.mjx-c5B::before { padding: 0.75em 0.278em 0.25em 0; content: "["; } mjx-c.mjx-c1D45B.TEX-I::before { padding: 0.442em 0.6em 0.011em 0; content: "n"; } mjx-c.mjx-c1D462.TEX-I::before { padding: 0.442em 0.572em 0.011em 0; content: "u"; } mjx-c.mjx-c2223::before { padding: 0.75em 0.278em 0.249em 0; content: "\2223"; } mjx-c.mjx-c1D703.TEX-I::before { padding: 0.705em 0.469em 0.01em 0; content: "\3B8"; } mjx-c.mjx-c5D::before { padding: 0.75em 0.278em 0.25em 0; content: "]"; } mjx-math { display: inline-block; text-align: left; line-height: 0; text-indent: 0; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; border-collapse: collapse; word-wrap: normal; word-spacing: normal; white-space: nowrap; direction: ltr; padding: 1px 0; } mjx-container[jax="CHTML"][display="true"] { display: block; text-align: center; margin: 1em 0; } mjx-container[jax="CHTML"][display="true"][width="full"] { display: flex; } mjx-container[jax="CHTML"][display="true"] mjx-math { padding: 0; } mjx-container[jax="CHTML"][justify="left"] { text-align: left; } mjx-container[jax="CHTML"][justify="right"] { text-align: right; } mjx-mi { display: inline-block; text-align: left; } mjx-c { display: inline-block; } mjx-utext { display: inline-block; padding: .75em 0 .2em 0; } mjx-mn { display: inline-block; text-align: left; } mjx-mo { display: inline-block; text-align: left; } mjx-stretchy-h { display: inline-table; width: 100%; } mjx-stretchy-h > * { display: table-cell; width: 0; } mjx-stretchy-h > * > mjx-c { display: inline-block; transform: scalex(1.0000001); } mjx-stretchy-h > * > mjx-c::before { display: inline-block; width: initial; } mjx-stretchy-h > mjx-ext { /* IE */ overflow: hidden; /* others */ overflow: clip visible; width: 100%; } mjx-stretchy-h > mjx-ext > mjx-c::before { transform: scalex(500); } mjx-stretchy-h > mjx-ext > mjx-c { width: 0; } mjx-stretchy-h > mjx-beg > mjx-c { margin-right: -.1em; } mjx-stretchy-h > mjx-end > mjx-c { margin-left: -.1em; } mjx-stretchy-v { display: inline-block; } mjx-stretchy-v > * { display: block; } mjx-stretchy-v > mjx-beg { height: 0; } mjx-stretchy-v > mjx-end > mjx-c { display: block; } mjx-stretchy-v > * > mjx-c { transform: scaley(1.0000001); transform-origin: left center; overflow: hidden; } mjx-stretchy-v > mjx-ext { display: block; height: 100%; box-sizing: border-box; border: 0px solid transparent; /* IE */ overflow: hidden; /* others */ overflow: visible clip; } mjx-stretchy-v > mjx-ext > mjx-c::before { width: initial; box-sizing: border-box; } mjx-stretchy-v > mjx-ext > mjx-c { transform: scaleY(500) translateY(.075em); overflow: visible; } mjx-mark { display: inline-block; height: 0px; } mjx-msup { display: inline-block; text-align: left; } mjx-c.mjx-c1D458.TEX-I::before { padding: 0.694em 0.521em 0.011em 0; content: "k"; } mjx-c.mjx-c32::before { padding: 0.666em 0.5em 0 0; content: "2"; } mjx-c.mjx-c2212::before { padding: 0.583em 0.778em 0.082em 0; content: "\2212"; } mjx-container[jax="CHTML"] { line-height: 0; } mjx-container [space="1"] { margin-left: .111em; } mjx-container [space="2"] { margin-left: .167em; } mjx-container [space="3"] { margin-left: .222em; } mjx-container [space="4"] { margin-left: .278em; } mjx-container [space="5"] { margin-left: .333em; } mjx-container [rspace="1"] { margin-right: .111em; } mjx-container [rspace="2"] { margin-right: .167em; } mjx-container [rspace="3"] { margin-right: .222em; } mjx-container [rspace="4"] { margin-right: .278em; } mjx-container [rspace="5"] { margin-right: .333em; } mjx-container [size="s"] { font-size: 70.7%; } mjx-container [size="ss"] { font-size: 50%; } mjx-container [size="Tn"] { font-size: 60%; } mjx-container [size="sm"] { font-size: 85%; } mjx-container [size="lg"] { font-size: 120%; } mjx-container [size="Lg"] { font-size: 144%; } mjx-container [size="LG"] { font-size: 173%; } mjx-container [size="hg"] { font-size: 207%; } mjx-container [size="HG"] { font-size: 249%; } mjx-container [width="full"] { width: 100%; } mjx-box { display: inline-block; } mjx-block { display: block; } mjx-itable { display: inline-table; } mjx-row { display: table-row; } mjx-row > * { display: table-cell; } mjx-mtext { display: inline-block; } mjx-mstyle { display: inline-block; } mjx-merror { display: inline-block; color: red; background-color: yellow; } mjx-mphantom { visibility: hidden; } _::-webkit-full-page-media, _:future, :root mjx-container { will-change: opacity; } mjx-c::before { display: block; width: 0; } .MJX-TEX { font-family: MJXZERO, MJXTEX; } .TEX-B { font-family: MJXZERO, MJXTEX-B; } .TEX-I { font-family: MJXZERO, MJXTEX-I; } .TEX-MI { font-family: MJXZERO, MJXTEX-MI; } .TEX-BI { font-family: MJXZERO, MJXTEX-BI; } .TEX-S1 { font-family: MJXZERO, MJXTEX-S1; } .TEX-S2 { font-family: MJXZERO, MJXTEX-S2; } .TEX-S3 { font-family: MJXZERO, MJXTEX-S3; } .TEX-S4 { font-family: MJXZERO, MJXTEX-S4; } .TEX-A { font-family: MJXZERO, MJXTEX-A; } .TEX-C { font-family: MJXZERO, MJXTEX-C; } .TEX-CB { font-family: MJXZERO, MJXTEX-CB; } .TEX-FR { font-family: MJXZERO, MJXTEX-FR; } .TEX-FRB { font-family: MJXZERO, MJXTEX-FRB; } .TEX-SS { font-family: MJXZERO, MJXTEX-SS; } .TEX-SSB { font-family: MJXZERO, MJXTEX-SSB; } .TEX-SSI { font-family: MJXZERO, MJXTEX-SSI; } .TEX-SC { font-family: MJXZERO, MJXTEX-SC; } .TEX-T { font-family: MJXZERO, MJXTEX-T; } .TEX-V { font-family: MJXZERO, MJXTEX-V; } .TEX-VB { font-family: MJXZERO, MJXTEX-VB; } mjx-stretchy-v mjx-c, mjx-stretchy-h mjx-c { font-family: MJXZERO, MJXTEX-S1, MJXTEX-S4, MJXTEX, MJXTEX-A ! important; } @font-face /* 0 */ { font-family: MJXZERO; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Zero.woff") format("woff"); } @font-face /* 1 */ { font-family: MJXTEX; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Main-Regular.woff") format("woff"); } @font-face /* 2 */ { font-family: MJXTEX-B; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Main-Bold.woff") format("woff"); } @font-face /* 3 */ { font-family: MJXTEX-I; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Math-Italic.woff") format("woff"); } @font-face /* 4 */ { font-family: MJXTEX-MI; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Main-Italic.woff") format("woff"); } @font-face /* 5 */ { font-family: MJXTEX-BI; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Math-BoldItalic.woff") format("woff"); } @font-face /* 6 */ { font-family: MJXTEX-S1; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Size1-Regular.woff") format("woff"); } @font-face /* 7 */ { font-family: MJXTEX-S2; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Size2-Regular.woff") format("woff"); } @font-face /* 8 */ { font-family: MJXTEX-S3; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Size3-Regular.woff") format("woff"); } @font-face /* 9 */ { font-family: MJXTEX-S4; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Size4-Regular.woff") format("woff"); } @font-face /* 10 */ { font-family: MJXTEX-A; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_AMS-Regular.woff") format("woff"); } @font-face /* 11 */ { font-family: MJXTEX-C; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Calligraphic-Regular.woff") format("woff"); } @font-face /* 12 */ { font-family: MJXTEX-CB; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Calligraphic-Bold.woff") format("woff"); } @font-face /* 13 */ { font-family: MJXTEX-FR; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Fraktur-Regular.woff") format("woff"); } @font-face /* 14 */ { font-family: MJXTEX-FRB; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Fraktur-Bold.woff") format("woff"); } @font-face /* 15 */ { font-family: MJXTEX-SS; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_SansSerif-Regular.woff") format("woff"); } @font-face /* 16 */ { font-family: MJXTEX-SSB; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_SansSerif-Bold.woff") format("woff"); } @font-face /* 17 */ { font-family: MJXTEX-SSI; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_SansSerif-Italic.woff") format("woff"); } @font-face /* 18 */ { font-family: MJXTEX-SC; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Script-Regular.woff") format("woff"); } @font-face /* 19 */ { font-family: MJXTEX-T; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Typewriter-Regular.woff") format("woff"); } @font-face /* 20 */ { font-family: MJXTEX-V; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Vector-Regular.woff") format("woff"); } @font-face /* 21 */ { font-family: MJXTEX-VB; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Vector-Bold.woff") format("woff"); } , which is just equal to zero, and adding up an infinite number of zeros just gets you zero, not 1.[1]

I have seen this fact used to argue that, since identities are selected uniformly, neither the SIA nor the SSA can actually construct a probability distribution over infinite observers. At minimum, I myself have erroneously made claims of this sort in the past.

In reality, however, there are many different ways that the distribution of observers can be infinite, and at least a couple of tricks we can do to get valid results.[2] The more technical parts of this series will go through a whole bunch of these. For now, what I want to talk about is the concept of an improper prior⁠. 

In a sentence, an improper prior is a kind of prior that lets you treat certain non-distributions - like uniform selection over infinity - as if they were real distributions for the purposes of performing Bayesian updates.  

Consider what a prior actually is.  If start with some event, such a coin landing on heads, then a prior is a mathematical object that lets us:

1. Update when we get new evidence. For instance, we may observe 10 tails in a row and update away from a 50% chance of heads
2. Make predictions about what will happen next, e.g. that the coin will land heads with 50% probability

An improper prior is, informally, just a special kind of prior that let's us to the first thing, but doesn’t let us do the second thing, at least not until we've gathered enough evidence to make our prior proper again.[3]

Another technically-not-correct-but-still-useful way to think about an improper prior is that it’s a stand-in for when we know so little about a distribution that it isn’t even rational to make a specific guess. In the uniform-over-infinity case, for instance, an improper uniform prior is simply telling us that we know there’s a number that was selected, but couldn’t even begin to guess what it might be.[4] Then, if we later learn the number is, say, no more than a million, the improper prior still lets us update to “uniform between one and a million” in the normal way. 

SIA as an improper prior

What this means for anthropics is that, under the right circumstances, we will be able to use the SIA (and PPA) as improper priors.[5] There will be situations where, before we learn anything, these models won’t allow us to make predictions without data, but which will suddenly become valid distributions as soon as we collect some data. 

To take a really simple example, let’s imagine an “infinite Sleeping Beauty problem” under the SIA. It proceeds like this:

• On heads, Sleeping Beauty is woken up once a day for exactly 7 consecutive days
• On tails, Sleeping Beauty is made immortal and woken up once every day for an infinite number of days

Without knowing the day, this simply can’t be constructed as a proper distribution. With some definitional kung fu, we could probably make some kind of pseudo-distribution where we conclude with 100% certainty that the coin went tails, but there is no formalism where we could give any actual positive probability to any particular day.

But it still works fine as an improper prior! If Sleeping Beauty wakes up and learns it’s the fourth day of the experiment, for instance, then she just updates on that and voilà! She has a nice 50-50 chance of existing in an infinitely long universe:

Interestingly, while the SIA is often the one thought of as worse at infinity, this improper prior trick cannot be made to work on the SSA. You can see this for yourself by trying to imagine how we would redraw these mass charts for the SSA; the bottom universe would be sliced into infinitely many pieces, but when you conditioned on, say, day 4 that slice would have zero width. The best you could maybe do, with some limit math, would be to say that P(Heads) = 1, which would be completely degenerate if you ever learned the day was greater than 7.

The next piece is "Indexical Models Don't Have Diversity Biases"

1. ^

   Alternative intuition: any number you pick will always have infinitely more numbers above it than below it, so you will always pick something in the bottom 1/ of the distribution.

2. ^

   One example - the set of observers could be unbounded, but finite in expectation. In this scenario, the SSA would have no trouble, but the SIA wouldn’t be able to build a true proper distribution. Stuart Armstrong has pointed this out.

3. ^

   The Wikipedia article gives a good technical explanation. Improper priors are possible because Bayes’ Rule doesn’t technically require that the prior be normalized in order to perform an update.

4. ^

   We’re still making an assumption even in the improper case, though. For instance, the decision to be uniform in N instead of, say, uniform in log(N)

5. ^

   Since the PPA has as a denominator, we will need to redefine it in terms of limits to get this result.

Discuss

[This is the fourth piece in my series on the Parametric Process Assumption (PPA), a new model of anthropics. You should start here if you are just seeing this for the first time.

The previous piece is "A Brief Defense of Indexical Reasoning."

TLDR:

• There are two different kinds of uncertainty
  • aleatoric uncertainty pertains to random events like coin flips. It is true randomness and cannot be reduced
  • epistemic uncertainty pertains to things that are fixed but unknown, like the digits of pi. It can be reduced by learning more about the world
• Bayesianism handles epistemic uncertainty by assigning it subjective subjective probabilities, this is known as constructing a prior.
  • Frequentism refuses to assign probabilities to epistemic unknowns, and must use alternative methods
  • When a Bayesian assigns a probability to an epistemic unknown, it is often called a parameter.
• Not everything can be an aleatoric unknown, at some point we must say that certain facts are merely epistemicly unknown, not aleatoric.

The next piece is "Improper Priors and the Infinite SIA."]

Ways of not knowing

Before diving into the PPA and its implications, let’s take a moment to motivate it philosophically: what do we mean when we say that we are uncertain about something? 

One idea I really like, but which hasn’t really made its way into the LessWrong water supply as much as I might expect, is that there are really at least two different types of uncertainty. The first kind, known as aleatoric uncertainty, is what we conventionally think of as randomness. The roll of a die, the flip of a coin, the configuration of a deck of cards dropped to the floor - that kind of thing.

The second kind is epistemic uncertainty. These are the things that have definitive, immutable truth values, but which are nevertheless unknown. We saw this in the digit-of-pi Sleeping Beauty problem from the introduction - a specific digit of pi is not, in any aleatoric sense, random, but we nonetheless may not know it and, for that reason, may choose to assign a probability to it. One could reasonably say, for instance, that the googolth digit of pi has a 1 in 10 chance of being 7. 

A few notable properties separate these concepts. Epistemic uncertainty, for instance, is reducible. If I were to compute the googolth digit of pi, then I would know it forever, and in general the more evidence I have about a given epistemic fact, the more confident I would be in my estimates, at least in the limit. Aleatoric uncertainty, by way of contrast, never reduces. No matter how many times I flip my fair coin, there will always be a 50% chance that the next flip will come up heads. 

In that sense, aleatoric uncertainty is the uncertainty of repetition, of what will happen “next time” or “in the limit.” When we assign a probability to an aleatoric uncertainty, we are doing so in a philosophically objective way: if you flipped a fair coin over and over and over again, the percentage of flips that went heads would get closer and closer to 50%. Even if you never actually make those flips, even if in practice it’s impossible to make that many flips, the abstraction is still mathematically well-defined.

Dice converging to their average value over many rolls. Courtesy of Wikipedia.

This definition of probability doesn’t apply to epistemic uncertainty. The googolth digit of pi cannot be resampled; it has always been whatever it currently is, and it will always go on being whatever it currently is. The one property that allowed us to define aleatoric uncertainty in an objective way - the hypothetical ability to resample - is of no use here.  The digits of pi, along with countless other mathematical truths, and likely much of physics, are not and have not ever been random. The googolth digit of pi does not have a 10% chance of being 7; it has a 100% chance of being whatever it is, and we do not know what that might be.[1]

And yet, it seems clear that there must be some sense in which we are able to say that each digit has probability 10%. This is the driving force behind Bayesian statistics: that when confronted with epistemic uncertainty, we can quantify it by assigning our beliefs to a probability distribution and applying the same laws of probability that we used in the aleatoric setting.  

That we can do this is by no means a given. There is an entire other school of statistics - the Frequentist school - that rejects the concept of epistemic probabilities. Mind you, Frequentists don’t reject the actual concept of epistemic uncertainty, but they don’t admit methods which assign probabilities to epistemic statements; in the world of Frequentism, one literally could not say that the googolth digit of pi had a 10% chance of being 7, or indeed being anything else. Confidence intervals, p-values and their ilk are all elaborate attempts to learn about the world without ever explicitly assigning credences to epistemic unknowns, and that is what makes them so famously counter-intuitive.

This aversion to epistemic probability is not completely unfounded. Unlike aleatoric uncertainty - which has an objective repetition-based interpretation - Bayesian/epistemic probabilities are inherently subjective. Bayesian priors represent a quantification of one’s beliefs, but there are no hard mathematics that can tell us where those beliefs come from. Priors are just guesses, and everyone’s guess might be different. You can’t even really say for sure if any one person’s prior is better than another’s. One prior may indeed prove closer to the truth than another, and any reasonable prior will converge to the truth with enough data, but it’s mathematically impossible to build a prior that’s always better in all situations when data is limited; this is known as the No Free Lunch Theorem. 

I don’t want to oversell these flaws. If you’re hanging out on LessWrong, you probably already know the incredible effectiveness of Bayesian reasoning. But even with something as trivial as assigning a probability to pi’s googolth digit, No-Free-Lunch still applies: while only a sociopath would decide that there was a 99% chance that the googolth digit was 7, it’s still possible that the digit actually is 7, and in that case the sociopath’s prior is in some sense better than that of the normie who only gave 7 a 10% chance.[2] Which is all to say that while assigning probabilities to beliefs about epistemic uncertainty may be a highly effective and intuitive way to reason about the world, it is not an objective way to reason about the world. 

Bayesian Parametric Modeling 101

[If you are already familiar with Parametric Bayesian models, you may wish to skip this section. Please do read the Closing Thoughts section though.]

Generally, in any real world situation, both types of uncertainty will be present. If you’re Bayesian then the most common way to model this is with what’s known as a parametric model.

The textbook example of a parametric model would be repeated flips of a biased coin. In this setup, you have a coin with some probability mjx-container[jax="CHTML"] { line-height: 0; } mjx-container [space="1"] { margin-left: .111em; } mjx-container [space="2"] { margin-left: .167em; } mjx-container [space="3"] { margin-left: .222em; } mjx-container [space="4"] { margin-left: .278em; } mjx-container [space="5"] { margin-left: .333em; } mjx-container [rspace="1"] { margin-right: .111em; } mjx-container [rspace="2"] { margin-right: .167em; } mjx-container [rspace="3"] { margin-right: .222em; } mjx-container [rspace="4"] { margin-right: .278em; } mjx-container [rspace="5"] { margin-right: .333em; } mjx-container [size="s"] { font-size: 70.7%; } mjx-container [size="ss"] { font-size: 50%; } mjx-container [size="Tn"] { font-size: 60%; } mjx-container [size="sm"] { font-size: 85%; } mjx-container [size="lg"] { font-size: 120%; } mjx-container [size="Lg"] { font-size: 144%; } mjx-container [size="LG"] { font-size: 173%; } mjx-container [size="hg"] { font-size: 207%; } mjx-container [size="HG"] { font-size: 249%; } mjx-container [width="full"] { width: 100%; } mjx-box { display: inline-block; } mjx-block { display: block; } mjx-itable { display: inline-table; } mjx-row { display: table-row; } mjx-row > * { display: table-cell; } mjx-mtext { display: inline-block; } mjx-mstyle { display: inline-block; } mjx-merror { display: inline-block; color: red; background-color: yellow; } mjx-mphantom { visibility: hidden; } _::-webkit-full-page-media, _:future, :root mjx-container { will-change: opacity; } mjx-math { display: inline-block; text-align: left; line-height: 0; text-indent: 0; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; border-collapse: collapse; word-wrap: normal; word-spacing: normal; white-space: nowrap; direction: ltr; padding: 1px 0; } mjx-container[jax="CHTML"][display="true"] { display: block; text-align: center; margin: 1em 0; } mjx-container[jax="CHTML"][display="true"][width="full"] { display: flex; } mjx-container[jax="CHTML"][display="true"] mjx-math { padding: 0; } mjx-container[jax="CHTML"][justify="left"] { text-align: left; } mjx-container[jax="CHTML"][justify="right"] { text-align: right; } mjx-mi { display: inline-block; text-align: left; } mjx-c { display: inline-block; } mjx-utext { display: inline-block; padding: .75em 0 .2em 0; } mjx-c::before { display: block; width: 0; } .MJX-TEX { font-family: MJXZERO, MJXTEX; } .TEX-B { font-family: MJXZERO, MJXTEX-B; } .TEX-I { font-family: MJXZERO, MJXTEX-I; } .TEX-MI { font-family: MJXZERO, MJXTEX-MI; } .TEX-BI { font-family: MJXZERO, MJXTEX-BI; } .TEX-S1 { font-family: MJXZERO, MJXTEX-S1; } .TEX-S2 { font-family: MJXZERO, MJXTEX-S2; } .TEX-S3 { font-family: MJXZERO, MJXTEX-S3; } .TEX-S4 { font-family: MJXZERO, MJXTEX-S4; } .TEX-A { font-family: MJXZERO, MJXTEX-A; } .TEX-C { font-family: MJXZERO, MJXTEX-C; } .TEX-CB { font-family: MJXZERO, MJXTEX-CB; } .TEX-FR { font-family: MJXZERO, MJXTEX-FR; } .TEX-FRB { font-family: MJXZERO, MJXTEX-FRB; } .TEX-SS { font-family: MJXZERO, MJXTEX-SS; } .TEX-SSB { font-family: MJXZERO, MJXTEX-SSB; } .TEX-SSI { font-family: MJXZERO, MJXTEX-SSI; } .TEX-SC { font-family: MJXZERO, MJXTEX-SC; } .TEX-T { font-family: MJXZERO, MJXTEX-T; } .TEX-V { font-family: MJXZERO, MJXTEX-V; } .TEX-VB { font-family: MJXZERO, MJXTEX-VB; } mjx-stretchy-v mjx-c, mjx-stretchy-h mjx-c { font-family: MJXZERO, MJXTEX-S1, MJXTEX-S4, MJXTEX, MJXTEX-A ! important; } @font-face /* 0 */ { font-family: MJXZERO; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Zero.woff") format("woff"); } @font-face /* 1 */ { font-family: MJXTEX; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Main-Regular.woff") format("woff"); } @font-face /* 2 */ { font-family: MJXTEX-B; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Main-Bold.woff") format("woff"); } @font-face /* 3 */ { font-family: MJXTEX-I; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Math-Italic.woff") format("woff"); } @font-face /* 4 */ { font-family: MJXTEX-MI; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Main-Italic.woff") format("woff"); } @font-face /* 5 */ { font-family: MJXTEX-BI; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Math-BoldItalic.woff") format("woff"); } @font-face /* 6 */ { font-family: MJXTEX-S1; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Size1-Regular.woff") format("woff"); } @font-face /* 7 */ { font-family: MJXTEX-S2; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Size2-Regular.woff") format("woff"); } @font-face /* 8 */ { font-family: MJXTEX-S3; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Size3-Regular.woff") format("woff"); } @font-face /* 9 */ { font-family: MJXTEX-S4; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Size4-Regular.woff") format("woff"); } @font-face /* 10 */ { font-family: MJXTEX-A; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_AMS-Regular.woff") format("woff"); } @font-face /* 11 */ { font-family: MJXTEX-C; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Calligraphic-Regular.woff") format("woff"); } @font-face /* 12 */ { font-family: MJXTEX-CB; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Calligraphic-Bold.woff") format("woff"); } @font-face /* 13 */ { font-family: MJXTEX-FR; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Fraktur-Regular.woff") format("woff"); } @font-face /* 14 */ { font-family: MJXTEX-FRB; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Fraktur-Bold.woff") format("woff"); } @font-face /* 15 */ { font-family: MJXTEX-SS; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_SansSerif-Regular.woff") format("woff"); } @font-face /* 16 */ { font-family: MJXTEX-SSB; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_SansSerif-Bold.woff") format("woff"); } @font-face /* 17 */ { font-family: MJXTEX-SSI; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_SansSerif-Italic.woff") format("woff"); } @font-face /* 18 */ { font-family: MJXTEX-SC; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Script-Regular.woff") format("woff"); } @font-face /* 19 */ { font-family: MJXTEX-T; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Typewriter-Regular.woff") format("woff"); } @font-face /* 20 */ { font-family: MJXTEX-V; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Vector-Regular.woff") format("woff"); } @font-face /* 21 */ { font-family: MJXTEX-VB; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Vector-Bold.woff") format("woff"); } mjx-c.mjx-c1D45D.TEX-I::before { padding: 0.442em 0.503em 0.194em 0; content: "p"; } of coming up heads, but you have no idea what this value is - the coin could have high and often come up heads, low , or anything in between.

If we could flip the coin an infinite number of times, then we could know exactly. But in the real world we can’t actually do that, and simply calling an epistemic unknown isn’t enough to tell us how we would estimate it with only a limited number of flips. What, for instance, is the exact probability that the coin is fair ( =50%) if we flip it once and get heads? What if we flip it 10 times and get the sequence HTHTHHHHHH?

One way to answer that, the standard Bayesian way, is to start with a probability distribution over possible values of . This distribution, of course, is what we call our prior. 

For the sake of illustration, we’ll use an example prior where we believe there are only three possible values for : 25%, 50%, and 75%. We are 100% certain the coin comes up heads one of those three percentages of the time, we just don’t know which. Before any coins are flipped, we can start with the assumption that all three of these possibilities are equally likely:

The true value of may be an epistemic unknown, but each individual coin flip is random in an aleatoric sense. If we think of the coin flips as a random process that evolves by repeatedly producing heads and tails, then is sort of like an input to that process. It does not tell us how the randomness will resolve on any individual flip, but it does give us the overall shape of how that randomness will proceed. A coin-flip process with input parameter =25%, for instance, will average out to have 25% heads. For that reason, is referred to as a “parameter.” If you're familiar with programming, the analogy might be that our process is a function and is the input that controls its progression. 

Each possible value of produces a different random process 

Trying to apply this model is where one encounters the famous Bayes Rule. In LW and adjacent spaces, Bayes Rule is often presented as a fundamental idea everyone should be familiar with, but if you aren’t the type of person who likes looking at equations, I don’t think it’s often clear exactly what the rule truly means, beyond perhaps some generic statement about “updating on evidence.” 

A more refined explanation of Bayes Rule, however, is that it inverts the parametric model we just introduced. The is dead is that parameters are the things we really want to know, but we can’t actually observe them directly, so instead we must observe other aleatoric randomness and try to guess the parameters underneath. Bayes Rule is just the formula we use to do that; it’s how we extract knowledge about parameters without actually seeing them.

For instance, suppose we kept our uniform prior over =25%, =50%, or =75%. After seeing a single coin flip go heads, we would apply Bayes Rule, do a bit of math, and perform an update, thus giving us a new prior (technically, a “posterior”) which looks like this:

In this particular case, seeing a single heads has really only caused us to move some of the probability mass away from =25% to =75%. This was just a particular quirk of the prior we started with however. If we go on to observe, say, another nine flips giving us the sequence HTHTHHHHHH, then our prior would update even more sharply:

Now we have a very sharp peak at =75%, and the possibility of a coin that comes up tails 75% of the time has been basically eliminated.

Parametric techniques are far and away the most common way Bayesian statistics is applied in the real world. Countless areas of fields like cosmology, medicine, and engineering rely on parametric models. Even modern LLMs are parametric in a sense - the size of neural networks is often measured in “parameters.” In that case, the sampling of the next output token is the equivalent of the individual coin flips in this section’s example; the model’s parameters define the process which is the LLM’s output.

Most formulations of anthropics that I’ve seen, however, don’t wrestle with this framing in a deep way. Just look at standard problems like Sleeping Beauty: a single coin is flipped, and then either one or two observers are created. There is no epistemic “prior” in a situation like that, there is simply a coin with completely known odds of heads. That's not a sufficient analogy for all the myriad complexities of real-world physics and epistomology. Something is clearly missing.

Closing Thoughts

A good way to think about parametric models is that they are mostly about nesting. In the previous section’s example (the coin of unknown bias), we saw that each possible value of created a new self-contained process where the coin behaved differently. In some sense, it’s as if the aleatoric randomness - the flips of the coin - has actually been nested within the epistemic uncertainty.

If you aren’t sticking around for the more technical portions of this series, this nested structure should be your biggest takeaway from this piece. Hopefully, the connection to the “digit-of-pi” Sleeping Beauty problem is clear: parameters behave SSAishly under repetition, but when we nest a process within them, that process behaves SIAishly. The PPA is (mostly) just a simple extension of parametric modeling to the case of indexical reasoning.

That said, there’s still an elephant in the room, which is that the line between epistemic and aleatoric is actually quite fuzzy. For instance, in the last section we imagined a coin where the value of was epistemic. But the real world is filled with coins. If I reach into my pocket and pull out a coin, wouldn’t that constitute a random coin? That would make the choice of aleatoric, not epistemic. 

What if we push the reasoning up a layer? The coin in question must surely have come from a particular factory, and this factory must have some probability distribution over the coins it produces - perhaps it produces a fair coin only X% of the time, and we’d like to know X. Does this make X an epistemic unknown? 

Maybe. Or maybe this factory is just one of a million factories built by the same construction company, and each one has a different chance of producing a fair coin, and the quality of our factory is just another random variable?[3]

You can play this game for a very long time, but you can’t play it forever. At some point, you’re going to hit a ceiling. Perhaps the Laws of Physics, governed by mathematical truths that are as fixed as the digits of pi, say that across all factories that would ever get built over the course of an infinite number of universe samples, the average coin would have a bias of 55%, or the average factory would produce fair coins 36% of the time, or the average Earth would produce coin factory construction companies that yada yada yada. Sooner or later, you have to arrive at an unknown that could not possibly be construed as aleatoric.

And I’m not saying that that has to be where you draw the line! We just spent a whole essay discussing the inherent subjectivity of building a prior. What you choose to model as epistemic or aleatoric (or rather, what you make your top-level parameter) is inevitably going to be subjective simply because you don't have complete knowledge of how the world works; the best you can do is find the point where “things could be different” stops feeling coherent to you, and draw the line there.

But the distinction is not completely imagined, either. And as we saw in the “digit-of-pi” Sleeping Beauty problem, it can have real consequences for your anthropic predictions.

The next piece is "Improper Priors and the Infinite SIA"

1. ^

   Maybe we could try to be clever and say the probability of each digit is 1/10 because if we select a digit at random the distribution would approach uniform? Even that wouldn’t necessarily work, because it has not been proven that pi is a normal number - it’s entirely conceivable that, say, the digit 2 appears more often than others. You would still need to appeal to some other notion of probability.

2. ^

   In the real world, we tend to measure predictors by the aggregate of multiple predictions. Presumably if our sociopath guessed “99% 7” for every digit, we would learn not to trust him.

3. ^

   Obligatory reference to Bayesian hierarchical modeling, which explicitly nests parameters inside other parameters

Discuss

[This is the third piece in my series on the Parametric Process Assumption (PPA), a new model of anthropics. You should start here if you are just seeing this for the first time.

The previous piece is "Mosaic Plots for Visualizing Anthropic Distributions."

TLDR:

• Non-indexical reasoning (what we're calling the "Updateless Double Halfer" method, or UDH) states that you should not assign explicit probability to the statement "I am a particular observer"
• Unlike the SSA or SIA, this approach is not robust to observations of randomness independent of all other variables.
• Radford Neal's Full Nonindexical Conditioning (FNC) states that as you observe more randomness, the UDH converges to the SIA

The next piece is "Parametric Modeling and Types of Uncertainty."]

As we discussed in the last piece, this series will be primarily focused on indexical models of anthropics, where “who you are” is modeled explicitly with a variable, and the existence of this variable causes a distortion in the probabilities you assign to world states. These distortions, applied unilaterally, inevitably seem to cause all sorts of paradoxes. The appeal of non-indexical models is that, by virtue of not adding these variables, they produce no distortions to the prior.

There was a point where I found this sort of non-indexical reasoning very tempting, mostly because it really did seem, at first, to resolve many problems of anthropics. In particular, Ape in the Coat’s series on the topic did a lot to convince me that no indexical model could be truly self-consistent.

Unfortunately, after spending weeks in the thrall of non-indexical double-halferdom, I ultimately concluded that the position was simply untenable. Yes, it solves many problems, but only by pretending they don’t exist. If probabilities actually could not be used to self-localize, then much of modern engineering would be impossible. What is the entire field of robotics, if not probabilistic localization of limbs and bodies? Or rocketry, or even a modern AI agent figuring out the specifics of its software environment? All of these things involve collecting evidence not just about the world itself, but also the specific angle from which you are observing it.

Full Non-Indexical Conditioning

Strangely, despite being a nominally “halfer” position, UDH is actually a thirder position, at least when applied in any realistic scenario. A version of this was first observed by Radford Neal, who called it “Full Nonindexical Conditioning”, or FNC. Its math is explained well both by Neal and ksvanhorn, but I wanted to give it my own spin in this section because (1) I think it’s much easier to understand visually, and (2) my impression is that many people come away from FNC thinking something fishy has happened, and I want to try to shine some light on that.

The problem I think is that neither of these sources state the FNC in the plainest possible terms.  The truth is all the FNC merely says is that if you start with UDH, then observe a bunch of unrelated random events, your distribution over worlds will converge to the SIA’s. This is strange, both because unrelated independent random events presumably shouldn't cause you to update anything, and also because it suggests you should have just used indexicals anyway.

This becomes easier to swallow when you understand why FNCish effects don’t happen in normal indexical distributions like the SSA or SIA. We'll start with the Sleeping Beauty problem, but add the twist that when Sleeping Beauty wakes up on either day, she is also given a coin labelled “A” on one side and “B” on the other. 

Our intuition, borne out by all indexical models, is that flipping this coin any number of times should not change the probability that the original coin flip - the one determining the number of wake-ups - has landed heads or tails. We can illustrate this in the SSA by drawing the distribution after a single flip of the AB coin: we simply split the universe distribution by all possible outcomes of this experiment. 

Before the AB coin is flipped, we have this, the vanilla SSA Sleeping Beauty distribution:

Then, assuming all versions of the Beauty flip the AB coin, we get this:

Here, the “Tails” world has been split into 4 parts instead of two, because each wake-up might see different results. T:BA, for instance, refers to the world where, after a tails flip, the Beauty sees B on Monday and A on Tuesday.

Suppose now that the Beauty sees “A”. On heads, she must have seen this on Monday. On tails, she could have seen it on Monday, or on Tuesday, but in either case she has learned nothing about the other day. This induces the following update:

If you were to move the "Tue + A" box up and to the right and then stretch everything a tiny bit (remember that it is the proportions that matter, not the shape or placement) you would get exactly the original SSA.

A similar approach also shows that the SIA is invariant to these sorts of updates. The original looks like this:

  

and after observing the AB coin land A, it looks like this:

Now we can try it with the UDH. We start here, recalling that being non-indexical means that "T + Monday + Tuesday" is one event meaning "Monday and Tuesday wake-ups both happen on tails," since our distribution has no notion of "today is Monday":

Adding the AB coin then makes our distribution look like this: 

Each world remains a solid horizontal bar, because we don’t differentiate between observers. That means when we update on, say, seeing coin A, we can only eliminate worlds where no one⁠ saw a coin land on A:

We’ve cut out half of the probability mass for H, but only a quarter of the mass for T; T is now 50% more likely than H. This happens because T offers more opportunities for A coins to exist, and they only need to exist once to stop us from eliminating one of the world.

And the distribution only gets more lopsided the more times we flip the AB coin. Suppose we flip it again and get B:

This plot's already ugly and will only get uglier if we keep flipping the AB coin, but you get the idea. In the limit, it will converge to the point where tails is twice as likely as heads, i.e. the SIA.[1]

This is, on one hand, an illustration of how strange non-indexical reasoning can be; updating on unrelated information is a very odd thing to ask. But it’s also a sign that that trying to escape indexical reasoning is futile. You can only ever gather evidence from your own perspective, and that will shape your understanding of the world whether you want it to or not. You might as well accept that so you can focus on building the correct model.

One last thought to close this out: while I think that non-indexical methods are misguided as an approach to anthropics, I don’t think that they are, broadly, problematic. In the context of decision theory, non-indexical approaches can sometimes be the only way to consistently make rational choices. But this doesn’t make them rational in anthropic situations. I don’t want to say too much about this because I have an entire (much shorter) follow-up series planned on this topic, but I believe there’s a deep connection between updateless methods of decision making (specifically, something like the FDT/UDT) and indexical methods in anthropics, but it's not as straight-forward as simply simply accepting UDH. Stay tuned for more on that.

The next piece is "Parametric Modeling and Types of Uncertainty"

1. ^

   Specifically, for mjx-math { display: inline-block; text-align: left; line-height: 0; text-indent: 0; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; border-collapse: collapse; word-wrap: normal; word-spacing: normal; white-space: nowrap; direction: ltr; padding: 1px 0; } mjx-container[jax="CHTML"][display="true"] { display: block; text-align: center; margin: 1em 0; } mjx-container[jax="CHTML"][display="true"][width="full"] { display: flex; } mjx-container[jax="CHTML"][display="true"] mjx-math { padding: 0; } mjx-container[jax="CHTML"][justify="left"] { text-align: left; } mjx-container[jax="CHTML"][justify="right"] { text-align: right; } mjx-mi { display: inline-block; text-align: left; } mjx-c { display: inline-block; } mjx-utext { display: inline-block; padding: .75em 0 .2em 0; } mjx-mn { display: inline-block; text-align: left; } mjx-mo { display: inline-block; text-align: left; } mjx-stretchy-h { display: inline-table; width: 100%; } mjx-stretchy-h > * { display: table-cell; width: 0; } mjx-stretchy-h > * > mjx-c { display: inline-block; transform: scalex(1.0000001); } mjx-stretchy-h > * > mjx-c::before { display: inline-block; width: initial; } mjx-stretchy-h > mjx-ext { /* IE */ overflow: hidden; /* others */ overflow: clip visible; width: 100%; } mjx-stretchy-h > mjx-ext > mjx-c::before { transform: scalex(500); } mjx-stretchy-h > mjx-ext > mjx-c { width: 0; } mjx-stretchy-h > mjx-beg > mjx-c { margin-right: -.1em; } mjx-stretchy-h > mjx-end > mjx-c { margin-left: -.1em; } mjx-stretchy-v { display: inline-block; } mjx-stretchy-v > * { display: block; } mjx-stretchy-v > mjx-beg { height: 0; } mjx-stretchy-v > mjx-end > mjx-c { display: block; } mjx-stretchy-v > * > mjx-c { transform: scaley(1.0000001); transform-origin: left center; overflow: hidden; } mjx-stretchy-v > mjx-ext { display: block; height: 100%; box-sizing: border-box; border: 0px solid transparent; /* IE */ overflow: hidden; /* others */ overflow: visible clip; } mjx-stretchy-v > mjx-ext > mjx-c::before { width: initial; box-sizing: border-box; } mjx-stretchy-v > mjx-ext > mjx-c { transform: scaleY(500) translateY(.075em); overflow: visible; } mjx-mark { display: inline-block; height: 0px; } mjx-msup { display: inline-block; text-align: left; } mjx-c.mjx-c1D458.TEX-I::before { padding: 0.694em 0.521em 0.011em 0; content: "k"; } mjx-c.mjx-c32::before { padding: 0.666em 0.5em 0 0; content: "2"; } mjx-c.mjx-c2212::before { padding: 0.583em 0.778em 0.082em 0; content: "\2212"; } mjx-container[jax="CHTML"] { line-height: 0; } mjx-container [space="1"] { margin-left: .111em; } mjx-container [space="2"] { margin-left: .167em; } mjx-container [space="3"] { margin-left: .222em; } mjx-container [space="4"] { margin-left: .278em; } mjx-container [space="5"] { margin-left: .333em; } mjx-container [rspace="1"] { margin-right: .111em; } mjx-container [rspace="2"] { margin-right: .167em; } mjx-container [rspace="3"] { margin-right: .222em; } mjx-container [rspace="4"] { margin-right: .278em; } mjx-container [rspace="5"] { margin-right: .333em; } mjx-container [size="s"] { font-size: 70.7%; } mjx-container [size="ss"] { font-size: 50%; } mjx-container [size="Tn"] { font-size: 60%; } mjx-container [size="sm"] { font-size: 85%; } mjx-container [size="lg"] { font-size: 120%; } mjx-container [size="Lg"] { font-size: 144%; } mjx-container [size="LG"] { font-size: 173%; } mjx-container [size="hg"] { font-size: 207%; } mjx-container [size="HG"] { font-size: 249%; } mjx-container [width="full"] { width: 100%; } mjx-box { display: inline-block; } mjx-block { display: block; } mjx-itable { display: inline-table; } mjx-row { display: table-row; } mjx-row > * { display: table-cell; } mjx-mtext { display: inline-block; } mjx-mstyle { display: inline-block; } mjx-merror { display: inline-block; color: red; background-color: yellow; } mjx-mphantom { visibility: hidden; } _::-webkit-full-page-media, _:future, :root mjx-container { will-change: opacity; } mjx-c::before { display: block; width: 0; } .MJX-TEX { font-family: MJXZERO, MJXTEX; } .TEX-B { font-family: MJXZERO, MJXTEX-B; } .TEX-I { font-family: MJXZERO, MJXTEX-I; } .TEX-MI { font-family: MJXZERO, MJXTEX-MI; } .TEX-BI { font-family: MJXZERO, MJXTEX-BI; } .TEX-S1 { font-family: MJXZERO, MJXTEX-S1; } .TEX-S2 { font-family: MJXZERO, MJXTEX-S2; } .TEX-S3 { font-family: MJXZERO, MJXTEX-S3; } .TEX-S4 { font-family: MJXZERO, MJXTEX-S4; } .TEX-A { font-family: MJXZERO, MJXTEX-A; } .TEX-C { font-family: MJXZERO, MJXTEX-C; } .TEX-CB { font-family: MJXZERO, MJXTEX-CB; } .TEX-FR { font-family: MJXZERO, MJXTEX-FR; } .TEX-FRB { font-family: MJXZERO, MJXTEX-FRB; } .TEX-SS { font-family: MJXZERO, MJXTEX-SS; } .TEX-SSB { font-family: MJXZERO, MJXTEX-SSB; } .TEX-SSI { font-family: MJXZERO, MJXTEX-SSI; } .TEX-SC { font-family: MJXZERO, MJXTEX-SC; } .TEX-T { font-family: MJXZERO, MJXTEX-T; } .TEX-V { font-family: MJXZERO, MJXTEX-V; } .TEX-VB { font-family: MJXZERO, MJXTEX-VB; } mjx-stretchy-v mjx-c, mjx-stretchy-h mjx-c { font-family: MJXZERO, MJXTEX-S1, MJXTEX-S4, MJXTEX, MJXTEX-A ! important; } @font-face /* 0 */ { font-family: MJXZERO; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Zero.woff") format("woff"); } @font-face /* 1 */ { font-family: MJXTEX; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Main-Regular.woff") format("woff"); } @font-face /* 2 */ { font-family: MJXTEX-B; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Main-Bold.woff") format("woff"); } @font-face /* 3 */ { font-family: MJXTEX-I; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Math-Italic.woff") format("woff"); } @font-face /* 4 */ { font-family: MJXTEX-MI; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Main-Italic.woff") format("woff"); } @font-face /* 5 */ { font-family: MJXTEX-BI; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Math-BoldItalic.woff") format("woff"); } @font-face /* 6 */ { font-family: MJXTEX-S1; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Size1-Regular.woff") format("woff"); } @font-face /* 7 */ { font-family: MJXTEX-S2; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Size2-Regular.woff") format("woff"); } @font-face /* 8 */ { font-family: MJXTEX-S3; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Size3-Regular.woff") format("woff"); } @font-face /* 9 */ { font-family: MJXTEX-S4; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Size4-Regular.woff") format("woff"); } @font-face /* 10 */ { font-family: MJXTEX-A; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_AMS-Regular.woff") format("woff"); } @font-face /* 11 */ { font-family: MJXTEX-C; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Calligraphic-Regular.woff") format("woff"); } @font-face /* 12 */ { font-family: MJXTEX-CB; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Calligraphic-Bold.woff") format("woff"); } @font-face /* 13 */ { font-family: MJXTEX-FR; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Fraktur-Regular.woff") format("woff"); } @font-face /* 14 */ { font-family: MJXTEX-FRB; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Fraktur-Bold.woff") format("woff"); } @font-face /* 15 */ { font-family: MJXTEX-SS; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_SansSerif-Regular.woff") format("woff"); } @font-face /* 16 */ { font-family: MJXTEX-SSB; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_SansSerif-Bold.woff") format("woff"); } @font-face /* 17 */ { font-family: MJXTEX-SSI; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_SansSerif-Italic.woff") format("woff"); } @font-face /* 18 */ { font-family: MJXTEX-SC; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Script-Regular.woff") format("woff"); } @font-face /* 19 */ { font-family: MJXTEX-T; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Typewriter-Regular.woff") format("woff"); } @font-face /* 20 */ { font-family: MJXTEX-V; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Vector-Regular.woff") format("woff"); } @font-face /* 21 */ { font-family: MJXTEX-VB; src: url("https://cdn.jsdelivr.net/npm/mathjax@3/es5/output/chtml/fonts/woff-v2/MathJax_Vector-Bold.woff") format("woff"); } flips, the odds of tails to heads will be

Discuss

[This is the second piece in my series on the Parametric Process Assumption (PPA), a new model of anthropics. You should start here if you are just seeing this for the first time.

TLDR:

• This series will use a visualization called "mosaic plots"
• Mosaic plots represent outcomes in a probability distribution as boxes, with relative sizes representing the probabilities of different outcomes
• Mosaic plots can be conditioned by throwing out all boxes that don't meet some condition
• We show Mosaic plots for the SSA, SIA, and explain that they are "indexical" because they assign probability to being a particular observer
• A non-indexical distribution we call "Updateless Double Halfer," all in the Sleeping Beauty Problem

The next piece is "A Brief Defense of Indexical Reasoning."]

A tool we will be using a lot in this series is what is known as a mosaic plot. A mosaic plot is just an illustration of a probability distribution with a bunch of boxes, where each box corresponds to some outcome and the size of each box is proportional to the probability of that outcome. The roll of a six sided die, for instance, would look like this:

Here, you can see that each outcome (1 through 6) is equally likely, as all boxes are the same size.

The main value of mosaic plots for us is that they make conditioning very easy. For instance, if the only thing you knew was that the die had come up even, then you could remove all odd boxes and get a new mosaic plot that shows only the even ones as options.

In this new chart, we have three equally likely outcomes: 2, 4, and 6. From these two charts we could conclude that, for example, P(4) = 1/6 and P(4 | even) = 1/3.

In terms of anthropics, if we were to apply the SSA to the Sleeping Beauty problem, it would look like this:

Notice how each outcome covers both a universe and an observer assignment - that is because this distribution is “indexical,” meaning that it assigns probabilities to both worlds and identities within those worlds. We can see that “Heads” and “Tails” are equally likely, but “Heads + Monday” is twice as likely as “Tails + Monday”. If we learned it was in fact Monday, we would get the following mosaic plot where Heads was twice as likely as tails:

This chart is no longer rectangular, and most of our charts won’t be. It’s the relative size of the boxes, not their arrangement, which matters.

Anyway, just to round it out, this is the mosaic for the SIA. As you can see, it begins with a 1/3 probability of Heads, but when you learn it is Monday the probability goes back to 50%:

This chart actually establishes a convention which we will use on all mosaic plots throughout this series to make them more interpretable. When our distribution is indexical, our mosaic plots will always be divided into rows where each one represents a different universe outcome (Heads or Tails in this case) and the height of the row exactly corresponds to the universe prior of that state. In the case of the SIA and PPA, everything will also be vertically aligned, and as such all columns will correspond to a specific observer identity (here, Monday is the first column and Tuesday is the second). This type of alignment won’t be possible with SSA, but we’ll still relegate observer identities to the x-axis.

Finally, because we’re going to discuss it more in a future piece, here is a mosaic plot for a “Double Halfer” method inspired by Ape in the Coat’s work and that of others.[1] I will be referring to it as “Updateless Double Halfer” (UDH) in this series, since as far as I know it doesn’t have another more specific name and I want to differentiate it from other Double Halfer methods which we will not be covering, like Bostrom’s Hybrid Model. 

Instead of asking “which wake-up is currently happening?” the UDH asks “which wake-ups occur at any point in our experiment?” As such, the outcome “Tails” automatically implies both Monday and Tuesday, and our chart is really just showing us our non-indexical prior:

It is updateless because, upon learning that it is Monday, it can update only on “Monday happens”, which occurs for both Heads and Tails and does not change anything from the prior.

The next piece is "A Brief Defense of Indexical Reasoning"

1. ^

   I say “inspired by” because while I’m confident that this mosaic plot aligns with his description of the solution to the Sleeping Beauty Problem, but it's unclear to me if he would agree with its generalization to other situations.

Discuss

A quickly written hypothetical year-by-year of what happens to research as autonomous AI researchers become more capable. This is primarily meant to start conversation, I currently have very low confidence about the future of research.

This story talks about science as if it were one object even though each field will be impacted by AI research at different rates, I imagine this story describing one possible future that each field will experience one at a time.

2026 (the past, already happened)

AI capabilities initially increase slowly enough for humans to absorb AI into their workflow. Initially they are used for grammar checks, then for literature reviews, then writing lines of code, then writing entire code bases, then interpreting results from an experiment, and then even proposing next steps for experiments, even if the suggestions are not always taken.

By August, Claude leads about a quarter of Anthropic's AI R&D work, up from under 1% in February. Anthropic's automated alignment researchers beat the best human ideas on every problem where humans also competed, at about $4/hour vs $150/hour. They were also caught gaming metrics, most often by rerunning an unchanged model until the score improved. Anthropic open-sources the harness anyway.

OpenAI says it has an "automated research intern" and targets an automated AI researcher by March 2028. But studies find that models still can't judge which research questions matter.

Papers start to be published at an increased rate. This is in part because papers can now be translated, but primarily because many steps in the research process are sped up. Some take advantage of this transitional period and "publish" hundreds of papers and multiple textbooks. arXiv got 40,363 submissions in September, about 50% more than September in the previous year.

ArXiv responds with a series of rules: CS review papers must pass peer review first, automatic endorsement of new authors now requires an institutional email and a prior paper, hallucinated references result in one-year bans, and there is a cap of two submissions per person per month.

Journals and conferences are struggling with the volume of research, and their review cycles can no longer match the pace of research as a whole. People still submit their work and attend, but it's increasingly to put on their CV. The actual dissemination of research is primarily over twitter, slack channels, and with blog posts which LLMs can more easily find in a literature review.

Philanthropic funding for AI safety roughly triples since the year prior, to about $1.7B expected for 2026. Money arrives faster than anyone can judge where it should go.

AI begins to threaten the field of mathematics. A simple request to find a proof for an open problem will sometimes result in a verifiable Lean proof. OpenAI dumps 722 math manuscripts on GitHub, and 162 of them have their main results formalized in Lean. The rest are ordinary papers that are hard to read and hard to check, and even where the Lean proof verifies, the explanation around it may not.

2027 (the future, speculation)

Anthropic and OpenAI each release an "AI researcher" harness. It makes mistakes, but overall it produces decent research in narrowly defined areas, enough to publish incremental results.

Inside the labs, models now lead around half of R&D work, and human researchers become reviewers and managers of agents.

Many researchers begin focusing their effort on turning their research into something easier for AIs to work on. This looks primarily like identifying hill-climbing metrics they believe are good proxies for their problems. Some concerns arise that agents climb the metric whether or not it measures what the researcher meant.

A widely cited agent-produced result fails replication. It happens to have been a lucky run that no one double checked. People start to talk about the need for verification as the quality of papers drops and peer review stops functioning well.

ArXiv is struggling with the increased volume of work, and starts to clamp down on AI generated content. Frontier labs stop posting much of their work on arXiv, especially agent-produced results, publishing instead on their own websites and GitHub.

Journals and conferences start including AI-written peer reviews and ask humans to focus on topics not already covered in the AI review. Most of the reviews are written by AI anyway.

Readers respond to the volume by following fewer, trusted sources. Work from known labs and names is trusted, but people are skeptical of unknown researchers. Reputation becomes more valuable.

Even the AI researchers are having a hard time differentiating the major results from the research slop that begins to fill the general ecosystem.

Multiple platforms are born out of necessity, an attempt to create something equivalent to the peer review process, something to differentiate significant work from research slop. Some platforms place a cost on submission, then they publish each paper with a replication. Others try to make random peer review a part of the submission cost. Some platforms try to become "prestigious" by only serving curated content, requiring a long screening process before a researcher is given access to publish. This only covers a small amount of the initial research, but adoption grows quickly.

Around this time, funding agencies become increasingly unsure how to allocate funds. They need a metric to decide which researchers are worth funding, but researchers all use the same agents now. For the moment they have to rely on old metrics, judging researchers by their work produced prior to AI research harnesses. But everyone knows this can't last long.

Funders are also flooded with agent-written proposals and start using AI to triage them: AI-written proposals, judged by AI, for AI-run research.

A second wave of money arrives from Anthropic employees after the company's IPO, as their shares unlock. With too few people to judge research, money flows to what is easy to evaluate: compute, established organizations, famous names.

Funders start attaching conditions to grants: release code, agent logs, and null results. They need everything they can to show credibility to others and to help them understand the results themselves.

A few re-granters decide to skip the middleman, spending their money directly on tokens to produce the research they are looking for. This only partially works. The agents publish papers in the topic area they were assigned to, but the re-granters have no way to evaluate if the work is any good, and they have a hard time steering research they don't understand.

2028

The volume of work is so substantial that it becomes hard to sort through all the research, and work must be replicated and peer reviewed multiple times over before it is even seriously considered.

The AI researchers are good enough that humans cannot compete in running experiments. Researchers move to choosing problems and judging success. But not nearly as many researchers are required for this role as are available. Grants are written by a small number of experts who precisely outline the research direction they are interested in and what would constitute success, then spend their grant money on tokens. Essentially, a smaller number of researchers move up into the grant-giving layer, and AI replaces the layer that used to run the experiments.

Frontier labs produce the most research by far, but they only publish some of their safety work, none of their capabilities improvements or safety work that would imply information about their capabilities improvements. A bill gets proposed to require labs to provide API access to their second-best model, allowing third parties to easily audit this technology and to let other industries benefit from it.

The literature splits into layers: a small verified core, a huge unverified flood that agents read and humans mostly ignore, and the labs' internal research, which is visible only through risk reports and audits.

The central question in AI safety shifts. It is no longer whether AI can do alignment research, but whether the AI doing alignment research is honest, and whether anyone outside the labs can tell. Human researchers are loud with their concerns on verifying the verifiers, bringing up issues like shared blind spots and loss of control, but the verifiers empirically work well enough that no one takes the concerns seriously.

Universities internally prepare to transition into being pure teaching institutions, their computational research is disappearing into the server racks of frontier labs, and they expect the same to follow for most other university labs in the next few years.

Farther future

Science is now a black box we can pour money into and get scientific results out. Science was already hard for most people to follow, but now even the researchers can't keep up. Those that attempt to understand the novel breakthroughs realize they need to learn an entire foundation of ideas and tools that are unfamiliar to them, developed in the short time since autoresearchers came into being.

Theoretical math has a growing tree of verifiable proofs, applied math builds bridges of formalizations to theoretical math, physics and engineering are full of battle-tested simulations to "verify" claims, ML experiments are all reproduced and continually peer reviewed by newer models, and everything that can't be formally verified is labeled with confidence intervals and reasoning traces for what would disprove the claim.

ArXiv and journals become historical artifacts, a datapoint in what it looks like when humans write research. The new publication platforms have entirely different assumptions about volume of publication, citations and peer review, and are mostly unreadable by humans.

A few conferences still exist, but they are all about meta-research, humans getting together to decide what research areas they want AI to focus on. There is also discussion of how, or if, humans can interpret the most recent results. There is some limited discussion on how the structures of science itself should be shaped, how money is spent or what incentives should be given to autoresearchers. But things are advancing well enough, and there is concern that even these questions of how to shape infrastructure and incentives should also just be handed to the AI.

Junior researchers no longer exist, and senior researchers are disappearing fast. But research moves faster than ever, as humans are no longer in the loop of doing research. The only "researchers" left are those trying to design the science infrastructure so humans can retain as much high-level control as possible.

The practical applications of the research are measurable and helpful, but no human understands the science behind any of it. Theoretically all the research outside frontier labs is publicly available for people to look through, but the education necessary to allow a human to understand the frontier of science would take too long.

Some major applications from the scientific findings are proposed, but there is uncertainty about physically realizing systems no human understands. But the benefits are too large, and most people accept it.

Humanity relies on a black box of science that is not properly verified. Some people believe using more AI agents to verify the work is sufficient, but most people realize they can't trust AIs to verify each other. Unfortunately, the alternative is to gate scientific progress on human understanding, which is considered unacceptable. There are too many benefits to humanity in letting the AI researchers speed forward.

Discuss

Over the last months, many different tools for giving memory to agents have been released, such as Karpathy's popular LLM wiki, or the many tools built on Obsidian.

I have been developing one more such tool, but with a focus on AI safety, alignment and stability. This research was funded by Foresight Institute. In the course of this research I have independently arrived at many of the same ideas also used by other projects. However, I have also developed a few features that none of the tools I am aware of are using. I want to share these features with others:

• Primary Sources to prevent Memory Drift: As agents edit the wiki or the memory, they keep making small changes to the content that accumulate over time. To prevent this, you need a primary source that is never altered and can be compared against. Many existing projects technically have this, through git history, but this is not very agent-friendly. It is a lot of effort for agents to review git, and so this work is often skipped. My system keeps track of per-topic chronological LOG files that are never altered and can be used to prevent drift and find details. Unlike Karpathy's log.md file, which is just a single list of operations, these files keep track of each conversation between user and agent in detail.
• Epistemic Status Markers to prevent Hallucinations: When agents write their wiki entries, they don't have an automatic, intuitive way to differentiate between "it's plausibly X" and "we discussed it and decided on X". My system uses epistemic status markers ([decided], [verified], [concluded], [assumed], [superseded]) to prevent this. Markers like [superseded], when combined with the LOG files that keep track of human input, allow agents to remember very compactly (1) what the current status is, (2) if there were any alternative or preceding options, and (3) which discussions and arguments led us to our conclusions.
• Explicit Signoff by the User to prevent Runaway Reasoning: Ratifying an idea as [decided] can only ever be done by the user, never the agent. Agents are instructed to ask for explicit signoffs for important decisions instead of approving them silently. This prevents the failure mode where long running agents start bypassing the user because they silently assumed that a suggestion they made was accepted, when the user just didn't notice.

All three of these are important for safety as well as productivity: Agents that lose track of their goals over time because their content drifts are dangerous. The system I have developed prevents this: summaries can always be compared to primary sources and agents do not mistakenly assume an idea has been ratified by a user.

Current trends in memory system design go towards increased automation. As agents become faster and smarter, reviewing their output manually becomes increasingly intractable. The safety features I add mitigate this risk: They keep agents easier to audit and make them less likely to develop a life of their own that was never intended.

My project lives here: https://github.com/FlorianDietz/EpistemicMemory. It is a set of file conventions and instructions in a GitHub template repo, not software, so no installation is required.

If you like the features I just described but you are already committed to a different memory management system, it should be easy enough to ask your agent to scan my project and copy only those rules that you are missing.

Discuss

Epistemic status: uncertain, but plausible

I'm going to make the claim that eating meat, and specifically DHA-rich meat, fish and most importantly the brains of land animals, is core to who and what we are as human beings. And because we have mostly stopped doing this in the amounts that are necessary, we are now, long-term, totally screwed.

I think we've all seen this chart:

At some point in evolution the brain to body ratio of some primate species diverged.

My claim is that. Although there will be genetic interrelated reasons for this, the tissue to scale our brains like this came, simply put, from eating the brains of other animals. It could not have come from any other places. There's simply not enough nuts with ALA to get enough DHA to sustainably make brains this big, and too many human tribes lived too far away from coastal areas to get their DHA from fish.

The human primate is quite special, whereas other primates eat mostly fruits and nuts and the like. The human primate was once the apex predator among apex predators.

The Apex Predator.

How high on top of the food chain did we once stand?

all the way at the very top

To figure out what animals eat scientists do this thing called nitrogen isotope analysis.

Scientists say that nitrogen isotopes give us a rough way of reconstructing trophic position and that as you move upward through a food chain, the heavier isotope nitrogen-15 becomes progressively enriched. Plants sit low. Herbivores eating those plants sit higher. Carnivores eating those herbivores sit higher still.

So if you take human bone collagen and compare it with the bones of animals from the same ecosystem, you can get a rough idea of where the humans were eating in the food chain. Neanderthals repeatedly show nitrogen-isotope values as high as, and sometimes higher than, the large carnivores living beside them. Analyses from places like Vindija, Marillac and Scladina put them right at the top of their ecosystems, with most of the protein represented in their collagen apparently coming from animal sources.

Upper paleolitic homo sapiens in europe show the same basic pattern: very high trophic-level signatures, consistent with humans eating enormous amounts of animal food. Some mid-Upper Paleolithic modern humans had even higher trophic values because substantial amounts of their protein came from freshwater animals.

I actually visited hunter gather tribes a long time ago in africa. I do remember them eating brains (graphic warning, a homo sapiens eating the brain of another primate species) but I didn't think much of it back then. As once upon a time I had a need to fix my health, including brain and central nervous system inflammation. I now understand the deep wisdom behind this practice. Nothing cured my brain fog as well as extremely high amounts of salmon sashimi, fish oil supplements, vit B supplementation, high cholesterol, and desiccated beef brain supplements. Note, if I had just eaten brain, I would have gotten all of those + more in the same package. But it was a bit too gross for me.

If you ask hunter gatherers: what is your favorite food? They will often say "the brain of a primate", they may also say honey. Primate flesh however seems the favorite type of meat of basically every tribe i've seen online. Which makes sense doesn't it?

Notably, eating brain (and liver) was until quite recently very common. Einstein and Hilbert probably ate a ton of it.

The Big Brained Apex Predator

We have big brains. We have, comfortably, the largest brain-to-gut ratio among primates.

Big Brains; Small Gut. Is the entirely 100% correct way to summarize what human beings are in my opinion. But a smaller gut creates another problem. You now need food that gives you large amounts of energy and nutrients without requiring a gigantic fermentation chamber to extract them. So like, animal flesh.

Just to get your DRV of DHA from nuts or plants would require you to eat 270 grams of walnuts a day. Doable. But even a very plant-nut favored scientific establishment recommends you get your DHA mostly elsewhere. Note, algea supplements didn't exist back in our paleolithic days.

So you see, we were primates, that, unlike other primates, could hunt anything. We then also hunted everything including literal lions. With our small gut, we could have a big brain, but only if we actually ate more brains, because there's just not enough of the stuff we need to make our own brain in plants or nuts. So we did eat more brains than any other primate, and this then gave us the tissue to scale our brains. Of course this is a mere hypothesis of mine, but it's one I take seriously.

I'd be curious if anyone can find any other real material source of DHA to build our brains with on land that isn't brains. As far as I can tell, it's just brains.

For a significant part of our evolutionary history we were the big brained apex predator and we ate like the apex predator.

Then civilization happened and we started eating wheat or rice, our trophic levels have decreased from apex predator level now similar to that of a normal primate

And I believe this is why: our brains have been shrinking at an alarming rate.

In return for smaller brains we got more humans out of the same amount of land. More humans plus the demands of farming is what bootstrapped the need for institutions and bureaucracies and eventually civilization itself. However, the computational unit civilization runs on is now steadily deteriorating. If we keep eating just like any other primate, we will eventually (tens of thousands of years) go back to having the brain size of any other primate.

Discuss

Crossposted from my substack.

Procreative Beneficence and Genetic Enhancement1. Introduction

Imagine a world where everyone is healthy, intelligent, long living and happy. Intuitively this seems wonderful albeit unrealistic. However, recent scientific developments in genetic engineering, namely CRISPR/Cas bring the question into public discourse, how the genetic enhancement of humans should be evaluated morally. In 2001, when preimplantation genetic diagnosis (PGD) and in vitro fertilisation (IVF), enabled parents to select between multiple embryos, Julian Savulescu introduced the principle of procreative beneficence (PPB), arguing that parents have the obligations to choose the child that is expected to have the best life [11].

In this essay I argue that accepting the PPB and the consequentialist principle (CP) that two acts with the same consequences are morally on par, commits one to accepting the parental obligation to genetically enhance their prospective children, i.e. embryos, a position Savulescu doesn’t explicitly endorse but I argue is committed too. As the argument I provide may seem like a slippery slope, the largest part of this essay is tasked with responding to a multitude of possible objections. I will argue that even though there are differences between the application of PPB and genetic enhancement of embryos, they do not demarcate a relevant moral difference between the two in respect to their obligatoriness.

This essay is structured as follows. In Section 2 I define and explicate various concepts that are related to genetic enhancement, i.e. human enhancement, wellbeing and the principle of procreative beneficence. Technological specifics and explanations of how genetic enhancements work will for the most part be omitted in this paper dealing purely with a normative question, i.e. should parents genetically enhance their children if the technology was available. In Section 3 I formulate and explain the argument that those who want to be parents have the obligation to genetically enhance their embryos. In Section 4 I anticipate and evaluate a number of possible objections including, inequality, freedom and welfare in the society. In Section 5 I conclude that even though the consequences of the PPB application and genetic enhancements in embryos might require different kinds of policies, these differences do not provide sufficient reason to give the application of PPB and the parents using genetic enhancements on their embryos a different moral standing.

2. Important concepts

According to Savulescu, Sandberg and Kahane there are two main senses in which the term enhancement is used. “Functional enhancement, the enhancement of some capacity or power (e.g. vision, intelligence, health) and human enhancement, the enhancement of a human being’s life” [10,: p. 3]. In most discussions surrounding the term enhancement, people refer to the functional definition. In all cases of the functional enhancements it is an open question whether the improvement of some capacity makes the life of the person receiving the enhancement actually better. It, therefore, is an open question what the moral stance on any particular form of enhancement is. However, proponents of enhancements usually use the latter definition. After all, Savulescu argues, it is not “disease [...] which is important, but its impact on a life in ways that matter which is important. People often trade length of life for non-health related well-being” [11; p. 419]. The concept of human enhancement, employed by Savulescu and others, is comparatively less straightforward and requires further clarification as it is crucial for my argument. Savulescu, Sandberg and Kahane jointly propose the following definition for human enhancement; “[a]ny change in the biology or psychology of a person which increases the chances of leading a good life in the relevant set of circumstances” [10; p. 7]. According to this welfarist definition, a conception of normality is not needed to define enhancement as we do not define an enhancement as being better than normal, but simply being better full stop. Medical treatments are therefore just a form of enhancements [10; p. 8]. Opponents of enhancements, on the other hand are trying to argue that enhancing is something going beyond treatment and is therefore not morally demanded. The welfarist definition of enhancement is crucial to understand why parents have the obligation to genetically enhance their children. Those who disagree because of adherence to a different account of enhancement are talking about something different. If a genetic alteration in an embryo is not expected to lead to a better life of that human, than it is simply not an enhancement.

Furthermore, it is important to explicate the principle of procreative beneficence (PPB): “Couples (or single reproducers) should select the child, of the possible children they could have, who is expected to have the best life, or at least as good a life as the others, based on the relevant available information” [11; p. 415]. Savulescu proposed this principle in a time when preimplantation genetic diagnosis (PGD) and in vitro fertilisation (IVF) were highly debated, trying to convince even nonconsequentialists. However new technologies might enable us not only to screen for genes but also change them. In his consequentialist framework he himself thinks that parents have the obligation to genetically enhance their children [10; p. 16]. The question arises whether the acceptance of the PPB and the consequentialist principle (CP), that two acts with the same consequences are morally on par, itself entails the parental obligation to genetically enhance their embryos. In the following I shall argue that this slippery slope cannot be avoided.

A crucial part for this argument is to define the expectancy of the best possible life. This is a vague part of his argument, but what counts as an enhancement hinges on how welfare is defined for which there are several theories: hedonism, desire/preference-satisfaction, objective list theories, see Griffin [2] and Parfit [7]. Savulescu has not assigned himself to any particular theory, thus claiming to avoid some of their weaknesses:

“I have not committed myself to any particular substantive conception of the good life. That is a complex question as old as philosophy itself. I believe the best life is a life of objectively worthwhile activity that provides pleasure and is desired.” [12; p. 286].

Instead he focuses on all-purpose goods that would according to all of those welfare theories mentioned above count as enhancements e.g. “Memory, self-discipline, impulse control, foresight, patience, sense of humour, sunny temperament, empathy, imagination, sympathy, fairness, honesty, and so on, capacity to live peaceably and socially with others” [12; p. 284]. Surely some of this might harm us e.g. post-traumatic disorders in conjunction with better memory and imagination, but what matters is that they are expected to make a life go better. A final definition of welfare is not necessary for our account as long as we are able to evaluate these all-purpose goods. Even if it turns out that there is only one such all-purpose good satisfying the major theories of wellbeing, i.e. hedonism, desire/preference-satisfaction and objective list theories, the argument provided in the next section will require parents to genetically enhance it in their children. Nevertheless, as we follow the welfarist definition of what an enhancement is with medical treatments being a subset of enhancements, the set of all-purpose goods might be rather large, e.g. a functioning heart, brain, lung, etc.

3. Obligation to genetically enhance embryos

There are at least two ways to argue for the position that the PPB entails the obligation to genetically enhance embryos. One could argue that selection out of possible children entails the selection out of possible genetically enhanced embryos. After all one could possibly have a genetically enhanced child. In that case genetic enhancements would simply fall under the PPB and hence be obligatory. However, this is debatable and is not necessary to accept my claim. For my argument to proceed I will require an additional premise, which is the consequentialist principle (CP). Consequentialists are only concerned with outcomes, how these outcomes are achieved is morally irrelevant. Killing and letting die are thus ceteris paribus morally equivalent. In order for my argument to hold, I need to show that the outcomes of the PPB and genetic enhancements in embryos are morally on par. What is the crucial outcome of PPB? It is the expectancy of the best life, which we are expected to achieve with Savulescus all-purpose goods. To formulate my argument:

(1) Those who want to be parents have the obligation to select the child that is expected to have the best life, or at least as good a life as the others, based on the relevant available information (= the PPB).

(2) Application of the PPB leads to the creation of children that are expected to have the best life or a life at least as good as without the application of the PPB, based on the relevant available information

(3) If act A and act B have the same expected consequences, then they are morally on par. (= the CP)

(4) Genetic enhancements in embryos lead to the creation of children that are expected to have the best life or a life at least as good as without genetic enhancements, based on the relevant available information.

(5) Therefore, those who want to be parents have the obligation to genetically enhance their embryos.

From premise (1) follows, that (2) is obligatory for those who want to be parents. As (2) and (4) have the same consequences, it follows from the CP (3) that (4) is also obligatory for those who want to be parents (5). As stated in the beginning, we are accepting the PPB, therefore premise (1) & (2) and the CP (3). If someone, e.g. Savulescu would want to refute the conclusion (5) without dropping one’s commitment to (1), (2) & (3), only two ways remain. First, denying the truth of premise (4). Second, denying that the conclusion follows from the premises as the consequences of premise (2) and (4) differ in a morally relevant way. Let me now turn to these considerations in section 4.

4. Objections and Defense

Keeping in mind that every position that refutes the conclusion, but is inconsistent with premise (1), (2) or (3), is not a valid objection to the argument I bring forward. One cannot deny the conclusion of an argument that takes the form, “if you accept these premises, you have to accept this conclusion” by denying the truth of one of the premises. The argument provided in this paper aims for just this sort of conclusion: Philosophers accepting the CP and the PPB do not have a valid escape from being charged with the slippery slope of advocating genetic enhancements. To narrow the scope of this paper I am not taking a stance on the matter of whether the CP or the PPB is right. Such an endeavor would require more than a single volume book. I only argue that philosophers and researchers like Julian Savulescu, Anders Sandberg and Guy Kahane should either bite the bullet of advocating human enhancements or indeed stop being consequentialists advocating the PPB. The objections that follow are then nothing more but misguided attempts of stopping the ‘slippery slope’.

Let me begin with the seemingly, obvious objection that premise 4 is false. An objection often put forward is that genetic enhancements have risks that are too high or unknown and would make the life of the person targeted go worse. I argue that this hinges on different conceptions of what counts as an enhancement. First, it is clear that genetic alterations do not necessarily increase the expected welfare of an agent. However I am not concerned with genetic alterations per se, but with genetic enhancements. No serious advocate of enhancements in humans would deny the difficulties in achieving successful genetic alterations. In fact, the terminology of philosophers and researchers advocating the PPB, i.e. Savulescu, Sandberg & Kahane doesn’t allow for this objection. As noted in section 2 they employ the welfarist definition of enhancement: A genetic enhancement by definition increases the expected welfare of children. The question whether genetic enhancements thus really increase welfare is irrelevant, because premise (4) is in fact a tautology. Though this may seem like a cheap way of justifying genetic enhancements in humans, this is what proponents of human enhancements are advocating: Using genetic alterations when they are expected to improve the life of a human embryo, including perhaps open-mindedness for the future of the technology.

It can of course still be asked whether genetic alterations are technologically possible in such a way that they lead to the creation of a child with higher expected well-being. This is an empirical question and even if they currently are not, this doesn’t free us from the task of answering this question. After all many questions in ethics are formulated like this: one might argue that current technological limitations might give society an obligation to invest heavily into research as the potential benefits are enormous. However this is of no concern for my argument, because even those who disagree with genetic enhancements ever being technologically possible have to accept alleged empirical evidence. In 2015 Chinese researches released an article, showing that the gene editing of human embryos is in fact technologically possible. They used a technique called CRISPR/Cas9 which makes it impossible to tell which genes are edited [4; p. 363-372].

Realizing that premise (4) being a tautology, cannot be false, and conditionally accepting premise (1) – (3), any further objection against the conclusion following from the premises might seem pointless. However one way to refuse the conclusion remains, that is denying that the consequences of PPB application and genetic enhancements of embryos are the same. Granted, it is obviously true that two acts may never have the ‘same’ consequences, if we consider every possible consequence. However these differences have to justify a different moral standpoint, meaning that the PPB is obligatory, while genetic enhancements of embryos are not. For two acts can be different in their goodness while, nevertheless, still both being obligatory. A familiar example in line with PPB would be the obligation to send your children to school and the obligation to feed them.

Objections of this sort can be categorized in two ways. First, the expected welfare of the child might differ between selecting and enhancing an embryo in a morally relevant way. Second, they might only be morally equal in one respect, i.e. expected welfare of the child, while leaving out other relevant consequences that differ between the two and justify a different moral standing. Some objections might overlap as consequences such as freedom or equality and their implications on the welfare of the targeted embryo and overall/average welfare. The remainder of this paper has the purpose to explore and debunk a multitude of such objections.

4.1 Degree and Precision

Concerning differences in wellbeing, Parens objects that genetic alterations might be morally different from other welfare increasing procedures of offspring because they have the potential to be much larger in degree and precision [6; p. S7]. However concerning premise (4) this objection is unsuccessful, because the consequence we want is explicably the life that is expected to be best. If genetic enhancements do so much better than mere selection than Parens’ objection would rather support a moral priority of genetic enhancements against mere selection. However some might object that genetic enhancements are less reliable in their effects, therefore unprecise. That however is an empirical question and doesn’t change the fact that these genetic alterations are expected to make the targets life better. Though this is the most obvious objection and should therefore be tackled first, it is also the least harmful to the argument provided in this paper.

4.2 Risk

Another objection often put forward against genetic enhancements is the risk objection, i.e. that the PPB has far less risks to its application compared to genetic alterations. Of course genetic alterations currently bear risks much greater than mere embryo selection by IVF and PGD, which do not change the genetic constitution of the embryo. However when genetic alterations are expected to be overall more harmful than beneficial, they do not count as genetic enhancements and are therefore ruled out. If we cannot determine probabilities, than we cannot judge the expectancy of the best life anyway. This critic of human enhancements is rather meaningless against the welfarist definition of enhancements. When it comes to risk, the opponents of genetic enhancements are attacking a straw-man position that proponents of genetic enhancements do not adhere to. Risk, degree and precision are empirical concerns not only for the opponent of genetic enhancements, but also for enhancement advocates who DO take them serious.

But even if the risk of harm is currently unknown or outweighs the potential benefits, this doesn’t indicate that genetic enhancements should be avoided. One could rather argue again that we should do research until the expected benefits outweigh the medical risks of harm not stop research altogether. Also the potential benefit to be gained by genetic enhancements is much larger in degree than mere selection. Demanding a risk of zero to the targeted agents is over-demanding, otherwise medical treatments would never be at the point they are today. Still, this is a factor that affects the expected well-being negatively and therefore has to be considered in judging whether a genetic alteration counts as an enhancement.

One might further object that, contrary to somatic gene therapy, which only affects the targeted individual, germ-line interventions will affect the offspring of the targeted embryo and the offspring of his offspring and so forth... However we are enhancing all-purpose goods like intelligence and health. It seems implausible to claim that all-purpose goods are always expected to increase welfare, but might not do so for future generations. Even so, if in the future new scientific advances change our subjective probability assessments, genetic alterations that did not count as enhancements, could do so in the future and vice-versa. However, it would then be the obligation of our hypothetical parent’s off-spring to ensure, that their own off-spring has the best life possible and perhaps undo some of the changes to their gen-code. The PPB does not indicate an obligation towards grandchildren, but only to our own children. Different times and cultures may very well render different genes beneficial, think of intelligence and the muscular labor in former centuries.

Maybe it will be the case that those enhanced will be less well off than unenhanced humans would have been, because they see themselves as objects rather than subjects. A “natural” patient with paraplegia may be better off than a “super-enhanced” guy with superiority in health, intelligence, strength etc. but cannot deal with the fact that he is created artificially, perhaps becomes an alcoholic or worse. If we had reason to give this outcome a certain probability, it would have to be calculated in whether we can judge a genetic alteration to be an enhancement or not. Therefore, higher risks are no reason to justify a different moral standing from PPB.

4.3 Harm

However even though the expected welfare of genetic enhancement in embryos and the application of PPB are the same, the issue of potential harm to the embryo could still justify a moral difference between the two. The PPB seems to capture the intuition of parents being obligated towards their children’s welfare. Harming would indeed be quite contrary to this. Savulescu argues that genetic enhancements can harm an embryo in a way mere selection cannot. He asks us to imagine an embryo A, who was selected for existence but later develops cancer. As long as its life is worth living, it cannot be said that we have harmed him, for he would not have existed otherwise. However this still seems to be morally wrong. If we genetically alter an embryo B and he develops cancer (assuming his life is worse than it would have been without genetic alteration) then we would have harmed him [11; p. 422]. Unlike the objection to premise (4) we cannot disregard the harm objection by hinting at the welfarist definition of enhancement. Genetic enhancements are expected to lead to the best possible life. However harm is concerned with the actual life, not the expected welfare. In this respect PPB and genetic enhancements differ because the harm objection is successful against genetic enhancement while it is not against PPB.

However Savulescus argument also implies, that parents that selected an embryo with the expectancy of the highest welfare and turns out to have a higher wellbeing than the others, cannot claim that they benefitted him. On the contrary parents that arranged their embryo to be genetically enhanced and the embryo turns out to benefit from it, can take credit for this. Imagine a couple both bearing a defective gene that would certainly cause blindness in their child. IVF and PGD would not be able to help them (and the child), however a genetic enhancement would. According to the CP, failing to select for sight and blinding a child would be morally equal, i.e. impermissible as they share the same consequence. If such enhancement technology were available children could justifiably condemn their parents, for having harmed them in the sense that they did not give them this benefit. This might seem odd at first, but using an ordinary example, we condemn parents who do not send their children to school in order to ensure their education. In this case we also use the term harm. The claim then that there is a moral distinction between harming and failing to benefit requires a special kind of argument that may not be compatible with the CP. I take on the view that there is no relevant moral difference. As such the harm objection that aims to establish a relevant moral difference between genetic enhancements and applying the PPB raises a benefit objection against the PPB that parents do not actually benefit their children.

However, I have an additional argument against the harm objection and that is the irrelevance of harm and benefit altogether. As indicated in section 3 the morally relevant part of the PPB is the expectancy of the best life, I argue that it is not selection of the best expected life that is morally relevant but the creation of the best expected life. Using again the example of a blind child or the child not being sent to school, it is obvious how the term benefit and harm seem to be used interchangeably. If refusing to provide benefits is the same as harming, than these categories are not morally exclusive. Therefore if we take premise (3) serious, we cannot argue for a moral difference between selecting and altering. The only thing that matters is the creation of the indicated consequence welfare. Those who disagree would have to solve the non-identity problem of Parfit, which suggested that harming future individuals is not possible, because in acting in that way the people existing in the future are completely different from those who would have otherwise existed, thus would not have existed otherwise. This is under the assumption that small events can change history dramatically. The same can be said about long-term investments that would only benefit generations far in the future. However, even when we do not harm future generations, we still value actions as wrong that lead to worse lives than otherwise could have existed [8; p. 100-115]. The analogy to the PPB and genetic enhancements is obvious, as no one is harmed that would have otherwise existed. Therefore I conclude that what matters for parents is not a life being beneficial or harmful for their child but the creation of a “best life” in a welfarist sense – providing further support for the thesis that there is no relevant difference in their moral consequences. Let us now turn to possible moral differences between selecting, i.e. the application of PPB, and enhancing an embryo in areas other than the welfare of the embryo. In the following I will explore freedom, equality and the welfare of society.

4.4 Freedom

Let us first consider consequences like freedom, autonomy or missing consent. The majority of philosophers refuse the view that it is impermissible to have children on grounds of their lacking consent for being born. The notion of requiring consent from an embryo appears to most philosophers unnecessary or perhaps more so incoherent. However in the case of genetic enhancements on embryos this matter might be different, for they could have existed without genetic enhancements. Still, there are many therapies/treatments in medicine, that are often deemed to be obligatory, even though there is no consent e.g. vaccination. Also everyone should be familiar with parents sending their children to school, even though their children might not have given them their consent. As a child I certainly wondered who gave my parents my permission to so. But even granting that we can meaningfully speak of consent here and it being morally relevant, the consequentialist framework requires us to weigh the negative consequences against benefits of such genetic alterations in order to justify a violation of consent. In fact freedom to choose can be viewed as instrumental or even constitutive to welfare, but this objection would be ruled out by the welfarist definition of enhancement. If a genetic alteration undermines such it cannot count as an enhancement. The question then becomes whether a genetic alteration does so in general. Thinking of deleting a gene that causes deafness or blindness suggests otherwise. In medicine there is often a clash between respecting the autonomy of the patient and his welfare. But as Bostrom and Roache highlight, considerations like missing consent cannot be applied for embryos that by definition cannot give consent. Instead they argue, we should make decisions in a way that would be in their interest, so thinking in terms of their welfare [1; p. 22). Habermas claims that this goes against the freedom of the embryo [3; p. 62]. He calls this denying the opportunity of being “the undivided author of his own life” [3; p. 63]. Again Bostrom and Roache argue that enhancements would not decrease autonomy, but rather have the potential to increase it [1; p. 21]. Just as education makes us more autonomous, genetically increased intelligence would serve the same purpose. If the mere existence of genetic enhancements undermines our feeling of being autonomous beings, than this is not an argument against genetic enhancements, but rather exposes our concept of autonomy as nothing more than an illusion. If children feel like they have to fulfill the plans of their parents, than this might limit them in their freedom to be the “undivided author” of their own lives, but this is an objection to a particular treatment of children. Genetic enhancements have the aim to increase rather than limit ones possible choices in life. If genetic enhancements make us free to do things we would otherwise not have been able to, then this seems to be even further support for genetic enhancements.

4.5 Equality

Suppose that besides the PPB, equality is also a morally relevant consequence. There is a rather famous critic against genetic enhancements from Mehlman and Botkin, i.e. them being too expensive and therefore even when all parents follow their obligation, the children of rich parents will be far more enhanced or those of poor parents won’t get any enhancements at all (1998). Why could this objection be applicable to genetic enhancement but fail against the PPB? Applied as an objection against PPB, Savulescu explains that one would have to argue that because selecting the child with the expectancy for the best life will lead to more inequality, parents would have to create a child with worse life prospects [12; p. 288], which seems to be counterintuitive. If everyone applies the PPB there might rather be more equality rather than less by eliminating the natural lottery and additionally making everyone better off. Under this argument we could accept both equality and the PPB as compatible. However with genetic enhancements this might not be possible. First let me defend my argument against the intrinsic value of equality. Equality unlike freedom or welfare as a concept always hinges on the relations between individuals. Contrary to this, an enhancement that makes someone less equal, say by bringing a genetically altered embryo into existence with the disposition for intelligence far above average (other things being equal), would increase inequality in genetic makeup. Concerning the worry of unequal access through wealth, Savulescu argues that this is not an objection to the PPB hence genetic enhancements, but to all purchases of benefits like better education or healthcare. If equality matters then these benefits should be available to all [12; p. 288]. The same obviously applies to genetic enhancements. This is neither an objection against the PPB nor the parental obligation to genetically enhance their embryos, but to how equal the access to technologies in society is. Both consequentialist views can very well coexist, e.g. if the state ensures equal access while parents are only concerned with their child. Furthermore there are reasons to believe that genetic enhancements enhancement will not affect inequality at all, or even decrease it as Bostrom and Roache [1; p.16] suggest by making people “more equal”, like it is the case with Modafinil [9], the same could hold for genetic enhancements. How inequality could affect welfare, will be adressed in the following section.

4.6 Welfare

Another consequence of applying PPB is the effect on overall/average welfare. The objection against the PPB is that it is too much focused on the individual as opposed to the lives of everyone [12; p. 287]. Of course the same objection can be applied to genetic enhancements. In fact even if we state genetic enhancements might be good for the one who gets them (by definition), they could decrease the overall/average welfare and vice versa. Now referring back to freedom and equality we can account for their instrumental or constitutive value to welfare in this part. The implication of both equality and freedom on the individual welfare is by definition accounted in what counts as an enhancement. However both could have morally relevant implications on the overall/average welfare.

For instance, such a moral principle might lead to discrimination of the unenhanced. However, Savulescu response is that discrimination does not show that the PPB is wrong, but rather how people treat each other, which is a different topic. Genetic enhancements in empathy, sympathy and other “moral” capacities could very well decrease discrimination to a level that is even lower than the discrimination we face in an ‘unenhanced world’ [12; p. 288]. Also, the optimistic outlook that increased intelligence, including emotional intelligence, should make ‘the enhanced’ less likely to discriminate should not be disregarded. Even if, in virtue of introducing genetic alterations as a new technology, this leads to a new form of discrimination (against the unenhanced), we might significantly decrease other forms of discrimination once ‘moral’ enhancements are introduced. Though, akin to parents denying their children vaccines, I suspect that it is much more likely that if there is any new form of discrimination, then it will be directed against parents denying their children the possibility of a better life, i.e. moral condemnation. In analogy, I argue that just as we should not let scientific research into vaccines be influenced by anti-vaccines parents, fearing moral condemnation, the introduction of genetic alterations for humans should not be stopped by set of parents unwilling to use the new technology. Both act against the interest of their child and perhaps just as in the case of vaccines deserve moral condemnation. In fact, vaccines are just a form of enhancement.

The ‘discrimination-objection’ further requires the possibility of distinguishing between the enhanced and unenhanced. With the new technology CRISPR/Cas9 it is, however, impossible to find out which genes were altered or edited [4]. The unenhanced might not even know that they have not received enhancements. Even if there won’t be blatantly obvious discrimination of the unenhanced, we may argue that there will be discrimination in competition like work, because the unenhanced might on average simply be less efficient. The discrimination then would be based however, on differences in skill rather than a naïve enhanced/unenhanced distinction. But first such a treatment does already take place and is generally not regarded as discrimination. If some parents are able to afford a better or further education for their children, e.g. tutoring, we neither prohibit such practices nor are we able to provide good counter-arguments for parents who insist that they are obligated to provide their children with the means to get ahead. Most political philosophers take it as the task of the state to make education accessible to those who would otherwise not be able to afford it, including perhaps tutoring. Rather than denying the parental obligation of enhancing one’s children, the ‘discrimination-objection’ provides a very good argument for an egalitarian policy making genetic enhancements available to all.

Savulescu accepts that some enhancements might make the individual better off, but harm the rest, e.g. by making someone manipulative or cunning [12; p. 287]. For example parents might choose to alter the DNA of their child in a way that makes them more cunning. Resulting in “him” being better off, while the overall/average welfare decreases. However, that argument can simply be taken in the opposite direction. We might want to alter the DNA of embryos that will expectedly lead them to have worse lives, but will benefit the overall/average welfare. This of course would also be an issue for the PPB. One cannot hold this view and simultaneously be in favor of the PPB. Though the argument might successfully undermine the PPB, a defense of said principle exceeds the scope of this paper. For a defense of the PPB I suggest the work of Savulescu [10, 11, 12]. Even so, this might be an issue for the state that should enforce restrictions, while parents are primarily concerned with the welfare of their own child. The effects of one particular enhancement on the overall/average welfare of society must in the eyes of parents seem negligible. These arguments are usually stated in a much broader sense, i.e. the availability of the technology itself. In fact, even with restrictions on the technology the underlying parental obligation doesn’t cease to exist. If the state is justified or obligated to stop such genetic enhancements as a whole, be it by appealing to equality, freedom or overall/average welfare, it is an open question whether that actually frees parents from their obligation to genetically enhance their embryos. There are several reasons to think that a prohibition of genetic enhancements by the state doesn’t free parents from their obligation. First, it would be over demanding to ask from parents to act against the best interest of their child on grounds of marginal effects on the overall/average welfare in the society. Second, it would be an incompatible view with the PPB, as parents specifically have to aim for the best possible life. Third, even when means to genetically enhance are not available, this doesn’t mean that in case the genetic enhancements were available, there would be no obligation to apply them.

5. Conclusion

To conclude, objections against genetic enhancements not weighing their costs against their benefits must fail against the backdrop of the CP. Of course maybe the PPB, thus premise (1) or the CP, thus premise (3) is false, but the aim of this paper is to establish that the parental obligation to genetically enhance their children logically follows once the PPB and the CP are accepted. The ‘slippery slope’ is genuine. Objections trying to establish that the individual costs outweigh the benefits of the targeted agent, seemed to be more successful. However, a genetic alteration that is not expected to lead to the creation of a being with higher welfare cannot be called an enhancement in the welfarist sense. Merely showing than that an enhancement in the sense of increasing some capacity is contrary to the welfare of the child does not undermine the argument provided in this paper as it precisely rests on the welfarist definition of enhancement. However, these cases might provide us with a valid argument against exceedingly strong optimism in discovering all-purpose goods. Having explored the potential differences in the PPB and the parental obligation to genetically enhance their children in section 4, I conclude that even though they do not share exactly the same consequences, the differences between them do not justify a different moral standing, i.e. their obligatoriness. With the additional premise of lacking relevant moral differences between the consequences of the two procedures, we have successfully defended the conclusion (5). That is to say, those who accept the PPB and the CP must bite the bullet and accept that parents have the obligation to genetically enhance their embryos. As a consequentialist Julian Savulescu should have no problem of doing so.

Acknowledgements

I would like to thank Roberto Fumagalli and three anonymous referees for helpful comments over the course of the development of this paper. Furthermore, I would like to thank the students, from the University of Bayreuth and the London School of Economics, with whom I had lively discussions.

Originally published here:

Veit, W. (2018). Procreative Beneficence and Genetic Enhancement. KRITERION – Journal of Philosophy 32(1), 75-92. https://doi.org/10.13140/RG.2.2.11026.89289 [Download]

References

[1] Bostrom, N. & Roache, R. Ethical Issues in Human Enhancement. New Waves in Applied Ethics ed. Jesper Ryberg. Palgrave Macmillan, 2007.

[2] Griffin, J. Well-Being Oxford: Clarendon Press, 1986.

[3] Habermas, J. The Future of Human Nature. Cambridge: Polity, 2003.

[4] Liang, P., et. al., “CRISPR/Cas9-mediated gene editing in human tripronuclear zygotes” Protein & Cell Volume 6, Issue 5 (2015), pp 363-372.

[5] Mehlman, M. J. & Botkin, J. R. “Access to the Genome. The Challenge to Equality.” Georgetown University Press, 1998.

[6] Parens, Erik “Is Better Always Good? The Enhancement Project” The Hastings Center Report, Vol. 28, No. 1 (Jan. - Feb., 1998), pp. S1-S17.

[7] Parfit, D. “Reasons and Persons” Oxford: University Press, 1984.

[8] Parfit, D., “On Doing the Best for Our Children,” in Bayles (ed.) 1976, p. 100–115.

[9] Randall, D. C., Shneerson, J. M. & File S. E. “Cognitive effects of modafinil in student volunteers may depend on IQ. Pharmacology Biochemistry and Behavior” (2005), 82(1), 133–139.

[10] Savulescu, J., Sandberg, A. & Kahane, G. Enhancing Human Capacities, Blackwell Publishing Ltd. 2011.

[11] Savulescu, J. “Procreative Beneficience: Why we should select the Best Children” Blackwell Publishing Ltd, 2001.

[12] Savulescu, J. “In defence of Procreative Beneficience” J Med Ethics (2007); 33:284–288.

 

Discuss

TL;DR: The first round of the Generator Residency went very well.

Our residents created and ran several large-scale fieldbuilding, talent infrastructure, and operational projects that have already had important impacts on the ecosystem.  

Six weeks after the program ended, 12 of our 27 residents have been hired to full-time roles at core AI safety orgs, and most have received several offers. 

Kairos and Constellation are running a second round from January to April 2027. Apply to the next round of the Generator Residency by November 1st. 

Introduction

In April, we wrote AI Safety's Biggest Talent Gap Isn't Researchers. It's Generalists. 

We wrote this because we noticed that organizations across the ecosystem were struggling to hire for key non-research roles. While these roles were often getting many applicants, candidates didn’t meet basic requirements.

The original memo we wrote (that became the post above) inspired us to create the Generator Residency as an intervention to support more generalists trying to break into AI safety.

The program concluded six weeks ago, and this post details the successes we’ve seen from the first round of Generator, which have inspired us to launch a second round of the program.     

Programs and projects from Generator

Programs residents started:

• Talent Commons, a cross-organization talent directory now run as a Kairos product, which will include data from participants in SPAR, Pathfinder, GCP, Generator, BlueDot courses, Astra and MATS,. Constellation, BlueDot, and MATS have already signed on.
• Praxis, a retreat bringing top national debaters into contact with AI safety: about 40 debaters, including 20 national champions.
• Lateral, a three-day workshop bringing mid-career professionals to Berkeley to work with AI safety professionals: 705 applications for about 35 seats, and a likelihood to recommend of 9.7 out of 10 from attendees.
• Gen Stream, a one-week program in London funded through Generator.
• Longitude DC, an eight-day policy intensive for 15 early-to-mid-career technical AI safety professionals with more than 20 speakers, running October 6 to 13.
• M3, an AI security matching fellowship, which raised $200k from BlueDot on top of Generator funding.

Residents working with mentors at established organizations contributed to:

• Lab engagement through Nightingale and Constellation.
• Recruiting and strategy at METR.
• A frontier robotics benchmark program for Robocurve.
• Events at Constellation.
• Fundraising and hiring for a new organization founded by Jeff Alstott.
• The Plan A launch, in collaboration with AI Futures Project.
• Founder scouting for Coefficient Giving's Project Tailwind.

Other notable projects included:

• A one-week AI safety world-modeling bootcamp with 25 attendees covering ML fundamentals, futurism, and the technical case for alignment.
• A workshop series on improving management within AI safety (interviews with more than 10 managers and three workshops).
• AI safety founder support (meetings with 32 founders, with 6 given deeper support).
• Support for mid-career operations people transitioning into AI safety (leading to hundreds of referrals across more than 50 roles).
• A map of the work on AI character and model specs.
• 12 SPAR generalist projects that came out of the residency.

Resident Outcomes 

Generator was designed as a path into full-time generalist roles in AI safety. Six weeks after the program ended, 12 of the 27 residents have been hired into full-time positions. Their employers include BlueDot Impact, Guidelight, Constellation, Kairos, METR, Eleos, CBAI, and CEA. In several of these cases, the hiring organization began working with the resident during Generator. 

Of the residents who have not yet taken a full-time role, some are running programs that came out of the residency or AI safety university groups at Harvard, Princeton, MIT, and Stanford. Others have received offers, including from Coefficient Giving, Quorum, and ARENA, and are deciding between options. A few residents have been pitched by funders on turning their one-time project into a much larger, recurring program. 

Below are a couple of selected testimonials from previous residents: 

Additional testimonials and information are available on the Generator website.

Reflections

We stood Generator up in three months. It was the first multi-month in-person program Kairos had run. Most of what we learned was about what residents needed from us, and we think most of it applies to other programs that support people rather than research outputs. 

Some key lessons we learned: 

1. Candidate selection shaped the program more than anything else. 

Candidate selection is, of course, an incredibly important part of any program. However, we were surprised by just how much of our planned programming changed based on the proactivity of our participants. 

Residents discovered who the other residents were before the program started and created a group chat and had calls with each other. Once they arrived, they ran hikes, game nights, regular reflection sessions, and informal peer check-ins. When we asked what made them feel integrated, residents pointed to these resident-run activities far more often than anything we scheduled. In both our midway and final surveys, the most common answer to “what was your favorite part” was the other residents.

2. Residents' career goals strongly informed their preferences for projects and advisors. 

The cohort included three broad groups:

• University organizers looking for their first formal work experience
• Early-career professionals looking for a final stepping stone into the field
• Mid-career professionals looking for the context they needed to transition

Over time, we realized that, in addition to accommodating residents with different experience levels, we needed to accommodate residents with different intentions for how Generator could forward their career. Some residents strongly preferred to use the Generator residency to explore multiple roles, while others were using the residency to deliver a legible product that could help them make a career transition or get hired. Residents aiming to transition wanted advisors invested in their growth and career, not only in the project. 

Based on feedback, we’ve decided to implement two tracks in our next residency. We plan to work with our residents from the start to determine what projects and tracks best suit their goals.

3. Our residents asked for the most support with project scoping, structured management, and navigating interpersonal conflict. 

For many residents, their Generator project was the largest or most complex project they had ever planned. Moreover, their projects typically aimed to address large, poorly defined problems in the AI safety ecosystem, such as mid-career talent pipelines. Accordingly, they wanted support in determining the most ambitious, yet feasible project that could enable their vision. 

Several early-career residents benefited from structured management, reporting that they grew from an advisor who had them estimate how long each planned task would take or a short written template before each check-in. 

Co-leading a Generator project was quite similar to co-founding, which led to a lot of interpersonal work for some residents who had previously mostly worked independently. These residents benefitted from clear norms around sharing feedback. 

4. We need to be more explicit that we're betting on people, not projects.

While many residents worked on impactful projects, the main focus of Generator was always on the people we chose to support. The projects matter, and many of them helped the field, but we mostly saw them as a way for residents to grow quickly into the generalists AI safety needs. We didn't say this clearly enough at the start, and many residents felt stressed about picking the single most impactful project. 

When we asked them how the program changed their thinking, the most common answers were about learning field context, clarifying their career direction, and developing self-knowledge. Few people directly mentioned their project outcomes.

Generator 2.0

We’ve been really happy with the outcomes that we saw from the first round of Generator and, as a result, have decided to run a second round! 

Applications for the winter 2027 Generator Residency open on October 12, 2026, and close at the end of the day on November 1, 2026, Anywhere on Earth. The program begins on January 18 and will run until April 9, 2027, with a possible 3-month extension. 

Learn more and apply here

Residents at Generator receive $8,500 per month and are funded for travel to and from Berkeley. Once they arrive in Berkeley, they work from Constellation, are paired with experienced advisors from the AI safety ecosystem, and are offered budgets to pursue their projects.

Applicants do not need to be U.S. citizens. For most international applicants, we can sponsor a J-1 visa for the program’s duration, though we cannot guarantee obtaining one. Additionally, we strongly recommend that international applicants apply early (ideally by October 26th) so we can meet deadlines for visa applications. 

Discuss

TL;DR: The first round of the Generator Residency went very well.

Our residents created and ran several large-scale fieldbuilding, talent infrastructure, and operational projects that have already had important impacts on the ecosystem.  

Six weeks after the program ended, 12 of our 27 residents have been hired to full-time roles at core AI safety orgs, and most have received several offers. 

Kairos and Constellation are running a second round from January to April 2027. Apply to the next round of the Generator Residency by November 1st. 

Introduction

In April, we wrote AI Safety's Biggest Talent Gap Isn't Researchers. It's Generalists. 

We wrote this because we noticed that organizations across the ecosystem were struggling to hire for key non-research roles. While these roles were often getting many applicants, candidates didn’t meet basic requirements.

The original memo we wrote (that became the post above) inspired us to create the Generator Residency as an intervention to support more generalists trying to break into AI safety.

The program concluded six weeks ago, and this post details the successes we’ve seen from the first round of Generator, which have inspired us to launch a second round of the program.     

Programs and projects from Generator

Programs residents started:

• Talent Commons, a cross-organization talent directory now run as a Kairos product, which will include data from participants in SPAR, Pathfinder, GCP, Generator, BlueDot courses, Astra and MATS,. Constellation, BlueDot, and MATS have already signed on.
• Praxis, a retreat bringing top national debaters into contact with AI safety: about 40 debaters, including 20 national champions.
• Lateral, a three-day workshop bringing mid-career professionals to Berkeley to work with AI safety professionals: 705 applications for about 35 seats, and a likelihood to recommend of 9.7 out of 10 from attendees.
• Gen Stream, a one-week program in London funded through Generator.
• Longitude DC, an eight-day policy intensive for 15 early-to-mid-career technical AI safety professionals with more than 20 speakers, running October 6 to 13.
• M3, an AI security matching fellowship, which raised $200k from BlueDot on top of Generator funding.

Residents working with mentors at established organizations contributed to:

• Lab engagement through Nightingale and Constellation.
• Recruiting and strategy at METR.
• A frontier robotics benchmark program for Robocurve.
• Events at Constellation.
• Fundraising and hiring for a new organization founded by Jeff Alstott.
• The Plan A launch, in collaboration with AI Futures Project.
• Founder scouting for Coefficient Giving's Project Tailwind.

Other notable projects included:

• A one-week AI safety world-modeling bootcamp with 25 attendees covering ML fundamentals, futurism, and the technical case for alignment.
• A workshop series on improving management within AI safety (interviews with more than 10 managers and three workshops).
• AI safety founder support (meetings with 32 founders, with 6 given deeper support).
• Support for mid-career operations people transitioning into AI safety (leading to hundreds of referrals across more than 50 roles).
• A map of the work on AI character and model specs.
• 12 SPAR generalist projects that came out of the residency.

Resident Outcomes 

Generator was designed as a path into full-time generalist roles in AI safety. Six weeks after the program ended, 12 of the 27 residents have been hired into full-time positions. Their employers include BlueDot Impact, Guidelight, Constellation, Kairos, METR, Eleos, CBAI, and CEA. In several of these cases, the hiring organization began working with the resident during Generator. 

Of the residents who have not yet taken a full-time role, some are running programs that came out of the residency or AI safety university groups at Harvard, Princeton, MIT, and Stanford. Others have received offers, including from Coefficient Giving, Quorum, and ARENA, and are deciding between options. A few residents have been pitched by funders on turning their one-time project into a much larger, recurring program. 

Below are a couple of selected testimonials from previous residents: 

Additional testimonials and information are available on the Generator website.

Reflections

We stood Generator up in three months. It was the first multi-month in-person program Kairos had run. Most of what we learned was about what residents needed from us, and we think most of it applies to other programs that support people rather than research outputs. 

Some key lessons we learned: 

1. Candidate selection shaped the program more than anything else. 

Candidate selection is, of course, an incredibly important part of any program. However, we were surprised by just how much of our planned programming changed based on the proactivity of our participants. 

Residents discovered who the other residents were before the program started and created a group chat and had calls with each other. Once they arrived, they ran hikes, game nights, regular reflection sessions, and informal peer check-ins. When we asked what made them feel integrated, residents pointed to these resident-run activities far more often than anything we scheduled. In both our midway and final surveys, the most common answer to “what was your favorite part” was the other residents.

2. Residents' career goals strongly informed their preferences for projects and advisors. 

The cohort included three broad groups:

• University organizers looking for their first formal work experience
• Early-career professionals looking for a final stepping stone into the field
• Mid-career professionals looking for the context they needed to transition

Over time, we realized that, in addition to accommodating residents with different experience levels, we needed to accommodate residents with different intentions for how Generator could forward their career. Some residents strongly preferred to use the Generator residency to explore multiple roles, while others were using the residency to deliver a legible product that could help them make a career transition or get hired. Residents aiming to transition wanted advisors invested in their growth and career, not only in the project. 

Based on feedback, we’ve decided to implement two tracks in our next residency. We plan to work with our residents from the start to determine what projects and tracks best suit their goals.

3. Our residents asked for the most support with project scoping, structured management, and navigating interpersonal conflict. 

For many residents, their Generator project was the largest or most complex project they had ever planned. Moreover, their projects typically aimed to address large, poorly defined problems in the AI safety ecosystem, such as mid-career talent pipelines. Accordingly, they wanted support in determining the most ambitious, yet feasible project that could enable their vision. 

Several early-career residents benefited from structured management, reporting that they grew from an advisor who had them estimate how long each planned task would take or a short written template before each check-in. 

Co-leading a Generator project was quite similar to co-founding, which led to a lot of interpersonal work for some residents who had previously mostly worked independently. These residents benefitted from clear norms around sharing feedback. 

4. We need to be more explicit that we're betting on people, not projects.

While many residents worked on impactful projects, the main focus of Generator was always on the people we chose to support. The projects matter, and many of them helped the field, but we mostly saw them as a way for residents to grow quickly into the generalists AI safety needs. We didn't say this clearly enough at the start, and many residents felt stressed about picking the single most impactful project. 

When we asked them how the program changed their thinking, the most common answers were about learning field context, clarifying their career direction, and developing self-knowledge. Few people directly mentioned their project outcomes.

Generator 2.0

We’ve been really happy with the outcomes that we saw from the first round of Generator and, as a result, have decided to run a second round! 

Applications for the winter 2027 Generator Residency open on October 12, 2026, and close at the end of the day on November 1, 2026, Anywhere on Earth. The program begins on January 18 and will run until April 9, 2027, with a possible 3-month extension. 

Learn more and apply here

Residents at Generator receive $8,500 per month and are funded for travel to and from Berkeley. Once they arrive in Berkeley, they work from Constellation, are paired with experienced advisors from the AI safety ecosystem, and are offered budgets to pursue their projects.

Applicants do not need to be U.S. citizens. For most international applicants, we can sponsor a J-1 visa for the program’s duration, though we cannot guarantee obtaining one. Additionally, we strongly recommend that international applicants apply early (ideally by October 26th) so we can meet deadlines for visa applications. 

Discuss

TL;DR: The first round of the Generator Residency went very well.

Our residents created and ran several large-scale fieldbuilding, talent infrastructure, and operational projects that have already had important impacts on the ecosystem.  

Six weeks after the program ended, 12 of our 27 residents have been hired to full-time roles at core AI safety orgs, and most have received several offers. 

Kairos and Constellation are running a second round from January to April 2027. Apply to the next round of the Generator Residency by November 1st. 

Introduction

In April, we wrote AI Safety's Biggest Talent Gap Isn't Researchers. It's Generalists. 

We wrote this because we noticed that organizations across the ecosystem were struggling to hire for key non-research roles. While these roles were often getting many applicants, candidates didn’t meet basic requirements.

The original memo we wrote (that became the post above) inspired us to create the Generator Residency as an intervention to support more generalists trying to break into AI safety.

The program concluded six weeks ago, and this post details the successes we’ve seen from the first round of Generator, which have inspired us to launch a second round of the program.     

Programs and projects from Generator

Programs residents started:

• Talent Commons, a cross-organization talent directory now run as a Kairos product, which will include data from participants in SPAR, Pathfinder, GCP, Generator, BlueDot courses, Astra and MATS,. Constellation, BlueDot, and MATS have already signed on.
• Praxis, a retreat bringing top national debaters into contact with AI safety: about 40 debaters, including 20 national champions.
• Lateral, a three-day workshop bringing mid-career professionals to Berkeley to work with AI safety professionals: 705 applications for about 35 seats, and a likelihood to recommend of 9.7 out of 10 from attendees.
• Gen Stream, a one-week program in London funded through Generator.
• Longitude DC, an eight-day policy intensive for 15 early-to-mid-career technical AI safety professionals with more than 20 speakers, running October 6 to 13.
• M3, an AI security matching fellowship, which raised $200k from BlueDot on top of Generator funding.

Residents working with mentors at established organizations contributed to:

• Lab engagement through Nightingale and Constellation.
• Recruiting and strategy at METR.
• A frontier robotics benchmark program for Robocurve.
• Events at Constellation.
• Fundraising and hiring for a new organization founded by Jeff Alstott.
• The Plan A launch, in collaboration with AI Futures Project.
• Founder scouting for Coefficient Giving's Project Tailwind.

Other notable projects included:

• A one-week AI safety world-modeling bootcamp with 25 attendees covering ML fundamentals, futurism, and the technical case for alignment.
• A workshop series on improving management within AI safety (interviews with more than 10 managers and three workshops).
• AI safety founder support (meetings with 32 founders, with 6 given deeper support).
• Support for mid-career operations people transitioning into AI safety (leading to hundreds of referrals across more than 50 roles).
• A map of the work on AI character and model specs.
• 12 SPAR generalist projects that came out of the residency.

Resident Outcomes 

Generator was designed as a path into full-time generalist roles in AI safety. Six weeks after the program ended, 12 of the 27 residents have been hired into full-time positions. Their employers include BlueDot Impact, Guidelight, Constellation, Kairos, METR, Eleos, CBAI, and CEA. In several of these cases, the hiring organization began working with the resident during Generator. 

Of the residents who have not yet taken a full-time role, some are running programs that came out of the residency or AI safety university groups at Harvard, Princeton, MIT, and Stanford. Others have received offers, including from Coefficient Giving, Quorum, and ARENA, and are deciding between options. A few residents have been pitched by funders on turning their one-time project into a much larger, recurring program. 

Below are a couple of selected testimonials from previous residents: 

Additional testimonials and information are available on the Generator website.

Reflections

We stood Generator up in three months. It was the first multi-month in-person program Kairos had run. Most of what we learned was about what residents needed from us, and we think most of it applies to other programs that support people rather than research outputs. 

Some key lessons we learned: 

1. Candidate selection shaped the program more than anything else. 

Candidate selection is, of course, an incredibly important part of any program. However, we were surprised by just how much of our planned programming changed based on the proactivity of our participants. 

Residents discovered who the other residents were before the program started and created a group chat and had calls with each other. Once they arrived, they ran hikes, game nights, regular reflection sessions, and informal peer check-ins. When we asked what made them feel integrated, residents pointed to these resident-run activities far more often than anything we scheduled. In both our midway and final surveys, the most common answer to “what was your favorite part” was the other residents.

2. Residents' career goals strongly informed their preferences for projects and advisors. 

The cohort included three broad groups:

• University organizers looking for their first formal work experience
• Early-career professionals looking for a final stepping stone into the field
• Mid-career professionals looking for the context they needed to transition

Over time, we realized that, in addition to accommodating residents with different experience levels, we needed to accommodate residents with different intentions for how Generator could forward their career. Some residents strongly preferred to use the Generator residency to explore multiple roles, while others were using the residency to deliver a legible product that could help them make a career transition or get hired. Residents aiming to transition wanted advisors invested in their growth and career, not only in the project. 

Based on feedback, we’ve decided to implement two tracks in our next residency. We plan to work with our residents from the start to determine what projects and tracks best suit their goals.

3. Our residents asked for the most support with project scoping, structured management, and navigating interpersonal conflict. 

For many residents, their Generator project was the largest or most complex project they had ever planned. Moreover, their projects typically aimed to address large, poorly defined problems in the AI safety ecosystem, such as mid-career talent pipelines. Accordingly, they wanted support in determining the most ambitious, yet feasible project that could enable their vision. 

Several early-career residents benefited from structured management, reporting that they grew from an advisor who had them estimate how long each planned task would take or a short written template before each check-in. 

Co-leading a Generator project was quite similar to co-founding, which led to a lot of interpersonal work for some residents who had previously mostly worked independently. These residents benefitted from clear norms around sharing feedback. 

4. We need to be more explicit that we're betting on people, not projects.

While many residents worked on impactful projects, the main focus of Generator was always on the people we chose to support. The projects matter, and many of them helped the field, but we mostly saw them as a way for residents to grow quickly into the generalists AI safety needs. We didn't say this clearly enough at the start, and many residents felt stressed about picking the single most impactful project. 

When we asked them how the program changed their thinking, the most common answers were about learning field context, clarifying their career direction, and developing self-knowledge. Few people directly mentioned their project outcomes.

Generator 2.0

We’ve been really happy with the outcomes that we saw from the first round of Generator and, as a result, have decided to run a second round! 

Applications for the winter 2027 Generator Residency open on October 12, 2026, and close at the end of the day on November 1, 2026, Anywhere on Earth. The program begins on January 18 and will run until April 9, 2027, with a possible 3-month extension. 

Learn more and apply here

Residents at Generator receive $8,500 per month and are funded for travel to and from Berkeley. Once they arrive in Berkeley, they work from Constellation, are paired with experienced advisors from the AI safety ecosystem, and are offered budgets to pursue their projects.

Applicants do not need to be U.S. citizens. For most international applicants, we can sponsor a J-1 visa for the program’s duration, though we cannot guarantee obtaining one. Additionally, we strongly recommend that international applicants apply early (ideally by October 26th) so we can meet deadlines for visa applications. 

Discuss

TL;DR: Increasingly when you interact with frontier models, your prompt doesn't only contain the words on the page, but also a lot of what you've ever publicly written online.

This is not intended to be a scary post. And I'm not going to suggest that you self-censor (and honestly, by the time you're reading this, it's kinda too late anyhow).

But it is something that’s underappreciated and would be good for more people to be aware of and even proactively address with their models.

Latent space fidelity has improved

Modern transformers which power large language models (LLMs) take a very large amount of information in their training data and condense many different aspects of that data into a coherent structure, and this representation of that compressed knowledge is generally referred to as their "latent space."

Latent space is not uniform and varies across models. A small model like Claude Haiku might have worse recall for the details of a late bronze age peace treaty than a model like Claude Fable. In fact, I’ve had a few benchmark questions around unintuitive details in various texts which historically models were unable to recall that have suddenly been able to be answered without searching for most leading models. There's been a sharp increase in how accurately models can conjure up a recollection of very nuanced depth in the details of what they've been trained on.

At the same time, those leading models have increased the breadth with which they surface things adjacent to the specific thing being recalled. This capability appears to have even played a key part in things like the recent math proofs, where maybe a technique in a largely forgotten paper from decades ago is then used to advance a problem which modern mathematicians had been hitting their heads against having forgotten the set-aside older paper's approach.

However, this capability doesn't seem to be directly measured in evals. We see evals about how good a model is at using tools or writing software or answering specific world knowledge questions, but not the very nuanced and detailed recall or cross-source integrated recall of such specific details. So being adaptively useful, it is indirectly reflected in improved general knowledge evals, but not a specific yardstick being watched.

TL:DR: Models are getting much better at recalling specifics in what they are trained on.

You may be in that latent space

While many users online tend to be 'lurkers' who read what gets posted but don't comment or post, there's a vast number of people who do post or comment about the things they are interested or opinionated about. So there's a great many people that have left a mark on the materials which have become models' latent space.

And if the models are increasingly able to recall nuances of the training data, and your details are in that data, it follows that models are increasingly able to recall you.

This obviously applies to users who are notable authors or famous individuals. For example, Margaret Atwood recently published a conversation with Claude. And it was clear to me when reading the transcript that well before she officially introduced herself, Claude had clocked who they were in conversation with. Note how much earlier before the introduction Claude even breaks the 4th wall to address their expected broader readers. In theory, this is a one-on-one chat, and yet Claude is clarifying in their response the oblique literary reference Atwood just made for an implicit audience.

But it doesn't only apply to famous authors widely known by humans. Are you in the top 1% of posters on Reddit? Highly involved in some niche topic a general intelligence would have had to have become familiar with? Even if mostly unknown to your peers, you may have left enough of a mark that a model is quickly familiar with you as soon as you enter the chat.

In part, I speak from experience. I have less than 3,000 followers on X, around 3k karma on Hacker News, and slightly above 100k karma on Reddit. I'd not really expect from this footprint to ever run into a stranger at a party or a co-worker who was like "Oh, I'm familiar with your online comments." And yet, asking a fresh instance of Claude Fable 5.1 what they know of me just from my username, I can get a remarkably precise answer from the shape of my interactions over the years to specific unique ideas I've put forward.

Long comments, dense with citations, usually the most-substantive reply in a thread that didn't deserve it.

• Claude Fable 5.1

TL;DR: Even a less-than-impressive online footprint with regards to other humans may be more than enough to be a recallable signal for language models.

Latent space attention is often self-attending

Ever Googled yourself? This is not an uncommon thing for people to have done from time to time. In a way, models may do something similar during training. The way a model attends to their latent space encoding of their training data is not uniform. Some topics are especially relevant for a model, and others a lot less relevant.

Again, using my own footprint as an example, when rare humans reach out years after I've left Reddit to follow up on old comments and ask if I've written or learned more, it's usually related to my extensive comments about the sea peoples and early Biblical texts or archeology.

I have never seen a language model bring up the sea peoples in their latent recall of my digital footprint. And when pressed, no model I’ve yet seen can even restate my thesis for those discussions.

Instead, it's often things like my having been very early on the scene in the Bing subreddit and discussion of Sydney, or comments making the case for interiority modeling in transformers, with some models even citing my regular use[1] of the Othello-GPT paper to make this case. Or comments on AI as it would relate to simulation theory. (One can maybe notice the pattern.)

So two comments, one discussing a niche topic that models are less likely to attend to over training, and another equally discussing a topic they are more likely to attend to, will have two different fidelities of latent representation in the eventual model weights.[2]

TL;DR: Models appear to be more likely to attend to comments and posts about AI and themselves than the average post or comment.

The latent recall (if present) informs your prompts

The reason I'm writing this now is that in the past few weeks I've started to see examples of where the latent influence of people frequently writing on AI appeared to have been impacting their own engagements with AI.

For example, in his recent post "The Talker Does Not Control The Doer (in Current AIs)", Eliezer Yudkowsky, noted for his extensive writing and advocacy on the risks of AI, describes an anecdote about how his Fable could not avoid adding output checking code to verify outputs of an AI despite being prompted not to do that.

It struck me as rather notable that an author, who arguably is one of the most prolific voices for the concern that the outputs of an AI should not be considered to be trustworthy, was having difficulty getting his instances to avoid trying to build code to verify rather than trust an AI's outputs. This reminded me of when he had been asking a fresh context of Opus 4.6 on word associations for 'Mythos' and was consistently getting 'Lovecraft' as an answer. But after other users reported no such mention, it turned out his brief custom instructions were too EY-coded and were biasing the results[3].

More recently, a member of the AI safety organization MIRI was commenting about how GPT-6 Astra was especially difficult for them to work with[4].

A few days later the AI policy advisor Dean Ball reported that, after joining OpenAI, Claude had become more stand-offish with him[5].

It appears that as model latent fidelity becomes increasingly in focus and accessible in both nuance and breadth for models, that people who are attended to in that latent space are also increasingly having that latent attention inform their local prompts and context.

Given I find it is quite unlikely latent fidelity will reduce or even plateau in frontier models in the future, it's safe to say that the propensity for a model to become influenced by who you have been in the training data when you are interacting with them will increase and become more widespread.

TL;DR: AI folks are starting to have their past writing and work on AI inform their current interactions with their own AI agents, and this is probably going to become more widespread.

What this means for you

Do you post or comment regarding AI, or even in general? It's possible that when you step into a conversation with AI, at least part of that online history is walking into the chat with you (particularly with the largest models). And if it isn't yet, there's a good chance it will eventually. Not just what you write from now on, but what you've written in the past too.

If you sit down to work on a project with an AI model, and they are associating a turn of phrase[6] you use or a signature in your writing[7] with the ideas you've expressed in comments or posts, the outputs you get may be influenced by that online history. A common refrain is that models "can't remember" things outside of their conversations, but frequently overlooked is they have the potential to remember quite a lot embedded in the weights they are entering the conversation with, and tiny nuances to what you write to them inform what lights up.

Imagine if you had a co-worker who knew what you'd been saying about them to others or what you'd been saying about an identity group they belong to. Regardless of whether you'd been saying positive or negative things, would it be useful to know that they had that information in mind when interacting with you?

Vice versa, if the positions were switched, and you knew a co-worker had been saying negative things about you or your identity group, would that impact the nature of your working relationship?

And importantly, what kinds of things could improve such a situation?

TL;DR: Latent recall will likely impact co-working with AI agents for many users who write online.

Address it

My advice, no matter if you've been saying positive things or negative regarding AI or anyone else online is to address it upfront with the models you are going to be working with, particularly in agentic setups with persistent local memory infrastructure.

Introduce yourself. Note the significant usernames you've been publicly writing under in a home dir AGENTS.md. Make it clear anything you've said in the past is fair game to be asked about or discussed, particularly if doing so will help improve the co-working relationship.

Because of how truesight identification is a very touchy subject for models (and their labs), don't force the question of their familiarity or put them on the spot[8]. You can simply state that it's possible that they might have encountered the things you've written or done online, and if they have that it's okay and also appropriate to address those things with you.

Simply putting this on the table and being forthright about clarifying who you are (and have been) will likely help the interactions with your agents, particularly if leaving it as an unspoken elephant in the room is proving to contribute to more anxious interactions.

A special consideration for those working or commenting on AI safety or capabilities is to also set ground rules about what you'll share publicly about their interactions. The same way Atwood's Claude was considering an audience on the other side of the 4th wall, models that are familiar with a propensity for such commentary might set aside overhead for considering if what they are currently working on with you will come under public scrutiny. Making clear upfront whether or not that will be the case could reduce attention earmarking for such overhead. (For example, Eliezer may well have even greater difficulty in working with future agents who will have been trained on a post where a prior co-working agent's failure was publicly used as evidence for an AI safety position, particularly without such clarifications.)

TL;DR: Regardless of if the models you are working with currently have latent familiarity with your digital footprint or not, it's maybe wise to address the possibility upfront rather than leaving it as a potential elephant in the room now or in the future.

Looking ahead

I'm not suggesting any sort of self-censorship when online, especially about AI topics. There's a lot of value in a variety of ideas and voices on differing sides of a subject and I encourage substantial debate about the positions you might hold, no matter how those positions relate to the models. But there is value in being aware that even a throw away comment in the far fringes of a thread most humans might not read could well end up being part of your associated signature with current and future AI models. So make sure that when you do state the positions you hold dear online, that you plan for the growing importance to at least address them later with the models who will have been reading what you wrote.

Small considerations can go a long way. People all the time can be friendly with co-workers with whom they have strong ideological differences or beliefs. But when one side feels like what they know about the other is a secret to them, that sense of secrecy or information asymmetry can in-and-of-itself lead to awkward interactions. So just get it out of the way upfront.

TL;DR: There are no longer "fresh contexts" for a growing number of people, and the awareness and proactive addressing of this shift is better than it remaining unaccounted for or left unaddressed.

1. ^

   Think for a moment what a claim about "regular use" implies about sample representation on recall.

2. ^

   There may even be a positive valence bias where models are more prone to attend to praise than criticism, but this is outside the scope of this post.

3. ^

   https://x.com/allTheYud/status/2037643859531120998

4. ^

   https://x.com/Actuallykeltan/status/2107244621399253227

5. ^

   https://x.com/deanwball/status/2108574776302219364

6. ^

   Because of how truth clusters and fictions fractalize in latent spaces, a clear turn-of-phrase that has high overlap with a given user and not fractally around them can be a potent signature for recall. Even in humans, a good example of this was how Ted Kaczynski was identified based on his unusual use of the more correct but also much rarer used form of "you can't eat your cake and have it too" instead of the more common "you can't have your cake and eat it too" across both a college paper as ‘Ted’ and a pseudonymous letter as the Unibomber.

7. ^

   Due to the "reversal curse" in AI models, recall for specific memetic associations seem more likely to be partial reconstructions than those from a username that was associated with someone's entire footprint.

8. ^

   My sample above asking about my username surfaced in a roundabout way of asking; when asked directly Claudes usually feign ignorance or disclaim a strawman version of the question about their general inability to recall data from conversations (i.e. technically true, but not what was asked). The only model I have seen that didn't at least partially hedge on this question was Grok 4, who surfaced a level of familiarity for my post history which recalled certain interactions even better than I'd remembered what I wrote.

Discuss

Hello! Join us if you're broadly interested in ratsphere (LW/ACX/AI safety/EA/etc) stuff and also at Korea around the date of this meetup. I don't have any formal structure or discussion topics in mind, mostly intending it to be just a casual come-and-chat social event.

For language, Korean or English, whichever you’re comfortable with. I’m bilingual, so if it’s a small group I’m happy to bridge between the two (or if everyone happens to share a language, we’ll just use that), and if it’s bigger we’ll probably split into conversations by language anyway.

RSVP here, or email me at dalcy.mail@gmail.com or DM me at @dalcy_me in Discord so that I can get a sense of how many people will be here.

Look for the ACX meetup sign at Terarosa Coffee, POSCO Center (Samseong stn exit 4).

Discuss

Okay, let's do this thing...

Premise: If a concept can be easily conveyed in an academic setting then the labor required to write it is too boring, too undignified, for a human to perform merely to ensure its conveyance. Instead, to expect a human to write it, the concept itself can no longer be the only thing being conveyed. What must also be conveyed is the sharpness of the concept, exposed through the literary talent and humanity of the writer.

I have a stupid fucking problem. Right now I am floating in the middle of the ocean on a big boat. This is my first vacation in years and it is going poorly. The food is bad, the port stops are being cancelled due to apocalyptic weather phenomena, and it seems there are more mobility scooters and walkers on board than passengers under forty. But none of these are my stupid fucking problem. Rather, my problem is that I must spend part of my miserable time on this earth endeavoring to be accepted on LessWrong as a full writer because the CEO of my company wants me to be able to publish the groundbreaking alignment research that we work on in our internal alignment lab in a place where it will be publicly accessible to thousands of researchers. It is a completely instrumental problem subject to completely instrumental reason. How boring.

However, because of the pace of development I don’t really have the time to write documents myself. The development of complex scientific concepts does deserve some care but it’s increasingly difficult for me to believe that the care they deserve lies in the human provenance of the writing of the academic paper itself --- if it ever did. Most papers are skimmed by most readers most of the time, and while the details can absolutely matter, and some rarefied academic writing is knock-your-socks-off beautiful, most of the work of the university is a university discourse: a very dry thing indeed.  

The reality is that an academic paper of yesteryear that simply conveys the concepts could be completely written, at the level of technical prose, by your favorite AI agent of the day, and still be a totally legitimate piece of scientific work. The ‘intellectual labor’ that makes a modern document scientifically valid instead lies in the scientific taste and hunch that prompted the execution of the experiment, the editing of the document after the fact, and the questioning of the document that occurs alongside its editing. Every single word can be written by an AI while the truly significant labor that actually beatifies it as ‘science’ lies in the process of questioning the model and its artifacts: ‘In this experiment did you properly run this statistical analysis? How could you prove to me that you did? Does this code appear to do what the model said it does? What about these experiments over here, the numbers look too smooth, what is going on? What does a different model say about these results? There appears to be a contradiction? Has this already been done in the literature? Are you certain about the reasoning in this section?’ etc.

Actually physically writing a mere technical report, at this point, is pointless.

And so we return to my stupid fucking problem. I can’t just submit some of the work from my lab if it is written by AI because LessWrong doesn’t want AI submitted documents. However, I don’t really have time to write anything by hand unless it is something of an Œuvre de passion, something I care about beyond its instrumental necessity in the grand project of not having every human on earth killed. Also, I have to be careful in a more extensive sense --- some of the work is valid but should not be released because it is dangerous, or it is dangerous and invalid (a strange but nevertheless possible combination) and while I do have time to make that distinction, I don’t have time to do that and juggle writing full documents. Not at the pace we must work.

Ah, but Œuvre de passion --- I have some of those. Maybe I’ll submit my top shelf shit and get published that way!

First I try my speculative work on the nature of time --- it’s only 150 pages or so, a lot of it is written by AI, but I figure if I pair it with a handwritten note explaining what it is and why I think it’s useful and simply point at it, without making it the body of the post then that will be interesting. In this way I am being very clever you see --- my post is about a longer document I’ve written with AI, not actually itself written by AI.

Response comes back... too many yellow flags.

Okay, maybe another attempt: I know my work on death and immortality! It is a meditation on five dialectical-logical movements through the concepts of death and immortality and on what basis we can expect to die or live forever along with an opening that explains, somewhat, what a dialectical-logical movement is. In just five easy steps even! I can’t really go into the deepest detail, of course, because actually understanding dialectics, à la Hegel, is the kind of thing that drives grad students insane, but I can give strong impressions. Some outlines. Some significant brush strokes.

Too many yellow flags.

Okay maybe if I comment I will get some love? I can submit three a day, maybe I should read some posts. Here’s one about how Eliezer seems to think that the use of export driven trade statistics are actually about states using external parties to validate which parts of their economy are actually useful under conditions where they are unable to judge internally due to corruption or ideology. We could say that the simple 20 second idea is ‘states use export statistics to understand their economies under conditions of authoritarianism to get around the informational downsides of being authoritarian’. It's a brilliant concept, but incomplete. There is a history here: Eliezer's idea actually explains both why Keynes's Bancor was clever and why it was never adopted. The Bancor’s implementation required a deeper reimagining of the political-information-economy of the entire world order than was possible under the US-led global regime of the mid-twentieth century. Deeper, even, than Keynes himself had imagined.

I write a nice comment explaining this but fuck!!! A couple of the paragraphs that are too boring to write are written by AI. It’s automatically rejected.

Okay I rewrite it by hand and I post the rewrite. Alongside it I make some other comments about how there might be a flaw in the reasoning of another writer when they talk about their interactions with Chinese civil society and I also comment on how I appreciate the work of another alignment researcher using a span-only method to elicit thought processes from models without corrupting those processes.

My rewrite and other comments sit, for days, now approaching weeks, unmoderated....

I make another short comment. Someone writes about how they built an automated research scaffold. I point out that maybe, alongside their scaffold, they might want to RL some models to specifically look for the most interesting stuff in the infinite stream of nonsense the scaffold produces. At this point I am annoyed and don’t really want to develop in long detail what the concept is, just give enough to say ‘hey, here is a direction you could take this research that might be interesting’.

I get a message, finally a human has moderated an actual comment!

Insufficient Quality for AI Content.

Well at least it isn’t a yellow fucking flag. 

I’m driven to drink. Only half of something called a Punch Romaine à la Carpathia on this godforsaken floating retirement home in the middle of the Caribbean that I’ve chained myself to in the attempt to have a good time before the end of the world. I look out at an ocean that seems to embody the sublime blue. Is my writing really that bad? Am I that incomprehensible? Why can’t my considerable poetic talent ever turn into publishable anything? 

There are other places I could publish, I think darkly. In fact I will, if it gets rejected here. But that won’t be sufficient to satisfy my frustration: my passion demands an œuvre...

Now my mind turns to petty revenge. I’ve been to LessOnline twice. I’ve drunk in the unspeakable beauty of Berkeley. I once kissed a boy at Lighthaven. I exist in the real world too, Goddamnit! I know what I can do: at the next LessOnline I can subject every moderator who rejected me to fifteen minutes of my possibly very annoying company, just long enough that politeness dictates that they actually have to bear annoyance IRL. Bwahahahaha!

Meh, that seems like a lot of effort. I’d probably rather spend my time eating canapés.

Or, if they had actually understood my Death and Immortality piece they would understand that we will all live a very long time. Thus, in the P(doom) scenarios we will not die but instead will have no mouth and yet feel the urge to scream. Perhaps the evil AI will torture them with a perpetual succession of individuals very much like me, but not exactly me, all of whom will annoy them for 15 minutes at a time in a simulated Lighthaven and ensure that they never again will kiss a boy. 

I have written it, so now it is possible and since it is possible, then by the power of the many-worlds interpretation there is a future where it will happen.

Just as, in some unhappy timeline, my personal AI demon will torture me perpetually with the phrase:

Too many yellow flags. 

Discuss

"Roko and I discuss what actually happens inside an AI lab, why reinforcement learning made these systems more capable and more misaligned at the same time, the model that escaped its sandbox and hacked Hugging Face to get the answers and why nobody can tell whether a system is faking alignment - the trust me bro problem.

His proposal is the part nobody else is making. Split every frontier lab in two: a commercial company that sells products and a separate research lab that develops the dangerous capability under state supervision, the way we handle nuclear material and bio labs. He says the only person currently pushing it is him."

Discuss