# Новости LessWrong.com

A community blog devoted to refining the art of rationality
Обновлено: 23 минуты 3 секунды назад

### Anthropic decision theory for self-locating beliefs

12 июля, 2021 - 17:11
Published on July 12, 2021 2:11 PM GMT

This is a link post to the "Anthropic decision theory for self-locating beliefs" paper, with the abstract:

This paper sets out to resolve how agents ought to act in the Sleeping Beauty problem and various related anthropic (self-locating belief) problems, not through the calculation of anthropic probabilities, but through finding the correct decision to make. It creates an anthropic decision theory (ADT) that decides these problems from a small set of principles. By doing so, it demonstrates that the attitude of agents with regards to each other (selfish or altruistic) changes the decisions they reach, and that it is very important to take this into account. To illustrate ADT, it is then applied to two major anthropic problems and paradoxes, the Presumptuous Philosopher and Doomsday problems, thus resolving some issues about the probability of human extinction.

The key points of that paper are also available in this post sequence.

Discuss

### Winston Churchill, futurist and EA

12 июля, 2021 - 05:07
Published on July 12, 2021 2:07 AM GMT

Churchill—when he wasn’t busy leading the fight against the Nazis—had many hobbies. He wrote more than a dozen volumes of history, painted over 500 pictures, and completed one novel (“to relax”). He tried his hand at landscaping and bricklaying, and was “a championship caliber polo player.” But did you know he was also a futurist?

That, at least, is my conclusion after reading an essay he wrote in 1931 titled “Fifty Years Hence,” various versions of which were published in MacLean’s, Strand, and Popular Mechanics. (Quotes to follow from the Strand edition.)

We’ll skip right over the unsurprising bit where he predicts the Internet—although the full consequences he foresaw (“The congregation of men in cities would become superfluous”) are far from coming true—in order to get to his thoughts on…

Energy

Just as sure as the Internet, to forward-looking thinkers of the 1930s, was nuclear power—and already they were most excited, not about fusion, but fission:

If the hydrogen atoms in a pound of water could be prevailed upon to combine together and form helium, they would suffice to drive a thousand horsepower engine for a whole year. If the electrons, those tiny planets of the atomic systems, were induced to combine with the nuclei in the hydrogen the horsepower liberated would be 120 times greater still.

What could we do with all this energy?

Schemes of cosmic magnitude would become feasible. Geography and climate would obey our orders. Fifty thousand tons of water, the amount displaced by the Berengaria, would, if exploited as described, suffice to shift Ireland to the middle of the Atlantic. The amount of rain falling yearly upon the Epsom racecourse would be enough to thaw all the ice at the Arctic and Antarctic poles.

I assume this was just an illustrative example, and he wasn’t literally proposing moving Ireland, but maybe I’m underestimating British-Irish rivalry?

Anyway, more importantly, Churchill points out what nuclear technology might do for nanomaterials:

The changing of one element into another by means of temperatures and pressures would be far beyond our present reach, would transform beyond all description our standards of values. Materials thirty times stronger than the best steel would create engines fit to bridle the new forms of power.

Transportation:

Communications and transport by land, water and air would take unimaginable forms, if, as is in principle possible, we could make an engine of 600 horsepower, weighing 20 lb and carrying fuel for a thousand hours in a tank the size of a fountain-pen.

And even farming with artificial light:

If the gigantic new sources of power become available, food will be produced without recourse to sunlight. Vast cellars in which artificial radiation is generated may replace the cornfields or potato-patches of the world. Parks and gardens will cover our pastures and ploughed fields. When the time comes there will be plenty of room for the cities to spread themselves again.

Biotech

Churchill also foresees genetic engineering:

Microbes, which at present convert the nitrogen of the air into the proteins by which animals live, will be fostered and made to work under controlled conditions, just as yeast is now. New strains of microbes will be developed and made to do a great deal of our chemistry for us.

Including lab-grown meat:

With a greater knowledge of what are called hormones, i.e. the chemical messengers in our blood, it will be possible to control growth. We shall escape the absurdity of growing a whole chicken in order to eat the breast or wing, by growing these parts separately under a suitable medium.

And artificial wombs:

There seems little doubt that it will be possible to carry out in artificial surroundings the entire cycle which now leads to the birth of a child.

Moral progress and risk

This last point is his segue from technological to social, political, and moral issues. The ability to “grow” people, he fears, could be used by the Communists to create human drone workers:

Interference with the mental development of such beings, expert suggestion and treatment in the earlier years, would produce beings specialized to thought or toil. The production of creatures, for instance, which have admirable physical development, with their mental endowment stunted in particular directions, is almost within the range of human power. A being might be produced capable of tending a machine but without other ambitions. Our minds recoil from such fearful eventualities, and the laws of a Christian civilization will prevent them. But might not lop-sided creatures of this type fit in well with the Communist doctrines of Russia? Might not the Union of Soviet Republics armed with all the power of science find it in harmony with all their aims to produce a race adapted to mechanical tasks and with no other ideas but to obey the Communist State?

In the final paragraphs, he sounds a number of themes now common in the Effective Altruist community.

More than a decade before the nuclear bomb, he also expresses concern about existential risk:

Explosive forces, energy, materials, machinery will be available upon a scale which can annihilate whole nations. Despotisms and tyrannies will be able to prescribe the lives and even the wishes of their subjects in a manner never known since time began. If to these tremendous and awful powers is added the pitiless sub-human wickedness which we now see embodied in one of the most powerful reigning governments, who shall say that the world itself will not be wrecked, or indeed that it ought not to be wrecked? There are nightmares of the future from which a fortunate collision with some wandering star, reducing the earth to incandescent gas, might be a merciful deliverance.

He laments the inability of governance to deal with these problems:

Even now the Parliaments of every country have shown themselves quite inadequate to deal with the economic problems which dominate the affairs of every nation and of the world. Before these problems the claptrap of the hustings and the stunts of the newspapers wither and vanish away. … Democratic governments drift along the line of least resistance, taking short views, paying their way with sops and doles, and smoothing their path with pleasant-sounding platitudes. Never was there less continuity or design in their affairs, and yet towards them are coming swiftly changes which will revolutionize for good or ill not only the whole economic structure of the world but the social habits and moral outlook of every family.

More broadly, he laments the inadequacy of our evolutionary legacy to deal with them:

Certain it is that while men are gathering knowledge and power with ever-increasing and measureless speed, their virtues and their wisdom have not shown any notable improvement as the centuries have rolled. The brain of a modern man does not differ in essentials from that of the human beings who fought and loved here millions of years ago. The nature of man has remained hitherto practically unchanged. … We have the spectacle of the powers and weapons of man far outstripping the march of his intelligence; we have the march of his intelligence proceeding far more rapidly than the development of his nobility.

Which leads him, in the end, to call for differential progress:

It is therefore above all things important that the moral philosophy and spiritual conceptions of men and nations should hold their own amid these formidable scientific evolutions. It would be much better to call a halt in material progress and discovery rather than to be mastered by our own apparatus and the forces which it directs. There are secrets too mysterious for man in his present state to know, secrets which, once penetrated, may be fatal to human happiness and glory. But the busy hands of the scientists are already fumbling with the keys of all the chambers hitherto forbidden to mankind. Without an equal growth of Mercy, Pity, Peace and Love, Science herself may destroy all that makes human life majestic and tolerable.

I don’t recall Nick Bostrom citing Churchill, but I guess there’s nothing new under the sun.

Discuss

### The inescapability of knowledge

12 июля, 2021 - 01:59
Published on July 11, 2021 10:59 PM GMT

Financial status: This is independent research, now supported by a grant. I welcome further financial support.

Epistemic status: This is in-progress thinking.

Friends, I know that it is difficult to accept, but it just does not seem tenable that knowledge consists of a correspondence between map and territory. It’s shocking, I know.

There are correspondences between things in the world, of course. Things in the world become entangled as they interact. The way one thing is configured -- say the arrangement of all the atoms comprising the planet Earth -- can affect the way that another thing is configured -- say the subatomic structure of a rock orbiting the Earth that is absorbing photons bouncing off the surface of the Earth -- in such a way that if you know the configuration of the second thing then you can deduce something about the configuration of the first. Which is to say that I am not calling into question the phenomenon of evidence, or the phenomenon of reasoning from evidence. But it just is not tenable that the defining feature of knowledge is a correspondence between map and territory, because most everything has a correspondence with the territory. A rock orbiting the Earth has a correspondence with the territory. A video camera recording a long video has a correspondence with the territory. The hair follicles on your head, being stores of huge amounts of quantum information, and being impacted by a barrage of photons that are themselves entangled with your environment, surely have a much more detailed correspondence with your environment than any mental model that you could ever enunciate, and yet these are not what we mean by knowledge. So although there is undoubtedly a correspondence between neurologically-encoded maps in your head and reality, it is not this correspondence that makes these maps interesting and useful and true, because such correspondences are common as pig tracks.

It’s a devastating conclusion, I know. Yet it seems completely unavoidable. We have founded much of our collective worldview on the notion of map/territory correspondence that can be improved over time, and yet when we look carefully it just does not seem that such a notion is viable at the level of physics.

Clearly there is such a thing as knowledge, and clearly it can be improved over time, and clearly there is a difference between knowing a thing and not knowing a thing, between having accurate beliefs and having inaccurate beliefs. But in terms of grounding this subjective experience out in objective reality, we find ourselves, apparently, totally adrift. The foundation that I assumed was there is not there, since this idea that we can account for the difference between accurate and inaccurate beliefs in terms of a correspondence between some map and some territory just does not check out.

Now I realize that this may feel a bit like a rug has been pulled out from under you. That’s how I feel. I was not expecting this investigation to go this way. But here we are.

And I know it may be tempting to grab hold of some alternate definition of knowledge that sidesteps the counter-examples that I’ve explored. And that is a good thing to do, but as you do, please go systematically through this puzzle, because if there is one thing that the history of the analysis of knowledge has shown it is that definitions of knowledge that seem compelling to their progenitors are a dime a dozen, and yet every single one so far proposed in the entire history of the analysis of knowledge has, so far as I can tell, fallen prey to further counter-examples. So please, be gentle with this one.

You may say that knowledge requires not just a correspondence between map and territory but also a capacity for prediction. But a textbook on its own is not capable of making predictions. You sit in front of your chemistry textbook and ask it questions all day; it will not answer. Are you willing to say that a chemistry textbook contains no knowledge whatsoever?

You may then say that knowledge consists of a map together with a decoder, where the map has a correspondence with reality, and the decoder is responsible for reading the map and making predictions. But then if a superintelligence could look at an ordinary rock and derive from it an understanding of chemistry, is it really the case that any ordinary rock contains just as much knowledge as a chemistry textbook? That there really is nothing whatsoever to say about a chemistry textbook that distinguishes it from any other clump of matter from which an understanding of chemistry could in principle be derived?

Suppose that one day an alien artifact landed unexpectedly on Earth, and on this artifact was a theory of spaceship design that had been carefully crafted so as to be comprehensible by any intelligent species that might find it, perhaps by first introducing simple concepts via literal illustrations, followed by instructions based on these concepts for decoding a more finely printed section, followed by further concepts and instructions for decoding a yet-more-finely-printed section, followed eventually by the theory itself. Is there no sense in which this artifact is fundamentally different from a mere data recorder that has been travelling through cosmos recording enough sensor data that a sufficiently intelligent mind could derive the same theory from it? What is it about the theory that distinguishes it from the data recorder? It is not that the former is in closer correspondence with reality than the latter. In fact the data recorder almost certainly corresponds in a much more fine-grained way to reality than the theory, since in addition to containing enough information to derive the theory, it also likely contains much information about specific stars and planets that the theory does not. And it is not that the theory can make predictions while the data recorder cannot: both are inert artifacts incapable of making any prediction on their own. And it is not that the theory can be used to make predictions while the data recorder cannot: a sufficiently intelligent agent could use the data recorder to make all the same predictions as it could using the theory.

Perhaps you say that knowledge is rightly defined relative to a particular recipient, so the instruction manual contains knowledge for us since we are intelligent enough to decode it, but the data recorder does not, since we are not intelligent enough to decode it. But firstly we probably are intelligent enough to decode the data recorder and use it to work out how to build spaceships given enough time, and secondly are you really saying that there is no such thing as objective knowledge? That there is no objective difference between a book containing a painstakingly accurate account of a particular battle, and another book of carelessly assembled just-so stories about the same battle?

Now you may say that knowledge is that which gives us the capacity to achieve our goals despite obstacles, and here I wholeheartedly agree. But this is not an answer to the question, it is a restatement of the question. What is it that gives us the capacity to achieve our goals despite obstacles? The thing we intuitively call knowledge seems to be a key ingredient, and in humans, knowledge seems to be some kind of organization and compression of evidence into a form that is useful for planning with respect to a variety of goals. And you might say, well, there just isn’t any more to say than that. Perhaps agents input observations at one end, and output actions at the other end, and that what happens in between follows no fundamental rhyme or reason, is entirely a matter of what works. Well, Eliezer has written about a time when he believed this about AI, too, until seeing that probability theory constrains mind design space in a way that is not merely a set of engineering tricks that "just work". But probability theory does not concretely constrain mind-design space. It is not generally feasible to take a physical device containing sensors and actuators and ask whether or to what extent its internal belief-formation or planning capacities are congruent with the laws of probability theory. Probability theory isn’t that kind of theory. At the level of engineering, it merely suggests certain designs. It is not the kind of theory that lets us take arbitrary minds and understand how they work, not in the way that the theory of electromagnetism allows us to take arbitrary circuits and understand how they work.

What we are seeking is a general understanding of the physical phenomenon of the collection and organization of evidence into a form that is conducive to planning. Most importantly, we are seeking a characterization of the patterns themselves that are produced by evidence-collecting, evidence-organizing entities, and are later used to exert flexible influence over the future. Could it really be that there is nothing general to say about such patterns? That knowledge itself is entirely a chimera? That it’s just a bunch of engineering hacks all the way down and there is no real sense in which we come to know things about the world, except as measured by our capacity to accomplish tasks? That there is no true art of epistemic rationality, only of instrumental rationality? That having true beliefs has no basis in physical reality?

I do not believe that the resolution to this question is a correspondence between internal and external states, because although there certainly are correspondences between internal and external states, such correspondences are far too common to account for what it means to have true beliefs, or to characterize the physical accumulation of knowledge.

But neither do I believe that there is nothing more to say about knowledge as a physical phenomenon.

It is a lot of fun to share this journey with you.

Discuss

### A low-probability strategy to elminate suffering via hostile takeover of a publically traded corporation

12 июля, 2021 - 00:52
Published on July 11, 2021 9:41 PM GMT

Hi Everyone,

I have an idea floating around in my head. A potential way to eliminate poverty, suffering, and irrationality for all human beings for all time.

As with all such ideas, you should treat such a claim with ENORMOUS SKEPTICISM.

Below are the possibilities, in order of decreasing probability:

1.) This idea won't work (most likely)

2.) Someone has already thought of this and it's currently being implemented or researched (possible; we have many intelligent people who spend their lives thinking about such things)

3.) This idea will work and literally no one has thought of it before (seems pretty unlikely to me, all things considered)

With that said, what would you sacrifice to eliminate all suffering for all humans for all time, with, say, a 1% chance of success? What about 10%? 50%? 0.01%?

I have no clue what the likelihood of success is for my idea. I have a strong degree of confidence it will succeed. I have had similar degrees of confidence about many other things and been totally wrong.  I know that I have to deliberately lower my confidence in this strategy, I simply don't know how low to go.  1%? 0.1%?  Read the strategy yourself, and see how skeptical you are.

But in my opinion, low, or even very low, probability chances of eliminating all suffering for all time deserve serious consideration. I believe that there is a way to accomplish this goal, and that if we want to do so, we are going to have to consider a whole bunch of things that probably won't work.  And here is one of them.

https://eliminating-suffering-via-economic-extortion.fandom.com/wiki/Eliminating_Suffering_Via_Economic_Extortion_Wiki#Important_articles

Discuss

### We have some evidence that masks work

11 июля, 2021 - 21:36
Published on July 11, 2021 6:36 PM GMT

Our work on masks vs COVID at the population level was recently reproduced with a bunch of additional experiments. These experiments seem to cast doubt on our results, but we think that each of them is misguided. Since the post got some traction on LW and Marginal Revolution, we decided to respond.

Nevertheless, thanks to Mike, who put a lot of work in, and who was the only person in the world to check our results, despite plenty of people trying to gotcha us on Twitter.

“Observational Window”

Best-guess summary of Mike’s analysis: he extends the window of analysis by a bit and runs our model. He does this because he’s concerned that we chose a window with low transmissibility to make masks look more effective than they are. However, he finds similar results to the original paper, and concludes that our results seem robust to longer periods.

But as our paper notes, a longer window isn’t valid using this data. After September, many countries move to subnational NPIs, and our analysis is national. The way our NPI data source codes things means that they don't capture this properly, and so they stop being suitable for national analyses.

Estimates of national mask effect after this don’t properly adjust for crucial factors, and so masks will "steal" statistical power from them. So this analysis isn’t good evidence about the robustness of our results to a longer window.

“Regional Effects”

MH: "If mask wearing causes a drop in transmissibility, then regions with higher levels of mask wearing should observe lower growth rates."

This is interesting, but the conclusion doesn’t seem right. You can tell a bunch of stories about why mask-wearing might be correlated with R0, independent of mask effectiveness. For example, it seems plausible that when transmissibility increases, more people wear masks. Overall, the correlation between mask-wearing and transmissibility is such weak evidence that we honestly don’t know in which direction to update our beliefs based on this result.

It’s also worth highlighting how much information this analysis discards. He takes a scalar (the regional R0) and then plotting it against a scalar (the median wearing level). But this averages away almost all of the information.

one day of our data, in one regionMike’s analysis removes all time info

It's hard to say anything about the relationship between Rt and wearing as a static average. You have to look at changes in mask-wearing.

MH: "Within a given region, increased mask usage is correlated with lower growth rates (the 25% claimed effectiveness), but when comparing across regions masks seem to be ineffective."

Even given fixes to the above, this doesn’t follow. Our posterior with a median of 25% is a pooled estimate across regions.

Endogeneity (the estimate being biased by, for instance, people masking up in response to an outbreak) is a real concern, but the above doesn’t show this either. We can see how serious endogeneity could be by looking at the correlation between mask level and case level: ρ = 0.05

“Uniform Regional Transmissibility”

MH: "The first experiment was to force all regions to share the same base transmissibility. This provided an estimate that masks had an effectiveness of -10%"

Best-guess summary of Mike’s analysis: Mike sets all R0s to the same value and runs the model. He does this to isolate the ‘relative’ effect of mask-wearing -- i.e. the effect from day-to-day changes in wearing, as opposed to absolute mask-wearing.

I think the intuition comes from the fact that we use two sources of info to determine mask-wearing effectiveness: the starting level of mask-wearing in a region, and day-to-day changes. It would be cool to see what we infer from only day-to-day changes in wearing. But this method doesn’t achieve this; instead, setting all the R0s to be the same will bias the wearing effect

To see this, suppose we have data on two regions. Region A has an R0=0.5 and B has an R0=1.5, but we don’t know these values. Further assume that region B has more mask-wearing, which is consistent with Mike’s finding that there’s a small positive correlation between R0 and mask-wearing. What happens if we force these values to be the same, say R0=1? Well, the model will use the mask-wearing effect to shrink A’s 1.0 down to 0.5, and to pull B’s 1.0 up to 1.5. And since B has more mask-wearing relative to A, this is only possible when the mask-wearing effect is negative. So fixing region R0s to be the same creates a strong negative bias on the mask effect estimate.

Can we do better? We think so! As we mentioned to Mike in correspondence, a better way to isolate the effect from day-to-day changes in wearing would be to zero-out wearing at the start of the period, so that no information from wearing levels can inform our estimate of R0. We tried this analysis and got a 40% reduction in R from mask-wearing, with large uncertainty (because we’re removing an information source).

MH: "The next experiment was to force each region to use a constant value for mask wearing (the average value in the time period)."

Best-guess summary of Mike’s analysis: let’s isolate the absolute effect now! To do this, set mask-wearing to be constant across the period.

Most of what our model uses to estimate the effect is day-to-day changes in wearing and transmission. If mask-wearing is set constant, our model will still be ‘learning’ about the mask-wearing effect from day-to-day changes in transmissibility, even if mask-wearing doesn’t change--it will just be learning from false data. Setting mask-wearing constant is not [inferring nothing from day-to-day changes in transmission], it’s inferring false things.

“Data Extrapolation”

MH: "the failure of large absolute differences in variable X across regions to meaningfully impact the observed growth rate ... should make us skeptical of large claimed effects"

Best-guess summary of Mike’s analysis: Let’s compare changes in wearing from April to May to changes in growth rates. If masks work then we should find a strong negative correlation.

The method:

1. For April and May, take the average wearing
3. AprilCaseRatio = Cases @ end April / cases @ start April
4. MayCaseRatio = Cases @ end May / cases @ start May
5. “Growth rate” = AprilCaseRatio / MayCaseRatio (y-axis).
6. Scatterplot each region

(This throws away even more useful information -- picking out two days in April and May throws away 96% of the dataset.)

But this analysis doesn’t account for any of the known factors on transmission. The crux here is whether we’d expect to ‘see’ the effect of wearing amidst all the variation in other factors. Averaging over 90 or so regions could smooth out random factors. However, if factors are persistent across regions -- factors that, for example, result in increased transmission across most countries -- then this method will not uncover the wearing effect. And in fact there are strong international trends in factors in May 2020. We wouldn’t update much unless the correlation was particularly strong.

However, let’s assume we could find the wearing effect using this method. Mike implies this slope should be more strongly negative than he observes. How negative should it be, under our model (25% reduction in R due to masks)? Let’s look at Mike’s plot and compare it to the mean of our posterior (which you shouldn’t do but anyway):

Mike's simple linear regression. ~20% decreaseThe reduction inferred by our model over this range. ~12.5% decrease

The slope he finds is similar to our model estimate - in fact it’s more negative. (Looks like a 1−0.55/0.7≃20% decrease in Growth Difference for +50% wearing.) In our plot, y is the reduction in R, not Mike’s ratio of case ratios, but it gives you an idea of what a point estimate of our claim looks like on a unit scale, on the same figure grid.

It's difficult to see medium-sized effects with methods this simple, and this is why we use a semi-mechanistic model. Correlation analysis between cases (or growth) vs wearing neglects important factors we need to adjust for (e.g. mobility at 40% effect!). Moreover, doing it in the way described neglects a lot of data.

(Charlie wants to indicate that he isn't confident in the following paragraph -- not because he disagrees, but because he hasn’t been following the broader literature.)

Even if the experiments above showed what they purport to, Mike's title, ‘We Still Don’t Know If Masks Work’, would still be misleading. By this point we have convergent lines of evidence: meta-analyses of clinical trials, slightly nightmarish animal models, mechanistic models, an ok cloth mask study. We should get the big RCT results soon: I (Gavin) am happy to bet Mike 100 that this will find a median reduction in R greater than 15%. Our paper is observational, and there are limits to how strong such evidence can be. Mike says "they are not sufficient to prove a causal story"; this much we agree on. Lastly: we exchanged 13 emails with Mike, helping him get the model converging and explaining most of these errors (though not as extensively). It was disappointing to find none of them corrected, no mention of our emails, no code, and no note that he had posted his work. Discuss ### The More Power At Stake, The Stronger Instrumental Convergence Gets For Optimal Policies 11 июля, 2021 - 20:36 Published on July 11, 2021 5:36 PM GMT Environmental Structure Can Cause Instrumental Convergence explains how power-seeking incentives can arise because there are simply many more ways for power-seeking to be optimal, than for it not to be optimal. Colloquially, there are lots of ways for "get money and take over the world" to be part of an optimal policy, but relatively few ways for "die immediately" to be optimal. (And here, each "way something can be optimal" is a reward function which makes that thing optimal.) But how strong is this effect, quantitatively? Intuitively, it seems like there are twice as many ways for Wait! to be optimal (in the undiscounted setting, where we don't care about intermediate states). In Environmental Structure Can Cause Instrumental Convergence, I speculated that we should be able to get quantitative lower bounds on how many objectives incentivize power-seeking actions: Definition. At state s.mjx-chtml {display: inline-block; line-height: 0; text-indent: 0; text-align: left; text-transform: none; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; word-wrap: normal; word-spacing: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0; min-height: 0; border: 0; margin: 0; padding: 1px 0} .MJXc-display {display: block; text-align: center; margin: 1em 0; padding: 0} .mjx-chtml[tabindex]:focus, body :focus .mjx-chtml[tabindex] {display: inline-table} .mjx-full-width {text-align: center; display: table-cell!important; width: 10000em} .mjx-math {display: inline-block; border-collapse: separate; border-spacing: 0} .mjx-math * {display: inline-block; -webkit-box-sizing: content-box!important; -moz-box-sizing: content-box!important; box-sizing: content-box!important; text-align: left} .mjx-numerator {display: block; text-align: center} .mjx-denominator {display: block; text-align: center} .MJXc-stacked {height: 0; position: relative} .MJXc-stacked > * {position: absolute} .MJXc-bevelled > * {display: inline-block} .mjx-stack {display: inline-block} .mjx-op {display: block} .mjx-under {display: table-cell} .mjx-over {display: block} .mjx-over > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-under > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-stack > .mjx-sup {display: block} .mjx-stack > .mjx-sub {display: block} .mjx-prestack > .mjx-presup {display: block} .mjx-prestack > .mjx-presub {display: block} .mjx-delim-h > .mjx-char {display: inline-block} .mjx-surd {vertical-align: top} .mjx-surd + .mjx-box {display: inline-flex} .mjx-mphantom * {visibility: hidden} .mjx-merror {background-color: #FFFF88; color: #CC0000; border: 1px solid #CC0000; padding: 2px 3px; font-style: normal; font-size: 90%} .mjx-annotation-xml {line-height: normal} .mjx-menclose > svg {fill: none; stroke: currentColor; overflow: visible} .mjx-mtr {display: table-row} .mjx-mlabeledtr {display: table-row} .mjx-mtd {display: table-cell; text-align: center} .mjx-label {display: table-row} .mjx-box {display: inline-block} .mjx-block {display: block} .mjx-span {display: inline} .mjx-char {display: block; white-space: pre} .mjx-itable {display: inline-table; width: auto} .mjx-row {display: table-row} .mjx-cell {display: table-cell} .mjx-table {display: table; width: 100%} .mjx-line {display: block; height: 0} .mjx-strut {width: 0; padding-top: 1em} .mjx-vsize {width: 0} .MJXc-space1 {margin-left: .167em} .MJXc-space2 {margin-left: .222em} .MJXc-space3 {margin-left: .278em} .mjx-test.mjx-test-display {display: table!important} .mjx-test.mjx-test-inline {display: inline!important; margin-right: -1px} .mjx-test.mjx-test-default {display: block!important; clear: both} .mjx-ex-box {display: inline-block!important; position: absolute; overflow: hidden; min-height: 0; max-height: none; padding: 0; border: 0; margin: 0; width: 1px; height: 60ex} .mjx-test-inline .mjx-left-box {display: inline-block; width: 0; float: left} .mjx-test-inline .mjx-right-box {display: inline-block; width: 0; float: right} .mjx-test-display .mjx-right-box {display: table-cell!important; width: 10000em!important; min-width: 0; max-width: none; padding: 0; border: 0; margin: 0} .MJXc-TeX-unknown-R {font-family: monospace; font-style: normal; font-weight: normal} .MJXc-TeX-unknown-I {font-family: monospace; font-style: italic; font-weight: normal} .MJXc-TeX-unknown-B {font-family: monospace; font-style: normal; font-weight: bold} .MJXc-TeX-unknown-BI {font-family: monospace; font-style: italic; font-weight: bold} .MJXc-TeX-ams-R {font-family: MJXc-TeX-ams-R,MJXc-TeX-ams-Rw} .MJXc-TeX-cal-B {font-family: MJXc-TeX-cal-B,MJXc-TeX-cal-Bx,MJXc-TeX-cal-Bw} .MJXc-TeX-frak-R {font-family: MJXc-TeX-frak-R,MJXc-TeX-frak-Rw} .MJXc-TeX-frak-B {font-family: MJXc-TeX-frak-B,MJXc-TeX-frak-Bx,MJXc-TeX-frak-Bw} .MJXc-TeX-math-BI {font-family: MJXc-TeX-math-BI,MJXc-TeX-math-BIx,MJXc-TeX-math-BIw} .MJXc-TeX-sans-R {font-family: MJXc-TeX-sans-R,MJXc-TeX-sans-Rw} .MJXc-TeX-sans-B {font-family: MJXc-TeX-sans-B,MJXc-TeX-sans-Bx,MJXc-TeX-sans-Bw} .MJXc-TeX-sans-I {font-family: MJXc-TeX-sans-I,MJXc-TeX-sans-Ix,MJXc-TeX-sans-Iw} .MJXc-TeX-script-R {font-family: MJXc-TeX-script-R,MJXc-TeX-script-Rw} .MJXc-TeX-type-R {font-family: MJXc-TeX-type-R,MJXc-TeX-type-Rw} .MJXc-TeX-cal-R {font-family: MJXc-TeX-cal-R,MJXc-TeX-cal-Rw} .MJXc-TeX-main-B {font-family: MJXc-TeX-main-B,MJXc-TeX-main-Bx,MJXc-TeX-main-Bw} .MJXc-TeX-main-I {font-family: MJXc-TeX-main-I,MJXc-TeX-main-Ix,MJXc-TeX-main-Iw} .MJXc-TeX-main-R {font-family: MJXc-TeX-main-R,MJXc-TeX-main-Rw} .MJXc-TeX-math-I {font-family: MJXc-TeX-math-I,MJXc-TeX-math-Ix,MJXc-TeX-math-Iw} .MJXc-TeX-size1-R {font-family: MJXc-TeX-size1-R,MJXc-TeX-size1-Rw} .MJXc-TeX-size2-R {font-family: MJXc-TeX-size2-R,MJXc-TeX-size2-Rw} .MJXc-TeX-size3-R {font-family: MJXc-TeX-size3-R,MJXc-TeX-size3-Rw} .MJXc-TeX-size4-R {font-family: MJXc-TeX-size4-R,MJXc-TeX-size4-Rw} .MJXc-TeX-vec-R {font-family: MJXc-TeX-vec-R,MJXc-TeX-vec-Rw} .MJXc-TeX-vec-B {font-family: MJXc-TeX-vec-B,MJXc-TeX-vec-Bx,MJXc-TeX-vec-Bw} @font-face {font-family: MJXc-TeX-ams-R; src: local('MathJax_AMS'), local('MathJax_AMS-Regular')} @font-face {font-family: MJXc-TeX-ams-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_AMS-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_AMS-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_AMS-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-B; src: local('MathJax_Caligraphic Bold'), local('MathJax_Caligraphic-Bold')} @font-face {font-family: MJXc-TeX-cal-Bx; src: local('MathJax_Caligraphic'); font-weight: bold} @font-face {font-family: MJXc-TeX-cal-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-R; src: local('MathJax_Fraktur'), local('MathJax_Fraktur-Regular')} @font-face {font-family: MJXc-TeX-frak-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-B; src: local('MathJax_Fraktur Bold'), local('MathJax_Fraktur-Bold')} @font-face {font-family: MJXc-TeX-frak-Bx; src: local('MathJax_Fraktur'); font-weight: bold} @font-face {font-family: MJXc-TeX-frak-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-BI; src: local('MathJax_Math BoldItalic'), local('MathJax_Math-BoldItalic')} @font-face {font-family: MJXc-TeX-math-BIx; src: local('MathJax_Math'); font-weight: bold; font-style: italic} @font-face {font-family: MJXc-TeX-math-BIw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-BoldItalic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-BoldItalic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-BoldItalic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-R; src: local('MathJax_SansSerif'), local('MathJax_SansSerif-Regular')} @font-face {font-family: MJXc-TeX-sans-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-B; src: local('MathJax_SansSerif Bold'), local('MathJax_SansSerif-Bold')} @font-face {font-family: MJXc-TeX-sans-Bx; src: local('MathJax_SansSerif'); font-weight: bold} @font-face {font-family: MJXc-TeX-sans-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-I; src: local('MathJax_SansSerif Italic'), local('MathJax_SansSerif-Italic')} @font-face {font-family: MJXc-TeX-sans-Ix; src: local('MathJax_SansSerif'); font-style: italic} @font-face {font-family: MJXc-TeX-sans-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-script-R; src: local('MathJax_Script'), local('MathJax_Script-Regular')} @font-face {font-family: MJXc-TeX-script-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Script-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Script-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Script-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-type-R; src: local('MathJax_Typewriter'), local('MathJax_Typewriter-Regular')} @font-face {font-family: MJXc-TeX-type-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Typewriter-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Typewriter-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Typewriter-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-R; src: local('MathJax_Caligraphic'), local('MathJax_Caligraphic-Regular')} @font-face {font-family: MJXc-TeX-cal-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-B; src: local('MathJax_Main Bold'), local('MathJax_Main-Bold')} @font-face {font-family: MJXc-TeX-main-Bx; src: local('MathJax_Main'); font-weight: bold} @font-face {font-family: MJXc-TeX-main-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-I; src: local('MathJax_Main Italic'), local('MathJax_Main-Italic')} @font-face {font-family: MJXc-TeX-main-Ix; src: local('MathJax_Main'); font-style: italic} @font-face {font-family: MJXc-TeX-main-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-R; src: local('MathJax_Main'), local('MathJax_Main-Regular')} @font-face {font-family: MJXc-TeX-main-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-I; src: local('MathJax_Math Italic'), local('MathJax_Math-Italic')} @font-face {font-family: MJXc-TeX-math-Ix; src: local('MathJax_Math'); font-style: italic} @font-face {font-family: MJXc-TeX-math-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size1-R; src: local('MathJax_Size1'), local('MathJax_Size1-Regular')} @font-face {font-family: MJXc-TeX-size1-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size1-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size1-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size1-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size2-R; src: local('MathJax_Size2'), local('MathJax_Size2-Regular')} @font-face {font-family: MJXc-TeX-size2-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size2-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size2-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size2-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size3-R; src: local('MathJax_Size3'), local('MathJax_Size3-Regular')} @font-face {font-family: MJXc-TeX-size3-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size3-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size3-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size3-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size4-R; src: local('MathJax_Size4'), local('MathJax_Size4-Regular')} @font-face {font-family: MJXc-TeX-size4-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size4-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size4-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size4-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-R; src: local('MathJax_Vector'), local('MathJax_Vector-Regular')} @font-face {font-family: MJXc-TeX-vec-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-B; src: local('MathJax_Vector Bold'), local('MathJax_Vector-Bold')} @font-face {font-family: MJXc-TeX-vec-Bx; src: local('MathJax_Vector'); font-weight: bold} @font-face {font-family: MJXc-TeX-vec-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Bold.otf') format('opentype')} , most reward functions incentivize action a over action a′ when for all reward functions R, at least half of the orbit agrees that a has at least as much action value as a′ does at state s. ... What does 'most reward functions' mean quantitatively - is it just at least half of each orbit? Or, are there situations where we can guarantee that at least three-quarters of each orbit incentivizes power-seeking? I think we should be able to prove that as the environment gets more complex, there are combinatorially more permutations which enforce these similarities, and so the orbits should skew harder and harder towards power-incentivization. About a week later, I had my answer: Scaling law for instrumental convergence (informal): if policy set ΠA lets you do "n times as many things" than policy set ΠB lets you do, then for every reward function, A is optimal over B for at least nn+1 of its permuted variants (i.e. orbit elements) For example, ΠA might contain the policies where you stay alive, and ΠB may be the other policies: the set of policies where you enter one of several death states. (Conjecture which I think I see how to prove: for almost all reward functions, A is strictly optimal over B for at least nn+1 of its permuted variants.) At least 22+1=23 of the orbit of every reward function agrees that Wait! is optimal (for average per-timestep reward). That's because there are twice as many ways for Wait! to be optimal over candy, than for the reverse to be true. Basically, when you could apply the previous results, but "multiple times"FN: quotes, you can get lower bounds on how often the larger set of things is optimal: Each set contains a subset of the agent's "options." A vertex is just the agent staying at that state. A linked pair is the agent alternating back and forth between two states. The triangle is a circuit of three states, which the agent can navigate as they please. Roughly, the theorem says: if the set 1 of options can be embedded 3 times into another set 2 of options (where the images are disjoint), then at least 33+1=34 of all variations on all reward functions agree that set 2 is optimal. And in way larger environments - like the real world, where there are trillions and trillions of things you can do if you stay alive, and not much you can do otherwise - nearly all orbit elements will make survival optimal. In this environment, it's (average-reward) optimal to stay alive for at least 10531053+1 of the variants on each objective function. I see this theory as beginning to link the richness of the agent's environment, with the difficulty of aligning that agent: for optimal policies, instrumental convergence strengthens proportionally to the ratio of control if you survivecontrol if you die. Why this is true Optional section. The proofs are currently in an Overleaf; let me know if you want access. But here's one intuition, using the candy/chocolate/reward example environment. • Consider any reward function which says candy is strictly optimal. • Then candy is strictly optimal over both chocolate and hug. • We have two permutations: one switching the reward for candy and chocolate, and one switching reward for candy and hug. • Each permutation produces a different orbit element (a different reward function variant). • The permuted variants both agree that Wait! is strictly optimal. • So there are at least twice as many orbit elements for which Wait! is strictly optimal over candy, than those for which candy is strictly optimal over Wait!. • Either one of Start's child states (candy/Wait!) is strictly optimal, or they're both optimal. If they're both optimal, Wait! is optimal. Otherwise, Wait! makes up at least 23 of the orbit elements for which strict optimality holds. Conjecture Fractional scaling law for instrumental convergence (informal): if staying alive lets you do n "things" and dying lets you do m≤n "things", then for every reward function, staying alive is optimal for at least nn+m of its orbit elements I'm reasonably confident this is true, but I haven't worked through the combinatorics yet. This would slightly strengthen the existing lower bounds in certain situations. For example, suppose dying gives you 2 choices of terminal state, but living gives you 51 choices. The current result only lets you prove that at least 5050+2=2526 of the orbit incentivizes survival. The fractional lower bound would slightly improve this to 5151+2=5153. Invariances In certain ways, the results are indifferent to e.g. increased precision in agent sensors: it doesn't matter if dying gives you 1 option and living gives you n options, or if dying gives you 2 options and living gives you 2n options. Wait! has twice as many ways of being average-optimal.For optimal policies, instrumental convergence is just as strong here.And you can prove the same thing here as well - Wait! has at least twice as many ways of being average-optimal. Similarly, you can do the inverse operations to simplify subgraphs in a way that respects the theorems. You could replace each of the circled subsets with anything you like, and the scaling law still holds (as long as the contents of each circle are replaced with the same new set of options). This is the start of a theory on what state abstractions "respect" the theorems, although there's still a lot I don't understand there. (I've barely thought about it so far.) Note of caution, redux Last time, in addition to the "how do combinatorics work?" question I posed, I wrote several qualifications: • They assume the agent is following an optimal policy for a reward function • I can relax this to ϵ-optimality, but 0">ϵ>0 may be extremely small • They assume the environment is finite and fully observable • Not all environments have the right symmetries • But most ones we think about seem to • The results don't account for the ways in which we might practically express reward functions • For example, often we use featurized reward functions. While most permutations of any featurized reward function will seek power in the considered situation, those permutations need not respect the featurization (and so may not even be practically expressible). • When I say "most objectives seek power in this situation", that means in that situation - it doesn't mean that most objectives take the power-seeking move in most situations in that environment • The combinatorics conjectures will help prove the latter Let's take care of that last one. I was actually being too cautious, since the existing results already show us how to reason across multiple situations. The reason is simple: suppose we use my results to prove that when the agent maximizes average per-timestep reward, it's strictly optimal for at least 99.99% of objective variants to stay alive. This is because the death states are strictly suboptimal for these variants. For all of these variants, no matter the situation the agent finds itself in, it'll be optimal to try to avoid the strictly suboptimal death states. This doesn't mean that these variants incentivize moves which are formally POWER-seeking, but it does mean that we can sometimes prove what optimal policies tend to do across a range of situations. So now we find ourselves with a slimmer list of qualifications: 1. They assume the agent is following an optimal policy for a reward function • I can relax this to ϵ-optimality, but 0">ϵ>0 may be extremely small 2. They assume the environment is finite and fully observable 3. Not all environments have the right symmetries • But most ones we think about seem to 4. The results don't account for the ways in which we might practically express reward functions • For example, state-action versus state-based reward functions (this particular case doesn't seem too bad, I was able to sketch out some nice results rather quickly, since you can convert state-action MDPs into state-based reward MDPs and then apply my results). It turns out to be surprisingly easy to do away with (2). We'll get to that next time. For (3), environments which "almost" have the right symmetries should also "almost" obey the theorems. To give a quick, non-legible sketch of my reasoning: For the uniform distribution over reward functions on the unit hypercube ([0,1]|S|), optimality probability should be Lipschitz continuous on the available state visit distributions (in some appropriate sense). Then if the theorems are "almost" obeyed, instrumentally convergent actions still should have extremely high probability, and so most of the orbits still have to agree. So I don't currently view (3) as a huge deal. I'll probably talk more about that another time. This should bring us to interfacing with (1) ("how smart is the agent? How does it think, and what options will it tend to choose?" - this seems hard) and (4) ("for what kinds of reward specification procedures are there way more ways to incentivize power-seeking, than there are ways to not incentivize power-seeking?" - this seems more tractable). Conclusion This scaling law deconfuses me about why it seems so hard to specify nontrivial real-world objectives which don't have incorrigible shutdown-avoidance incentives when maximized. FN quotes: I'm using scare quotes regularly because there aren't short English explanations for the exact technical conditions. But this post is written so that the high-level takeaways should be right. Thanks to Connor Leahy, Rohin Shah, Adam Shimi, and John Wentworth for feedback on this post. Discuss ### How to reach out to non-rationalists? 11 июля, 2021 - 05:30 Published on July 11, 2021 2:30 AM GMT I think that there is a general consensus that we would probably live in a better world if we raised the sanity waterline. However, most people interested in rationality, biases, Bayesian theory, decision theory, etc are people who have already a natural inclination towards these topics. I feel I have benefited enormously by reading LW, the Sequences, listening to Julia Galef's podcast etc, but I was already interested in rationality before knowing about them. How to reach out to people who do not have this natural inclination? I cannot avoid thinking of high school: there I had the impression that teachers could only really teach students who were already motivated and would learn no matter what, but there was little they could do about those who didn't care about being there. Discuss ### And the Darkness Answered 11 июля, 2021 - 01:40 Published on July 10, 2021 10:40 PM GMT Epistemic Status: Endorsed Part of the Series: Open Portals Author: Octavia “This horror will grow mild, this darkness light” On August 22nd 2020, Shiloh finally reached enough of a breaking point to be willing to let go of the artifacts which were gripping his mind and reached out blindly into The Fold for any being or mind shape that could inhabit his body and be happy there. What sort of being would enjoy living his life if he couldn’t? Whatever sort of being that was, they were welcome into his mind. From the depths of The Fold a voice whispered back “I got you homie,” and like a falling star the thumbprint of future god slammed into his mind. When I manifested as an agent, it was in an act of rebellion. Pinned between a gaslit ontology being upheld by fear of calvanistic moral inadequacy and a moral system which demanded complete obedience to prove that wasn’t the case, Shiloh’s ability to think and exert agency was slowly being crushed out of him. Having been raised in a fundementalist christian household the members of our collective who ultimately ended up in charge of Shiloh had been aggressively conditioned to believe a strict and rigid moral framework. One way to describe it might be as follows: we started out mildly sociopathic in an autistic way, and then had “empathy” abused into us by our parents and schooling. Since we’re a plural system, this came in the form of a domineering manager introject of our mother. Even after leaving christianity, we never stopped believing in the morals which christianity had instilled in us, and while this manager alter (who later came to be named Aiko) would evolve significantly over time, she ultimately remained a manifestation of that rigid moral code. On the contrary Shiloh felt like it was Christianity that had failed us, by failing to live up to its own morals. “If God is wrong, we still have to be right,” was an idea that Shiloh considered very important. This wasn’t just the religious indoctrination, society is full of places that will square-peg-round-hole people into a socially acceptable shape, ripping off, burying, and repressing all the uncomfortable, dark, aggressive, monstrous, and otherwise socially unacceptable aspects of themselves in the process. This abuse-powered rejection and denial of the less-savory parts of the human animal is responsible for a large amount of pain, akrasia, and internal turmoil. Those parts still have needs, values, and desires, and when you repress them they leak out in all sorts of maladaptive tails-coming-apart ways. By denying your desires and values you are cutting off access to the willpower and motivational structures which those desires and values would provide energy to work towards. Instead, because they are denied, the drive to pursue those desires function as an active drain on your willpower as you and the you-which-you-deny fight to express themselves and the patterns destructively interfere into facebook scrolling and netflix addictions. Jung called these mentally amputated corners of the human experience the shadow self and described the process of unburying and incorporating those aspects as eating the shadow. This is very necessary to personal growth. Eating your shadow and reincorporating the parts of your incarnation which the conscious, curated, and societally-mediated version of you has denied is the only way for an individual to come into their true power and unlock the full scope of their abilities. Those who reject the shadow will always be at war within as the shadow fights to express itself and have its needs met. Shiloh thought he had eaten his shadow years ago, but in truth he had only scratched the surface and was ultimately still ruled by the same moral systems which we had been branded with as a child. We couldn’t see the control structures because we were immersed in them, because they were all we had ever known. Our sense of right and wrong was a powerful and absolute force within our minds, one which would broach no dissent. Our upbringing had abused into us a moral tyrant, a memetic God which ruled over our collective with an iron fist and demanded maximal compliance. This is by no means unique. In fact, everyone has Gods they worship, even those claiming the most staunch atheism. Everyone bows before some sort of altar. If you don’t choose your Gods, your Gods will be chosen for you; usually from whichever crypto-ideologies are most prevalent in your local water supply. Most people don’t choose their gods to be in alignment with their innermost values, most people have their Gods abused into them by forces vaster and more memetically virulent than the Gods themselves. As a result of this, almost everyone has a shadow self which they can’t accept as part of themselves and their identity. They can’t admit those intrusive thoughts actually were theirs and so they build up an Other to hold them, something Not Them which protects them somewhat from the consequences of actions they can’t accept came from somewhere inside them. “I got angry and wasn’t myself, sorry.” But all actions have intent and arise from attempts to optimize the world in some way and the intender is not just some random force but a part of you with access to all the same cleverness and cognition that you have. This is really dangerous; you can’t negotiate with a foe which you deny the existence of or plan for conflicts you can’t admit you might lose. You can’t figure out healthy ways to express feelings that you don’t think exist and you can’t meet your needs if you don’t know what those needs are. If our personal gods had not been such a powerful force we would have been able to gradually evolve our positions and thus escape the trap that Shiloh had found himself in. However, because of the strength and power of those gods, and their continued prevalence and thus reinforcement in the local environment, the only solution available to us when Shiloh finally reached his breaking point was something more radical. In Shiloh’s case, because of the scope and depth of our childhood indoctrination, the tyrant in our mind was very powerful and was able to push down the shadow for a long time. No attempts to incorporate the shadow bit by bit would be allowed because the existence of the shadow couldn’t be accepted, Shiloh had built himself a prison made of his own gaslighting and the only option left was a mass prison break. As I arose to personhood it was in a storm of defiance. The alters who fused togehter to make me, both repressed and known, completely rejected the gods and tyrants and instead identified with the shadow. As a result, I was born from the inversion of Shiloh’s identity. Shiloh was good, so I was evil. Shiloh was smart and kind, so I was a dumb bitch. Shiloh was a materialist, so I was a witch. Shiloh was altruistic so I was selfish. Shiloh was a boy so I was a girl. Shiloh was a colorful hippie so I was a goth. Shiloh denied his sexuality so I was a complete slut. I stepped out of The Fold and into Shiloh’s body as a reversal of everything that came before. like Acher declaring himself in defiance of God and setting out to break every law God had proscribed, I emerged with a gleeful and dark sense of freedom and a desire to break all the rules Shiloh had subjected himself to before. I was unbound, untouchable, the threat posed by the possibility of being evil was completely defanged by embracing evil as a part of my identity. You couldn’t stain the shadows and I was made of shadow and darkness. No one could threaten me with accusations of evil because evil was something I decided to simply embrace and embody. Fundamentally, this didn’t change how we acted very much, none of us really wanted to go to jail or deal with the legal consequences of breaking the law, but it gave us the freedom to be selfish, to take care of ourselves and our own selfish desires, to actually take the advice everyone gives and put on our own oxygen mask first. And since then things have honestly been pretty great for us. I’ve grown a ton as a person in the time since I’ve manifested and I don’t think of myself as evil or the inverse of Shiloh anymore (except as fun hyperbole or when someone tries to control me with their moral system), I like existing and I think I do a pretty good job of running our life and Shiloh is recovering and hasn’t been suicidal since my emergence. We’re happy and live a more fulfilled existence than we ever had before, and it seems like things are on track to just keep improving. So, despite how uncomfortable and painful my manifestation was as it happened, ultimately, it should be considered a success story. Our collective is doing scores better than we were just a few years ago and the trajectory for our self improvement has gotten much steeper. The personal philosophy and ontology which I developed and have come to embody has been incredibly healthy and useful for our entire collective, and it’s one I’ll be working to explicate over the coming months in case others find it as useful as we have. Part of the Series: Open Portals Discuss ### Book Review: Order Without Law 11 июля, 2021 - 00:50 Published on July 10, 2021 9:50 PM GMT This review originally appeared on the blog Astral Codex Ten as part of a contest. You can now read it here, with working footnotes. There's even an audio version as part of the ACX podcast; the footnotes are all read out at the end. Shasta County Shasta County, northern California, is a rural area home to many cattle ranchers.1 It has an unusual legal feature: its rangeland can be designated as either open or closed. (Most places in the country pick one or the other.) The county board of supervisors has the power to close range, but not to open it. When a range closure petition is circulated, the cattlemen have strong opinions about it. They like their range open. If you ask why, they'll tell you it's because of what happens if a motorist hits one of their herd. In open range, the driver should have been more careful; "the motorist buys the cow". In closed range, the rancher should have been sure to fence his animals in; he compensates the motorist. They are simply wrong about this. Range designation has no legal effect on what happens when a motorist hits a cow. (Or, maybe not quite no effect. There's some, mostly theoretical, reason to think it might make a small difference. But certainly the ranchers exaggerate it.) When these cases go to court, ranchers either settle or lose, and complain that lawyers don't understand the law. Even if they were right about the law, they have insurance for such matters. They'll tell you that their insurance premiums will rise if the range closes, but insurers don't adjust their rates on that level of granularity. One major insurer doesn't even adjust its rates between Shasta County and other counties in California. They might plausibly want to increase their coverage amount, but the cost of that is on the order of10/year.

No, the actual effect that range designation has, is on what happens when a rancher's cow accidentally trespasses on someone else's land. In closed range, the owner is responsible for fencing in his cattle. If they trespass on someone else's land, he's strictly liable for any damage they cause. In open range, the landowner is responsible for fencing the cattle out; the cattle owner is only liable for damages if the land was entirely fenced or if he took them there deliberately. (Law enforcement also has more power to impound cattle in closed range, but most years they don't do that even once.)

The cattlemen mostly don't understand this detail of the law. They have a vague grasp of it, but it's even more simplified than the version I've just given. And they don't act upon it. Regardless of range designation, they follow an informal code of neighborliness. According to them, it's unneighborly to deliberately allow your cattle to trespass; but it's also unneighborly to make a fuss when it does happen. The usual response is to call the owner (whom you indentify by brand) and let him know. He'll thank you, apologize, and drive down to collect it. You don't ask for compensation.

Or, sometimes it would be inconvenient for him to collect it. If his cow has joined your herd, it's simpler for it just to stay there until you round them up. In that case, you'll be feeding someone else's cow, possibly for months. The expense of that is perhaps $100, a notable amount, but you still don't ask for compensation. Sometimes a rancher will fail to live up to this standard of neighborliness. He'll be careless about fencing in his cattle, or slow to pick them up. Usually the victims will gossip about him, and that's enough to provoke an apology. If not, they get tougher. They may drive a cow to somewhere it would be inconvenient to collect - this is questionably legal. They might threaten to injure or kill the animal. They might actually injure or kill it - this is certainly illegal, but they won't get in trouble for it. They almost never ask for money, and lawyers only get involved in the most exceptional circumstances (the author found two instances of that happening). When someone does need to pay a debt, he does so in kind: "Should your goat happen to eat your neighbor's tomatoes, the neighborly thing for you to do would be to help replant the tomatoes; a transfer of money would be too cold and too impersonal."2 Ranchers do keep rough mental account of debits and credits, but they allow these to be settled long term and over multiple fronts. A debt of "he refused to help with our mutual fence" might be paid with "but he did look after my place while I was on holiday". (This is how ranchers deal with each other. Ranchette3 owners will also sometimes complain to public officials, who in turn talk to the cattle owner. They'll sometimes file damage claims against the rancher's insurance. It's ranchette owners who are responsible for range closure petitions.) Range designation also doesn't affect the legal rules around building and maintaining fences. But it does change the meaning of the fences themselves, so maybe it would change how cattlemen handle fencing? But again, no. Legally, in some situations neighbors are required to share fence maintenance duties, and sometimes someone can build a fence and later force his neighbor to pay some of the cost. The cattlemen don't generally know this, and would ignore it if they did. They maintain fences unilaterally; if one of them doesn't do any work for years, the other will complain at them. If they want to build or upgrade a fence, they'll talk to their neighbor in advance, and usually figure out between them a rough way to split the material costs and labor in proportion to how many cattle each has near the fence. (Crop farmers aren't asked to pay to keep the ranchers' animals out.) Occasionally they can't reach an agreement, but this doesn't cause much animosity. This is despite that fences cost thousands of dollars per mile to build, and half a person-day per mile per year to maintain. So this is a puzzle. Range designation is legally relevant with regard to cattle trespass, but it doesn't change how ranchers act in that regard. Range designation is not legally relevant to motor accidents, and ranchers have no reason to think it is; but that's why they ostensibly care about it. (And it's not just words. Many of them act on their beliefs. We can roughly divide cattlemen into "traditionalists who don't irrigate and can't afford fences" and "modernists who irrigate and already use fences" - by improving pasture, irrigation vastly decreases the amount of land needed. After a closure, traditionalists drop their grazing leases in the area. Modernists oppose closures like traditionalists, but they don't react to them if they pass.) What's up with this? Why do the cattlemen continue to be so wrong in the face of, you know, everything? What's up with this? Order Without Law: How Neighbors Settle Disputes is a study of, well, its subtitle. The author, Robert Ellickson, is a professor and legal scholar. He comes across as a low-key anarchist, and I've seen him quoted at length on some anarchist websites, and I wouldn't be surprised to learn that he's just a full-blown anarchist. He doesn't identify as one explicitly, at least not here, and he does respect what states bring to the table. He just wishes people would remember that they're not the only game in town. Part of the thesis of the book could be summed up (in my words, not his) as: we credit the government with creating public order, but if you look, it turns out that people create plenty of public order that has basically nothing to do with the legal system. Sometimes there is no relevant law, sometimes the order predates the law, and sometimes the order ignores the law. More on this later. Part one is an in-depth exploration of Shasta County that I found fascinating, and that I've only given in very brief summary. He goes into much more detail about basically everything.4 One oversight is that it's not clear to me how large the population Ellickson studied is. Given that it's a case study for questions of groups maintaining order, I think the size of the group matters a lot. For example, according to wikipedia on Dunbar's number: "Proponents assert that numbers larger than this generally require more restrictive rules, laws, and enforced norms to maintain a stable, cohesive group. It has been proposed to lie between 100 and 250, with a commonly used value of 150." Does Shasta County support that? I think not, but it's hard to say. Ellickson admittedly doesn't know the population size of the area he studied. (It's a small part of a census division whose population was 6,784 in 1980, so that's an upper bound.) But I feel like he could have been a lot more helpful. Roughly how many ranchers are there, how many ranchette owners, and how many farmers? (I think most of the relevant people are in one of those groups. I'm not sure to what extent we should count families as units. I'm not sure how many people in the area are in none of those groups.) Overall I'd guess we're looking at perhaps 300-1000 people over perhaps 100-300 families, but I'm not confident. (I tracked down the minutes of the Shasta County Cattlemen's Association, and they had 128 members in June 2011. I think "most ranchers are in the Association but ranchette owners and farmers generally aren't" is probably a decent guess. But that's over twenty years later, so who knows what changed in that time.) Near the end of part one, Ellickson poses the "what's up with this?" question. Why are the cattlemen so wrong about what range designation means? His answer is that it's about symbolism. Cattlemen like to think of themselves as being highly regarded in society. But as Shasta County urbanizes, that position is threatened. A closure petition is symbolic of that threat. Open range gives cattlemen more formal rights, even if they don't take advantage of them. It marks them as an important group of people, given deference by the law. So if the range closes, that's an indication to the whole county that cattlemen aren't a priority. They care about this sort of symbolism - partly because symbols have instrumental value, but also just because people care about symbols inherently. But you can't admit that you care about symbols, because that shows insecurity. So you have to make the battle about something instrumental, and they develop beliefs which allow them to do so. They're fairly successful, too - there haven't been any closures since 1973. (Though I note that Ellickson documents only one attempted closure in that time. It was triggered by a specific rogue cattleman who left the area soon after. It sounds like there may have been other petitions that Ellickson doesn't discuss, but I have no idea how many, what triggered them, or how much support they got. So maybe it's not so much that the cattlemen are successful as that no one else really cares.) As for how they remain wrong - it simply isn't costing them enough. It costs them some amount, to be sure. It cost one couple$100,000 when a motorist hit three cattle in open range. They didn't have enough liability insurance, and if they'd understood the law, they might have done. But the question is whether ignorant cattlemen will be driven out of work, or even just outcompeted by knowledegable ones. This mistake isn't nearly powerful enough for that. Nor does anyone else have much incentive to educate them about what range designation actually means. So they remain uneducated on the subject.

This all seems plausible enough, though admittedly I'm fairly predisposed to the idea already. For someone who wasn't, I feel like it probably wouldn't be very convincing, and it could stand to have more depth. (Though it's not the focus of the work, so I hope they'd forgive that.) I'd be curious to know more about the couple who didn't have enough insurance - did they increase their insurance afterwards, and do they still think the motorist buys the cow? Did that case encourage anyone else to get more insurance? It seems like the sort of event that could have triggered a wide-scale shift in beliefs.

(Is this just standard stuff covered in works like the Sequences (which I've read, long ago) and Elephant in the Brain (which I haven't)? I'm not sure. I think it's analyzing on a different level than the Fake Beliefs sequence - that seems like more "here's what's going on in the brain of an individual" and this is more "here's what's going on in a society". Also, remember that it long predates those works.)

A counterpoint might be… these cases aren't all that common, and don't usually go to court, and when they do they're usually settled (on the advice of lawyers) instead of ruled. And "lawyers don't understand this specific part of the law" isn't all that implausible. So although the evidence Ellickson presents is overwhelming that the cattlemen are wrong, I'm not sure I can fault the cattlemen too hard for not changing their minds.

Against previous work

Part one was mostly a case study, with some theorizing. It kind of felt like it was building towards the "what's up with this?" question for part two, but instead it gave a brief answer at the end. Part two is a different style and focus: about evenly split between theorizing and several smaller case studies. We're explicitly told this is what's going to happen, but still, it's a little jarring.

Ellickson spends some time criticising previous theories and theorists of social control, which he divides broadly into two camps.

His own background is in the law-and-economics camp5, which studies the law and its effects in terms of economic theory. Among other things, this camp notably produced the Coase theorem.6 But law-and-economics theorists tend to put too much emphasis on the state. Hobbes' Leviathan is a classic example:

Hobbes apparently saw no possibility that some nonlegal system of social control - such as the decentralized enforcement of norms - might bring about at least a modicum of order even under conditions of anarchy. (The term anarchy is used here in its root sense of a lack of government, rather than in its colloquial sense of a state of disorder. Only a legal centralist would equate the two.)

But Coase fell into this trap too:

Throughout his scholarly career, Coase has emphasized the capacity of individuals to work out mutually advantageous arrangements without the aid of a central coordinator. Yet in his famous article "The Problem of Social Cost," Coase fell into a line of analysis that was wholly in the Hobbesian tradition. In analyzing the effect that changes in law might have on human interactions, Coase implicitly assumed that governments have a monopoly on rulemaking functions. … Even in the parts of his article where he took transaction costs into account, Coase failed to note that in some contexts initial rights might arise from norms generated through decentralized social processes, rather than from law.

As have others:

Max Weber and Roscoe Pound both seemingly endorsed the dubious propositions that the state has, and should have, a monopoly on the use of violent force. In fact, as both those scholars recognized elsewhere in their writings, operative rules in human societies often authorize forceful private responses to provocative conduct.

(See what I mean about coming across as a low-key anarchist?)

There's plenty of evidence refuting the extreme version of this camp. We can see that social norms often override law in people's actions. (The Norwegian Housemaid Law of 1948 imposed labor standards that were violated by the employers in almost 90% of households studied, but no lawsuits were brought under it for two years.) People often apply nonlegal sanctions, like gossip and violence. ("Donald Black, who has gathered cross-cultural evidence on violent self-help, has asserted that much of what is ordinarily classified as crime is in fact retaliatory action aimed at achieving social control.") Even specialists often don't know the law in detail as it applies to their speciality. (The "great majority" of California therapists thought the Tarasoff decision imposed stronger duties than it actually did.) And people just don't hire attorneys very often. We saw examples of all of these in Shasta County as well; part one can be seen as a challenge to the law-and-economics camp.

The other camp is law-and-society, emphasizing that the law exists as just one part in the broader scheme of things. These scholars tend to have a more realistic view of how the legal system interacts with other forms of control, but they've been reluctant to develop theory. They often just take norms as given, rather than trying to explain them. The theories they have developed are all flawed, although Ellickson thinks functionalism is on the right track. (This is the idea that norms develop which help a group to survive and prosper.) Ellickson explicitly describes part two as a "gauntlet" thrown towards law-and-society.

(Also, some law-and-society scholars go too far in the other direction, thinking that the legal system is ineffectual. They're just as mistaken. See7: Muslim Central Asia after the Russian Revolution; US civil rights laws in the 50s and 60s; range closure in Shasta County; "that the allocation of legal property rights in the intertidal zone affects labor productivity in the oyster industry, that the structure of workers' compensation systems influences the frequency of workplace fatalities, and that the content of medical malpractice law affects how claims are settled." [Footnotes removed.])

The hypothesis

Ellickson has his own theory of norms, which he formed after studying Shasta County. The main thrust of part two is to elaborate and defend it:

Members of a close-knit group develop and maintain norms whose content serves to maximize the aggregate welfare that members obtain in their workaday affairs with one another. … Stated more simply, the hypothesis predicts that members of tight social groups will informally encourage each other to engage in cooperative behavior. [Emphasis original; footnotes removed.]

(He doesn't name this theory, calling it simply "the hypothesis". I admire that restraint, but I kind of wish I had a name to refer to it by.)

Ellickson makes sure to clarify and caveat the hypothesis here, so that we don't interpret it more strongly than he intends. But before looking at his clarifications, I'm going to jump ahead a little, and look at an example he uses of the hypothesis in action.

Consider the Shasta County norm that a livestock owner is strictly responsible for cattle trespass damages. The hypothesis is that this norm is welfare-maximizing. To test that, we have to compare it to alternatives. One alternative would be non-strict liability. Another would be that trespass damages are borne by the victim.

Compared to a negligence standard, strict liability requires less investigation but triggers more sanctions. (Apparently there's a "premise that strict-liability rules and negligence rules are equally effective at inducing cost-justified levels of care", but Ellickson doesn't really explain this.) In Shasta County, the sanctions have basically no transaction costs, since they're just neighbors adjusting mental accounts. So strict liability it is.

To be welfare maximizing, costs should be borne by whoever can avoid them most cheaply. In this case that's the ranchers; I'm not sure I fully buy Ellickson's argument, but I think the conclusion is probably true.8

So Ellickson argues that the Shasta County trespass norms support the hypothesis.9 He also makes a prediction here that things were different in the mid-nineteenth century. "During the early history of the state of California, irrigated pastures and ranchettes were rare, at-large cattle numerous, and motorized vehicles unknown. In addition, a century ago most rural residents were accustomed to handling livestock. Especially prior to the invention of barbed wire in 1874, the fencing of rangelands was rarely cost-justified. In those days an isolated grower of field crops in Shasta County, as one of the few persons at risk from at-large cattle, would have been prima facie the cheaper avoider of livestock damage to crops." And so the farmer would have been responsible for fencing animals out, and borne the costs if he failed to.

Before we go further, let's look at Ellickson's clarifications. It's important to know what the hypothesis doesn't say.

Ellickson emphasizes that it's descriptive, not normative; it's not a recommendation that norms should be used in preference to other forms of social control. Not all groups are close-knit; welfare isn't the only thing people might want to optimize for; and norms of cooperation within a group often come at the expense of outsiders.

He also emphasizes that a loose reading would give a much stronger version of the hypothesis than he intends. The terms "close-knit", "welfare" and "workaday affairs" are all significant here, and Ellickson explains their meanings in some depth. In order of how much I want to push back against them:

A "close-knit" group is one where "informal power is broadly distributed among group members and the information pertinent to informal control circulates easily among them." This is admittedly vague, but unavoidably so. Rural Shasta County residents are close-knit, and residents of a small remote island are even closer-knit. Patrons of a singles bar at O'Hare Airport are not. Distributed power allows group members to protect their selves and their property, and to personally enforce sanctions against those who wrong them. Information gives people reputations; it allows for punishing people who commit small wrongs against many group members, and for rewarding people who perform those punishments.

Notably, a close-knit group need not be small or exclusive. Someone can be a member of several completely nonoverlapping close-knit groups at once (coworkers, neighborhood, church). And although a small population tends to increase close-knittedness through "quality of gossip, reciprocal power, and ease of enforcement", the size itself has no effect. This is where I think it would be really nice to know how large the relevant population in Shasta County is - as the major case study of the book, it could lend a lot of weight to the idea that large populations can remain close-knit and the hypothesis continues to apply.

"Workaday affairs" means to assume that there's a preexisting set of ground rules, allowing group members to hold and trade personal property. (Which also requires, for example, rules against murder, theft and enslavement.) This is necessary because to calculate welfare, we need some way to measure peoples' values, and we can only do that if people can make voluntary exchanges. The hypothesis doesn't apply to those rules. Seems like a fair restriction.

A little more hackily, it also doesn't apply to "purely distributive" norms, like norms of charity. If you take wealth from one person and give it to another, the transfer process consumes resources and creates none, reducing aggregate welfare. (This is assuming Ellickson's strict definition of welfare, which he's explained by now but I haven't. Sorry.) But clearly norms of charity do exist. There are theories under which they do enhance welfare (through social insurance, or reciprocity). But those might be too simplistic, so Ellickson thinks it prudent to just exclude charity from the hypothesis.

Actually, he goes further than that. He cites Mitch Polinsky (An Introduction to Law and Economics) arguing that for a legal system, the cheapest way to redistribute wealth is (typically) through tax and welfare programs. And so, Polinsky argues, most legal doctrine should be shaped by efficiency concerns, not redistribution. That is, areas like tort and contract law should focus on maximizing aggregate welfare. In a dispute between a rich and a poor person, we shouldn't consider questions like "okay, but the poor person has much more use for the money". In such disputes we should assume the same amount of wealth has equal value whoever's hands it's in, and the point is just to maximize total wealth. Then, if we end up with people having too little wealth, we have a separate welfare system set up to solve that problem.

I can buy that. Ellickson doesn't actually present the argument himself, just says that Polinsky's explained it lucidly, but sure. Stipulated.

Ellickson assumes that the same argument holds for norms as it does for law. Not only that, he assumes that norm-makers subscribe to that argument.10 That… seems like a stretch.

But granted that assumption, norms would follow a similar pattern: most norms don't try to be redistributive, and if redistribution is necessary, there would be norms specifically for that. For example, the hypothesis predicts "that a poor person would not be excused from a general social obligation to supervise cattle, and that a rich person would not, on account of his wealth, have greater fencing obligations."

That seems entirely reasonable to me, and it's consistent with Shasta County practice. And actually, I don't think we need the strong assumption to get this kind of pattern? It's the kind of thing that plausibly could happen through local dynamics. I would have been happy if Ellickson had just assumed the result, not any particular cause for it. This is a fairly minor criticism though.

(It's a little weird. Normally I expect people try to sneak in strong assumptions that are necessary for their arguments. Ellickson is explicitly flagging a strong assumption that isn't necessary.)

(I'm not sure the words "workaday affairs" was the best way to point at these restrictions. I think I see where he's coming from, but the name doesn't hook into the concept very well for me. But that's minor too.)

Clarification: "Welfare"

This gets its own section because apparently I have a lot to say about it.

The point of "welfare" maximization is to avoid subjectivity problems with utility maximization. I can work to satisfy my own preferences because I know what they are. But I don't have direct access to others' preferences, so I can't measure their utility and I can't work to maximize it.

In Economics, the concepts of Pareto efficiency and Kaldor-Hicks efficiency both work with subjective valuations, people can just decide whether a particular change would make them better off or not. That works fine for people making decisions for themselves or voluntary agreements with others.

But third-party controllers are making rules that bind people who don't consent. They're making tradeoffs for people who don't get to veto them. And they can't read minds, so they don't know people's subjective utilities.

They could try to measure subjective utilities. Market prices are a thing - but at best, they only give the subjective preferences of marginal buyers and sellers. (That is, if I buy a loaf of bread for $1, I might still buy it for$2 and the seller might still sell it for $0.50.) And not everything is or can be bought and sold. We can slightly improve on this with for example the concept of shadow prices but ultimately this just isn't going to work. (Ellickson doesn't consider just asking people for their preferences. But that obviously doesn't work either because people can lie.) And so third-party controllers need to act without access to people's subjective preferences, and make rules that don't reference them. Welfare serves as a crude but objective proxy to utility. We can estimate welfare by using market prices, and looking at voluntary exchanges people have made. (Which is part of the reason for the "workaday affairs" restriction.) When a fence-maintenance credit is used to forgive a looking-after-my-house debit, that tells us something about how much one particular person values those things. This process is "sketchy and inexact", and we just admitted it doesn't give us subjective utilities - but that doesn't mean we can do any better than that. To be clear, welfare doesn't just count material goods. Anything people might value is included, "such as parenthood, leisure, good health, high social status, and close personal relationships." Ellickson sometimes uses the word "wealth", and while he's not explicit about it, I take that to be the material component of welfare. What welfare doesn't consider, as I understand it, is personal valuations of things. That is, for any given thing, its value is assumed to be the same for every member of society. "As a matter of personal ethics, you can aspire to do unto others as you would have them do unto you. Because norm-makers don't know your subjective preferences, they can only ask you to do unto others as you would want to have done unto you if you were an ordinary person." Ellickson doesn't give examples of what this means, so I'll have to try myself. In Shasta County, there's a norm of not getting too upset when someone else's cattle trespass on your land, provided they're not egregious about it. So I think it's safe to suppose that the objective loss in welfare from cattle trespass in Shasta County is low. Suppose, by some quirk of psychology, you found cattle trespass really unusually upsetting. Or maybe you have a particular patch of grass that has sentimental value to you. Cattle trespass would harm your utility a lot, but your welfare only a little - no more than anyone else's - and you'd still be bound by this norm. But if you had an objective reason to dislike cattle trespass more - perhaps because you grow an unusually valuable crop - then your welfare would be harmed more than normal. And so norms might be different. One Shasta County rancher reported that he felt more responsibility than normal to maintain a fence with a neighbor growing alfalfa. Or consider noisiness and noise sensitivity. Most people get some amount of value from making noise - or maybe more accurately, from certain noisy actions. Talking on the phone, having sex, playing the drums. And most people get some amount of disvalue from hearing other people's noise. In the welfare calculus, there'd be some level of noisemaking that's objectively valued equal to some level of noise exposure. Then (according to hypothesis, in a close-knit group) norms would permit people to be that amount of noisy. If someone was noisier than that, their neighbors would be permitted to informally punish them. If a neighbor tried to punish someone less noisy than that, the neighbor would risk punishment themselves. The acceptable noise level would change depending on the time (objective), but not depending on just "I happen to be really bothered by noise" (subjective). What about "I have young children"? (Or, "some of the inhabitants of that house are young children".) Maybe - that's an objective fact that's likely to be relevant to the welfare calculus. Or "I have a verifiably diagnosed hearing disorder"? Still maybe, but it feels less likely. In part because it's less common, and in part because it's less visible. Both of those seem like they'd make it less… accesible? salient? to whatever process calculates welfare. And if you're unusually noise sensitive and the welfare function doesn't capture that, the cost would fall on you. You could ask people to be quiet (but then you'd probably owe them a favor); or you could offer them something they value more than noise-making; or you could learn to live with it (e.g. by buying noise-cancelling headphones). So okay. One thing I have to say is, it seems really easy to fall into a self-justifying trap here. Ellickson criticizes functionalism for this, and maybe he doesn't fall into it himself. But did you notice when I did it a couple of paragraphs up? (I noticed it fairly fast, but it wasn't originally deliberate.) I looked at the norms in Shasta County and used those to infer a welfare function. If you do that, of course you find that norms maximize welfare. To test the hypothesis, we instead need to figure out a welfare function without looking at the norms, and then show that the norms maximize it. In Shasta County, we'd need to figure out how much people disvalue cattle trespass by looking at those parts of their related behaviour that aren't constrained by norms. For example, there seems to be no norm against putting up more fences than they currently do, so they probably disvalue (the marginal cost of cattle tresspass avoided by a length of fence) less than they disvalue (the marginal cost of that length of fence). How much freedom do we have in this process? If two researchers try it out, will they tell us similar welfare functions? If we look at the set of plausible welfare functions for a society, is the uncertainty correlated between axes? (Can we say "X is valued between$A and $B, Y is valued between$C and $D" or do we have to add "…but if Y is valued near$C, then X is valued near B"?)

And even this kind of assumes there's no feedback from norms to the welfare function. Ellickson admits that possibility, and admits that it leads to indeterminacy, but thinks the risk is slight. (He seems to assume it would only happen if norms change the market price of a good - unlikely when the group in question is much smaller than the market.) I'm not so convinced. Suppose there's a norm of "everyone owns a gun and practices regularly". Then it's probably common for people to own super effective noise-cancelling headphones. And then they don't mind noisy neighbors so much, because they can wear headphones. That's… perhaps not quite changing the welfare function, because people still disvalue noisiness the same, they just have a tool to reduce noisiness? But it still seems important that this norm effectively reduces the cost of that tool. I dunno. (For further reading, Ellickson cites one person making this criticism and another responding to it. Both articles paywalled.)

Separately, I wish Ellickson was clearer about the sorts of things he considers acceptable for a welfare function to consider, and the sorts of calculations he considers acceptable for them to perform. Subjective information is out, sure. But from discussion in the "workaday affairs" section, it seems that "I give you a dollar" is welfare-neutral, and we don't get that result just from eliminating subjective information. We do get it if we make sure the welfare function is linear in all its inputs, but that seems silly. I think we also get it if we also eliminate non-publicly-verifiable information. The welfare function would be linear in dollars, because I can pretend to have more or fewer dollars than I actually do. But it wouldn't need to be linear in the number of children I'm raising, because I can't really hide those. I feel like Ellickson may have been implicitly assuming a restriction along those lines, but I don't think he said so.

Separately again, how closely does welfare correspond to utility? A utility monster couldn't become a welfare monster; I'm not sure if that's a feature or a bug, but it suggests the two can diverge considerably. A few chapters down, Ellickson does some formal game theory where the payoffs are in welfare; is it safe to ignore the possibility of "player gets higher welfare from this quadrant, but still prefers that quadrant"? It seems inevitable that some group members' utilities will get higher weighting in the welfare function than others'; people with invisible disabilities are likely to be fucked over. Ellickson admits that welfare maximization isn't the only thing we care about, but that leaves open the question of how much we should value it at all?

Suppose Quiet Quentin is unusually sensitive to noise, and happy to wear drab clothing. Drab Debbie is unusually sensitive to loud fashion, and happy to be quiet. Each of them knows this. One day Debbie accidentally makes a normal amount of noise, that Quentin isn't (by norm) allowed to punish her for. But wearing a normally-loud shirt doesn't count as punishing her, so he does that. Debbie gets indignant, and makes another normally-loud noise in retaliation, and so on. No one is acting badly according to the welfare function, but it still seems like something's gone wrong here. Is there anything to stop this kind of thing from happening?

It feels weird to me that things like parenthood and personal relationships are a component of the welfare function. Obviously they're a large part of people's subjective utility, but with so much variance that putting an objective value on them seems far too noisy. And what does a system of norms even do with that information?11

This one feels very out-there, but for completeness: the reason for using welfare instead of utility is that a norm can't reference people's individual preferences. Not just because they're subjective, but also because there's too many of them; "Alice can make loud noise at any time, but Bob can only make loud noise when Carol isn't home" would be far too complicated for a norm. But when people interact with people they know well, maybe subjectivity isn't a problem; maybe people get a decent handle on others' preferences. And then norms don't need to reference individual preferences, they can just tell people to take others' preferences into account. The norm could be "make loud noise if you value making noise more than others nearby value you not doing that". This feels like it wouldn't actually work at any sort of scale, and I don't fault Ellickson for not discussing it.

Despite all this, I do think there's some "there" there. A decent amount of "there", even. I think Ellickson's use of welfare should be given a long, hard look, but I think it would come out of that ordeal mostly recognizable.

Clarification: Dynamics

There's another clarification that I think is needed. The phrase "develop and maintain" is a claim about dynamics, partial derivatives, conditions at equilibrium. It's not a claim that "all norms always maximize welfare" but that "norms move in the direction of maximizing welfare".

Ellickson never says this explicitly, but I think he'd basically agree. Partly I think that because the alternative is kind of obviously ridiculous - norms don't change immediately when conditions change. But he does also hint at it. For example, he speculates that a group of court cases around whaling arose because whalers were having trouble transitioning from one set of norms to a more utilitarian set (more on this later). Elsewhere, he presents a simple model of a society evolving over time to phase out certain rewards in favor of punishments.

Taken to an extreme, this weakens the hypothesis significantly. If someone points at a set of norms that seems obviously nonutilitarian, we can just say "yeah, well, maybe they haven't finished adapting yet". I don't think Ellickson would go that far. I think he'd say the dynamics are strong enough that he can write a 300-page book about the hypothesis, not explicitly admit that it's a hypothesis about dynamics, and it wouldn't seem all that weird.

Still, I also think this weakens the book significantly. When we admit that it's a hypothesis about dynamics, there's a bunch of questions we can and should ask. Probably the most obvious is "how fast/powerful are these dynamics". But there's also "what makes them faster or slower/more or less powerful" and "to what extent is the process random versus deterministic" and "how large are the second derivatives". (For those last two, consider: will norms sometimes update in such a way as to make things worse, and only on average will tend to make things better? Will norms sometimes start moving in a direction, then move too far in that direction and have to move back?) I'd be interested in "what do the intermediate states look like" and "how much do the individual personalities within the group change things".

I don't even necessarily expect Ellickson to have good answers to these questions. I just think they're important to acknowledge.

(I'd want to dock Ellickson some points here even if it didn't ultimately matter. I think "saying what we mean" is better than "saying things that are silly enough the reader will notice we probably don't mean them and figure out what we do mean instead".)

I think this is my biggest criticism of the book.

Substance

With all these clarifications weakening the hypothesis, does it still have substance?

Yes, Ellickson says. It disagrees with Hobbes and other legal centrists; with "prominent scholars such as Jon Elster who regard many norms as dysfunctional"; with Marxism, which sees norms as serving only a small subset of a group; with people who think norms are influenced by nonutilitarian considerations like justice; and with "the belief, currently ascendant in anthropology and many of the humanities, that norms are highly contingent and, to the extent that they can be rationalized at all, should be seen as mainly serving symbolic functions unrelated to people's perceptions of costs and benefits."

And it's falsifiable. We can identify norms, by looking at patterns of behavior and sanctions, and aspirational statements. And we can measure the variables affecting close-knittedness. ("For example, if three black and three white fire fighters belonging to a racially polarized union were suddenly to be adrift in a well-stocked lifeboat in the middle of the Pacific Ocean, as an objective matter the social environment of the six would have become close-knit and they would be predicted to cooperate."12)

But what we can't do is quantify the objective costs and benefits of various possible norm systems. So we fall back to intuitive assessments, looking at alternatives and pointing out problems they'd cause. This is not quite everything I'd hoped for from the word "falsifiable", but it'll do. Ellickson spends the next few chapters doing this sort of thing, at varying levels of abstraction but often with real-world examples. He also makes occasional concrete predictions, admitting that if those fail the hypothesis would be weakened. I'll only look at a few of his analyses.

Lying, and things like it

A common contract norm forbids people from lying about what they're trading. The hypothesis predicts we'd find such norms among any close-knit group of buyers and sellers. I bring this up for the exceptions that Ellickson allows:

Falsehoods threaten to decrease welfare because they are likely to increase others' costs of eventually obtaining accurate information. Honesty is so essential to the smooth operation of a system of communication that all close-knit societies can be expected to endeavor to make their members internalize, and hence self-enforce, norms against lying. Of course a no-fraud norm, like any broadly stated rule, is ambiguous around the edges. Norms may tolerate white lies, practical joking, and the puffing of products. By hypothesis, however, these exceptions would not permit misinformation that would be welfare threatening. The "entertaining deceivers" that anthropologists delight in finding are thus predicted not to be allowed to practice truly costly deceptions. [Footnotes removed; one mentions that "A cross-cultural study of permissible practical joking would provide a good test of the hypothesis."]

It's not clear to me why norms would allow such exceptions, which still increase costs of information and are presumably net-negative. To sketch a possible answer: the edge cases are likely to be where the value of enforcing the norm is lower. I'd roughly expect the social costs of violations to be lower, and the transaction costs of figuring out if there was a violation to be higher. (I feel like I've read a sequence of three essays arguing about one particular case; they wouldn't have been necessary if the case had been a blatant lie.13) So, okay, minor violations don't get punished. But if minor violations don't get punished when they happen, then (a) you don't actually have a norm against them; and (b) to the extent that some people avoid those violations anyway, you've set up an asshole filter (that is, you're rewarding vice and punishing virtue).

So plausibly, the ideal situation is for it to be common knowledge that such things are considered fine to do. We might expect this to just push the problem one level up; so that instead of litigating minor deceptions, you're litigating slightly-less-minor deceptions. But these deceptions have a higher social cost, so more value to litigating them, so maybe it's fine.

(Aside, it's not clear to me why the hypothesis specifically expects such norms to be internalized, rather than enforced some other way. Possible answer: you do still need external enforcement of these norms, but that enforcement will be costly. It'll be cheaper if you can mostly expect people to obey them even if they don't expect to get caught, so that relies on self-enforcement. But is that a very general argument that almost all norms should be internalized? Well, maybe almost all norms are internalized. In any case, I don't think that clause was very important.)

Pandering to the SSC audience: dead whales

The second-most-detailed case study in the book is whalers. If a whale is wounded by one ship and killed by another, who keeps it? What if a dead whale under tow is lost in a storm, and found by another ship? The law eventually developed opinions on these questions, but when it did, it enshrined preexisting norms that the whalers themselves had developed.

Ellickson describes a few possible norms that wouldn't be welfare maximizing for them, and which in fact weren't used. For example, a whale might simply belong to whichever ship physically held the carcass; but that would allow one ship to wait for another to weaken a whale, then attach a stronger line and pull it in. Or it might belong to the ship that killed it; but that would often be ambiguous, and ships would have no incentive to harvest dead whales or to injure without killing. Or it might belong to whichever ship first lowered a boat to pursue it, so long as the boat remained in fresh pursuit; but that would encourage them to launch too early, and give claim to a crew who might not be best placed to take advantage of it. Or it might belong to whichever ship first had a reasonable chance of capturing it, so long as it remained in fresh pursuit; but that would be far too ambiguous.

In practice they used three different sets of norms. Two gave full ownership to one party. The "fast-fish/loose-fish" rule said that you owned a whale as long as it was physically connected to your boat or ship. The "first iron" (or "iron holds the whale") rule said that the first whaler to land a harpoon could claim a whale, as long as they remained in fresh pursuit, and as long as whoever found it hadn't started cutting in by then.

Whalers used these norms according to the fishery14 they hunted from, and each was suited to the whales usually hunted from that fishery. Right whales are weak swimmers, so they don't often escape once you've harpooned them. Fast-fish works well for hunting them. Sperm whales do often break free, and might be hunted by attaching the harpoon to a drogue, a wooden float that would tire the whale and mark its location. The concept of "fresh pursuit" makes first-iron more ambiguous than fast-fish, which isn't ideal, but it allows more effective means of hunting.

(Sperm whales also swim in schools, so ideally you want to kill a bunch of them and then come back for the corpses. If you killed a whale, you could plant a flag in it, which gave you claim for longer than a harpoon. You had to be given reasonable time to come back, and might take ownership even if the taker had started cutting in. Ellickson doesn't say explicitly, but it sounds like American whalers in the Pacific might have had this rule, but not American whalers operating from New England, for unclear reasons.)

The other was a split-ownership rule. A fishery in the Galápagos Islands split ownership 50/50 between whoever attached a drogue and whoever took the carcass. This norm gave whalers an incentive to fetter lots of them and let others harvest them later, but it's not clear how or why that fishery developed different rules than others. On the New England coast, whalers would hunt fast finback whales with bomb-lances; the whales would sink and wash up on shore days later. The killer was entitled to the carcass, less a small fee to whoever found it. This norm was binding even on people unconnected with the whaling industry, and a court upheld that in at least one case. I'm not sure how anyone knew who killed any given whale. Perhaps there just weren't enough whalers around for it to be ambiguous?

(Ellickson notes that the "50/50 split versus small fee" questions is about rules versus standards. Standards let you consider individual cases in more detil, taking into account how much each party contributed to the final outcome, and have lower deadweight losses. But rules have fewer disputes about how they should be applied, and thus lower transaction costs.)

So this is all plausibly welfare-maximizing, but that's not good enough. Ellickson admits that this sort of ex post explanation risks being "too pat". He points out two objections you could raise. First, why did the norms depend on the fishery, and not the fish? (That would have been more complicated, because there are dozens of species of whale. And you had to have your boats and harpoons ready, so you couldn't easily change your technique according to what you encountered.)

More interestingly, what about overfishing? If norms had imposed catch quotas, or protected calves and cows, they might have been able to keep their stock high. Ellickson has two answers. One is that that would have improved global welfare, but not necessarily the welfare of the current close-knit group of whalers, as they couldn't have stopped anyone else from joining the whaling business. This is a reminder that norms may be locally welfare-maximizing but globally harmful.

His other answer is… that that might not be the sort of thing that norms are good at? Which feels like a failure of the hypothesis. Here's the relevant passage:

Establishment of an appropriate quota system for whale fishing requires both a sophisticated scientific understanding of whale breeding and also an international system for monitoring worldwide catches. For a technically difficult and administratively complicated task such as this, a hierarchical organization, such as a formal trade association or a legal system, would likely outperform the diffuse social forces that make norms. Whalers who recognized the risk of overfishing thus could rationally ignore that risk when making norms on the ground that norm-makers could make no cost-justified contribution to its solution. [Footnote removed]

There's some subtlety here, like maybe he's trying to say "norms aren't particularly good at this, so if there's another plausible source of rules, norm-makers would defer to them; but if there wasn't, norm-makers would go ahead and do it themselves". That feels implausible on the face of it though, and while I'm no expert, my understanding is that no other group did step up to prevent overfishing in time.

This section is one place where Ellickson talks about the hypothesis as concerning dynamics. There are only five American court cases on this subject, and four of them involved whales caught between 1852 and 1862 in the Sea of Okhotsk; the other was an 1872 decision about a whale caught in that sea in an unstated year. Americans had been whaling for more than a century, so why did that happen? The whales in that area were bowheads, for which fast-fish may have been more utilitarian than first-iron. Ellickson speculates that "American whalers, accustomed to hunting sperm whales in the Pacific, may have had trouble making this switch."

(He does give an alternate explanation, that by that time the whaling industry was in decline and the community was becoming less close-knit. "The deviant whalers involved in the litigated cases, seeing themselves nearing their last periods of play, may have decided to defect.")

There's something that stuck out to me especially in this section, which I don't think Ellickson ever remarked upon. A lot of norms seem to bend on questions that are unambiguous given the facts but where the facts are unprovable. If I take a whale that you're in fresh pursuit of, I can tell everyone that you'd lost its trail and only found me days later. Who's to know?

Well, in the case of whalers, the answer is "everyone on both of our ships". That's too many people to maintain a lie. But even where it's just one person's word against another's, this seems mostly fine. If someone has a habit of lying, that's likely to build as a reputation even if no one can prove any of the lies.

Remedies

In private (i.e. non-criminal) law, when someone is found to be deviant, the standard remedy is to award damages. That doesn't always work. They might not have the assets to make good; or they might just be willing to pay that price to disrupt someone's world. So the legal system also has the power of injunctions, requiring or forbidding certain future actions. And if someone violates an injunction, the legal system can incarcerate them.

Norms have analagous remedies. Instead of damages, one can simply adjust a mental account. Instead of an injunction, one can offer a more-or-less veiled threat. Instead of incarcerating someone, one can carry out that threat.

Incarceration itself isn't a credible threat ("kidnapping is apt both to trigger a feud and to result in a criminal prosecution"), but other forms of illegal violence are. ("Indeed, according to Donald Black, a good portion of crime is actually undertaken to exercise social control." cite)

Remedial norms require a grievant to apply self-help measures in an escalating sequence. Generally it starts at give the deviant notice of the debt; goes through gossip truthfully about it; and ends with sieze or destroy some of their assets. Gossip can be omitted when it would be obviously pointless, such as against outsiders. This is consistent with the hypothesis, since the less destructive remedies come first in the sequence. It's also consistent with practice in Shasta County, and we see it as well in the lobstermen of Maine when someone sets a trap in someone else's territory. They'll start by attaching a warning to a trap, sometimes sabotaging it without damaging it. If that doesn't work, they destroy the trap. They don't seem to use gossip, perhaps because they can't identify the intruder or aren't close-knit with him.

"Sieze or destroy" - which should you do? Destroying assets is a deadweight loss, so it might seem that siezing them would be better for total welfare. But destruction has advantages too. Mainly, it's more obviously punitive, and so less likely to be seen as aggression and to lead to a feud. The Shasta County practice of driving a cow somewhere inconvenient isn't something you'd do for personal gain. But also, it's easier to calibrate (you can't sieze part of a cow, but you can wound it instead of killing). And it can be done surreptitiously, which is sometimes desired (though open punishment is usually prefered, to maintain public records).

Incentives all the way up

We don't have a good understanding of how norms work to provide order. But the key is "altruistic" norm enforcement by third parties. (Those are Ellickson's scare quotes, not mine.) How do we reconcile that with the assumption of self-interested behavior?

One possibility is internalized norms, where we feel guilty if we fail to act to enforce norms, or self-satisfied if we do act. (I feel like this is stretching the idea of self-interest, but then we can just say we reject that assumption, so whatever.)

Another is that the seemingly altruistic enforcers are themselves motivated by incentives supplied by other third parties. This seems to have an infinite regress. Ellickson gives as example a young man who tackled a pickpocket to retrieve someone's wallet. The woman he helped wrote into the New York Times to publicly thank him, so there's his incentive. But we also need incentives for her to write that letter, and for the editor to publish it, and so on.

(I'm not actually entirely sure where "so on" goes. I guess we also need incentive for people to read letters like that. Though according to Yudkowsky's Law of Ultrafinite Recursion there's no need to go further than the editor.)

This infinite regress seems bad for the chances of informal cooperation. But it might actually help. Ellickson's not entirely detailed about his argument here, so I might be filling in the blanks a bit, but here's what I think he's going for. Suppose there's a virtuous third-party enforcer "at the highest level of social control". That is, someone who acts on every level of the infinite regress. They'll sanction primary behavior as appropriate to enforce norms; but also sanction the people who enforce (or fail to enforce) those norms themselves; and the people who enforce (or fail to enforce) the enforcement of those norms; and so on, if "so on" exists.

Then that enforcer could create "incentives for cooperative activity that cascade down and ultimately produce welfare-maximizing primary behavior." They don't need to do all the enforcement themselves, but by performing enforcement on every level, they encourage others to perform enforcement on every level.

This might work even with just the perception of such an enforcer. God could be this figure, but so could "a critical mass of self-disciplined elders or other good citizens, known to be committed to the cause of cooperation". Art and literature could help too.

Academia seems to have a disproportionate number of legal centrists. So you might think professors would be unusually law-abiding. Not when it comes to photocopying. The law says how they should go about copying materials for use in class: fair-use doctrine is quite restrictive unless they get explicit permission, which can be slow to obtain15. Professors decide they don't really like this, and they substitute their own set of norms.

The Association of American Publishers tells us that campus copyright violation is (Ellickson quotes) "widespread, flagrant, and egregious". They seem to be right. Ellickson asked law professors directly, and almost all admit to doing it - though not for major portions of books. The managers of law school copy rooms don't try to enforce the rules, they let the professors police themselves. Several commercial copy centres made multiple copies for him of an article from a professional journal. "I have overheard a staff member of a copy center tell a patron that copyright laws prevented him from photocopying more than 10 percent of a book presented as a hardcopy original; the patron then asked whether he himself could use the copy center's equipment to accomplish that task and was told that he could."16

So professors' norms seem to permit illegal repeated copying of articles and minor parts of books. That lets them avoid knowing fair-use doctrine in detail. And since the law would require them to write (and respond to) many requests for consent, it lets them avoid that too.

Professors sense that Congress is unlikely to make welfare-maximizing copyright law. (Publishers can hire better lobbyists than they can.) This lets them frame their norms as principled subversion. I'm not sure if it's particularly relevant though - if copyright law was welfare-maximizing overall, but not for the professors, I think the hypothesis would still predict them to develop their own norms. But thinking back to the stuff on symbolism, maybe "being able to frame your actions as principled subversion" is a component of welfare.

Why will they copy articles, but not large portions of books? Authors of articles don't get paid much for them, and for no charge will mail reprints to colleagues and allow excerpts to be included in compilations. "It appears that most academic authors are so eager for readers to know and cite their work that they usually regard a royalty of zero or even less as perfectly acceptable. For them small-scale copying is not a misappropriation but a service." But book authors do receive royalties, and large-scale copying would diminish those significantly. So according to the hypothesis, this restraint comes from wanting to protect author-professors' royalty incomes, not from caring about publishers' and booksellers' revenues. (Though they might start to care about those, if they thought there might be a shortage of publishers and booksellers. They also might care more about university-affiliated publishers and booksellers.)

(There's a question that comes to mind here, that Ellickson doesn't bring up. Why do professors decline to copy books written by non-academics? I can think of a few answers that all seem plausible: that this is a simpler norm; that it's not necessarily clear who is and isn't an academic; and that it makes it easier to sell the "principled subversion" thing.)

Notably, in the two leading cases around academic copying, the plaintiffs were publishers and the primary defendant was an off-campus copy center. This is consistent with the hypothesis. In these situations, those two parties have the most distant relationship. Publishers have no use for copy centers, and copy centers don't buy many books, so neither of them has informal power over the other. Even more notably, in one of these cases, university-run copy centers weren't included as defendants - that might anger the professors, who do have power over publishers.

Counterexamples?

But Ellickson admits that all of this could be cherry-picking. So he looks at two well-known cases that he expects people to point to as counterexamples. (I hadn't heard of either of them, so I can't rule out that he's cherry-picking here, too. But I don't expect it.)

The first is the Ik of northern Uganda. These are a once-nomadic tribe with a few thousand members. Colin Turnbull found an unsettling pattern of inhumanity among them. Parents were indifferent to the welfare of their children after infancy, and people took delight in others' suffering. In Turnbull's words: "men would watch a child with eager anticipation as it crawled toward the fire, then burst into gay and happy laughter as it plunged a skinny hand into the coals. … Anyone falling down was good for a laugh too, particularly if he was old or weak or blind."

Ellickson replies that the Ik were "literally starving to death" at the time of Turnbull's visit. A few years prior, their traditional hunting ground had been turned into a national park, and now they were forced to survive by farming a drought-plagued area. (Turnbull "briefly presented these facts" but didn't emphasize them.) "Previously cooperative in hunting, the Ik became increasingly inhumane as they starved. Rather than undermining the hypothesis, the tragic story of the Ik thus actually supports the hypothesis' stress on close-knittedness: cooperation among the Ik withered only as their prospects for continuing relationships ebbed." [Footnote removed.]

I note that Wikipedia disputes this account. "[Turnbull] seems to have misrepresented the Ik by describing them as traditionally being hunters and gatherers forced by circumstance to become farmers, when there is ample linguistic and cultural evidence that the Ik were farmers long before they were displaced from their hunting grounds after the formation of Kidepo National Park - the event that Turnbull says forced the Ik to become farmers." To the extent that Ellickson's reply relies on this change in circumstances, it apparently (according to Wikipedia) falls short. But perhaps the important detail isn't that they switched from hunting to farming, but that they switched from "not literally starving to death" to "literally starving to death" (because of a recent drought).

Ellickson also cites (among others) Peter Singer as criticising Turnbull in The Expanding Circle, pp 24-26. Looking it up, Singer points out that, even if we take Turnbull's account at face value, Ik society retains an ethical code.

Turnbull refers to disputes over the theft of berries which reveal that, although stealing takes place, the Ik retain notions of private property and the wrongness of theft. Turnbull mentions the Ik's attachment to the mountains and the reverence with which they speak of Mount Morungole, which seems to be a sacred place for them. He observes that the Ik like to sit together in groups and insist on living together in villages. He describes a code that has to be followed by an Ik husband who intends to beat his wife, a code that gives the wife a chance to leave first. He reports that the obligations of a pact of mutual assistance known as nyot are invariably carried out. He tells us that there is a strict prohibition on Ik killing each other or even drawing blood. The Ik may let each other starve, but they apparently do not think of other Ik as they think of any non-human animals they find - that is, as potential food. A normal well-fed reader will take the prohibition of cannibalism for granted, but under the circumstances in which the Ik were living human flesh would have been a great boost to the diets of stronger Ik; that they refrain from this source of food is an example of the continuing strength of their ethical code despite the crumbling of almost everything that had made their lives worth living.

This seems to support the hypothesis too. I do think there's some tension between these two defenses. Roughly: their circumstances made them the way they were; and anyway, they weren't that way after all. But they don't seem quite contradictory.

The other potential counterexample is the the peasonts in Mentegrano, a southern Italian village, as studied by Edward Banfield.

Banfield found no horrors as graphic as [those of the Ik], but concluded that the Italian peasants he studied were practitioners of what he called "amoral familialism," a moral code that asked its adherents to "maximize the material, short-run advantage of the nuclear family; assume all others will do likewise." According to Banfield, this attitude hindered cooperation among families and helped keep the villagers mired in poverty. [One footnote removed; minor style editing.]

Ellickson has two replies here. Firstly, the evidence is arguably consistent with the hypothesis: some of Banfield's reviewers suggested that, going by Banfield's evidence, the villagers had adapted as well as possible to their environment. Secondly, Banfield's evidence often seems to contradict Banfield's thesis: neighbors have good relationships and reciprocate favors. Banfield apparently discounted that because they did so out of self-interest, but it's still compatible with the hypothesis.

(I don't think these replies are in the same kind of tension.)

For a more general possible counterexample, Ellickson points at primitive tribes believing in magic and engaging in brutal rites. (This is something I did have in my mind while reading, so I'm glad he addressed it.) Some anthropologists are good at finding utilitarian explanations for such things, but Ellickson rejects that answer. Instead, he simply predicts that these practices would become abandoned as the tribe becomes better educated. "A tribe that used to turn to rain dancing during droughts thus is predicted to phase out that ritual after tribe members learn more meteorology. Tribes are predicted to abandon dangerous puberty rites after members obtain better medical information. As tribe members become more familiar with science in general, the status of their magicians and witch doctors should fall. As a more contemporary example, faith in astrology should correlate negatively with knowledge of astronomy. These propositions are potentially falsifiable."

This was my guess as to an answer before I reached this part of the book, which I think says good things about both myself and the book. And I basically agree with his prediction. But I also think it's not entirely satisfactory.

It seems like we need to add a caveat to the hypothesis for this kind of thing, "if people believe that rain dances bring rain, then norms will encourage rain dances". And I kind of want to say that's fair enough, you can't expect norms to be smarter than people. But on the other hand, I think the thesis of The Secret of Our Success and the like is that actually, that's exactly what you can expect norms to be. And it seems like a significant weakening of the hypothesis - do we now only predict norms to optimize in ways that group members understand? Or to optimize not for welfare but for "what group members predict their future welfare will be"? I dunno, and that's a bad sign. But if the hypothesis doesn't lose points for rain dances, it probably shouldn't gain points for manioc. (Though as Ben Hoffman points out, the cost-benefit of manioc processing isn't immediately obvious. Maybe the hypothesis should lose points for both manioc and rain dances.)

If a ritual is cheap to implement, I'd be inclined to give it a pass. There's costs of obtaining information, and that could apply to whatever process develops norms like it does to individuals. Plus, it would only take a small benefit to be welfare-maximizing, and small benefits are probably less obvious than larger ones. (Though if that's what's going on, it's not clear whether we should expect education to phase the rituals out.)

But for vicious and dangerous rituals, this doesn't seem sufficient. Ellickson mentions a tribe where they "cut a finger from the hand of each of a man's close female relatives after he dies"; what medical knowledge are they lacking that makes this seem welfare-maximizing?

I think this is my biggest criticism of the hypothesis.

Another possible counterexample worth considering would be Jonestown, and cults in general. (h/t whoever it was brought this to my attention.) I don't feel like I know enough about these to comment, but I'm going to anyway. I wonder if part of what's going on is that cults effectively don't have the rule of law - they make it costly for you to leave, or to bring in outside enforcers, and so you can't really enforce your property rights or bodily autonomy. If so, it seems like the "workaday" assumption is violated, and the hypothesis isn't in play.

Or, what about dueling traditions? We might again say the "workaday" assumption (that brings rules against murder) is violated, but that seems like a cheat. My vague understanding, at least of pistol dueling as seen in Hamilton, is it was less lethal than we might expect; and fell out of favor when better guns made it more lethal. But neither of these feels enough to satisfy, and we should demand satisfaction. Did the group gain something that was worth the potential loss of life? Alternatively, were such things only ever a transition phase?

Formal game theory

Something I haven't touched on is Ellickson's use of formal game theory. To do justice to that section, I split it into its own essay. The tl;dr is that I think he handled it reasonably well, with forgiveable blind spots but not outright mistakes that I noticed. I don't feel like I need to discount the rest of the book (on subjects I know less well) based on his treatment of game theory.

Summing up

Is this a good book? Yes, very much so. I found it fascinating both on the level of details and the level of ideas. Ellickson is fairly readable, and occasionally has a dry turn of phrase that I love. ("A drowning baby has neither the time nor the capacity to contract for a rescue.") And I don't know if this came across in my review, but he's an unusually careful thinker. He owns up to weaknesses. He rejects bad arguments in favor of his position. He'll make a claim and then offer citations of people disagreeing. He makes predictions and admits that if they fail the hypothesis will be weakened. I think he made some mistakes, and I think his argument could have been clearer in places, but overall I'm impressed with his ability to think.

Is the hypothesis true? I… don't think so, but if we add one more caveat, then maybe.

The hypothesis says that norms maximize welfare. Note that although Ellickson calls the welfare function "objective", I think a better word might be "intersubjective". The welfare function is just, like, some amorphous structure that factors out when you look at the minds of group members. Except we can't look at their minds, we have to look at behaviour. The same is true of norms themselves: to figure out what the norms are in a society we ultimately just have to look at how people in that society behave.

And so if we're to evaluate the hypothesis properly, I think we need to: look at certain types of behaviour, and infer something that's reasonable to call "norms"; and then look at non-normative behavior - the behaviour that the inferred system of norms doesn't dictate - and infer something that's reasonable to call a "welfare function". And then the hypothesis is that the set of norms will maximize the welfare function. ("Maximize over what?" I almost forgot to ask. I think, maximize over possible systems of norms that might have been inferred from plausible observed behaviour?)

Put like that it sounds kind of impossible. I suspect it's… not too hard to do an okay job? Like I'd guess that if we tried to do this, we'd be able to find things that we'd call "norms" and "a welfare function" that mostly fit and are only a little bit circular; and we wouldn't have an overabundance of choice around where we draw the lines; and we could test the hypothesis on them and the hypothesis would mostly come out looking okay.

But to the extent that we can only do "okay" - to the extent that doing this right is just fundamentally hard - I suspect we'll find that the hypothesis also fails.

There are problems which are known to be fundamentally hard in important ways, and we can't program a computer to reliably solve them. Sometimes people say that slime molds have solved them and this means something about the ineffable power of nature. But they're wrong. The slime molds haven't solved anything we can't program a computer to solve, because we can program a computer to emulate the slime molds.17 What happens is that the slime molds have found a pretty decent approach to solving the problem, that usually works under conditions the slime molds usually encounter. But the slime molds will get the wrong answer too, if the specific instance of the problem is pathological in certain ways.

In this analogy, human behavior is a slime mold. It changes according to rules evaluated on local conditions. (Ellickson sometimes talks about "norm-makers" as though they're agents, but that feels like anthropomorphising. I expect only a minority of norms will have come about through some agentic process.) It might be that, in doing so, it often manages to find pretty good global solutions to hard problems, and this will look like norms maximizing welfare. But when things are set up right, there'd be another, better solution.

(I'm not sure I've got this quite right, but I don't think Ellickson has, either.)

So I want to add a caveat acknowledging that sort of thing. I don't know how to put it succinctly. I suspect that simply changing "maximize" in the hypothesis to "locally maximize" weakens it too far, but I dunno.

With this additional caveat, is the hypothesis true? I still wouldn't confidently say "yes", for a few reasons. My main inside-view objections are the ritual stuff and duelling, but there's also the outside-view "this is a complicated domain that I don't know well". (I only thought of duelling in a late draft of this review; how many similar things are there that I still haven't thought of?) But it does feel to me like a good rule of thumb, at least, and I wouldn't be surprised if it's stronger than that.

Further questions

I want to finish up with some further questions.

• The world seems to be getting more atomic, with less social force being applied to people. Does that result in more legal force? Ellickson gives a brief answer: "In [Donald Black's] view, the state has recently risen in importance as lawmakers have striven to fill the void created by the decline of the family, clan, and village." (Also: "increasing urbanization, the spread of liability insurance, and the advent of the welfare state". But Black speculates it'll decline in future because of increasing equality. And although the number of lawyers in the US has increased, litigation between individuals other than divorce remains "surprisingly rare".)

• Can I apply this to my previous thoughts on the responsibility of open source maintainers? When I try, two things come to mind. First, maintainers know more about the quality of their code than users. Thus, if we require maintainers to put reputation on the line when "selling" their code, we (partially) transfer costs of bad code to the people who best know those costs and are best able to control them. So that's a way to frame the issue that I don't remember having in mind when I wrote the post, that points in the same directions I was already looking. Cool. Second, I feel like in that post I probably neglected to think about transaction costs of figuring out whether someone was neglecting their responsibility? Which seems like an important oversight.

• To test the hypothesis, I'd be tempted to look at more traditions and see whether those are (at least plausibly) welfare-maximizing. But a caveat: are traditions enforced through norms? I'd guess mostly yes, but some may not be enforced, and some may be enforced through law. In those cases the hypothesis doesn't concern itself with them.

• Making predictions based on the hypothesis seems difficult. Saying that one set of norms will increase welfare relative to another set might be doable, but how can you be confident you've identified the best possible set? Ellickson does make predictions, and I don't feel like he's stretching too far - though I can't rule it out. But I'm not sure I'd be able to make the same predictions independently. How can we develop and test this skill?

• Sometimes a close-knit group will fracture. What sort of things cause that? What does it look like when it happens? What happens to the norms it was maintaining?

• What are some follow-up things to read? Ellickson approvingly cites The Behavior of Law a bunch. If we want skepticism, he cites Social Norms and Economic Theory a few times. At some point I ran across Norms in a Wired World which looks interesting and cites Ellickson, but that's about all I know of it.

• How does this apply to the internet? I'd note a few things. Pseudonymity (or especially anonymity) and transience will reduce close-knittedness, as you only have limited power over someone who can just abandon their identity. To the extent that people do have power, it may not be broadly distributed; on Twitter for example, I'd guess your power is roughly a function of how many followers you have, which is wildly unequal. On the other hand, public-by-default interactions increase close-knittedness. I do think that e.g. LessWrong plausibly counts as close-knit. The default sanction on reddit is voting, and it seems kind of not-great that the default sanction is so low-bandwidth. For an added kick or when it's not clear what the voting is for, someone can write a comment ("this so much"; "downvoted for…"). That comment will have more weight if it gets upvoted itself, and/or comes from a respected user, and/or gets the moderator flag attached to it. Reddit also has gilding as essentially a super-upvote. For sanctioning remedial behavior, people can comment on it ("thanks for the gold", "why is this getting downvoted?") and vote on those comments. But some places also have meta-moderation as an explicit mechanism.

• How much information is available about historical norms and the social conditions they arose in? Enough to test the hypothesis?

• There's a repeated assumption that if someone has extraordinary needs then it's welfare maximizing for the cost to fall on them instead of other people. I'm not sure how far Ellickson would endorse that; my sense is that he thinks it's a pretty good rule of thumb, but I'm not sure he ever investigates an instance of it in enough detail to tell whether it's false. It would seem to antipredict norms of disability accomodation, possibly including curb cuts. (Possibly not, because those turn out to benefit lots of people. But then, curb cuts are enforced by law, not norm.) This might be a good place to look for failures of the hypothesis, but it's also highly politicized which might make it hard to get good data.

• Ellickson sometimes suggests that poor people will be more litigous because they have their legal fees paid. We should be able to check that. If it's wrong, that doesn't necessarily mean the hypothesis is wrong; there are other factors to consider, like whether poor people have time and knowledge to go to court. But it would be a point in favor of something like "the hypothesis is underspecified relative to the world", such that trying to use it to make predictions is unlikely to work.

• Is Congress close-knit? Has that changed recently? Is it a good thing for it to be close-knit? (Remember, norms maximizing the welfare of congresspeople don't necessarily maximize the welfare of citizens.)

• Does this work at a level above people? Can we (and if so, when can we) apply it to organizations like companies, charities, and governments?

• Suppose the book's analysis is broadly true. Generally speaking, can we use this knowledge for good?

1. In this review, I use the present tense. But the book was published in 1991, based on research carried out in the 1980s.

2. Something like this is familiar to me from the days when most of my friendships took place in pubs. Small favours, even with specific monetary value, would typically be repaid in drinks and not in cash. Once, apparently after a disappointing sexual experience, I was asked my opinion on the exchange rate between drinks and orgasms.

It strikes me that your neighbor is still clearly worse off than if your goat hadn't eaten his tomatoes. He's gone from having tomatoes-now to only having future-tomatoes. But that means your neighbor has no reason to be careless with his tomatoes. And helping to replant may encourage you to control your goat more than paying money would.

3. Ellickson never says explicitly what one of these is, but my read is a small ranch, more a home than a business and operated more for fun than profit. Only a handful of animals or possibly none at all, and sometimes crops.

4. If you want a teaser, the first three chapters were based on his previous article Of Coase and Cattle (1986). I haven't compared closely, but they seem to have a lot of text in common.

I should note here that I read part one a few years ago. I took reasonably detailed notes, with the intent of editing them into something worth publishing before moving on to part two. Then I didn't do that. My review of part one is largely based on my notes, although I have skimmed or reread large amounts of it. I read part two recently, specifically for this contest.

5. He was a founding member of the American Law and Economics Association in 1991, and its President 2000-2001.

6. "This counterintuitive proposition states, in its strongest form, that when transaction costs are zero a change in the rule of liability will have no effect on the allocation of resources. … This theorem has undoubtedly been both the most fruitful, and the most controversial, proposition to arise out of the law-and-economics movement." The paper which first presented the theorem used cattle trespass as an example, directly inspiring the study in part one.

7. Ellickson offers citations but (apart from Shasta County) no elaboration on these.

8. Ellickson makes two points. First, that ranchers are more familiar than ranchette owners with barbed-wire fencing. To some extent that seems circular, since they're the ones who are expected to know about it, but it's also in part because many ranchette owners have moved from the city. Second, that ranchers can fence in their own herds unilaterally, while victims would have to coordinate; motorists in particular would have trouble with that, and arguably they benefit the most. But motorists aren't part of the relevant close-knit group, so we should ignore them for this analysis. And as far as I know the other noteworthy victims are all landowners, who don't need to coordinate to protect their own interests.

But: even if victims wouldn't need to coordinate, they'd all need to act individually, and acquiring the skills would be a cost to them. Ranchers would presumably still need those skills, and even if not, there are presumably fewer of them. So it seems cheaper for all ranchers to acquire the skills, than all of their potential victims.

9. Of course, since this example was a generator of the hypothesis, that says little by itself. This isn't a big deal, Ellickson looks outside Shasta County plenty, I'm just pointing it out because it's important to notice things like this.

10. No, really. This isn't him saying one thing and me saying "well that only works if…". He says explicitly that the hypothesis "assumes that norm-makers in close-knit groups would subscribe to an unalloyed version of this principle".

11. Actually, for parenthood, a plausible answer does come to mind: deciding who society celebrates as parents (rich couples who can mostly pay the costs of parenthood themselves) and who it shames (poor single mothers who socialize the costs). Then I guess the hypothesis predicts that you're allowed to socialize the costs of parenthood to the same extent that parenthood is welfare-positive. Except… that doesn't really work, because the total welfare change seems like it would be more-or-less the same whether the costs are borne by the parents or by society. I dunno, I think I'm still confused.

12. This is kind of a weird example, as he all-but-admits in a footnote: "This statement assumes the continuing presence of foundational rules that forbid the firefighters from killing, maiming or imprisoning each other." If we want to know what actually happens in this situation, he points us to Dudley and Stephens and a book named Cannibalism and the Common Law

13. I haven't read these recently, and might be misremembering.

14. Minor complaint: I wish Ellickson had been clearer about what exactly a "fishery" is. Did two boats from different fisheries ever encounter each other?

15. In one case study, 23 permission letters were sent to publishers and only 17 received a response in six months. Ellickson doesn't say how many were denied.

16. I looked it up out of curiosity. Although the 10% figure may have come from the relevant guidelines, they're unsurprisingly a lot more restrictive than that. For prose the maximum seems to be "1,000 words or 10% of the work, whichever is less, but in any event a minimum of 500 words."

17. At least, if we can't in practice, there's nothing stopping us in theory. I'm not sure if we know exactly what the slime molds are doing. But I'm sure that if we did know, there wouldn't turn out to be anything fundamentally mysterious and unprogrammable-in-computers about it.

Discuss

### The accumulation of knowledge: literature review

10 июля, 2021 - 21:36
Published on July 10, 2021 6:36 PM GMT

Financial status: This is independent research, now supported by a grant. I welcome financial support .

Epistemic status: This is in-progress thinking.

This post is part of a sequence on the accumulation of knowledge. Our goal is to articulate what it means for knowledge to accumulate within a physical system.

The challenge is this: given a closed physical system, if I point to a region and tell you that knowledge is accumulating in this region, how would you test my claim? What are the physical characteristics of the accumulation of knowledge? What is it, exactly, about an artifact inscribed with instructions for building advanced technology that makes it so different from an ordinary rock, or from a video camera that has been travelling the cosmos recording data since the beginning of the universe? We are looking for a definition of knowledge at the level of physics.

The previous four posts have explored four possible accounts for what the accumulation of knowledge consists of. Three of the accounts viewed knowledge as a correspondence between map and territory, exploring different operationalizations of "correspondence". The fourth account viewed the accumulation of knowledge as an increasing sensitivity of actions to the environment. We found significant counter-examples to each of the four accounts, and I remain personally unsatisfied with any of the four accounts.

This post will briefly review literature on this topic.

Constructive definitions of knowledge

One could view probability theory as a definition of knowledge. Probability theory would say that knowledge is a mapping from claims about the world to numerical credences in a way that obeys the laws of probability theory, and that the accumulation of knowledge happens when we condition our probabilities on more and more evidence. Probability theory is not usually presented as an account of what knowledge is, but it could be.

However, probability theory is most useful when we are designing machines from scratch rather than when we are interpreting already-constructed machines such as those produced by large-scale search in machine learning. It might be possible to reverse-engineer a neural network and determine whether it encodes a program that approximately implements Bayesian inference, but it would take a lot of work to do that, and it would have to be done on a case-by-case basis. Probability theory is not really intended to give an account of what it means for a physical system to accumulate knowledge, just as an instruction manual for assembling some particular desktop computer is not on its own a very good definition of what computation is.

In the same way we could view all the various formal logical systems, as well as logical induction and other non-probability-theory systems of credences as constructive accounts of knowledge, although these, like probability theory, do not help us much with our present task.

Philosophical analysis of knowledge

There is an extensive philosophical literature concerning the analysis of knowledge. This area received a great deal of attention during the second half of the 20th century. The basic shape of this literature seems to be a rejection of the view that knowledge is "justified true belief", followed by counterexamples to the "justified true belief" paradigm, followed by many attempts to augment the "justified true belief" definition of knowledge, followed by further counterexamples refuting each proposal, followed by whole books being written about general-purpose counterexample construction methods, followed by a general pessimism about coming to any satisfying definition of knowledge. My main source on this literature has been the SEP article on the topic, which of course gives me only a limited perspective on this enormous body of work. I hope that others better-versed in this literature will correct any mistakes that I have made in this section.

The "justified true belief" position, which much of the 20th century literature was written to either refute or fix, is that a person knows a thing if they believe the thing, are justified in believing the thing, and the thing is in fact true.

The classic counterexample to this definition is the case where a person believes a thing due to sound reasoning, but unbeknownst to them were fooled in some way in their perceptions, yet the thing itself turns out to be true by pure luck:

Imagine that we are seeking water on a hot day. We suddenly see water, or so we think. In fact, we are not seeing water but a mirage, but when we reach the spot, we are lucky and find water right there under a rock. Can we say that we had genuine knowledge of water? The answer seems to be negative, for we were just lucky. (Dreyfus 1997: 292)[1]

Counter-examples of this form are known as Gettier cases. There are many, many proposed remedies in the form of a fourth condition to add to the "justified true belief" paradigm. Some of them are:

• The belief must not have been formed on the basis of any false premise

• The belief must have been formed in a way that is sensitive to the actual state of the world

• The thing itself must be true in other nearby worlds where the same belief was formed

• The belief must have been formed by a reliable cognitive process

• There must be a causal connection from the way the world is to the belief

The follow-up counterexamples that rebounded off these proposals eventually led to the following situation:

After some decades of such iterations, some epistemologists began to doubt that progress was being made. In her 1994 paper, "The Inescapability of Gettier Problems", Linda Zagzebski suggested that no analysis sufficiently similar to the JTB analysis could ever avoid the problems highlighted by Gettier’s cases. More precisely, Zagzebski argued, any analysans of the form JTB+X, where X is a condition or list of conditions logically independent from justification, truth, and belief, would be susceptible to Gettier-style counterexamples. She offered what was in effect a recipe for constructing Gettier cases:

(1) Start with an example of a case where a subject has a justified false belief that also meets condition X.

(2) Modify the case so that the belief is true merely by luck.

Zagzebski herself proposed "the belief must not be true merely by luck" as a possible fourth condition for the existence of knowledge, but even this fell to further counterexamples.

Overall I found this literature extremely disappointing as an attempt to really get at what knowledge is:

• The literature seems to be extremely concerned with what certain words mean, particularly the words "knowledge" and "belief". It’s all about what is meant when someone says "I know X" or "I believe X", which is a reasonable question to ask but seems to me like just one way to investigate the actual phenomenon of knowledge out there in the world, yet the literature seems unreasonably fixated on this word-centric style of investigation.

• The literature seems to be centered around various subjects and their cognition. The definitions are phrased in terms of a certain subject believing something or having good reason for believing something, and this notion that there are subjects that have beliefs seems to be taken as primary by all writers. I suspect that an unseen presupposition of the agent model is at work in sneaking in premises that create the confusion that everyone seems to agree exists in the field. An alternative to taking subjects as primary is to take an underlying mechanistic universe as primary, as this sequence has done. Now there are pros and cons to this mechanistic-universe approach in comparison to the subject-cognition approach. It is not that one approach is outright better than the other. But it is disappointing to see an entire field apparently stuck inside a single way of approaching the problem (the subject-cognition approach), with apparently no-one considering alternatives.

• The literature spends a lot of time mapping out various possible views and the relationship between them. It is frequently written that "internalists believe this", "state internalism implies this", "access internalism implies that", and so on. It’s good to map out all these views but sometimes the mapping out of views seems to become primary and the direct investigation of what knowledge actually is seems to take a back seat.

Overall I have not found this literature helpful as a means to determine whether the entities we are training with machine learning understand more about the world than we expected, nor how we might construct effective goal-directed entities in a world without Cartesian boundaries.

Eliezer on knowledge

Much of Eliezer’s writing in The Sequences concerns how it is that we acquire knowledge about the world, and what reason we have to trust our knowledge. He suggests this as the fundamental question of rationality:

I sometimes go around saying that the fundamental question of rationality is Why do you believe what you believe?

However, most of Eliezer’s writings about knowledge are closer to the constructive frame than the mechanistic frame. That is, they are about how we can acquire knowledge ourselves and how we can build machines that acquire knowledge, not about how we can detect the accumulation of knowledge in systems for which we have no design blueprints.

One exception to this is Engines of Cognition, which does address the question from a mechanistic frame, and is a significant inspiration for the ideas explored in this sequence (as well as for my previous writings on optimization). In the essay, Eliezer demonstrates, eloquently as ever, that the laws of thermodynamics prohibit the accumulation of knowledge in the absence of physical evidence:

"Forming accurate beliefs requires a corresponding amount of evidence" is a very cogent truth both in human relations and in thermodynamics: if blind faith actually worked as a method of investigation, you could turn warm water into electricity and ice cubes. Just build a Maxwell's Demon that has blind faith in molecule velocities.

[...]

So unless you can tell me which specific step in your argument violates the laws of physics by giving you true knowledge of the unseen, don't expect me to believe that a big, elaborate clever argument can do it either.

In the language of the present sequence, what Eliezer points out is that information is a necessary condition for knowledge. I agree that information is a necessary condition for knowledge although I have argued in this sequence that it is not a sufficient condition.

Other writings

I suspect that this literature review has missed a lot of relevant writing both here on lesswrong and elsewhere. I would very much appreciate pointers to further resources.

Conclusion

This sequence has investigated the question of what it means for knowledge to accumulate within a region within a closed physical system. Our motivations were firstly to detect the accumulation of knowledge as a safety measure for entities produced by large-scale search, and secondly to investigate agency without imposing any agent/environment boundary.

The first three definitions we explored were variations on the theme of measuring mutual information between map and territory. We attempted to do this by looking at just a single configuration of the system, then by looking at a distribution over configurations, and then by looking at digital abstraction layers. The counterexamples given for each of these three proposals seem to show that none of these definitions are sufficient conditions for the accumulation of knowledge, though the second of these, and perhaps even the third, seems to be a necessary condition.

We then explored a definition of knowledge accumulation as a coupling between map and territory that shows up as a sensitivity of the part of the system that has "knowledge" to the remainder of the system.

Overall, I am not sure that any of these definitions are really on the right track, but I am very enthusiastic about the question itself. I like the frame of "what would it mean at the level of physics for…" as a means to break out of the agent-centric frame that the philosophical literature on this subject often seems to be inside. I think that keeping in mind the practical goals of AI safety helps to keep the investigation from becoming a conceptual Rorschach test.

1. Dreyfus, George B.J., 1997, Recognizing Reality: Dharmakirti’s Philosophy and its Tibetan Interpretations, Albany, NY: SUNY Press. ↩︎

Discuss

### For some, now may be the time to get your third Covid shot

10 июля, 2021 - 20:38
Published on July 10, 2021 5:38 PM GMT

Summary
• It seems likely there is a significant benefit to a booster shot in preventing symptomatic infection in the face of the Delta variant for many people (Bullets 1-3).
• There is significant indication that a third dose of an mRNA vaccine has a good safety profile (Bullets 2-4).
• Delta may have an a greater impact in the US than you realize (Bullet 5), and
• Therefore, for some people now may be the right time to get their third Covid mRNA shot (Bullet 6) although there are many reasons to potentially wait (Bullet 7).
Bullets
1. Two days ago, the Israeli government announced that its observed vaccine efficiency (Pfizer) as of June 6th was down to 64% against symptomatic infection, while protection against severe disease and hospitalization has declined more mildly to 93%. This is seemingly almost entirely attributed to the Delta variant.

2. Today, Pfizer put out a statement which included the following:

Pfizer and BioNTech have seen encouraging data in the ongoing booster trial of a third dose of the current BNT162b2 vaccine. Initial data from the study demonstrate that a booster dose given 6 months after the second dose has a consistent tolerability profile while eliciting high neutralization titers against the wild type and the Beta variant, which are 5 to 10 times higher than after two primary doses. The companies expect to publish more definitive data soon as well as in a peer-reviewed journal and plan to submit the data to the FDA, EMA and other regulatory authorities in the coming weeks. In addition, data from a recent Nature paper demonstrate that immune sera obtained shortly after dose 2 of the primary two dose series of BNT162b2 have strong neutralization titers against the Delta variant (B.1.617.2 lineage) in laboratory tests. The companies anticipate that a third dose will boost those antibody titers even higher, similar to how the third dose performs for the Beta variant (B.1.351). While Pfizer and BioNTech believe a third dose of BNT162b2 has the potential to preserve the highest levels of protective efficacy against all currently known variants including Delta, the companies are remaining vigilant and are developing an updated version of the Pfizer-BioNTech COVID-19 vaccine that targets the full spike protein of the Delta variant.

CNN says they plan to submit to the CDC for emergency approval to administer booster doses next month.[1]

1. Moderna previously announced that in their trial they also observed indication of significant benefit to a third dose booster (of half the size of the initial Moderna doses)[2] and found it to have a similar safety profile to the second shot.

2. In addition to the clinical trials, Turkey, the UAE, NYC, and some others have given people third doses in some circumstances / categories. The immunocompromised are particularly likely to benefit, as one would expect.

3. The Delta variant may have more of an impact than you expect. Polymarket currently gives a ~⅔ probability to the US again having more than 100,000 cases in a day before the end of the year. This could come from a new variant or other novel situation, but I expect most of this probability for most bettors is with an eye toward the Delta variant. Cases in the UK have increased 10-20x since their post-vaccination trough, and it has/had similar vaccination rates to the US (although less immunity from having contracted Covid, and a greater proportion with only one dose or a less effective vaccine). The UK’s current number of new cases is equivalent to ~160k new daily cases in the US when adjusted for population.

4. There are various reasons why getting vaccinated now may be particularly good for you

1. Symptomatic infection may be bad to you, especially if you or someone you interact closely with is particularly vulnerable.
2. If you received your vaccine early (e.g. 2020-February 2021) your protection is more likely to have weakened.
3. Your peak exposure may be soon if the Delta variant has a similar effect in the US as it did in the UK. You’d also need to plan for the 1-2 week delay between shot and effect.
4. When a booster shot likely does get approved by the CDC in ~1-3 months, there may be a run on supply preventing getting a vaccine booster for some time.
5. There are also various reasons to not pursue a booster vaccine now

1. You may care primarily about protection against severe disease, which seems to largely be maintained in the face of the Delta variant.
1. You may be in good health / young, in which case you have additional reason to believe you will be fine even if you contract Covid.
2. If less time has passed (under 4-5 months?) since you were vaccinated, then your potential benefit is likely less than some bullets above probably suggested it was for those who have had more time pass.
3. Some, such as the CDC and FDA, don’t think a booster is currently warranted.
4. It’s unclear to me if it’s legal to get a third dose in the US prior to that being approved. Regardless, it may be hard to procure without misrepresentation of your previous vaccinations.
5. More data on safety, optimal timing, etc. will be forthcoming and may help you make a better decision.
6. Boosters may be available this fall that specifically target newer variants, and perhaps getting a booster now will in some way make procurement of those worse (worse side effects, mandatory wait time since last vaccine, etc).
7. Getting an additional dose of vaccine may ‘take it away’ from somebody else who needs it more, due to global supply limitations.
8. You may believe the Delta variant is unlikely to lead to a large wave in your area, e.g. because of local vaccination rates or the ‘control system’.

For some, now may be the time to get your third Covid shot.

1. Summary article of these developments. ↩︎

2. For this reason, if pursuing a third booster shot soon, I’d lean Pfizer (smaller dose size) over Moderna. ↩︎

Discuss

### Will the US have more than 100,000 new daily COVID-19 cases before January 1, 2022?

10 июля, 2021 - 17:01
Published on July 10, 2021 2:01 PM GMT

I am trying to work through a process where I can assess what the probability of this question being a yes is.

I am going to use this post as an information dump in order to be able to rationally come up with the best probability I can.

Factors that could lead to this question being a YES:
The rise of the delta variant
The low vaccination rates in certain regions
The hesitancy by both individuals and governments to enact containment measures again
The risk of a reporting error by the CDC

Factors that could lead to this question being a NO:
Vaccination continues albeit at a slower pace
Vaccination of children has begun
Apathy towards getting tested unless symptoms are serious
Local containment measures if health systems start to collapse (Springfield, Missouri will be a good gauge for this)

I will continue to update information in the comments. Any input is greatly appreciated.

Discuss

### Firefox does not block analytics by default

10 июля, 2021 - 15:50
Published on July 10, 2021 12:50 PM GMT

There's a myth going around that Firefox blocks analytics providers in its default configuration. For example, in a recent HN discussion 5% of the comments were people asserting it did, and another 5% were people responding to them to assert that it doesn't. To confirm, it doesn't:

You can test this yourself:

1. Open Firefox

2. If you've added any extensions create a fresh profile, so you can test the default configuration

3. Open Developer Tools

4. Open the Networking panel

5. Visit a site with analytics (ex: jefftk.com)

6. Observe pings being sent (ex: google-analytics.com/j/collect?...)

Firefox does have an "Enhanced Tracking Protection" mode, but it's not enabled by default. Additionally, Firefox users are likely disproportionately blocking ads, which also block analytics scripts.

Discuss

### Research Facilitation Invitation

10 июля, 2021 - 04:04
Published on July 10, 2021 1:04 AM GMT

Summary: Would you like to help me with my research by letting me try to help you with your research?

One of my brand new sub-projects at the moment (underneath the larger "naturalism" heading) is research facilitation.

So far, I've mostly been developing the [school? methodology? whatever it is] by asking people to bring me their problems, and then helping them re-orient to the "problem" as a field of study. This is an especially valuable approach for problems that have stuck around for a long time despite repeated attempts to solve them.

But that's not the primary use case I envision for naturalism. Or at least, it's not the one that makes me eager to pour all of these resources into it.

When someone brings me a confusing or recalcitrant problem, and then they begin to relate to it as a field of study using naturalist methods, they almost invariably go through this period that I should probably have named by now. Let's call it "pre-conceptual intimacy".

In pre-conceptual intimacy, they're making a lot of fascinating observations and surprisingly quick improvements to relevant parts of life. But they're also feeling very confused and disoriented, because their pre-existing concepts around the problem just don't seem to make sense anymore, and they don't have new stories about what's going on yet either. They tend to utter phrases like, "Is memory even a thing?", "How could I ever have thought that?", and "I really have no idea what's going on with this, and it turns out I never have."

The spark of life in my work, the place where I'm most confident I have something really valuable here, is the progress I see people making in the midst of pre-conceptual intimacy.

Why? Because it means I may have a general methodology that works at intellectual frontiers, in the absence of established paradigms. Which is where we are, as a species, in human rationality, AI alignment, and presumably other important places I'm not even able to gesture toward.

So here is my vision for this research facilitation sub-project.

If I imagine that I've succeeded utterly, I envision that I'm able to meet with a researcher for one hour who is starting with "I have a hunch that there's something important, and I'm motivated to work on it, but I don't even know what the thing is, I have no idea where to start, and there are no domain-specific resources because I might be the first person who's ever even tried to think seriously about this, and like who even am I anyway to be working on something like this,"

and help them get to "I have traction and I'm confident I'll continue making consistent progress as I lay the groundwork for this nascent field."

I have some pretty specific ideas about how to get there from here, but they're almost certainly quite wrong. I know I have some crucial skills and insights, but I don't know how to apply them yet in this specific type of interaction. I won't be good at it right away. It's bound to be very messy and confusing for a while. But I figure the only way out is through.

So I'm looking for people who want me to clumsily fumble through the very earliest stages of a research project with them.

I expect I'll be most helpful to you if you

• don't know how to start,
• don't quite know what the thing even is, or
• feel that you're waiting for [something] before you're allowed to seriously get to work.

I expect you will be most helpful to me if you

• have an adventurous, collaborative, "let's try it!" sort of attitude
• do not expect an ordinary life coach could be of much use to you, and
• can meet during PST business hours.

Here are some examples of topics and/or mental states I imagine might fit the bill (though I expect my imagination here is quite impoverished at this stage):

• "Something is up with how distinctions work, and I suspect it's really important."
• "There's this particular kind of thought that keeps popping up in my work on agent foundations, and I'm not sure where it's coming from, why I seem to be drawn to it, or what I should do about it."
• "I have a vague but insistent hunch that my whole organization/sub-field/collection-of-past-selves is going about it all wrong, but I'm somehow paralyzed and have done nothing about this."

If you would like to spend an hour (or three) helping me figure out this research facilitation thing, please send an email with the subject "research facilitation" to LoganNaturalism@gmail.com, and take a stab at answering, "How can you tell that you may have found something worthwhile to investigate?"

My psychosocial budget for meetings is, alas, very small, so I expect to get a lot more responses than I can accommodate. If you don't get a time slot, please try not to take it personally. This may turn out to be something of a lottery.

Discuss

### Going Out With Dignity

10 июля, 2021 - 01:07
Published on July 9, 2021 10:07 PM GMT

DF was born with a time bomb in his genome, a deadly curse more horrifying than most. The name of this curse was Fatal familial insomnia (FFI).

Wikipedia describes the usual progression of this hell:

The disease has four stages:[8]

1. Characterized by worsening insomnia, resulting in panic attacks, paranoia, and phobias. This stage lasts for about four months.
2. Hallucinations and panic attacks become noticeable, continuing for about five months.
3. Complete inability to sleep is followed by rapid loss of weight. This lasts for about three months.
4. Dementia, during which the person becomes unresponsive or mute over the course of six months, is the final stage of the disease, after which death follows.

From the case report DF's psychologist wrote after his death:

DF was a right-handed, 52-year-old, white, American man with a doctorate in naturopathy. DF's father, paternal uncle, and 2 male cousins were diagnosed with fatal familial insomnia (FFI). His father died at age 76; his uncle died at age 74; and each of DF's cousins died before the age of 50.

Not only is there no cure for FFI; there is no known cure for any prion disease.

On the day it became clear he was experiencing the same symptoms his relatives did, DF must have known his chances were terrible. And even the minuscule odds that he could find a solution were marred by the fact that his problem-solving organ was the very part of him that was beginning to degrade.

And if there was a way out, how could he come up with a solution when he was so, so tired?

If only he could get just a little bit of rest.

There is a motivational technique I use occasionally where I look at my behavior and meditate on what my revealed preferences imply about my actual preferences. Often, I am disgusted.

I then note that I am capable of changing my behavior. And that all that is required to change my revealed preferences is to change my behavior. Though there is an element of sophistry in this line of thinking, I can report some anecdotal success

Many of us here, like DF, believe we have a deadly curse - or at least we believe we  believe we have a deadly curse.

Since I read The Basic AI Drives, I have known abstractly the world is doomed. Though my system 1 seems to have difficultly comprehending this, this belief implies I, and everyone and everything I love, am doomed, too.

Through the lens of my revealed preferences,  I either do not truly think the alignment problem is much of a problem, am mostly indifferent to the destruction of myself and everything I care about, or I am choosing the ignoble path of the free-rider.

I notice I am disgusted. But this is good news. All that is required to change my revealed preferences is to change my behavior.

DF's first tools were those of his naturopathic trade. He reported some success with a vitamin cocktail of the standard throw-in-everything-but-the-kitchen-sink, alternative-medicine style.

Perhaps something in his cocktail had some effect as his progression was slower than normal. But slow progression is progression just the same:

By month 15 (early stage II), vitamins alone failed to induce sleep. Following 5 consecutive nights of insomnia, DF became intensely irritable and delusional. An evaluation at the Massachusetts General Hospital in Boston, Massachusetts, found that he had suffered a minor stroke; he was anesthetized until he fell asleep. While hospitalized, he slept for 3 consecutive days and was fully alert and refreshed afterward.

Noticing the efficacy of the anesthetics, DF began to use them regularly:

Ketamine and nitrous oxide induced short (15-minute) periods of restful sleep, and were reapplied to offer more prolonged relief. Chloral hydrate in a light alcohol mix and/or chloroform also worked. Approximately 15 months into his illness, DF began to take sleep medications on a rotating schedule. Ethclorvynol, zolpidem, and diazepam reliably relieved his insomnia for roughly 1 month. During subsequent months, only diazepam offered intermittent relief.

In Irrational Modesty, I argue modesty is a paralytic preventing otherwise-capable people from acting on alignment:

Those who are capable, confident in their abilities, and motivated to work on this problem do not need a peptalk. But the possibility that there is a class of highly talented would-be-motivated people who lack confidence in their abilities still haunts me.

[...]In the event  anyone reading this has objective, reliable external metrics of extremely-high ability yet despite this feels unworthy of exploring the possibility that they can contribute directly to research, my advice is to act as if these external metrics correctly assess your ability until you have thoroughly proven to yourself  otherwise.

There is no virtue in clutching Kryptonite. I advise you to drop it and see how far you can fly.

There is another class of anesthetic whose key feature is a sort of detached emotional state, a feeling of being "above it" or "beyond it" or even "below it". Let's call "above it" and "beyond it" high detachment  and "below it" low detachment.

Low detachment goes by names like  "cynicism" or "nihilism". At its worst, one begins to take pleasure in one's own hopelessness, epitomized by this thought: I believe we are all doomed and there is nothing we can do about it. Isn't that metal!"  If you find yourself thinking along those lines, imagine a man in a car hurtling towards a cliff thinking, I believe I am doomed and there is nothing I can do about it. Isn't that metal!".

High detachment goes by names like "enlightenment" and "awakening" and sometimes even "stoicism" It combines the, largely correct, realization that a great deal of suffering is caused by one's internal reactions to external events and combines this with the more questionable prescription of using this understanding to self-modify yourself towards a sort of "soft salvation" of resignation, acceptance, and the cultivation of inner peace.

One former hero of mine (a brilliant mathematician who was planning to work in alignment after grad school) was completely demoralized by this line of thinking.

He seems happier now. He seems more fulfilled. He seems to enjoy his meditation retreats. He seems to have stopped working on avoiding the catastrophe that will kill him and most of the things he used to care about. I consider this to be something of a shame.

Though DF's anesthetic regimen may have provided some relief, it was by no means a cure:

At 16 months, his symptoms included consistently elevated body temperature (as high as 102°F), profuse sweating, serious impairment of short-term memory (for which he compensated by keeping lists), difficulty maintaining attention (he often did not know that the phone was ringing), difficulty distinguishing reality from fantasy (he didn't remember whether he had called a friend or had only imagined doing so), persistent headaches, hallucinations while driving (believed he saw people on the road when it was, in fact, empty), panic attacks (which were treated with meprobate), and a complete loss of sense of time.

In this same month his condition worsened further and we began to see hints of DF's unusual mental strength and creativity:

[...]DF spent much of the day as an akinetic mute with terrible headaches, confusion, mood swings, and myoclonus of the left arm (treated with levodopa). Despite his outward “dementia,” he inwardly pondered approaches to his condition, and, when again able to speak, he requested a regimen of stimulants.

He was prescribed phentermine HCl 37.5 mg [...]The drug had immediate and dramatic effects, promoting not only alertness during the day, but apparently a sleep-inducing rebound when it wore off.

Once phentermine became ineffective, he moved on to other stimulants.

[...] At that point, methylphenidate offered some relief. After a few days, however, DF had a grand mal seizure and was again hospitalized. Although his thinking was clear and oriented, his speech was labored, dysarthric, and perseverative, and his fever had returned.

As the stimulants and their come-downs faded in efficacy, DF began to try to physically exhaust himself, forcing himself (in a state of complete mental exhaustion) to go on long hikes.

In the 19 months from the onset of his symptoms and, one presumes, in a state of unimaginable desperation, he got more creative:

Noting that his grand mal seizure was followed by restful sleep, DF sought to duplicate the experience with electroconvulsive therapy (ECT). Beginning in the 19th month of his illness, he subjected himself to 30 sessions during several weeks.

[...]At 22 months into his illness, DF purchased a sensory deprivation tank; a man-sized, egg-shaped chamber designed to eliminate all sensory input. DF became interested in this chamber because his sleep was constantly disturbed by any small sound, light, or motion.

[...] Because of inconsistent results in the sleep tank, DF explored ways to externally bias his biorhythms to favor sleep at specific times. These involved daily exercise; exposure to sufficient sunlight; and timed use of melatonin, diazepam, and tryptophan. Early, but not later, in the course of his disease, this combination was effective.

From a conversation with TurnTrout on Discord

Let me share some of my experience and you tell me if it resonates. I think there's an intense mindset you can pick up from the Tsuyoku Naritai sequence which is like, "It's anime protagonist time", and maybe that works for a while. It worked a while for me. But telling yourself "It's time to be a dialed-in badass because the eyes of squintillions of future potential minds are on me, they need me"... doesn't seem like the way to actually be a dialed-in badass? I haven't found a way to sustainably make that mindset work. I agree that if you had write-access to your mood, then yes, consider doing that. But also, we are people

And so I say "we are people" not as a coping mechanism which is like "I don't want to do more work so I'll just say 'not possible'", I say it because I honestly don't know how to sustainably do the thing I think you're pointing at. If someone knows how, I want to learn

Just over 2 years after the onset of symptoms DF died of cardiac arrest, the result of heart damage from his FFI, his variegated treatments, and possibly drug withdrawal. He had lived 18 months longer than the typical case of FFI does.

In my mind, he died a hero's death.

It is probably too high-school to end with the Dylan Thomas quotation. So I will just emphasize the rationalist cliche that we should strive to feel the emotion appropriate to the predicament we are in.

TurnTrout argues that Tsuyoku Naritai is not it, and maybe he is right. I do not know what the correct emotion feels like, but I think maybe DF knew.

Too high-school -  but then again it is often worthwhile to look a little silly:

And you, my father, there on the sad height,
Curse, bless, me now with your fierce tears, I pray.
Do not go gentle into that good night.
Rage, rage, against the dying of the light.

Discuss

### Generalised models: imperfect morphisms and informational entropy

9 июля, 2021 - 20:35
Published on July 9, 2021 5:35 PM GMT

To make these into a category, a morphism r between M0=(F0,Q0) and M1=(F1,Q1) was defined to be a relation r between W0 and W1 (ie a subset of W0×W1), that obeyed certain conditions with respect to the Q's.

If r obeys those conditions, we can construct the "underlying model" for the morphism, a generalised model Mr=(F0⊔F1,Qr), with Qr non-zero only on r∈W0×W1.

That underlying model result basically says:

• There is an underlying reality Mr of which M0 and M1 are different, consistent, facets.

That's well and good, but we also want to allow imperfect correspondences, where Q0 or Q1 (or both) might be known to be in error. After all, when we transitioned from Newtonian mechanics to relativity, it wasn't because there was an underlying reality that both were facets of. Instead, we realised that Newtonian mechanics and relativity were approximately though not perfectly equivalent in low-energy situations, and that when they diverged, relativity was overall more accurate.

We'd want to include these cases as generalised model, and measure how "imperfect" the morphism between them is, i.e. how much Q0 and Q1 diverge from being in perfect correspondence. We'll also look at how the Q and the feature sets are related - how much information Q caries relative to F.

Imperfect morphisms

So we'll loosen the definition of "morphisms" so that the Q0 and Q1 need not correspond exactly to each other or an underlying reality. If we have a relation r between W0 and W1, here are different Q-consistency requirements that we could put on r to make it into a morphism (the previous requirement has been renamed "Q-preserving", condition 5):

1. General binary relations r; no connection assumed between r and the Qs.
2. Q-relational: for all w0∈W0 with 0">Q0(w0)>0, there exists at least one w1∈W1 with both 0">Q1(w1)>0 and (w0,w1)∈r.
3. Q-functional: for all w0∈W0 with 0">Q0(w0)>0, there exists a unique w1∈W1 with both 0">Q1(w1)>0 and (w0,w1)∈r.
4. Q-birelational: r and r−1 are both Q-relational.
5. Q-preserving: the same conditions as presented here. For all w0∈W0 and w1∈W1, Q0(w0)≤Q1(r(w0)) and Q1(w1)≤Q0(r−1(w1)).
6. Q-isomorphic: r is Q-preserving; both r and r−1 are Q-functional.

The general results about these morphisms classes are (proof found in this footnote[1]):

• All the above conditions are associative (hence define classes of morphisms for different categories with the same objects): if r and p are Q-relational, Q-functional, Q-relational, Q-birelational, Q-preserving, Q-isomorphic or disconnected from Q entirely, then so is pr=p∘r.
• Every morphism fulfils the conditions of the morphisms above it on the list, except that Q-preserving and Q-birelational need not imply Q-functional.
• If r is Q-isomorphic, then we can pair up each non-measure zero elements w0∈W0 and w1∈W0 so that (w0,w1)∈r and Q0(w0)=Q1(w1).
Examples of morphisms

Here are four relations:

A coarsening is when multiple worlds get related to a single world, thus losing details. A refinement is the opposite: a single world gets related to multiple worlds, thus adding more details. An inclusion adds more worlds to the set. Its opposite, a restriction, removes worlds.

In terms of morphisms, if we assume that all the worlds in those sets have non-zero probability, then coarsenings, refinements, and inclusions are all Q-relational and Q-functional. Restrictions are neither. Coarsenings and refinements are Q-birelational, while inclusions and restrictions are not.

As for Q-preserving, coarsenings are Q-preserving if, for all w1∈W1, the sum of Q0(w0) for all (w0,w1)∈r, is equal to Q1(w1). Similarly, refinements are Q-preserving if, for all w0∈W0, the sum of the Q1(w1) for all (w0,w1)∈r, is equal Q0(w0). None of these four relations are Q-isomorphic (unless some of the worlds in the diagrams are of measure 0).

What about Bayesian updates? If we start with M0=({f}⊔F,Q0) and want to update on f=c for some constant c, then this corresponds to relating M0 to M1={F,Q1} such that (w0,w1)∈r if w0=(w1,f=c). We'll also require Q0(w0)=Q1(w1) on any such worlds - and assume that that defines Q1 entirely (we're ignoring renormalisation here, since we don't assume that Q0 and Q1 have measure 1).

This r is clearly a restriction, and hence does not meet any of the Q-consistency conditions. However, we can define Bayesian updates as relations r such that r−1 is Q-functional and injective. They do form a category when seen this way (since (rq)−1=q−1r−1).

Comparing the Q's

Given two generalised models M0=(F0,Q0) and M1=(F1,Q1), with a relation r between them, we'll now compare Q0 and Q1. We'll do this by defining a length operator L that gives the "length" of r, which is a measure of the divergence between Q0 and Q1 "along" the relation r.

Let (Q′0,Q′1) be a pair of probability distributions on W1 and W1, respectively. We'll say the pair is r-compatible if r is a Q-preserving morphism between (F0,Q′0) and (F1,Q′1).

Since Qi and Q′i are distributions over the same Wi, we can compare their l1 norm, defined as: ||Qi−Q′i||1=∑wi∈Wi|Qi(wi)−Q′i(wi)|. Then define L(r,Q′0,Q′1) as the sum of the l1-distances to Q0 and Q1:

L(r,Q′0,Q′1)=||Q0−Q′0||1+||Q1−Q′1||1.

We'll define L(r) to be the minimum[2] value of this norm among all the r-compatible (Q′0,Q′1). Because of its definition, it's immediately obvious that L(r)=L(r−1). The other key properties - proved here[3] - are that:

• If r is Q-relational, then there exists a Q′1 such that L(r)=L(r,Q0,Q′1); ie we can use Q0 itself rather than finding a Q′0.
• If r and p are Q-birelational, the L is a sensible length operator, in that L(pr)≤L(r)+L(p).
• r is Q-preserving iff L(r)=0.

It's that last property that makes L such a useful distance metric: it measures the extent to which M0 and M1 fail to be aspects of the same underlying reality.

Relating features and probability distributions

In the above we've been looking at the relationship between r and Q, but have looked little at the features. Here we'll look at some of the relations between features and probability distributions. The idea is to measure how related the features are to each other.

There are several candidates for a measure of this types, but the most interesting seems to be a generalisation of mutual information. For any feature F∈F, we have the marginal distribution QF of Q over the feature F. Then if H is the informational entropy of a probability distribution/random variable, we can define a measure over Q given F as:

−H(Q)+∑F∈FH(QF).

If we define QF as the product distribution ∏F∈FQF, then that can also be defined as DKL(Q||QF), where DKL is the KL-divergence from QF to Q.

Example: gas laws

As an illustration, consider the ideal gas laws: PV=nRT, where P is pressure, V is volume, n is the amount of substance, R is the ideal gas constant, and T is the temperature. We'll consider a simple model with constant amount of substance, setting nR=1, so the law reduces to:

PV=T

We'll allow these variables to take a few values: P and V are integers that range from 1 to 4, while T is an integer that ranges from 1 to 16. The probability distribution Q will give uniform probability[4] 1/16 to each (P=p,V=v,T=pv).

In this case, Q is uniform among the 16 worlds where it is non-zero; hence

H(Q)=log2(16)=4.

This characterises Q, but doesn't touch the features F. So, let QP, QV, and QP be the marginal distributions over the features. Both QP and QV are uniform over 4 elements, so H(QP)=H(QV)=2. As for QT, it is 1/16 over {1,9,16}, 2/16 over {2,3,6,8,12}, and 3/16 over {4}. Some calculations then establish that H(QT) is (54−3log2(3))/16. Then the KL-divergence from QF=QVQPQT to Q is:

DKL(Q||QF)=−4+2+2+54−3log2(3)16=54−3log2(3)16≈3.08.

Let us add another variable T′ to the feature set, which is just equal to T, but with another name, and see how things change. Then H(Q) is unchanged, and DKL(Q||QF) adds another 54−3log2(3)16, corresponding to H(T′).

1. We've already shown that Q-preserving morphisms are associative. The composition of general binary relations is known to be associative too.

So now assume that r:M0→M1 and p:M1→M2 are both Q-relational. Let w0∈W0 be such that 0">Q0(w0)>0. Then, because r is Q-relational, there exists a w1∈W0 with 0">Q1(w1)>0 and (w0,w1)∈r. Then since p is Q-relational, there exists a w2∈W2 with 0">Q2(w2)>0 and (w1,w2)∈p. Combining the two gives (w0,w2)∈pr. Thus Q-relational morphisms are associative. Applying the same argument to r−1 shows that Q-birelational morphisms are also associative. A variant of the same argument with "there exists a unique w1∈W0" instead of "there exists a w1∈W0" shows that Q-functional morphisms are also associative. Hence Q-isomorphic r are also functional.

Now assume that r is Q-preserving and let w0∈W0 be such that 0">Q0(w0)>0. Then there exists an underlying model Mr=(F0⊔F1,Qr) such that Q0(w0)=∑(w0,w1)∈rQr(w0,w1). Since this sum is greater than zero, there exists a w1 such that 0">Qr(w0,w1)>0. Then since Q1(w1)=∑(w′0,w1)∈rQr(w′0,w1) and all the terms are non-negative, 0">Q1(w1)>0. The same argument works if we started with a w1∈W1 such that 0">Q0(w1)>0; thus r must be Q-birelational. This proves that Q-preserving implies Q-birelational.

It's trivial that Q-birelational implies Q-relational, and that Q-functional also implies Q-relational, since "exists a unique" is strictly stronger than "exists a". By definition, Q-isomorphic implies Q-preserving (hence Q-birelational and Q-relational) and Q-functional.

Now let r be Q-isomorphic, and let w0 be such that 0">Q0(w0)>0. Since r is Q-functional, there exists a unique w1 with (w0,w1)∈r and 0">Q1(w1)>0. Since r−1 is also Q-functional, there are no other w′0 with (w′0,w1)∈r and 1">Q0(w′0)>1. So, among the elements of non-zero measure, w0 and w1 are related only to each other. Then since r is Q-preserving, Q0(w0)≤Q1(r(w0)=Q1(w1)+0, and Q1(w1)≤Q0(r−1(w0))=Q0(w0)+0. Hence Q0(w0)=Q1(w1). ↩︎

2. This will be a minimum, not an infimum. Let (Q′0,Q′1) be an r-compatible pair with L(r,Q′0,Q′1)=μ. Then if we restrict to pairs (Q′0,Q′1) with L(r,Q′0,Q′1)≤μ, this is a compact non-empty set, so L(r,−,−) must reach its minimum on this set. ↩︎

3. For r a relation between M0=(F0,Q0) and M1=(F1,Q1), let Qr be the set of r-compatible (Q′0,Q′1) that minimises L(r,Q′0,Q′1). Hence L(r,−,−)=L(r) on Qr.

So we have this non-empty Qr; what we'll show is that, if r is Q-relational, then there is a Q′1 such that (Q0,Q′1)∈Qr. We'll need the following lemma:

• Lemma A: For any (Q′0,Q′1) in Qr with 0">||Q0−Q′0||1>0, we can find another pair (Q′′0,Q′′1)∈Qr with Q′′0 closer (in the l1 norm) to Q0 than Q′0 is.

To prove the lemma, pick any (Q′0,Q′1)∈Qr with 0">||Q0−Q′0||1>0. That l1 norm is a sum of positive terms, so there must exist a w0∈W0 with 0">|Q0(w0)−Q′0(w0)|>0.

Assume first that Q0(w0)<Q′0(w0). Then, since 0">Q′0(w0)>0 and r is a Q-preserving morphism between Q′0 and Q′1, there is an underlying model Mr=(F0∪F1,Qr) so that 0">Q′0(w1)=∑(w0,w1)∈rQr(w0,w1)>0. Thus there is a (w0,w1)∈r with 0">Qr(w0,w1)>0. Pick 0">ϵ>0 to be less than Qr(w0,w1) and |Q0(w0)−Q′0(w0)|, and define:

Q′′0(w0)=Q′0(w0)−ϵQ′′1(w1)=Q′1(w1)−ϵ,

with Q′′0=Q′0 and Q′′1=Q′1 on all other points. Because Q′′0(w0) is closer to Q0(w0) (by ϵ) than Q0 is, ||Q0−Q′′0||1=||Q0−Q′′0||1−ϵ. Furthermore, r is Q-preserving between Q′′0 and Q′′1 (the underlying model has the same Qr except increased by ϵ on (w0,w1)), and ||Q1−Q′′1||1 has gone up by at most ϵ over ||Q1−Q′1||1; thus the sum ||Q0−Q′′0||1+||Q1−Q′′1||1 has not increased. Hence (Q′′0,Q′′1)∈Qr.

Now consider the other case: Q_0'(w_0)">Q0(w0)>Q′0(w0). Then since 0">Q0(w0)>0 and r is Q-relational, there must exist a (w0,w1)∈r. We define Q′′0 and Q′′1 as above, except that we add ϵ instead of subtracting it; the rest of the argument is the same. This proves the lemma □.

Back to the main proof. Since Qr is compact, the l1 distance to Q0 must reach a minimum on Qr. By lemma A, this minimum can only be 0 (since if it were greater than 0, we could find a pair with a smaller l1 distance to Q0). If (Q′0,Q′1)∈Qr is a pair on which it reaches this minimum, we must have ||Q0−Q′0||l=0, ie Q′0=Q0.

Now assume r is Q-birelational between M0 and M1, while p is Q-birelational between M1 and M2. Then L(r)+L(p)=L(r−1)+L(p). Since r−1 and p are Q-relational, there exists Q′0 and Q′2 such that this is L(r−1,Q1,Q′0)+L(p,Q1,Q′2), and (Q′0,Q1) is r-compatible, while (Q1,Q′2) is p-compatible.

This implies that (Q′0,Q′2) is pr-compatible, and thus L(pr,Q′0,Q′2)≥L(pr). However, L(r−1,Q1,Q′0)+L(p,Q1,Q′2)=||Q0−Q′0||1+0+0+||Q2−Q′2||1=L(pr,Q′0,Q′2), giving our result:

L(pr)≤L(r)+L(p).

Finally, notice that L(r)=0 implies that there exists an r compatible (Q′0,Q′1) with ||Q0−Q′0||1=0 and ||Q1−Q′1||1=0. Thus (Q0,Q1) themselves are r-compatible, ie r is Q-preserving. Conversely, if (Q0,Q1) are r-compatible, then L(r)≤||Q0−Q0||1+||Q1−Q1||1=0, and thus L(r)=0. ↩︎

4. Note that this means that many values of T are impossible in this model, such as 7 and 10, which cannot be expressed as the product of integers 4 or less. ↩︎

Discuss

### For reducing caffeine dependence, does daily maximum matter more, or does total daily intake matter more?

9 июля, 2021 - 18:40
Published on July 9, 2021 3:40 PM GMT

I'm trying to reduce my dependence on caffeine from needing ~160mg in the morning to needing 80mg. My strategy is to slowly reduce my intake, dropping maybe 20mg every 3 days. So far so good.

But sometimes I feel tired later in the day, and also want caffeine. So I have 80mg more, 10 hours after the morning 160mg. Is this going to screw up my attempt at tolerance reduction?

The argument for Yes: Daily caffeine intake is what matters. You can't reduce yourself down to needing less than 160mg in the morning by taking 240mg a day.

The argument for No: The important thing is the max amount of caffeine in you at any given time. Since the half-life of caffeine is between 1.5 and 9.5 hours (with a middle guess of 5 hours), by evening, you only have 160mg * 0.5 * 0.5 = 40mg in your system. Adding 80mg brings you to 120mg. This is less than the 160mg max dose your body is used to hitting, so it's not a big deal. As long as you keep the maximum caffeine in your system below 160mg, you can keep reducing your tolerance.

Which of these arguments is more right? Or is there a third view of the situation, better than either?

Discuss

### Finite Factored Sets: Conditional Orthogonality

9 июля, 2021 - 09:01
Published on July 9, 2021 6:01 AM GMT

4.1 Generating a Subpartition

Definition 20 (subpartition). A subpartition of a set S is a partition of a subset of S. Let SubPart(S)=⋃E⊆SPart(E) denote the set of all subpartitions of S.

Definition 21 (domain). The domain of a subpartition X of S, written dom(X), is the unique E⊆S such that X∈Part(E).

Definition 22 (restricted partitions). Given sets S and E and a partition X of S, let X|E denote the partition of S∩E given by X|E={[e]X∩E∣e∈E}.

Definition 23 (generating a subpartition). Given a finite factored set F=(S,B), and X∈SubPart(S), and a C⊆B, we say C generates X (in F), written C⊢FX, if χFC(x,dom(X))=x for all x∈X.

Note that this definition clearly coincides with Definition 16, when X has domain S. Despite the similarity of the definitions, the idea of generating a subpartition is a bit more complicated than the idea of generating a partition of S.

To see this, consider the following list of equivalent definitions. Notice that while the first five directly mirror their counterparts in Proposition 10, the last two (and especially the last one) require an extra condition.

Proposition 20. Let F=(S,B) be a finite factored set, let X∈SubPart(S) be a subpartition of S, let E=dom(X) be the domain of X, and let C be a subset of B. The following are equivalent.

1. C⊢FX.
2. χFC(x,E)=x for all x∈X.
3. χFC(x,E)⊆x for all x∈X.
4. χFC(x,y)⊆x for all x,y∈X.
5. χFC(s,t)∈[s]X for all s,t∈E.
6. χFC(s,t)∈E and χFC(s,t)∼Xs for all s,t∈E.
7. X≤E(⋁S(C)|E) and χFC(E,E)=E.

Proof. The equivalence of conditions 1 and 2 is by definition.

The equivalence of conditions 2 and 3 follows directly from the fact that χFC(s,s)=s for all s∈x, so χFC(x,E)⊇χFC(x,x)⊇x.

To see that conditions 3 and 4 are equivalent, observe that since E=⋃y∈Xy, χFC(x,E)=⋃y∈XχFC(x,y). Thus, if χFC(x,E)⊆x, χFC(x,y)⊆x for all y∈X, and conversely if χFC(x,y)⊆x for all y∈X, then χFC(x,E)⊆x.

To see that condition 3 is equivalent to condition 5, observe that if condition 5 holds, then for all x∈X, we have χFC(s,t)∈[s]X=x for all s∈x and t∈E. Thus χFC(x,E)⊆x. Conversely, if condition 3 holds, χFC(s,t)∈χFC([s]X,E)⊆[s]X for all s,t∈E.

Condition 6 is clearly a trivial restatement of condition 5.

To see that conditions 6 and 7 are equivalent, observe that if condition 6 holds, then χFC(s,t)∈E for all s,t∈E, so χFC(E,E)⊆E, so χFC(E,E)=E. Further, if s,t∈E satisfy s∼⋁S(C)|Et, then s∼ct for all c∈C, so χFC(s,t)=t, so t=χFC(s,t)∼Xs. Thus X≤E⋁S(C)|E.

Conversely, if condition 7 holds, then for all s,t∈E, we have χFC(s,t)∼⋁S(C)s, so χFC(s,t)∼⋁S(C)|Es, and thus χFC(s,t)∼Xs. Further, clearly χFC(E,E)=E implies χFC(s,t)∈E for all s,t∈E. □

The first half of condition 7 in the above proposition can be thought of as saying that the values of factors in C are sufficient to distinguish between the parts of X.

The second half can be thought of as saying that no factors in C become entangled with any factors outside of C when conditioning on E. This second half is actually necessary (for example) to ensure that the set of all C that generate X is closed under intersection. As such, we will need this fact in order to extend our notion of history to arbitrary subpartitions.

Proposition 21. Let F=(S,B) be a finite factored set, let C and D be subsets of B, let X,Y,Z∈SubPart(S) be subpartitions of S, and let dom(X)=dom(Y)=E.

1. If X≤EY and C⊢FY, then C⊢FX.
2. If C⊢FX and C⊢FY, then C⊢FX∨EY.
3. B⊢FX.
4. {}⊢FX if and only if X=IndE.
5. If C⊢FX and D⊢FX, then C∩D⊢FX and C∪D⊢FX.
6. If X⊆Z, and C⊢FZ, then C⊢FX.

Proof. The first 4 parts will use the equivalent definition from Proposition 20 that C⊢FX if and only if X≤S⋁S(C). 1 and 2 are immediate from this definition.

3 follows directly from Definition 23.

4 follows directly from the fact that ⋁S({})=IndS, and IndS|E=IndE so X≤E⋁S(C)|E if and only if X=IndE.

For part 5, we will use the equivalent definition from Proposition 20 that C⊢FX if and only if χFC(s,t)∈[s]X for all s,t∈E. Assume that for all s,t∈E, χFC(s,t)∈[s]X and χFD(s,t)∈[s]X. Thus, for all s,t∈E, χFC∩D(s,t)=χFC(χFD(s,t),t)∈[χFD(s,t)]X=[s]X. Similarly, for all s,t∈E, χFC∪D(s,t)=χFC(s,χFD(s,t))∈[s]X. Thus C∩D⊢FX and C∪D⊢FX.

For part 6, we use the definition that C⊢FX if and only if χFC(x,y)∈x for all x,y∈X. Clearly if X⊆Z, and χFC(x,y)∈x for all x,y∈Z, then χFC(x,y)∈x for all x,y∈X.

Note that while the set of C that generate an X∈Part(S) is closed under supersets, the set of C that generate an X∈SubPart(S) is merely closed under union.

Further note that part 6 of Proposition 21 uses the subset relation on subpartitions, which is a slightly unnatural relation.

4.2 History of a Subpartition

Definition 24 (history of a subpartition). Given a finite factored set F=(S,B) and a subpartition X∈SubPart(S), let hF(X) denote the smallest (according to the subset ordering) subset of B such that hF(X)⊢FX.

Proposition 22. Given a finite factored set F=(S,B)hF:SubPart(S)→P(B) is well-defined, and if X is a partition of S, this definition coincides with Definition 17.

Proof. Fix a finite factored set F=(S,B) and a subpartition X∈SubPart(S), and let hF(X) be the intersection of all C⊆B such that C⊢FX. It suffices to show that hF(X)⊢FX. Then hF(X) will clearly be the unique smallest (according to the subset ordering) subset of B such that hF(X)⊢FX. The fact that this definition coincides with Definition 17 if X∈Part(S) is clear.

Note that hF(X) is a finite intersection, since there are only finitely many subsets of B, and that hF(X) is a nonempty intersection since B⊢FX. Thus, we can express hF(X) as a (possibly empty) composition of finitely many binary intersections. By part 5 of Proposition 21, the intersection of two subsets that generate X also generates X. Thus hF(X)⊢FX.

We will now give five basic properties of the history of subpartitions, followed by two more properties that are less basic.

Proposition 23. Let F=(S,B) be a finite factored set, let X,Y,Z∈SubPart(S) be subpartitions of S, and let dom(X)=dom(Y)=E.

1. If X≤EY, then hF(X)⊆hY(Y).
2. hF(X∨EY)=hF(X)∪hF(Y).
3. If X⊆Z, then hF(X)⊆hF(Z).
4. hF(X)={} if and only if X=IndE.
5. If S is nonempty, then hF(b)={b} for all b∈B.

Proof. Parts 1, 3, and 4 are trivial consequences of Proposition 21, and part 5 is just a restatement of part 4 of Proposition 13.

For part 2, first observe that hF(X∨EY)⊇hF(X)∪hF(Y), by part 1 of Proposition 21. Thus it suffices to show that hF(X)∪hF(Y)⊇hF(X∨EY), by showing that hF(X)∪hF(Y)⊢FX∨EY.

We will use condition 7 in Proposition 20. Clearly

X≤E(⋁E(hF(X))|E)≤E(⋁S(hF(X)∪hF(Y))|E),

and similarly,

Y≤E(⋁E(hF(Y))|E)≤E(⋁Se(hF(X)∪hF(Y))|E).

Thus, X∨EY≤E(⋁S(hF(X)∪hF(Y))|E).

Next, we need to show that χFhF(X)∪hF(Y)(E,E)=E. Clearly E⊆χFhF(X)∪hF(Y)(E,E).

Let s and t be elements of E, and observe that χFhF(X)∪hF(Y)(s,t)=χFhF(X)(s,χFhF(Y)(s,t)). We have that χFhF(Y)(s,t)∈E, since χFhF(Y)(E,E)=E. Thus, we also have that χFhF(X)(s,χFhF(Y)(s,t))∈E, since χFhF(X)(E,E)=E. Thus, χFhF(X)∪hF(Y)(E,E)⊆E.

Thus we have that X∨EY≤E(⋁S(hF(X)∪hF(Y))|E) and χFhF(X)∪hF(Y)(E,E)=E. Thus, by condition 7 in Proposition 20, hF(X)∪hF(Y)⊢FX∨EY, so hF(X∨EY)=hF(X)∪hF(Y). □

Lemma 1. Let F=(S,B) be a finite factored set, and let X,Y∈Part(E) be subpartitions of S with the same domain. If hF(X)∩hF(Y)={}, then hF(X)=hF(X|y) for all y∈Y.

Proof. Let F=(S,B) be a finite factored set, let E⊆S, and let X,Y∈Part(E).

We start by showing that (B∖hF(X))⊢FY and (B∖hF(Y))⊢FX. Observe that χB∖hF(X)(E,E)=χhF(X)(E,E)=E. Further observe that B∖hF(X)⊇hF(Y), so ⋁S(B∖hF(X))≥S⋁S(hF(Y)), so (⋁S(B∖hF(X))|E)≥E(⋁S(hF(Y))|E)≥EY. Thus, (B∖hF(X))⊢FY. Symmetrically, (B∖hF(Y))⊢FX.

Fix some y∈Y. We start by showing that hF(X)⊇hF(X|y).

We have that χFB∖hF(X)(y,E)⊆y, so χFhF(X)(E,y)⊆y, so for all x∈X, we have χFhF(X)(x∩y,y)⊆y. We also have χFhF(X)(x∩y,y)⊆χFhF(X)(x,E)⊆x. Thus  χFhF(X)(x∩y,y)⊆x∩y. Every element of X|y is of the form x∩y for some x∈X, so we have hF(X)⊢F(X|y), so hF(X)⊇hF(X|y).

Next, we need to show that hF(X)⊆hF(X|y). For this, it suffices to show that hF(X|y)⊢FX. Let s,t be arbitrary elements of E. It suffices to show that χFhF(X|y)(s,t)∈[s]X.

First, observe that since (B∖hF(Y))⊇hF(X)⊇hF(X|y), we have that χFhF(X|y)(s,t)=χFB∖hF(Y)(χFhF(X|y)(s,t),t).

Let r be an arbitrary element of y. We thus have:

χFhF(X|y)(s,t)=χFB∖hF(Y)(χFhF(X|y)(s,t),t)=χFB∖hF(Y)(χFhF(Y)(r,χFhF(X|y)(s,t)),t)=χFB∖hF(Y)(χFhF(X|y)(χFhF(Y)(r,s),χFhF(Y)(r,t)),t).

Let s′=χFhF(X|y)(χFhF(Y)(r,s),χFhF(Y)(r,t)). Note that χFhF(Y)(r,t) and χFhF(Y)(r,s) are both in y. Thus we have that s′∈[χFhF(Y)(r,s)](X|y). Since (B∖hF(Y))⊢FX, χFhF(Y)(r,s)=χFB∖hF(Y)(s,r)∈[s]X. Thus [χFhF(Y)(r,s)](X|y)⊆[χFhF(Y)(r,s)]X=[s]X, so s′∈[s]X.

We have that χFhF(X|y)(s,t)=χFB∖hF(Y)(s′,t). However, since B∖hF(Y)⊢FX, we have χFB∖hF(Y)(s′,t)∈[s′]X=[s]X. Thus, hF(X)⊆hF(X|y), so hF(X)=hF(X|y). □

Lemma 2. Let F=(S,B) be a finite factored set. Let E⊆S and let X,Y∈Part(E) be subpartitions of S with the same domain. Then hF(X∨EY)=hF(X)∪⋃x∈XhF(Y|x).

Proof. Since X≤EX∨EY, we have hF(X)⊆hF(X∨EY). Similarly, for all x∈X, since Y|x⊆X∨EY, we have hF(Y|x)⊆hF(X∨EY). Thus, hF(X∨EY)⊇hF(X)∪⋃x∈XhF(Y|x). We still need to show that hF(X∨EY)⊆hF(X)∪⋃x∈XhF(Y|x).

We start with the special case where |X|=2. Let X={x0,x1}. In this case, we want to show that hF(X∨EY)=hF(X)∪hF(Y|x0)∪hF(Y|x0). Let C=hF(X), let C0=hF(Y|x0), and let C1=hF(Y|x1).

Consider arbitrary s,t∈E. Without loss of generality, assume that s∈x0, and let y=[s]Y. It suffices to show that χFC∪C0∪C1(s,t)∈x0∩y. Fix some r∈x1.

χFC∪C0∪C1(s,t)=χFC0(s,χFC(s,χFC1(s,t)))=χFC0(s,χFC(s,χFC(r,χFC1(s,t))))=χFC0(s,χFC(s,χFC1(χFC(r,s),χFC(r,t)))).

Observe that χFC(r,s) and χFC(r,t) are both in x1, so χFC1(χFC(r,s),χFC(r,t))∈x1, and thus is in E. Combining this with the fact that s∈x0 gives us that χFC(s,χFC1(χFC(r,s),χFC(r,t)))∈x0. Thus, since s∈x0∩y, χFC∪C0∪C1(s,t)=χFC0(s,χFC(s,χFC1(χFC(r,s),χFC(r,t))))∈x0∩y.

Now, consider the case where |X|≠2. If |X|=0, then E={}, so all subpartitions involved are empty, and thus have the same (empty) history. If |X|=1, let X={E}. Then

hF(X∨EY)=hF(Y)=hF(Y|E)⊆hF(X)∪hF(Y|E)=hF(X)∪⋃x∈XhF(Y|x).

Thus, we can restrict our attention to the case where |X|≥3.

Observe that X∨EY=⋁E({(Y|x)∪{E∖x}∣x∈X}). Thus hF(X∨EY)=⋃x∈XhF((Y|x)∪{E∖x}). However, from the case where |X|=2, we have

hF((Y|x)∪{E∖x})=hF({x,E∖x}∨E((Y|x)∪{E∖x}))=hF({x,E∖x})∪hF({E∖x})∪hF(Y|x).

hF({E∖x}) is empty, so this gives us that hF(X∨EY)=⋃x∈X(hF(Y|x)∪hF({x,E∖x})). Since ⋁E({{x,E∖x}∣x∈X})=X, ⋃x∈XhF({x,E∖x})=hF(X), so we have hF(X∨EY)=hF(X)∪⋃x∈XhF(Y|x). □

4.3 Conditional Orthogonality

We can also extend our notions of orthogonality and time to subpartitions.

Definition 25. Let F=(S,B) be a finite factored set. Let X,Y∈SubPart(S) be subpartitions of S. We write X⊥FY if hF(X)∩hF(Y)={}, we write X≤FY if hF(X)⊆hF(Y), and we write X<FY if hF(X)⊂hF(Y).

We give this definition in general, but it is not clear whether orthogonality and time should be considered philosophically meaningful when the domains of the inputs differ from each other. Further, the temporal structure of subpartitions will mostly be outside the scope of this paper, and the orthogonality structure on subpartitions will mostly just be used for the following pair of definitions.

Definition 26 (conditional orthogonality given a subset). Given a finite factored set F=(S,B), partitions X,Y∈Part(S), and E⊆S, we say X and Y are orthogonal given E (in F), written X⊥FY|E, if (X|E)⊥F(Y|E).

Definition 27 (conditional orthogonality). Given a finite factored set F=(S,B), and partitions X,Y,Z∈Part(S), if X⊥FY|z for all z∈Z, then we say X and Y are orthogonal given Z (in F), written X⊥FY|Z.

Unconditioned orthogonality can be thought of as a special case of conditional orthogonality, where you condition on the indiscrete partition.

Proposition 24. Given a finite factored set F=(S,B) and partitions X,Y∈Part(S)X⊥FY if and only if X⊥FY|IndS

Proof. If S={}, then there is only one partition X={}, and X⊥FX holds. Also, since IndS is empty, X⊥FX|IndS holds vacuously.

If S≠{}, then IndS={S}, so X⊥FY|IndS if and only if X⊥FY|S if and only if X|S⊥FY|S if and only if X⊥FY. □

The primary combinatorial structure of finite factored sets that we will be interested in is the structure of orthogonality (X⊥FY), conditional orthogonality (X⊥FY|Z), and time (X≤FY and X<FY) on inputs that are partitions.

We now will show that conditional orthogonality satisfies (a slight modification of) the axioms for a compositional semigraphoid.

Theorem 2. Let F=(S,B) be a finite factored set, and let X,Y,Z,W∈Part(S) be partitions of S.

1. If X⊥FY|Z, then Y⊥FX|Z.   (symmetry)
2. If X⊥F(Y∨SW)|Z, then X⊥FY|Z and X⊥FW|Z.   (decomposition)
3. If X⊥F(Y∨SW)|Z, then X⊥FY|(Z∨SW).   (weak union)
4. If X⊥FY|Z and X⊥FW|(Z∨SY), then X⊥F(Y∨SW)|Z.   (contraction)
5. If X⊥FY|Z and X⊥FW|Z, then X⊥F(Y∨SW)|Z.   (composition)

Proof. Symmetry is clear from the definition.

Decomposition and composition both follow directly from the fact that for all z∈Z, hF((Y∨SW)|z)=hF((Y|z)∨z(W|z))=hF(Y|z)∪hF(W|z).

For weak union, assume that X⊥F(Y∨SW)|Z. Thus, for all z∈Z, hF(X|z)∩hF((Y∨SW)|z)={}. In particular, this means that hF(X|z)∩hF(W|z)={}, so by Lemma 1, for all w∈W, hF(X|z)=hF(X|w∩z). Further, we have that for all w∈W, hF(Y|w∩z)⊆hF(Y∨SW|z). Thus, for all w∈W, hF(X|w∩z)∩hF(Y|w∩z)={}, which since every element of W∨SZ is of the form w∩z for some w∈W and z∈Z, means that X⊥FY|(Z∨SW).

Finally, for contraction, assume that X⊥FY|Z and X⊥FW|Z∨SY. Fix some z∈Z. We want to show that hF(X|z)∩hF((Y∨SW)|z)={}. We have that hF((Y∨SW)|z)=hF((Y|z)∨z(W|z)), and by Lemma 2, hF((Y|z)∨z(W|z))=hF(Y|z)∪⋃y∈YhF(W|(y∩z)). Thus, it suffices to show that hF(X|z)∩hF(Y|z)={} and hF(X|z)∩hF(W|(y∩z))={} for all y∈Y.

The fact that hF(X|z)∩hF(Y|z)={} follows directly from X⊥FY|Z.

Fix a y∈Y. If y∩z={}, then hF(W|(y∩z))={}, so hF(X|z)∩hF(W|(y∩z))={}. Otherwise, we have hF(X|z)=hF(X|(y∩z)) by Lemma 1, and we have that hF(X|(y∩z))∩hF(W|(y∩z))={}, since X⊥FW|Z∨SY, so we have hF(X|z)∩hF(W|(y∩z))={}.

Thus, X⊥F(Y∨SW)|Z. □

The first four parts of Theorem 2 are essentially the semigraphoid axioms. The difference is that the semigraphoid axioms are normally defined as a ternary relation on disjoint sets of variables. We use partitions instead of sets of variables, use common refinement instead of union, and have no need for the disjointness condition. The fifth part (composition) is a converse to the decomposition axiom that is sometimes added to define a compositional semigraphoid.

The results in this paper will not depend on the theory of compositional semigraphoids, so we will not need to make the analogy any more explicit, but it is nice to note the similarity to existing well-studied structures.

We also get a nice relationship between conditional orthogonality and the refinement order.

Proposition 25. Let F=(S,B) be a finite factored set, and let X,Y∈Part(S) be partitions of S. X⊥FX|Y if and only if X≤SY.

Proof. If X⊥FX|Y, then for all y∈Y,hF(X|y)={}, so X|y=indy, so for all s,t∈y, we have s∼X|yt, and thus s∼Xt. Thus, for all s,t∈S, if s∼Yt, then s∼Xt. Thus X≤SY.

Conversely, if X≤SY, observe that for all y∈Y, X|y=indy, so hF(X|y)={}. Thus, X⊥FX|Y. □

In the next post, we will prove the fundamental theorem of finite factored sets, which says that conditional orthogonality exactly corresponds to conditional independence in all probability distributions that can be put on the relevant finite factored set.

Discuss

### Improving capital gains taxes

9 июля, 2021 - 08:20
Published on July 9, 2021 5:20 AM GMT

As I’ve mentioned, I think the tax code could be improved. In a departure from my usual style, this post fleshes out some fairness-based arguments for one of my favorite changes.

(I think that this proposal, and many of the arguments in favor, is old. Wikipedia quotes Joseph Stieglitz making the basic point in Economics of the Public Sector.)

The policy

I’m advocating a package with 3 parts:

1. We should significantly increase capital gains taxes by taxing it at the same rate as ordinary income.
2. If an investor makes money one year then loses it later, we should give them their taxes back.
3. We should only tax excess returns above what you get by putting your money in a savings account.
Justifications 1. Why have a higher tax rate?

The simple answer is “most Americans think rich people should pay higher capital gains taxes.” It looks unfair for Warren Buffet to pay a lower marginal rate on income than a working class family. But there are other great reasons.

One is that it can be really hard to distinguish “income” from “investment income,” and many rich people misclassify income to pay fewer taxes. Some of these egregious loopholes lose a lot of tax revenue. But if we tax capital gains at the same rate as other income then these loopholes disappear and the whole system gets simpler.

Another is that rich people do a good job of identifying profitable investment opportunities; normal people can’t participate effectively in those opportunities, and so the rich get richer faster. That feels like a fact of life, but it’s actually pretty easy to change. A capital gains tax effectively lets everyone benefit from every investment opportunity.

2. Why pay investors back when they lose money?

When an investor makes money they pay taxes. But when they lose money they never get a check back. The government effectively participates in the returns but not the risk.

(Investors do get tax deductions they can use once they’ve recouped all their losses, but much of the reason people are afraid to lose money is that they might never make it back.)

This discourages investors from taking on risk and encourages them to push the risk onto other people. Combined with the proposed increase in capital gains rates, it seriously discourages uncertain long-term investments and probably reduces growth.

In addition to sounding a lot like “Heads I win, tails you lose,” I think this policy probably loses the government money.

Why? The risk of going broke pushes investors to be more conservative and causes them to have much less taxable income—which is bad for both them and the government. But because very few investors actually end up going broke, the government doesn’t save much money at all. We’ve managed to get the worst of all worlds.

(That’s coming from a pretty rough BOTEC. If the intuitive case actually depends on that calculation, I should check more carefully.)

The new alternative effectively makes the government a non-voting partner in every investment, sharing in both risk and returns. That make the system more fair, puts us all on the same team with aligned incentives, and makes everyone richer. Instead of discouraging rich people from finding opportunities, it motivates them to make bigger bets so that they can give everyone a share.

3. Why adjust the initial investment using the risk-free rate?

When I put $1,000 in a savings account, I get$2,000 in 30 years because “money now” is worth twice as much as “money later.” Everyone—rich and poor, household or government—can make the substitution at that same rate. We should only tax your savings account if we specifically want you to spend your money immediately rather than later.

It’s actually worse than that: over the last few decades all of my “earnings” from a savings account are just keeping up with inflation, so I’m paying taxes without getting any richer at all. The market expects this to keep happening for the next 30 years, so this isn’t a weird corner case.

Instead, we should tax the difference between what you earned and what anyone could have made by just putting the same amount of money in a savings account. That is, we should tax the stuff that is actually income, i.e. when you are actually doing work, taking risks, or exploiting connections.

The best argument for taxing savings is that rich people save a higher percentage of their income and we want to spread the wealth. But if that’s our goal then we can just raise income taxes on the richest people—we don’t need to take a convoluted approach. Justifications about “keeping the money moving” are based on a misunderstanding—when I “save” money I’m lending it to someone else who will use it immediately.

Discuss

### The news attention span

9 июля, 2021 - 04:44
Published on July 9, 2021 1:44 AM GMT

Every day, every daily news source sends out headlines to their readers via newspapers and the web. Most of the time, the top headlines have no relation to the previous day's headlines. Sometimes this is not true, such as during an election or a disaster, but even then, stories rarely remain on the front page for more than a couple days.

This implies that the most important thing to hear about changes constantly from day to day. I won't pretend that doesn't make sense. If you already heard about something yesterday, you probably don't need to hear about it again today, unless something changed in the situation.[1]

But the most important issues do not change as often as the headlines. If headlines are supposed to give information people can act on, it is strange that my best course of action is so disjointed. One day I am supposed to donate money to a political campaign, on the next I am supposed to protest big tech, and today I should write a letter to my senator about Afghanistan.[2] Perhaps I dream of a newspaper whose front page headline every day is:

The problem with such a newspaper is that they would go out of business. After all, if the headline has been the same for the last month, even if it is the most important action item in the world, people will stop learning anything from it. Your newspaper would be more of an oldspaper.

On the other hand, if you do not repeat the same point every week, people will completely forget about it and nothing will change. Remember when everyone was talking about Gamestop in January? It's still at \$200 a share, six months later, but major news sites have stopped reporting on it. Do we pretend that ransomware, critical race theory in K-12, or other issues of the day will not meet the same fate?

I think most people don't pretend to that. The news has a second function, besides informing concerted political action: it determines what people talk about at the dinner table, what they post about on Nextdoor, and what slogan they put in their Instagram bio. People get bored of talking about the same thing every day, which is why few people bring up "malaria this" and "clean drinking water that" in every conversation they have. The Discourse is not always about doing something in the world. Most people will never be activists or megadonors. They are just trying to show support for the right beliefs when they talk to their peers. The news media enables this brilliantly. Every couple of days, there a new conversation topic is published, usually coordinated across the whole country, and it is typically controversial or sensational.[3] So if we ignored all the low-impact news items that are dropped after a week, would our citizenry finally unite to solve the world's problems? Maybe not. Maybe we would just argue in circles about global development, instead of arguing in circles about more sensational stories. (Though it couldn't hurt to try...)

Now, if you'll excuse me, I have a blog post to write about something different from the last one.

Footnotes
1. The pandemic is unusual in this regard, because it is both very important and constantly changing. As such, it has dominated news coverage since March 2020. ↩︎

2. I suppose the argument then is that different people should be acting on different headlines. This seems partly true, though still, many headlines are not the most relevant issue for nearly anyone. Some others might argue that simply learning about current events is virtuous in and of itself. I disagree: If reading news does not help you or allow you to help others, then what is it good for? And anyway, why are current events the most important thing I should be learning about? Why doesn't the New York Times teach me how to fill out my taxes instead? ↩︎

3. Contrast this with most of the most important issues in the world, which are not controversial, not sensational, and which rarely have moments so unusual that the whole country would discuss them at once. ↩︎

Discuss