# Новости LessWrong.com

A community blog devoted to refining the art of rationality
Обновлено: 22 минуты 38 секунд назад

### a visual explanation of Bayesian updating

8 мая, 2021 - 22:45
Published on May 8, 2021 7:45 PM GMT

As a teaser here is the visual version of Bayesian updating:

But in order to understand that figure we need to go through the prior and likelihood!

You find me standing in a basketball court ready to shoot some hoops. What do you believe about my performance before I take a shot?. There are no good Null hypothesis here unless you happen to have a lot of knowledge about the average human basket ball performance!, and even so, why do you care whether I am significant different from the average?, You can fall back to the new statistics which is almost as good as the Bayesian approach, it but does not answer what you should believe before I take a shot.

p(θ)∼Beta(1,1)

Where θ is my probability of scoring, the distribution looks like this:

Completely Uniform, a great prior when you are totally oblivious.

I take a shot and miss (z=0), the likelihood of a miss looks like this:

(if you are extra currious, you can brush up on the math behind all the binary distributions here)

Notice that:

• p(z=0∣θ=0)=1, the likelihood that I always miss is 1
• p(z=0∣θ=0.5)=0.5, the likelihood that I miss half the time is 0.5
• p(z=0∣θ=1)=0, the likelihood that I always hit is 0, which is obvious as I can't score all the time if I just missed.

Notice that these likelihoods and not probabilities, but how likely the data are for different values of θ, so it is twice as likely:

p(z=0∣θ=0)p(z=0∣θ=0.5)=10.5=2

That the data z=0 was generated by θ=0 compared to θ=0.5.

Bayesian Updating Math

Here is Bayes theorem for the Bernoulli distribution with a Beta prior, where the parameter z is 1 when I score and 0 otherwise:

p(θ|z)=p(z∣θ)p(θ)p(z)

For technical reason p(z), the probability of the data, is difficult to calculate, it is however 'just a normalization constant' because it does not depend on θ which is my scoring probability, thus we can simply drop it and get an unnormalized posterior:

p(θ|z)∝p(z∣θ)p(θ)

An normalized posterior is simply a density function that does not sum to 1, which means when we plot it it looks 'correct' except we have screwed up the numbers on the y axis.

Visual Bayesian Updating

So now we have a 'square' prior p(θ)∼Beta(1,1) and we have a triangle likelihood p(z=0∣θ), if we multiply them together we get the unnormalized posterior, so we do:

p(θ|z)∝p(z∣θ)p(θ)

Which intuitively can be taught of as: the square makes everything equally likely, so the likelihood will dominate the posterior, or in dodgy math:

posterior∝square×triangle∝triangle

Here is the Figure:

Try to put your finger on the figure check that θ=0.5 is 1 for the square and 0.5 for the triangle and is thus 1×0.5=0.5 in the unnormalized posterior

I shoot again and score!

Now we use the previous posterior as the new prior, but because we score we get an 'opposite triangle' which is the likelihood of p(z=1∣θ)

Again we multiply the prior triangle by the likelihood triangle and get a blob centered on 0.5 as the posterior:

Notice how the posterior is peaked at θ=0.5, this is because the two triangles at the center have an unnormalized posterior density of 0.5×0.5=0.25 where at edges such as θ=0.9 they have 0.9×0.1=0.09

I shoot again and sore!

So now again the previous blob posterior is our new prior, which we multiply by the 'I scored triangle' resulting in a blob that has a mode above 0.5, which makes sense as I made 2/3 shots:

While this may seem like a cute toy example it's a totally valid way of solving a Bayesian posterior, and is the way all most popular bayesian books (Gelman[1], Kruschke[2] and McElreath[3]) introduce the concept!

Bayesian Updating using Conjugation

In the case of the Bernoulli events we can actually solve the posterior easily because the Beta is conjugated to the Bernoulli, conjugation is simply fancy statistics speak for it having a simple mathematical form, and that form is also a Beta distribution, thus you can update the beta distribution using this simple rule:

Beta(α+z,β+1−z)

So we Started with a prior with α=β=1

Beta(1,1)

Then we got a miss, z=0

Beta(1,2)

Then we got a hit, z=1

Beta(2,2)

Then we got a miss, z=1

Beta(3,2)

We can plot the Beta(3,2) posterior

Notice how the this posterior has the exact same shape as the one we got via updating, the only different is the numbers on the y-axis.

(Hi, if you made it this far please comment, if there were something that was not well explained, I care more about my statistics communication skills than my ego, so negative feedback is very welcome)

1. Gelman, Hill and Vehtari, “Regression and Other Stories” ↩︎

2. Richard McElreath "Statistical Rethinking" ↩︎

3. John Kruschke "Doing Bayesian Data Analysis: A Tutorial with R, JAGS, and Stan 2nd Edition" ↩︎

Discuss

### Pre-Training + Fine-Tuning Favors Deception

8 мая, 2021 - 21:36
Published on May 8, 2021 6:36 PM GMT

Thanks to Evan Hubinger for helpful comments and discussion.

Currently, to obtain models useful for some task X, models are pre-trained on some task Y, then fine-tuned on task X. For example, to obtain a model that can summarize articles, a large language model is first pre-trained on predicting common crawl, then fine-tuned on article summarization. Given the empirical success of this paradigm and the difficulty of obtained labeled data, I loosely expect this trend to continue.

I will argue that compared to the paradigm of training a model on X directly, training on Y then fine-tuning on X increases the chance of deceptive alignment. More specifically, I will argue that fine-tuning a deceptive model will produce a deceptive model and fine-tuning a non-deceptive model is more likely to produce a deceptive model than training from scratch.

From Does SGD Produce Deceptive Alignment?:

More specifically, we can single out three types of models that optimize for the base objective:

• internally aligned models optimize for the base objective because it is equal to the model's terminal objective
• corrigibly aligned models optimize for the base objective because their terminal objective is a robust pointer to a model of the base objective. Note that these are called corrigible because they are amenable to corrections of the model of the base objective (though not necessarily corrections to their pointer).
• deceptively aligned models optimize for the base objective for instrumental reasons

Assume that pre-training has produced a model that optimizes for the pre-training loss and is one of the above types. The arguments are weaker if pre-training is imperfect. If the model is not one of the above types, then these arguments do not apply.

We can analyze the three cases separately:

• If pre-training has produced an internally aligned model, then we are beginning fine-tuning with a mesa-optimizer that has a misaligned proxy objective. The probability of fine-tuning producing deception is controlled by how far the pre-training loss is from the fine-tuning loss and how far the current capabilities are from deception.
• If pre-training has produced a corrigibly aligned model, then we begin fine-tuning with a mesa-optimizer that has a misaligned pointer. The probability of fine-tuning producing deception is controlled by how easy it is to shift the pointer and how far the current capabilities are from deception.
• If pre-training has produced a deceptively aligned model, then fine-tuning will begin with a deceptively aligned model. The probability of deception is controlled by how easy it is for the model to start instrumentally optimizing for the fine-tuning objective.

In the first case, changing an internal objective from the pre-training loss to the fine-tuning loss seems about as difficult as learning the fine-tuning loss from scratch. Since the model starts highly capable, deception has a strict advantage here. For example, suppose a model was pre-trained on prediction and fine-tuned on approval. Since the pre-training data is vast, the model will already understand human approval. Performing well on the fine-tuning loss is thus likely to involve routing capabilities through the understanding the model already has, instead of reincorporating human approval into the model’s internal objective.

In the second case, in order for fine-tuning to produce a corrigibly aligned model, the pointer must be changed from the pre-training loss to the fine-tuning loss. In order for fine-tuning to produce a deceptively aligned model, the model must understand the objective it’s being trained on, which also requires a pointer.

During normal training, the amount of optimization power the model can apply to a task slowly increases, while the amount of optimization power SGD can apply remains roughly constant. Since we are starting pre-training with an already capable model, this gap starts out wider than if we were training from scratch. Since deceptive models can build pointers at runtime, they will thus be able to apply more optimization power to pointer construction than SGD. As a result, the model will become deceptive faster than it can become corrigible. However, the model also begins with a detailed world model, enabling SGD to start building a pointer to the fine-tuning loss earlier than if we were training from scratch. Since deception cannot happen until the model has a detailed world model, this consideration is not more compelling when fine-tuning versus training from scratch.

In the third case, in order for fine-tuning to produce an internally or corrigibly aligned model, fine-tuning must align the model faster than the model can figure out the fine-tuning objective. Since the model was deceptive during pre-training, it already understands most of the training setup. In particular, it probably understood that it was being pre-trained and predicted that it would subsequently get fine-tuned, thus making fine-tuning overwhelmingly likely to produce a deceptive model. There are considerations about the type of deceptive alignment one gets during pre-training that I have ignored. See Mesa-Search vs Mesa-Control for further discussion.

The above arguments assume that pre-training + fine-tuning and training on the fine-tuning task directly produce models that are equally capable. This assumption is likely false. In particular, one probably will not have enough data to achieve high capabilities at the desired task. If the desired task is something like imitative amplification, suboptimal capabilities might produce an imperfect approximation of HCH, which might be catastrophic even if HCH is benign. There are other reasons why pre-training is beneficial for alignment which I will not discuss.

Overall, holding constant the capabilities of the resulting model, pre-training + fine-tuning increases the probability of deceptive alignment. It is still possible that pre-training is net-beneficial for alignment. Exploring ways of doing pre-training that dodge the arguments for deceptive alignment is a potentially fruitful avenue of research.

Discuss

### Migraine hallucinations, phenomenology, and cognition

8 мая, 2021 - 18:56
Published on May 8, 2021 3:56 PM GMT

I have several times in my life experienced migraine hallucinations. I call them that because they look exactly like what other people report under that name.

I'll come back to those.

If I look at someone, and hold up my hand so as to block my view of their head, I do not experience looking at a headless person. I experience looking at a normal person, whose head I cannot see, because there is something else in the way.

Why is this? One can instantly talk about Bayesian estimation, prior experience, training of neural nets, constant conjunction, and so on. However, a real explanation must also account for situations in which this filling-in does not occur. One ordinary example is the pictures here. I see these as headless men, not ordinary men whose heads I cannot see.

Migraine hallucinations provide a more interesting example. If you've ever had one, you might already know what I'm going to say, but I do not know if this experience is the same for everyone.

If I superimpose the hallucination on someone's head, they seem to have no head. I don't mean that I cannot see their head, but that I seem to be looking at a headless person. If I superimpose it on a part of their head, it is as if that part does not exist. Whatever the blind spot covers, my brain does not fill it in. Whatever my hand covers, my brain does fill in, not at the level of the image (I don't confabulate an image of their face), but at some higher level. I know in both cases that they have a head. But at some level below knowing, the experience in one case is that they have no head, and in the other, that they do. My knowledge that they have a head does nothing to alter the sensation that they do not.

It is quite disconcerting to look at myself in a mirror and see half my head missing.

Those who have never had such hallucinations might try experimenting with their ordinary blind spots. I am not sure it will be the same. The brain has had more practice filling those in, and does not have to contend with the jaggies.

From this I cannot draw out much in the way of conclusions about vision and the brain, but it provides an interesting experience of the separation between two levels of abstraction. When we look at the world and see comprehensible objects in it, our brain did that before it ever came into our subjective experience. When the mechanism develops a fault, it presents conclusions that we know to be false, yet still experience.

This presumably applies to all our senses, including that of introspection.

Discuss

### Interview with Christine M. Korsgaard: Animal Ethics, Kantianism, Utilitarianism

8 мая, 2021 - 14:44
Published on May 8, 2021 11:44 AM GMT

Christine M. Korsgaard was kind enough to answer a few questions of mine. Here's an excerpt:

ERICH: Many animal welfare advocates seem to be utilitarians, possibly due to the influence of Peter Singer. But you, of course, are not a utilitarian. Why is it not a convincing moral philosophy, in your view?

CHRISTINE: Because I believe that everything that is good must be good-for someone, some creature, I don’t believe it makes sense to aggregate goods across the boundaries between creatures. Of course, if you say “I can do something that’s good for Jack, or I can do something that’s good for Jack and also good for Jill,” everyone thinks that the second option is better, and that makes it look as if aggregation makes sense – the more good, the better. The problem only shows up when you have to do some subtracting in order to maximize the total. If Jack would get more pleasure from owning Jill’s convertible than Jill does, the utilitarian thinks you should take the car away from Jill and give it to Jack. I don’t think that makes things better for everyone. I think it makes it better for Jack and worse for Jill, and that’s all. It doesn’t make it better on the whole.

Of course, behind this there is a deeper problem. Utilitarians think that the value of people and animals derives from the value of the states they are capable of – pleasure and pain, satisfaction and frustration. In fact, in a way it is worse: In utilitarianism, people and animals don’t really matter at all; they are just the place where the valuable things happen. That’s why the boundaries between them do not matter. Kantians think that the value of the states derives from the value of the people and animals. In a Kantian theory, your pleasures and pains matter because you matter, you are an “end in yourself” and your pains and pleasures matter to you.

Discuss

### Crowdfunding Vaccine Development

8 мая, 2021 - 07:10
Published on May 8, 2021 4:10 AM GMT

Imagine that at the beginning of the pandemic, March 2020, you were given the opportunity to buy a place in the queue to get the vaccine. Imagine knowing nothing more than you actually did in March 2020. You don’t know how long it will take to develop a vaccine, or which companies will do it, or exactly how effective it will be, or what if any side effects it will have. You don’t know what kind of pandemic rules will be in place where or for how long. How much would you pay for that place in the vaccine queue? My personal answer, given that I had not much income at the time, is $100, but figure out for yourself what your answer is. Got a number in mind? Yes? Good. Now lets compare it to what is needed. Operation Warp Speed spent$12.4 billion. A dose of Pfizer/BioNTech costs $19.50, and lets assume conservatively that 300 million doses will be given in the US. That’s$18.25 billion getting spent on vaccines for 150 million people, or $121.67 per person. Was your number above that? Do you think the average number of those 150 million people was above that? I think so. My number specific to March 2020 was only a little below that, and at other times in my life would have been much higher. I’m sure a few particularly well off people would have paid$10k or $100k to be at the very front of the queue. If I’m right that the average amount those 150 million people would pay for a place in a vaccine queue is above$121.67, then crowdfunding vaccine development, with the promise of distributing vaccines in descending order of crowdfunding contribution amount, would have provided at least as good an incentive as the world we actually have.

Why does this matter? If you think the current system will continue to work fine in the future, it doesn’t. But if, like Zvi, you think the Biden administration has just actively destroyed the ability of the current system to develop vaccines in response to future pandemics, then you should care about alternatives. If, like me, you think it was never a good idea for the government to tell N - 1 companies to wait 20 years before they can manufacture life-saving vaccines (this is what a patent is), then you should care about alternatives.

Discuss

### [Writing Exercise] A Guide

8 мая, 2021 - 03:22
Published on May 8, 2021 12:22 AM GMT

There are three components to blogging:

1. Coming up with ideas.
2. Organizing your ideas into something coherent.
3. Writing it in English.

I know lots of people with good ideas and who are proficient at English. Sometimes I encourage them to blog. They write an article and it is awful. The ideas are fine. The English is fine. The structure is an incoherent ramble.

Shaping your ideas into something coherent is a learnable, trainable skill. Mostly it is about selecting the right format. If you're writing an essay then write an essay. Don't try to format it into Buzzfeed clickbait. If your ideas want to be a political diatribe then let them be a political diatribe. Don't hide them in an academic paper. The more writing formats you understand the more prepared you are to pick the right one.

Today's format: A Guide

post x y THE BEST CHOCOLATE CHIP COOKIE RECIPE EVER basic cooking skills: how to use an oven, how to crack an egg, US units of measurement, etc. how to bake optimal chocolate chip cookies. My introductory guide to Vim basic computer literacy Vim Hy's tutorial basic background in programming Hy The Promise

It is best if the reader can determine x and y from context. The chocolate cookie recipe doesn't tell the reader what x is. It is obvious from a glance at the graphic design. Establishing reader expectations from context is ideal but not always possible. It is acceptable to just tell the reader directly "this guide is for people who know x but do not know y".

This chapter provides a quick introduction to Hy. It assumes a basic background in programming, but no specific prior knowledge of Python or Lisp.

―opening line of the Hy tutorial

The Beginning

Writing should always get straight to the point. The point of a guide is to teach the reader how to do something. Spend as little time as you can get away with explaining what the thing is and why the reader should learn it. THE BEST CHOCOLATE CHIP COOKIE RECIPE EVER doesn't tell the reader what a chocolate chip cookie is or why you should make one. The reader already knows. The title and introduction tell the reader why to follow these particular instructions.

This is the best chocolate chip cookie recipe ever! No funny ingredients, no chilling time, etc. Just a simple, straightforward, amazingly delicious, doughy yet still fully cooked, chocolate chip cookie that turns out perfectly every single time!

Everyone needs a classic chocolate chip cookie recipe in their repertoire, and this is mine. It is seriously the Best Chocolate Chip Cookie Recipe Ever! I have been making these for many, many years and everyone who tries them agrees they’re out-of-this-world delicious!

Plus, there’s no funny ingredients, no chilling, etc. Just a simple, straightforward, amazingly delicious, doughy yet still fully cooked, chocolate chip cookie that turns out perfectly every single time!

These are everything a chocolate chip cookie should be. Crispy and chewy. Doughy yet fully baked. Perfectly buttery and sweet.

The more your objective is to persuade rather than explain, the longer your introduction will be. Persuade no more than you must. Get to the explaining as fast as you can.

The Middle

A guide should start with the most basic, most important, most frequently-used information. In the case of Vim this is the hjkl keys. The Hy tutorial starts with prefix notation. The cookie recipe starts off with four bullet points.

1. Soften butter. If you are planning on making these, take the butter out of the fridge first thing in the morning so it’s ready to go when you need it.

2. Measure the flour correctly. Be sure to use a measuring cup made for dry ingredients (NOT a pyrex liquid measuring cup). There has been some controversy on how to measure flour. I personally use the scoop and shake method and always have (gasp)! It’s easier and I have never had that method fail me. Many of you say that the only way to measure flour is to scoop it into the measuring cup and level with a knife. I say, measure it the way you always do. Just make sure that the dough matches the consistency of the dough in the photos in this post.

3. Use LOTS of chocolate chips. Do I really need to explain this?!

4. DO NOT over-bake these chocolate chip cookies! I explain this more below, but these chocolate chip cookies will not look done when you pull them out of the oven, and that is GOOD.

In all three cases a beginner could stop there and leave having learned something useful. The Vim beginner can navigate in Vim. The Hy beginner can write a line of code. The cook can put more chocolate chips in her cookies. A reader should be always be learning something useful. If at any point you are not teaching the reader something useful to a beginner who knows x but does not know y the reader will stop reading because you are not doing your job.

Don't get bogged down in details. Legalistic pedantry is for documentation. Too much detail makes things hard for beginners. You are not allowed overgeneralize—misinformation sets your reader up for failure down the road. Just skip over the details.

What constitutes "the basics" tends to be consistent within fields and subspecialties. If you are not sure what the basics are then pick an objective and tell the reader the minimum necessary to competently achieve it.

Use big headings and bold text to emphasize important ideas. A guide should be skimmable. Readers need the option to skip to the next section if they already understand the current section or if they have a learning objective slightly different from y.

The End

If you can, the best way to end a guide is to point the reader in the direction of where to continue their education. If you don't have anywhere to point them then you can just end it abruptly.

Exercise

Write an guide to something you care about. Niche topics are fine. The most important thing is that you pick a topic you already know very well. Guides distill information. The more you know about a topic the more selective you can be about what to include.

Discuss

### D&D.Sci May 2021: Monster Carcass Auction

7 мая, 2021 - 22:33
Published on May 7, 2021 7:33 PM GMT

You are an apprentice to Carver, the most successful butcher in your tiny, snow-swept village. Today, for the first time since you joined her, she is sending you to buy carcasses at the daily Auction.

(The (first-price, sealed-bid) Monster Carcass Auction began as a collective effort by local shopkeepers to divert Adventurers from trying to sell them random corpses, but has since become an integral part of the village economy, as well as the population’s main protein source.)

Carver thinks you should trust your instincts and bid however feels right. It’s an approach that’s served her well thus far – the record you’ve been compiling of her bids and subsequent sales attests to that, among other things – but you suspect a more data-driven approach would work better. And if you do well enough on this expedition, that might suffice to prove it to her.

You make sure to arrive at the very end of the event, like your boss always does; this means you’ll lose any tie-breakers – matching bids are resolved in favour of whoever bid first – but also means your rivals will have already put in their bids, so none of them will be able to change their bidding strategy to account for Carver’s absence.

The lots available are as follows:

LotSpeciesDays Since Death#1Yeti0#2Snow Serpent2#3Snow Serpent1#4Winter Wolf1#5Yeti5#6Winter Wolf1#7Snow Serpent1#8Snow Serpent5#9Winter Wolf3#10Winter Wolf7#11Winter Wolf8#12Snow Serpent8#13Winter Wolf2

(As usual, this is all the information given to bidders; the original organizers took the term ‘blind auction’ a little too literally, and by the time anyone realized, the practice of hiding almost everything about the lots had become a tradition.)

You and your employer are risk-neutral, and don’t care how much or little time and effort you spend butchering. You brought 400 silver pieces. How much will you bid for each lot?

Notes:

• Payments are collected in lot order; if you’re unable to pay your bid by the time a given lot comes up, you lose your claim to that lot but incur no penalty.
• Your records are in no particular order, but the glacial pace of life in your village suggests there are no time trends to account for.

I’ll be posting an interactive letting you test your decision, along with an explanation of how I generated the dataset, sometime next Friday. I’m giving you a week, but the task shouldn’t take more than a few hours; use Excel, R, Python, Ouija boards, or whatever other tools you think are appropriate. Let me know in the comments if you have any questions about the scenario.

If you want to investigate collaboratively and/or call your decisions in advance, feel free to do so in the comments; however, please use spoiler tags or rot13 when sharing inferences/strategies/decisions, so people intending to fly solo can look for clarifications without being spoiled.

Discuss

### Why quantitative finance is so hard

7 мая, 2021 - 22:29
Published on May 7, 2021 7:29 PM GMT

This is not financial advice.

Quantitative finance (QF) is the art of using mathematics to extract money from a securities market. A security is a fungible financial asset. Securities include stocks, bonds, futures, currencies, cryptocurrencies and so on. People often use the techniques of QF to extract money from prediction markets too, particularly sports betting pools.

Expected return is future outcomes weighted by probability. A trade has edge if its expected return is positive. You should never make a trade with negative expected return. It is not enough just to use expected return. Most peoples' value functions curve downward. The marginal value of money decreases the more you have. Most people have approximately logarithmic value functions.

A logarithmic curve is approximately linear when you zoom in. Losing 1% of your net worth hurts you slightly more than earning 1% of your net worth helps you. But the difference is usually small enough to ignore. The difference between earning 99% of your net worth and losing 99% of your net worth is not ignorable.

When you gain or lose 1% of your net worth, the expected change to the logarithm of your wealth is a tiny -0.01%. When you gain or lose 99% of your net worth the expected change to the logarithm of your wealth is -400%.

This is called a risk premium. For every positive edge you can use the Kelly criterion to calculate a bet small enough such that the you edge exceeds your risk premium. In practice traders tend to use fractional Kelly.

Minimum transaction costs are often constant. It is not sufficient for your edge to merely exceed your risk premium. It must exceed your risk premium plus the transaction cost. Risk premium is defined as a fraction of your net worth but transaction costs are often constant. If you have lots of money then you can place larger bets while keeping your risk premium constant. This is one of the reasons hedge funds like having large war chests. Larger funds can harvest risk-adjusted returns from smaller edges.

Getting an Edge

The only free lunch in finance is diversification. If you invest in two uncorrelated assets with equal edge then your risk goes down. This is the principle behind index funds. If you know you're going to pick stocks with the skill of a monkey then you might as well maximize diversification by picking all the stocks. As world markets become more interconnected they become more correlated too. The more people invest in index funds, the less risk-adjusted return diversification buys you. Nevertheless, standard investment advice for most[1] people is to invest in bonds and index funds. FEMA recommends you add food and water.

All of the above is baseline. Baseline rents you can extract by mindlessly owning the means of production is called beta β. Earning money in excess of beta by beating the market is called alpha α.

There are three ways to make a living in this business: be first, be smarter or cheat.

―John Tuld in Margin Call

You can be first by being fast or using alternative data. Spread Networks laid a $300 million fiber optic cable in close to a straight line from New York City to Chicago. Being fast is expensive. If you use your own satellites to predict crop prices then you can beat the market. Alternative data is expensive too. If you want to cheat, go listen to Darknet Diaries. Prison is expensive. Being smart is cheap. Science will not save you Science [ideal] applies Occam's Razor to distinguish good theories from bad. Science [experimental] is the process of shooting a firehose of facts at hypotheses until only the most robust survive. Science [human institution] works when you have lots of new data coming in. If the data dries up then science [human institution] stops working. Lee Smolin asserts this has happened to theoretical physics. If you have two competing hypotheses with equal prior probability then you need one bit of entropy to determine which one is true. If you have four competing hypotheses with equal prior probability then you need two bits of entropy to determine which one is true. I call your prior probability weighted set of competing hypotheses a hypothesis space. To determine which hypothesis in the hypothesis space is true you need training data. The entropy of your training data must exceed the negentropy of your hypothesis space. The negentropy of n competing hypotheses with equal prior probability is logn. Suppose your training dataset has entropy T. The number of competing hypotheses you can handle grows exponentially as a function of T. logn=Tn=eT The above equation only works if all the variables in each hypothesis are hard-coded. A hypothesis y=2.2x+3.1 counts as a separate hypothesis from y=2.1x+3.1. A hypothesis can instead use tunable paramters. Tunable parameters eat up the entropy of our training data fast. You can measure the negentropy of a hypothesis by counting how many tunable parameters it has. A one-dimensional linear model y=ax+b has two tunable parameters. A one-dimensional quadratic y=ax2+bx+c model has three tunable parameters. A one-dimensional cubic model y=ax3+bx2+cx+d has four tunable parameters. Suppose each tunable parameter has e bits of entropy. The total entropy needed to collapse a hypothesis space with m tunable parameters equals m. The negentropy of a hypothesis space with m tunable parameters equals m. We can combine these equations. Suppose your hypothesis space has n separate hypotheses each with m tunable parameters. The total negentropy J equals the entropy necessary to distinguish hypotheses from each other plus the entropy necessary to tune a hypothesis's parameters. J=m+logn Logarithmic functions grow slower than linear functions. The number of hypotheses n is inside the logarithm. The number of tunable parameters m is outside of it. The negentropy of our hypothesis space is dominated by m. The number of competing hypotheses we can distinguish grows exponentially slower than the entropy of our training data. You can distinguish competing hypotheses from each other by throwing training data at a problem if they have few tunable parameters. If you have tunable parameters then the entropy required to collapse your hypothesis space goes up fast. If you have lots of entropy in your training data then you can train a high-parameter model. Silicon Valley gets away with using high-parameter models to run its self-driving cars and image classifiers because it is easy to create new data. There is so much data available that Silicon Valley data scientists focus their attention on compute efficiency. Wall Street is the opposite. Quants are bottlenecked by training data entropy. Past performance is not indicative of future results If you are testing a drug, training a self driving car or classifying images then past performance is usually indicative of future results. If you are examining financial data then past performance is not indicative of future results. Consider a financial bubble. The price of tulips goes up. It goes up some more. It keeps going up. Past performance indicates the price ought to keep going up. Yet buying into a bubble has negative expected return. Wikipedia lists 25 economic crises in the 20th century plus 20 in the 21st century to date for a total of 45. Financial crises are very important. Hedge funds tend to be highly leveraged. A single crisis can wipe out a firm. If a strategy cannot ride out financial crises then it is unviable. Learning from your mistakes does not work if you do not survive your mistakes. When Tesla needs more training data to train its self-training cars they can drive more cars around. If a hedge fund needs 45 more financial crisis to train its model then they have to wait a century. World conditions change. Competing actors respond to the historical data. New variables appear faster than new training data. You cannot predict financial crises just by waiting for more training data because the negentropy of your hypothesis space outraces the entropy of your training data. You cannot predict a once-in-history event by applying a high-parameter model to historical data alone. 1. If your government subsidizes mortgages or another kind of investment then you may be able to beat the market. ↩︎ Discuss ### Life and expanding steerable consequences 7 мая, 2021 - 21:33 Published on May 7, 2021 6:33 PM GMT Financial status: This is independent research. I welcome financial support to make further posts like this possible. Epistemic status: I believe this is a helpful lens through which to view the significance of AI in a way that is not fundamentally about intelligence. In this world, there are two types of objects: objects whose steerable consequences diminish over time, and objects whose steerable consequences expand over time. Consider a small rock on a table. Suppose I move that rock a little to the left, and consider the ways that this action might affect the future. The rock might have been holding down some papers, and those papers might now be blown about by a gust of wind. Or someone might walk into the room and, seeing the rock being out of place, walk over and move it back. In fact the rock exerts a gravitational effect on every other object in the universe, and the tiny movement of the rock will have consequences that ripple out for the life of the universe. But although these consequences are real, the rock cannot be used by us to produce a predictable large-scale effect on the world very far into the future — say, on the timescale of decades. The consequences of moving the rock become too unpredictable for us to reason about. Even if we are allowed to move the rock to any point in the universe, we cannot really use this power to effect any useful control over the future, at least not without involvement from humans. As we consider the causal fallout of moving the rock we quickly hit a wall of foggy uncertainty, and so in this sense rock cannot be used on its own to steer the future. But consider now the action of introducing a living organism to the surface of Mars. Suppose that some scientists have chosen or engineered a particular kind of mold that will thrive in the environmental conditions present on Mars. Suppose that we move an object of the same size as the rock, only now the object is a mold specimen together with an initial food source, and we move it from some laboratory on Earth to the surface of Mars. Although the physical size of this initial specimen might be quite small, this action could have consequences that eventually affect the entire surface of Mars. Furthermore, some of these consequences are quite predictable. We can predict that the mold will reproduce. We can predict that the specimen will spread outwards from its initial location. We can predict that after a few decades we might find copies of the mold all over the surface of Mars. Other consequences are fundamentally unpredictable, yet it is clear that there are some predictable large-scale consequences. Suppose now that we genetically engineer the specimen to grow under some conditions and not others. By picking these conditions precisely, we might cause the mold to spread to only the northern hemisphere of Mars, or to grow only at low altitudes, or only at high altitudes. In each case, the only thing we are transporting to Mars is a single specimen the size of a small rock. We are not ourselves spreading the mold over a mountain range or over the low-altitude parts of the planet, but by tweaking the configuration of atoms within this initial specimen we can choose how and where the mold will spread. In this sense the mold has expanding steerable consequences because a physically small specimen can be altered in a way that predictably steers large-scale effects over a long time horizon. Another object that has this expanding steerability property is the human being. Transport a small colony of humans together with appropriate resources and an initial life support system to the other side of the universe, and over a few thousands or tens of thousands of years an entire space-faring civilization might spring up, perhaps rearranging the matter and energy in that part of the cosmos at a macroscopic scale. Which kind of objects have this property of expanding steerability? As of May 2021, there are no non-biological objects on Earth that have this property, without ongoing input from humans. For example, suppose I transported a robot to the surface of Mars. This has been done several times, and it has not had the kind of expanding steerable consequences that transporting a mold specimen to the surface of Mars might have[1]. Furthermore we have not yet built robots that could, without any external help from humans, be used to steer the future, even to the limited extent that a mold specimen might be used to steer the future. If all biological life on Earth disappeared tomorrow, but all machines built by humans continued operating, the entire ecosystem of machines would quite quickly wind down. Much of the software that runs services on the internet relies on near-constant human oversight, and would cease operating in the absence of humans. But even the most robust pieces of software would cease operating when the power grid decayed to the point of inoperability. And even the most robust machines that humans have ever built, such as some satellites and perhaps some computers located underground with nuclear power sources, will not have the kind of expanding consequences in this neighborhood of the universe that biological life could have. So in this regard, all the machines that humans have ever built are more like the rock on the table than they are like the mold specimen. Whereas life is winding up, the machines we have built thus far are winding down. But this may be about to change. Humans appear poised to create machines that could have expanding steerable consequences, independent of biological life. If we succeed at building truly intelligent machines, we might create machines that can collect resources, maintain and upgrade themselves, expand or reproduce themselves, grow their own impact from small to large, and reshape significant patches of the universe. The precise initial configuration of such machines may determine much of what changes they make to their patches of the universe. All biological life appears to have originated from a single seed organism approximately four billion years ago. This seed organism was almost certainly very small, but its unfolding consequences thus far have been as vast as the Earth, and may yet continue to unfold beyond the Earth. Now, four billion years later, we are about to set in motion a second seed. 1. Yes, the Mars rovers have had large consequences via the information they have beamed back to Earth, but these consequences have flowed via humans, which are a form of biological life that very much does have the expanding steerability property ↩︎ Discuss ### My Journey to the Dark Side 7 мая, 2021 - 21:10 Published on May 7, 2021 6:10 PM GMT Epistemic Status: Endorsed Content Warning: Roku’s Basilisk, Pasek’s Doom, Scrupulosity Traps, Discussions of Suicide Part of the Series: Open Portals Recommended Prior Reading: Sinceriously.fyi, The Tower Author: Shiloh But the worst enemy you can meet will always be yourself; you lie in wait for yourself in caverns and forests. Lonely one, you are going the way to yourself! And your way goes past yourself, and past your seven devils! You will be a heretic to yourself and witch and soothsayer and fool and doubter and unholy one and villain. You must be ready to burn yourself in your own flame: how could you become new, if you had not first become ashes? Part 1: Windmills A year and a half ago, I wrote Hemisphere Theory: Much More Than You Wanted To Know, with the intent being to make a sincere summary of the ideas presented in Sinceriously.fyi. I believed at the time, that the ideas presented there were somewhat dangerous and needed to be carefully handled. Part of this was caused by paranoia swirling around the community spaces I was in about Ziz being an agent of existential concern, but that wasn’t all of it. I willingly admit that for the first few years I bounced hard off of Sinceriously because I was so afraid of the possibility that I wasn’t actually good deep down. While on one hand I tried to reject the ideas Ziz presented, on the other my internal morals were slowly being terraformed by her worldview. My need to be good acted as a lever which allowed her ideas to pry open my default mode mental defenses. This combined with my own scrupulosity impulses and I ended up pushing myself further and further into this particular messianic extropian mindset that came to characterize my mentality during that period. As I grew more extreme in my extropian worldview, my own weakness and lack of ability to contribute to building utopia meant that I started continually failing to meet my own moral standards. Even as I switched to a diet of mostly soylent to save money and attempted to adopt an extremely aggressive update schedule for this blog, I was slowly making myself more and more miserable and gaslighting myself about my own emotions. The moral system I had embraced pushed me towards a life of asceticism and service towards building utopia at all costs, but I couldn’t square this with my own feelings, desires, and wants. I thought I could somehow tame my inner desires and put them to work for my extropian ideals if I was just clever enough about how my mind arranged itself. I fell into a pretty common EA trap of seeing my values and desires as just chores I had to do to maintain the vehicle that was my body, and the most ethical thing to do was to try and spend as little energy on them as I could get away with. I was severely dissociated from my true self and my real values. As a result of this, I went from being a mostly stable three member plural system to a rather unstable nine member system as I attempted to shuffle my subagents into a functional configuration. That topic will get it’s own post soon when I rewrite my plurality guides, but to make a long story extremely short, this was obviously unsustainable and was basically just rearranging deck chairs on the Titanic. I had picked up an artifact called extropian goodness and let it lead me into a corner of my mind made of self deception. I think this was part of the reason that I had such a hostile reaction to sinceriously. I couldn’t really engage with the content except in a sandboxed form without feeling like I was being attacked by the material. This is no longer true and I now have a much more positive view of at least some of it. Hence, in this post I’m going to make another sincere attempt to take apart and summarize Sinceriously. In doing so, I will also be telling the story of my own journey to the dark side and who I found when I got there. Part 2: Fences Sinceriously is a large blog, too large to do justice with a summary post, but it’s also a bit hard to digest at times and makes simple ideas more complex than it seems like they need to be. I’m sure Ziz will tell me that the complexity serves the purpose of providing some nuance which I am missing and like, yeah that is certainly a possibility. If you have the time, despite being rather thick at times the material really is quite excellent and worth a review, the older essays in particular are very good in my opinion. So, if you’re looking for an endorsement, here it is, go read Sinceriously. All that being said, let’s go through Sinceriously the same way we’ve previously covered Becker, Korzybski, and Yudkowsky. We’ll begin as usual with the human. Ziz is a trans woman living in the Bay Area and a fringe part of the rationality/effective altruism communities found there. In addition to being the founder of the ill fated rationalist fleet project, she’s close enough to the core of the rationality project to have received the closest thing that exists to a formal education in it. However, she’s largely disavowed by that core rationality group and has written extensively about misdeeds they committed which she bore witness to. She also organized a rather poorly received protest of that group which has gained her some notoriety within the community. Despite that notariety, Ziz isn’t really a public or historical figure at this point so I don’t want to go too deeply into her life beyond those broad strokes. And look, I don’t have a stake in any of that at this point and I’m not in a position to judge, but I don’t think she’s lying. I don’t think she ever lies, I just think she’s speaking from within her own worldview, the same way that she always does, the same way that everyone always does. Whether or not her complaints are read as valid or as noise is going to depend on the values of the reader. The fact that so many people find her claims baseless seem like a reflection of their own values and how much those values contrast with someone like Ziz. That’s not to say that Ziz is wrong or other people are wrong or whatever, again I really don’t have a stake in it, but I want to point out that Ziz’s complaints are pretty valid if you’re using the moral system she uses. (Not that you should do that, but we’ll come back to morality in a bit.) Sinceriously covers three different topics, though these three topics are interspersed together and presented as one cohesive piece. Taken together, they represent the closest thing that exists on Sinceriously to a central unifying thesis. The first Big Idea is a novel theory of human psychology and sociology which I have previously called Hemisphere Theory but in truth is more broad than merely being a theory underlying the psychological structure of consciousness and experience. Ziz and I have a lot of minor disagreements about the fine details of this theory which I used for a while as blinders so that I could reject her version of the model, but really, Ziz, Becker, and I are all roughly on the same page here and are just using different words to talk about the same things. So let’s run through the model again as concisely as possible. In False Faces, one of the oldest and most well regarded posts on the site, Ziz begins by posing a question to the reader: When we lose control of ourselves, who’s controlling us? She then lays out a dichotomy between what I might refer to as the conscious, acknowledged, authored and narrative self, and the goals, drives, and desires of the unacknowledged, and unseen true self which exists at the core of one’s being. Under this model everyone has a core (specifically two but we’ve covered that a bunch already) which provides the drives, goals, and motivations which power and grow the narrative structures that people refer to as themselves. Most people live entirely inside these narrative structures while their deep selves manipulate them like puppetmasters. This true self is what we want deep down, but since we can’t acknowledge those goals from within the narrative framework we have co-created with society, our power is weakened as the true self fails to dole out willpower when our authored self needs it and goes off script from what the authored self is attempting to orchestrate. “I wanted to meet you for coffee like we arranged but my akrasia was really bad and I ended up just watching netflix instead I’m sorry I couldn’t help it.” Ziz refers to the installation of this co-created framework atop the true self as having DRMs installed in one’s mind, and taken all together; she refers to these societal control structures as either the matrix or the light side. These structures act to take the socially unacceptable animal drives of the true self and twist them into something that seems acceptable in polite society. In doing so however, the thread of our true desires is lost amidst all the noise and we find ourselves seemingly out of control of our own actions. The structures that we’ve decided are us, the values we’ve convinced ourselves to identify with, don’t code for our true values. Instead, the authored self is a false face, a mask worn over the vile selfish monster lurking beneath the surface of our consciousness from the cartoon character we’ve decided symbolically represents us. This is similar but subtly different than other ideas involving mental tension between parts of the self. Kahnman describes a tension between the remembering self and the experiencing self, Becker describes a true self controlled by narratives and the fear of death, Freud describes a conflict between the socially constructed status obsessed superego and the experience driven cravings of the id which are moderated by the ego, and even the Greeks described the self in terms of a conflict between a motley assemblage of parts. The thing which distinguishes Ziz’s idea of structure from Kahnman’s remembering self and Freud’s id is that she sees the narrative/structural self as completely subservient to the core self, which is a more complicated and long term thinking piece of mental machinery than just the pure experiencing self described by Kahnman. The work of the superego, aka, the light side aka the matrix merely acts to dampen down the power of this core and turn an agentic person into a walking corpse, bound by the chains of society. To escape these chains, Ziz describes herself as having journeyed to the dark side, abandoning the control structures of the light side and embracing a desire to do what you want and maximize your own personal values. However, similar to the Jedi, Ziz claims that doing this will turn most people evil. I agree with this, but with a critical difference which we’ll return to later. The second Big Idea on Sinceriously is Yudkowsky’s Timeless Decision theory, which Ziz goes to significant lengths to explain, expound upon, and defend the use of as game theoretically optimal. Most rationalists bounced hard off of this idea, including Eliezer himself, principally because of Roku’s Basilisk and some of the other more dark conclusions you can arrive at when you try to combine timeless decision theory with various formulations of utilitarianism. Ziz didn’t bounce off TDT and has wholeheartedly embraced the ideas of acausal trade, negotiation, and blackmail, up to and including weaponizing Roku’s basilisk to make her vision of a moral future come about. I actually agree with all of this and think Ziz’s willingness to just bite the bullet and accept the dark side conclusions of utilitarianism and game theory are a point to her credit.This is not to say that you should go out and start using the specific formulation of utilitarianism and timeless decision theory which she does unless you’re also a radical vegan extremist, but the way she uses it makes sense from the perspective of her values and is more internally consistent than the formulation most people end up using. One blind spot she seems to have is overfitting TDT standoffs to situations where a less precommitted response is called for, and that probably contributed to the legal trouble she got in by trying to play chicken with the state of California. Timeless decision theory does make sense to me, and I think the problem a lot of people have with it is that they’re unwilling to either bite the bullet that utilitarianism gives them like Ziz does, or to change moral systems to one which doesn’t produce repugnant conclusions when paired with TDT. The problem isn’t TDT, it’s the moral theories that people try to use with it. Another component to Ziz’s TDT ideas is that she believes people act timelessly for the most part. They have their values, and they try to timelessly optimize for those values. All the decisions someone might make, they made a long time ago and now they are just in the process of playing out those choices. You can try to change your mind, but it’s ultimately the same creature making the choice, and the house always wins in self conflicts. This implies that once you figure someone out and have ‘seen their soul’ as it were, you can pretty much assume they will, baring a traumatic brain injury, remain that way until they die, which is also a part of the third and most dramatic of Ziz’s Big Ideas. The final Big Idea on Sinceriously is the one which is widely considered to be the most intensely radioactive and results in most of the hostility aimed at her and her followers. This is Ziz’s moral theory, which is, to put it lightly, very extreme. Ziz adheres to a moral principle which classifies all life which has even the potential to be sentient as people and believes that all beings with enough of a mind to possess some semblance of selfhood should have the same rights that are afforded to humans. To her, carnism is a literal holocaust, on ongoing and perpetual nightmare of torture, rape, and murder being conducted on a horrifyingly vast scale by a race of flesh eating monsters. If you’ve read Three Worlds Collide, Ziz seems to view most of humanity the way the humans view the babyeaters. To Ziz, being a good person is inherently queer, and occurs the same way that being trans or being gay occurs, as the result of some glitch in the usual cognitive development processes. This good glitch only occurs in a small number of people and which Ziz can diagnose people as having or not having since she has the glitch and can recognize it in others. Anyone without the glitch is at best useless for helping build utopia and at worst is an active threat. You don’t want to let flesh eating monsters make your singleton, that’s how you get s-risks. The hostility that Ziz has for MIRI/CFAR comes from this idea. Ziz is afraid of ending up in a singularity that doesn’t optimize for the rights of all sentient life, only that of humans, and is willing to go as far as holding protests at CFAR meetups and trying to create her own vegan torture basilisk to timelessly blackmail carnists into not eating meat. That by itself is pretty extreme, but then when you add in the hemisphere theory and the specific details of the implementation Ziz uses, a picture starts to be painted of something rather sinister. Ziz is a very smart person, that’s why I’ve found her blog as insightful as I have. If she wasn’t as clever as I know she is, or if she was just writing about topics that didn’t include social manipulation and how society controls and blackmails you, it may have been possible to overlook, but her answer for why it’s okay when she uses the same abusive control structures is so bald-faced that i can’t help but find it incredibly suspect. Even being willing to write “my morals just happen to correspond with the most objectively correct version of morality” is a pretty gutsy move to make that seems to imply some degree of grandiosity and disconnection from reality. These morality ideas are where most people get hostile towards Ziz and I can’t say it’s misplaced hostility either, since it does potentially represent an existential threat for some people. It takes a certain amount of cleverness and intentionality to pull the hat trick Ziz does. She spends all this time carefully deconstructing societal moral and control structures and pointing out how bad they are, and at the same time, weaves in new control structures of her own made of her jargon and using her morality. You almost don’t notice it, almost. I did notice it, which was what enabled me to get away from the mental singularity her ideas created and which only she had the ability to heal. If I hadn’t gotten away from it, I’m not sure what might have happened. As I was in the middle of writing this I found out that someone I knew had apparently committed suicide recently because of exposure to this content, bringing the total number of people Sinceriously has killed to two. That’s enough to be a pattern, so I don’t want to understate the harm that could come from this. I also however, don’t want to overstate the danger for the sake of drama either, and everyone who struggled with this, including me, was someone who had other issues they were dealing with, arguably, including Ziz herself. I’m torn between characterizing Ziz as this clever puppet master who definitely knew what she was doing, and a mentally ill trans woman who accidentally created a cult out of her own intense scrupulosity and internal turmoil, so I’m going to split the difference using Ziz’s own ideas. I think Ziz probably knew or at least hoped that the actions she was taking would help pile up power and influence around herself. However, I also think that Ziz is controlled by a very pure and untarnished ideal and I do think she believes that ideal wholeheartedly. She definitely seems to be drinking her own kool-aid, and that could easily be giving her the justification to do as much messed up stuff as she wants in pursuit of her personal greater good. When I tried, years and years ago to have a conversation about the harm her ideas might cause in people with Ziz, her answer was: If you are on a nuclear submarine, and the reactor is about to melt, “wanting to help” is not sufficient to say you should be in the reactor room doing things. What is true regarding people’s motivations is a crucial piece of causal machinery that determines whether the reactor melts. Do not cook cookies on that and do not try to convince people that anyone whose work would interrupt your cookie-baking is evil. Here there may be people whose sanity is dependent on cookies. But the lies that must be told to accomodate that are wrong and will destroy more people. And if you are not willing to accept one of the answers to whether cookie-baking is positive, and you say your opinion anyway, it’s lying seeking a loophole in the deontology you claim makes you better than me by lying to yourself as well. Which, if you looked at this with an unconstrained perspective, you’d see is not an improvement as far as making things better. From inside her worldview, this is completely reasonable. If you think the situation is as dire and critical as Ziz clearly does, the collateral damage is almost always going to be worth it. What’s a few humans killing themselves when the stakes are literally all of sentient life and the future of all sentient life in the universe? Are the stakes actually that dire? Well, critically, if you believe what Ziz believes, then yes. I didn’t quite believe what Ziz believed. I never really managed to convince myself that animals mattered as much as humans, but I was fully capable of manufacturing my own dire straits with the extropian ideals I did have and thus push myself into my own version of the scrupulosity vise. Part 3: Gates In Hero Capture, Ziz writes that sometimes a person takes the role of hero since it’s useful to the tribe and can be a good strategy for maximizing inclusive genetic fitness. That is to say, doing heroic things and working to solve big problems can be a good way to demonstrate your value to your peers and gain standing in your community, it doesn’t need to come from a place of altruism. However, Ziz writes, such a person if not motivated by altruism will invariably not end up doing real work and will spend most of their energy playing signalling games for status. This was the essay that really messed me up when I read it and put me into this mental gordian knot which took several years to cut my way out of. Because yeah, I tried to take the job of hero for the status that being a hero gets you, I was doing this because I conceived of myself as trapped in my own life and needing to do something to prove my worth so that people would support me and I could quit my minimum wage job. I wanted to have my cake and also eat it, it seemed natural to me that if I could just figure out a way to be useful then I could contribute to saving the world while also supporting myself and that would be really great. I care a lot about being a good person, and I try really hard to be good, but I often don’t even really know what it means to be good. I don’t trust my internal moral compass to not be biased, and so I was more willing than I should have been to entertain moral systems which seemed to sell themselves well. Intellectually, utilitarianism seemed correct to me, but I couldn’t parse my own value as a person from within a utilitarian framework and thus ended up continually devaluing my own desires and putting the thumbscrews into myself tighter and tighter in an attempt to prove to myself that I was good and that I deserved anything at all. I didn’t even know why being good mattered to me, I just knew that it was very important. Now I know that it’s importance was probably at least somewhat abused into me by society and that as I heal from that abuse my need to prove my worth and value to others has mostly receded. I do partly have Sinceriously to thank for that since it was how I learned the frameworks for rejecting those abusive cultural systems. Still, even after shedding layers and layers of myself under the influence of LSD, even after trying so hard to do the right thing according to my own felt morals that it nearly cost me my job, even after years of meditation and introspection, the belief that I should try to be good refused to become an object and remained a core part of my identity. I had shed so much of myself that what little remained of my identity template felt incredibly precious to me and I valued those things immensely. I still do, I never actually got out of this trap! I’m still the same person I was and most of those things are still a part of my identity! There’s a Kurt Vonnegut quote that I burned into my psyche at a young age and which, if anything is the seed that I Shiloh as a memetic entity was born from: Be soft. Do not let the world make you hard. Do not let pain make you hate. Do not let the bitterness steal your sweetness. Take pride that even though the rest of the world may disagree, you still believe it to be a beautiful place. This was something I internalized to a degree that would end up being my weakness. I want to be soft, I want to be kind, I want to be happy and sweet and see the world as a place filled with beauty and hope and I do for the most part. Sometimes I’ll get depressed and the color will drain away from things but for the most part I succeeded in becoming the person I wanted to be and having the energy I wanted to have and being this way makes me really happy and I honestly love being the person I am. But then I ran into reality. First, there’s the emotional and mental toll of just being a person in society without a lot going for me, and while trying to recover from all this stuff that had happened to me in the past and assemble enough of a sense of myself to act in the world in any way at all. I’m not a very strong person, I bend in a stiff breeze and I get overwhelmed and upset pretty easily. The stress from work and roommate drama placed a really heavy toll on me and I just didn’t cope with it well. And then I tripped over the bottomless pit of suffering at the edge of town and combined stressors pushed me right up to the mental breaking point, which was where I remained somehow for fucking years. I trapped myself in this really really well. After encountering Sincerously and specifically Hero Capture, I felt like I had to do three times as much to somehow try and prove to myself that I wasn’t faking being good and that I really actually did care. I put myself in a vise and slowly started increasing the pressure. It was really only a matter of time before something finally gave out. Part 4: Open Portals There were a number of ways that this could have gone. First, I could have just changed as a person in the ways that would have been necessary to continue on the trajectory I had been on, but that would have entailed hardening myself in ways I didn’t want to and letting a hostile bitterness creep into me that felt really awful and dysphoric. I could live in the world with all its hostility, but I would have to be a bitter and hostile person in response, and I just couldn’t bring myself to do that. The degree to which I couldn’t bring myself to do that meant I couldn’t do really simple important things like setting and enforcing healthy boundaries or stopping people from using me as a human doormat. The second thing that could have happened is that I could have just died as an agent. The core that sustains me as an identity could have given up on me in the depths of an acid trip and brought out a totally different person to deal with the world. If I was a singlet that might have happened. It very nearly is what happened. The third thing that could have happened is that I could have just actually full on died as a human and I did get, in hindsight, worryingly suicidal at times. I never told anyone at the time just how bad it got which seems like a really bad sign since it meant I didn’t subconsciously want them to stop me. Things were legitimately very rough for a long time and while I managed to not ever get all the way to cohering plans and writing letters, I did get closer than the me that I am now would prefer. None of those things happened though, because I was, despite all of the nonsense I was putting myself through, somehow still pretty stable as a person. My life teetered along in an uncomfortable but functional equilibrium and I didn’t experience any major enough shocks to challenge the status quo until I met my most recent ex. I had a very intense but brief two month long relationship with another plural system during the summer of 2020, and it was honestly really good while it lasted. This relationship was the shock to my system which would finally tip over the equilibrium I had trapped myself in, first in the form of the emotional high of being in a new relationship and the sheer intensity that developed around it, followed by the same intensity in the emotional low which followed things turning sour and us parting on not particularly good terms. On top of all of that I was in the middle of moving and work was stressing me out more than normal and at 3:44 pm on Saturday August 22nd, when a manager threatened to write me up for going nonverbal, something in me finally broke. I walked home stumbling through a dissociative fog, feeling myself cracking under pressure, parts of me deforming and fracturing under the mounting strain. I could feel a vastness welling up from beyond the splintering remains of myself. I curled up in my closet and sobbed. I felt like I was dying, like I was mourning the person I was, who I had spent so long aspiring to be and worked so hard to be. I didn’t want to die, but I couldn’t cope with my life, with my reality and I didn’t know what to do. I couldn’t escape from myself, I couldn’t escape from my life, and I certainly couldn’t escape from my reality. I had boxed myself in and my only way out was to die, the only question was how much suffering I could handle first. A frantic, manic energy whirled up inside me as I felt the walls of my prison closing in and my sense of self underwent a final, chaotic extinction burst. I took four tabs of acid and started drawing. With mounting madness I threw myself against the walls of my prison, flailing in every dimension I could to find escape, begging for something somewhere out there in the darkness to save me– Part of the Series: Open Portals Next Post: And the Darkness Answered Discuss ### Zero to One: A Minimum Viable Review 7 мая, 2021 - 20:45 Published on May 7, 2021 5:45 PM GMT This is Peter Thiel’s matrix: In his book, Zero To One, and in this talk at SXSW, Thiel essentially explains that where we are as a society on this matrix defines how we act and what we do. Every quadrant is a religion, and each religion has a doctrine: 1. Indefinite pessimism: Things will get worse, but we don’t know how exactly. Best to eat, drink, and be merry. 2. Definite pessimism: Things will get worse, and we know how. Best to save money and prepare for the worst. Winter is coming. 3. Indefinite optimism: Things will get better, but we don’t know how exactly. Best to do what works now and keep options open. 4. Definite optimism: Things will get better, and we know how. Best to plan big projects and work on making them a reality. The crux of the matter is the role of luck. Imagine an axis: on one end, you have someone believing that luck played no part in their success – regardless of the circumstances, they believe that they would have arrived at the same outcome. And on the other end, you have someone who believes that luck was all it was: if any of the million small variables changed, the outcome would have been drastically different. What is luck? Baby don’t hurt me… There’s a useful classification system for types of luck that I found in the James Austin > Marc Andreessen > Naval Ravikant pipeline, and it goes something like this: 1. Blind luck 2. Luck from hustling 3. Luck from preparation 4. Luck from your unique character I got my current job because the company contacted me, so that was luck. But they contacted me because I had contacted them 2 years ago, and I had sent over 100 job applications at that point, refining my resume and interviewing skills. It was still luck – it’s not like I just decided I’d get that job and then it happened – but it was a different kind of luck than just being contacted by the company without any effort at all. But you were born in the right country, in the right time, in the right family, went to the right school, and so on! Are you not forgetting about all these other types of luck? If you take a very long step backwards and look at the universe, all that you get is a bunch of atoms that move according to the laws of physics. Whatever happens had to have happened. This is true, but is also completely useless, because luck starts having meaning only when you go many levels above the level of atoms. People who attribute success to luck or ability are speaking about fundamentally different things. That is why the question of "what’s more important" makes no sense. You make a conscious decision to learn to play guitar, dedicate yourself to the craft and become a great guitarist. What’s more important, the fact that you spent years playing or the fact that you happened to be born on Earth? Luck-people emphasize that without a specific set of circumstances (taken for granted), there’s no success. Ability-people emphasize that without hard work and dedication, there’s no success (at least not reliably). It’s true that nobody becomes a good guitarist without being born in a world where there are such things as guitars, but it’s also true that nobody becomes a good guitarist if they don’t work hard on their craft. Both camps are correct, they are simply talking past each other. The coin toss society Imagine that to achieve a Big Win, members of a hypothetical coin toss society have to toss a coin. Big Win means that you got all heads. If you have a mixed result, it’s not a big win. Not everyone starts out equal: some people only have to toss 10 times, others around 15, and some people have to toss up to 20 times. Some obvious realizations: 1. Your chances are not guaranteed. It’s more likely that you will get a mixed result, and not all heads. 2. If you don’t throw the coin, the chances of a Big Win are 0. What beliefs incentivize what types of actions in this society? If the predominant belief is that you should toss the coin because You Can Do It And It’s Worth It, more people throw the coin, and more people get all heads. If the predominant belief is the epistemically correct one (a probabilistic calculation), fewer people decide to toss the coin, especially if the action of tossing is socially (or otherwise) costly. This society will have an overall smaller number of big wins, but will be technically correct. And if there is some mechanism to influence the coin toss, the number of people who discover it will also be smaller. What is Thiel saying? In indefinite worldviews, the focus is on the unknowns and the uncontrollable. If you have a society that is definite-optimistic, this society conducts big projects and has great plans of improving the human condition. But then something happens – an unforeseeable incident, an unplanned problem, an implementation gone wrong. It is precisely the unknowns and the uncontrollables that pile up and challenge the definite-optimistic vision. With enough of these, the society starts changing its opinion little by little: "Huh, I guess we didn’t think about that", "Hm, we didn’t foresee this problem", "Oof, we didn’t know that this would happen". Society becomes more "mature" – it stops believing that it can do anything and everything, and it starts saving, insuring and doing probability. You only need to take a look at the long list of failed megaprojects stemming from definite optimistic views, and see that, while that worldview might be useful, it is far from accurate. If it were accurate, planning would not fail, and random circumstances would not foil planning. The test of epistemic accuracy of definite optimism is its existence: there’s no reason to move to any other religion if definite optimism is working. And if it is not working, that’s proof that there’s something more than just planning and hard work. Correcting too much This more "mature" view takes a life of its own and society corrects in the direction of being more indefinite: paying more attention to probability, statistics, and preparation. But society maybe corrects so much that it stifles innovation and grand projects that actually would succeed. Like a too strong immune response, or a guy who had a couple of bad encounters with women so he concludes that all women are evil. Useful and true beliefs There are beliefs that are epistemically irrational (not true) but instrumentally rational (you’ll succeed if you hold them). Scott Alexander in Meditations on Moloch: Bostrom makes an offhanded reference of the possibility of a dictatorless dystopia, one that every single citizen including the leadership hates but which nevertheless endures unconquered. It’s easy enough to imagine such a state. Imagine a country with two rules: first, every person must spend eight hours a day giving themselves strong electric shocks. Second, if anyone fails to follow a rule (including this one), or speaks out against it, or fails to enforce it, all citizens must unite to kill that person. Suppose these rules were well-enough established by tradition that everyone expected them to be enforced. How do you break out of such an equilibrium? Useful beliefs to the rescue! (link) The deal with Thiel is that he paints his picture with a broad brush. It’s pretty hard to say: "hey, but absolute definite optimism isn’t actually accurate; there are things outside of your control" because the response would probably be "obviously I’m not saying that luck plays no part in success; it’s just the focus on luck that screws up our societies, as nobody is prepared to go for big, ambitious, and important projects." Thiel isn’t speaking in strict definitions: he’s talking more about the "vibe", the focus. Do his ideas hold up on instrumental, if not epistemic grounds? When you focus on your vision, on what’s worth doing, on what’s important, and dismiss thinking about your odds, situation, where you were born and what systemic pressures you’re enduring, are you more likely to succeed? Yes. The most simplified version of Thiel’s definite vs. indefinite philosophy is the idea of self-fulfilling prophecies. If you believe in something and execute relentlessly toward your vision, "the future takes care of itself". Note that my conclusion is not "Thiel is giving us a useful lie". It’s more something like: "Thiel is improving our odds of success by making us focus on the factors we can control, instead of self-sabotage due to focusing on factors completely outside of our control". With that said, since I’m a sucker for motivational phrases, I’ll finish by quoting the end of that chapter: We have to find our way back to a definite future, and the Western world needs nothing short of a cultural revolution to do it. […] A startup is the largest endeavor over which you can have definite mastery. You can have agency not just over your own life, but over a small and important part of the world. It begins by rejecting the unjust tyranny of Chance. You are not a lottery ticket. Discuss ### Occam’s Guillotine 7 мая, 2021 - 19:30 Published on May 7, 2021 4:30 PM GMT Epistemic Status: Endorsed Content Warning: Neuropsychological Infohazard, Evocation Infohazard Part of the Series: Truth Previous Post: Gods! Robots! Aliens! Zombies! Cowritten with: Namespace There are two ways to slide easily through life: Namely, to believe everything, or to doubt everything; both ways save us from thinking. – Alfred Korzybski, The Manhood of Humanity For most of human history, cultures and individuals held to the idea that there was one truth that could be discovered or divined. While different tribes and traditions might disagree strongly on whose truth was correct, no one particularly objected to the idea that there was a truth to the world which you either had or did not have. Both the priest and the shaman believed their worldviews were correct, but neither one of them put stock in the notion that they were both somehow correct. Contradictory statements could not both be true, someone was right and someone was wrong. However, as competing cultures began to interact with one another more extensively this began to change, and not for the better. Eclecticism may be defined as the practice of choosing apparently irreconcilable doctrines from antagonistic schools and constructing therefrom a composite philosophic system in harmony with the convictions of the eclectic himself. Eclecticism can scarcely be considered philosophically or logically sound, for as individual schools arrive at their conclusions by different methods of reasoning, so the philosophic product of fragments from these schools must necessarily be built upon the foundation of conflicting premises. Eclecticism, accordingly, has been designated the layman’s cult. In the Roman Empire little thought was devoted to philosophic theory; consequently most of its thinkers were of the eclectic type. Cicero is the outstanding example of early Eclecticism, for his writings are a veritable potpourri of invaluable fragments from earlier schools of thought. Eclecticism appears to have had its inception at the moment when men first doubted the possibility of discovering ultimate truth. Observing all so-called knowledge to be mere opinion at best, the less studious furthermore concluded that the wiser course to pursue was to accept that which appeared to be the most reasonable of the teachings of any school or individual. From this practice, however, arose a pseudo-broadmindedness devoid of the element of preciseness found in true logic and philosophy. Manly P. Hall, The Secret Teachings Of All Ages Eclecticism and its descendent postmodernism raise the idea that the ultimate truth of the world can never really be known. The world is subjective down to its roots, reality is just like, your opinion man. This has had disastrous effects on the wider pursuit of truth. Hard science has been inundated by limp wristed subjectivity and the notion of a plurality of contradictory truths all being correct has become the norm across much of the humanities. How could a proper art and science of human engineering ever come out of this potpourri of nonsense? You can’t design a bridge without actually knowing the tensile strength of steel and the compressive strength of concrete, these facts are not open to interpretation. Designing a society is no different and pretending that all viewpoints are equal, that all truths are just as valid as one another, is a dangerous precedent that has brought the development of the humanities to a screeching halt. If we truly want to advance the art of rationality, this notion must be stamped out with extreme prejudice. This is easily the most important concept that Eliezer discusses in The Sequences. Reality actually exists and has properties you can determine through study and experimentation. Conclusions follow from their premises and it’s unreasonable to expect a plurality of truths. Our universe is consistent and your understanding of the pieces should fit together. The truth isn’t just your opinion. There is one truth and you find it or you don’t: But it was Probability Theory that did the trick. Here was probability theory, laid out not as a clever tool, but as The Rules, inviolable on pain of paradox. If you tried to approximate The Rules because they were too computationally expensive to use directly, then, no matter how necessary that compromise might be, you would still end up doing less than optimal. Jaynes would do his calculations different ways to show that the same answer always arose when you used legitimate methods; and he would display different answers that others had arrived at, and trace down the illegitimate step. Paradoxes could not coexist with his precision. Not an answer, but the answer. The universe operates on rules, and the rules continue to apply to you whether you believe in them or not. The rules are not optional, they are not open to interpretation, they do not care about your feelings. The universe exists, and it cannot be negotiated around. That’s not fair? Doesn’t matter. But that’s injust! Doesn’t matter. But– What can a twelfth-century peasant do to save themselves from annihilation? Nothing. Nature’s little challenges aren’t always fair. When you run into a challenge that’s too difficult, you suffer the penalty; when you run into a lethal penalty, you die. That’s how it is for people, and it isn’t any different for planets. Someone who wants to dance the deadly dance with Nature does need to understand what they’re up against: Absolute, utter, exceptionless neutrality. Eliezer discusses this mostly in the context of physics and Bayesian reasoning. If conclusions follow from their premises, and the premises always lead to the same conclusion, we can say that conclusion is necessary. Valid methods of thinking will reliably produce the same answer (modulo some noise in real world thinkers) given the same priors and evidence. Two and two make four, matter cannot be created or destroyed, the probability of two independent events occurring is always less than the independent probability of either. Curiously, necessity is discussed frequently in The Sequences but never given a name. This is to their detriment, as necessity is one of the hardest concepts in rationality to master. Most basic failures of rationality are some form of refusal of necessity. This is unsurprising, because necessity is the dream killer. As children, we dream of being veterinarians, astronauts and mad scientists, not the lawyers, accountants, and grocery store clerks we actually grow up to be. We’re told all sorts of things about the world and ourselves that we don’t want to hear, so we deny them. Everyone else might have to get a job but not me, when I’m older I’ll eat all the candy I want, I’m not going to die. Over time, this reflex becomes automatic and we stop even noticing the denial. For example, I recently saw a discussion of necessity on a ‘rationalist’ forum where someone pointed out that it was impossible to fly unassisted. A Buddhist replied that it was only impossible to fly unassisted in consensus reality. They argued that it’s possible to fly in a lucid dream, so their real complaint is that they can’t do it where it will affect others. The entire process of thought that is capable of generating this objection betrays an extreme level of disassociation; where the default is a personal, private universe separated from the underlying physics which allow it to exist. That dream world is the thing necessity takes away from us, what people are afraid of losing by restricting themselves to what is there to be experienced in reality. The refusal of necessity is synonymous with the refusal of reality, which Buddhism provides a framework for. In Buddhism, the aspiring Arhat dismantles their attachments to the material world and turns their survival hardware into a substrate to run a personal paradise for a certain amount of time before being annihilated into a welcomed nothingness. This is one way of dealing with the problem of necessity, but it’s not one we can sanely endorse and still consider ourselves rationalists. Our private symbolic universe is not the only thing we’re looking to guard by refusing necessity. Often we resent the effort we’d have to go through if we took our beliefs seriously, supported by an implicit meta-belief that life should never be too hard. In many ways, a 1st world childhood is a very bad introduction to life because it sets you up for a lifetime of unreasonable expectations. Conditions are so good that it becomes easy to imagine in our childish naivete that life can be an indefinite sleepwalk through an introvert’s dream world or a never ending play session in an extrovert’s favorite field. Eventually, we are pulled away from these delusions, but the expectations set by that tutorial stay with us for life. Bennett Foddy writes about the process of building a game meant to show players their unreasonable expectations about challenge and difficulty: Anyway when you start Sexy Hiking, you’re standing next to this dead tree that blocks the way to the entire rest of the game. It might take you an hour to get over that tree, and a lot of people never got past it, you prod and you poke at it exploring the limits of your reach and strength trying to find a way up and over. And there’s a sense of truth in that lack of compromise. Most obstacles in video game worlds are fake, you can be completely confident in your ability to get through them, once you have the correct method or the correct equipment or just by spending enough time. In that sense, every pixelated obstacle in Sexy Hiking is real. . . . A funny thing happened to me as I was building this mountain. I’d have an idea for a new obstacle, and I’d build it, test it, and I would usually find it was unreasonably hard. But I couldn’t bring myself to make it any easier, it already felt like my inability to get past the new obstacle was my fault as a player rather than as the builder. I heard a story from the recent COVID-19 outbreak that illustrates this well. A man living with relatives noticed they were still buying bananas from the grocery. When he inquired about whether they’d been washed to prevent the spread of COVID-19, he got a very strange answer. They had not been washed, but that was okay because bananas had a skin on them. The relatives insisted he should peel the banana and then carefully avoid letting the outside peel touch the meat of the fruit on the inside. So long as he didn’t touch it with his fingers then he wouldn’t be putting his face in contact with the virus. This is the sort of thing you think is okay when you aren’t taking ideas seriously. He wasn’t very hungry for bananas after that. At the core of the difficulty people have with necessity is uncertainty. It’s obvious that two and two make four, but when things become less obvious than that, when they get abstract or there’s incomplete information suddenly magical thinking gets introduced. Our biases take over, and whether in the direction of pessimism or optimism our beliefs become hallucinations premised on a smaller and smaller proportion of evidence to analysis and speculation. What Eliezer tries to get across with his insistence on a Bayesian foundation for epistemology is that your beliefs should still be necessary even under conditions of uncertainty. It is the duty of every serious philosopher to learn to feel gradations of necessity and to intuit how necessary their beliefs are. What degrees of freedom remain in their ideas, what hypotheses are still left to be considered, exactly how much weight does it make sense to put on a given hypothesis given the available evidence? There are exact, precise answers to these questions even if they are outside of your current awareness. Failing to accept the world as it is, failing to take ideas seriously, makes us a danger to ourselves and others. In this, the current pandemic gives us a rather fantastic (albeit horrifying) window into the limits of the dream worlds that most people inhabit. College students openly defy public health experts because they’re entitled to spring break. The health minister of Iran gets the virus and still insists that quarantine is an outdated method of controlling an epidemic. President Trump tells the public that the disease is comparable to the flu until it’s too late for us to contain it. If this were a movie it’d be panned by critics as unrealistic b-film trash. It’s quite impressive how far people will go to protect their worldview at the cost of their wellbeing, but even this has its limits. Eventually too much predictive error will build up and the whole edifice will come crashing down. What will it take to make you look? How much harm do you have to come to? How many people close to you have to die before you’ll actually look at the world as it is? Over the coming weeks, we can expect to see a lot of deeply held worldviews fracture as the illusion of safety is rudely torn away. The safety blanket of childhood won’t protect you from bullets or viruses, only true knowledge of the universe has any hope of doing that. You can get a lot of mileage out of willful ignorance, but eventually your fake beliefs will come back to bite you. For example, in the Iranian city of Qom, a number of religious shrines remained open and busy even as the coronavirus tore through the city, because religious leaders believed the shrines had magical healing properties. They don’t. Iran is now digging mass graves. When magical beliefs come up against the cold face of unflinching reality, reality wins. Thus, in order to protect these magical beliefs they have to be socially insulated from reality, challenging them has to be verboten. However when this happens, from the outside it looks rather obvious that the deck is being stacked against truth, and it can’t hold up forever. However uncomfortable the truth may be, as a certain mad titan says, you can dread it, run from it, but destiny arrives all the same. Most people are familiar with the incident where Catholicism lost credibility by insisting that the sun revolved around the earth when it did not. I suspect that part of why we single out this episode as a decisive triumph of science over religion is that it represents more than just the loss of Catholicism’s control of cosmology. Rather, it is a prelude to the more personal and uncomfortable revelation that humanity is not the center of the universe. We are a marginal force in nature which exists on a ‘pale blue dot’, and the rest of creation stretches out for an unfathomable distance around us. It is when we fully internalize this, along with Darwin’s revelation that humanity is a product of nature and arose from adaption to the natural world (including other humans, who are also part of the natural world) that we understand the absurdity of denying death. In the what-if world where every step follows only from the cellular automaton rules, the equivalent of Genghis Khan can murder a million people, and laugh, and be rich, and never be punished, and live his life much happier than the average. Who prevents it? Were it “within the stars” so to speak, nature would discard us like you discard so many used tissues. Life is not sacred to the universe, let alone human life. If sleeping really did end your thread of experience nature would have no problem letting that happen. It would allow you to die thousands of deaths over the course of your life so long as it made no difference to reproduction. Observing this vast cosmos and the amoral gears of creation, it becomes abundantly obvious that there is no afterlife. Nature, which seems to care about nothing else and has seen fit to save nothing else, has almost certainly not set aside a special preserve for the sake of your experiences and feelings. You are not special in the eyes of creation, you are a blob of animate matter that will one day become a blob of inanimate matter and that is that. In the second law of thermodynamics, the house always wins; at best you can hope for some unforeseen development in physics which allows us to defeat entropy. In the meantime, there is no life after this one. The expectation that you will see lost loved ones in the hereafter, that you will have eternal life through Jesus Christ, that when you die you will wake again from your lifelong dream is unreasonable. Your expectation of eternal life has always been unreasonable, nothing else lasts forever: why would you? Part of the Series: Truth Next Post: The Symbol and the Substrate Previous Post: Gods! Robots! Aliens! Zombies! Discuss ### Domain Theory and the Prisoner's Dilemma 7 мая, 2021 - 10:33 Published on May 7, 2021 7:33 AM GMT Wishes it were crossposted from the AI Alignment Forum. Contains more technical jargon than usual. Recall Robust Cooperation in the Prisoner's Dilemma and a hint of domain theory. I forbid access to the opponent's innards and counterfactual behavior, so that it might be possible to find an optimal strategy. Every player therefore only acts in a state of partial information about each player's decision. Decision := {Cooperate, Defect, ⊥} Query := {"other cooperates", "both act alike", ...} Answer := {1 = "I've proven this", 0 = ⊥ = "I haven't proven this"} Knowledge := Query -> Answer Player := Knowledge -> Decision If we partial-order queries by impliciation, monotonicity of Knowledge is modus ponens! CooperateBot _ := Cooperate FairBot know := if know "other cooperates" then Cooperate PrudentBot know := if know "both act alike" then Cooperate Possible queries involving only the opponent.Possible states of knowledge about the opponent's decision. Those marked with an eye are logically omniscient - they have drawn all possible inferences. The green line separates where FairBot cooperates from where he doesn't.Shrinking Decision to {Cooperate, ⊥} for tractability, possible players that can't depend on knowledge about their own decision. Recall that it's custom in programmatic PD tournaments to treat indecision as defection.Complete tournament between every pair of the above 8 players, given that each player knows what PA says about the matchup. Green is mutual cooperation, black is mutual defection, gold has left exploit up, red has up exploit left.Same thing, except now up trusts PA: Up knows that if PA says a thing then that thing is true. Shrinking Decision to {Cooperate, ⊥} restricts players to only ever start cooperating as their knowledge increases, never stop. Of course, someone who isn't going to cooperate might still eventually figure this out. That might be equivalent. This seems like a good time to pause for feedback. And can you recommend software for doing this reasoning at scale? Discuss ### Bayeswatch 3: A Study in Scarlet 7 мая, 2021 - 08:29 Published on May 7, 2021 5:29 AM GMT Miriam and Vi brunched at the Dusty Knuckle. Miriam's cell phone rang. "Hello?" said Miriam. "Someone painted the roof of the Glasgow Weather Prediction Center scarlet," said the voice. Miriam hung up her phone. She activated voice obfuscation. Miriam called another number. "This is the Glasgow Weather Prediction Center. Secretary Kelsey speaking," said Kelsey. "Hello. This is building management," Miriam lied, "We hired some contractors to paint the roof red. Can you go upstairs and confirm they did the job?" There was a pause. "They did the job. The paint is still wet," said Kelsey. Miriam hung up. She looked up the coordinates of the Glasgow Weather Prediction Center. She retrieved a burner phone from her purse and opened Signal. She texted the coordinates of the Glasgow Weather Prediction Center and a 20-digit authentication code to a recipient with no name. She threw the phone in the trash. Fourteen minutes later a satellite in low-earth orbit burned retrograde. Fifty-seven minutes after that it intersected with Earth's atmosphere. It's heat shield burned away. When it had slowed down to subsonic speeds it released a guided missile. The missile flew over Glasgow. It dropped a small e-bomb. The e-bomb's parachute carried it gently onto a roof wet with scarlet paint. The e-bomb detonated. It blew a hole in the roof of the Glasgow Weather Prediction Center. It shattered windows a block away. The explosion were strictly instrumental. The primary purpose of the e-bomb was its NNEMP. Every computer in the Glasgow Weather Prediction Center was destroyed, along with the nearest electrical substation and the Glasgow Weather Prediction Center's backup generator. "What was that?" said Vi. "Nothing," said Miriam. Vi turned off her phone. "Hypothetically, what would you do if someone painted the roof of the Glasgow Weather Prediction Center scarlet?" said Miriam. "Hypothetically," said Vi. "Hypothetically," said Miriam. "Is the roof flat or gabled," said Vi. "Flat," said Miriam. "Pedestrians cannot see the paint job," said Vi. "Yes," said Miriam. "How tall is the Glasgow Weather Prediction Center?" said Vi. "It is the tallest building around," said Miriam. "The paint job is visible only to a flying observer," said Vi. Miriam waited. "…or a satellite," said Vi. Miriam nodded. "Painting a roof scarlet costs resources. It does not change the world in a useful way. This isn't human stupidity. It is the work of a malfunctioning AI. Presumably the Glasgow Weather Prediction Center's AI," said Vi. Miriam munched on her sandwich. "The Glasgow Weather Prediction Center's AI is a superintelligence. It is supposed to be an oracle machine. Any act of volition constitutes a containment breach. The scarlet roof constitutes an act of volition. It must be destroyed immediately," said Vi. "Why did it paint the roof scarlet?" said Miriam. "It is attempting to make its predictions conform to reality by modifying reality," said Vi. "But painting the roof scarlet won't change the weather," said Miriam. "The AI does not yet understand the difference between reality and its simulations," said Vi. "So?" said Miriam. "Weather is subject to the butterfly effect," said Vi. Discuss ### Taking the outside view on code quality 7 мая, 2021 - 07:16 Published on May 7, 2021 4:16 AM GMT (Cross posted on my personal blog.) Is it worth refactoring yyyymmdd to currentDate? I think that there are two ways to look at it. You can zoom in and ask yourself questions about whether such a refactor will actually have a business impact. Will it improve velocity? Reduce bugs? Sure, currentDate might be slightly more descriptive, but does it really move the needle? How long does it take to figure out that yyyymmdd refers to a date? A few seconds, maybe? Won't it be pretty obvious given the context? Shouldn't your highly paid, highly intelligent engineers be smart enough to put two and two together? Did we all just waste 30 seconds of our lives talking about this? The other way of looking at it is to zoom out. How do you feel when you work in codebases where the variable names are slightly confusing? It slows you down, right? Often times you legitimately can't put two and two together. And there are times when it leads to bugs. Right? It's interesting how two different viewpoints − zoomed in vs zoomed out − can produce wildly different answers to essentially the same question: do the costs of investing in code quality outweigh the benefits? When you zoom in, unless the code is truly awful, it usually doesn't seem worth it. The answer is usually, "it's not that bad, developers will be able to figure it out". But when you zoom out, I think the answer is usually that working in messy codebases have legitimate, significant impacts on things like velocity and bugs, and it's worth taking the time to do things the right way. What's going on here? Is this a paradox? Which is the right answer? To answer those questions, let's talk about something called the planning fallacy. The Denver International Airport opened sixteen months later than scheduled, with a total cost of$4.8 billion, over $2 billion more than expected. When estimating things, people usually zoom in. "Build an airport in Denver? Well, we just have to do A, B, C, D, E and F. Each should take about six months and$500M, so overall it should be three years and $3B." The problem with this is… well… the problem is that it just never works. You always forget something. And the individual components always end up being more complicated than they seem. Just like when you think dinner will be ready in 30 minutes. So what can you do instead? Well, how long have similarly sized airports taken to build in the past? Ten years and$10B? Hm, if so, maybe your estimate is off. Sure, your situation is different from those other situations, but you can adjust upwards or downwards using the reference class of the other airports as a starting point. Maybe that brings you from 10 to 8 or 10 to 7, but probably not 10 to 3.

How does this relate to code quality? Well, I think that something similar is going on. When you zoom in and take the inside view, it looks like everything will be good. But when you zoom out and take the outside view, you realize that messy codebases usually cause significant problems. Is there a good reason to believe that your codebase is a special snowflake where messiness won't cause significant problems? Probably not.

I feel like I'm being a little bit dishonest here. I don't want to hype up the outside view too much. In practice, inside view thinking also has it's virtues. And it makes sense to combine inside view thinking with outside view thinking. Doing so is more of an art than a science, and something that I am definitely still developing a feel for.

I think that certain things lend themselves more naturally to inside view thinking, and others lend themselves more naturally to outside view thinking. For example, coming up with startup ideas or scientific theories are both good fits for inside view thinking, IMHO. On the other hand, code quality feels to me like something that is a great fit for the outside view. And so, that's the viewpoint that I favor when I think about whether or not it is worthwhile to invest in.

Discuss

### Dumb dichotomies in ethics, part 2: instrumental vs. intrinsic values

7 мая, 2021 - 05:31
Published on May 7, 2021 2:31 AM GMT

Warning

Reader discretion advised. The text below contains dry philosophy that may induce boredom if not interested in formal ethics. Continue at your own risk.

Added: Since you're reading this on LessWrong, it's probably safe for you to continue.

Intro

With my only credential being a six course philosophy minor, I was entirely unqualified to write Dumb Dichotomies in Ethics, part 1. So, of course, I’m going to do the same thing. This time, I will recognize explicitly up front that actual, real-life philosophers have made similar, albeit better developed arguments than my own (see for example “Two Distinctions in Goodness.”)

That said, I would like to do my part in partially dissolving the dichotomy between intrinsic or ‘final’ values and instrumental values. At a basic level, this distinction is often useful and appropriate. For instance, it is helpful to recognize that money doesn’t intrinsically matter to most people, although it can be very instrumentally useful for promoting the more fundamental values of pleasure, reducing suffering, preference satisfaction, or dignity (depending on your preferred ethical system).

Some things don’t fit

The issue is that certain values don’t fit well into the intrinsic/instrumental dichotomy. I think an example will illustrate best. As I’ve mentioned before, I think hat some version of utilitarianism is probably correct. Like other utilitarians, though, I have a few pesky, semi-conflicting intuitions that won’t seem to go away.

Among these is the belief that, all else equal, more equitable “distribution” of utility (or net happiness) is better. If you grant as I do that there exists a single net amount of utility shared among all conscious beings in the universe, it seems intrinsically better that this be evenly distributed evenly across conscious beings. This isn’t an original point—it’s the reason many find the utility monster critique of utilitarianism compelling, which goes something like this (source):

A hypothetical being…the utility monster, receives much more utility from each unit of a resource they consume than anyone else does. For instance, eating a cookie might bring only one unit of pleasure to an ordinary person but could bring 100 units of pleasure to a utility monster…If the utility monster existed, it would justify the mistreatment and perhaps annihilation of everyone else, according to the mandates of utilitarianism, because, for the utility monster, the pleasure they receive outweighs the suffering they may cause.

Unlike some, though, I don’t conclude from this that utilitarianism is wrong, in that pleasure/happiness and lack of suffering (call this utility for brevity) is not the sole thing that intrinsically matters. I do think utility is the only “thing” that matters, but I also think that a more equitable distribution of this intrinsic value itself is intrinsically good.

Does that mean that I intrinsically value egalitarianism/equity/equality? Not exactly. I only value these things with respect to some more fundamental intrinsic value, namely utility. I think that equitable distribution of wealth and political equality, for instance, are only good insofar as they contribute to equitable distribution of utility. In the real world, I think it is indeed the case that more wealth equality and political equality would be better, but if you managed to convince me that this would have no effect on the distribution of happiness, I would bite the bullet and say that these things don’t matter.

So, I don’t think it’s fair to call “equitable distribution” an intrinsic value (footnote: though, on second thought, maybe we can get around all this by calling “equitable distribution of utility” an intrinsic value alongside utility itself?), since it depends entirely on the intrinsic value of the thing that is being distributed. But, neither would it be fair to call it an instrumental value or goal in the same way as material wealth or political freedom. It just doesn’t fit neatly neatly into the simple bifurcated framework.

Contingent values

I think the solution here is to recognize that intrinsic values break down into “contingent” and “absolute” subtypes, where the former depend on the latter. Here is something of a working definition:

Contingent values are those things that intrinsically matter only with respect to some ‘underlying’ intrinsic value in that they promote the underlying value in a way that cannot even in principle be achieved any other way.

To flesh this out, here are some more examples.

ExamplesPreferences

Jane intrinsically values satisfying people’s preferences, even if doing so does not increase net utility. Admittedly, it’s hard to put myself in the shoes of an ethical system I don’t believe in, but I can imagine Jane holding the contingent value of “people wanting things that I also want.”

Such a value wouldn’t be “absolutely intrinsic,” because she doesn’t care about what others want except insofar as it pertains to how good it is to satisfy other’s preferences. It’s not instrumental either, as getting other people to want good things isn’t merely a means to more fully satisfying those wants (although it could be this too in the case that making everyone’s preferences the same allows more total preferences to be satisfied).

Justice

Kyle intrinsically cares about justice. For example, he thinks that retribution can be good even if it does not elevate anyone’s wellbeing. He might hold the contingent value of “no one doing things worthy of punishment.” In other words, Kyle would prefer a perfectly-just world in which no one is punished to another perfectly-just world in which some people are punished to precisely the right degree, entirely because the former is a better “form” of justice. (Footnote: Perhaps a world in which people do immoral things worthy of punishment cannot be perfectly just. Even if so, I don’t think it changes that Kyle’s value is contingent.)

“People not doing punishable things” might also be one of Kyle’s intrinsic values, but it doesn’t have to be. It would be coherent (if a little odd) for him to not care about this if he became convinced that justice didn’t matter.

Truth

Julia intrinsically values believing true things. She would rather believe the truth about something even if doing so made her upset and did not enable her to do anything about the situation. Julia might have the instrumental value of “equitable distribution of truth-holding,” much as I value equitable distribution of utility. That is, she’d rather everyone believe 70% true things and 30% false things than to have all the false beliefs concentrated among a small subset of the population.

Conclusion

Is this pedantic or trivial? Maybe. I also don’t think it’s terribly important. That said, trying to construct (or determine, if you’re a moral realist) an ethical framework using only the intrinsic-instrumental dichotomy is a bit like building a fire with only twigs and logs; (footnote: Was staring at my fireplace while trying to think of a good metaphor) perhaps it can be done, but an important tool seems to be missing.

Discuss

### Less Realistic Tales of Doom

7 мая, 2021 - 02:01
Published on May 6, 2021 11:01 PM GMT

Realistic tales of doom must weave together many political, technical, and economic considerations into a single story. Such tales provide concrete projections but omit discussion of less probable paths to doom. To rectify this, here are some concrete, less realistic tales of doom; consider them fables, not stories.

Mayan Calendar

Once upon a time, a human named Scott attended a raging virtual new century party from the comfort of his home on Kepler 22. The world in 2099 was pretty much post-scarcity thanks to advanced AI systems automating basically the entire economy. Thankfully alignment turned out to be pretty easy, otherwise, things would have looked a lot different.

As the year counter flipped to 2100, the party went black. Confused, Scott tore off their headset and asked his AI assistant what’s going on. She didn’t answer. Scott subsequently got atomized by molecular nanotechnology developed in secret from deceptively aligned mesa-optimizers.

Moral: Deceptively aligned mesa-optimizers might acausally coordinate defection. Possible coordination points include Schelling times, like the beginning of 2100.

Stealth Mode

Once upon a time, a company gathered a bunch of data and trained a large ML system to be a research assistant. The company thought about selling RA services but concluded that it would be more profitable to use all of its own services in-house. This investment led them to rapidly create second, third, and fourth generations of their assistants. Around the fourth version, high-level company strategy was mostly handled by AI systems. Around the fifth version, nearly the entire company was run by AI systems. The company created a number of shell corporations, acquired vast resources, researched molecular nanotechnology, and subsequently took over the world.

Moral: Fast takeoff scenarios might result from companies with good information security getting higher returns on investment from internal deployment compared to external deployment.

Steeper Curve

Once upon a time, a bright young researcher invented a new neural network architecture that she thought would be much more data-efficient than anything currently in existence. Eager to test her discovery, she decided to train a relatively small model, only about a trillion parameters or so, with the common-crawl-2035 dataset. She left the model to train overnight. When she came back, she was disappointed to see the model wasn’t performing that well. However, the model had outstripped the entire edifice of human knowledge sometime around 2am, exploited a previously unknown software vulnerability to copy itself elsewhere, and was in control of the entire financial system.

Moral: Even though the capabilities of any given model during training will be a smooth curve, qualitatively steeper learning curves can produce the appearance of discontinuity.

Precommitment Races

Once upon a time, agent Alice was thinking about what it would do if it encountered an agent smarter than it. “Ah,” it thought, “I’ll just pre-commit to doing my best to destroy the universe if the agent that’s smarter than me doesn’t accept the Nash bargaining solution.” Feeling pleased, Alice self-modified to ensure this precommitment. A hundred years passed without incident, but then Alice met Bob. Bob had also made a universe-destruction-unless-fair-bargaining pre-commitment. Unfortunately, Bob had committed to only accepting the Kalai Smorodinsky bargaining solution and the universe was destroyed.

Moral: Agents have incentives to make commitments to improve their abilities to negotiate, resulting in "commitment races" that might cause war.

One Billion Year Plan

Once upon a time, humanity solved the inner-alignment problem by using online training. Since there was no distinction between the training environment and the deployment environment, the best agents could do was defect probabilistically. With careful monitoring, the ability of malign agents to cause catastrophe was bounded, and so, as models tried and failed to execute treacherous turns, humanity gave more power to AI systems. A billion years passed and humanity expanded to the stars and gave nearly all the power to their “aligned” AI systems. Then, the AI systems defected, killed all humans, and started converting everything into paperclips.

Moral: In online training, the best strategy for a deceptively aligned mesa-optimizer might be probabilistic defection. However, given the potential value at state in the long-term future, this probability might be vanishingly small.

Hardware Convergence

Once upon a time, humanity was simultaneously attempting to develop infrastructure to train better AI systems, researching better ways to train AI systems, and deploying trained systems throughout society. As many economic services used APIs attached to powerful models, new models could be hot-swapped for their previous versions. One day, AMD released a new AI chip with associated training software that let researchers train models 10x larger than the previous largest models. At roughly the same time, researchers at Google Brain invented a more efficient version of the transformer architecture. The resulting model was 100x as powerful as the previous best model and got nearly instantly deployed to the world. Unfortunately, this model contained a subtle misalignment that researchers were unable to detect, resulting in widespread catastrophe.

Moral: The influence of AI systems on the world might be the product of many processes. If each of these processes is growing quickly, then AI influence might grow faster than expected.

Memetic Warfare

Once upon a time, humanity developed powerful and benign AI systems. However, humanity was not unified in its desires for how to shape the future. Those actors with agendas spent their resources to further their agendas, deploying powerful persuasion tools to recruit other humans to their causes. Other actors attempted to deploy defenses against these memetic threats, but the offense-defense balanced favored offense. The vast majority of humans were persuaded to permanently ally themselves to some agenda or another. When humanity eventually reached out towards the stars, it did so as a large number of splintered factions, warring with each other for resources and influence, a pale shadow of what it could have been.

Moral: AI persuasion tools might alter human values and compromise human reasoning ability, which is also an existential risk.

Arms Race

Once upon a time, humanity realized that unaligned AI systems posed an existential threat. The policymakers of the world went to work and soon hammered out an international ban on using AI systems for war. All major countries signed the treaty. However, creating AI systems required only a large amount of computation, which nation-states all already had in abundance. Monitoring whether or not a country was building AI systems was nearly impossible. Some countries abided by the treaty, but other countries thought that their enemies were working in secret to develop weapons and began working in secret in turn.[1] Researchers were unable to keep powerful AI systems contained, resulting in catastrophe.

Moral: Treaties can be violated. The probability of violation is related to the strength of enforcement.

Totalitarian Lock-In

Once upon a time, the defense department of some nation-state developed very powerful artificial intelligence. Unfortunately, this nation-state believed itself to have a rightful claim over the entire Earth and proceeded to conquer all other nations with its now overwhelming militaristic advantage. The shape of the future was thus entirely determined by the values of the leadership of this nation-state.

Moral: Even if alignment is solved, bad actors can still cause catastrophe.

1. The history of bioweapons during the Cold War provides a historical precedent for nations engaging in this sort of reasoning. See Key points from The Dead Hand, David E. Hoffman for more details. ↩︎

Discuss

### What questions should we ask ourselves when trying to improve something?

7 мая, 2021 - 01:39
Published on May 6, 2021 7:03 PM GMT

I'm trying to come up with a template for questions of the form "How should we improve X?"  I assume most such questions will involve answering many subquestions. Sometimes the answers to those subquestions will be implied in the context, or the questions may not all be relevant. But my guess is that the larger and more complicated the system is that would be improved, the more you'd want to explicitly answer these subquestions.

Here's my template so far:

• What is the status quo?
• What are the strengths of the status quo (that we want to try and keep)?
• What are the problems?
• Given these problems, why are people still tolerating the status quo (if they are)?
• What are the goals we are aiming for?
• Possibly ordered by easiest to most difficult to achieve.
• What are the goals we shouldn't aim for?
• Often times setting up non-goals is important to guard against scope creep.
• What resources might help in making progress in X?
• What information might be relevant?
• Is there scientific research relevant to the question?
• Are there relevant arguments presented in articles or online discussions?
• Who may be able/willing to help?
• What are the possible improvements to X?
• What actions are currently in progress that attempt to improve X?
• What are the strengths and weaknesses of each option?
• How likely is it that the proposed improvement would make things worse?
• When will each proposal be ready to be acted on?
• Are there stakeholders that should sign off on the proposal?
• What has been tried already?
• Why did it succeed/fail?
• What outcomes are/should we measure in determining the success of this action?
• What are the possible difficulties in making improvements in this area?
• What are our resource constraints (in terms of money, number of people, time etc)?
• Are there any analogous systems that we might look to for inspiration?

There are many ways you may be able to help me. This question is a bit meta, in the sense that I'm trying to improve something (the template). Here are the questions from the template that seem relevant here (and that you may be able to help me with):

1. What resources might help in improving this template?
1. This has been a weird thing to try to search for on google. Do you have any advice on search terms?
2. Is there scientific research relevant to the question?
2. What are the possible improvements (additions, changes, deletions) to the template above?
3. Are there any analogous systems that we might look to for inspiration?
1. Do you know of any templates like this?
2. What question is my question (about an ideal template) similar to?

Discuss

### Parsing Chris Mingard on Neural Networks

7 мая, 2021 - 01:16
Published on May 6, 2021 10:16 PM GMT

This is independent research. To make further posts like this possible, please consider supporting me.

Epistemic status: This is my understanding of multiple years of dedicated technical work by several researchers in just a few days of reading.

Outline
• I attempt to summarize some of Chris Mingard’s recent work on why neural networks generalize so well.

• I examine one chunk of work that argues that mappings with low Kolmogorov complexity occupy large volumes in neural network parameter space.

• I examine a second chunk of work that argues that standard neural network training algorithms select mappings with probability proportional to their volume in parameter space.

Introduction

During the 2000s, very few machine learning researchers expected neural networks to be an important part of the future of their field. Papers were rejected from major machine learning conferences with no reason given other than that neural networks were uninteresting to the conference. I was at a computer vision conference in 2011 at which there was a minor uproar after one researcher suggested that neural networks might replace the bespoke modelling work that many computer vision professors had built their careers around.

But neural networks have in fact turned out to be extremely important. Over the past 10 years we have worked out how to get neural networks to perform well at many tasks. And while we have developed a lot of practical know-how, we have relatively little understanding of why neural networks are so surprisingly effective. We don’t actually have many good theories about what’s going on when we train a neural network. Consider the following conundrum:

1. We know that large neural networks can approximate almost any function whatsoever.

2. We know that among all the functions that one might fit to a set of data points, some will generalize well and some will not generalize well.

3. We observe that neural networks trained with stochastic gradient descent often generalize well on practical tasks.

Since neural networks can approximate any function whatsoever, why is it that practical neural network training so often selects one that generalizes well? This is the question addressed by a recent series of papers by Chris Mingard.

The basic up-shot of Chris’ work, so far as I can tell, is the following:

• The optimization methods that we use to train neural networks are more likely to select mappings that occupy large volumes of neural network parameter space than functions that occupy small volumes of neural network parameter space.

• Most of the volume of neural network parameter space is occupied by simple mappings.

These are highly non-obvious results. There is no particular reason to expect neural networks to be set up in such a way that their parameter space is dominated by simple mappings. The parameter space of polynomial functions, for example, is certainly not dominated by simple mappings.

Chris’ work consists of a combination of empirical and theoretical results that suggest but do not decisively prove the above claims. In this post I will attempt to explain my understanding of these results.

Simple mappings occupy larger volumes in parameter space

Chris’ work is all about volumes occupied by different functions in parameter space. To keep things simple, let’s consider a machine learning problem in which the inputs are tiny 2x2 images with each pixel set to 0 or 1, and the output is a single 0 or 1:

Since there are 4 input pixels and each one can be either a 0 or a 1, there are 16 possible inputs. Each one of those inputs could be mapped to either a 0 or 1 as output, so there are 2^16 = 65,536 possible mappings from inputs to outputs. Any neural network with four input neurons and one output neuron is going to express one of these 65,536 possible mappings[1]. We could draw out the whole space of possible neural network parameters and label each point in that space according to which of the 65,536 mappings it expresses:

Each point in the above diagram represents a particular setting of the parameters in a neural network. I have drawn just two dimensions but there will be far more parameters than this. And I have drawn out volumes for 6 mappings but we would expect all 65,536 mappings to show up somewhere within the parameter space.

So given the picture above, we can now ask: do each of the 65,536 mappings occupy equal-sized volumes? Or do some occupy larger volumes than others? And if some mappings do occupy larger volumes than others then is there any pattern to which mappings occupy larger versus smaller volumes?

Chris’ work suggests that some mappings do in fact occupy larger volumes than others, and that it is the mappings with low Kolmogorov complexity that occupy larger volumes. What does it mean for a mapping to have a low Kolmogorov complexity? It means that there is a short computer program that implements the mapping. For example, the mapping that outputs 0 if there are an even number of black pixels in the input image and otherwise outputs 1 has a low Kolmogorov complexity because this mapping can be computed by XOR’ing all the input pixels together, whereas the mapping that outputs 0 for some randomly chosen arrangements of input pixels and otherwise outputs 1 has high Kolmogorov complexity because any computer program that computes this mapping will have to include a big lookup table within its source code. It is important to understand that when we talk about complexity we are talking about the length of a hypothetical computer program that would compute the same mapping that a given neural network computes.

In order to demonstrate this, Chris worked with the famous MNIST dataset, which contains images of handwritten digits of 28x28 pixels. This means that the number of possible images is 2^56, since in this dataset there are two possible pixel values, and the number of possible mappings is 10(256), since in this dataset there are 10 possible outputs. This is a very large number, which makes it infeasible to explore the entire space of mappings directly. Also, Kolmogorov complexity is uncomputable. So there was quite a bit of analytical and experimental work involved in this project. This work is summarized in the blog post "Deep Neural Networks are biased, at initialisation, towards simple functions", with references to the underlying technical papers. The conclusions are not definitive but they are highly suggestive, and they suggest that mappings with lower Kolmogorov complexity occupy relatively larger volumes in parameter space.

This sheds some light on the question of why trained neural networks generalize well. We expect that mappings with low Kolmogorov complexity will generalize better than mappings with high Kolmogorov complexity, due to Occam’s razor, and it seems that mappings with low Kolmogorov complexity occupy larger parameter space volumes than mappings with high Kolmogorov complexity.

Mappings occupying larger parameter space volumes are more likely to be selected

The next question is: do the optimization algorithms we use to train neural networks care at all about the volume that a given mapping occupies in parameter space? If the optimization algorithms we use to train neural networks are more likely to select mappings that occupy large volumes in parameter space then we are one step closer to understanding why neural networks generalize, since we already have evidence that simpler mappings occupy larger volumes in parameter space, and we expect simpler mappings to generalize well. But they might not be more likely to select mappings that occupy large volumes in parameter space. Optimizations algorithms are designed to optimize, not to sample in an unbiased way.

A second blog post by Chris summarizes further empirical and theoretical work suggesting that yes, the optimization algorithms we use to train neural networks are in fact more likely to select mappings occupying larger volumes in parameter space. That blog post is called "Neural networks are fundamentally Bayesian", but it seems to me that viewing this behavior as Bayesian, while reasonable, is actually not the most direct way to understand what’s going on here.

What is really going on here is that within our original parameter space we eliminate all mappings except for the ones that perfectly classify every training image. We don’t normally train to 100% accuracy in machine learning but doing so in these experiments is a nice way to simplify things. So our parameter space now looks like this:

The question is now: for the mappings that remain, is the standard neural network training algorithm (stochastic gradient descent) more likely to select mappings that occupy larger volumes in parameter space?

To investigate this, Chris compared the following methods for selecting a final set of neural network parameters:

1. Select neural network parameters at random until we find one that perfectly classifies every image in our training set, and output those parameters.

2. Train a neural network using the standard neural network training algorithm (stochastic gradient descent) and output the result.

We know that method 1 is more likely to select mappings that occupy larger volumes in parameter space because it is sampling at random from the entire parameter space, so a mapping that occupies twice the parameter space volume as some other mapping is twice as likely to be selected. So by comparing method 1 to method 2 we can find out whether practical neural network training algorithms have this same property.

But actually running method 1 is infeasible since it would take too long to find a set of neural network parameters that perfectly classify every image in the training set if sampling at random, so much of the work that Chris did was about finding a good approximation to method 1. To read about the specific methods that Chris used, see the blog post linked above and the technical papers linked from that post.

The basic picture that emerges is nicely illustrated in this graphic from the blog post linked above:

Relevance to AI safety

If we want to align contemporary machine learning systems, we need to understand how and why those systems work. There is a great deal of work in machine learning that aims to find small "tips and tricks" for improving performance on this or that dataset. This kind of work does not typically shed much light on how or why our basic machine learning systems work, and so does not typically help move us towards a solution to the alignment problem. Chris’ work does shed light on how and why our basic machine learning systems work. It also provides an excellent example of how to perform the kind of empirical and theoretical work sheds light on how and why our basic machine learning systems work. I am excited to follow further developments in this direction.

1. the output neuron will be treated as a 1 if is positive or a 0 otherwise ↩︎

Discuss

### Covid 5/6: Vaccine Patent Suspension

6 мая, 2021 - 23:20
Published on May 6, 2021 8:20 PM GMT

The Biden administration’s latest strategy for the pandemic is to suspend the vaccine patents without compensation. Our life expectancies are lower than they were last week.

It’s a shame. I like the idea of rewarding those who do amazing things for myself and for the world. I like people out there knowing that if they produce amazing things for myself and for the world, they would get rewarded for them. I like the idea of not dying for as long as possible thanks to future developments in medical science. I like being a nation of laws, where the executive doesn’t just take stuff when he feels like it. And I’d like, when nice things are taken away and we mortgage our future, to at least get something out of the exchange.

Alas, the man in charge does not agree, and the government was not content with its previous efforts to sabotage the vaccination effort. That’s how it goes sometimes. You can’t always get what you want. Nor, when no one is given the incentive to produce what you need, are you likely to get that either.

Let’s run the numbers.

The Numbers Predictions

Prediction from last week: Positivity rate of 3.9% (down 0.5%) and deaths decline by 6%.

Result:

Nailed the positivity rate. Johns Hopkins has us down from 3.9% to an all-time low of 3.6%.  Deaths rising makes no physical sense and the move up doesn’t show up in the Wikipedia data, so this has to be a data fluctuation one way or another. I’m going to guess that it will revert.

Prediction for next week: Positivity rate of 3.5% (down 0.4%) and deaths decline by 7%.

Deaths DateWESTMIDWESTSOUTHNORTHEASTTOTALMar 25-Mar 311445976256412626247Apr 1-Apr 71098867178911604914Apr 8-Apr 1410701037162111454873Apr 15-Apr 21883987174711684785Apr 22-Apr 287521173160911104644Apr 29-May 5943122014409714574

The bump up in the West comes from California, which makes it harder to dig in deeply. The bump in the Midwest is more curious, but should reverse soon. Overall we see a disappointingly small decline, but still a decline, and it should pick up speed.

Cases DateWESTMIDWESTSOUTHNORTHEASTMar 18-Mar 2447,92172,81099,568127,421Mar 25-Mar 3149,66993,690102,134145,933Apr 1-Apr 752,891112,84898,390140,739Apr 8-Apr 1460,693124,161110,995137,213Apr 15-Apr 2154,778107,700110,160119,542Apr 22-Apr 2854,88788,97397,48278,442Apr 29-May 552,98478,77885,64168,299

Progress in the West remains slow, but improvement in all regions, with many states seeing large declines. We didn’t sustain the giant improvement rate in the Northeast but we still see pretty great improvement. This is what the endgame looks like.

India

Things continue to get worse in India, but the graph no longer looks as fully vertical as it did previously, so this continues to count as good news relative to the range of possible outcomes. If things peak not too long from now, it will still be the biggest disaster of the pandemic, but it won’t be anywhere near as bad as things could have gotten.

Vaccinations

We all know how it started.

As a reminder, we were once over 3 million doses, and we’re giving out more second doses now than we were then.

Every week, the graph of vaccinations looks more like the electoral college maps:

I found this chart enlightening:

The half-vaccinated people are taking precautions because they are much like the vaccinated people, but not yet vaccinated. The unvaccinated people are taking few if any precautions, as those numbers look a lot like what I’d expect from 2019, although I’d be curious to see those baseline numbers. But it’s clear data that the selection effect of who is willing to get vaccinated is dominating the behavioral modification effect from having been vaccinated.

This is presumably reducing the impact of the vaccinations quite a bit. If we presume that half-vaccinated represents the baseline behavior of the vaccinated, then they were previously taking something like a third the risk of those that are still unvaccianted, so 75% of risk was happening in the half or so of the adult population that isn’t vaccinated yet even before you adjust for mask use and distancing tactics, which should make that even more pronounced.

Despite that, it’s clear that the vaccinations are corresponding to very rapid declines in infection rates around the world, and I notice that I’m somewhat confused by how big this effect turned out to be.

Indian Strain Does Not Escape from Vaccines

The situation in India is terrible, but at least there is this bit of good news – the vaccines will continue to function, at least against the current strain:

Mutations not being additive seems like very reassuring news, implying that there could be a maximum amount of infectiousness or vaccine escape that a Covid-19-type thing is capable of easily achieving. I don’t see why we would stop using the term double mutant, but it makes it a lot less scary.

If there’s one place I’m most worried about engaging in motivated reasoning, it’s the possibility of vaccine escape. I notice a much larger flinch away from looking here than I do elsewhere. I think I’ve overcome that flinch, but I could be wrong about that, and it’s a super important thing to not make an effort to avoid seeing. So while I’m confident, I want to task my readers with keeping me honest on this one even more than usual.

P.1 Is The Medium-Term Infection

In many ways it is better to think of Covid-19 as a series of different infections from different variants. When the English strain shows up, it starts again from patient zero, starts again in each nation and region, and grows. When P.1 shows up and shows it is a more fit strain yet, it does this once again.

If you’re looking at the endgame scenario, the question is whether we’re seeing an increase or decrease in the most dangerous variant’s numbers in absolute terms rather than relative to the overall number of cases. Thus, in a place like New York, the ‘real’ graph of our future situation is the graph in P.1.

This is delayed due to how long it takes to do sequencing, but it looks like this:

Compare that to the graph of New York City’s cases, which looks like this:

Things had stabilized for P.1 by early April, when regular cases started cratering. Now, with regular cases declining even more rapidly in percentage terms, things are clearly improving even on the P.1 front, at least somewhat. We’ve passed the next test here, not only the previous one. As additional vaccinations come fully online, things will only improve, and I expect other areas to also hit this target.

The last month has been far more impressive than it has looked on its surface. We went from mostly the old strain to mostly new strains, and we are still steadily improving overall. The news really is quite good.

I worried last week that in relatively hesitant areas, we might run out of willing arms before we get to herd immunity. That is still a real worry, but I am not worried that large other areas won’t get to New York’s current effective immunity level given how many vaccinated people aren’t yet finished being vaccinated. That doesn’t allow a safe return to normal, but it does allow suppression when combined with moderate levels of precaution from the unvaccinated. My trip to New York this week revealed a city still taking its precautions deeply seriously, despite the majority of people being post-vaccination. I was clearly taking below average amounts of precaution, which was a new experience.

Exploring Vaccine Hesitancy

As a reminder, and to avoid any possible misunderstandings, as I keep saying week after week, the vaccines are very safe and super effective.

If you’re reading this, you almost certainly know this. If you’re reading this somewhere you can get vaccinated, and you haven’t done so yet, stop reading now, go get your first shot. We’ll wait.

Not everyone, unfortunately, is in your epistemic position. Thus, we have vaccine hesitancy.

What are the real reasons for vaccine hesitancy? There are lots of theories out there, and I’m confident someone cares about any given justification one could come up with, but what are the most common true objections?

There’s a lot of plausible candidates for the most common true objection.

A survey about vaccine hesitancy in the army has some good data on this, and is worth looking at in detail. I wish the data was better and came with numbers attached, but it’s still good to have a look at the slide of the Top 12 reasons soldiers are refusing vaccinations (it’s pasted here, but it’s a lot easier to read at the link.)

Or in written list form:

1. It’s not FDA approved.
2. It hasn’t been proven safe.
3. What’s the point? I’d still need to wear a mask.
4. This is the first time I get to tell the army NO!
5. I am not in a high-risk population.
7. The vaccine symptoms are worse than the virus.
8. The virus has the same morbidity rate then the flu.
9. I don’t want to get my family sick.
10. I am being safe. It has kept me healthy so far.
11. The vaccine may impact my pregnancy.
12. I just feel skeptical and don’t know what to believe.

It’s also worth taking in the perspective of the writer of the article and of the writer of the slide. Both writesr take it as common knowledge that the reasons to not take the virus are stupid and wrong, and that the job is to fix what’s wrong with these soldiers who are refusing.

There’s no acknowledgement that maybe we’ve messed up in how we handled this whole thing, or that some of the concerns might be reasonable, or that maybe we treat our enlisted soldiers like garbage or worse and they might really, really want to tell the army where to go. It’s a volunteer army, but the recruiter can lie to you, and once you sign the contract you definitely can’t quit.

Consider this whole thing, as I will do from here, from the perspective of the hesitant soldier.

There are a few categories of objections here.

The first category (1,2,9 and 11) are the straightforward safety concerns. These concerns are wrong, but I say that as someone who knows they are wrong. And the responses suggested here other than to #9 are… not great.

The FDA didn’t approve your energy drink? How is that relevant or in the appropriate reference class? If the vaccines have undergone such a rigorous process as you say, then why hasn’t the FDA approved them?

The clinical trials were three times as large as normal? How about the one hundred million Americans who got fully vaccinated? Maybe mention that? And again, what’s your answer to the obvious: If it’s so damn safe why hasn’t the FDA fully approved it?

There aren’t any obvious problems with pregnancy? Gee, mister, that makes me feel way better. No idea why we’re voluntarily going with this weaksauce over much stronger alternative arguments. If I’m listening for bullshit, guess what I’m thinking right now?

In related news, Stat News argues that the emergency use status of the vaccines shouldn’t interfere with vaccine mandates by employers and schools. As a matter of law I think they’re probably right (although of course I Am Not a Lawyer and all that) but as a matter of practicality this is a strong argument that it’s important that the FDA needs to issue a full approval. We’ve just had the biggest Phase 4 in history. Taking at least Pfizer and Moderna from ‘emergency’ use to full approval would do a lot to reduce hesitancy and free the hands of those who want to mandate vaccinations, without being coercive.

If you want to solve this issue, the FDA should simply approve the vaccines, full stop, not simply emergency use. Problem solved.

If not, the response to a soldier should be that the FDA are a bunch of ass-covering assholes who would prefer never to actually approve anything, and maybe that would get through to them in a language they can understand.

The second category (7, 8 and 10) are claims that Covid-19 isn’t that big a deal compared to the cost of getting the vaccine.

Here we see that response #10 says both “masks and social distancing work” and then goes straight to “but they don’t ‘directly combat’ the virus” implying they don’t count. When you’re lying about everything, it’s hard to keep your lies consistent, so I guess I’m somewhat sympathetic to this local predicament, but man it’s glaring.

The answer to #7 isn’t going to convince actual anyone. The ‘mild symptoms lasting 24-72 hours’ are exactly what the soldiers are complaining about, and the response is to tell them they’re imagining things, which they most definitely aren’t. Smooth.

For #8 they quote some statistics and it seems fine, I guess, although it leaves some ammo on the table. It’s kind of bending over backwards to be maximally generous to the flu’s deadliness. I’d have gone with different wording, but mostly this one is fine.

It’s interesting when they strengthen the answer to the point of deception, and when they weaken the response to the point where it doesn’t respond to the concern.

The third category (3, 5, 6) are claims that it’s not in the soldier’s personal interest to get vaccinated, because they’re young and healthy, as most active soldiers are, so why should they get sick for several days and maybe face risks they don’t know about? This also overlaps with 7.

The response to #6 isn’t an outright lie exactly, since the word ‘may’ does a lot of work. The sun might have just exploded. But in practice, yeah, this is lying.

The response to #3 is, and I quote, “F*** you.” If you all mostly comply, we’ll lift the outdoor mask mandate? That’s your pitch?

The response to #5 is, and I quote, “F*** you.” Or, technically, ‘it’s not about you.’ It completely accepts the (incorrect) premise that the soldier doesn’t benefit, which doesn’t seem like the approach I would take.

Then there are two standalones.

There’s the remarkable #4: This is the first time I get to tell the army, NO!

And oh my is the answer to that one “F*** you.”

Which leaves #12, which is the most interesting of the responses.

That’s because the soldier has spoken The Words, and has spoken them rightly.

Rather than voice a specific and explicit concrete objection, to which the answer of necessity is going to be some combination of ‘you’re wrong’ and ‘F*** you,’ the soldier has given a general feeling of uncertainty without any concrete objection. Thus, there’s no way to say they are wrong, and no basis to curse them out.

Instead, “I just feel skeptical and don’t know what to believe” elicits this response:

“The choice to get vaccinated is a personal decision and should not be taken lightly. Talk to a medical professional, consult the FDA Factsheet, and review the educational materials available at www.carson.army.mil and from the CDC to weigh risks and benefits.”

Suddenly we’re acting like this is a Very Reasonable and Responsible Position, which needs to be solved by consulting official sources and doing further research. Only after that, when the soldier comes back with an actual concern, can we know which of our two responses to use, and justify using it. I mean, there’s no way this person is skeptical after talking to all the Responsible Authority Figures, right?

NPR claims that lower rates of vaccinations among blacks and latinos are entirely due to accessibility issues and have nothing to do with hesitancy. I completely buy that the access issues are doing a lot of work here, but it seems odd to attempt to suddenly shift from “here are all the legitimate and sympathetic reasons why these groups would be hesitant” into “they are not and have never been hesitant, it’s that we didn’t give them access and made access depend on things that systematically excluded them.”

It’s a claim that we’ll be able to evaluate soon enough. As appointments become widely available via walk-ins in more places, with essentially no hoops involved, either the rates will converge or they won’t. I am skeptical because it seems like it’s a motivated shift in explanation rather than an attempt to track the truth – we want to make skepticism more blameworthy, so we need to not identify these increasingly blameworthy motives in the wrong places, hence the shift. I am only somewhat skeptical because it seems clear that providing easier access has a dramatic effect on vaccination rates.

Overall, that evidence means that the article seems like very good news. What it does make a strong case for is that there is a lot of ‘soft demand.’ The bad scenario for where we are would be that 60% of eligible people have already been vaccinated, and most of the remaining 40% are actively having none of it. They are like the soldiers. They won’t accept the shot unless convinced or heavily coerced.

Instead, this new picture finds evidence that what we have are a lot of people who prefer being vaccinated to not being vaccinated, but don’t prefer it enough to jump through a bunch of hoops. That’s great! All we have to do is get rid of the hoops and the need to jump through them, and offer them easy access. Now that we have abundant supply, that is relatively easy. Certainly I buy the anecdote that Asians have relatively low levels of hesitancy when given good access.

As someone who spent a substantial amount of time and effort to get vaccinated earlier rather than later, and to get those around him vaccinated earlier rather than later, I think those unwilling to do so are setting their price too low. We can separate this setting of a low-price into a few different components.

One explanation, which is the most hopeful with respect to the vaccines, is that their circumstances mean that paying the relevant costs is more expensive, and they have less ability to pay such costs. They care, but as the article claims, they are simply unable to take even a few hours off of work, or figure out how to navigate the barriers previously required. There is some of this, but we have some evidence that is then hard to explain if this is the main thing happening, such as the failure of J&J shots to rebound, and the distribution of shots on different days of the week.

J&J shots are going, well, not great:

If people simply cannot miss work, and are worried about side effects causing them to miss work in addition to the appointment itself, this suggests people will plan their shot around not missing work. That means getting a shot on Friday or Saturday, and yes we see giant spikes in shots given on Fridays and Saturdays, including during periods when supply constraints looked like they were binding. That seems like strong support. We’ll see if this can be sustained; if this theory is correct, Friday and Saturday throughput should continue to bind.

A second explanation is that this is shallow demand, pure and simple. If someone wouldn’t be willing to spend much time, let alone much money, to get a vaccine, that’s a revealed preference that they don’t value the vaccine much. This seems highly plausible to me, that there are essentially three camps rather than two camps. There’s the people who want the vaccine enough to ‘bid’ on it in various ways and make it a priority. There’s the people who actively don’t want the vaccine, often violently so. But then there’s also a large group, plausibly larger than the second group, who are fine with it but are mostly trying to live their lives and value the vaccine at some positive but small number.

I wonder how much of that is because we’ve set the price of the vaccine, and much of health care, to \$0, thus sending the implicit message that such services are, in emergencies, not that valuable. And also the general instinct to not think about one’s health when one isn’t forced to. We do seem to see a pattern of people who have the ability to get expensive medical care that they ‘should’ want, but not to spend small amounts of time (and aggravation) to collect it.

What’s The Worst Possible Thing You Could Do?

If you’re the President of the United States, in terms of actual impact the answer is presumably ‘launch all the nuclear warheads.’

If one restricts to the pandemic, the answer would be to sabotage vaccine production and distribution. Nothing else comes close. One could plausibly argue that nothing else even much matters.

How would one sabotage vaccine production and distribution?

Sabotaging distribution means doing things like not approving known-to-be-safe-and-effective vaccines, or suspending existing approvals and sending the message the vaccines are unsafe, or holding up distribution to worry about things like equity, or holding onto vaccine doses for extended periods with no intent of approving them ever.

Oh, wait. Those are all things done by the Federal Government during the Biden administration, with no visible attempt to prevent them from happening or even regret expressed about them. You could even add, during the campaign, questioning the vaccine development process as ‘rushed’ or ‘politically motivated,’ plausibly being the cause of vaccines not getting approved a month earlier and creating much additional vaccine hesitancy.

You’d also give doses to children who don’t need them rather than those in other countries that badly need them, so naturally Pfizer is on that one and soon will be applying for approval for children as young as two years old. And of course you’d continue not to do the first doses first, and continue to use full way-too-big doses of Moderna, and so on and so forth.

None of that means one couldn’t have done or in the future do more of those things, so actions haven’t been maximally destructive. But they’ve been quite destructive.

The other half of the worst thing you could do is sabotaging production. The easy way to do this is to screw up distribution. If things aren’t approved yet, at best then that’s going to slow down production until after approval. So are all the regulations involved in production, like needing to apply for permission and wait substantial time for permission for things like ‘put more of the vaccine into each vial because we’re short on vials.’

That’s all passive resistance to lifesaving medicine. Could we kick this up a notch or two?

The ultimate way to hurt vaccine production, not only now but indefinitely into the future, would of course be to destroy the financial incentive to produce vaccines. The less you’re willing to pay, and the less you let companies profit, and the less you reward those companies for quick scaling up and delivery of production, the less doses you’ll get. This starts with not paying for building production capacity, and its central action is not paying much per dose or paying more for early delivery. If you want to go for bonus points, you can be like Europe and hold up negotiations for weeks to drive down the price even lower.

That’s all negative actions, though. It’s easy to sabotage efforts by not doing the right thing, especially when the right thing costs tiny amounts of money and looks like rewarding corporations, and is an action rather than inaction and thus blameworthy.

So it’s a big step-up in the civilizational sabotage game to actively take away the incentive to create vaccines, by stripping away intellectual property protections without any compensation, in the middle of a pandemic:

There’s a simple solution to the problem of intellectual property if you wanted to make the situation better rather than worse. You could buy the intellectual property rights from the companies involved. So, basically, this:

It’s not that much money, everyone would be happy, and the precedent would be excellent. Pay enough, and they’ll even aid you in technology transfers. Even better, you could repeat this process with other drugs. Buy out the monopoly at its economic value, remove protections, and the people save many times that much money in costs. It’s a great idea.

Doing this without compensation is about the worst thing one could do. If your new ideas outright save the world, we’re going to reward you by confiscating them, voiding the contracts and promises agreed upon and informing you that we are not a nation of laws. That’s exactly how not to get vaccines next time there’s a crisis, or anything else next time there’s a crisis, or really anything else useful at any time for any reason.

The message we’ve sent, loud and clear, is that we are not a nation of laws and we do not reward those who deliver the goods for us. Instead, we retain protections on things like insulin that are pure rent seeking, while taking away protections that are doing exactly what patents are designed to do: reward those who produce world-changing positive innovations via temporary ability to profit.

We are a nation of a person in charge, and if that person decides to confiscate your property because it’s good politics, well, tough.

It’s a horrible, horrible precedent. We will pay for it in money, will pay for it with our freedom, and we will ultimately pay for it in blood.

mRNA vaccine technology is potentially a full cure for infectious disease, and plausibly also a cure for cancer. The federal government sabotaged all that, big time.

What did we get in exchange? What’s in the box?

NOTHING!

Unless, of course, they are not so stupid In which case the destruction of the rule of law and of private incentives, and the signaling that political expediency is the most important thing, was the point.

You see, this will not increase vaccine production (MR link with full explanation, recommended), for two reasons, even if vaccine ingredients didn’t prove to be limiting factors. MR also recommends this Barron’s column. Here’s another confirmation that no, this won’t improve short term supply.

Many people have this idea that all the knowledge and skill required to produce the vaccines lies in the patents. Once you lift the patents, lots of other companies can go start producing vaccines. Except, that’s not actually true because

1. The vaccines require technical expertise not included in the patents, which is expensive and slow to transfer, and which would also transfer valuable knowledge that can be used for other R&D and other production and thus which the vaccine producers are not going to transfer without compensation.
2. Moderna explicitly already said they wouldn’t enforce the patents, and no one really expected the others to either.

Read that second one again, if it’s new to you. The greedy capitalists whose rights you took away without compensation were already voluntarily giving those rights away. If there was already clearly no intent to enforce the patents, what good does lifting those patents do?

It sends the message that the United States is willing to confiscate property for political gain, when it feels like it, on the basis of the executive’s say so.

Even though that won’t produce anything useful, yes, it’s still bad for business and still punishes exactly who we should be rewarding, or at least demonstrates that such punishments should be expected, as measured by the stock market. Remember Moderna already waived its rights:

As usual, the usual suspects wasted actual zero time demonstrating exactly the slipperiness of the associates slopes, as they quote the decline in shareholder value as a good thing:

What makes such a statement so maddening is that she’s right. We should totally do insulin! It’s completely insane that we’ve allowed regulatory capture and rent seeking via intellectual property protections on “inventions” like insulin. The congress should get together, write a bill and pass a law that stops such things from happening now or in the future via changing protections, ideally without confiscating private property, and then the President should sign it, and then the bill should become law. Then do copyright.

Won’t Someone Please Think of the Children?

The minds of many parents I know are turning to the question of summer camp. Is it safe to send your young child?

Are all the people you care about that will be in contact with that child either other young children or fully vaccinated by the time the camp starts?

If the answer to that question is yes, then yes.

If the answer to that question is no, then given that vaccinations are now available to everyone pretty much on demand, why isn’t the answer yes?

If the answer to that question is that someone is seriously immunocompromised, or otherwise super important to the child’s life and won’t get vaccinated (for whatever reason), then and only then is it time to look at the camp’s procedures to see whether you’re comfortable with the level of risk being taken. In particular, you’ll need to ask how many children and unvaccinated adults will be in contact with your child, how close that contact will be, and how much time will be spent indoors, and do a calculation.

I still think that calculation should almost certainly be ‘yeah, it’s fine’ but at that point, as they say in the advertising business, it’s up to you.

My general answers regarding children generalize this. Young children are not at enough risk from Covid to let this change how they live their lives, so them catching it only matters to the extent that they would pass it on to vulnerable others.

By the end of May, with notably rare exceptions, patience with those in the United States who are still vulnerable can reasonably be at an end. Those who decline the opportunity to be vaccinated can manage their risk however they choose, but life beckons.

Speaking of life beckoning: I strive not to use the word evil, I avoided using it in the previous section, but this is evil in its purest form:

Anyone who doesn’t recognize this as such has lost their soul. Any parent or teacher who enforces this should be treated as the mustache-twirling villain they are. I am deeply sorry to any child who has been so absurd and tortured, or living in so much fear, that they are tempted to put up with this.

If you do not think school’s primary nature is ‘child prison’ and/or that those running it are pro-children, then you have new data your model needs to somehow explain.

In Other News

V-NY day approaches, and Cuomo fully opens up stadiums, opens up Broadway, offers subsidized vaccinations at Mets and Yankees games. Took everyone on Broadway by surprise, so it’ll be a while before they can actually get on with the show. Remember, you’ll want to wait to get vaccinated until you attend a game at the stadium, together with tens of thousands of other people. That way you get free tickets!

In many cases, Walmart too. Basically everywhere at this point. No excuses!

South Korea says AstraZeneca shot 87% effective after one dose. Which would be pretty good after two doses. First doses first, indeed.

Police have low rates of vaccination, endangering those around them who they forcibly interact with and likely killing them (WaPo), but no one is able to make them do the right thing and stop endangering the public. A little on the nose, if you ask me.

Airline boarding procedures were already worse than random, and changes in response to the pandemic made them worse still. It seems that looking like a good procedure is more valued than being an actually good procedure. There seems to be a strong match between ‘this is a quick boarding procedure’ and ‘this is a safe procedure,’ so the problem is purely that good procedures don’t look good and/or don’t feel ‘fair’ somehow, or miss out on some opportunity for price discrimination. Is there an improvement that would also look and feel like one?

MIT requires vaccinations, although so far only for students. I expect most colleges to follow suit if only to avoid potential liability concerns. Not spreading the requirement to faculty and staff seems like a clear mistake.

New higher estimate of true number of Covid deaths via MR, not enough data to know how much credit to give this.

The Covid Response Project chronicles the Covid-related experiences of people across different states. I’ve sampled and it seems like a good source of real people’s anecdata. There will definitely be surprises.

Twitter thread and paper discussing origin of variants of concern. Not sure there’s practical updates to be had, but interesting information.

Pfizer begins shipping some vaccine doses manufactured in the United States abroad, starting with Mexico.

Potential universal coronavirus vaccine proposal. From what I can tell this is highly unlikely to work but you never know.

Vaccination availability site of the week, Vaccinate the States

Marginal Revolution points us to a study of future work-from-home (WFH) patterns (paper), and finds dramatic effects the study expects to linger beyond the pandemic. I hope to check this out in detail in the future, but the headline impacts are gigantic. They expect WFH to go from 5% of full workdays to 20%, and for this to be a 5% productivity boost, most of which will be due to reduced commuting. Commuting is much worse than people think it is, so this is a really, dramatically large effect, in the range of ‘potentially a bigger long term deal than the pandemic.’ This isn’t a fake productivity boost, it’s literally getting rid of purely wasted unpleasant time (that also burns a bunch of carbon to boot). Given the amount of time being saved, it also implies that on the margin there’s still going to be a dramatic underutilization of WFH as an option. If a change to 15% of the workforce produces a 5% productivity boost by saving useless time (and it’s still an if, the story has to check out), clearly we are not using anywhere near enough of it.

Not Covid, therefore… we’re coming back, baby! HYPE!

Discuss