# Новости LessWrong.com

A community blog devoted to refining the art of rationality
Обновлено: 24 минуты 12 секунд назад

1 мая, 2019 - 18:43
Published on May 1, 2019 3:43 PM UTC

If it’s worth saying, but not worth its own post, you can put it here.

Also, if you are new to LessWrong and want to introduce yourself, this is the place to do it. Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are welcome. If you want to explore the community more, I recommend reading the Library, checking recent Curated posts, and seeing if there are any meetups in your area.

The Open Thread sequence is here.

Discuss

### Nash equilibriums can be arbitrarily bad

1 мая, 2019 - 17:58
Published on May 1, 2019 2:58 PM UTC

Consider the following game, called "Almost Free Lunches". You name any dollar-and-cent amount between £0 and £1,000,000.00; your opponent does likewise. Then you will both get whichever amount named was lowest.

On top of that, the person who named the highest amount must give £0.02 to the other. If you tie, no extra money changes hands.

What's the Nash equilibrium of this game? Well:

• The only Nash equilibrium of Almost Free Lunches is for both of you to name £0.00.

Proof: Suppose player A has a probability distribution pA over possible amounts to name, and player B has a probability distribution pB over possible amounts. Let mA be the highest amount such that pA(mA) is non-zero; let mB be the same, for B. Assume that (pA,pB) is a Nash equilibrium.

Assume further that mA≥mB (if that's not the case, then just switch the labels A and B). Then either ">mA> £0.00 or mA= £0.00 (and hence both players select £0.00).

We'll now rule out ">mA> £0.00. If ">mB> £0.00, then player A can improve their score by replacing mA with m′A=mB−£0.01. To see this, assume that player B has said nB, and player A has said mA. If nB<m′A<mA, then player A can say m′A just as well as mA - either choice gives them the same amount (namely, nB− £0.02).

There remain two other cases. If nB=m′A, then m′A is superior to mA, getting m′A (rather than m′A− £0.02). And if nB=mB, then m′A gets m′A+ £0.02 =mB+ £0.01, rather than mB (if mA=mB) or mB−£0.02 (if m_B">mA>mB).

Finally, if mB= £0.00, then player A gets -£0.02 unless they also say £0.00.

Hence if ">mA> £0.00, the pA cannot be part of Nash Equilibrium. Thus mA= £0.00 and hence the only Nash Equilibrium is at both players saying £0.00.

Minmax and maximin

The minmax and maximin values are also both terrible, and also equal to £0.00.

This shows that choosing the Nash Equilibrium can be worse than almost every other option. We can of course increase the maximal amount, and get the Nash Equilibrium to be arbitrarily worse than any reasonable solution (I would just say either £1,000,000.00 or £999,999.99, and leave it at that).

But we can also make the Nash Equilibrium arbitrarily close to the worst possible outcome, and that without even requiring more than three options for each player.

Assume that there are four ordered amounts of money/utility: n_2 > n_1 > n_0">n3>n2>n1>n0. Each player can name n2 or n1. Then if they both name the same, they get that amount of utility. If they name different ones, then then player naming n2 gets n0, and the player naming n1 gets n3.

By the same argument as above, the only Nash equilibrium is for both to name n1. The maximum possible amount is n3; the maximum they can get if they both coordinate is n2, the Nash equilibrium is n1, and the worst option is n0. We can set n1=n0+ϵ and n3=n2+ϵ for arbitrarily tiny 0">ϵ>0, while setting n2 to be larger than n1 by some arbitrarily high amount.

So the situation is as bad as it could possibly be.

Note that this is a variant of the prisoner's dilemma with different numbers. You could describe it as "Your companion goes to a hideous jail if and only if you defect (and vice versa). Those that don't defect will also get a dust speck in their eye."

Discuss

1 мая, 2019 - 17:43
Published on May 1, 2019 2:43 PM UTC

Discuss

### Has government or industry had greater past success in maintaining really powerful technological secrets?

1 мая, 2019 - 05:24
Published on May 1, 2019 2:24 AM UTC

Context: As part of our efforts of working on Open Questions the LessWrong team has been reaching out to various researchers we know and asked them about questions they would be interested in getting answered.

This question was given to us by Ryan Carey and some of the answers below are the result of us trying to answer the question for a day on a private LessWrong instance, copied over to allow other people to contribute and read what we wrote.

Discuss

### A quick map of consciousness

1 мая, 2019 - 05:17
Published on May 1, 2019 2:17 AM UTC

Original post: http://bearlamp.com.au/a-quick-map-of-consciousness/

Prior knowledge: Many maps lightly held, Leaky concepts, Boundaries

Map and territory: mind to reality – To be presented alongside the caveat, “what is good?”

(Well “good” is in the map, not the territory.  This diagram very quickly becomes a mess, but before that happens, let’s talk about reifying the parts of this model to see if it’s useful)

To me right now, it seems like consciousness is the ladder between the map and the territory.  In the diagram, on the left is a thought, suggesting that “this is an apple” on the right, pictured is a red apple.  When the attention points at a red apple, the consciousness is filled with a map of declarative definition that labels, names and concludes that this is an apple.

Consciousness seems to be a label generating machine.  Something fundamental about brains is that they map the territory.  They quest towards mapping the territory.

That’s.Just.What.They.Do.

This brings us to the question of – how do I have a good life.  I have 3 strategies:

1. [content] Look at different apples

2. [map] modify so that there are more positive opinions of apples

3. [relationality] appreciate looking at rotten apples if that’s what’s to look at today.

Content

If I look at dead apples all day, I’m not going to auto-magically have a great day.  On the other hand if I look at great apples, I’m going to be impressed and delighted.  The apple could be replaced with beautiful artwork, nice sunsets, tasty food, nice music.  Whatever strikes in the heart of desire to be attended to. Improve the content is a reasonable and helpful strategy sometimes.

Sometimes it’s not the content that’s the problem.  Maybe there’s nothing wrong with apples but they make me puke.  Then I can try the map.

Map

If every time I see an apple I remember that one time I bit an apple and found half a worm, maybe there’s some work I can do so that I don’t keep thinking worms when I see an apple.  Even sunsets are irrelevant when I’m too busy on my phone. If art galleries remind me of my ex, music reminds me of screeching cats (not in a good way), food reminds me of how fat I am (and how I can’t take care of my body). Maybe the work to be done is in the map.  Sometimes with more and less force, the map can be trained to be less miserable when presented with stimuli. Usually the good stuff is found by passing through the uncomfortable, not avoiding it.

Sometimes I can’t shift the content.  I’m living in the developing world, sometimes sickness and suffering is visible.  Sometimes it’s a very real awareness that if I’m not careful it could be me. That’s where the 3rd method comes in.

There’s parts of the map that start to relate to other parts of the map. That’s what I start to call “relationality”.

Relationality

I look at an apple.  It reminds me of the time I bit into a worm.  How I relate to that content is flexible. I can feel bad about being dumb that time, or I can look at it and laugh about how ridiculous that was. Maybe thinking of worm-apple-gate is my minds way of warning me to be careful it doesn’t happen again.  That time I went to see the sunset and could not get off my phone, I was upset about something, maybe I’m being reminded to be kind to myself, now I know better. Screeching cats – Hilarious! Food makes me fat, but it’s really really good food.  So tasty! Maybe the question of balancing good food and living!life is worth considering.

I have a chance to see how I’m relating to the content, and I can travel to different maps.

How?  Slowly.

That process of “travel to different maps” needs to be done in the way of being that travels all the way down the ladder.  If I brute force the attention to move elsewhere, my relationality is “brute force”. My map says, “I gotta brute force my way around here” or “that’s not important” and my content becomes all about the things I avoid.  Sure I can brute force my content to be butterflies not machine guns, but that’s not going to substantially change a map with trouble brewing. I can’t always control what I see. but I can work towards relating to those experiences better.

This post has been quick and dirty. I hope to build on it later.

Discuss

### Natural Structures and Definitions

1 мая, 2019 - 03:05
Published on May 1, 2019 12:05 AM UTC

There's a sense in which definitions are arbitrary. Words are made by humans and no-one can stop me from calling red blue and blue red if I really want to. So when people ask questions like, "What is consciousness?" or "What is free-will?", it seems quite reasonable to respond, "Just pick a definition. These terms can be defined many different ways and it's completely your choice which one you choose to use".

This may appear to dissolve the question, however, I would suggest that such an answer often misunderstands what the asker is attempting. Typically the asker is concerned by more than the linguistic question, but instead with attempting to understand the ontology or structure of reality. And it may be the case that this structure includes a substructure that naturally fits with our intuitions of what consciousness is or what freewill is or it may be the case (as per the standard LW view) that such a structure doesn't exist.

What makes this especially confusing is that many people will conditionally accept the "It's arbitrary" in those cases where they are convinced that such a natural doesn't exist, while pointing out the natural structure otherwise. Here's an example. Let's suppose it was common knowledge that we all have souls. Then whenever someone asked about the definition of consciousness, we'd be tempted to point to the soul, just as whenever people ask about the definition of trees, we'd be tempted to talk about leaves and branches. The arguments for being able to use language arbitrarily remain, it's just that one definition suffices for 95% of cases, so we don't bring up that argument.

Appendix

Here are some possible interpretations of, "What is X?":

• What does term X intrinsically mean? (no intrinsic meaning exists)
• What natural structure (if any) corresponds to X?
• What are some useful interpretations of the term X?
• How is the term X used in society?

These kinds of discussions tend to work better if everyone is on the same page about what is being asked.

Discuss

### Site will be down for 10 minutes at 1:25PM PST: Testing out some new deployment infrastructure

30 апреля, 2019 - 23:07
Published on April 30, 2019 8:07 PM UTC

After we had a bug yesterday that caused the site to be sluggish and a bit broken for an hour or so, we decided to improve our deployment infrastructure a bit. This is a test run of the new infrastructure which will cause the site to be down for something like 5-15 minutes. Sorry for the inconvenience.

Discuss

### Change A View: An interesting online community

30 апреля, 2019 - 21:34
https://changeaview.com/static/d4f67fa49ebd3e36059288668f361097.png

### LW is behaving wonkily, apologies

30 апреля, 2019 - 02:23
Published on April 29, 2019 11:23 PM UTC

We're diagnosing a problem wherein LessWrong is slow (in particular for logged in users – it loads much faster in incognito windows), and some comments are double-posted.

Hopefully will be fixed by end of day, just letting people know that we're working on it.

Discuss

### Pecking Order and Flight Leadership

29 апреля, 2019 - 23:30
Published on April 29, 2019 8:30 PM UTC

It was recently pointed out to me that humans are weird, compared to other social animals, in that we conflate the pecking order with the group decision-making process.

The pecking order, for instance in birds, is literally the ranking of who gets to eat first when food is scarce.

We can also call it a “dominance hierarchy”, but the words “dominance” and “hierarchy” call up associations with human governance systems like aristocracy and monarchy, where the king or chief is both the decisionmaker for the group and the person entitled to the most abundant resources.

In birds, it’s not like that. Being top chicken doesn’t come with the job of “leading” the other chickens anywhere; it just entitles you to eat better (or have better access to other desirable resources).  In fact, group decisionmaking (like deciding when and where to migrate) does occur in birds, but not necessarily according to the “pecking order”.  Leadership (setting the direction of the group) and dominance (being high in the pecking order) are completely independent in pigeons, for instance.  Pigeons have stable, transitive hierarchies of flight leadership, and they have stable pecking order hierarchies, and these hierarchies do not correlate.

Logically, it isn’t necessary for the individual who decides what others shall do to also be the individual who gets the most goodies.  They can be related — one of the things you can do with the power to give instructions is to instruct others to give you more goodies. But you can, at least with nonhuman animals, separate pecking-order hierarchies from decision-making hierarchies.

You can even set this up as a 2×2:

High rank in pecking order, high decision-making power: Lord

High rank in pecking order, low decision-making power: Eloi

Low rank in pecking order, high decision-making power: Morlock

Low rank in pecking order, low decision-making power: Vassal

“Eloi” and “Morlocks” are, of course, borrowed from H.G. Wells’ The Time Machine, which depicted a human species divided between the privileged, childlike Eloi, and the monstrous underground Morlocks, who farm them for food.  Eloi enjoy but don’t decide; Morlocks decide but don’t enjoy.

The other archetypal example of someone with low rank in the pecking order but high decision-making power is the prophet. Biblical prophets told people what to do — they could even give instructions to the king — but they did not enjoy positions of privilege, palaces, many wives, hereditary lands, or anything like that.  They did sometimes have the power to threaten or punish, which is a sort of “executive” power, but not the power to personally enjoy more resources than others.

In American common parlance, “leadership” or “dominance” generally means both being at the top of a pecking order and being a decision-maker for the group.  My intuition and experience says that if somebody wants to be the decision-maker for the group but doesn’t seem to be conspicuously seeking & enjoying goodies in zero-sum contexts — in other words, if somebody behaves like a Morlock or prophet — they will read as not behaving like a “leader”, and will fail to get a certain kind of emotional trust and buy-in and active participation from others.

My previous post on hierarchy conflated pecking-order hierarchies with decision-making hierarchies. I said that people-telling-others-what-to-do (decision-making hierarchy) “usually goes along with” special privileges or luxuries for the superiors (pecking-order hierarchy.)  But, in fact, they are different things, and the distinction matters.

Most of the practical advantages of hierarchy in organizations come from decision-making hierarchy.  A tree structure, or chain of command, helps get decisions made more efficiently than many-to-many deliberative assemblies.  Many of the inefficiencies of hierarchy in organizations (expensive displays of deference, poor communication across power distance) are more about pecking-order hierarchy.  “So just have decision-making hierarchy without pecking-order hierarchy!” But that’s rule-by-prophets, and in practice people seem to HATE prophets.

The other model for leadership is the “good king”, of the kind that Siderea writes about in this series of posts on Watership Down.  The good king is not just sitting on top of the pecking order enjoying luxury at the expense of his people. He listens to his people and empowers them to do their best; he shares their privations; he is genuinely committed to the common good. But he’s still a king, not a prophet. (In Watership Down, there actually is a prophet — Fiver — and Hazel, the king, is notable for listening to Fiver, while bad leaders ignore their prophets.)

My guess is that the “good king” does sit on top of a pecking-order hierarchy, but a very mild and public-spirited one.  He’s generous, as opposed to greedy; but generosity implies that he could be greedy if he wanted to. He shares credit with others who do good work, instead of hogging all the credit for himself; but being the one to give credit itself makes him seem central and powerful.

A “good king” seems more emotionally sustainable for humans than just having a “prophet”, but it could be that there’s a way to implement pigeon-like parallel hierarchies for resource-enjoyment and decision-making, or other structures I haven’t thought of yet.

Discuss

### Reference request: human as "backup"

29 апреля, 2019 - 20:13
Published on April 29, 2019 5:13 PM UTC

I remember reading some paper that suggests humans to take a specialized role inside super AI, like mitochondria, or like a biological "backup". The argument was that humans might have specialized functions that they can do well, like mitochondria is good at making energy. And humans might be good at surviving, so might be a good "biological backup".

I feel like it's by Vernor Vinge, but I just checked and he didn't write that. I googled around and couldn't find it either. Can you find from which paper this might have come from?

Discuss

29 апреля, 2019 - 18:51
Published on April 29, 2019 3:51 PM UTC

Cross-posted from Putanumonit.

Follow-up to Shopping for Happiness.

My old Galaxy smartphone recently gave up the ghost, and I upgraded to the new model for $750. My friend was surprised when I told him. The old model is now available for$250, is the new one really three times better?

“Three times” better can mean several things, but in my post on spending money wisely I came up with the metric that should guide purchasing decisions: happiness gained per unit of time spent experiencing a thing, or :-)/hr. By this metric, since the new phone costs 3x as much, unless it provides 3x the :-)/hr it’s worse in terms of $/:-). That means I’m getting less happiness per dollar spent. I like my new phone a lot: the screen is bigger, the battery lasts all day and night, I can use it for blogging. It brings me at least 25% more :-) than the old phone. But, it doesn’t make me 200% happier. And yet I feel like I’m getting a good deal. When my friend asked how I would justify this decision I warned him not to trust my explanation – since I already bought the phone, any justification may just be a post hoc rationalization. That caveat aside, my justification is that the price of the phone is a red herring. What I really care about is the value. Ask yourself: how much would you be willing to pay for your smartphone if it was the only model available for sale? Whether they “ruined a generation” or not, but I think that smartphones are awesome and immensely improve my life. If I had to choose between no phone at all or a Galaxy smartphone, I’d pay at least$4,000 for the old model and $5,000 (25% more) for the new one. That means I’d be willing to pay$1,000 more for the upgrade, and they only charge me $500 more ($750 vs. $250) for it. The fact that smartphones cost less than what I’m willing to pay is just a wonderful bonus born of engineering ingenuity and market competition. I square this with the$/:-) disparity by noting that my goal is to maximize :-) over all the money I spend, not in each category separately.

Consider a toy example of a world in which only two product categories exist: jackets and smartphones. You have $1,000 to spend, and four products to choose from: 1. Galaxy S7,$250, 4,000 :-)
2. Galaxy Note 9, $750, 5,000 :-) 3. Regular jacket,$250, 500 :-)
4. Fucking jacket, $750, 1,000 :-) Given the constraint that you can only enjoy one smartphone at a time, the most happiness is bought by purchasing the Galaxy Note 9 and a regular jacket – 5,500 :-). The S7 does better in terms of$/:-) but it doesn’t leave you with great options for the remaining $750. It’s better to spend more on categories of products where you get a lot of :-) and spend the minimum in low-value categories instead of looking to optimize within each category separately. This example is not too far from the real case for me. The fact that I’d be willing to pay ~10x the asking price for a smartphone is a sign that smartphones (along with soap, tea, and underwear) are high-value categories. If I have enough money to buy great things in each of those I should do that before looking elsewhere. For things that I value little compared to their average price (cars, jewelry, whiskey) even a good within-category deal is a bad deal overall. Decomposing Value One more thing to consider is that the value of a purchase is made up of several factors. Marketing theory usually breaks those into four types: 1. Functional value – the direct use of a thing, the problem it solves. The value of a spoon is mostly functional. 2. Social value – the connection with other people and the signaling value of the thing. The value of a college degree is mostly social. 3. Psychological value – the happiness resulting from merely having the thing. The value of a framed family photo is mostly psychological. 4. Monetary value – the financial benefit generated by owning or reselling the thing. The value of stocks and bonds is mostly monetary. Companies use this breakdown to market product to consumers, but as a consumer, you can flip this around to figure out what you’re looking for and how much it’s worth. For example, I can decompose the value of a tailored suit: 1.$50 of functional value – keeping me warm and not-naked.
2. $2,000 of social value – a requirement for certain jobs and social events. 3.$300 of psychological value – I feel like I look good in it.
4. 0 monetary value – no one is going to pay much for a suit tailored to someone else.

This means that I’d be willing to pay up to $2,350 for a good suit (thankfully, I can find one for a fraction of that price), but I won’t pay much extra for a suit that is slightly better looking for me or does a better job of keeping me warm – most of the suit’s value is social. In rich countries, people tend to spend a lot more money on social value than on other kinds. Making me warm in New York costs a lot less than making me cool. For an opposite example, I almost always buy the cheapest overnight airline tickets I can find, even if an airline with great food and service costs only 10% more. The value of a plane ticket to me is almost purely functional – getting me to another city. I don’t care to pay more than$15 for an in-flight meal, let alone hundreds of dollars for business class or more polite flight attendants.

Advanced Putanumonit users don’t have to limit themselves to the four types of value described above. You can goal factor any purchase and break it down to your personalized components of value. And after doing this exercise, truly advanced and enlightened users may decide not to spend their money on anything except peanut butter. Unfortunately, I’m not there yet.

Discuss

### My poorly done attempt of formulating a game with incomplete information.

29 апреля, 2019 - 07:36
Published on April 29, 2019 4:36 AM UTC

Epistemic Status: I was bored so I wrote this, forgetting that I was probably making a poor attempt of reinventing some wheel, and realising how I forgot to solve

Motivation: To formulate the stereotypical “Obedience Test” in game theory terms.

The idea came into my mind when I saw the “Three CIA Candidates” joke:

The goal of the test was stated as “To know that you will follow instructions, no matter what.”, and apparently the sure-fire way to know this is to let the subject make a choice of harming themselves (or troubling their conscience) or to disobey command. Obedience is taken as the willingness to put oneself and their own conscience under harm in order to obey the command.

And the command in the obedience test probably serves little or no higher purpose other than being a test, and would probably appear so. (Arguably, if the subject believes the test serves a higher purpose, the instructor may still need to find out what happens when him and the purpose contradicts each other)

This pattern seems sensible, although maybe not logically vigorous.

I can think of a few other examples with similar logic:

God tested whether Abraham would be willing to sacrifice his beloved son Isaac, and when Abraham was about to kill Isaac the Angel of the Lord (to God’s credit compared to the other examples) stopped Abraham and acknowledged that Abraham showed his fear towards God.

Ancient Chinese general Wu Qi allegedly (this incident’s historicity was under question) murdered his wife, who was from the State of Qi (the enemy of the State of Lu, which Wu Qi was serving under), as the Lu leadership was hesitant to assign Wu Qi as an army commander concerning his loyalty.

Tokugawa Ieyasu executed his own wife Lady Tsukiyama and forced his first son Nobuyasu to suicide, when his ally/master Oda Nobunaga showed allegations that Tsukiyama conspired against the Oda Clan.

And I guess it is the same logic that lies behind gang initiation murders and the same logic that makes fraternity/sorority hazing rituals dangerous.

Now let’s try to formulate the process:

(This game has incomplete information, and so I haven't solved it just yet)

We will have a “monarch” M, who gives commands to the “subject” S, and M would like to figure out how “obedient” (denoted by b, a positive real number) was to him/her. M chooses a test with “repulsiveness” (denoted by r, a positive real number), and S can choose to either pass or fail the test, with payoff to S as follows:

Pass: -r

Fail: -b

Intuitively, if S finds the test’s repulsiveness worse than disobeying the command, S fails the test, if S finds the test less repulsive than disobeying the command, S obeys and passes the test.

Hence in the naïve first round, S passing the test would mean that b is at least as large as r, which assures M that S is at least somewhat obedient.

However, there is probably little purpose knowing whether S is obedient or not unless M wants to assign S some job to do.

The second round: the job, tax rate, and defection.

Assuming S is an extremely capable subject and the only reason why S might fail a job is disobedience/disloyalty to M.

Define a “tax rate” t as a real number between 0 and 1, and t is known to both M and S before the start of the game.

M will choose a job with value v, a positive real number, and assign the job to S.

S can choose to stay loyal or defect as response to the job, with payoffs as following:

(ie: M and S divides the job’s value according to the pre-defined tax rate)

(ie: S defects and receives all utilities from the job, but suffers from the consequences from disobeying M)

In the second round, S will defect when tv>b (when the share of job value M took away outweighs S’s loyalty to M), and M aims for tv=b (when S’s obedience is fully utilised), and since M does not know b, this has to be inferred from the first round.

Putting the two rounds together, the game can go three ways (ignoring the possibility that S fails the test yet M still assigns S a job with only negative knowledge about S’s obedience)

Assume that t and b are pre-defined, t is known to both M and S, b is only known to S.

1. M chooses r, S fails test. Payoffs: M:0, S:-b

2. M chooses r, S passes test.

2a. M chooses v, S stays loyal. Payoffs: M: tv, S: (1-t)v-r

2b. M chooses v, S defects. Payoffs: M: -tv, S: v-b-r

Now since the second round will yield positive utility to S, expecting this means S will choose to pass the test even if r>b.

In the first round S does not know what value v will be, but we can still work something out on what would happen in this case:

If S chooses to pass the test, it means the expected v is high enough such that either

(v-r-b)>-b or (1-t)v-r>-b has to be true.

When r<b, (1-t)v-r>-b is always true since (1-t)v>=0

When r>b this becomes somewhat different:

Either (1-t)v>r-b (loyal), or v-r>0 (defect)

By my rather problematic construction of the game, the expected payoff of defection vs failing the test is only dependent on v and r, not b. This means if S expects a valuable job following the test, passing it and then defect when given the job is always better than failing the test. I’m not sure if this makes sense, although I suspect it might be the logic behind the trope, and perhaps it is a routine for undercover agents (who basically have 0 loyalty whatsoever)

On a second thought, since M does not have any information whatsoever before S passes or fails the test, one of r or v has to be exogenous. While in the formulation of the game, it appears that r is exogenous (since M has no information in the first round), I think it is more realistic to assume that v is exogenous.

M knows v at the beginning of the game, and S does not know v until the start of the second round.

S knows b, which M will never know for sure.

Now M’s goal becomes: set r (ranging from excessive drinking to kick a dog to shoot your spouse), such that if S passes the test, the expected utility of M of passing v to S is positive. (In this simple formulation, it means M believes tv<b is more likely than not)

Yet I still haven't figured out how to solve this game yet, good on me.

(Afterthought: I think this is definitely a rather primitive attempt in making a principal-agent game, and it is probably reinventing a wheel somewhere, not exactly the "screening game" but maybe something close. And my lack of studying in game theory specifics is showing, so despite my best efforts I cannot find any clue for solving this game)

I guess that will be the end of my struggle into game theory today, until I got enough time to revisit the textbooks.

Discuss

29 апреля, 2019 - 00:29
Published on April 28, 2019 9:29 PM UTC

Originally posted at https://www.threemonkeymind.com/wgir/mario-kart/.

“Rationality techiques helped me to become less bad at Mario Kart” isn’t the most compelling elevator pitch for reading The Sequences, but it illustrates the usefulness of reevaluating the things you think you know every so often, especially when you’re making decisions based on the things you think you know.

Rationality gives you a mental toolkit to help you update your beliefs based on evidence. This is not always easy. Orwell once remarked that to see what is in front of one’s nose needs a constant struggle and it’s useful to have mental techniques, habits, and sayings to help you model the world as best as you can.

I recently got back into playing Mario Kart 8 on my own Switch. This wasn’t my first time playing Mario Kart, though. Back in the day, I managed to get first place in all Mario Kart cups at the highest difficulty. I’d played about three hours of Double Dash, and I’d played a bit of Mario Kart 8 at a friend’s house. When my friends play multiplayer, they all race at 200cc, so I tried to pick a character and loadout that was up to the task[1]. Because I didn’t know any better, I opted for Yoshi in the Standard Kart, Standard tires, and the Super Glider. I remembered reading that bikes were “more advanced”, so I didn’t try them initially. After a couple rounds of doing abysmally poorly, I tried one of the bikes thinking they’d be, at the very least, very different. After all, the motorcycle was very different from the cars in Stunt Race FX. I was right — I went from “abysmal” all the way up to “mediocre” just by changing my vehicle type. A massive improvement!

Several months later, I was sitting at home when I decided I wanted to get gold trophies for each of the cups in the single-player version of the game. I figured that I’d start out with the most basic loadout, but in the interim I forgot how well I did on the bike. Remembering that “bikes were for advanced players”, I ended up picking the same racer and loadout: Yoshi, Standard Kart, Standard tires, Super Glider. With this loadout, I was only mediocre. While getting 5th place is definitely better than trailing everybody else in 12th place, I wouldn’t be able to get a gold Flower Cup trophy anytime soon if all I could do was 5th place on the first circuit in the cup.

While I didn’t expect to dominate Flower Cup, I consistently did much worse than I hoped. My biggest problem was oversteering and understeering while sliding. The kart never seemed to do what I expected it to, especially since I was still getting used to the new post-slide boost mechanic.

At this point I decided to try the “more advanced” Standard Bike instead as I remembered doing a bit better with it at the friend’s house.

This seemed to be the change I needed; I took first place in Flower Cup’s first circuit, Mario Circuit. The bike behaved in ways I expected. I over- and understeered much, much less.

Now then. I had a belief of “bikes are more advanced”, but clearly this belief was keeping me from doing what I wanted to. The minimal-fanfare way to update your beliefs in this case amounts to shrugging your shoulders and reasoning “guess that half-remembered meme about bikes was way less useful for me than memories of how much better I am on a bike”. Generally, one doesn’t need to spend too much time mulling over one’s cognitive errors.

That said, sometimes it’s useful to pause a moment and mull over an error in cognition. After all, I’d like to make this error less often in the future. I like snowclones as much as the next guy, so I looked up the Litany of Tarski and amused myself by writing this:

If I am better at Mario Kart on a bike,
I desire to believe that I am better at Mario Kart on a bike.
If I am better at Mario Kart on a kart,
I desire to believe that I am better at Mario Kart on a kart.
Let me not become attached to beliefs I may not want.

From what I’ve experienced, both with Mario Kart and in other situations, the Litany of Tarski is handy to keep in mind when faced with conflicting information sources of imperfect reliability. It’s even more useful to keep in mind if there’s a chance you over- or undertrust a given information source, as I overtrusted what was probably a throwaway phrase in a GameFAQs guide.

1. A racer who’s viable in slower circuits can be too slow to compete in faster circuits. In the original Mario Kart, for example, I chose Toad (slow top speed, easy to control), but eventually had to pick Yoshi (moderate top speed, accelerates quickly, somewhat harder to control) in order to keep up with the computer-controlled opponents. ↩︎

Discuss

### Rationality Vienna Meetup June 2019

29 апреля, 2019 - 00:05
Published on April 28, 2019 9:05 PM UTC

Open to all people interested in epistemology, overcoming biases, science, self-transformation and all topics related.

Schedule:
15:00 - 15:30: arrival & socializing with tea/coffee
15:30 - 16:00: round of introductions (if new attendees - warmly welcome!)
16:00 - 18:00 open microphone & discussion TBD
~18:00: quickly cleaning the room together

Followed by dinner together in the city.
You are also welcome to only join for the dinner.

How to find the meetup room: https://my.pcloud.com/publink/show?code=XZGJrE7Z8p72zalBbmH4xELOhiViwHpLNhpy

Discuss

### Rationality Vienna Meetup May 2019

29 апреля, 2019 - 00:01
Published on April 28, 2019 9:01 PM UTC

Open to all people interested in epistemology, overcoming biases, science, self-transformation and all topics related.

Schedule:
15:00 - 15:30: arrival & socializing with tea/coffee
15:30 - 16:00: round of introductions (if new attendees - warmly welcome!)
16:00 - 18:00 open microphone & discussion TBD
~18:00: quickly cleaning the room together

Followed by dinner together in the city.
You are also welcome to only join for the dinner.

How to find the meetup room: https://my.pcloud.com/publink/show?code=XZGJrE7Z8p72zalBbmH4xELOhiViwHpLNhpy

Discuss

### Recent updates to gwern.net (2017–2019)

28 апреля, 2019 - 23:18
Published on April 28, 2019 8:18 PM UTC

Previously: 2011/2012–2013/2013–2014/2014–2015/2015–2016/2016–2017.

"Iram indeed is gone with all its Rose, / And Jamshyd's Seven-ring'd Cup where no one knows; / But still the Vine her Ancient Ruby yields / And still a Garden by the Water blows."

An index of my recent writings, by topic:

This will be the last gwern.net changelog I post on LW, as it seems increasingly redundant with my newsletter & subreddit, and my occasional regular LW link submissions. Please subscribe to those for future updates.

Discuss

### Player vs. Character: A Two-Level Model of Ethics

28 апреля, 2019 - 21:40
Published on April 28, 2019 6:40 PM UTC

Epistemic Status: Confident

This idea is actually due to my husband, Andrew Rettek, but since he doesn’t blog, and I want to be able to refer to it later, I thought I’d write it up here.

In many games, such as Magic: The Gathering, Hearthstone, or Dungeons and Dragons, there’s a two-phase process. First, the player constructs a deck or character from a very large sample space of possibilities.  This is a particular combination of strengths and weaknesses and capabilities for action, which the player thinks can be successful against other decks/characters or at winning in the game universe.

The choice of character often determines the strategies that character can use in the second phase, which is actual gameplay.  In gameplay, the character can only use the affordances that it’s been previously set up with.

This means that there are two separate places where a player needs to get things right: first, in designing a strong character/deck, and second, in executing the optimal strategies for that character/deck during gameplay.

(This is in contrast to games like chess or go, which are single-level; the capacities of black and white are set by the rules of the game, and the only problem is how to execute the optimal strategy. Obviously, even single-level games can already be complex!)

The idea is that human behavior works very much like a two-level game.

The “player” is the whole mind, choosing subconscious strategies.  The “elephant“, not the “rider.”  The player is very influenced by evolutionary pressure; it is built to direct behavior in ways that increase inclusive fitness.  The player directs what we perceive, do, think, and feel.

The player creates what we experience as “personality”, fairly early in life; it notices what strategies and skills work for us and invests in those at the expense of others.  It builds our “character sheet”, so to speak.

Note that even things that seem like “innate” talents, like the savant skills or hyperacute senses sometimes observed in autistic people, can be observed to be tightly linked to feedback loops in early childhood. In other words, savants practice the thing they like and are good at, and gain “superhuman” skill at it.  They “practice” along a faster and more hyperspecialized path than what we think of as a neurotypical “practicing hard,” but it’s still a learning process.  Savant skills are more rigidly fixed and seemingly “automatic” than non-savant skills, but they still change over time — e.g. Stephen Wiltshire, a savant artist who manifested an ability to draw hyper-accurate perspective drawings in early childhood, has changed and adapted his art style as he grew up, and even acquired new savant talents in music.  If even savant talents are subject to learning and incentives/rewards, certainly ordinary strengths, weaknesses, and personality types are likely to be “strategic” or “evolved” in this sense.

The player determines what we find rewarding or unrewarding.  The player determines what we notice and what we overlook; things come to our attention if it suits the player’s strategy, and not otherwise.  The player gives us emotions when it’s strategic to do so.  The player sets up our subconscious evaluations of what is good for us and bad for us, which we experience as “liking” or “disliking.”

The character is what executing the player’s strategies feels like from the inside.  If the player has decided that a task is unimportant, the character will experience “forgetting” to do it.  If the player has decided that alliance with someone will be in our interests, the character will experience “liking” that person.  Sometimes the player will notice and seize opportunities in a very strategic way that feels to the character like “being lucky” or “being in the right place at the right time.”

This is where confusion often sets in. People will often protest “but I did care about that thing, I just forgot” or “but I’m not that Machiavellian, I’m just doing what comes naturally.”  This is true, because when we talk about ourselves and our experiences, we’re speaking “in character”, as our character.  The strategy is not going on at a conscious level. In fact, I don’t believe we (characters) have direct access to the player; we can only infer what it’s doing, based on what patterns of behavior (or thought or emotion or perception) we observe in ourselves and others.

Evolutionary psychology refers to the player’s strategy, not the character’s. (It’s unclear which animals even have characters in the way we do; some animals’ behavior may all be “subconscious”.)  So when someone speaking in an evolutionary-psychology mode says that babies are manipulating their parents to not have more children, for instance, that obviously doesn’t mean that my baby is a cynically manipulative evil genius.  To him, it probably just feels like “I want to nurse at night. I miss Mama.”  It’s perfectly innocent. But of course, this has the effect that I can’t have more children until I wean him, and that’s to his interest (or, at least, it was in the ancestral environment when food was more scarce.)

Szaszian or evolutionary analysis of mental illness is absurd if you think of it as applying to the character — of course nobody wakes up in the morning and decides to have a mental illness. It’s not “strategic” in that sense. (If it were, we wouldn’t call it mental illness, we’d call it feigning.)  But at the player level, it can be fruitful to ask “what strategy could this behavior be serving the person?” or “what experiences could have made this behavior adaptive at one point in time?” or “what incentives are shaping this behavior?”  (And, of course, externally visible “behavior” isn’t the only thing the player produces: thoughts, feelings, and perceptions are also produced by the brain.)

It may make more sense to frame it as “what strategy is your brain executing?” rather than “what strategy are you executing?” since people generally identify as their characters, not their players.

Now, let’s talk morality.

Our intuitions about praise and blame are driven by moral sentiments. We have emotional responses of sympathy and antipathy, towards behavior of which we approve and disapprove. These are driven by the player, which creates incentives and strategic behavior patterns for our characters to play out in everyday life.  The character engages in coalition-building with other characters, forms and breaks alliances with other characters, honors and shames characters according to their behavior, signals to other characters, etc.

When we, speaking as our characters, say “that person is good” or “that person is bad”, we are making one move in an overall strategy that our players have created.  That strategy is the determination of when, in general, we will call things or people “good” or “bad”.

This is precisely what Nietzsche meant by “beyond good and evil.”  Our notions of “good” and “evil” are character-level notions, encoded by our players.

Imagine that somewhere in our brains, the player has drawn two cartoons, marked “hero” and “villain”, that we consult whenever we want to check whether to call another person “good” or “evil.” (That’s an oversimplification, of course, it’s just for illustrative purposes.)  Now, is the choice of cartoons itself good or evil?  Well, the character checks… “Ok, is it more like the hero cartoon or the villain cartoon?”  The answer is “ummmm….type error.”

The player is not like a hero or a villain. It is not like a person at all, in the usual (character-level) sense. Characters have feelings! Players don’t have feelings; they are beings of pure strategy that create feelings.  Characters can have virtues or vices! Players don’t; they create virtues or vices, strategically, when they build the “character sheet” of a character’s skills and motivations.  Characters can be evaluated according to moral standards; players set those moral standards.  Players, compared to we characters, are hyperintelligent Lovecraftian creatures that we cannot relate to socially.  They are beyond good and evil.

However! There is another, very different sense in which players can be evaluated as “moral agents”, even though our moral sentiments don’t apply to them.

We can observe what various game-theoretic strategies do and how they perform.  Some, like “tit for tat”, perform well on the whole.  Tit-for-tat-playing agents cooperate with each other. They can survive pretty well even if there are different kinds of agents in the population; and a population composed entirely of tit-for-tat-ers is stable and well-off.

While we can’t call cellular automata performing game strategies “good guys” or “bad guys” in a sentimental or socially-judgmental way (they’re not people), we can totally make objective claims about which strategies dominate others, or how strategies interact with one another. This is an empirical and theoretical field of science.

And there is a kind of “”morality”” which I almost hesitate to call morality because it isn’t very much like social-sentiment-morality at all, but which is very important, which simply distinguishes the performance of different strategies.  Not “like the hero cartoon” or “like the villain cartoon”, but “win” and “lose.”

At this level you can say “look, objectively, people who set up their tables of values in this way, calling X good and Y evil, are gonna die.”  Or “this strategy is conducting a campaign of unsustainable exploitation, which will work well in the short run, but will flame out when it runs out of resources, and so it’s gonna die.”  Or “this strategy is going to lose to that strategy.”  Or “this strategy is fine in the best-case scenario, but it’s not robust to noise, and if there are any negative shocks to the system, it’s going to result in everybody dying.

“But what if a losing strategy is good?” Well, if you are in that value system, of course you’ll say it’s good.  Also, you will lose.

Mother Teresa is a saint, in the literal sense: she was canonized by the Roman Catholic Church. Also, she provided poor medical care for the sick and destitute — unsterilized needles, no pain relief, conditions in which tuberculosis could and did spread.  Was she a good person? It depends on your value system, and, obviously, according to some value systems she was.  But, it seems, that a population that places Mother Teresa as its ideal (relative to, say, Florence Nightingale) will be a population with more deaths from illness, not fewer, and more pain, not less.  A strategy that says “showing care for the dying is better than promoting health” will lose to one that actually can reward actions that promote health.  (To be fair, for most of human history we didn’t have ways to heal the sick that were clearly better than Mother Teresa’s, and even today we don’t have credit-allocation systems that reliably reward the things that keep people alive and healthy; it would be wrong to dump on Catholicism too much here.)  That’s the “player-level” analysis of the situation.

Some game-theoretic strategies (what Nietzsche would call “tables of values”) are more survival-promoting than others.  That’s the sense in which you can get from “is” to “ought.”  The Golden Rule (Hillel’s, Jesus’s, Confucius’s, etc) is a “law” of game theory, in the sense that it is a universal, abstract fact, which even a Lovecraftian alien intelligence would recognize, that it’s an effective strategy, which is why it keeps being rediscovered around the world.

But you can’t adjudicate between character strategies just by being a character playing your strategy.  For instance, a Democrat usually can’t convert a Republican just by being a Democrat at him. To change a player’s strategy is more like “getting the bodymind to change its fundamental assessments of what is in its best interests.”  Which can happen, and can happen deliberately and with the guidance of the intellect! But not without some…what you might call, wiggling things around.

The way I think the intellect plays into “metaprogramming” the player is indirect; you can infer what the player is doing, do some formal analysis about how that will play out, comprehend (again at the “merely” intellectual level) if there’s an error or something that’s no longer relevant/adaptive, plug that new understanding into some change that the intellect can affect (maybe “let’s try this experiment”), and maybe somewhere down the chain of causality the “player”‘s strategy changes. (Exposure therapy is a simple example, probably much simpler than most: add some experiences of the thing not being dangerous and the player determines it really isn’t dangerous and stops generating fear emotions.)

You don’t get changes in player strategies just by executing social praise/blame algorithms though; those algorithms are for interacting with other characters.  Metaprogramming is… I want to say “cold” or “nonjudgmental” or “asocial” but none of those words are quite right, because they describe character traits or personalities or mental states and it’s not a character-level thing at all.  It’s a thing Lovecraftian intelligences can do to themselves, in their peculiar tentacled way.

Discuss

### The Forces of Blandness and the Disagreeable Majority

28 апреля, 2019 - 21:20
Published on April 28, 2019 6:20 PM UTC

There are a few data points that have been making me see “the discourse” differently lately.

1. Large Majorities Dislike Political Correctness.

That’s the title of this Atlantic article that came out in October, and is based on this study from the think tank More in Common which opposes political polarization.

The results of the 8000-person poll of a nationally-representative sample of Americans are pretty striking. About 80% of Americans think “political correctness is a problem”; and even when you restrict to self-identified liberals, Democrats, or people of color, large majorities agree with the statement. The study identifies “progressive activists” (8% of Americans) as a younger, more extreme, more educated, more politically active left-wing cluster, and even within this cluster, a full 25% agree with “political correctness is a problem.”

And lots of people who agree with statements about hate speech being bad, white people starting out with advantages in life, sexual harassment being a problem, etc, also think political correctness is a problem.

Being “politically incorrect” isn’t just a white thing, a male thing, or even a conservative thing. It’s a hugely common thing.

2. Support for free speech is common, and growing, not shrinking. And it’s not the most left-wing people who most oppose free speech, but the moderate liberals.

Political scientist Justin Murphy has done studies about this, based on the General Social Survey, a large poll on social attitudes that’s been running for decades.

Since the 1970’s, Americans have become more tolerant of allowing people with controversial views to speak in public — communists, people proposing military coups, homosexuals, and opponents of “all churches and religions.”  Racism is the exception to the rule — people haven’t become more tolerant of racist speech, even as they have become more tolerant of other varieties of speech.

Keep in mind that legal censorship and centralization of political speech were way more prevalent in mid-20th century America than they are today.  Cable television networks didn’t exist till the 1970’s. The Fairness Doctrine didn’t end until 1987. Satellite radio, which allowed obscene language that was regulated on conventional radio and television, only began in 1988, Fox News was founded in 1996, and, of course, the blogosphere didn’t really begin until the early 2000’s.

Murphy notes that “extreme liberals” are consistently the most supportive of permitting controversial speech, and that in fact they have increased their rates of tolerating even racist speech. People who rate themselves as “moderately liberal” and “slightly liberal”, however, have sharply declined in their willingness to tolerate racist speech.  If there’s been a “backlash against free speech”, it’s on the moderate left, not the far left.

3. Calls for speech restrictions often come from moderates.

Things like this essay by Renee diResta, which I found chilling — a call for social media to be actively regulated by the US military, which says we should treat people spreading opinions that weaken trust in “the legitimacy of government, the persistence of societal cohesion, even our ability to respond to the impending climate crisis” as “digital combatants.” DiResta says, “More authoritarian regimes, by contrast, would simply turn off the internet. An admirable commitment to the principle of free speech in peace time turns into a sucker position against adversarial psy-ops in wartime.”

Who is DiResta? She’s a writer, technologist, adviser to Congress and the State Department, and the director of research at something called New Knowledge, a firm offering corporations a new kind of service: using algorithms to bury social media scandals that would make them look bad.

In other words, she’s an influential moderate; well-connected in corporate and government worlds, and very troubled by the crisis of declining trust in traditional institutions that the open Internet has enabled.

An Alternative Paradigm: Moderate, Measured Elites vs. The Chaotic, Offensive Populace

What if “free speech” vs. “restricted speech” isn’t a right-vs.-left thing at all?

Lots of people, who are by no means all political conservatives, want the right to say offensive things.

Verbal conflict just isn’t that big a problem to most people, apparently.  And how likely you are to violate vs. observe verbal taboos varies a lot based on personality and socioeconomic class.

Swearing is an interesting example of a verbal taboo that’s not especially politicized.  Socially low-ranking people swear more. Swearing is negatively correlated with agreeableness. Men swear more than women.  Swearing is commonly associated with  being working-class, though I haven’t found published evidence of this.  Swearing is “inappropriate” in office settings, religious settings, or whenever we’re expected to be formal or respectful.

It’s often corporate caution that drives speech codes that restrict political controversy, obscenity, and muckraking/whistleblowing. It’s not just racist or far-right opinions that get silenced; media and social-media corporations worry about offending prudes, homophobes, Muslim extremists, the Chinese government, the US military, etc, etc.

Some people clearly do have strong ideological opinions about what speech they want to see allowed vs. banned, but I don’t see that as the main driver of what rules actually get put into place.  What I think is going on is that decisionmakers in media and PR, and corporate and government elites generally, have a lower tolerance for verbal conflict and taboo violations than the typical individual.

The growth of lots and lots of outlets for more “unofficial” or “raw” self-expression — blogs, yes, but before that cable TV and satellite radio, and long before that, the culture of “journalism” in 18th century America where every guy with a printing press could publish a “newspaper” full of opinions and scurrilous insults  — tends to go along with more rudeness, more cursing, more sexual explicitness, more political extremism in all directions, more “trashy” or “lowest common denominator” media, more misinformation and “dumbing down”, but also some innovative/intellectual “niche” media.

Chaos is a centrifugal force; it increases the chance of any unexpected outcome. Good things, bad things, existential threats, brilliant ideas, and a lot of weird, gross, and disturbing stuff.

Some people like parts of that (it’s hard to like everything about chaos), and others find even a little chaos threatening.  The most passionate opponents of chaos are likely to be powerful, since change can only knock them off their pedestals.

I think we’re currently in an era of unusually large amounts of free speech that elites are starting to get spooked by and defend against.  Most people have high, perhaps even growing, tolerance for controversy and offense, but some find it unacceptable, and these people are disproportionately influential.

Discuss