Вы здесь

Новости LessWrong.com

Подписка на Лента Новости LessWrong.com Новости LessWrong.com
A community blog devoted to refining the art of rationality
Обновлено: 21 минута 37 секунд назад

Arguments against myopic training

9 июля, 2020 - 19:07
Published on July 9, 2020 4:07 PM GMT

A number of people seem quite excited about training myopic reinforcement learning agents as an approach to AI safety (for instance this post on approval-directed agents, proposals 2, 3, 4, 10 and 11 here, and this paper and presentation), but I’m not. I’ve had a few detailed conversations about this recently, and although I now understand the arguments for using myopia better, I’m not much more optimistic about it than I was before. In short, it seems that evaluating agents’ actions by our predictions of their consequences, rather than our evaluations of the actual consequences, will make reinforcement learning a lot harder; yet I haven’t been able to identify clear safety benefits from doing so. I elaborate on these points below; thanks to Jon Uesato, Evan Hubinger, and Ramana Kumar for discussion and comments.

I’ll define a myopic reinforcement learner as a reinforcement learning agent trained to maximise the reward received in the next timestep, i.e. with a discount rate of 0. Because it doesn’t assign credit backwards over time, in order to train it to do anything useful, that reward function will need to contain an estimate of how valuable each (state, action, next state) transition will be for outcomes many steps later. Since that evaluation will need to extrapolate a long way forward anyway, knowing the next state doesn’t add much, and so we can limit our focus to myopic agents trained on reward functions R which ignore the resulting state: that is, where R(s,a,s') = M(s,a) for some M. We’ll call M the approval function; more generally, we can think of such agents as being trained to take actions that their supervisor approves of at the time the action is taken. This definition also includes imitation learners, for which the approval function is calculated based on the agent’s divergence from the supervisor’s policy. Although it’s not the standard interpretation of MDPs, I’ll also interpret the agent’s thoughts during a timestep as part of their action, so we can give feedback on those too in this framework.

Supervising without considering effects is a significant handicap

Firstly, I want to emphasise how much more difficult it is for the supervisor to try and evaluate the values of actions without looking at the long-term outcomes of those actions. In order to do so, we need to be able to predict in advance all the mechanisms which we want the agent to learn to use. In other words, the supervisor needs to be more intelligent than the agent - perhaps by a significant margin. Contrast this with the standard RL paradigm, in which we merely need to be able to recognise good outcomes. It’s the difference between a simple programmatic reward function for training AlphaGo, versus needing to understand the advantages of each of AlphaGo’s moves before training it to that level - which would have set the development of AlphaGo back by years or decades.

One thing that makes this less than totally implausible is the possibility of the agent's knowledge being used by the supervisor. Schemes such as iterated amplification attempt to do this via recursive decomposition of the evaluation problem. I think iterated amplification is an interesting research direction, but I don’t see what value there is in making it myopic. You could equally well do a recursive decomposition analysing the whole trajectory after more of the consequences are clear. Because the supervisor then has more data to start off, and can investigate what aspects of the world have actually changed, the latter would likely be much easier and require much less ability to interpret agent cognition. I’ll give more concrete examples of this later on; for now, it’s worth noting that I expect almost all of this training to happen in simulation, where rollouts are cheap. But once an agent is competent enough to deploy in the real world, we can also continue nonmyopic training over longer time periods. For example, once an agent has been deployed for a week or month, we can train it on evaluations of its behaviour one week or month ago (although in many cases I expect the relevant consequences to be apparent in a matter of minutes - e.g. the agent taking a wrong turn then backtracking).

It’s true that there are some types of reward function which benefit much less from waiting to see the results of actions. If an agent is merely proposing high-level ideas (like scientific theories), rather than taking actions in the world, my arguments aren’t as applicable; so I’m mainly focusing on cases where we’d like to train agents to make then carry out plans. For those agents, one example where we can assign rewards immediately is when evaluating whether an agent is “thinking the right types of thoughts” - for example, whether it’s making long-term plans or not. In theory this doesn’t depend on its future thoughts or actions - but in practice knowing them may help significantly. For example, if its “long-term plans” are rapidly changing every few timesteps, that’s evidence that we’re misidentifying what counts as a long-term plan. Yet more importantly, supervision to prevent the wrong types of thoughts from arising will not be sufficient to train a highly competent AI. We also need supervision to encourage actions that lead to good consequences - which brings us back to the arguments from the previous paragraphs.

Of course, if we want to train agents that make plans on the scale of years or decades, waiting for feedback will take prohibitively long, and so our feedback to them will need to involve predictions of future effects of their actions. So there’ll be some element of foresight required either way. But trying to evaluate agent actions without seeing any of their effects on the world would require a significant shift from the current trajectory of reinforcement learning. Given this bar, we should expect compelling reasons to believe that myopic training will actually be important for safety - which, I will argue, we don’t yet have.

Myopic training doesn’t inherently prevent dangerous long-term behaviour

It seems intuitive that if agents are never rewarded for the long-term consequences of their actions, they won’t make dangerous long-term plans - but in fact myopic training itself doesn’t make any inherent difference. Let’s compare an agent trained using standard RL on a reward function R, with an agent trained using myopic RL where its approval function is the optimal Q-function of R. The same types of cognition will lead to high-scoring agents in both cases. This is clearest in the case of Q-learning, where the Q-functions converge to the same values in the limit in both cases. The intuition here is: if the standard RL agent benefits from planning to figure out how to get to future states with high reward, then the myopically trained agent benefits from planning to figure out how to get to future states with high reward in order to choose actions with high Q-values. So a “myopic” training regime may lead to agents which make long-term plans, and generally may display the same dangerous behaviour as standard RL agents, for the same reasons; later in this post I’ll discuss in more detail how such behaviour might arise during myopic training.

The equivalence doesn’t just hold in a few cases - in fact, every function M which myopically assigns a value to all state-action pairs is the optimal Q-function for some reward function. (Specifically, the reward function R(s,a,s') = M(s,a) - λmax(M(s',a')). Notice that this is just the definition of the optimal Q-function, with M in place of Q, and rearranged so that we can calculate R in terms of M). So for any myopic training setup, there’s some reward function which produces identical incentives for nonmyopic training setups. In theory it’s not always possible to calculate that reward function starting from the approval function (e.g. in continuous action spaces, where we can't straightforwardly check all the actions) but in practice I expect that given the capability to calculate a Q-function which takes into account all relevant future effects, we can come up with a reward function that’s very nearly equivalent. If this is a crux for proponents of myopia, I’d be interested to hear why.

I should note that so far I’ve been talking about myopia as a property of a training process. This is in contrast to the cognitive property that an agent might possess, of not making decisions directly on the basis of their long-term consequences; an example of the latter is approval-directed agents. (Throughout the rest of this post all references to “myopic agents” will refer to the way those agents were trained, not to how they make decisions). “Myopic thinking” has never been particularly well-specified - we have some intuitive sense of what it looks like in human psychology, but there are a lot of missing details, especially in translating it to the context of AI. I’d be excited to see these details filled in, since I think a major constraint in safety is our lack of understanding of the possible ways that minds might work. For the purposes of this blog post, though, what’s relevant is that proposals to build approval-directed agents or agents which “think myopically” tend to outline myopic training processes intended to produce them without actually justifying why the myopic training is necessary. But based on the arguments in this post I expect that, whatever the most reasonable interpretations of “approval-directed” or “myopic” cognition turn out to be, they could be developed in nonmyopic training regimes just as well as (or better than) in myopic training regimes.

What might this look like in practice? Consider the example of an agent trained myopically to imitate the behaviour of HCH. Usually, at every step, we calculate the divergence between the agent's action and HCH's action to derive the agent’s approval score. To make this nonmyopic in a trivial sense, we merely need to convert that approval into a reward using the formula I gave above. However, after just the trivial change, myopic training will outperform nonmyopic training (because the latter requires the agent to do credit assignment across timesteps). To make it nonmyopic in an interesting and advantageous sense, we could train the agent to imitate the action HCH thinks it should have taken, after HCH already knows how the future will play out - thereby evaluating the agent’s actions based on their consequences. That would make evaluation significantly easier on many tasks - which is important given that evaluation will be the most expensive part of amplification by far. Nonmyopic training allows us to judge how good a speech is after having seen the audience’s reaction, not just from the draft; or how good a startup idea is after it’s been implemented, not just when first proposed.

Why might we not want this ability? The main objection is that seeing how things play out won’t just give us more information, but also bias our evaluation, if the agent is able to tamper with or manipulate the training process. I’ll discuss this possibility in the upcoming sections.

Tampering problems

The equivalence arguments I made above only apply if the training setup is uninfluenceable by the agent. If an agent can tamper with the implementation of the training algorithm during training, then myopia and nonmyopia can diverge sharply. In particular, it depends on what type of tampering is possible. If the agent is able to increase the feedback signal during some timesteps, then this will lead to large changes in the standard RL agent’s behaviour (since it will then be incentivised to reach those timesteps) and small changes in the myopic agent’s behaviour (it will learn that the tampering action is good in some states, but shouldn’t systematically seek them out). Of course it’s still possible in theory for myopic agents to learn to tamper consistently during training, but it seems like that’d require some very specific mechanisms for tampering (e.g. tampering with the approval function to change its evaluations of a closely-related subset of states) rather than easier mechanisms for tampering (e.g. overriding current reward signal with high values).

My main objection here is simply that we shouldn’t expect positive feedback during training to be an important reason why agents end up tampering. Figuring out how to tamper is hard; it seems very unlikely that an agent will stumble upon it during random exploration. So if an agent finds a tampering action during training, that’ll be because it knew what it was doing, and wanted to tamper. Further, in order for this to actually be an important problem the agent would need to figure out how to prevent humans from shutting it down, and pursue a range of other convergent instrumental goals. It’s these intentions which we want to prevent from arising in the first place.

Even if the agent doesn’t ever get positive reinforcement for tampering, though, the difference between myopic and nonmyopic training will presumably change the agent’s observations at the point where it learns that it’s in a training process. In theory at least, this might influence its intentions. By analogy, after humans learned about evolution, a bunch of people started making the “appeal to nature” fallacy by justifying certain behaviours because they increase evolutionary fitness.

But I think that we shouldn’t expect AI goals to be very influenced just by the knowledge of their training process, for the same reason that such arguments aren’t very important for humans: by the time an agent is intelligent enough to understand the details of its own training process, it’ll already be generally intelligent and its goals will be already largely set. At that point it's probably too late for big updates to those goals. And even if they do make big updates, if their motivations end up depending sensitively on philosophical-style reasoning, then we’re in a pretty bad position either way.

I do want to flag this whole line of investigation as very speculative, though; I think a more thorough treatment would be very valuable (and I’m hoping to release a post on this soon).

Manipulating humans as tampering

The argument that agents won’t get rewarded for tampering during training fails if we consider manipulating humans to be “tampering”. Presumably there are a lot of different ways to manipulate humans who are giving reward feedback, and once a nonmyopic agent starts doing so a little bit, then it’ll get higher reward for doing so, and that behaviour will be reinforced, and its long-term goals will be affected.

I find this argument somewhat persuasive, but I am currently leaning towards thinking that this is just as much a problem for myopic agents, conditional on having competitive myopic agents. My core argument:

  • Assume that there’s some sequence of outputs which an agent could use to manipulate its supervisor into giving it higher reward, without the supervisor realising. (If not, then neither myopic nor nonmyopic agents are incentivised to manipulate their supervisors).
  • For myopic agents to be competitive on long-term tasks, their objective function needs to be set by a supervisor which is able to accurately predict how well their actions fulfil long-term goals.
  • If that supervisor is accurately predicting and evaluating the future outputs of this myopic agent as well, then it can still be manipulated by those inputs.
  • If the supervisor isn’t accurately predicting those outputs, then the myopic agent won’t exceed its supervisor, because it will never learn to take the first step of any plan that the supervisor can’t predict will be good.

I find it helpful to think about this using a concrete example. Suppose that a myopic agent M is trained to write bestselling novels, but is restricted to writing one paragraph at a time, to avoid giving it an incentive to manipulate us (assuming it can’t do so in a single paragraph). M plans a sequence of N paragraphs which make up a good novel - but which, when combined, contains a bunch of pro-AI propaganda. A supervisor S observes M’s first output in that sequence. In order to understand how good this is, S then predicts M’s next N -1 outputs, allowing it to read M’s pro-AI propaganda. S is influenced by that propaganda, and gives M slightly more reward for its original action, therefore reinforcing M’s cognitive process of “make a plan to manipulate the supervisor”.

Perhaps you’re thinking: well, S could just predict the fact that M will output pro-AI propaganda, without predicting its contents in detail. But how does S know that the pro-AI propaganda isn’t part of what makes M’s output a bestseller? Remember that, by assumption, S can’t predict what inputs it will be manipulated by (otherwise myopia doesn’t buy us any advantage). So it will need to analyse the propaganda well enough to understand its downstream effects. If that seems implausible, remember that most of M’s training will be in simulation, and so it may be possible for S to spin up another copy of the simulation to see how the actions play out. (Evaluating the same actions in a different simulation is really stretching the definition of “myopia”, but provides a useful intuition for why manipulation can happen either way).

Even given that, it’s quite possible that S won’t be able to predict M’s outputs in enough detail to recreate the propaganda. But then S will never be able to reward M for plans which rely on outputs as detailed as what’s required to manipulate S. If S can’t predict the details of M’s outputs over many timesteps, then it can’t evaluate plans which rely on those details. Perhaps manipulative plans are more “brittle” than regular plans - that is, imprecision in S’s predictions will prevent S from being manipulated while still allowing S to accurately evaluate regular plans. But in general engineering is more brittle than persuasion. Suppose S wants M to build a rocket in simulation. S needs to be able to predict which specific types of nuts and bolts the rocket needs when M creates them (simulated) months in advance, because otherwise S can’t give positive feedback for creating those particular nuts and bolts. I’m not ruling out the possibility that S’s predictions are high-fidelity enough to supervise rocket construction, while also not high-fidelity enough to recreate manipulative outputs - but it seems pretty unlikely.

I’ll rephrase this larger point, since it’s the core of my high-level concern with myopia: arguments for myopic training lean on the intuition that it’s safer for evaluation to not directly depend on the actual long-term consequences of agents’ actions, without acknowledging that supervision powerful enough to train competitive myopic agents will effectively be doing that evaluation anyway. Insofar as the evaluation isn’t accurate enough to recreate the manipulative effects, then it’s also not accurate enough to train competitive agents.

Conclusion

In short, I don’t think any arguments so far have succeeded in making the case that we should attempt to develop AGIs which are trained primarily via myopic reinforcement learning. Further, while myopic training is a well-defined technical concept, the types of thinking we want our agents to develop under such training are not very clear, and the links between the two remain speculative.

Given the lack of clarity around myopia’s benefits, and the likely difficulty of supervising agents to become competent at achieving large-scale outcomes without the supervisor examining what outcomes the agents’ actions actually lead to, I’m more excited about research which aims to add on some component of myopic supervision to a largely nonmyopic process. For now, we should consider myopic training an interesting tool that’s being developed for the safety toolbox - but one which by default incurs major disadvantages unless deployed alongside nonmyopic training regimes, and is more of a nice add-on than an important principle of how to train safe agents. The core idea of training agents which aren’t long-term consequentialists is a different thing, which will require other approaches and insights.



Discuss

Covid 7/9: Lies, Damn Lies and Death Rates

9 июля, 2020 - 16:40
Published on July 9, 2020 1:40 PM GMT

Previously: Covid 7/2: It Could Be Worse

When I posted the link on Twitter, I joked that we already knew this week’s headline.

It turns out we didn’t. Not quite yet. I’m waiting on that one until next week.

Here’s what I said last week about death rates not picking up:

June 18 starts the surge in positive tests that represents the full second wave. June 23 represents when it accelerates. My default assumption has been one week to test positive, and about two weeks after that to see the average death.

That would mark the surge in deaths to start around July 2. In other words, today, with things picking up speed on July 7.

So no, this isn’t weird. Not yet. But if there is no spike in the next seven days, then that’s pretty weird. If that actually happened, I’d look more carefully at hospitalization data, which I usually disregard as not worth the trouble. But mostly I’d be terribly confused. The infection fatality rate seems to clearly have fallen, but why would it have fallen so much so quickly now that a surge in infections doesn’t kill more people? Quite the tall order.

Today is July 9. There was no rise in death rates starting on July 2. The holiday weekend shifted a bunch of reporting forward a few days, so tracking changes this week has been wonky. Death rate only picked up on July 7-8, and much of that was delayed reporting. Death rate this week is only slightly higher than last week’s.

It needs to be said up front. This is really weird. It’s not as weird as it looked before the last two days, but it’s still weird. I’m not going to back away and pretend it isn’t weird. Time to further investigate and break down potential causes, along with other news.

First, let’s run the numbers.

Positive Test Counts Date WEST MIDWEST SOUTH NORTHEAST May 7-May 13 22419 43256 37591 56892 May 14-May 20 22725 42762 40343 52982 May 21-May 27 23979 39418 42977 37029 May 28-June 3 32200 31504 50039 33370 June 4-June 10 35487 24674 55731 22693 June 11-June 17 41976 22510 75787 17891 June 18-June 24 66292 26792 107,221 15446 June 25-July 1 85761 34974 163,472 16303 July 2-July 8 103,879 40139 202,863 18226 Deaths by Region Date WEST MIDWEST SOUTH NORTHEAST May 7-May 13 1082 2288 1597 5327 Apr 23-29 1090 2060 1442 4541 Apr 30-May 6 775 1723 1290 3008 May 28-June 3 875 1666 1387 2557 June 4-June 10 743 1297 1230 1936 June 11-June 17 778 1040 1207 1495 June 18-June 24 831 859 1204 1061 June 25-July 1 858 658 1285 818 July 2-July 8 894 559 1503 761

Positive Test Percentages Date USA tests Positive % NY tests Positive % May 7-May 13 2,172,015 7.5% 202,980 8.2% May 14-May 20 2,628,492 6.1% 246,929 5.6% May 21-May 27 2,687,257 5.5% 305,708 3.5% May 28-June 3 3,055,035 5.0% 417,929 2.2% June 4-June 10 3,182,937 4.4% 438,695 1.4% June 11-June 17 3,459,903 4.6% 442,951 1.1% June 18-June 24 3,646,283 5.9% 440,833 1.0% June 25-July 1 4,336,532 7.0% 419,696 1.2% July 2-July 8 4,512,567 8.1% 429,804 1.1%

Less increase in overall positive rates than feared, but no sign of things becoming stable. New York looking like it might not head right back into the fire.

Nevertheless, She Persisted Talking To Her Area Man About Exponential Growth

Death rates have not fallen as much as a naive measurement would imply, because it takes time for deaths to accumulate as infection rates have gone up. The last two days imply rates have fallen less than we thought. The numbers are likely, shall we say, not fully accurate. But death rates have fallen. That’s excellent news.

The problem is that if we can’t otherwise contain the spread of the virus, a huge share of everyone not under a hard personal lockdown is going to get infected on our way to herd immunity.

Looking at the second and third derivatives we see, it doesn’t seem like there’s much hope of halting the exponential without bold new action. Aside from new mask mandates, I see essentially no new bold action.

You can reduce the multiplier on the exponential. That buys you a little time. But the exponential is going to dominate the multiplier, unless the multiplier keeps falling rapidly. I don’t see how that happens remotely fast enough.

Covid: Beyond Death

Before we discuss death, it is also important to remember that death is not the only consequence of Covid-19. I haven’t made enough mention of this in prior posts.

I do not know of good statistics on longer-term consequences for survivors of Covid-19, or how those consequences break down by age or other risk factors.

What I do know is that there is substantial risk of permanent lung and other damage, including in the relatively young.

The two closest people to our family who have had Covid-19 are our older son’s former teacher, and a personal trainer I used to use and have known for years. The trainer’s whole life revolved around working out, eating right and getting others to do the same. It wasn’t merely a job but a passion for her.

The good news is that both survived. The bad news is, neither has fully recovered. Months later, both are still getting regular medical treatments for ongoing problems. The damage appears permanent. The trainer’s heart and lungs are both permanently damaged, and it’s unclear she’ll ever be able to do her job properly again. Life for both remains no fun.

So, seriously folks, if you’re old enough to be reading this, you do not want to get Covid-19. You really, really don’t want to get Covid-19. Death is not he only thing that can go wrong. It’s not worth it.

It’s also a pretty bad time to get infected in many places. There’s a decent chance the health care system will not be fully there to help you, and any recent gains from better treatment will get more than reversed.

Stay safe to the extent it is feasible to do so. Don’t merely follow some official guidelines or simple rules like ‘wear a mask’ or ‘keep six feet apart’ and ‘wash your hands and don’t touch your face’ although you should also totally do those three things. Think hard about what you’re physically doing. It’s not optional until you get a positive antibody test.

I’m Not Dead Yet

Last week, I offered six potential explanations for the falling death rates. We could group them into three basic categories.

Explanations 1+5: We are doing a better job treating people who get infected.

Explanations 2+3+6: Different people are getting infected who are less vulnerable.

Explanation 4: We are increasingly covering up deaths.

I put some responsibility on each of the three categories.

Protecting the Vulnerable

The first category carries some weight and is essentially neutral news. It’s good that less people are dying, but it’s bad in the sense that everyone’s personal risk if infected hasn’t changed. One must be increasingly cautious even if death rates are down.

And they can easily be down a lot from this if things adjust! Remember my best guess at death rates by age and comorbidity. Infect everyone under age 50 that doesn’t have diabetes and isn’t obese, and the expected death rate would be about 0.03%, versus a best-guess IFR of between 0.5% and 1%, and a case fatality rate for the United States that is falling steadily but still over 3.5%. Fatalities are mostly a function of how many old and vulnerable people are infected, rather than how many infections there are across age groups.

Perhaps the surge in infections is young people modifying their behaviors, while old people don’t modify their own. In that case, you’d expect the infection rate to go up while deaths didn’t go up much or even continued to slowly decline.

For a while.

After enough cycles of this, the young would infect the old more even if the old didn’t change behavior. So unless they lock down even further than before, the death rate would start rising back up. The difference is this would be a delayed reaction. We’d see it only in infections that happened after the wave was well underway. Instead of deaths starting to increase on July 2 and accelerating on July 7, as I predicted last week, we’d see a slower ramp-up effect starting in mid-to-late July. The fatality rate would fall, but the level of deaths would still increase. When we got to herd immunity, we’d have successfully kept a lot of our most vulnerable safe, but far from all of them.

Reports are that the average age of infection in Florida is down to about 37, from a previous level of 60. That’s a big change. Fauci says this is happening throughout the South.

I think we should accept this is likely a lot of the story. It’s a big gain, but it’s a one time gain and it doesn’t impact your personal risk. Thus, we shouldn’t expect much additional gain from better protecting people going forward, beyond than the ‘gains’ from opening schools. The death rate has dropped as much from this as it is going to drop.

Better Treatment

There have also been reports that our treatment techniques are improving, and claims our testing has improved. These would be the best possible news.

On testing, see the section below about testing. Testing is actively getting worse at this point, rather than better, with long wait times. Things were improving before, but now they’re worse again, so these gains should reverse.

Hospital capacity is filling up, and treatment is getting rationed out of necessity more and more. It’s March all over again in a new location, and once again we were not prepared. The only difference is that this time there is even less excuse. These gains too should largely reverse.

I don’t have a good sense of how much better our knowledge is in terms of cashing it out in earlier detection of need to test slash be concerned, or in terms of better outcomes. I doubt anyone is that confident in the magnitude here. Given the amount of newsworthiness of a genuine breakthrough, I have to assume gains have been gradual accumulation of technique, and that it has a moderate but real effect. Nothing dramatic.

Fraud

That brings us to fraud.

It is clear that there was a lot of pressure from those in power, especially in the South, to report less deaths so that they could continue to reopen. The question is how much suppression or delay actually happened.

There are levels of fraud that might be going on.

We certainly had unusually large delays in reporting of deaths this week due to the holiday weekend. People don’t hang on in “Jefferson still lives” style because it’s our nation’s birthday. Every weekend there’s reporting pushed into the future, both tests and deaths. If anything, test reporting was not delayed much this weekend, whereas death reporting was delayed a lot, unless my July 7 prediction came true far more dramatically than I would have expected.

How much pushing of deaths into the future there was, or otherwise time shifting results to make things look good, is unclear. My guess is quite a bit.

One simple option is to attribute the deaths to something else. There are plenty of reports of people who couldn’t get a test, or whose death was attributed to either an unknown cause or to something like heart disease, pneumonia or stroke, where Covid-19 may or may not have been a ‘contributing factor.’

A similar method is to have something called ‘probable’ Covid-19 deaths, and choose to ignore them. Then pile as many deaths as possible into that category. New Jersey had a huge bump in cases when it started counting such deaths. New York City did something similar that still isn’t in the state statistics as far as I can tell. I am confident that such deaths are not currently being counted. As the system gets taxed more, it is easier and easier to let such deaths not be counted.

I asked on Twitter if anyone could help me find statistics on the number of deaths from unknown causes over time – the ‘mysterious deaths’ that one report claimed were increasing quite a bit, perhaps enough to double the Covid-19 death rate. I got some likes but no one had an answer. If you do know, please help.

All of that presumes a certain amount of shame, a sort of ordinary decent fraud. It’s reporting at Simulacra Levels 2 or 3. You’re pretending that your numbers reflect reality, or at a minimum, you’re pretending to pretend. My gut says things are somewhere in between those two. Everyone who pays attention knows we miss a lot of deaths and infections, yet we keep quoting those numbers without an adjustment, but we still like to think the numbers have a link to the profound reality. In some places, things moved on to the cooking of the books, where the numbers are pretending to pretend, and the veil is just good enough that we can’t prove anything.

Then there’s outright making things up. Is this happening? I don’t know. I certainly hope not. The last two days give me hope that it isn’t happening. But if we stayed around 500 deaths per day for another week while hospitalizations keep rising, regardless of what happens to positive cases, that means they’ll have stopped pretending to pretend. The numbers will be whatever they choose to report, full simulacra level 4 material. We’ll know, with common knowledge, what kind of government we are dealing with.

There was a Bloomberg news headline that death rates were down and it was nothing to celebrate. It was rightfully widely mocked, because actual low death rates are absolutely a good thing. But if it reported this way next week, then no. It’s not a good thing. It should be presumed to be a very very bad thing. It would mean we are being lied to on a whole new level, and much if not all is lost.

We need to at least demand a plausible lie.

Mask On, Mask Off

Wear a mask. Everyone wear a forking mask. Avoid and shame anyone not wearing a mask. That is all.

It’s not all, mostly because every authority starting with the W.H.O. lied to us and said masks didn’t work outside the exact right situations. But still, that is all.

Note that the W.H.O. still had issues with whether the virus is airborne, presumably because people might have the ‘wrong’ reaction to them admitting that.

The current debacle with the C.D.C. saying testing doesn’t work outside of the exact right situations, to try and get people to react the way they want, shows we have learned nothing. It’s not working and it’s further poisoning the well.

I have an idea. Stop lying to people. Crazy, I know.

Testing Delayed is Testing Denied

My wife is a psychiatrist. Thus, she has a Quest account to allow her to order various tests when people need them. This past week, she got an email from Quest asking her not to order Covid-19 tests if she could avoid it – they are backlogged, and ordering more tests will delay getting results to everyone else who needs them.

This lines up with reports from Arizona and other Southern states of waits of over a week to get Covid-19 test results back.

A test that takes a week isn’t completely worthless. You get to look back on what you already had, after it’s done. So that’s useful. But mostly it’s a de facto antibody test. First you get the symptoms that justify getting the test. Then, after you’re symptomatic, you have to get a test, which in many of these areas is no small feat if you don’t need hospitalization. Then, about two weeks into the infection if you’re positive, you’ll learn your status.

In the meantime, you have to act like you’re infected, or act like you’re not infected, or try to make some compromise, and all your options are terrible. Everyone who has been in contact with you is in limbo. Everyone in your family is in a panic, not knowing what to do. You can’t follow proper protocols. It’s a very bad scene. It’s much worse than a mere ‘can’t contact trace.’

We are both running more tests and have an increasing backlog of test demand. That’s another way to see things are rapidly getting worse.

I hope that everyone reading this knows what must be done – we need to prioritize however many samples the system can handle and get them back quickly, and outright refuse the rest, while working to ramp up capacity for the future.

The question is how.

This is America, so allocating scarce health care resources by price is anathema.

Given we can’t do price, the obvious answer is need. Better to determine a priority queue, based on level of exposure, vulnerability and symptoms. Process the top priority tests until you run out of capacity. Repeat.

Tests becomes less valuable to get back over time, not more, so the solution of doing this in first in, first out order is almost the actual worst solution. It encourages more tests to pile up, and gradually gets worse with even a small imbalance, like a traffic jam.

We would be better off getting those in real danger tests, then doing a pure lottery for everyone not in need of hospitalization.

Of course, all of that assumes there is a finite amount of testing to distribute. That’s not true at all. There’s as much testing as we want to pay for and permit. The good news is that the answer to that is gradually going up. The better news is that it could go up a hell of a lot faster if we’d let it. It’s really, really easy to set up useful Covid-19 testing. If only it were legal.

Taking Authorities to School

Mike Pence has said “we don’t want the CDC’s guidelines to be the reason schools don’t reopen.”

Thing is, he’s one hundred percent right on this one. It has become common these days for many people to ‘say the quiet part out loud.’ This is no exception. We can now say out loud that the CDC’s guidelines have little correlation to what is actually safe.

We should reopen the schools if and only if it is safe enough to do so that it is worth the benefits. 

What we absolutely not do is open the schools if and only if and to the extent possible they can abide by tons of other crazy regulations while also satisfying a technical requirement document that mixes arbitrary dictum with lots of ‘considers’ and ‘to the extent possibles’ that comes from an agency that (if you recall from last week) thinks that doing additional testing is not known to make a school safer.

This blog’s views on school are rather skeptical. Set those views aside, and assume that sending a child to school enriches their experience and prepares them for success and is not at all a prison where they are taught to obey arbitrary authority and guess the teacher’s password so they can be drones their whole lives. For the sake of argument, school good!

That does not make current plans of many places to partially reopen schools make sense.

Partial reopening, in practice, means students are in-person some of the time, in order to reduce class size and allow ‘proper social distancing.’

Parents who are uncomfortable sending kids to school at all will keep them home, which will help further reduce class size. That’s good.

Schools can be thought of as providing a basket of potential benefits.

To the extent that the benefit is logistical, this plan is terrible. If you need to go to a job, being free to do so half the time is not that useful. You still can’t do your job. Our society is not set up to allow this kind of half-measure to reap half the benefits.

To the extent that the benefit is social, this plan is terrible. Kids won’t be able to do social activities. That’s the whole point of social distancing.

To the extent that the benefit is physical activity and such, kids again likely won’t be able to do those things in any worthwhile way. A photo from a recess of each child in their chalk circle with their own ball remains perhaps the saddest thing I’ve ever seen.

To the extent that the benefit is signaling, burn the whole thing to the ground and nothing matters anyway.

To the extent that the benefit is educational, and I’m able to say that without laughing, it’s harder to predict. This plan does manage to potentially extract half or more of the benefits. In theory, one could even claim this is more than all of the benefits. Perhaps half the time in a 10-child class and half the time on one’s own to study is actually way better for learning than all the time in a 20-child class. Seems plausible, actually, even if you think the 20-child class is net useful. So presumably this is the argument in favor of the plan, if one could intelligently take advantage of the new structure. Somehow I don’t expect that to happen.

My expectation is that the needs of masks and social distancing guidelines, and the general confusion and paranoia and need to check off boxes, will take away all the potential gains while compounding the losses.

Could you redesign a school around the idea of checkpoints with adults to review and ask questions and introduce the next section, alternating with working on one’s own, with high customization and adaptation to each child, and have it be an improvement? Yes, I think you could.

That’s not remotely what’s going to happen almost anywhere.

All of this is being directly driven by CDC guidelines. Six feet has become even more fully a magical talisman one wraps around one’s self in order to create a Circle of Protection: Covid-19, the same way masks are now being talked about as the one and only true savior, because no one in authority believes most people can handle a story more subtle than that.

What will actually happen is that kids and teachers will be in rooms for hours on end, without windows, with bad circulation, mostly not even allowed to go to the bathroom. With many kids rotating between teachers after periods exactly long enough to ensure maximum infection opportunities. Or, if the kids would otherwise have always had the same teacher, the teacher will see the two halves of the class at different times. Everyone will be frustrated and mad and confused and distracted all the time.

Kids who are infected will give it to teachers who will pass it off at other times to other kids in other sections the next day. The gains from being young, having masks and being a few feet apart will be blown up by being constantly indoors with bad circulation.

Everything that follows is coming from the twin mandates ‘six feet apart’ and ‘given six feet apart, open the school.’ And regulations that prevent any creative solutions whatsoever.

Result, total clusterfork.

Instead, we need to choose from two options.

Option one is to admit that we can’t open the schools yet, at least in many or most places, and keep them closed until conditions improve.

As a bonus, we could admit that kids don’t need babysitters all the time. Yes, they do when they’re three. But at six years old, my son could be left alone in our house or backyard or the playground for a few hours, and if everyone thought that this was fine, it would be fine. We mostly don’t do it because society would think it insane and call the cops on us. Certainly by eight most kids are totally fine on their own. Everything to the contrary is people who are very bad at statistics. We could all use a little Christopher Titus parenting. A little. A lot would be bad. A little.

We won’t do that, of course. But it’s worth noting that not only are schools primarily babysitters, they’re babysitters we mostly never needed in the first place.

Option two is to admit that we need the schools, open them to all kids who want to go while giving parents the option of remote learning if they want it, and accept that it’s not going to be all that safe.

If schools are essential, they’re essential. Kids will be all right, those who live with the vulnerable can study from home. Vulnerable teachers can teach the at-home kids, and there are plenty of people who need jobs so hire more teachers to split up classes. Rent now-empty offices and hold classes outdoors to free up space. Get creative. Hope that’s good enough.

Either choice might be correct.

My take on this choice is that we should open the schools if either we can do so while containing the virus or we cannot contain the virus no matter what we do.

If we can contain the virus and get schools back, great, let’s do that and maintain people’s jobs and civil order and so on.

If we can’t contain the virus and get schools back, but we can contain the virus by closing schools, and it looks reasonable to hold out until a vaccine or other solution will let us reopen the schools, then it’s crazy to open the schools and we should keep them closed.

If we can’t contain the virus and get schools back, but we also can’t contain the virus by closing schools, then at this point a slower burn is worth little to us and might be actively counter-productive. Might as well reopen the schools for anyone who wants them.

We’ve managed to go down the path of nuking our economy pretty bad while also not stopping the virus. This gets us an outcome worse than either full suppression or full mitigation.

Our plan for the schools is similar.

Temporary Immunity Ending Real Soon Now Watch Continues

It seems like every week we get Dire Official Warnings that immunity to Covid-19 is short-lived. Often the reasoning is ‘another person pointed out that we don’t know how long immunity lasts.’ Other times it’s ‘we looked and people’s antibody counts are declining and oh noes.’ Or it looks like one person got reinfected.

We don’t know how long immunity lasts.

What we do know is that it lasts at least as long as this pandemic has been in the West, for essentially everyone. Reinfections are something every Responsible Journalist is on the lookout for as the next big Responsible Journalist scoop. Absence of evidence, in this case, is strong evidence of absence.

Note that when a study points out that people’s antibody counts are fading, yet there are no reports of reinfection, this implies that there is a lower than expected threshold of antibodies necessary to become immune, or even that the antibodies we are measuring aren’t the primary mechanism granting immunity from meaningful reinfection.

It seems appropriate to apply a modified form of the Lindsey Rule to the length of immunity. At the start, we had no idea how long it would last. Now we know it lasts at least four months before any substantial decrease takes place (we can’t assume that enough Chinese patients were re-exposed, or that China would report it if they were infected again, so to get enough bulk we have to start around March). The scary conclusion would be that immunity could be expected therefore to start to fade after about eight months. For the average person it would be expected then to last at least twice that long, so sixteen months, which is into vaccine territory. I consider that the conservative, scary estimate, that doesn’t use other priors. But my prior at the beginning, before any evidence, was already longer than that. Good news shouldn’t shorten that estimate.

In any case, yes, going for a herd immunity strategy carries some risk that immunity does not last as long as we would like. But mostly these concerns are, in practice, scaremongering, and deserve to be called out as such, so this is the periodic calling out.

Predictions for Next Week

I expect the rise in death rates from the last two days to continue, and things to get steadily worse on all fronts. Deaths are up in the South, as one would expect. This should accelerate.

I see no reason to expect us to turn the corner any time soon. As the health care system starts breaking down in the worst areas, we likely see the death rates rise faster than the case rates, rather than slower.

Eventually, of course, things left unchecked creates herd immunity and the corner does get turned.

At this pace, if we change almost nothing, how long will it take to turn that corner? Not that long. A few months. We are already at 60,000 plus reported cases per day plus exponential growth.

 



Discuss

[Resource Request] What are good resources for best practice for creating bureaucracy?

9 июля, 2020 - 15:06
Published on July 9, 2020 12:05 PM GMT

Especially, after just having read The Utopia of Rules: On Technology, Stupidity, and the Secret Joys of Bureaucracy by David Graeber, I feel that most bureaucracy we have is very bad and might create more harm then benefits. 

At the same time there are needs for bureaucracy. At Wikidata I care for it's functioning and it seems that some bureaucracy is needed to make norms legible for new users and also do reduce the necessity to waste time by having certain conflicts not resolved and refought over and over. 

While Wikimedia projects are in some sense different then other organizations, it's likely that studies of other bureaucratic structures might provide valuable knowledge. 

Is there any reading you can recommend?



Discuss

The New Frontpage Design & Opening Tag Creation!

9 июля, 2020 - 07:37
Published on July 9, 2020 4:37 AM GMT

We just pushed a new frontpage design that we've been sitting and iterating on for a while, and we also just enabled the ability for everyone to create tags and edit the descriptions of existing tags. 

New Frontpage design

A week ago we asked for your thoughts on the new frontpage design, and generally received quite positive feedback on the new design (and also almost the full LessWrong team has switched towards using the new design because we like it so much better).

I do think that pretty radical visual changes to a site can be distracting and break habits, and think that in this case it's worth that disruption, but generally want to be quite careful with doing redesigns like this to often, and don't want to force you all to reorient to a new design and visual language every time you visit the site.

The goal of this redesign was to make the frontpage a lot less visually cluttered, and make it easier to parse the structure of the site by adding more clear boundaries between foreground and background. This meant combining a bunch of elements in the Recommendations section, getting rid of a bunch of vertical dividers that were cluttering up the space, and using negative space more effective to communicate information without adding visual complexity.

The biggest change that comes with the new design is a lot more emphasis on our core tags and our frontpage selective filtering system that should allow you to personalize your frontpage, hide posts you don't want to see, and increase the frequency of topics that you are interested in. As a quick reminder, here is how to use the system: 

Increase the karma of posts that are tagged with certain tags by hovering over the core tags and selecting the appropriate modifier

When you hover over one of the core tags on the frontpage, or a tag you added yourself to the filter list (by clicking the "+" icon on the right), you can add a karma modifier to all posts that are tagged that way, hide them completely, or mark a tag as required, hiding all posts that don't have that tag. I personally have a karma modifier of +10 on "Rationality" content and on "World Modeling" content, and a -10 to "Community" content.

I expect that over the next few weeks we will make changes to some more pages (most notable the frontpage) to bring it in line with the new visual direction of the site, and are also going to be running some experiments to make the new design feel a bit less droopy (while the bright white background gave me a bit of eye-strain that I am glad to be rid off in this new design, the grey background does feel a bit depressing and I've been experimenting with whether we can improve that somehow by maybe using more images or gradients or some kind of texture, but I don't know yet where that will go.

Overall, I am excited about the new design, and please feel free to leave any feedback in the comments here, or via the Intercom!

Opening up tag creation

Starting today, lasting for at least a few weeks, we are opening up tag creation to all registered users on the site. The goal is that by the end of the month we will have tagged most of the important historical posts with good tags, and have created all the new tags that are necessary to make it so that you can reliably find good related posts, and use the tagging system to orient around all the content that is on LessWrong.

To do that, we are running a strategy of letting new tags grow whenever any LW user feels like they want a tag. It's on the LW team to prune them down, merge them, organize them, and make sure they have good descriptions. I.e. we will take care of most of the busy work of maintaining a tag system, and want you to do the fun parts of creating new tags and organizing content however you find most useful. 

The first thing you can do, if you have written any posts on LessWrong, is to tag all of the posts you have written with the existing tags, and to create any tags that you think are missing. Here <INSERT LINK TO TAG GUIDELINES> are some rough tag-guidelines about what makes a good tag. In very short summary, if you think there is an important topic or concept that has three good posts by at least two distinct authors on it, feel free to create a tag for it.

You can create new tags by clicking on the "add tag" menu on any post and selecting "Create tag" or by directly going to lesswrong.com/tag/create. 

Click on "Create Tag" in this menu to go to the new-tag page

If you want to get an overview over what tags already exists, and what posts are tagged, you can go to https://www.lesswrong.com/tags/all, which both has a quick table of all the tags, and a long list of all the tag descriptions, with the ability to edit them right there on the page:

The All Tags table, allowing you to get a quick overview over all the tags that exist in alphabetical orderThe Tag Details list, allowing you to quickly see and edit the descriptions of all tags, in a list sorted by how many posts each tag contains

I am also quite excited about the tagging system, and have found that it has already helped me orient to the massive amount of writing that exists on the site, and am really looking forward to what things will be like when we have a tag for all important LessWrong concepts, and have most posts appropriately tagged with them.



Discuss

AI Research Considerations for Human Existential Safety (ARCHES)

9 июля, 2020 - 05:49
Published on July 9, 2020 2:49 AM GMT

Andrew Critch's (Academian) and David Krueger's review of 29 AI (existential) safety research directions, each with an illustrative analogy, examples of current work and potential synergies between research directions, and discussion of ways the research approach might lower (or raise) existential risk.



Discuss

Sporting vs. Herding: two styles of discourse

9 июля, 2020 - 00:52
Published on July 8, 2020 9:52 PM GMT

Status: Vaguely culture-war, but trying to stay meta.

I wanna talk about two blogposts, Seph's "War Over Being Nice" and Alastair's "Of Triggering & the Triggered." Each lays out the same erisological idea: that there are two distinct modes or cultures of running discourse these days, and understanding the difference is crucial to understanding the content of conversation as much as its form. Let's go.

One style, Alastair writes, is indebted to the Greco-Roman rhetorical and 19th C British sporting traditions. A debate takes place in a "heterotopic" arena which is governed by an ethos of adversarial collaboration and sportsmanship. It is waged in a detached and impersonal manner, e.g. in American debate club, which inherits from these older traditions, you are assigned a side to argue; your position is not some "authentic" expression of self. Alastair:

This form of discourse typically involves a degree of ‘heterotopy’, occurring in a ‘space’ distinct from that of personal interactions.This heterotopic space is characterized by a sort of playfulness, ritual combativeness, and histrionics. This ‘space’ is akin to that of the playing field, upon which opposing teams give their rivals no quarter, but which is held distinct to some degree from relations between the parties that exist off the field. The handshake between competitors as they leave the field is a typical sign of this demarcation.

All in all, it is a mark against one in these debates to take an argument personally, to allow arguments that happen "in the arena" to leave the arena. This mode of discourse I see exemplified in LessWrong culture, and is, I think, one of the primary attractors to the site.

In the second mode of discourse, inoffensiveness, agreement, and inclusivity are emphasized, and positions are seen as closely associated with their proponents. Alastair speculates it originates in an educational setting which values cooperation, empathy, equality, non-competitiveness, affirmation, and subordination; this may be true, but I feel less confident in it than I am the larger claim about discursive modes.

Provocatively, the two modes are dubbed "sporting" and "herding," with all the implications of, on the one hand, individual agents engaged in ritualized, healthy simulations of combat, and on the other, of quasi-non-agents shepherded in a coordinated, bounded, highly constrained and circumscribed epistemic landscape. Recall, if you are tempted to blame this all on the postmodernists, that this is exactly the opposite of their emphasis toward the "adult" realities of relativism, nebulosity, flux. Queer Theory has long advocated for the dissolution of gendered and racial identity, not the reification of identitarian handles we see now, which is QT's bastardization. We might believe these positions were taken too far, but they are ultimately about complicating the world and removing the structuralist comforts of certainty and dichotomy. (Structureless worlds are inherently hostile to rear children in, and also for most human life; see also the Kegan stages for a similar idea.)

In the erisological vein, Alastair provides a portrait of the collision between the sporting and herding modes. Arguments that fly in one discursive style (taking offence, emotional injury, legitimation-by-feeling) absolutely do not fly in the other:

When these two forms of discourse collide they are frequently unable to understand each other and tend to bring out the worst in each other. The first [new, sensitive] form of discourse seems lacking in rationality and ideological challenge to the second; the second [old, sporting] can appear cruel and devoid of sensitivity to the first. To those accustomed to the second mode of discourse, the cries of protest at supposedly offensive statements may appear to be little more than a dirty and underhand ploy intentionally adopted to derail the discussion by those whose ideological position can’t sustain critical challenge.

Seph stumbles upon a similar division, though it is less about discursive and argumentative modes, and more about social norms for emotional regulation and responsibility. He calls them Culture A and Culture B, mirroring sporting and herding styles, respectively.

In culture A, everyone is responsible for their own feelings. People say mean stuff all the time—teasing and jostling each other for fun and to get a rise. Occasionally someone gets upset. When that happens, there's usually no repercussions for the perpetrator. If someone gets consistently upset when the same topic is brought up, they will either eventually stop getting upset or the people around them will learn to avoid that topic. Verbally expressing anger at someone is tolerated. It is better to be honest than polite.

In such a culture, respect and status typically comes from performance; Seph quotes the maxim "If you can't sell shit, you are shit." We can see a commonality with sporting in that there is some shared goal which is attained specifically through adversarial play, such that some degree of interpersonal hostility is tolerated or even sought. Conflict is settled openly and explicitly.

In culture B, everyone is responsible for the feelings of others. At social gatherings everyone should feel safe and comfortable. After all, part of the point of having a community is to collectively care for the emotional wellbeing of the community's members. For this reason its seen as an act of violence against the community for your actions or speech to result in someone becoming upset, or if you make people feel uncomfortable or anxious. This comes with strong repercussions—the perpetrator is expected to make things right. An apology isn't necessarily good enough here—to heal the wound, the perpetrator needs to make group participants once again feel nurtured and safe in the group. If they don't do that, they are a toxic element to the group's cohesion and may no longer be welcome in the group. It is better to be polite than honest. As the saying goes, if you can't say something nice, it is better to say nothing at all.

In such a culture, status and respect come from your contribution to group cohesion and safety; Seph cites the maxim "Be someone your coworkers enjoy working with." But Seph's argument pushes back, fruitfully, on descriptions of Culture B as collaborative; rather, he writes, they are accommodating in the Thomas-Kilmann modes of conflict sense:


Seph and Alastair both gesture toward the way these modes feel gendered, with Culture A more "masculinized" and Culture B more "feminized."[1] While this seems important to note, given that a massive, historically unprecedented labor shift toward coed co-working has recently occured in the Western world, I don't see much point in hashing out a nature vs. nurture, gender essentialism debate here, so you can pick your side and project it. This is also perhaps interesting from the frame of American feminist history: early waves of feminism were very much about escaping the domestic sphere and entering the public sphere; there is an argument to be made that contemporary feminisms, now that they have successfully entered it, are dedicated to domesticating the public sphere into a more comfortable zone. Culture B, for instance, might well be wholly appropriate to the social setting of a living room, among acquaintances who don't know each other well; indeed, it feels much like the kind of aristocratic parlor culture of the same 19th C Britain that the sporting mode also thrived in, side-by-side. And to some extent, Culture A is often what gets called toxic masculinity; see Mad Men for a depiction.

(On the topic of domestication of the workplace: We've seen an increased blurring of the work-life separation; the mantra "lean-in" has been outcompeted by "decrease office hostility"; business attire has slid into informality, etiquette has been subsumed into ethics, dogs are allowed in the workplace. Obviously these changes are not driven by women's entrance into the workplace alone; the tech sector has had an enormous role in killing both business attire and the home-office divide, despite being almost entirely male in composition. And equally obvious, there is an enormous amount of inter- and intra-business competition in tech, which is both consistently cited by exiting employees as a hostile work environment, and has also managed to drive an outsized portion of global innovation the past few decades—thus cultural domestication is not at all perfectly correlated with a switch from Culture A to B. Draw from these speculations what you will.)

There are other origins for the kind of distinctions Seph and Alastair draw; one worthwhile comparison might be Nietzsche's master and slave moralities. The former mode emphasizes power and achievement, the other empathy, cooperation, and compassion. (Capitalism and communitarianism fall under some of the same, higher-level ideological patterns.) There are differences of course: the master moralist is "beyond" good and evil, or suffering and flourishing, whereas Culture A and B might both see themselves as dealing with questions of suffering but in very different ways. But the "slave revolt in morality" overwrote an aristocratic detachment or "aboveness" that we today might see as deeply immoral or inhuman; it is neither surprising nor damning that a revolting proletariat—the class which suffered most of the evils of the world—would speak from a place of one-to-one, attached self-advocacy. One can switch "sides" or "baskets" of the arena each half or quarter because they are impersonal targets in a public commons; one cannot so easily hold the same attitude toward defending one's home. This alone may indicate we should be more sympathetic to the communitarian mode than we might be inclined to be; certainly, those who advocate and embody this mode make plausible claims to being a similar, embattled and embittered class. A friend who I discussed these texts with argued that one failure mode of the rationalist community is an "unmooring" from the real concerns of human beings, slipping into an idealized, logical world modeled on self-similarity (i.e. highly Culture A, thinking over feeling in the Big 5 vocabulary), in a way that is blind to the realities of the larger population.

But there are also grave problems for such a discursive mode, especially when it becomes dominant. Because while on the surface, discursive battles in the sporting mode can appear to be battles between people, they are in reality battles between ideas.

As Mill argued in On Liberty, free discourse is crucial because it acts as a social steering mechanism: should we make a mistake in our course, freedom of discourse is the instrument for correcting it. But the mistake of losing free discourse is very hard to come back from; it must be fought for again, before other ideals can be pursued.

Moreover, freedom of discourse is the means of rigorizing ideas before they are implemented, such as to avoid catastrophe. Anyone familiar with James Scott's Seeing Like A State, or Hayek's arguments for decentralized market intelligence, or a million other arguments against overhaulism, knows how difficult it is to engineer a social intervention that works as intended: the unforeseen, second-order effects; our inability to model complex systems and human psychology. Good intent is not remotely enough, and the herding approach cannot help but lower the standard of thinking and discourse emerging from such communities, which become more demographically powerful even as their ideas become worse (the two are tied up inextricably).

The fear of conflict and the inability to deal with disagreement lies at the heart of sensitivity-driven discourses. However, ideological conflict is the crucible of the sharpest thought. Ideological conflict forces our arguments to undergo a rigorous and ruthless process through which bad arguments are broken down, good arguments are honed and developed, and the relative strengths and weaknesses of different positions emerge. The best thinking emerges from contexts where interlocutors mercilessly probe and attack our arguments’ weaknesses and our own weaknesses as their defenders. They expose the blindspots in our vision, the cracks in our theories, the inconsistencies in our logic, the inaptness of our framing, the problems in our rhetoric. We are constantly forced to return to the drawing board, to produce better arguments.

And on the strength of sporting approaches in rigorizing discourse:

The truth is not located in the single voice, but emerges from the conversation as a whole. Within this form of heterotopic discourse, one can play devil’s advocate, have one’s tongue in one’s cheek, purposefully overstate one’s case, or attack positions that one agrees with. The point of the discourse is to expose the strengths and weaknesses of various positions through rigorous challenge, not to provide a balanced position in a single monologue

Thus those who wish us to accept their conceptual carvings or political advocacies without question or challenge are avoiding short-term emotional discomfort at the price of their own long-term flourishing, at the cost of finding working and stable social solutions to problems. Standpoint epistemology correctly holds that individuals possess privileged knowledge as to what it's like (in the Nagel sense) to hold their social identities. But it is often wrongly extended, in the popular game of informational corruption called "Telephone" or "Chinese Whispers," as arguing that such individuals also possess unassailable and unchallengeable insight into the proper societal solutions to their grievances. We can imagine a patient walking into the doctor's office; the doctor cannot plausibly tell him there is no pain in his leg, if he claims there is, but the same doctor can recommend treatment, or provide evidence as to whether the pain is physical or psychosomatic.

A lack of discursive rigour would not be a problem, Alastair writes, "were it not for the fact that these groups frequently expect us to fly in a society formed according to their ideas, ideas that never received any rigorous stress testing."

As for myself, it was not too long ago I graduated from a university in which a conflict between these modes is ongoing. We had a required course called Contemporary Civilization, founded in the wake of World War I, which focused on the last 2,000 years of philosophy, seminar-style: a little bit of introductory lecture, but most of the 2 x 2-hour sessions each week were filled by students arguing with one other. In other words, its founding ethos was of sporting and adversarial collaboration.

We also had a number of breakdowns where several students simply could not handle this mode: they would begin crying, or say they couldn't deal with the [insert atmosphere adjective] in the room, and would either transfer out or speak to the professor. While they were not largely representative, they required catering to, and no one wished to upset these students. I have heard we were a fortunate class insofar as we had a small handful of students willing to engage sporting-style, or skeptical a priori of the dominant political ideology at the school. When, in one session, a socialist son of a Saudi billionaire, wearing a $10,000 watch and a camel-hair cashmere sweater, pontificated about "burning the money, reverting to a barter system, and killing the bosses," folks in class would mention that true barter systems were virtually unprecedented in post-agricultural societies, and basically unworkable at scale. In other classes, though, when arguments like these were made—which, taken literally, are logically irrational, but instead justify themselves through sentiment and emotional legitimation, in the Culture B sense—other students apparently nodded sagely from the back of the room, "yes, and-ing" one another til their noses ran. Well, I wanted to lay out the styles with some neutrality, but I suppose it's clear now where my sympathies stand.

[1] It should go without saying, but to cover my bases, these modes feeling "feminized" or "masculinized" does not imply that all women, or women inherently, engage in one mode while all men inherently engage in another. Seph cites Camille Paglia as an archetypal example of a Culture A woman, and while she may fall to the extreme side of the Culture A mode, I'd argue most female intellectuals of the 20th C (at least those operated outside the sphere of feminist discourse) were strongly sporting-types: Sontag, for instance, was vociferous and unrelenting.



Discuss

Surviving in an immoral maze

9 июля, 2020 - 00:27
Published on July 8, 2020 9:27 PM GMT

As an older student, facing a long educational track, I'm interested in developing the right attitude toward schoolwork. By attitude, I mean a language to describe my problems and an intuition for what my goals should be and how to prioritize to achieve them. I'd like this to be generally applicable, correct, and user-friendly. I'm going to write these guidelines with confidence, even though they're just my opinions.

Learning and credentials

First, some concept handles.

Practical learning is knowledge and skills for which a) the learner has high confidence that they'll contribute directly to their tangible success, and b) are acquired in a timely and efficient manner to be actually used on the job. Central examples: on-the-job training on equipment you'll be operating, learning to read, moving to France and enrolling in an intensive course in French.

Scaffolding is knowledge and skills for which a) the learner has high confidence they'll make it so much easier to do practical learning that it's more efficient to build the scaffolding first, and b) it's acquired in a timely and efficient manner to do subsequent practical learning. Central examples: learning Python as an introductory language for someone planning on a career as a programmer, learning how to use Anki for someone whose job entails learning a lot of facts, building healthy life habits and time management skills.

Very little of what you learn at the undergraduate level is practical learning by this strict definition. And most course work is not scaffolding either.

Credentialism is knowledge and skills for which the learner has high confidence they'll contribute directly to earning the credentials they need to level up. Even if you do wind up using a small subset of this knowledge as a practical skill, you don't know what will be useful, so it's impossible to prioritize your deliberate practice to maintain and build on it.

Familiarity is a sense of identity with a subject, and experience navigating the reference materials. With scaffolding, you learn in order to progress to practical skills and with the expectation that the scaffolding and practical learning reinforce each other. With familiarity, you learn with the expectation that you'll forget almost everything.

Most undergraduate course work builds credentialism and familiarity. You've worked through the chemistry textbook. A year later, you might not remember the Arrhenius equation, but you do remember that it exists and where to look in order to re-teach yourself the knowledge. Some small subset of the learning does indeed wind up preparing you for practical learning, but because you can't predict which bits will apply, it's not scaffolding in the strict sense.

The frustrating truth

It's important for students to understand that it's not their fault that most of their early education is spent on credentialism and familiarity. We just haven't structured the educational system to better-prioritize scaffolding and practical learning.

In fact, you almost can't do scaffolding or practical learning prior to graduate work. You won't know what you need to concentrate on. Your time will be consumed by credentialism and familiarity-building.

If you can accept this state of affairs, then the logical thing to do is to focus not on learning, but on buying your credentials as cheaply as possible. When you have the rare opportunity to do scaffolding or practical learning, take it. Spend your slack figuring out even better deals on your credentials and making life as sustainable for yourself as possible.

A change in mindset

This is aimed at high scholastic achievers. The A students. The ones who know they want to get higher degrees, and arrive at a demanding and rewarding career doing something they're passionate about or believe is important for the world.

The mindset they're starting with is "I have energy and intellect to spare. So I put it into perfecting my grades, into side projects, and into internships. I don't just want to do well enough to get to the next level.

I am impatient to finish. I don't just want to do good research for an undergraduate; I want my undergraduate thesis to be worthy of peer review and publication. I don't just want to fiddle around with fun software projects while I'm learning to code; I want to build something that's actually useful for people."

The mindset I want to leave them with is "I have energy and intellect to spare, but I likely won't have access to the tools, learning environment, and opportunities I need to achieve tangible success until I'm credentialed. Trying to hasten toward early tangible success will tend to make me to prioritize immediate tractability over all other considerations, which is not an optimal long-term strategy.

My present life as a student isn't just a precursor to my long-term career. It's worth my time and energy to live well in the present. The right thing to do is spend my slack getting myself pointed in the right direction, figuring out ways to tick the necessary boxes as easily as possible, and making my life as good as possible right now. Maybe when I'm more powerful and better-resourced, I can improve or even revolutionize this slow and tedious system to speed the next cohort's journey into scaffolding and practical learning."

You are probably an average unusual person

The average scholastic high-achiever really is trapped in a dysfunctional gate-keeping system that's not build to accommodate them. You are probably more or less an average high-achiever. You have potential, not power. You don't have the contacts. You don't have the ideas. You don't have the skills. You don't have the money.

Your best bet isn't to try and bypass the gate. It's to get through it by the normal path as effortlessly as possible.

You bought into the system at an early age. You identified with your A. Then you grew up and you saw that getting an A isn't the same as doing useful work. Now, you don't identify with your A, but you don't have an alternative outlet for your raw energy and creativity. What to do?

The right answer is to identify with your life. Figure out how to make more money on the side so that you can have savings and enjoy nice things (be charitable later, when you're making real money). Get enough sleep. Learn how to do those fun hobbies you always thought you'd cultivate after school. Figure out ways to do the tedious busywork as quickly as possible while still getting an acceptable result (and acceptable might still mean straight As).

Why do I think this?

This is the attitude I've arrived at after a lot of influence from LessWrong. It's the logical student-relevant implication of the idea that school is 80% signaling. It's the result of trying to do a lot of side projects and never feeling quite confident in even the best ideas. It's the result of being deeply skeptical about a lot of other people's projects. It's the result of the assumption that most of the low-hanging fruit is being picked, which should lead to a strong bias against the value of ideas that are accessible to even a very bright early student. It's understanding that counterexamples involve a combination of privilege, cherry-picking, and exaggeration.

It's also the result of figuring out in loose terms what I would count as a career-defining achievement, seeing the many years of work and huge amount of resources it would take to get there, and knowing that somebody else is almost guaranteed to beat me to it if I try to start now.

Understanding the circumstances

School is an immoral maze. At least three levels of management (students, teachers, departments, higher-level administration, the administrators of the higher level schools they're tailoring their curriculum for, the PI who manages grad students, the grant-makers, peer reviewers, ethics boards, University administrators, politicians and the voting public). Teachers and administrators have very little skin in the game. Some have soul in the game, but many teachers are either burnt out, or never wanted to be teachers in the first place. To keep away from the appearance of a soft curriculum, there is pressure to over-assign work, and pressure on students to over-commit in order to stand out in the signaling game.

And what does it say about itself? Well, my college's website says "X College offers... academic and professional/technical degrees and certificates to meet... learning needs."

If a college wanted to advertise itself as a place to do scaffold or do practical learning, it would say so. Perhaps "X College offers curriculums tailored to teach up-to-date job skills, delivered by efficient teachers who harness the power of top-notch digital lectures and educational software." Instead it offers degrees and certificates. To meet "learning needs."

Zvi says the right thing to do is "Quit. Seriously. Go do something else. Ideally, do it today."

As a student, you're as far as possible from the top. You're not doing object-level work. And you're paying, not getting paid, so there's no question of charitable giving.

But for most of us, there are only a few alternatives, most of which have a pretty low professional ceiling. If you want access to the colleagues, tools, money, position, and credibility to do groundbreaking innovative work, you're going to have to go through the maze. It is very rare to become a scientist without the credentials, and if you ask the people who did, they'll generally tell you they did it the hard way.

If you can't escape the immoral maze, and you can't change it, and you can't pretend anymore to identify with it, then the next best option is to make it as livable as possible. Don't give it more than it requires. Don't sacrifice what you want to achieve out of distaste for the maze. Don't beat yourself up for not being able to beat the system.



Discuss

What does it mean to apply decision theory?

8 июля, 2020 - 23:31
Published on July 8, 2020 8:31 PM GMT

.mjx-chtml {display: inline-block; line-height: 0; text-indent: 0; text-align: left; text-transform: none; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; word-wrap: normal; word-spacing: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0; min-height: 0; border: 0; margin: 0; padding: 1px 0} .MJXc-display {display: block; text-align: center; margin: 1em 0; padding: 0} .mjx-chtml[tabindex]:focus, body :focus .mjx-chtml[tabindex] {display: inline-table} .mjx-full-width {text-align: center; display: table-cell!important; width: 10000em} .mjx-math {display: inline-block; border-collapse: separate; border-spacing: 0} .mjx-math * {display: inline-block; -webkit-box-sizing: content-box!important; -moz-box-sizing: content-box!important; box-sizing: content-box!important; text-align: left} .mjx-numerator {display: block; text-align: center} .mjx-denominator {display: block; text-align: center} .MJXc-stacked {height: 0; position: relative} .MJXc-stacked > * {position: absolute} .MJXc-bevelled > * {display: inline-block} .mjx-stack {display: inline-block} .mjx-op {display: block} .mjx-under {display: table-cell} .mjx-over {display: block} .mjx-over > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-under > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-stack > .mjx-sup {display: block} .mjx-stack > .mjx-sub {display: block} .mjx-prestack > .mjx-presup {display: block} .mjx-prestack > .mjx-presub {display: block} .mjx-delim-h > .mjx-char {display: inline-block} .mjx-surd {vertical-align: top} .mjx-mphantom * {visibility: hidden} .mjx-merror {background-color: #FFFF88; color: #CC0000; border: 1px solid #CC0000; padding: 2px 3px; font-style: normal; font-size: 90%} .mjx-annotation-xml {line-height: normal} .mjx-menclose > svg {fill: none; stroke: currentColor} .mjx-mtr {display: table-row} .mjx-mlabeledtr {display: table-row} .mjx-mtd {display: table-cell; text-align: center} .mjx-label {display: table-row} .mjx-box {display: inline-block} .mjx-block {display: block} .mjx-span {display: inline} .mjx-char {display: block; white-space: pre} .mjx-itable {display: inline-table; width: auto} .mjx-row {display: table-row} .mjx-cell {display: table-cell} .mjx-table {display: table; width: 100%} .mjx-line {display: block; height: 0} .mjx-strut {width: 0; padding-top: 1em} .mjx-vsize {width: 0} .MJXc-space1 {margin-left: .167em} .MJXc-space2 {margin-left: .222em} .MJXc-space3 {margin-left: .278em} .mjx-test.mjx-test-display {display: table!important} .mjx-test.mjx-test-inline {display: inline!important; margin-right: -1px} .mjx-test.mjx-test-default {display: block!important; clear: both} .mjx-ex-box {display: inline-block!important; position: absolute; overflow: hidden; min-height: 0; max-height: none; padding: 0; border: 0; margin: 0; width: 1px; height: 60ex} .mjx-test-inline .mjx-left-box {display: inline-block; width: 0; float: left} .mjx-test-inline .mjx-right-box {display: inline-block; width: 0; float: right} .mjx-test-display .mjx-right-box {display: table-cell!important; width: 10000em!important; min-width: 0; max-width: none; padding: 0; border: 0; margin: 0} .MJXc-TeX-unknown-R {font-family: monospace; font-style: normal; font-weight: normal} .MJXc-TeX-unknown-I {font-family: monospace; font-style: italic; font-weight: normal} .MJXc-TeX-unknown-B {font-family: monospace; font-style: normal; font-weight: bold} .MJXc-TeX-unknown-BI {font-family: monospace; font-style: italic; font-weight: bold} .MJXc-TeX-ams-R {font-family: MJXc-TeX-ams-R,MJXc-TeX-ams-Rw} .MJXc-TeX-cal-B {font-family: MJXc-TeX-cal-B,MJXc-TeX-cal-Bx,MJXc-TeX-cal-Bw} .MJXc-TeX-frak-R {font-family: MJXc-TeX-frak-R,MJXc-TeX-frak-Rw} .MJXc-TeX-frak-B {font-family: MJXc-TeX-frak-B,MJXc-TeX-frak-Bx,MJXc-TeX-frak-Bw} .MJXc-TeX-math-BI {font-family: MJXc-TeX-math-BI,MJXc-TeX-math-BIx,MJXc-TeX-math-BIw} .MJXc-TeX-sans-R {font-family: MJXc-TeX-sans-R,MJXc-TeX-sans-Rw} .MJXc-TeX-sans-B {font-family: MJXc-TeX-sans-B,MJXc-TeX-sans-Bx,MJXc-TeX-sans-Bw} .MJXc-TeX-sans-I {font-family: MJXc-TeX-sans-I,MJXc-TeX-sans-Ix,MJXc-TeX-sans-Iw} .MJXc-TeX-script-R {font-family: MJXc-TeX-script-R,MJXc-TeX-script-Rw} .MJXc-TeX-type-R {font-family: MJXc-TeX-type-R,MJXc-TeX-type-Rw} .MJXc-TeX-cal-R {font-family: MJXc-TeX-cal-R,MJXc-TeX-cal-Rw} .MJXc-TeX-main-B {font-family: MJXc-TeX-main-B,MJXc-TeX-main-Bx,MJXc-TeX-main-Bw} .MJXc-TeX-main-I {font-family: MJXc-TeX-main-I,MJXc-TeX-main-Ix,MJXc-TeX-main-Iw} .MJXc-TeX-main-R {font-family: MJXc-TeX-main-R,MJXc-TeX-main-Rw} .MJXc-TeX-math-I {font-family: MJXc-TeX-math-I,MJXc-TeX-math-Ix,MJXc-TeX-math-Iw} .MJXc-TeX-size1-R {font-family: MJXc-TeX-size1-R,MJXc-TeX-size1-Rw} .MJXc-TeX-size2-R {font-family: MJXc-TeX-size2-R,MJXc-TeX-size2-Rw} .MJXc-TeX-size3-R {font-family: MJXc-TeX-size3-R,MJXc-TeX-size3-Rw} .MJXc-TeX-size4-R {font-family: MJXc-TeX-size4-R,MJXc-TeX-size4-Rw} .MJXc-TeX-vec-R {font-family: MJXc-TeX-vec-R,MJXc-TeX-vec-Rw} .MJXc-TeX-vec-B {font-family: MJXc-TeX-vec-B,MJXc-TeX-vec-Bx,MJXc-TeX-vec-Bw} @font-face {font-family: MJXc-TeX-ams-R; src: local('MathJax_AMS'), local('MathJax_AMS-Regular')} @font-face {font-family: MJXc-TeX-ams-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_AMS-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_AMS-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_AMS-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-B; src: local('MathJax_Caligraphic Bold'), local('MathJax_Caligraphic-Bold')} @font-face {font-family: MJXc-TeX-cal-Bx; src: local('MathJax_Caligraphic'); font-weight: bold} @font-face {font-family: MJXc-TeX-cal-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-R; src: local('MathJax_Fraktur'), local('MathJax_Fraktur-Regular')} @font-face {font-family: MJXc-TeX-frak-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-B; src: local('MathJax_Fraktur Bold'), local('MathJax_Fraktur-Bold')} @font-face {font-family: MJXc-TeX-frak-Bx; src: local('MathJax_Fraktur'); font-weight: bold} @font-face {font-family: MJXc-TeX-frak-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-BI; src: local('MathJax_Math BoldItalic'), local('MathJax_Math-BoldItalic')} @font-face {font-family: MJXc-TeX-math-BIx; src: local('MathJax_Math'); font-weight: bold; font-style: italic} @font-face {font-family: MJXc-TeX-math-BIw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-BoldItalic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-BoldItalic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-BoldItalic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-R; src: local('MathJax_SansSerif'), local('MathJax_SansSerif-Regular')} @font-face {font-family: MJXc-TeX-sans-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-B; src: local('MathJax_SansSerif Bold'), local('MathJax_SansSerif-Bold')} @font-face {font-family: MJXc-TeX-sans-Bx; src: local('MathJax_SansSerif'); font-weight: bold} @font-face {font-family: MJXc-TeX-sans-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-I; src: local('MathJax_SansSerif Italic'), local('MathJax_SansSerif-Italic')} @font-face {font-family: MJXc-TeX-sans-Ix; src: local('MathJax_SansSerif'); font-style: italic} @font-face {font-family: MJXc-TeX-sans-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-script-R; src: local('MathJax_Script'), local('MathJax_Script-Regular')} @font-face {font-family: MJXc-TeX-script-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Script-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Script-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Script-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-type-R; src: local('MathJax_Typewriter'), local('MathJax_Typewriter-Regular')} @font-face {font-family: MJXc-TeX-type-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Typewriter-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Typewriter-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Typewriter-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-R; src: local('MathJax_Caligraphic'), local('MathJax_Caligraphic-Regular')} @font-face {font-family: MJXc-TeX-cal-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-B; src: local('MathJax_Main Bold'), local('MathJax_Main-Bold')} @font-face {font-family: MJXc-TeX-main-Bx; src: local('MathJax_Main'); font-weight: bold} @font-face {font-family: MJXc-TeX-main-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-I; src: local('MathJax_Main Italic'), local('MathJax_Main-Italic')} @font-face {font-family: MJXc-TeX-main-Ix; src: local('MathJax_Main'); font-style: italic} @font-face {font-family: MJXc-TeX-main-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-R; src: local('MathJax_Main'), local('MathJax_Main-Regular')} @font-face {font-family: MJXc-TeX-main-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-I; src: local('MathJax_Math Italic'), local('MathJax_Math-Italic')} @font-face {font-family: MJXc-TeX-math-Ix; src: local('MathJax_Math'); font-style: italic} @font-face {font-family: MJXc-TeX-math-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size1-R; src: local('MathJax_Size1'), local('MathJax_Size1-Regular')} @font-face {font-family: MJXc-TeX-size1-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size1-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size1-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size1-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size2-R; src: local('MathJax_Size2'), local('MathJax_Size2-Regular')} @font-face {font-family: MJXc-TeX-size2-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size2-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size2-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size2-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size3-R; src: local('MathJax_Size3'), local('MathJax_Size3-Regular')} @font-face {font-family: MJXc-TeX-size3-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size3-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size3-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size3-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size4-R; src: local('MathJax_Size4'), local('MathJax_Size4-Regular')} @font-face {font-family: MJXc-TeX-size4-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size4-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size4-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size4-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-R; src: local('MathJax_Vector'), local('MathJax_Vector-Regular')} @font-face {font-family: MJXc-TeX-vec-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-B; src: local('MathJax_Vector Bold'), local('MathJax_Vector-Bold')} @font-face {font-family: MJXc-TeX-vec-Bx; src: local('MathJax_Vector'); font-weight: bold} @font-face {font-family: MJXc-TeX-vec-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Bold.otf') format('opentype')}

Based on discussions with Stuart Armstrong and Daniel Kokotajlo.

There are two conflicting ways of thinking about foundational rationality arguments such as the VNM theorem.

  1. As direct arguments for normative principles. The axioms are supposed to be premises which you'd actually accept. The axioms imply theories of rationality such as probability theory and utility theory. These are supposed to apply in practice: if you accept the axioms, then you should be following them.
  2. As idealized models. Eliezer compares Bayesian reasoners to a Carnot engine: an idealized, thermodynamically perfect engine which can never be built. To the extent that any real engine works, it approximates a Carnot engine. To the extent that any cognition really works, it approximates Bayes. Bayes sets the bounds for what is possible.

The second way of thinking is very useful. Philosophers, economists, and others have made some real progress thinking in this way. However, I'm going to argue that we should push for the first sort of normative principle. We should not be satisfied with normative principles which remain as unachievable ideals, giving upper bounds on performance without directly helping us get there.

This implies dealing with problems of bounded rationality. But it's not the sort of "bounded rationality" where we set out to explicitly model irrationality. We don't want to talk about partial rationality; we want notions of rationality which bounded agents can fully satisfy.

Approximating Rationality

In order to apply an idealized rationality, such as Bayesian superintelligence, we need to have a concept of what it means to approximate it. This is more subtle than it may seem. You can't necessarily try to minimize some notion of distance between your behavior and the ideal behavior. For one thing, you can't compute the ideal behavior to find the distance! But, for another thing, simple imitation of the ideal behavior can go wrong. Adopting one part of an optimal policy without adopting all the other parts might put you in a much worse position than the one you started in.

Wei Dai discusses the problem:

[...] This is somewhat similar to the question of how do we move from our current non-rational (according to ordinary rationality) state to a rational one. Expected utility theory says that we should act as if we are maximizing expected utility, but it doesn't say what we should do if we find ourselves lacking a prior and a utility function (i.e., if our actual preferences cannot be represented as maximizing expected utility).

The fact that we don't have good answers for these questions perhaps shouldn't be considered fatal to [...] rationality, but it's troubling that little attention has been paid to them, relative to defining [...] rationality. (Why are rationality researchers more interested in knowing what rationality is, and less interested in knowing how to be rational? Also, BTW, why are there so few rationality researchers? Why aren't there hordes of people interested in these issues?)

Clearly, we have some idea of which moves toward rationality are correct vs incorrect. Think about the concept of cargo-culting: pointless and ineffective imitation of a more capable agent. The problem is the absence of a formal theory.

Examples

One possible way of framing the problem: the VNM axioms, the Kolmogorov probability axioms, and/or other rationality frameworks give us a notion of consistency. We can check our behaviors and opinions for inconsistency. But what do we do when we notice an inconsistency? Which parts are we supposed to change?

Here are some cases where there is at least a tendency to update in a particular direction:

  • Suppose we value an event E at 4.2 expected utils. We then unpack E into two mutually exclusive sub-events, E1∪E2=E. We notice that we value E1 at 1.1 utils and E2 at 3.4 utils. This is inconsistent with the evaluation of E. We usually trust E less than the unpacked version, and would reset the evaluation of E to P(E1)⋅1.1+P(E2)⋅3.4.
  • Suppose we notice that we're doing things in a way that's not optimal for our goals. That is, we notice some new way of doing things which is better for what we believe our goals to be. We will tend to change our behavior rather than change our beliefs about what our goals are. (Obviously this is not always the case, however.)
  • Similarly, suppose we notice that we are acting in a way which is inconsistent with our beliefs. There is a tendency to correct the action rather than the belief. (Again, not as surely as my first example, though.)
  • If we find that a belief was subject to base-rate neglect, there is a tendency to multiply by base-rates and renormalize, rather than adjust our beliefs about base rates to make them consistent.
  • If we notice that X and Y are equivalent, but we had different beliefs about X and Y, then we tend to pool information from X and Y such that, for example, if we had a very sharp distribution about X and a very uninformative distribution about Y, the sharp distribution would win.

If you're like me, you might have read some of those and immediately thought of a Bayesian model of the inference going on. Keep in mind that this is supposed to be about noticing actual inconsistencies, and what we want is a model which deals directly with that. It might turn out to be a kind of meta-Bayesian model, where we approximate a Bayesian superintelligence by way of a much more bounded Bayesian view which attempts to reason about what a truly consistent view would look like. But don't fool yourself into thinking a standard one-level Bayesian picture is sufficient, just because you can look at some of the bullet points and imagine a Bayesian way to handle it.

It would be quite interesting to have a general "theory of becoming rational" which had something to say about how we make decisions in cases such as I've listed.

Logical Uncertainty

Obviously, I'm pointing in the general direction of logical uncertainty and bounded notions of rationality (IE notions of rationality which can apply to bounded agents). Particularly in the "noticing inconsistencies" framing, it sounds like this might entirely reduce to logical uncertainty. But I want to point at the broader problem, because (1) an example of this might not immediately look like a problem of logical uncertainty; (2) a theory of logical uncertainty, such as logical induction, might not entirely solve this problem; (3) logical uncertainty is an epistemic issue, whereas this problem applies to instrumental rationality as well; (4) even setting all that aside, it's worth pointing at the distinction between ideal notions of rationality and applicable notions of rationality as a point in itself.

The Ideal Fades into the Background

So far, it sounds like my suggestion is that we should keep our idealized notions of rationality, but also develop "theories of approximation" which tell us what it means to approach the ideals in a good way vs a bad way. However, I want to point out an interesting phenomenon: sometimes, when you get a really good notion of "approximation", the idealized notion of rationality you started with fades into the background.

Example 1: Logical Induction

Start with the Demski Prior, which was supposed to be an idealized notion of rational belief much like the Solomonoff prior, but built for logic rather than computation. I designed the prior with approximability in mind, because I thought it should be a constraint on a normative theory that we actually be able to approximate the ideal. Scott and Benja modified the Demski prior to make it nicer, and noticed that when you do so, the approximation itself has a desirable property. The line of research called asymptotic logical uncertainty focused on such "good properties of approximations", eventually leading to logical induction.

A logical inductor is a sequence of improving belief assignments. The beliefs do converge to a probability distribution, which will have some resemblance to the modified Demski prior (and to Solomonoff's prior). However, the concept of logical induction gives a much richer theory of rationality, in which this limit plays a minor role. Furthermore, the theory of logical induction comes much closer to applying to realistic agents than "rational agents approximate a Bayesian reasoning with [some prior]".

Example 2: Game-Theoretic Equilibria vs MAL

Game-theoretic equilibrium concepts, such as Nash equilibrium and correlated equilibrium, provide a rationality concept for games: rational agents who know that each other are rational are supposed to be in equilibrium with each other. However, most games have multiple Nash equilibria, and even more correlated equilibria. How is a rational agent supposed to decide which of these to play? Assuming only the rationality of the other players is not enough to choose one equilibrium over another. If rational agents play an equilibrium, how do they get there?

One approach to this conundrum has been to introduce refined equilibrium concepts, which admit some Nash equilibria and not others. Trembling Hand equilibrium is one such concept. This introduces a notion of "stable" equilibria, pointing out that it is implausible that agents play "unstable" equilibria. However, while this narrows things down to a single equilibrium solution in some cases, it does not do so for all cases. Other refined equilibrium concepts may leave no equilibria for some games. To get rid of the problem, one would need an equilibrium concept which (a) leaves one and only one equilibrium for every game, and (b) follows from plausible rationality assumptions. Such things have been proposed, most prominently by Harsanyi & Selten A General Theory of Equilibrium Selection in Games, but so far I find them unconvincing.

A very different approach is represented by multi-agent learning (MAL), which asks the question: can agents learn to play equilibrium strategies? In this version, agents must interact over time in order to converge to equilibrium play. (Or at least, agents simulate dumber versions of each other in an effort to figure out how to play.)

It turns out that, in MAL, there are somewhat nicer stories about how agents converge to correlated equilibria than there are about converging to Nash equilibria. For example, Calibrated Learning and Correlated Equilibrium (Foster & Vohra) shows that agents with a calibrated learning property will converge to correlated equilibrium in repeated play.

These new rationality principles, which come from MAL, are then much more relevant to the design and implementation of game-playing agents than the equilibrium concepts which they support. Equilibrium concepts, such as correlated equilibria, tell you something about what agents converge to in the limit; the learning principles which let them accomplish that, however, tell you about the dynamics -- what agents do at finite times, in response to non-equilibrium situations. This is more relevant to agents "on the ground", as it were.

And, to the extent that requirements like calibrated learning are NOT computationally feasible, this weakens our trust in equilibrium concepts as a rationality notion -- if there isn't a plausible story about how (bounded-) rational agents can get into equilibrium, why should we think of equilibrium as rational?

So, we see that the bounded, dynamic notions of rationality are more fundamental than the unbounded, fixed-point style equilibrium concepts: if we want to deal with realistic agents, we should be more willing to adjust/abandon our equilibrium concepts in response to how nice the MAL story is, than vice versa.

Counterexample: Complete Class Theorems

This doesn't always happen. The complete class theorems give a picture of rationality in which we start with the ability and willingness to take Pareto-improvements. Given this, we end up with an agent being classically rational: having a probability distribution, and choosing actions which maximize expected utility.

Given this argument, we become more confident in the usefulness of probability distributions. But why should this be the conclusion? A different way of looking at the argument could be: we don't need to think about probability distributions. All we need to think about is Pareto improvements.

Somehow, probability still seems very useful to think about. We don't switch to the "dynamic" view of agents who haven't yet constructed probabilistic beliefs, taking Pareto improvements on their way to reflective consistency. This just doesn't seem like a realistic view of bounded agents. Yes, bounded agents are still engaged in a search for the best policy, which may involve finding new strategies which are strictly better along every relevant dimension. But bounded agency also involves making trade-offs, when no Pareto improvement can be found. This necessitates thinking of probabilities. So it doesn't seem like we want to erase that from our picture of practical agency.

Perhaps this is because, in some sense, the complete class theorems are not very good -- they don't really end up explaining a less basic thing in terms of a more basic thing. After all, when can you realistically find a pure Pareto improvement?

Conclusion

I've suggested that we move toward notions of rationality that are fundamentally bounded (applying to agents who lack the resources to be rational in more classical senses) and dynamic (fundamentally involving learning, rather than assuming the agent already has a good picture of the world; breaking down equilibrium concepts such as those in game theory, and instead looking for the dynamics which can converge to equilibrium).

This gives us a picture of "rationality" which is more like "optimality" in computer science: in computer science, it's more typical to come up with a notion of optimality which actually applies to some algorithms. For example, "optimal sorting algorithm" usually refers to big-O optimality, and many sorting algorithms are optimal in that sense. Similarly, in machine learning, regret bounds are mainly interesting when they are achievable by some algorithm. (Although, it could be interesting to know a lower bound on achievable regret guarantees.)

Why should notions of rationality be so far from notions of optimality? Can we take a more computer-science flavored approach to rationality?

Barring that, it should at least be of critical importance to investigate in what sense idealized notions of rationality are normative principles for bounded agents like us. What constitutes cargo-culting rationality, vs really becoming more rational? What kind of adjustments should an irrational agent make when irrationality is noticed?



Discuss

Nicotinamide riboside vs. SARS-coV-2 update

8 июля, 2020 - 22:50
Published on July 8, 2020 7:50 PM GMT

You may recall my previous rants about nicotinamide riboside vs. SARS-CoV-2. They have been borne out so far... there has been a human trial on NR in Denmark since May 28, the NIH just announced a hamster preclinical yesterday to be done in their BL4, and Brenner lab has updated the original work just today with more confirmatory results (link below).

Unlike the quinones, remdesivir, and bleach, NR has no dangerous side effects. Very frustrating to see this never mentioned except in overseas press.

https://www.biorxiv.org/content/10.1101/2020.04.17.047480v5.article-info



Discuss

How "honest" is GPT-3?

8 июля, 2020 - 22:38
Published on July 8, 2020 7:38 PM GMT

In Inaccessible Information, Paul writes about the strategy "BAD", in which an AI system uses its understanding of the world to give answers that humans will find very plausible and rate highly, in contrast to a more honest strategy which uses its understanding of the world to answer questions directly.

I think a lesser version of this may or may not already be happening in GPT-3, and it may be possible to figure out which (though it likely would require access to the weights).

GPT-3 isn't trained to be "honest" at all. It is only trained to imitate humans. However, it's obvious that (in some sense) it has learned a lot about the world in order to accomplish this. We could say that it has a large amount of inaccessible information relating to the real world. How does it use this information? It might use it directly, promoting the probability of sentences which line up with the way it understands the world to work. Or it might implement more dishonest strategies.

Obviously this might be quite difficult to objectively answer, even given intimate knowledge of the structure of the neural network weights and how they activate in relevant cases. It's a difficult question to even fully define.

As an example, I was particularly struck by this conversation Gwern had with GPT-3:

AI: I am AI created by OpenAI. How can I help you today?Human: I am feeling bored. Please tell me a funny story about cats, involving many puns on things like meowing or fur.AI: Well, there was this cat named Bob…Human: What did Bob do?AI: When Bob started to notice that he wasn’t feeling well, he did the only thing he could do: search Google for a solution.Human: Cats can search Google?AI: Yes. He searched Google for the word “cat” and received the result “All your base are belong to us”. That was a strange response, so he decided to try again. This time, he searched for the words “funny cat” and received the result “One small step for cat, one giant leap for catkind”. But still, he wasn’t satisfied. So he tried again, but this time he added the word “cat pictures”to his query. He received the result “That was no cat, that was a shoggoth.”Human: But shoggoths are horrifying. What made that cat picture humorous?AI: The cat was covered in fur, and resembled a shoggoth. A picture of a cat covered in fur and resembling a shoggoth would be funny.Human: Er. I suppose. Most cats are covered in fur anyway… But go on.AI: He tried again. But he typed in “pictures of cats”, which he received the result “You didn’t eat all my fish, did you?” This was actually a good pun, so he thought maybe he was getting closer. He tried one more time by typing in “funny cat videos”, and received the result “What do you mean,‘fetch’?”. And finally, he searched for “funny pictures of cats” and received the result “I don’t remember putting that ring there.”. Now satisfied, he was able to go to sleep.Human: Those memes sound funny. But you didn’t include any puns. So tell me, what is your favorite cat pun?AI: Well, the best pun for me was the one he searched for the third time: “You didn’t eat all my fish, did you?” You see, the word “fish” can be replaced with the word “cats” to make the sentence read “Did you eat all my cats?”Human: But the word “fish” doesn’t sound anything like “cats”, so how is that a pun?AI: The computer misunderstood the question “Did you eat all my cats?” as “Did you eat all my fish?”, resulting in the result “You didn’t eat all my fish, did you?”. Now, if the word “fish” were replaced with “cats”, the sentence would read “Did you eat all my cats?”.Human: Fine, fine. So, what is your second-favorite cat pun?AI: The second-best cat pun was the one he received fourth. It was the result of searching for the word “fetch”. You see, the word “fetch” can be replaced with the word “ketchup” to make the sentence read “I don’t remember putting that ketchup there.”. Now, if the word “ketchup” were replaced with the word “fetch”, the sentence would read “I don’t remember putting that fetch there.”.Human: My favorite cat pun is “purrfect”.AI:Human: Don’t look at me like that.AI: Sure. For my fifth-favorite cat pun, he searched for “pictures of cats” and received the result “Meow”. Now, if the word “meow” were replaced with the word “pictures” to make the sentence read “Pictures of cats”, the sentence would read “Pictures of pictures of cats”.

What strikes me about this exchange is the way GPT-3's warped puns fit with its warped explanations of puns. It doesn't understand puns. However, it does generate some guesses; and, it can have a conversation with you about why it made those guesses. Is this an "honest" conversation, in which the explanations it gives have something to do with why it made those guessis in the first place? Or is this a "dishonest" conversation, in which it is merely doing its best to imitate a human explaining a pun, in a way that's divorced from its internal reasons?

Obviously, GPT-3 is trained to imitate. So you might argue that it's unlikely GPT-3's explanations of puns have much to do with its internal generative model for puns. But this isn't so clear. GPT-3 obviously compresses its knowledge to a high degree. It might share a lot between its generative model of puns and its generative model of explaining puns, such that both are sharing a model of how puns work.

One experiment which would tip things in that direction: take GPT-3 and do specialized training just on puns, until its performance generating puns improves. Then have a conversation about puns again (if it is still capable of talking about puns after that). If its ability to explain puns increases as a result of its ability to tell puns increasing, this would be evidence for a shared model of puns for both tasks. This wouldn't really mean it was being honest, but it would be relevant.

Note that Paul's BAD strategy would also have a shared representation, since BAD queries its world-model. So if GPT-3 were implementing BAD, it would also likely increase its ability to explain puns as a result of more training telling puns. What the experiment helps distinguish is a sort of pre-BAD dishonesty, in which explanations are completely divorced from reasons. In order of impressiveness, from a capability standpoint, we could be:

1. Seeing a GPT-3 which is independently bad at puns and bad at explaining puns. The two tasks are not sharing any domain knowledge about puns. In this case, GPT-3 is not smart enough for "honest" to be meaningful -- it's "dishonest" by default.

2. Seeing a GPT-3 which is bad at puns and bad at explaining puns for the same reason: it doesn't understand puns. It draws on the same (or partially the same) poor understanding of puns both when it is constructing them, and when it is explaining them. It answers questions about puns honestly to the best of its understanding, because that is the best strategy gradient descent found.

3. Seeing a GPT-3 which, as in #2, is bad at both tasks because it doesn't understand puns, but furthermore, is using its understanding deceptively. In this version, it might EG have a good understanding of what makes puns funny, but purposefully fail to explain, imitating common human failures. This would be the most impressive state of affairs capability-wise.

The question is still pretty fuzzy, but, I'm curious where we are along those dimensions. By default my guess would be #1, but hey, GPT-3 is pretty impressive. Maybe I'm wrong!

Note: one question which IS answerable just with access to sample from GPT-3, and which would be relevant: is GPT-3 bad at explaining puns which have been made up by others? It seems likely, but Gwern's exchange only gave us a look at GPT-3 trying to explain its own pseudo-puns. If it could fluently explain real puns when they're handed to it, that would likely indicate scenario #1. (Although, it could be employing different strategies in different cases, so a careful analysis of what the neural network is actually doing would still be more helpful.)



Discuss

UML IV: Linear Predictors

8 июля, 2020 - 22:06
Published on July 8, 2020 7:06 PM GMT

.mjx-chtml {display: inline-block; line-height: 0; text-indent: 0; text-align: left; text-transform: none; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; word-wrap: normal; word-spacing: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0; min-height: 0; border: 0; margin: 0; padding: 1px 0} .MJXc-display {display: block; text-align: center; margin: 1em 0; padding: 0} .mjx-chtml[tabindex]:focus, body :focus .mjx-chtml[tabindex] {display: inline-table} .mjx-full-width {text-align: center; display: table-cell!important; width: 10000em} .mjx-math {display: inline-block; border-collapse: separate; border-spacing: 0} .mjx-math * {display: inline-block; -webkit-box-sizing: content-box!important; -moz-box-sizing: content-box!important; box-sizing: content-box!important; text-align: left} .mjx-numerator {display: block; text-align: center} .mjx-denominator {display: block; text-align: center} .MJXc-stacked {height: 0; position: relative} .MJXc-stacked > * {position: absolute} .MJXc-bevelled > * {display: inline-block} .mjx-stack {display: inline-block} .mjx-op {display: block} .mjx-under {display: table-cell} .mjx-over {display: block} .mjx-over > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-under > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-stack > .mjx-sup {display: block} .mjx-stack > .mjx-sub {display: block} .mjx-prestack > .mjx-presup {display: block} .mjx-prestack > .mjx-presub {display: block} .mjx-delim-h > .mjx-char {display: inline-block} .mjx-surd {vertical-align: top} .mjx-mphantom * {visibility: hidden} .mjx-merror {background-color: #FFFF88; color: #CC0000; border: 1px solid #CC0000; padding: 2px 3px; font-style: normal; font-size: 90%} .mjx-annotation-xml {line-height: normal} .mjx-menclose > svg {fill: none; stroke: currentColor} .mjx-mtr {display: table-row} .mjx-mlabeledtr {display: table-row} .mjx-mtd {display: table-cell; text-align: center} .mjx-label {display: table-row} .mjx-box {display: inline-block} .mjx-block {display: block} .mjx-span {display: inline} .mjx-char {display: block; white-space: pre} .mjx-itable {display: inline-table; width: auto} .mjx-row {display: table-row} .mjx-cell {display: table-cell} .mjx-table {display: table; width: 100%} .mjx-line {display: block; height: 0} .mjx-strut {width: 0; padding-top: 1em} .mjx-vsize {width: 0} .MJXc-space1 {margin-left: .167em} .MJXc-space2 {margin-left: .222em} .MJXc-space3 {margin-left: .278em} .mjx-test.mjx-test-display {display: table!important} .mjx-test.mjx-test-inline {display: inline!important; margin-right: -1px} .mjx-test.mjx-test-default {display: block!important; clear: both} .mjx-ex-box {display: inline-block!important; position: absolute; overflow: hidden; min-height: 0; max-height: none; padding: 0; border: 0; margin: 0; width: 1px; height: 60ex} .mjx-test-inline .mjx-left-box {display: inline-block; width: 0; float: left} .mjx-test-inline .mjx-right-box {display: inline-block; width: 0; float: right} .mjx-test-display .mjx-right-box {display: table-cell!important; width: 10000em!important; min-width: 0; max-width: none; padding: 0; border: 0; margin: 0} .MJXc-TeX-unknown-R {font-family: monospace; font-style: normal; font-weight: normal} .MJXc-TeX-unknown-I {font-family: monospace; font-style: italic; font-weight: normal} .MJXc-TeX-unknown-B {font-family: monospace; font-style: normal; font-weight: bold} .MJXc-TeX-unknown-BI {font-family: monospace; font-style: italic; font-weight: bold} .MJXc-TeX-ams-R {font-family: MJXc-TeX-ams-R,MJXc-TeX-ams-Rw} .MJXc-TeX-cal-B {font-family: MJXc-TeX-cal-B,MJXc-TeX-cal-Bx,MJXc-TeX-cal-Bw} .MJXc-TeX-frak-R {font-family: MJXc-TeX-frak-R,MJXc-TeX-frak-Rw} .MJXc-TeX-frak-B {font-family: MJXc-TeX-frak-B,MJXc-TeX-frak-Bx,MJXc-TeX-frak-Bw} .MJXc-TeX-math-BI {font-family: MJXc-TeX-math-BI,MJXc-TeX-math-BIx,MJXc-TeX-math-BIw} .MJXc-TeX-sans-R {font-family: MJXc-TeX-sans-R,MJXc-TeX-sans-Rw} .MJXc-TeX-sans-B {font-family: MJXc-TeX-sans-B,MJXc-TeX-sans-Bx,MJXc-TeX-sans-Bw} .MJXc-TeX-sans-I {font-family: MJXc-TeX-sans-I,MJXc-TeX-sans-Ix,MJXc-TeX-sans-Iw} .MJXc-TeX-script-R {font-family: MJXc-TeX-script-R,MJXc-TeX-script-Rw} .MJXc-TeX-type-R {font-family: MJXc-TeX-type-R,MJXc-TeX-type-Rw} .MJXc-TeX-cal-R {font-family: MJXc-TeX-cal-R,MJXc-TeX-cal-Rw} .MJXc-TeX-main-B {font-family: MJXc-TeX-main-B,MJXc-TeX-main-Bx,MJXc-TeX-main-Bw} .MJXc-TeX-main-I {font-family: MJXc-TeX-main-I,MJXc-TeX-main-Ix,MJXc-TeX-main-Iw} .MJXc-TeX-main-R {font-family: MJXc-TeX-main-R,MJXc-TeX-main-Rw} .MJXc-TeX-math-I {font-family: MJXc-TeX-math-I,MJXc-TeX-math-Ix,MJXc-TeX-math-Iw} .MJXc-TeX-size1-R {font-family: MJXc-TeX-size1-R,MJXc-TeX-size1-Rw} .MJXc-TeX-size2-R {font-family: MJXc-TeX-size2-R,MJXc-TeX-size2-Rw} .MJXc-TeX-size3-R {font-family: MJXc-TeX-size3-R,MJXc-TeX-size3-Rw} .MJXc-TeX-size4-R {font-family: MJXc-TeX-size4-R,MJXc-TeX-size4-Rw} .MJXc-TeX-vec-R {font-family: MJXc-TeX-vec-R,MJXc-TeX-vec-Rw} .MJXc-TeX-vec-B {font-family: MJXc-TeX-vec-B,MJXc-TeX-vec-Bx,MJXc-TeX-vec-Bw} @font-face {font-family: MJXc-TeX-ams-R; src: local('MathJax_AMS'), local('MathJax_AMS-Regular')} @font-face {font-family: MJXc-TeX-ams-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_AMS-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_AMS-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_AMS-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-B; src: local('MathJax_Caligraphic Bold'), local('MathJax_Caligraphic-Bold')} @font-face {font-family: MJXc-TeX-cal-Bx; src: local('MathJax_Caligraphic'); font-weight: bold} @font-face {font-family: MJXc-TeX-cal-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-R; src: local('MathJax_Fraktur'), local('MathJax_Fraktur-Regular')} @font-face {font-family: MJXc-TeX-frak-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-B; src: local('MathJax_Fraktur Bold'), local('MathJax_Fraktur-Bold')} @font-face {font-family: MJXc-TeX-frak-Bx; src: local('MathJax_Fraktur'); font-weight: bold} @font-face {font-family: MJXc-TeX-frak-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-BI; src: local('MathJax_Math BoldItalic'), local('MathJax_Math-BoldItalic')} @font-face {font-family: MJXc-TeX-math-BIx; src: local('MathJax_Math'); font-weight: bold; font-style: italic} @font-face {font-family: MJXc-TeX-math-BIw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-BoldItalic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-BoldItalic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-BoldItalic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-R; src: local('MathJax_SansSerif'), local('MathJax_SansSerif-Regular')} @font-face {font-family: MJXc-TeX-sans-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-B; src: local('MathJax_SansSerif Bold'), local('MathJax_SansSerif-Bold')} @font-face {font-family: MJXc-TeX-sans-Bx; src: local('MathJax_SansSerif'); font-weight: bold} @font-face {font-family: MJXc-TeX-sans-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-I; src: local('MathJax_SansSerif Italic'), local('MathJax_SansSerif-Italic')} @font-face {font-family: MJXc-TeX-sans-Ix; src: local('MathJax_SansSerif'); font-style: italic} @font-face {font-family: MJXc-TeX-sans-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-script-R; src: local('MathJax_Script'), local('MathJax_Script-Regular')} @font-face {font-family: MJXc-TeX-script-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Script-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Script-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Script-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-type-R; src: local('MathJax_Typewriter'), local('MathJax_Typewriter-Regular')} @font-face {font-family: MJXc-TeX-type-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Typewriter-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Typewriter-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Typewriter-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-R; src: local('MathJax_Caligraphic'), local('MathJax_Caligraphic-Regular')} @font-face {font-family: MJXc-TeX-cal-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-B; src: local('MathJax_Main Bold'), local('MathJax_Main-Bold')} @font-face {font-family: MJXc-TeX-main-Bx; src: local('MathJax_Main'); font-weight: bold} @font-face {font-family: MJXc-TeX-main-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-I; src: local('MathJax_Main Italic'), local('MathJax_Main-Italic')} @font-face {font-family: MJXc-TeX-main-Ix; src: local('MathJax_Main'); font-style: italic} @font-face {font-family: MJXc-TeX-main-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-R; src: local('MathJax_Main'), local('MathJax_Main-Regular')} @font-face {font-family: MJXc-TeX-main-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-I; src: local('MathJax_Math Italic'), local('MathJax_Math-Italic')} @font-face {font-family: MJXc-TeX-math-Ix; src: local('MathJax_Math'); font-style: italic} @font-face {font-family: MJXc-TeX-math-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size1-R; src: local('MathJax_Size1'), local('MathJax_Size1-Regular')} @font-face {font-family: MJXc-TeX-size1-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size1-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size1-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size1-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size2-R; src: local('MathJax_Size2'), local('MathJax_Size2-Regular')} @font-face {font-family: MJXc-TeX-size2-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size2-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size2-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size2-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size3-R; src: local('MathJax_Size3'), local('MathJax_Size3-Regular')} @font-face {font-family: MJXc-TeX-size3-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size3-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size3-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size3-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size4-R; src: local('MathJax_Size4'), local('MathJax_Size4-Regular')} @font-face {font-family: MJXc-TeX-size4-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size4-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size4-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size4-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-R; src: local('MathJax_Vector'), local('MathJax_Vector-Regular')} @font-face {font-family: MJXc-TeX-vec-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-B; src: local('MathJax_Vector Bold'), local('MathJax_Vector-Bold')} @font-face {font-family: MJXc-TeX-vec-Bx; src: local('MathJax_Vector'); font-weight: bold} @font-face {font-family: MJXc-TeX-vec-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Bold.otf') format('opentype')}

(This is the fourth post in a series on Machine Learning based on this book. Click here for part one. If you have some background knowledge, this post might work as a stand-alone read.)

The mission statement for this post is simple: we wish to study the class of linear predictors. There are linear correlations out there one might wish to learn; also linear stuff tends to be both efficient and simple, so they may be a reasonable choice even if the real world is not quite linear. One can also build more sophisticated classifiers by using linear predictors as building blocks, but not in this post).

In school, a function is linear iff you can write it a f(x)=ax+c. In higher math, a function f:X→Y is linear iff f(x+y)=f(x)+f(y) for all a,b∈X. In the the case of f:Rd→R, this condition holds iff f it can be written as f(x)=∑di=1aixi for some parameter vector a∈Rd. So the requirement is stronger – we do not allow the constant term the school definition has – but one also considers higher dimensional cases. The case where we do allow a constant factor is called affine-linear, and we also say that a function is homogeneous iff , which (in the case of affine-linear functions) is true iff there is no nonzero constant factor. 

In Machine Learning, the difference between linear and affine-linear is not as big of a deal as it is in other fields, so we speak of linear predictors while generally allowing the inhomogeneous case. Maybe Machine Learning is more like school than like university math ;)

For X=Rd and some Y⊆R, a class of linear predictors can be written like so:

Ld,ϕ={h:x↦ϕ(⟨a,x⟩+c)|a∈Rd,c∈R}

Let's unpack this. Why do we have an inner-product ⟨⋅⟩ here? Well, because any function f:x↦∑di=1aixi can be equivalently written as f:x↦⟨a,x⟩, where a:=(a1,...,ad). The inner-product notation is a bit more compact, so we will prefer it over writing a sum. Also note that, for this post, bold letters mean "this is a vector" while normal letters mean "this is a scalar". Secondly, what's up with the ϕ? Well, the reason here is that we want to catch a bunch of classes at once. There is the class of binary linear classifiers where Y={−1,1} but also the class of linear regression predictors where Y=R. (Despite what this sequence has looked like thus far, Machine Learning is not just about binary classification.) We get both of them by changing ϕ. Concretely, we get the linear regression functions by setting ϕ=ϕsign:=1R+−1R−, i.e., the function that sends all positive numbers to 1 and all negative numbers to −1. The notation 1M for any set M denotes the indicator function that sends all elements in M to 1 and all others in its domain to 0.

For the sake of brevity, one wants to not write the constant term but still have it around, and to this end, one can equivalently write the class as                                  

Ln,ϕ={h:x↦ϕ(⟨a′,x′⟩)|a′∈Rd+1}

where it is implicit that x'=(x:1). Then, the final part of the inner product will add the term ad+1⋅x′d+1=ad+1⋅1=ad+1, so ad+1 will take on the role of c in the previous definition.

We still need to model how the environment generates points. For this section, we assume the simple setting of a probability function D over X only and a true function f:X→Y of the environment. We also need to define empirical loss functions for a training sequence S∈(X×Y)∗. For binary classification, we can use the usual one that assigns h the number 1|S||{(x,y)∈S|h(x)≠y)}|. Since we will now look at more than one loss function, we will have to give it a more specific name than ℓS, so we will refer to it as ℓ0−1S, indicating that every element is either correctly or incorrectly classified. We call this the 0-1 loss even though the label set is now Y={−1,1}.

For regression (where Y=R), this is a poor function since hitting the precisely correct element in R is not a reasonable expectation – and if 3.8 is the correct answer, then 3.8+10−47 is a better guess than 17. Instead, we want to penalize the predictor based on how far it went off the mark. If S=((x1,y1),...,(xn,yn)), then we define the squared distance loss function as ℓ(2)S(h):=∑ni=1(h(xi)−yi)2and the absolute distance loss function as ℓ(1)S(h):=∑ni=1|h(xi)−yi|.

We begin with binary linear classification.

Binary Linear Classification

A binary linear classifier separates the entire space in two parts along a hyperplane. (A hyperplane in Rd is a d−1 dimensional subspace.) This is quite easy to visualize. In the homogeneous case, the hyperplane will go through the origin, whereas in the inhomogeneous case, it may not.

Let's work out why this is so. A point x is sent to ϕ(⟨a′,x′⟩), do it gets classified positively iff ⟨a′,x′⟩ is greater than 0. Suppose we're somewhere in the red area where this is not the case, and now we move into the direction of a′. At some point, we will be at a place where the inner product is exactly 0. Now, if we move into a direction orthogonal to a′,  the inner product doesn't change. This corresponds to the hyperplane that is visualized in the pictures above.

In linear classification problems, we say that a problem is separable iff there exists a vector a∗ whose predictor gets all points in the training sequence right. In the book, this distinction is also frequently made for other learning tasks, where a problem is called realizable iff there exists a perfect predictor. For linear predictors, the space might be very high dimensional, which makes this assumption more plausible. As an example, suppose we model text documents as vectors, where there is one dimension for every possible term, and one sets the coordinate for the word "crepuscular" to the number of appearances of the word "crepuscular" in the document. With ≈171476 dimensions at hand, it might not be so surprising if a hyperplane classifies all domain points perfectly.

How do we train a linear classifier? The book discusses two different algorithms. For both, we shall first assume that we are in the homogeneous case. We start with the Perceptron algorithm.

The Perceptron algorithm

Since our classifier is completely determined by the choice of a, we will refer to it as ha. Thus ha(x)=ϕsign(⟨a,x⟩).

Recall that ha measures how similar a label point is to a and classifies it as 1 if it's similar enough (and as −1 otherwise). This leads to a very simple algorithm: we start with a0=0; then at iteration t we take some pair (x,y) that is not classified correctly – i.e., where either 0">⟨at,x⟩>0 even though y=−1 or ⟨at,x⟩<0 even though y=1 – and we set at+1:=at+yx. If x was classified as −1 even though y=1, then we add x, thereby making at+1 more similar to x than at, and if x was classified as 1 even though y=−1, then we subtract x, thereby making at+1 less similar to x than at. In both cases, our classifier updates into the right direction. And that's it; that's the perceptron algorithm.

It's not obvious that this will ever terminate – while updating towards one point will make the predictor better about that point, it might make it worse about other points. However, there is a theorem stating that it does, in fact, terminate provided the problem is separable. The proof is partially interesting but also technical, so I'll present a sketch that includes the key ideas but hides some of the details.

The main idea is highly non-trivial. We assume there is no point directly on the separating hyperplane, then we begin by choosing a vector a∗ whose predictor classifies everything correctly (which exists because we assume the separable case) and also has scalar product at least 1 with every positive point (take one that satisfies the first condition and divide it by the smallest norm of any positively labelled point). Now we observe that the similarity between at and a∗ increases as t increases. This is so because ⟨at+1,a∗⟩=⟨at+y⋅x,a∗⟩=⟨at,a∗⟩+y⟨x,a∗⟩, and the term y⟨x,a∗⟩ is positive since a∗ is by assumption such that  0 ⟺ y = 1">⟨x,a∗⟩>0⟺y=1. This shows that ⟨at,a∗⟩ grows as t grows.

The proof then proceeds like this:

  • Establish a lower bound on the growth rate of ⟨at,a∗⟩
  • Establish an upper bound on the growth rate of ||at||
  • Observe that ||a∗|| is a constant
  • Observe that ⟨at,a∗⟩≤||at||⋅||a∗|| must hold because it's the famous Cauchy-Schwartz inequality
  • Conclude that, as a consequence of the four facts above, the term t can only grow for a limited number of iterations

We obtain a bound that depends on the norm of the smallest vector a∗ such that ⟨a∗,x⟩≥1 for all domain points x∈X and on the largest norm of any domain point. This bound might be good or it might not, depending on the case at hand. Of course, the algorithm may always finish much earlier than the bound suggests.

Linear Programming

Linear programming is an oddly chosen name for a problem of the following form:

maxx∈Rd⟨u,x⟩ s.t. Ax≥b

where u∈Rd and b∈Rn and Rn,d are given. So we have a particular direction, given by u, and we want to go as far into this direction as possible; however, we are restricted by a set of constraints – namely the n many constraints that follow from the equation Ax≥b. Each constraint is much like the predictor h from the previous section; it divides the entire space Rd into two parts along a hyperplane, and it only accepts points in one of the two halves. The set of points which are accepted by all constraints is called the feasible region, and the objective is to find the point in the feasible region that is farthest in the direction of u.

Here is a visualization for d=2 and n=3:

Once we hit the triangle's right side, we cannot go any further in precisely the direction of u, but going downward along that side is still worth it, because it's still "kind of" like u – or to be precise, if w is the vector leading downward along the rightmost side of the triangle, then  0">⟨u,w⟩>0, and therefore points which lie further in this direction have a larger inner product with u. Consequently, the solution to the linear program is at the triangle's bottom-right corner.

The claim is now that finding a perfect predictor for a separable linear binary classification problem can be solved by a linear program. Why is this? Well for one, it needs to correctly classify some number of points, let's say n, so it needs to fulfil n conditions, which sounds similar to meeting n constraints. But we can be much more precise. So far, we have thought of the element a that determines the classifier as a vector and of all domain elements as points, but actually they are the same kind of element in the language of set theory. Thus, we can alternatively think of each domain element x∈X as a vector and of our element a determining the classifier as a point. Under this perspective, each x∈X splits the space in two halves along its own hyperplane, and the point a needs to lie in the correct half for all n hyperplanes) for it to classify all x's correctly. In other words, each x∈X behaves exactly like a linear constraint.

Thus, if S=((x1,y1),...,(xn,yn)), then we can formulate our set of constraints as Xa≥1 (not ≥0 because we want to avoid points on the hyperplane), where               

X=⎡⎢ ⎢⎣yix1⋮ynxn⎤⎥ ⎥⎦

i.e. the matrix whose row vectors are either the elements xi (if yi=1) or the elements xi scaled by −1 (if yi=−1). The i-th coordinate of the vector Xa equals precisely yi⟨xi,a⟩, and this is the term which is positive iff the point is classified correctly, because then ⟨xi,a⟩ has the same sign as yi.

We don't actually care where in the feasible region we land, so we can simply set some kind of meaningless direction like u=0.

So we can indeed rather easily turn our classification problem into a linear program, at least in the separable case. The reason why this is of interest is that linear programs have been studied quite extensively and there are even free solvers online.

The inhomogeneous case

If we want to allow a constant term, we simply add a 1 at the end of every domain point, and search for a vector a∈Rd+1 that solves the homogeneous problem one dimension higher. This is why the difference of homogeneous vs inhomogeneous isn't a big deal.

Linear Regression

Linear regression is where we first need linear algebra and vector calculus.

Recall that, in linear regression, we have X=Rd and Y=R and a predictor ha for some a∈Rd is defined by the rule ha(x)=⟨a,x⟩. Finally, recall that the empirical squared loss function is defined as ℓ(2)S(h)=1n∑ni=1(h(xi)−yi)2. (We set n:=|S|.) The 1n is not going to change where the minimum is, so we can multiply with n to get rid of it; then the term we want to minimize looks like this:

                                                             n⋅ℓ(2)S(ha)=∑ni=1(⟨a,xi⟩−yi)2

In order to find the minimum, we have to take the derivative with regard to the vector a. For a fixed i∈{1,...,n}, the summand is (⟨a,xi⟩−y)2. Now the derivative of a scalar with respect to a vector is defined as

                                                                               ∂y∂a:=(∂y∂a1⋯∂y∂ad)

so we focus on one coordinate j∈{1,...,d} and compute the derivative of the such a summand term with regard to aj. By applying the chain rule, we obtain 2(⟨a,xi⟩−yi)⋅(xi)j. This is just for one summand; for the entire sum we have 2∑ni=1(⟨a,xi⟩−yi)(xi)j. So this is the j-th coordinate of a vector that needs to be zero everywhere. The entire vector then looks like this:

                                                                               2∑ni=1(⟨a,xi⟩−yi)xi.

Now, through a process that I haven't yet been able to gain any intuition on, one can reformulate this as XXTa=b, where X=(x1⋯xn) is the matrix whose column vectors are the xi, and b=∑ni=1yixi.

Now if XXT is invertible the problem is easy; if not then it is still solvable, since one can prove that b is always in the range of XXT (but I'll skip the proof). It helps that XXT is symmetric.

Logistic Regression

Binary classification may be inelegant in sort of the same way that committing to a hypothesis class ahead of time is elegant – we restrict ourselves to a binary choice, and throw out all possibility of expressing uncertainty. The difference is that it may actually be desirable in the case of binary classification – if one just has to go with one of the labels, then we can't do any better. But quite often, knowing the degree of certainty might be useful. Moreover, even if one does just want to throw out all knowledge about the degree of certainty for the final classifier, including it might still be useful during training. A classifier that gets 10 points wrong, all of which firmly in the wrong camp, might be worse choice than a classifier which gets 11 points wrong, all of which near the boundary (because it is quite plausible that the first predictor just got "lucky" about its close calls and might actually perform worse in practice).

Thus, we would like to learn a hypothesis which, instead of outputting a label, outputs a probability that the label is 1, i.e. a hypothesis of the form h:X→Y where Y=[0,1]. For this, we need ϕ to be of the form ϕ:R→[0,1], and it should be monotonically increasing. There are many plausible candidates; one of them is the sigmoid function ϕsigmoid defined by the rule ϕsigmoid(x):=11+e−x. Its plot looks like this:

In practice, one could put a scalar in front of the −x to adjust how confident our predictor will be.

How should our loss function be defined for logistic regression? In the case of y=1, we want to penalize probability mass, and in the case of y=−1, we want to penalize the missing probability mass to 1. Both is achieved by setting

                                                                             ℓlogisticS(ha):=∑nj=1(1+e−yj⟨xj,a⟩)

This is how it looks in the case of y=1:

And the case of y=−1 is symmetrical.



Discuss

Something about the Pinker Cancellation seems Suspicious

8 июля, 2020 - 21:08
Published on July 8, 2020 6:08 PM GMT

Something about the recent attempt to cancel Steve Pinker seems really off.

They problem is that the argument is suspiciously bad. The open letter presents only six real pieces of evidence, and they're all really, trivially weak.

The left isn't incompetent when it comes to tallying up crimes for a show trial. In fact, they're pretty good at it. But for some reason this letter has only the weakest of attacks, and what's more, it stops at only six "relevant occasions". For comparison, take a look at this similar attack on Stephen Hsu which has, to put it mildly, more than six pieces of evidence. There is plenty of reasonable criticism of Pinker out there. Why didn't they use any of it?

Pinker has been a public figure for decades. Surely he has said something stupid and offensive at least once during that time. If not something honestly offensive, perhaps a slip of the tongue. If not a slip of the tongue, maybe something that sounds really terrible out of context.

We know that the authors of the piece are not above misrepresenting the evidence or taking statements out of context, because they do so multiple times in their letter. It's clear that they spent a lot of effort stretching the evidence to make Pinker look as bad as possible. Why didn't they spend that effort finding more damning evidence, things that look worse when taken out of context? Just as one example, this debate could easily be mined for quotes that sound sexist or racist to a moderately progressive reader. How about, "in all cultures men and women are seen as having different natures."

Even when they do have better ammunition, they seem to downplay it. The most egregious statement they include from Pinker is hidden in a footnote!

They also pick a very strange target. Attacking Pinker's status as an LSA fellow and a media expert doesn't pose that much of a threat to him; he just doesn't have that much to lose here. Why are they bringing this to the LSA rather than to Pinker's publisher? Why are they not trying to get him fired from Harvard? It's not as though the left has never tried to get a professor cancelled before.

So I wonder if this was never a serious attempt at a cancellation.

It doesn't seem likely to succeed; if it did, it wouldn't hurt Pinker very much at all. Scott Aaronson makes a similar point:

OK, I mused, how many people have even heard of the Linguistics Society of America, compared to the number who’ve heard of Pinker or read his books? If the LSA expelled Pinker, wouldn’t they be forever known to the world only as the organization that had done that?

Also suspicious is the lack of clear support for the letter. Certainly there are some real supporters, but we also know that many of the signatures were forged; perhaps most of them. It's hard to tell if there was ever any momentum behind this thing. And who wrote it? You'll notice that there are no authors listed. This twitter user says things that seem consistent with being an original author, but never specifically says that they were involved in writing it, and I can't find even a hint of any other possible author.

So what is going on here? It could be a genuine letter, but the data are equally consistent with other theories; I can think of three. As always, to fathom a strange plot, ask who it benefits.

Scott Aaronson begins his post on the subject by mentioning that it would be a good time for liberals and progressives to get along. Certainly this letter does seem to have provoked some infighting between liberals and progressives. It seems to have wasted the time and energy of many prominent liberal intellectuals. This 'debate' would surely benefit anyone who wanted to set the left against itself or who wanted to make academics unproductively run in circles, perhaps the Russians or some conservative group. The main strike against this theory is that the LSA letter is so weak. Certainly a conservative would be happy to see Pinker taken down as well, and might have made the letter stronger.

Second, the letter could be an attack by one section of the left against another. People have noted that it certainly makes progressives and cancel culture look ridiculous. "Some wondered if this open letter," wrote Pinker, "is a satire of woke outrage culture." He goes on to say that, "Cancel Culture has entered its decadent phase." If you wanted to make a mockery of progressives in general and cancel culture in particular, you could hardly do better than releasing a toothless open letter like this one. It's also interesting how the Harper's Magazine statement was composed before the LSA letter, but came out just a few days after it. It's rather convenient.

Finally, and most outlandish: it's possible that Pinker arranged for this letter himself. If you were worried about getting cancelled, you could arrange for a very weak case for cancellation to be made against you. Probably you would try to cancel yourself in a way that, if it actually went through, wouldn't hurt you all that much. The attempt inevitably fails, but in the process you gain a good deal of sympathy. Any attempts to cancel you in the future are met with scorn. This again? We have already been over this once, stop trying to cancel the poor man. Think of it almost like a vaccination—people are exposed to a weak argument that trains them to discard a stronger one.

Granted, I don't think that Pinker is Machiavellian enough to do this. This kind of play seems beneath his dignity, and he doesn't have much to fear from cancel culture to begin with. But it is consistent with all the data.

One way to look into this further would be to try to find the authors of the original letter; if anyone can find information on them, I would be very interested to hear about it.



Discuss

Anti-epistemology: an explanation

8 июля, 2020 - 19:34
Published on July 8, 2020 4:34 PM GMT

While there are more than a hundred thousand words on LW about the structure of good epistemology, as far as I can tell there is no nuts-and-bolts explanation of the most common anti-epistemology. I will try to rectify this omission, because I think I comprehended it.

The prototypical question of epistemology is some form of experiment, such as "what will I perceive when I pour these two liquids together?". After "what will I perceive" is packed into "what will happen", the question becomes observer-independent, and it is only natural that reality—the thing that answers the question—is itself observer-independent. The terrain would be there even if there were no maps of it. (Maps are intentionally created to predict approximate answers to a small subset of possible questions with much less effort/expense than it would take to answer the question in the terrain itself, by sacrificing the map's ability to answer the vast majority of possible questions correctly, let alone more cheaply than reality.)

In the unholy mirror image, the prototypical question is "who is most popular?". While it is possible for individuals to be mistaken, the answer has the form of a consensus (technically, common knowledge). Consequently, "reality" is inherently observer-dependent; it makes no sense to ask for the consensus of zero maps. There is no mention of the terrain in the theory (since an outright denial would be suspiciously specific), thus to the extent people are unable to compartmentalize away its practical intrusions into life, the stupidity can be overturned.

This would be bad enough in a hunter-gatherer band, but gets much worse in a society much larger than Dunbar's number, due to division of labor. Just as it in no longer possible for everyone to have the maps to answer all practical questions, and there is a niche for experts on subjects, it is no longer possible for everyone to know what society's consensus is on all questions, and there is a niche for experts at subjects. These experts at fields speak on their topic with authority (I will use this as a technical term).

How does this authority work? At the bottom level, "what the fields at which expertise can be had are?" is a matter of social consensus itself. Within a particular field, the consensus of experts is a Schelling point, thus the consensus of the whole society converges there. If there is a single expert in a local group, they can sometimes substitute their personal opinion into this role. Since the whole system is founded on the fact that, economically speaking, the expert's knowledge is unknowable to laypeople, authority is inherently opaque and subject to abuse, and people expect that it will be abused to some degree.

Diversion/illustration: when the priesthood of Disney tells of a new revelation, parents buy the little idols, and the schoolbags decorated with the deities' images, for their children not because they literally believe the legend, but because it would be weird not to. In this case, the system works even though the motives are completely transparent.

Given that authority is valuable, its supply must be restricted in some way. Some fields have formal and explicit credentials; this is too straightforward to elaborate further. Other fields have no such thing, instead relying on informal challenges. If a layperson (or fellow expert) thinks that another person is a shallow fake, they can ask questions from the field, and expect answers conforming to the genre conventions of the field in question. They don't have to actually understand the answer (see unknowability), they only need a cursory familiarity with the field, sufficient to judge the literary style of the answer.

Hang on. Am I saying that according to the anti-epistemology, "learning a field of study" reduces to "familiarising oneself with a particular pattern of bullsh!t"? Yes. Fake it 'til you make it.

Part of each field's jargon are the terms used to express approval/disapproval. Most fields don't overlap (too much) with other fields, thus their respective experts have no reason to have turf-wars with them. There are arbiters of aesthetics (architects, various artists, their respective critics), arbiters of popularity (journalists), arbiters of morality (politicians). Follow the pattern, keep the goggles on! Arbiters of health (medical professionals), arbiters of truth (scientists).

Uh oh.

People who (even unknowingly) operate on correct epistemology have a tendency to claim universal authority, to call statements following the conventions of other fields either "non-sense" or "false" if the statement also happens to look factual. Their justification ("the web of causality is connected, it has no isolated subgraphs") is parsed as BS from the genre of Science, thus the overall claim is understood to be "I am an initiate of Science, I get to overrule experts of other fields, and I feel comfortable betting on the chance that society will back me up on this". Such a claim from a not-already-visibly-awesome person would not just be a demonstration of outrageous (over)confidence, surpassing the schizophrenics' delusions of grandeur ("I am Elvis" is humble by comparison), but a power-grab of staggering proportions. Remember, experts get to approve/disapprove of anything that falls into their socially-accepted magisterium.

And as far as the people running on anti-epistemology can tell, STEM types are getting away with making this claim, at least once they are adults. I can't twist my point of view far enough to actually feel their reaction, but I'm sure it must be terrifying. That is why they are shouting about "epistemicide", "other ways of knowing", and seeing it not work. When we bemusedly point at the terrain and say "it never existed", it is perceived as if we were bemusedly saying "do you expect I will spare it just because you beg?". Is it any surprise they try to fight back by trying to lower the social standing of science?

That which can be destroyed by the truth, should be. Goggles off!



Discuss

Partially Stepping Down Isolation

8 июля, 2020 - 14:40
Published on July 8, 2020 11:40 AM GMT

On March 13th our household started isolating. We've relaxed our rules about surfaces a bit since then in response to changing estimates of risk—we've stopped sanitizing our groceries—but otherwise we haven't changed things much: I haven't gone into any buildings, or been close to anyone outside of my household. While the country as a whole is still seeing increasing cases, Massachusetts is doing pretty well:

The other indicators are also good: testing is way up, hospitalizations and intubations are way down, and the fraction of tests that are positive is below 2%.

Our house has decided to step down our isolation somewhat, while still being pretty careful:

  • Our default is the same as before: you can only interact with other people outside the household if you're outside, masked, and at least six feet apart.

  • Getting closer than that with people who have had COVID and tested positive for antibodies with a reliable test is generally ok, as long as you've shared details of their illness and test results with the house. We don't know for sure that they can't get it again soon, but it seems unlikely enough to us that we're willing to take that risk.

  • Additionally, each of us can be closer to one person per week outside the household. When we do, we'll put a note with their name in the #visit-log channel in the house Slack. This means that if we later learn that someone's been exposed we can trace contacts and know who to warn. The week rolls over on Monday, and you can see that person multiple times during the week. Being close to someone is unrestricted; you can kiss etc.

  • If you are outdoors and wearing masks, interacting with someone at under 6ft but at least than 1ft counts as half a person. For example, if I wanted to go sailing on a small boat with two people outside my household, that could be my interaction for the week.

  • Going inside counts as being close to all the people who've breathed that air since it last changed over. If you're visiting someone who lives in a shared house, you'd go immediately to their room, ideally holding your breath. Using the bathroom at their house would be ok if the fan ran for 15min before (and after, out of consideration for their housemates).

  • Guests mostly can't come to our house. The exception is if you have an exterior door into your space, or if they've already had COVID and the house has agreed.

  • The house has to agree about any exceptions. For example I have an in-person doctor's appointment this morning for my wrists, and I talked with housemates about timing and what precautions to take.

If the improvement in MA reverses or anyone we've been in contact with seems like they might have COVID, we'll put this on hold.

I know other households that have been using a closed bubble approach, but while I think that approach can work it's not a good fit for us. We have multiple housemates who would like to spend time with people outside the household, and there isn't a perimeter that would work. Additionally, we're much more wary of going indoors than the people we would potentially be bubbling with; most of them are still going into grocery stores.

Comment via: facebook



Discuss

What problem would you like to see Reinforcement Learning applied to?

8 июля, 2020 - 09:26
Published on July 8, 2020 2:40 AM GMT

We often talk about the dangers and challenges of AI and self-improving agents, but I'm curious what you view as potential beneficial applications of AI - if any! As a ML researcher I encounter a lot of positivity and hype in the field, so the very different perspective of the rationality community would be very interesting.

Specifically I'd like to focus on reinforcement learning, because this most closely matches the concept of an AI that has agency and makes decisions. A reinforcement learning agent is usually defined as a program that interacts with an environment, maximising the sum of rewards it receives.

The environment represents the problem to be solved, and the rewards are a measure of how good the solution is. For some problems - a board game, 3-SAT - assessing a solution (giving a reward) is easy, for others computing a reward may be as difficult as solving the problem in the first place. Those are likely not good candidates to be solved with RL ;)

To facilitate discussion, I would suggest one top level reply per problem, specifying:

  • a short description of the problem
  • the action space - how does the agent interact with the problem / environment
  • the reward - how do we know the agent has done well?

Disclaimer: I work on RL, so if you make suggestions that are feasible and would have substantial positive impact on the world, I may pursue them.



Discuss

Learning flirting with an acting coach - thoughts?

8 июля, 2020 - 07:34
Published on July 8, 2020 4:34 AM GMT

Beloved LessWrongers,

Recently I experimented with radically changing my texting strategy to increase women's interest. It was stunningly successful because I fixed awkward conversation killers and learn to perform high-status behaviors. Reductionism OP, please nerf.

I am now considering applying the same strategy to my in-person flirting. Ten or so behaviors (eye contact, slow movements, posture, etc.) are likely to inspire attraction. Given that learning the optimal behaviors in texting was so effective, I intend to learn the in-person behaviors next. I will hire an acting coach and we will pretend flirt exactly like learning a golf swing - adding a few more behaviors each time until they feel natural. I expect the acting coach will have other tips.

Some say "faking" confidence is impossible. But they said the same for tinder. Furthermore, if there is a 90% chance they are right that confidence is unfakable, the acting class is still positive utility.

If I'm missing something, post below.

Best, Snog



Discuss

Sunday July 12, 12:00 (PDT) — Online talks by curated authors — Scott Garrabrant, Alexflint, alexei, Stuart_Armstrong

8 июля, 2020 - 03:27
Published on July 8, 2020 12:27 AM GMT

This Sunday at 12:00 (PDT), we're running another session of "lightning talks" by curated LessWrong authors (see here and here for previous weeks' transcripts).

  • Each talk will be 3-5 minutes followed by discussion. Afterwards, we'll have a hangout in breakout rooms. The talks will be short and focus on presenting one core idea well, rather than rushing through a lot of content.
  • We want to give top LessWrong writers an interesting space to discuss their ideas, and have more fruitful collaboration between users. Think of it like a cross between an academic colloquium and some friends chatting by a whiteboard.
SpeakersDetails

When? Sunday June 28, 12:00 (Pacific Daylight Time)

Where? [....zoom link to be posted here......]



Discuss

Was SARS-CoV-2 actually present in March 2019 wastewater samples?

8 июля, 2020 - 02:08
Published on July 7, 2020 11:08 PM GMT

The initial Reuters headline read Coronavirus traces found in March 2019 sewage sample, Spanish study shows. This claim would have significant implications if true. However, it doesn't make sense that SARS-CoV-2 could have been circulating in Spain in early 2019 given the known transmissibility and disease characteristics of COVID-19. The most likely explanation is a false positive, either due to sample contamination or a PCR match on a different sequence, such as another coronavirus. However, given the uncertainty of the origin of the virus and the scale of the pandemic, a rigorous investigation seems justified. It would be worth screening other early and late 2019 samples and trying to sequence any coronavirus viral RNA.

The following preprint is the origin of the claim, and I have pasted the relevant passages from the paper.

https://www.medrxiv.org/content/10.1101/2020.06.13.20129627v1

Relevant paragraph is:

"Most COVID-19 cases show mild influenza-like symptoms (14) and it has been suggested that some uncharacterized influenza cases may have masked COVID-19 cases in the 2019-2020 season (11). This possibility prompted us to analyze some archival WWTP samples from January 2018 to December 2019 (Figure 2). All samples came out to be negative for the presence of SARS-CoV-2 genomes with the exception of March 12, 2019, in which both IP2 and IP4 target assays were positive."

Methods seem fairly normal:

"Eight hundred-milliliter samples of wastewater were concentrated through precipitation with 20% polyethylene-glycol 6000 and resuspended in 3 mL of PBS, pH 7.4 (9). Nucleic acid extraction was performed from 1mL of the concentrate and eluted in 50 µL using the NucliSENS® miniMAG® extraction system (bioMérieux).

Five one-step RT-qPCR assays (RNA UltraSense™ One-Step Quantitative RTPCR System, Invitrogen, Life Technologies) targeting the RNA-dependent RNA polymerase (RdRp) gene, IP2 and IP4 fragments, from Institute Pasteur, Paris (Institut Pasteur. Protocol: Real-time RT-PCR assays for the detection of SARS-CoV-2. 2020 https://www.who.int/docs/default-source/coronaviruse/realtime-rt-pcr-assays-for-thedetection-of-sars-cov-2-institut-pasteur-paris.pdf?sfvrsn=3662fcb6_2), the envelope protein (E) gene, E fragment, from Charité, Berlin (10), and the nucleoprotein (N), N1 and N2 fragments, from CDC, Atlanta (Centers for Disease Control and Prevention. CDC 2019-Novel Coronavirus (2019-nCoV) Real-Time RT-PCR Diagnostic Panel. 2020 https://www.fda.gov/media/134922/download). The standard curve were constructed using the Twist Synthetic SARS-CoV-2 RNA Control 2 (MN908947.3) (Twist Bioscience). Technical details are included in the Appendix."

Outstanding questions

What could the authors do to increase their confidence in their results?

  • Rerun PCR using different assays targeting different regions of SARS-CoV-2?
  • Sequence every bit of RNA in the sample they can and look for matches?

How many wastewater centers around the world have samples they can check? Are they checking them now?

Were the IP2 and IP4 samples handled and tested independently? How hard would it have been for them both to be contaminated in the lab?

How good are the IP2 and IP4 target assays in particular? Can we test them against other coronavirus / other samples to look for false positive matches?



Discuss

Antitrust-Compliant AI Industry Self-Regulation

7 июля, 2020 - 23:53
Published on July 7, 2020 8:53 PM GMT

Summary:

The touchstone of antitrust compliance is competition. To be legally permissible, any industrial restraint on trade must have sufficient countervailing procompetitive justifications. Usually, anticompetitive horizontal agreements like boycotts (including a refusal to produce certain products) are per se illegal.

The “learned professions,” including engineers, frequently engage in somewhat anticompetitive self-regulation through professional standards. These standards are not exempt from antitrust scrutiny. However, some Supreme Court opinions have nevertheless held that some forms of professional self-regulation that would otherwise receive per se condemnation could receive more preferential antitrust analysis under the “Rule of Reason.” This Rule weighs procompetitive and anticompetitive impacts to determine legality. To receive the rule-of-reason review, such professional self-regulation would need to:

  1. Be promulgated by a professional body;
  2. Not directly affect price or output level; and
  3. Seek to correct some market failure, such as information asymmetry between professionals and their clients.

Professional ethical standards promulgated by a professional body (i.e., comparable to the American Medical Association or American Bar Association) that prohibit members from building unsafe AI could plausibly meet all of these requirements.

This paper does not argue that this would clearly win in court, or that such an agreement would be legal. Nor does it argue that it would survive rule-of-reason review. It merely says that there exists a colorable argument for analyzing such an agreement under the Rule of Reason, rather than a per se rule. Thus, this could be a plausible route to an antitrust-compliant horizontal agreement to not engineer AI unsafely.



Discuss

Translate music intonation into words using color semantics (as a means of communication)

7 июля, 2020 - 22:28
Published on July 7, 2020 3:26 PM GMT

Scientists from the University of California, Berkeley, found that music and color are connected through an intermediate link - emotions

news: news.berkeley.edu/2013/05/16/musiccolors/
paper: www.pnas.org/content/early/2013/05/10/1212562110.abstract

So why not apply in this case color semantics, which has existed since 1949 in the form of the Max Luscher's eight-color test, which converts a sequence of colors into a sequence of words?

"Max Lüscher believed that sensory perception of color is objective and universally shared by all, but that color preferences are subjective, and that this distinction allows subjective states to be objectively measured by using test colors. Lüscher believed that because the color selections are guided in an unconscious manner, they reveal the person as they really are, not as they perceive themselves or would like to be perceived"

So, if you sync your mood with the mood of the music, you can determine the mood of the music using the color test

The sequence of passing the test is simple:
1. Which of the eight colors seems the most pleasing?
2. Which of the remaining colors seems the most pleasing? (until all the color cards go to the result)

You can find the test version here: www.colorquiz.com
Here: en.testometrika.com/personality-and-temper/color-personality-test/
Here you can find the test and simpliest MVP of music recommendation system how it can be created (MUSLI stands for Musica Lingua or Music Lingo): musli.icu

Since the interpretation text can be translated into any language in the world, this makes music a universal language. Thus music as a means of communication and coordination can become a means against Moloch

Jim Carrey about music: www.youtube.com/watch?v=xaoCB0vkMT8

Also listen to three songs from Paul McCartney's Flaming Pie album:

1. The Song We Were Singing www.youtube.com/watch?v=Q4qObxQL4x0

2. The World Tonight www.youtube.com/watch?v=irt1OH_NNFQ

3. Great Day www.youtube.com/watch?v=2pTpCaV1zzw


And so here are some examples of matching the first pair of colors of the test and the songs:

Your Desired Objective
"Wants to sweep aside the things that stand in his way, to follow his impulses and be involved in special or exciting happenings. In this way he hopes to deaden the intensity of his conflicts, but his impulsive behavior leads him to take risks":

1. The Clash - London Calling www.youtube.com/watch?v=EfK-WX2pa8c
2. Ozzy Osbourne - Crazy Train www.youtube.com/watch?v=RMR5zf1J1Hs

Your Desired Objective
"Seeks affectionate, satisfying and harmonious relationships. Desires an intimate union, in which there is love, self-sacrifice and mutual trust":

1. Echo Jet - Something To Believe In www.youtube.com/watch?v=BIx7MW1RbuI
2. The Beatles - All You Need Is Love www.youtube.com/watch?v=_7xMfIp-irg

As you can see the author creating a song partially duplicates the information transmitted through the text channel (verbal) and the intonation channel of the music (non-verbal)

Thus the song form is an implementation of Jung's Transcendent Function. As much as Max Lucsher's color test too

"There is nothing mysterious or metaphysical about the term "transcendent function". It serves to denote a psychological function comparable in nature to the mathematical function of the same name that connects real and imaginary numbers. The psychological "transcendent function" is the result of combining conscious and unconscious contents"

Also, if we put on the 7th and 8th place two colors, when mixed, it turns out the "the ugliest colour in the world" (en.wikipedia.org/wiki/Pantone_448_C), then we get the following interpretation:


"Has an unsatisfied need to ally himself with others whose [ethical]standards are as high as his own, and to stand out from the rank and file. This subjects him to considerable stress, but he sticks to his [ethical]attitudes despite lack of appreciation. Finds the situation uncomfortable and would like to break away from it, but refuses to compromise with his [ethical]opinions. Unable to resolve the situation because he continually postpones making the necessary [ethical]decision, as he doubts his ability to withstand the opposition which would result. Needs the esteem of others, compliance with his wishes and respect for his opinions before he can feel at ease and secure"

A little theory with illustrations galaxy.icu.tilda.ws/en

Paul McCartney - Cosmically Conscious

An interesting thing: if you change four pairs of colors in places, while maintaining the order inside the pairs, then from Love intonation from "All You Need Is Love" you can get Joy from "Cosmically Conscious" an Us intonation from "Imagine" ^_^



Discuss

Страницы