## Вы здесь

### Conjecture Workshop

Новости LessWrong.com - 16 мая, 2020 - 01:41
Published on May 15, 2020 10:41 PM GMT

The conjecture workshop is an activity I’ve run three times now in a conference/meetup-style setting. Feedback has been unusually positive, so I’m writing it up here to codify the basic idea and hopefully get independent feedback from others who try it.

The overall goal is to translate intuitive ideas into mathematical conjectures.

Format

People break into groups of 2-3. Within each group, one person serves as “conjecturer”, and the other one or two serve as “questioners”. Roles can rotate over the course of the allotted time (~1 hr for sessions so far).

To start, the conjecturer should have some intuitive claim in mind that they want to formalize. Some AI-oriented examples from previous sessions:

• In beneficial comprehensive AI services, security services and planning services necessarily have to be "agenty"
• There exists some “universal” algorithm which performs about-as-well as any other on optimization problems with bounded runtime
• Coarse-grained models of the world form a category, and we can always construct pullback/pushout models within that category

Note that these are quite fuzzy and leave out a lot of the idea; fully explaining the ideas intuitively (even without formalization) takes much longer than the short blurbs above. Indeed, the ideas usually start out even fuzzier than the blurbs above - just summarizing them in a sentence is hard.

Once the conjecturer has something in mind, they try to explain the claim to the questioners, intuitively. The questioners’ job is to regularly interrupt, and ask questions to help formalize what the conjecturer is saying. Common examples include:

• “You’ve mentioned <thing> a couple of times. Should we give it a name?”
• “What type of thing is <thing we just named>? Any constraints on it?”
• “Ok, so we claim/assume that <thing> is <better/larger/simpler/etc> than <other thing> in some intuitive sense. How do we quantify that?”
• “So we want to assume <intuitive assumption>. What does that mean, mathematically?”
• “Does the model so far accurately capture your intuitions, at least the parts which intuitively seem relevant to the claim?”

The questioners should also ask general explanation-support questions, like “can you give an example?”, “can you repeat/clarify that last part?”, and repeating back the questioner’s understanding of the claim so far. In particular, the questioners should remind the conjecturer to write down any components of the mathematics (i.e. variable/function definitions, assumptions, claim itself, etc) as they come up.

Key point: PROVING OR DISPROVING THE CLAIM IS NOT THE GOAL. For purposes of this activity, we do not care whether or not the claim is true; the goal is simply to formalize it enough that it could be mathematically proven/disproven. (One minor exception: if the claim seems trivially true/false at some point, that’s often evidence that some key piece of the conjecturer’s intuition has not been captured.)

Value Model

The idea behind the exercise is that translation is a scarce resource - in this case, translation of intuitions into mathematical formalism. Often, a major bottleneck to theoretical/modelling work is simply expressing the ideas mathematically.

Focusing on a particular conjecture also helps avoid a “model ALL the things” failure mode. Since there’s one particular claim we want to set up, we just work out the pieces necessary for that claim, not for a whole theory of everything.

Discuss

### Could We Give an AI a Solution?

Новости LessWrong.com - 16 мая, 2020 - 00:38
Published on May 15, 2020 9:38 PM GMT

One of the biggest problems in AI Alignment is how difficult it is to condense morality into one utility function. You want to maximize happiness? The AI will plug us into wireheading machines, and we will experience nothing but pure joy, with none of the other things that we care about, like friendship and science. Okay, then try maximizing complex human experiences that lead to happiness. The AI will select the one thing that would satisfy this criteria best, and then have us replay that one thing over and over and over.

Some people have suggested that we just skip over that part. Tell the AI to do whatever would satisfy a human's preferences. Then it will modify the human's preferences so that they like, for example, hydrogen, and then make lots and lots of hydrogen. Even if you tell the AI "Investigate what humans consider 'morality' and formalize it as a utility function, then maximize that utility function," you run into the problem that humans often don't have consistent values.

The conclusion is that it's really hard to write a utility function for an AI that will match ours.

So what if we don't give an AI that type of utility function?

It is also difficult to come up with an ideal universe of the future, primarily due to boredom. But it seems like it shouldn't be too hard. Suppose humans had access to virtual reality technology, with the ability to create worlds with anything they wanted. Not literally anything- there might be constraints on complexity due to computing power, and causing sentient beings pain would need to be prohibited- but almost anything. Let's label this potential universe VirtualFuture.

Now give the AI a utility function solely devoted to VirtualFuture, and tell it to bring VirtualFuture to existence, as soon as possible, with as high probability as possible. Just maximize VirtualFuture, and nothing else. If we can create a universe where humans are in control of their own worlds, and those worlds can be almost anything they imagine, it's hard to see how that would go wrong.

Potential problem number 1: A utility function devoted entirely to one thing only applies to universes where that one value is possible. If the AI finds that it can't create VirtualFuture, it won't do anything to distinguish between other values. But I doubt that would be much of a problem- being an Artificial Intelligence, it should be sufficiently smart that creating VirtualFuture will be easy. There would be almost no chance that this would matter.

Potential problem number 2: VirtualFuture might not be perfect. Humans might create other sentient beings, causing an exponential increase in computing power required, and eventually even the AI wouldn't have enough computers. Also, humans might cause each other pain. The description of VirtualFuture would have to include both a nonperson predicate and a restriction stopping serious pain.

Potential problem number 3: Even if VirtualFuture is significantly preferable to our current world (which I believe it is) it certainly isn't the maximally best possible universe. A fully Friendly AI, with all of our values, would certainly be able to find something better than what we can come up with. This means that a VF-AI would, while being much better than an Unfriendly AI, would still not be as good as a Friendly AI.

In summary, while there are still many problems to work out, programming an AI with VirtualFuture seems much easier than alignment, even if it's not as good. MIRI should probably begin work on a full description of VirtualFuture, complete with nonperson predicates and the necessary restrictions. While it's inferior to an aligned AI, a VF-AI could be created much faster, and if MIRI or other alignment organizations can't find a way to make a full human utility function, the VF-AI might be sufficient for us to win the AI race before x-risk can destroy us.

Discuss

### [Repost] Movable Housing for Scalable Cities

Новости LessWrong.com - 16 мая, 2020 - 00:21
Published on May 15, 2020 9:21 PM GMT

First posted on Steemit on July 30, 2016.

0: Summary.

The non-scalability of current major cities poses a challenge to economic growth; it is a burden that falls disproportionately on those least able to bear it.

Unfortunately, any would-be startup city must confront the network effects that pose a tremendous potential energy barrier to the attractiveness of a city that isn't yet mature. From day one, you need to compete with the hedonic attractiveness of San Francisco, in the eyes of the person who decides where to locate the company.

Movable houses mated to modular foundations – nice large customized houses, not small ugly trailers – can make cities more scalable and more hedonically attractive:

• By making it easier for groups to relocate and untangle themselves in a coordinated way, movable houses can increase the natural dispersion of the city as it grows larger.
• Since people could buy and retain customized houses manufactured with economies of scale, movable houses could have better technology and amenities not available today. (Contrast your current house to a modern car.)
• Movable houses could allow unprecedented opportunities to live next to your friends, or to groups of similar-minded people. You and your friends just need to find a set of available modular foundations close together.
• Movable housing might help on key points of governance and realpolitik, for example by making land value taxes more attractive, and decreasing exit costs.

(This was written in response to Y Combinator's request for ideas about the design of new cities.)

1: Introduction: The challenge of scalable cities.

Manhattan doesn't scale well. San Francisco doesn't scale well. Tokyo and London don't scale well. On the present system, each added resident comes with some tiny added degree in inconvenience for every other resident already living in a city, most notably in the form of traffic jams and housing prices. Yes, every added programmer also provides some tax revenue; but the city infrastructure, qua infrastructure, is not scaling well.

This is not all due to anti-housing housing policy, though we've certainly gone to every possible length to shoot ourselves in that foot. Limited road bandwidth is a real issue. The air in Manhattan is genuinely dirtier than in Berkeley – I acquire a sore throat every time I visit New York City.

Building cities that can scale better is a problem of overwhelming human importance. Some analyses have suggested that increased housing costs have eaten huge chunks out of middle-class income; that housing costs may be responsible all on their own for the stagnating middle class. The costs and inconveniences of our most active and growing cities, prevent people in less active economic areas from seeking better fortunes. Cities that don't scale are problems measured in trillions of dollars of lost economic growth and a crushing amount of real human despair.

1.1: All-robotic car fleets are an obvious first step.

The technology for an all-robotic car fleet is probably already available... if we take all the non-robotic cars off the road.

A robotic car fleet makes cities more scalable in many, many ways. The most obvious boost is widening space, compressing time, and removing the cost in stamina to travel. Robotic cars that don't need stoplights or speed limits might let you get anywhere within 6 miles in 10 minutes. (Say, 2 minutes to get to a thoroughfare, 6 minutes to go 6 miles at 60mph, and 2 minutes to go the last mile.) A 6-mile radius is a lot of territory (113 square miles). You can be 10 minutes away from a lot of friends.

Robotic cars lend themselves to fleets that show up when needed, rather than individually owned cars. If every individual car is being used much more regularly, that changes the cost/benefit calculation for batteries versus gasoline. Which implies less pollution per added resident in concentrated populations. Fewer parking lots implies more space for people, and so on.

But robotic cars, by themselves, may not be enough. Designing a new city is the last place you want to stop and congratulate yourself on the improvements you already have planned.

1.2: The first business challenge is being hedonically attractive to corporate decisionmakers.

Cities run on network effects. Moving to a location with 1000 companies that might want to hire you is far less risky to moving to a location with 3 companies that might want to hire you.

Any new city faces a huge potential energy gradient during its youngest and most vulnerable stages. The gravity of Manhattan and Tokyo and London is absolutely enormous, drawing away your potential residents. You need to offer something extremely attractive about your new city if you want companies to move there instead of San Francisco. And your city's first critical business challenge is about attracting companies, not humans; humans go where there are jobs.

"But we'll have lower housing prices!" you cry. Alas, it's a sad fact of life that the people who decide where to locate their companies face different incentives than the average employee in those countries. (As in the Dilbert cartoon where the CEO is moving the company to what happens to be his old hometown, because that way he can get free babysitting from his grandparents.) I once asked a Google employee why Google didn't start an offshoot campus somewhere with lower housing costs – surely, I ventured, Google had the scale to do that, and enough employees who dispreferred city living. My friend replied that no project manager at Google would want to be so far from the center of corporate politics.

The cofounder who decides to locate their hot startup in San Francisco may have plenty of reasonable and selfless motives to do so – better access to venture capital, better access to programmers. But it seems worth noticing regardless that this decision is being made by someone who can probably afford more Ubers and higher rents than other company employees. If the company did locate itself somewhere cheaper, it could perhaps pay lower salaries and so capture part of the gains from lower housing costs. The decisionmaker who decides where to locate the company may capture a part of the company's gains from those lower salaries. But quantitatively speaking, those fractions do not multiply out to 100%. The incentives are not aligned between the decisionmaker and the rank-and-file.

The overwhelming proportion of the real human good accomplished by a more scalable city comes in the form of schoolteachers who can afford the rent. But if you want that city to boot up successfully, you need to think about how to make it personally, hedonically attractive to the decisionmakers who choose where to put their companies.

Robotic cars help here too! Even compared to taking a private car through traffic, taking a robotic car through an absence of traffic might be more attractive to a CEO.

Slap some flying drones riding the cars, to take off and make deliveries as the destination approaches, and you can have amenities that non-robotized cities just don't have! If delivering hot meals across a 3-mile radius became truly cheap and ubiqitous, so that even non-CEOs could afford it, then we would see much greater specialization on selling meals. Meal-creation businesses would compete on cost and healthiness, instead of restaurants competing on location and dining experiences. It could become typical to order a not-so-expensive, non-high-calorie, home-cooked-style hot meal that would have taken you an hour to make from scratch, and have it delivered in 10 minutes. Maybe the CEO of a big company has a home chef who can compete with that – but it's an attractive amenity for the CEO of any middle-sized company or hot startup.

Again, the vast majority of real good comes from giving the thousands of other residents access to healthy food without a huge cost in personal labor. But you have to appeal to the decisionmakers before anyone else gets that chance.

Having remarked on this cool and futuristic amenity, we are still in no position to relax and declare ourselves done. We need every possible attractive feature for our shiny new city to exert a draw that is remotely competitive with Manhattan.

1.3: Inside or outside the US?

I will digress at this point – though it will turn out to not really be a digression – to consider the possibility of locating the New City outside the United States (and outside the UK and the EU and Australia).

By far the hugest attraction of this possibility for cofounders would be EASIER IMMIGRATION. If your city is otherwise competitive in terms of first world amenities and safety, has lower housing prices, and cofounders can bring people there without Eternal Visa Hell, startups will flock to you that couldn't form anywhere else.

Avoiding regulatory molasses will be another huge attraction for some companies and potential companies. Cough cough PATENT MONSTERS cough cough.

If the new city were located in, say, a special district negotiated with the government of Uruguay, this might permit some basic utopian legal features. For example, not living in a country where marijuana is theoretically illegal if you're white and rich, and potentially life-destroyingly illegal if you're black or poor. For many cofounders or company-owners, this is a point of principle rather than practice; but to some people it is a matter of practice. Or some people, with decision-making power even, may dislike at a deep emotional level the prospect of living in a country where a SWAT team can kick down their door at any time.

The United States is unique among nations in insisting that it can tax the income of its citizens even if they live and earn outside the US. But the tax policy surrounding your city still matters to anyone who isn't a US citizen, or to any US citizen earning not much more than $92,000 per year. Act 20/22 in Puerto Rico enables people moving to Puerto Rico for the first time to negotiate a two-decade contractual exemption from US federal income taxes. Puerto Rico is just about the only place on Earth, inside or outside the US, that a US citizen can go to not pay US federal taxes. For this reason I'm fond of saying that the three obvious places to start a new city are 2 hours from the Bay Area, Puerto Rico if you're going to be located in the US but not near any particular existing metropolitan area, and Uruguay (hurricane and earthquake free, extremely sensible and stable government). Not paying federal taxes in Puerto Rico would be a hugely favorable attractive gradient – for the decisionmakers that locate companies, in particular, but heck, even for programmers making a dinky$150K per year.

International development has its own negatives. Most obviously, the extra barrier posed by being a lot further away from existing centers of gravity; compared to starting up 2 hours from the Bay Area, or in coastal Oregon, or Nevada. It's one thing to live a 2-hour drive away from venture capitalists, and quite another to be separated from them by a 6-hour plane flight.

There may also be language issues – especially if your company needs to interact with government agencies whose employees don't speak your language. Google Translate will only continue to improve in the future, lowering language barriers further; but the technology isn't there right now to render international communication frictionless.

These barriers will be particularly fatal during the early stages, when the potential energy barriers surrounding your city are the highest.

What we really want – one might think – is to initially start up our city 2 hours from the Bay Area. But to have some mysterious feature that made it unusually easy for many of the city's inhabitants and whole companies to relocate to coastal Oregon, to Puerto Rico, or to Uruguay, once the system had been proven.

Especially if this mysterious new feature also improved the hedonic attractiveness, the cost profile, and the fundamental scalability of our startup city...

2: Movable housing.

Let's assume at the beginning that we're talking about an early town small enough to be composed mainly of houses instead of apartment buildings.

Imagine that in this town, houses are portable objects with standard connections that match up to standard modular house-foundations. We can disconnect a house from its water and electricity and Internet cables, have a standard vehicle pick the house up from its foundations, and gently drive the house over to somewhere else inside the city.

We could also imagine apartments that slot into towers on rails. We can imagine office buildings built up from modules that could be taken apart at need. But as we'll soon see, this new city might be able to scale a lot further before it needed apartment buildings and office towers.

Movable housing would require some amount of new technology, and more importantly, a green-field city – the roads need to take the weight (unless we can replace or supplement carriers with skyhooks or other lifters); the robotic-vehicle idiom would also help, because it means we can clear any road of cars as a house-carrier is passing. I'm pretty sure all of this is within reach of human technology; the question is whether it's too expensive.

And whether anything is 'too expensive', of course, depends on what benefits you're buying.

Benefit #1: You can live closer to where you most want to live.

In the world today, when you want to move to a new city or district, you hunt around until you find a house that is shaped sorta like you want a house to be shaped, which isn't located too far from where you wish you actually lived. And all of your friends and coworkers – network effects will shortly become very important here – are implementing that same process.

Movable housing decouples the problem of finding a good location from the problem of finding a good house. It disaggregates the two businesses, as my father would say.

To find a nice place to live, you just need to find any available plot whose modular foundations match your current house. (There'd better not be more than three varieties of foundation, though, or we're back to the same old matching hell.)

Benefit #2: You can own a better house.

A modern car contains vastly more interesting technology than a modern house. That's because there are big-company car manufacturers with centralized factories and R&D departments competing to offer the car that you'll like the best.

Imagine if instead, a boutique car-constructor company sauntered over to a parking space and built a car there over a year or so. And then you looked around for an available parking space that wasn't too far from your house and had an associated car that was acceptable. If no car was good enough, you could buy a parking space and its car, trash that car, and have a different car handcrafted in that parking space over the next year by the small boutique car company of your choice.

Cars would be a lot less advanced, to put it mildly.

You wouldn't realize what you were missing. You'd never have seen cars with electronic stability control and automatic parallel parking and collision alarms and radio keyfobs. Maybe some rich people would have hand-constructed cars with primitive versions of those features, to go with the gold-plated ashtrays.

So if you can buy a house from a company that has a real R&D department and is actually trying to please the customer and has real competition… well, I'm not an entire R&D department myself. But my personal starting list of feature requests for an Apple iHouse might include:

• Completely blackout shutters over my windows, which open gradually and automatically at the time of morning when I actually wake up.
• Noiseproofing. Extremely serious noiseproofing, with thicker walls, better-sealed doors and windows, active sound cancellation, and maybe white noise generation if that still wasn't enough. When I close my door I want silence.
• Centralized, extremely powerful LED lighting modules with the light carried by optical fibers to whichever room I was actually using at the time. Dimming to red as my bedtime approached, of course.
• Centralized humidification with per-room control plus per-room temperature control, so I'm not running a loud hot humidifier in my bedroom at night. (I need a lot of humidity, personally.)
• A Jacuzzi on the roof, open to the stars. (You think this is some kind of huge luxury and that I'm spoiled for even wanting it, but that's because you live in a civilization where everything house-related costs too much.)
• Anti-insect laser bug-zappers on the roof which fry any mosquitos, wasps, hornets, other stingy things, or even houseflies that get too close. (Yes, I know we're all supposed to be too tough to be bothered by little things like insects; just like there was a time when British people made fun of people who carried umbrellas on drizzly days. I observe that you do, in fact, work inside an office instead of outdoors. I bet those little annoyances you're supposed to be too tough to care about have an awful lot to do with that cultural decision in practice.)

Finally, bigger rooms! More storage space! Relatively empty square meters in a house just should not cost that much on the margin!

Extra marginal people impose costs on cities. Electricity costs money. The central LED lighting modules and Powerwall batteries will cost money. An extra square meter of floor or ceiling, even an extra story on a house, should not cost that much more on the margins.

It does feel to me like the small, crowded apartments are an arcane penance that we are imposing on ourselves. Even people who don't like expanding cities, ought to see more people as being the issue, not more square meters of living space; having more square meters per person seems like something our civilization should be able to manage. Our ancestors lived in bigger houses. We really ought to be able to get that back.

(This does mean tolerating higher roofs, or larger apartment-tower frames for modular apartments on rails, or less grass on the hill. Or living further away with lower local population density; see Benefit #4.)

Some of us have chosen to live in large house-complexes with friends (though it's not obvious how much of this is just due to modern housing costs). Imagine a central 'house' that has the kitchen and the game room and the Jacuzzi, and six modules radiating out from it, each module with a bedroom, bathroom, storage space, and small reading room. Want to try out a different group house for a month? Pick up the bedroom module, connect it to a different house, put it back down again.

Your own wishlist would look different, I expect. And in another 5 years the Apple iHouse 4e would offer us features that neither of us imagined. But one point is sure: under the present system of the world, we'll never get a chance to find out.

Benefit #3: Friends living near each other.

All sufficiently tight clusters of friends can find a set of empty foundations, rent them, and live literally next door to each other.

We could have neighbors again.

We could have tribes again.

Yes yes, I know, you're thinking of that one person who's friends with a bunch of your friends but who you don't want on the same block as you. But really, human beings have been living in tribes for a while. You can probably work it out. Many people could benefit a lot from living in close proximity to fifty people whose company you can tolerate – not necessarily your coworkers, even; you might be choosing to live next to your fellow D&D 3.5e roleplayers who also like dubstep.

It could give us back something human that we lost, and start to make headway against lives that are sad, never mind the poverty.

This notion will admittedly create some coordination problems: "Does everybody in the block need to be in our group for us to have the kind of... parties... that we want?" and "How do we try to make sure that a lot of foundations go empty at the same time so a new group can move in?" and "Uh, what if 60% of the group prefers somebody not move in to their cluster, is that going to be enforceable at all and doesn't that imply a whole new level of local government?"

I can think of some obvious policies to try, but I have a strong suspicion that policy v.1 won't work all that well and it will take until policy v.3 before work tolerably well. Sometimes you just gotta try things.

Benefit #4: Small towns can scale better.

And not just to larger sizes, but to higher levels of economic activity.

This is the prediction that I'm least sure about. But one of the reasons why I'm talking about movable houses, instead of modules on rails in a huge apartment-tower frame… is that maybe, a lot of us won't even need the huge apartment-frames?

Maybe you, dear reader, positively prefer to live in a huge apartment building. Maybe you don't care where a robotic car can take you in 5 minutes. Maybe what you really want, more than any other style of life, is to be able to walk out onto your sidewalk and see lots and lots of human faces (even unfamiliar ones), then turn right and walk into that lovely coffeeshop that is downstairs and half a block to the right of your eighth-floor apartment. Maybe you like the hustle and the bustle. There is no arguing with terminal components of the utility function, as David Hume observed.

Some of us – and I'm not saying we have better preferences, but we have different preferences – some of us would rather live in Rivendell.

Or failing that, we'd like to live in a quiet little house on a green hill, where the technologically advanced soundproofing doesn't need to work that hard.

If we had robotic cars and movable houses, it would be a lot easier for all of us who work at my nonprofit to not need to live downtown – not even in downtown Berkeley. Part of the reason we located in downtown Berkeley is that it has sufficient density of housing that we originally could, and new employees still can, find available non-horrible apartments within walking distance of the office.

But in a city with robotic cars and movable houses, we could perhaps all live in the same section of green hills, 10 miles or 20 miles from the skyscrapers, and move our office modules there too. Robotic cars could teleport us to the large, centralized supermarket that served a big area and therefore had just as much selection as a supermarket in a big city. Or some of us might live closer to the skyscrapers, or with the Burning Man tribe; and for those who made that choice, a robotic car would take them to the green hilly workplace in 24 minutes while they browsed their cellphones.

24 minutes isn't far from the time it takes to cross San Francisco right now! Anyone who considers that good enough could live anywhere within a 20-mile circle, served by robotic cars with no traffic lights cruising at 60mph down the central throughways.

Even so, some offices and some people would need to be closer to the center of gravity, or would just yield in preference to the siren call of network effects. Sometimes there are just too many graph links that all need to be located within 5 minutes of one another. Those companies might still need to live in office modules slotted into towering office-complex-frames at the New City's center.

But if you can coordinate locations more cheaply, move more cheaply, and travel more cheaply because of the robotic cars, then it becomes a whole lot more feasible to have more software companies located in the quiet hills. It's not a panacea and it won't work for every organization, but more people will be able to live in real houses like our parents owned.

Even more of us will be able to live in Rivendell when virtual reality tech matures, which will loosen (though not cut) the bonds of spatial distance that much further. Or VR might not really make any pragmatic difference, but it's worth trying to think 5 years ahead at least; the headsets will improve. It will be one more marginal force exerting a bit more quantitative push towards the attractiveness of living in a bigger house in a quieter place, if your new city can offer that with fewer than usual disadvantages.

Benefit #5: Land value taxes.

Economists since Adam Smith have observed that land is the ideal thing to tax. Literally, tax the square meters of planetary surface. We can't make more land, so it's not like a land tax discourages the production of land, the way that income taxes discourage work. Somebody is going to collect the implicit rent on land; so long as the relevant collector doesn't tax more than the price point at which the supply of land balances the demand for land, the tax doesn't change that price. It's a frictionless tax on a HUGE flow of land rent, and the alternative to taxing that huge flow of land rent, is frictionful taxes. Taxing almost anything other than land (except carbon dioxide emissions), before taxing land, is one of those insanely stupid aspects of modern civilization that make economists want to stab themselves to death with a butter knife.

With movable housing, the houses are moving around and the foundations are staying in one place. This makes it even more obvious that you might as well have the rent on the foundations supporting government services.

I pay $2500/month on my little 2-bedroom apartment in Berkeley, the supermajority of which is certainly land rent. That land rent is probably more than I pay in state and federal income taxes. Which makes me want to gouge out my eyes with a spoon. Because – no offense to my innocent landlord who paid a corresponding price for that land and is not earning an excess profit in on it after mortgage costs – I am paying two huge taxes where an even slightly saner system could be charging me one tax. Yes, you can see how it might go wrong if one government tries to own all the modular foundations and therefore all the urban land. (The more so if there are no competing cities: see benefit #6.) But the economic factors here are huge enough that it seems worth trying to do things the sane way just once. Since most taxes are federal, making real headway on eliminating the 'double tax' (income tax plus land-rent) might require locating in Puerto Rico or Uruguay. But even without that, to whatever extent the modular foundations have a natural equilibrium rent, that rent can provide for fire trucks, maybe even health care if the land rents equilibrate high enough – without additional taxes. Where, again, the current model is to have the residents pay one stream of highly frictional tax-rent to the government, and an entirely separate stream of land-rent. This isn't really a technology problem. But having houses moving around, so that the rent collected on the stolid immobile foundations is entirely separate from any handcrafted structure nailed to them, makes the solution that much more obvious and maybe more politically feasible. Benefit #6: Exit threats and political relocations. When Patri Friedman proposed 'seasteading', a lot of the hoped-for good systemic properties came from the fact that sea-based platforms would be easier to move around. Movable housing can be seen as trying to get several similar systemic properties, without taking on the engineering or political challenges of building at sea. This analogy extends over to one of the primary political ideas motivating seasteading, the notion of "voice and exit". When your BATNA to staying in a place is to pick up your houses and/or your office modules and move them somewhere else... then that creates a different negotiating position than when you need to destroy your painstakingly handcrafted structures, create new handcrafted structures somewhere else, rearrange all of your personal possessions, etcetera. Yes, there's still friction – you have friends who may not follow you, maybe your kids end up going to a different school. But there's less friction with movable houses, the coordination problems are that much easier to solve in groups; it could make a quantitative difference. This improved BATNA could apply at the level of a whole city that negotiated a special economic zone with some state or country. Maybe it's wacky to think that "Kay thanks bye" would ever be a plausible reply on that scale if the host country tries to "alter the bargain, pray I do not alter it any further". There might just be too many non-movable objects creating too much inertia. But even having a large fraction of the potential victims, having the option of putting their movable modules on a cargo ship and heading elsewhere, could make a difference. It matters quantitatively to a victimizer whether making conditions worse for their victims means that 20% of their victims leave or 2% of their victims leave. I'm trying not to go on too much of a rant here. But one of the enormous overlooked questions of the modern age is how poverty still manages to exist, when agricultural and economic productivity have risen by a factor of literally 100 since the time when 98% of the population was farming. We have fewer poor people, to be sure, the life of the lowest income quartile is a lot less horrible than it was in the 13th century. But there's still some sense in which it seems a little embarrassing to imagine going back to a world where people managed to survive despite being 100 times less productive, telling them that we are now 100 times wealthier, and then having to explain why there are any horribly poor and desperate people in our country at all. When a condition is that sticky, we should suspect it to be an equilibrium with strong restoring forces. There must be some powerful factor that makes some people be poor, no matter how much wealth is flowing around – a factor that gets stronger as more wealth flows, even by a factor of 100. One of the obvious forces that could be stabilizing a Poverty Equilibrium is if the standard state of affairs, for human civilizations in general, is for there to always be a few groups here or there that can extract a little more value. The Ferguson Police Department, issuing 3 warrants per household per year, is one obvious example of this idiom. But you should also be thinking of taxi medallions, licensed haircutters, NIMBY house-owners, and health insurance companies without much statewide competition. I don't mean to single out one group as a target for the Two Minutes Hate. There can just be these endless small sets of local factors with the power to drain one more dollar; and these factors will collectively go on draining one more dollar until they can't drain any more dollars without some victims dying. Actually, the equilibrium for multiple extractive forces is a commons problem – Alice knows that if she doesn't steal a dollar from your pocket, Bob will steal it instead, so Alice might as well steal that dollar even if the result is disastrous. Which means that in many cases the little extractions do continue past the point where people riot. This is one reason I'm skeptical of the ability of a Guaranteed Basic Income to solve poverty in general, leaving aside various other technical problems. We increased economic productivity by a factor of 100 and there are still poor people. Is a GBI really going to be the last marginal improvement that solves it all? A GBI might still help – just like increasing economic productivity by a factor of 100 helped the people who are still living lives of awful suffering and desperation. But after you introduce a GBI, I'm guessing, there will be a number of factors that start to extract one more dollar here, one more dollar there. The Ferguson police department issues another arrest warrant per household, the state increases its court costs, hey, people can afford it now, they've got a GBI right? And what do you know, almost everyone will still have to get awful jobs just to survive. So it's not at all a side issue, or a mere bugaboo of the independent-minded, to think about the political power of a cheaper exit. To consider whether mature VR, and to a lesser extent, movable housing, might make it a little bit harder to extract value from victims anchored too solidly to run away. The mobility of labor might affect how fast the poverty equilibrium restores itself. I'm not saying that corporate taxes are the correct level of organization on which to have any tax at all... but it does happen to be the case that taxing corporate profits located in your country is very hard to do, at least to large corporations, because they just locate their profits somewhere else. Making individual human beings and small companies more mobile would grant them some of the same power of resistance. No, let me be more blunt. If your shiny new city would otherwise be generating a huge amount of excess value for the people inside the new city, and the people inside the city have no credible threat of exit, the people inside your city will not be allowed to keep that value. There are things in the ecology that like to eat free energy, and your city will not be allowed to keep that energy indefinitely if it is so temptingly available for a little more taking every year. It could be eaten by any level of regional government, or any organization empowered by any level of regional government. If you're dumb enough to let somebody patent the connection scheme of the modular foundations, they can let you build out the city, watch to see how much excess wealth is being generated, and then jack up fees to try to capture nearly all of that value. It could be an invasion of patent monsters under a national jurisdiction that permits them. It only takes one factor that can threaten to shut down your whole process, to extract nearly all of the free energy from your city. So if your well-meaning goal is to make your new city generate lots of excess wealth that the people inside the city get to keep, you'd better make it as cheap as possible for coordinated groups to leave. Benefit #7: Lower-frictional flow of people through economies. I'll finish by remarking that, in a very generic and boring sense, friction is usually bad for economies and reduced friction is usually good. It is a terrifying sign of stagnation that people in the United States, especially the less wealthy states, are moving house less often. Housing prices are probably a bigger part of that problem than any one-time cost of shuffling possessions. But every little bit helps, and if you reduce the physical cost and emotional wear of moving from Point A to Point B, it will matter some. Alleged benefits #3 to #6 mostly reduce to, "Some nice things might happen if we reduce the cost and friction of being somewhere else". So benefit #7 is just the generic observation that movable houses would reduce economic friction in a very generic way and therefore other nice things might happen as well. We could also see movable housing as a kind of modularity that has the same kind of benefits as modularity in code; you can think about fewer things at a time. Time to extend the city? No need to plan Housing Projects and Development. Your job is just to dig a bunch of new foundations, and hook them up to water and electricity and Internet and roads. There, now your city is bigger. Everything else will sort itself out. 3: Conclusion. I would put high odds against any of this happening in real life before further events are derailed by a near-lightspeed expanding front of von Neumann machines eating the galaxy (as happens in both good and bad AI scenarios). But if advanced AI does happen to take that long, or if Elon Musk gets bitten by the movable housing bug and makes it happen in two years, I'd enjoy living in Rivendell for some of the remaining time. Imagine living in a higher-tech bigger nicer house, with robotic cars to teleport you wherever, and drones to deliver hot meals. Imagine that the rents not being so damned high – or even that 18th-century utopia where you just pay rent instead of rent plus tax – has produced a decrease of economic friction and a corresponding economic boom, so that it's less hard for people to make a living and get by. Is that a little more like that legendary future our parents were promised, of which it is said that instead we got 140 characters? Better steel factories didn't just produce shinier steel, back in the day; people made more tools that required steel, and used those steel tools to make other things. Scalable cities are something in that class. Movable houses, I think, might be a significant incremental piece of something in that class. To me, it seems nearly certain that none of that will actually happen. This essay is not a prediction of future glories ahead of us, more of a wistful sigh. If we lived in a civilization where we could have nice things at that complexity level, we'd already have nicer versions of much simpler things. But so long as Y Combinator is asking for essays on new cities anyway, this seemed worth writing up. I hope you enjoyed it! PS: Please pave the sidewalks with the bouncy kind of pavement so that people can run on sidewalks without destroying their knees. Discuss ### The EMH Aten't Dead Новости LessWrong.com - 15 мая, 2020 - 23:44 Published on May 15, 2020 8:44 PM GMT Cross-posting from my personal blog, but written primarily for Less Wrong after recent discussion here. There are whispers that the Efficient-Market Hypothesis is dead. Eliezer's faith has been shaken. Scott says EMH may have been the real victim of the coronavirus. The EMH states that “asset prices reflect all available information”. The direct implication is that if you don’t have any non-available information, you shouldn’t expect to be able to beat the market, except by chance. But some people were able to preempt the corona crash, without any special knowledge! Jacob mentioned selling some of his stocks before the market reacted. Wei Dai bought out-of-the-money 'put' options, and took a very handsome profit. Others shorted the market. These people were reading the same news and reports as everyone else. They profited on the basis of public information that should have been priced in. And so, the EMH is dead, or dying, or at the very least, has a very nasty-sounding cough. I think that rumours of the death of efficient markets have been greatly exaggerated. It seems to me the EMH is very much alive-and-kicking, and the recent discussion often involves common misunderstandings that it might be helpful to iron out. This necessarily involves pissing on people's parade, which is not much fun. So it's important to say upfront that although I don't know Wei Dai, he is no doubt a brilliant guy, that Jacob is my favourite blogger in the diaspora, that I would give my left testicle to have Scott's writing talent and ridiculous work ethic, that Eliezer is a legend whose work I have personally benefited from greatly, etc. But in the spirit of the whole rationality thing, I want to gently challenge what looks more like a case of 'back-slaps for the boys' than a death knell for efficient markets. First: how the heck did the market get the coronavirus so wrong? The Great Coronavirus Trade Lots of people initially underreacted to COVID-19. We are only human. But the stockmarket is not only human—it’s meant to be better than this. Here’s Scott, in A Failure, But Not of Prediction: The stock market is a giant coordinated attempt to predict the economy, and it reached an all-time high on February 12, suggesting that analysts expected the economy to do great over the following few months. On February 20th it fell in a way that suggested a mild inconvenience to the economy, but it didn’t really start plummeting until mid-March – the same time the media finally got a clue. These aren’t empty suits on cable TV with no skin in the game. These are the best predictive institutions we have, and they got it wrong. But… this isn't how it went down. As AllAmericanBreakfast and others pointed out in the comments, the market started reacting in the last week of February, with news headlines directly linking the decline to the ‘coronavirus’. By the time we get to mid-March, we’re not far off the bottom. (You can confirm this for yourself in a few seconds by looking at a chart of the relevant time period.) For whatever reason, COVID-19 seems to be a magnet for revisionist history and/or wishful thinking. In other comments under the same post, the notion that people from our ‘tribe’ did especially well also comes under serious question—in fact, it looks like many of the names mentioned seem to have jumped on the bandwagon after it was obvious, and certainly long after the market was moving. The facts are that the market reacted faster than almost all of us. But not before a few prescient people placed their bets! So now the question becomes: why didn’t the market react earlier than February 20, like those smart people did? The null hypothesis is that the market reacted exactly appropriately on the basis of the information available. After all, there were other potential pandemics in the recent past that were successfully contained or eradicated. On February 20, there were only 4 known cases in Italy. We were a long ways from the bloodbath that was coming. Maybe it was correct to move cautiously until further information came in? Here's AllAmericanBreakfast: [At that time] it may have been plausible for many people to think this would continue to play out like SARS – East Asia would solve their problem, everyone else would watch airport arrivals and quarantine them effectively, and within a few weeks everything would stabilize and gradually go away. By Feb. 27 it was clear that this wasn’t happening, since community spread was very clear from the public data in Italy and Iran, and probably also clear from genetic data in the United States and elsewhere. So we reached a tipping point in those next few days, at which point, the market started responding more vigorously. If the null hypothesis is true, then those early trades were not quite as prescient as they look. We might be making the mistake of ‘resulting’, and confusing the reality we ended up in with all the others which were possible at the time, in which those traders lost their shirts. It's really hard to have a useful object-level discussion about this, because these events are one-offs (this is the same argument we get into every year on Scott's predictions threads!) It's not like we can run the experiment again, and thank goodness for that. Nevertheless, Wei Dai suggested that this was the final nail in the coffin of EMH—at least for him. I want to pause here to give mad props to Wei Dai for being totally open about everything, and especially for doing the following: • warning that options are dangerous, and you can easily lose the lot • generic disclaimer about seeking financial advice • updating the thread after he ended up losing 80% of his paper profits • mentioning selection bias, and saying we'd be right to discount his evidence Which only leaves the initial claim that "at least for me this puts a final nail in the coffin of EMH." This is a polite way of hinting that you might be a brilliant investing wizard with the power to beat the market. Honestly, after making such a beautiful trade—and my gosh it really was beautiful—whom amongst us could resist that temptation? Certainly not me. And anyway, it might even be true! In making sense of claims of this nature, the first thing we have to establish: what does it even mean to be able to beat the market? Can Uncle George Beat the Market? Uncle George really likes his new iPhone. Man, these things are nifty! The dancing poop emoji is hilarious. On the strength of this insight, George dials his broker and loads up on AAPL stock. the chosen one! the scourge of efficient markets! the stuff of eugene fama’s nightmares! Over the next year, AAPL stock goes up 15 per cent, while the broader S&P 500 only goes up 10 per cent. George becomes insufferable at family dinners as he holds forth on his stock-picking powers. Guess the market isn’t so ‘efficient’ after all, huh. Suck it, Eugene Fama! So: did Uncle George beat the market? In the narrowest possible sense… yes. In the sense in which we aim to string words together so that they mean things: no, of course not. By this definition, every single trade leads to one of the two parties ‘beating the market’. Millions of people beat the market while I wrote this sentence. I can flip a coin between Pepsi and Coke right now, and have a 50 per cent chance of becoming a market-beating genius. The Uncle George example makes it glaringly obvious that a successful trade does not somehow ‘break’ efficient markets. And yet, this is the same naive criticism constantly leveled against the EMH: if the market moves in literally any direction, that must mean it was wrong before! My cousin who sold/bought before it went up/down beat the market! Same goes for the Great Coronavirus Trade. The fact that some people got out of the market early is not even the tiniest bit surprising. Investors constantly think the market is going to crash, for any number of plausible reasons. This is the default state of affairs: we have successfully predicted 73 of the last five market crashes, etc. These predictions are almost always wrong: And almost all the people who make them would have been much better off taking the boring ‘buy and hold forever’ strategy. But even a stopped clock is right twice a day. And of course, we’re much more likely to hear about the occasional brilliant successes than the near-constant dull failures. So the most naive criticism of the EMH boils down to ‘it's possible to make a good trade’. This is just a property of trading. It tells us exactly nothing about the market’s efficiency, or lack thereof. But some people really do beat the market—and not in the trivial sense. They're not merely stopped clocks, or highly visible 'survivors'. I’ll suggest a definition later on which strips out the effect of randomness. Before we get there—doesn’t the concession that people can non-trivially beat the market already drive a stake through the EMH’s heart? This is the second great misunderstanding: there is no conflict between the EMH and beating the market. That’s how the market gets efficient! You find an information asymmetry that isn’t priced in yet, and in exploiting it, you move the market a little further towards efficiency. Let’s call this information asymmetry an ‘edge’. If the EMH is true—or even just true-ish—that doesn’t mean the market can’t be beat. It means: You shouldn’t expect to beat the market without a unique edge, except by chance Now, this usually gets simplified down to ‘you can’t beat the market’. And most of the time, this simplification is good enough: you might get lucky and win in the Uncle George sense, but over an investing lifetime, you’ll almost certainly revert to the mean (which isn’t matching the market return—it’s underperforming it). But if you can find some kind of edge, you really can win! So, what might a genuine edge look like? Anomalies Exist! The Uncle Georges of the world don’t have an edge. All of their thoughts have already been thunk by someone else (probably by millions of someone elses). Instead, their fortunes are entirely at the mercy of the myriad other forces that drive stock prices: consumer demand, workplace harassment scandals, money printers going brrr, the exact virulence of a novel coronavirus, the price of cheese in Spain last Friday afternoon, etc. All of this stuff—billions of inputs processed by the greatest collective intelligence ever built—is a black box unto us mere mortals. It’s impossible to assign perfect causal explanations to stock prices, which means we can pick whichever story suits us best. As a market reporter, this was pretty much my whole job: calling brokers and economists to wrap a plausible narrative around totally inexplicable events, and generate sage nodding of heads. All Uncle George can see is that he placed his bet, and AAPL went up. It was the poop emoji for sure! And so, Uncle George spends his days dishing out hot stock tips on online forums, oblivious to the fact that his success was meaningless. [uncle georging intensifies] What does a real edge look like? A century ago, investors started noticing they could consistently pick up bargains by running very simple formulas over stock prices. The most famous is the ‘value investing’ approach developed by Ben Graham, and used by Warren Buffett and Charlie Munger. There was a genuine, big old inefficiency in the markets, and these guys had a great time exploiting it. I think this might be the image most people have in their head when they think of ‘beating the market’—diligently studying The Intelligent Investor and learning about PE ratios or whatever. But this is like trying to use a stone-age axe against a fighter jet. The Ben Graham information asymmetry has long since disappeared because…markets are efficient(ish)! Once the formula was widely known, it stopped working. Investors developed more sophisticated versions, more formulas, more pricing models. Once those got out, they stopped working too. Now there’s a great debate as to whether even the most complicated descendants of value might be totally dead. In which case, the anomaly has officially gone for good. Either way, this is not how Buffett gets his edge, and it hasn’t been for decades. Here’s Munger: The trouble with what I call the classic Ben Graham concept is that gradually the world wised up and those real obvious bargains disappeared. You could run your Geiger counter over the rubble and it wouldn’t click. Buffett’s most brilliant achievement is weaving this folksy legend that he is a cute old grandpa who beats the market by backing the best companies. Let's take a look at how market-beating investors really make their money. Modern Edges are Completely Ridiculousgreatest showman on earth1. The Warren Buffett Halo Effect In recent decades, Buffett has made a killing through juicy private deals which are completely out of reach of the average investor. Like, six billion dollar deals with three billion in preference rights and a guaranteed dividend. Like, lobbying the government to bail out the banks, then carving off a huge piece of the action. Like, being able to play around with Berkshire Hathaway’s$115 billion insurance float. Much of his fortune is built on taxpayer largesse.

Warren Buffett’s brand is so powerful that at this point, his success is a self-fulfilling prophecy: when Berkshire invests in a stock, everyone else piles in after him and drive the price up. Buffett even lends out his ‘halo’ to companies that need it—most famously during the GFC—so long as they give him a generous discount to the market price, of course. (Matt Levine has written some fascinating columns about this).

And yet, and yet… Berkshire Hathaway has underperformed for the last decade. Buffett would have been better off to take his own advice and put it all in index funds.

2. Hedge funds with armies of drones

There you are, sitting in your home office going through Walmart’s quarterly report and calculating PE ratios or whatever. Meanwhile, the professionals are using an army of drones to monitor the movement of shopping carts in Walmart parking lots in real time.

See also: sending foot soldiers out to every branch of a bakery chain at the close of business each day, because the numbered dockets start out at zero, and thus contain live sales data unavailable to the market.

And so, when renowned hedge fund manager Michael Steinhardt was asked the most important thing average investors could learn from him, here’s what he suggested:

“I’m their competition.”

And yet, and yet…almost all hedge funds underperform. Not all of them are trying to beat the market, but the tools at their disposal gives us a sense of the difficulty here.

If multiple people have access to the same information, the speed in which you can bring it to market also matters. So, we have high-frequency traders.

One firm spent \$300 million laying a direct cable between Chicago to New Jersey. They cut straight through mountains and crossed rivers. The cable stretched 1331 kilometres. And they did this to shave four milliseconds off their transmission time.

And yet, and yet…microwaves came along and rendered the whole project obsolete. Trying to get an edge is expensive.

4. Being willing and able to commit felonies

And yet, and yet…even when criminals have advance access to earnings reports, they still don’t do all that well, which is evidence for the very strongest form of the EMH (the one that no-one, including me, believes can possibly be true).

So…what sort of edge do us lesser mortals have?

If we mumble something about having ‘good intuition’, or ‘subscribing to the Wall Street Journal' then we should consider the strong possibility that we are Uncle George.

If the answer involves ‘fundamental analysis’ or ‘Fibonacci retracements’, we're still in Uncle George territory. The only difference is that doing something complicated makes it easier to internally justify the belief that we know a secret no-one else does. But it is still (probably!) a mistaken belief.

The EMH Gets Stronger With Every Attack

So we know for sure that market-beating edges exist—I’ve even written them down for everyone to see!

I can only dream of possessing a halo effect so strong that everyone piles into a stock right after I announce I have graced it with my favour. I don’t have an army of drones at my command, or the ability to bore through mountains to shave milliseconds off my trading times, or a weekly round of golf with the CEO of a Fortune 500 company.

The market is never perfectly efficient. But relative to me, it might as well be.

Critics have pointed out plenty of cases in which the EMH doesn’t jive with reality—and they are absolutely right. So this is where it gets really weird.

The EMH is the only theory that grows stronger with every attack against it.

Every edge is constantly at risk of being gobbled up by an efficient-ish market. The ones I’ve mentioned can only be publicly knowledge because they're somewhat defensible: they’re based on personal relationships, capital investment, proprietary technology, etc. But they disappear too.

Most edges can’t even be spoken out loud without disappearing. If stocks systematically rise on the third Thursday of each month but only under a waxing moon, and then someone writes about it in public, you can kiss that anomaly goodbye. The EMH sucks it into its gigantic heaving maw, and it’s gone forever.

In other words: every time someone picks a hole in the theory and points out an inefficiency, they make the predictions generated by the EMH more robust! It’s like some freaky shoggoth thing that Just. Won’t. Die.

you may not like it, but this is what peak efficiency looks like

Which gets us to the totally justified criticism of the theory: the only reason the EMH can pull this stunt is because it’s bullshit science.

It’s unfalsifiable! It responds to criticism by saying, ‘OK, good point, but now that I’ve factored that in, you should believe in my theory even more.’

And…we really should?

The only way I can think about the EMH without going insane is to remind myself that it generates a useful heuristic. It’s not a stable law, like we might find in hard sciences. It’s not perfectly accurate. At any given point in time, there are always competing models that do a better job of describing reality. But all those other models can stop working at any moment, with no warning! By the time you find out their predictive power is gone, it’s too late, and you probably lost a bunch of money! By contrast, the EMH is a reliable model—reliably vague and hand-wavy, yes, but also reliably useful.

We know there are inefficiencies in the market. In the fullness of time, they will be absorbed into the gelatinous alien-god’s hivemind. But before that happens, maybe we can make some money off of them.

So now we come to the final test. How do you tell if you’ve really found a market-beating edge—that is, a model of reality that has more predictive power than the EMH—or you’re fooling yourself like Uncle George?

If You’re so Smart, Why Aren’t You Rich?

Everyone knows a secret about themselves, or the people they know well, or can arbitrage some opportunity in a niche that few people are paying attention to. These illiquid private ‘markets’ are much more fertile hunting grounds for asymmetries, and something I encourage everyone to think about.

But the public security markets are a gigantic agglomeration of everyone’s predictions, which constantly hoovers up every new fragment of information, and recalibrates itself in real time. Your challenge is to try and predict why this giant meta-prediction is wrong, and in which direction.

If you think you can reliably beat, say, an index fund that passively tracks the S&P 500, this is a much stronger claim than it first appears.

For one thing, you’re claiming to be better than Warren Buffett, who has failed to pull this off in the last 10 years, despite his huge advantages, and has started saying the game is so hard that everyone should just buy index funds. But that’s nothing. You are also claiming that you have the power to beat the greatest collective intelligence humanity has ever created.

This is an extraordinary claim, and the thing about extraordinary claims is that they require extraordinary evidence.

Uncle George’s AAPL trade ain’t going to cut it. Here is the extraordinary evidence that I would personally want to see before agreeing that an investor can beat the market:

1. Big heaps of money

This is the one area of life where there really is no dodging that most venerable of sick burns: if you’re so smart, why aren’t you rich?

So the first piece of evidence I would accept is the fact that someone is very, very rich. Sitting atop a big old pile of cash. And they’d probably also open a hedge fund so they can take other people’s money and turn it into millions more.

2. Track record of outperformance

Maybe a genuine market-beater doesn’t have enough starting capital to make big piles of money on a relatively slim edge, and for some reason is unable to come up with any scheme to beg or borrow more? Or the anomaly is real, but disappears before they can get filthy rich?

In these scenarios, I would also accept a complete record of out-of-sample investment returns over time—no backtests! no selecting the best trades!—as compared against the appropriate risk-adjusted benchmark.

These evidential standards work both in the case that the EMH is ‘dead’, i.e. you can reliably beat the market using public domain info which ought to already be priced in, and in the case that it’s not, i.e. you really do need novel information to get an edge.

As for evidence that the EMH really is dead…hmm. It’s not a proper theory to begin with. But I guess it would be ‘dead’ when the predictions it generates stop being accurate or useful? Which would look something like: ‘finding out that plenty of people meet the standards above, despite having never been in possession of a scrap of information that wasn’t already available to the market’.

In doing so, we’d have to be very careful to make sure we aren’t just looking at the Uncle Georges who unwittingly drew the winning lottery ticket. After all, we should expect plenty of investors to beat the market for a long time—even for years on end—entirely by chance.

The Great Coin-Tossing Experiment

Say we held a national coin-flipping contest. After 15 rounds, one in every ~32,800 people would have managed to call every single toss correctly, perfectly predicting a sequence like this:

H T H H T T H H H H T H T T T

Pretty impressive, huh!

Well, only in a world where we don’t know about probability. In that world, we might mistake blind randomness for skill. The lucky few winners would be hailed as the heroes of their hometowns, do interviews with breathless breakfast TV hosts, and explain that it’s all about the precise flick of the wrist. Aspiring flippers would queue up to buy the inevitable best-selling book, Flip Me Off, and pay exorbitant sums for one-on-one coaching sessions with the master tossers.

Depressingly, this is exactly what happens in the world of investing. What does it mean to achieve the kind of success which only happens by chance with 0.0003 probability? In the United States alone, it means you end up with 10,000 lucky dopes who are indistinguishable from brilliant investors.

And fund managers don’t need to do anywhere near that well to attract a market-beating aura. They’re incentivised to swing for the fences, increasing the odds they beat the market in some highly visible fashion over some shorter period—say, a lucky season or two. They inevitably regress to the mean, sometimes crashing and burning in spectacular fashion, but it doesn’t matter so long as they manage to hose naive investors in the meantime.

We can never entirely rule out the effect of randomness—there will always be some tiny chance that Warren Buffett is really just the world’s greatest coin-flipper—but we have to draw the line somewhere, or the standard is impossibly high.

Once the odds of a fluke get pretty slim—someone is super duper rich, and they’ve made a ton of consistently good trades over time—I’d happily congratulate them on their market-beating prowess, give them all my money to invest, listen eagerly to their advice, etc.

Being good Bayesians, this is obviously a spectrum: if they are not at that point, but trending in the right direction, I would be less skeptical, etc. But the bar has to be pretty high, or there's no way to separate skill from randomness.

Scott and Eliezer have both alluded to their comments being informed by private information. Here's a reddit comment in which Scott responds to criticism of his 'EMH is the real victim' riff:

I think we're in an asymmetric position, in that I know these people pretty well, I know they've thought about efficient market before, and they're the sort of people I would expect to beat the market if anyone could. I agree that if you just hear some blogger say he saw some people beat the market once, that's not much evidence.

Eliezer definitely understands the EMH, because the descriptions of it in Inadequate Equilibria are among the most brilliant and insightful I've ever come across. And Scott is obviously super smart.

I would love to know what their evidential standards are, but I'm explicitly not calling them out, or any of the people mentioned earlier in the post. No-one is under the slightest obligation to share private evidence, and I would be thrilled if those folks were indeed the market-beating chosen ones.

But I am saying that people, in general, make these kind of claims all the time—in good faith and with no malicious intent—and in general, taking their advice is an extraordinarily bad idea.

A heuristic: if you (or someone you know) is confident they can beat the market, and yet you notice you are not sitting atop an enormous pile of wealth, it's at least worth considering the possibility that you might be fooling yourselves.

The Four Types of Investors

There are very obvious and well-known reasons why everyone loves to think they can beat the market: overconfidence, confirmation bias, ‘resulting’, selective memory, survivorship bias, etc.

These forces are so powerful that many people—myself included—blithely ignore the vast piles of evidence that suggest beating the market is incredibly difficult, and go ahead and try anyway. All of us think we are special, and (almost) all of us are wrong.

Even if we don't personally harbour this particular fantasy, there's also a natural tendency to want our tribe or our friends to be the brilliant visionaries who were ahead of the action, possess sweet market-beating skills, etc.

So we can roughly place investors into one of the following four quadrants:

('Losers' and 'winners' here is tongue-in-cheek, and not a value judgment: literally, losing/winning this game by either successfully beating the market, or failing to do so)

Deluded losers (‘Suckers’)

“Apple stock really is undervalued, but the market hasn’t recognised it yet. I just got unlucky—it was because of [elaborate rationalization]. Also, even if I got it wrong this time, I was really close. Next time!

“What’s that? Do I track my portfolio returns over time, and compare against the relevant risk-adjusted benchmark to see whether I’m actually outperforming? Well, there’s no need. I usually do pretty well for myself, and I’m expecting to improve—in fact, I just picked up this classic book called The Intelligent Investor…

Deluded winners (‘Dumb Luck’)

“I knew Apple stock was undervalued! And I remember that other time I made a really good trade, too. Guess I’m pretty good at this game!

“…What’s that? I might have just got lucky? Hah, no. I even did the Fibonacci retracements and everything."

Realistic losers (‘Clear-Eyed Fools’)

“I keep a meticulous record of my portfolio returns, which forces me to acknowledge the fact that even though I occasionally do well, I am underperforming my benchmarks on a risk-adjusted basis. I am under no delusions about my prospects of finding an edge, and I know I really ought to take Warren Buffett’s advice and put all my money in index funds.

“But I enjoy playing the markets! It's like how a night in Vegas can have negative EV but still be positive utility, because of all the non-financial factors. So I’m gonna keep gambling with a small part of my portfolio, just for shits and giggles. In the event that I ‘win’, I will try really hard to resist the incredible internal pressure to start thinking of myself as a brilliant investing guru."

Realistic winners (‘Chosen One’)

“I keep a meticulous record of my portfolio returns, which have outperformed the appropriate risk-adjusted benchmarks to such a degree that I am confident I have found a genuine informational asymmetry. I will of course never tell anyone about it, or it will become useless.

“And I can never be entirely sure: it’s also possible that I just got lucky. But at the very least, I am sitting atop great piles of money, which is pretty nice.”

***

The vast majority of people who actively trade their account are ‘Suckers’. Some smaller number fall into the ‘Dumb Luck’ quadrant (Uncle George would stay there if he never places another trade, but he almost certainly won’t be able to help himself.)

The right-hand quadrants are much more sparsely populated. I guess there are a few ‘Clear-Eyed Losers’ floating around, and a tiny handful of ‘Chosen Ones’.

This rough distribution is probably not too controversial. The question is, which one are you?

(I made a poll on my website at this point in the post, just for a bit of fun: the results so far are brilliant)

Trying to Beat the Market is Like Crack for Smart People

There is a tendency for smart people to wander into areas they know very little about, and think they can do better than the actual experts who have years or decades of domain-specific knowledge, on the basis of being very smart, or having read some blog posts online about being more rational.

This would be OK if it was just a bit cringe. I love armchair pontificating as much as the next guy! The consequences are usually limited to mildly annoying the people who actually know what they’re talking about, and much eye-rolling when you triumphantly reinvent the wheel.

There is some upside too: reinventing the wheel is fun, because you get to, like, invent wheels. And very occasionally, it might even be true! No doubt smart outsiders are occasionally able to breeze into a new field and exploit some obvious inefficiencies.

But…oh boy. It’s really not true of this particular domain. And it’s not harmless either.

The central prediction generated by the EMH is that you should not expect to be able to beat the market (in the non-trivial sense) unless you have unique information or some similar edge.

This prediction is tested every day. We have great piles of evidence which suggest that it is correct: the vast majority of active investors do really badly.

Crucially, it’s not only regular schmucks who underperform. So do paid professionals, and active managers, and hedge funds, and all sorts of brilliant people who have made this their life’s work.

It's possible that I am not making many friends with this post. I certainly feel pretty nervous about publishing it. Everyone who thinks they can beat the market will have their hackles up! If it helps at all, I am not claiming the high ground. I have made every dumb investing mistake you could think of, and then a few more besides. I am painfully aware of how hard this is.

These days I would put myself in the ‘Clear-Eyed Fool’ quadrant, but only by a fingernail. It’s a constant battle even to stay there. I still do clever things that contradict my own boring advice, and annoyingly, am rewarded for my hubris just often enough to start entertaining the thought that I’m a brilliant investing genius after all. Then I force myself to calculate the IRR on my publicly-traded investments, and compare it against appropriate benchmarks, and manage to get a fingernail-hold back on boring old reality.

To the extent that I have succeeded as an investor, and I am doing quite nicely thank you, it has only come through forcing myself to acknowledge the main prediction that emerges from the very-much-alive-and-kicking EMH. The huge and underappreciated benefit of doing so is that I occasionally divert some of my attention elsewhere, to domains where I actually do have an edge—and then I win.

Discussion

The EMH is a weird, counterintuitive, freaky-ass shoggoth of a thing and it still confuses the heck out of me, even after almost a decade of writing about finance. I almost certainly made some mistakes in the post—in the process of writing it, I noticed several ways in which my initial beliefs were subtly wrong, which has already been super useful for me, and helped me understand some of the (valid) objections people have raised.

So I would also like to use this post to open up the floor to any and all EMH discussion, and try to benefit from the smaller-but-still-powerful collective intelligence that is Less Wrong.

I'm going to add comments with some of my own questions and uncertainty. It would be nice to become less confused together, and try to get a better sense of where we should apply our efforts at the margin.

Discuss

### Dishwasher Filter

Новости LessWrong.com - 15 мая, 2020 - 23:30
Published on May 15, 2020 8:30 PM GMT

A few months ago our dishwasher started building up food inside the sprayer arms. It happened pretty suddenly: within a few days I was needing to pick things out daily. I initially thought the dishwasher had been run with the filter not properly seated and food had gotten into the water recirculation system. I reseated the filters and waited a while, but it wasn't getting better.

It all still mostly worked, but only if we scraped our dishes well and if I spent a frustratingly large fraction of my time picking at the sprayer arms with a bent paperclip. Still less work than handwashing, though not by much. I read the manual, forums, etc; couldn't figure it out. Eventually we called the manufacturer and they sent someone out. They pulled the filter, and saw that the macro filter (left) was present but the micro filter (right) just wasn't there.

As soon as the tech pointed out that the filter was missing this very clearly explained what we'd been seeing. I ordered a new one, installed it, and the dishwasher worked perfectly again. What I really don't understand, though, is how the micro filter could have gone missing?

Discuss

### Subspace optima

Новости LessWrong.com - 15 мая, 2020 - 23:02
Published on May 15, 2020 12:38 PM GMT

The term "global optimum" and "local optimum" have come from mathematical terminology and entered daily language. They are useful ways of thinking in every day life. Another useful concept, which I don't hear people talk about much is "subspace optimum": A point maximizes a function not in the whole space, but in a subspace. You have to move along a different dimension than those of the subspace in order to improve. A subspace optimum doesn't have to be a local optimum either, because even a small change along the new dimension might yield improvements. If you're in a subspace optimum, this requires a different attitude to get to a global optimum, than if you're in a local optimum, which makes me think it's good for the term to be part of every day language.

• When you're in a local optimum, you have to do something quite different from what you're doing to improve.
• When you're in a subspace optimum, you have to notice dimensions along which you could be doing things differently that you didn't even notice before, but small changes along those new dimensions might already help. You're applying constraints to yourself that you could let go.

Regarding how it looks subjectively:

• The phrase: "am I in a local optimum?" generates curiosity about whether you maybe should undertake a quite different plan from the one you're taking now. (Should I do a different project, rather than make local changes to the project I'm taking?)
• The phrase: "am I in a subspace optimum?" generates curiosity about whether you maybe are not noticing (possibly small) changes you could be making across dimensions you haven't been considering. (Should I optimize/adjust the way I'm doing my project across different dimensions/variables than the ones I've been optimizing over so far?)

My impression is that somewhat often when people informally use the term local optimum, they are in fact talking about a subspace optimum.

Bonus for the theoretically inclined: A local subspace optimum is one where you can improve by temporarily doing things differently along dimension X, moving around in a bigger space, while eventually ending up on a different, better, point in the same subspace.

Discuss

### Craving, suffering, and predictive processing (three characteristics series)

Новости LessWrong.com - 15 мая, 2020 - 16:21
Published on May 15, 2020 1:21 PM GMT

This is the third post of the "a non-mystical explanation of insight meditation and the three characteristics of existence" series. I originally intended this post to more closely connect no-self and unsatisfactoriness, but then decided on focusing on unsatisfactoriness in this post and relating it to no-self in the next one.

Unsatisfactoriness

In the previous post, I discussed some of the ways that the mind seems to construct a notion of a self. In this post, I will talk about a specific form of motivation, which Buddhism commonly refers to as craving (taṇh&#x101; in the original Pali). Some discussions distinguish between craving (in the sense of wanting positive things) and aversion (wanting to avoid negative things); this article uses the definition where both desire and aversion are considered subtypes of craving.

My model is that craving is generated by a particular set of motivational subsystems within the brain. Craving is not the only form of motivation that a person has, but it normally tends to be the loudest and most dominant. As a form of motivation, craving has some advantages:

• People tend to experience a strong craving to pursue positive states and avoid negative states. If they had less craving, they might not do this with an equal zeal.
• To some extent, craving looks to me like a mechanism that shifts behaviors from exploration to exploitation.
• In an earlier post, Building up to an Internal Family Systems model, I suggested that the human mind might incorporate mechanisms that acted as priority overrides to avoid repeating particular catastrophic events. Craving feels like a major component of how this is implemented in the mind.
• Craving tends to be automatic and visceral. A strong craving to eat when hungry may cause a person to get food when they need it, even if they did not intellectually understand the need to eat.

At the same time, craving also has a number of disadvantages:

• Craving superficially looks like it cares about outcomes. However, it actually cares about positive or negative feelings (valence). This can lead to behaviors that are akin to wireheading in that they suppress the unpleasant feeling while doing nothing about the problem. If thinking about death makes you feel unpleasant and going to the doctor reminds you of your mortality, you may avoid doctors - even if this actually increases your risk of dying.
• Craving narrows your perception, making you only pay attention to things which seem immediately relevant for your craving. For example, if you have a craving for sex and go to a party with the goal of finding someone to sleep with, you may see everyone only in terms of “will sleep with me” or “will not sleep with me”. This may not be the best possible way of classifying everyone you meet.
• Strong craving may cause premature exploitation. If you have a strong craving to achieve a particular goal, you may not want to do anything that looks like moving away from it, even if that would actually help you achieve it better. For example, if you intensely crave a feeling of accomplishment, you may get stuck playing video games that make you feel like you are accomplishing something, even if there was something else that you could do that was more fulfilling in the long term.
• Multiple conflicting cravings may cause you to thrash around in an unsuccessful attempt to fulfill all of them. If you crave to get your toothache fixed, but also a craving to avoid dentists, you may put off the dentist visit even as you continue to suffer from your toothache.
• Craving seems to act in part by creating self-fulfilling prophecies; making you strongly believe that you are going to achieve something, so as to cause you to do it. The stronger the craving, the stronger the false beliefs injected into your consciousness. This may warp your reasoning in all kinds of ways: updating to believe an unpleasant fact may subjectively feel like you are allowing that fact to become true by believing in it, incentivizing you to come up with ways to avoid believing in it.
• Finally, although craving is often motivated by a desire to avoid unsatisfactory experiences, it is actually the very thing that causes dissatisfaction in the first place. Craving assumes that negative feelings are intrinsically unpleasant, when in reality they only become unpleasant when craving resists them.

Given all of these disadvantages, it may be a good idea to try to shift one’s motivation to be more driven by subsystems that are not motivated by craving. It seems to me that everything that can be accomplished via craving, can in principle be accomplished by non-craving-based motivation as well.

Fortunately, there are several ways of achieving this. For one, a craving for some outcome X tends to implicitly involve at least two assumptions:

1. achieving X is necessary for being happy or avoiding suffering
2. one cannot achieve X except by having a craving for it

Both of these assumptions are false, but subsystems associated with craving have a built-in bias to selectively sample evidence which supports these assumptions, making them frequently feel compelling. Still, it is possible to give the brain evidence which lets it know that these assumptions are wrong: that it is possible to achieve X without having craving for it, and that one can feel good regardless of achieving X.

A predictive processing look on unsatisfactoriness

I find that a promising way of looking at unsatisfactoriness and craving and their impact on decision-making comes from the predictive processing (PP) model about the brain. My claim is not that craving would work exactly like this, but something roughly like this seems like a promising analogy.

Good introductions to PP include this book review as well as the actual book in question... but for the purposes of this discussion, you really only need to know two things:

• According to PP, the brain is constantly attempting to find a model of the world (or hypothesis) that would both explain and predict the incoming sensory data. For example, if I upset you, my brain might predict that you are going to yell at me next. If the next thing that I hear is you yelling at me, then the prediction and the data match, and my brain considers its hypothesis validated. If you do not yell at me, then the predicted and experienced sense data conflict, sending off an error signal to force a revision to the model.
• Besides changing the model, another way in which the brain can react to reality not matching the prediction is by changing reality. For example, my brain might predict that I am going to type a particular sentence, and then fulfill that prediction by moving my fingers so as to write that sentence. PP goes so far as to claim that this is the mechanism behind all of our actions: a part of your brain predicts that you are going to do something, and then you do it so as to fulfill the prediction.

Next I am going to say a few words about a phenomenon called binocular rivalry and how it is interpreted within the PP paradigm. I promise that this is going to be relevant for the topic of craving and suffering in a bit, so please stay with me.

Binocular rivalry, first discovered in 1593 and extensively studied since then, is what happens when your left eye is shown one picture (e.g. an image of Isaac Newton), and your right eye is shown another (e.g. an image of a house) in the right. People report that their experience keeps alternating between seeing Isaac Newton and seeing a house. They might also see a brief mashup of the two, but such Newton-houses are short-lived and quickly fall apart before settling to a stable image of either Newton or a house.

Image credit: Schwartz et al. (2012), Multistability in perception: binding sensory modalities, an overview. Philosophical Transactions of the Royal Society B, 367, 896-905.

Predictive processing explains what’s happening as follows. The brain is trying to form a stable hypothesis of what exactly the image data that the eyes are sending represents: is it seeing Newton, or is it seeing a house? Sometimes the brain briefly considers the hybrid hypothesis of a Newton-house mashup, but this is quickly rejected: faces and houses do not exist as occupying the same place at the same scale at the same time, so this idea is clearly nonsensical. (At least, nonsensical outside highly unnatural and contrived experimental setups that psychologists subject people to.)

Your conscious experience alternating between the two images reflects the brain switching between the hypotheses of “this is Isaac Newton” and “this is a house”; the currently-winning hypothesis is simply what you experience reality as.

Suppose that the brain ends up settling on the hypothesis of “I am seeing Isaac Newton”; this matches the input from the Newton-seeing eye. As a result, there is no error signal that would arise from a mismatch between the hypothesis and the Newton-seeing eye’s input. For a moment, the brain is satisfied that it has found a workable answer.

However, if one really was seeing Isaac Newton, then the other eye should not keep sending an image of a house. The hypothesis and the house-seeing eye’s input do have a mismatch, kicking off a strong error signal which lowers the brain’s confidence in the hypothesis of “I am seeing Isaac Newton”.

The brain goes looking for a hypothesis which would better satisfy the strong error signal… and then finds that the hypothesis of “I am seeing a house” serves to entirely quiet the error signal from the house-seeing eye. Success?

But even as the brain settles on the hypothesis of “I am seeing a house”, this then contradicts the input coming from the Newton-seeing eye.

The brain is again momentarily satisfied, before the incoming error signal from the hypothesis/Newton-eye mismatch drives down the probability of the “I am seeing a house” hypothesis, causing the brain to eventually go back to the “I am seeing Isaac Newton” hypothesis... and then back to seeing a house, and then to seeing a Newton, and...

One way of phrasing this is that there are two subsystems, each of which are transmitting a particular set of constraints (about seeing Newton and a house). The brain is then trying and failing to find a hypothesis which would fulfill both sets of constraints, while also respecting everything else that it knows about the world.

As I will explain next, my feeling is that something similar is going on with unsatisfactoriness. Craving creates constraints about what the world should be like, and the brain tries to find an action which would fulfill all of the constraints, while also taking into account everything else that it knows about the world. Suffering/unsatisfactoriness emerges when all of the constraints are impossible to fulfill, either because achieving them takes time, or because the brain is unable to find any scenario that could fulfill all of them even in theory.

Predictive processing and psychological suffering

There are two broad categories of suffering: mental and physical discomfort. Let’s start with the case of psychological suffering, as it seems most directly analogous to what we just covered.

Let’s suppose that I have broken an important promise that I have made to a friend. I feel guilty about this, and want to confess what I have done. We might say that I have a craving to avoid the feeling of guilt, and the associated craving subsystem sends a prediction to my consciousness: I will stop feeling guilty.

In the previous discussion, an inference mechanism in the brain was looking for a hypothesis that would satisfy the constraints imposed by the sensory data. In this case, the same thing is happening, but

• the hypothesis that it is looking for is a possible action that I could take, that would lead to the constraint being fulfilled
• the sensory data is not actually coming from the senses, but is internally generated by the craving and represents the outcome that the craving subsystem would like to see realized

My brain searches for a possible world that would fulfill the provided constraints, and comes up with the idea of just admitting the truth of what I have done. It predicts that if I were to do this, I would stop feeling guilty over not admitting my broken promise. This satisfies the constraint of not feeling guilty.

However, as my brain further predicts what it expects to happen as a consequence, it notes that my friend will probably get quite angry. This triggers another kind of craving: to not experience the feeling of getting yelled at. This generates its own goal/prediction: that nobody will be angry with me. This acts as a further constraint for the plan that the brain needs to find.

As the constraint of “nobody will be angry at me” seems incompatible with the plan of “I will admit the truth”, this generates an error signal, driving down the probability of this plan. My brain abandons this plan, and then considers the alternative plan of “I will just stay quiet and not say anything”. This matches the constraint of “nobody will be angry at me” quite well, driving down the error signal from that particular plan/constraint mismatch… but then, if I don’t say anything, I will continue feeling guilty.

The mismatch with the constraint of “I will stop feeling guilty” drives up the error signal, causing the “I will just stay quiet” plan to be abandoned. At worst, my mind may find it impossible to find any plan which would fulfill both constraints, keeping me in an endless loop of alternating between two unviable scenarios.

There are some interesting aspects about the phenomenology of such a situation, which feel like they fit the PP model quite well. In particular, it may feel like if I just focus on a particular craving enough, thinking about my desired outcome hard enough will make it true.

Recall that under the PP framework, goals happen because a part of the brain assumes that they will happen, after which it changes reality to make that belief true. So focusing really hard on a craving for X makes it feel like X will become true, because the craving is literally rewriting an aspect of my subjective reality to make me think that X will become true.

When I focus hard on the craving, I am temporarily guiding my attention away from the parts of my mind which are pointing out the obstacles in the way of X coming true. That is, those parts have less of a chance to incorporate their constraints into the plan that my brain is trying to develop. This momentarily reduces the motion away from this plan, making it seem more plausible that the desired outcome will in fact become real.

Conversely, letting go of this craving, may feel like it is literally making the undesired outcome more real, rather than like I am coming more to terms with reality. This is most obvious in cases where one has a craving for an outcome that is impossible for certain, such as in the case of grieving about a friend’s death. Even after it is certain that someone is dead, there may still be persistent thoughts of if only I had done X, with an implicit additional flavor of if I just want to have done X really hard, things will change, and I can’t stop focusing on this possibility because my friend needs to be alive.

In this form, craving may lead to all kinds of rationalization and biased reasoning: a part of your mind is literally making you believe that X is true, because it wants you to find a strategy where X is true. This hallucinated belief may constrain all of your plans and models about the world in the same sense as getting direct sensory evidence about X being true would constrain your brain’s models. For example, if I have a very strong urge to believe that someone is interested in me, then this may cause me to interpret any of his words and expressions in a way compatible with this belief, regardless of how implausible and far-spread of a distortion this requires.

The case of physical pain

Similar principles apply to the case of physical pain.

We should first note that pain does not necessarily need to be aversive: for example, people may enjoy the pain of exercise, hot spices or sexual masochism. Morphine may also have an effect where people report that they still experience the pain but no longer mind it.

And, relevant for our topic, people practicing meditation find that by shifting their attention towards pain, it can become less aversive. The meditation teacher Shinzen Young writes that

... pain is one thing, and resistance to the pain is something else, and when the two come together you have an experience of suffering, that is to say, 'suffering equals pain multiplied by resistance.' You'll be able to see that's true not only for physical pain, but also for emotional pain and it’s true not only for little pains but also for big pains. It's true for every kind of pain no matter how big, how small, or what causes it. Whenever there is resistance there is suffering. As soon as you can see that, you gain an insight into the nature of "pain as a problem" and as soon as you gain that insight, you'll begin to have some freedom. You come to realize that as long as we are alive we can't avoid pain. It's built into our nervous system. But we can certainly learn to experience pain without it being a problem. (Young, 1994)

What does it mean to say that resisting pain creates suffering?

In the discussion about binocular rivalry, we might have said that when the mind settled on a hypothesis of seeing Isaac Newton, this hypothesis was resisted by the sensory data coming from the house-seeing eye. The mind would have settled on the hypothesis of “I am seeing Isaac Newton”, if not for that resistance. Likewise, in the preceding discussion, the decision to admit the truth was resisted by the desire to not get yelled at.

Suppose that you have a sore muscle, which hurts whenever you put weight on it. Like sensory data coming from your eyes, this constrains the possible interpretations of what you might be experiencing: your brain might settle on the hypothesis of “I am feeling pain”.

But the experience of this hypothesis then triggers a resistance to that pain: a craving subsystem wired to detect pain and resist it by projecting a form of internally-generated sense data, effectively claiming that you are not in pain. There are now again two incompatible streams of data that need to be reconciled, one saying that you are in pain, and another which says that you are not.

In the case of binocular rivalry, both of the streams were generated by sensory information. In the discussion about psychological suffering, both of the streams were generated by craving. In this case, craving generates one of the streams and sensory information generates the other.

On the left, a persistent pain signal is strong enough to dominate consciousness. On the right, a craving for not being in pain attempts to constrain consciousness so that it doesn’t include the pain.

Now if you stop putting weight on the sore muscle, the pain goes away, fulfilling the prediction of “I am not in pain”. As soon as your brain figures this out, your motor cortex can incorporate the craving-generated constraint of “I will not be in pain” into its planning. It generates different plans of how to move your body, and whenever it predicts that one of them would violate the constraint of “I will not be in pain”, it will revise its plan. The end result is that you end up moving in ways that avoid putting weight on your sore muscle. If you miscalculate, the resulting pain will cause a rapid error signal that causes you to adjust your movement again.

What if the pain is more persistent, and bothers you no matter how much you try to avoid moving? Or if the circumstances force you to put weight on the sore muscle?

In that case, the brain will continue looking for a possible hypothesis that would fulfill the constraint of “I am not in pain”. For example, maybe you have previously taken painkillers that have helped with your pain. In that case, your mind may seize upon the hypothesis that “by taking painkillers, my pain will cease”.

As your mind predicts the likely consequences of taking painkillers, it notices that in this simulation, the constraint of “I am not in pain” gets fulfilled, driving down the error signal between the hypothesis and the “I am not in pain” constraint. However, if the brain could suppress the craving-for-pain-relief merely by imagining a scenario where the pain was gone, then it would never need to take any actions: it could just hallucinate pleasant states. Helping keep it anchored into reality is the fact that simply imagining the painkillers has not done anything to the pain signal itself: the imagined state does not match your actual sense data. There is still an error signal generated between the mismatch of the imagined “I have taken painkillers and am free of pain” scenario, and the fact that the pain is not gone yet.

Your brain imagines a possible experience: taking painkillers and being free of pain. This imagined scenario fulfills the constraint of “I have no pain”. However, it does not fulfill the constraint of actually matching your sense data: you have not yet taken painkillers and are still in pain.

Fortunately, if painkillers are actually available, your mind is not locked into a state where the two constraints of “I’m in pain” and “I’m not in pain” remain equally impossible to achieve. It can take actions - such as making you walk towards the medicine cabinet - that get you closer towards being able to fulfill both of these constraints.

There are studies suggesting that physical pain and psychological pain share similar neural mechanisms [citation]. And in meditation, one may notice that psychological discomfort and suffering involves avoiding unpleasant sensations in the same way as physical pain does; the same mechanism has been recruited for more abstract planning.

When the brain predicts that a particular experience would produce an unpleasant sensation, craving resists that prediction and tries to find another way. Similarly, if the brain predicts that something will not produce a pleasant sensation, craving may also resist that aspect of reality.

Now, this process as described has a structural equivalence to binocular rivalry, but as far as I know, binocular rivalry does not involve any particular discomfort. Suffering obviously does.

Being in pain is generally bad: it is usually better to try to avoid ending up in painful states, as well as try to get out of painful states once you are in them. This is also true for other states, such as hunger, that do not necessarily feel painful, but still have a negative emotional tone. Suppose that whenever craving generates a self-fulfilling prediction which resists your direct sensory experience, this generates a signal we might call “unsatisfactoriness”.

The stronger the conflict between the experience and the craving, the stronger the unsatisfactoriness - so that a mild pain that is easy to ignore only causes a little unsatisfactoriness, and an excruciating pain that generates a strong resistance causes immense suffering. The brain is then wired to use this unsatisfactoriness as a training signal, attempting to avoid situations that have previously included high levels of it, and to keep looking for ways out if it currently has a lot of it.

It is also worth noting what it means for you to be paralyzed by two strong, mutually opposing cravings. Consider again the situation where I am torn between admitting the truth to my friend, and staying quiet. We might think that this is a situation where the overall system is uncertain of the correct course of action: some subsystems are trying to force the action of confronting the situation, others are trying to force the action of avoiding it. Both courses of action are predicted to lead to some kind of loss.

In general, it is a bad thing if a system ends up in a situation where it has to choose between two different kinds of losses, and has high internal uncertainty of the right action. A system should avoid such dilemmas, either by avoiding the situations themselves or by finding a way to reconcile the conflicting priorities.

Craving-based and non-craving-based motivation

What I have written so far might be taken to suggest that craving is a requirement for all action and planning. However, the Buddhist claim is that craving is actually just one of at least two different motivational systems in the brain. Given that neuroscience suggests the existence of at least three different motivational systems, this should not seem particularly implausible.

Let’s take another look at the types of processes related to binocular rivalry versus craving.

Craving acts by actively introducing false beliefs into one’s reasoning. If craving could just do this completely uninhibited, rewriting all experience to match one’s desires, nobody would ever do anything: they would just sit still, enjoying a craving-driven hallucination of a world where everything was perfect.

In contrast, in the case of binocular rivalry, no system is feeding the reasoning process any false beliefs: all the constraints emerge directly from the sense data and previous life-experience. To the extent that the system can be said to have a preference over either the “I am seeing a house” or the “I am seeing Isaac Newton” hypothesis, it is just “if seeing a house is the most likely hypothesis, then I prefer to see a house; if seeing Newton is the most likely hypothesis, then I prefer to see Newton”. The computation does not have an intrinsic attachment to any particular outcome, nor will it hallucinate a particular experience if it has no good reason to.

Likewise, it seems like there are modes of doing and being which are similar in the respect that one is focused on process rather than outcome: taking whatever actions are best-suited for the situation at hand, regardless of what their outcome might be. In these situations, little unsatisfactoriness seems to be present.

In an earlier post, I discussed a proposal where an autonomously acting robot has two decision-making systems. The first system just figures out whatever actions would maximize its rewards and tries to take those actions. The second “Blocker” system tries to predict whether or not a human overseer would approve of any given action, and prevents the first system from doing anything that would be disapproved of. We then have two evaluation systems: “what would bring the maximum reward” (running on a lower priority) and “would a human overseer approve of a proposed action” (taking precedence in case of a disagreement).

It seems to me that there is something similar going on with craving. There are processes which are neutrally just trying to figure out the best action; and when those processes hit upon particularly good or bad outcomes, craving is formed in an attempt to force the system into repeating or avoiding those outcomes in the future.

Suppose that you are in a situation where the best possible course of action only has a 10% chance of getting you through alive. If you are in a non-craving-driven state, you may focus on getting at least that 10% chance together, since that’s the best that you can do.

In contrast, the kind of behavior that is typical for craving is realizing that you have a significant chance of dying, deciding that this thought is completely unacceptable, and refusing to go on before you have an approach where the thought of death isn’t so stark.

Both systems have their upsides and downsides. If it is true that a 10% chance of survival really is the best that you can do, then you should clearly just focus on getting the probability even that high. The craving which causes trouble by thrashing around is only going to make things worse. On the other hand, maybe this estimate is flawed and you could achieve a higher probability of survival by doing something else. In that case, the craving absolutely refusing to go on until you have figured out something better might be the right action.

There is also another major difference, in that craving does not really care about outcomes. Rather, it cares about avoiding positive or negative feelings. In the case of avoiding death, craving-oriented systems are primarily reacting to the thought of death… which may make them reject even plans which would reduce the risk of death, if those plans involved needing to think about death too much.

This becomes particularly obvious in the case of things like going to the dentist in order to have an operation you know will be unpleasant. You may find yourself highly averse to going, as you crave the comfort of not needing to suffer from the unpleasantness. At the same time, you also know that the operation will benefit you in the long term: any unpleasantness will just be a passing state of mind, rather than permanent damage. But avoiding unpleasantness - including the very thought of experiencing something unpleasant - is just what craving is about.

In contrast, if you are in a state of equanimity with little craving, you still recognize the thoughts of going to the dentist as having negative valence, but this negative valence does not bother you, because you do not have a craving to avoid it. You can choose whatever option seems best, regardless of what kind of content this ends up producing in your consciousness.

Of course, choosing correctly requires you to actually know what is best. Expert meditators have been known to sometimes ignore extreme physical pain that should have caused them to seek medical aid. And they probably would have sought help, if not for their ability to drop their resistance to pain and experience it with extreme equanimity.

Negative-valence states tend to correlate with states which are bad for the achievement of our goals. That is the reason why we are wired to avoid them. But the correlation is only partial, so if you focus too much on avoiding unpleasantness, you are falling victim to Goodhart’s Law: optimizing a measure so much that you sacrifice the goals that the measure was supposed to track. Equanimity gives you the ability to ignore your consciously experienced suffering, so you don't need to pay additional mental costs for taking actions which further your goals. This can be useful, if you are strategic about actually achieving your goals.

But while Goodharting on a measure is a failure mode, so is ignoring the measure entirely. Unpleasantness does still correlate with things that make it harder to realize your values, and the need to avoid displeasure normally operates as an automatic feedback mechanism. It is possible to have high equanimity and weaken this mechanism, without being smart about it and doing nothing to develop alternative mechanisms. In that case you are just trading Goodhart’s Law for the opposite failure mode.

In the beginning of this post, I mentioned a few other disadvantages that craving has, which I have not yet mentioned explicitly. Let’s take a quick look at those.

Craving narrows your perception, making you only pay attention to things that seem immediately relevant for your craving.

In predictive processing, attention is conceptualized as giving increased weighting to those features of the sensory data that seem most useful for making successful predictions about the task at hand. If you have strong craving to achieve a particular outcome, your mind will focus on those aspects of the sensory data that seem useful for realizing your craving.

Strong craving may cause premature exploitation. If you have a strong craving to achieve a particular goal, you may not want to do anything that looks like moving away from it, even if that would actually help you achieve it better.

Suppose that you have a strong craving to experience a feeling of accomplishment: this means that the craving is strongly projecting a constraint of “I will feel accomplished” into your planning, causing an error signal if you consider any plan which does not fulfill the constraint. If you are thinking about a multistep plan which will take time before you feel accomplished, it will start out by you not feeling accomplished. This contradicts the constraint of “I will feel accomplished”, causing that plan to be rejected in favor of ones that bring you even some accomplishment right away.

Craving and suffering

We might summarize the unsatisfactoriness-related parts of the above as follows:

• Craving tries to get us into pleasant states of consciousness.
• But pleasant states of consciousness are those without craving.
• Thus, there are subsystems which are trying to get us into pleasant states of consciousness by creating constant craving, which is the exact opposite of a pleasant state.

We can somewhat rephrase this as:

• The default state of human psychology involves a degree of almost constant dissatisfaction with one’s state of consciousness.
• This dissatisfaction is created by the craving.
• The dissatisfaction can be ended by eliminating craving.

… which, if correct, might be interpreted to roughly equal the first three of Buddhism’s Four Noble Truths: the fourth is “Buddhism’s Noble Eightfold Path is a way to end craving”.

A more rationalist framing might be that the craving is essentially acting in a way that looks similar to wireheading: pursuing pleasure and happiness even if that sacrifices your ability to impact the world. Reducing the influence of the craving makes your motivations less driven by wireheading-like impulses, and more able to see the world clearly even if it is painful. Thus, reducing craving may be valuable even if one does not care about suffering less.

This gives rise to the question - how exactly does one reduce craving? And what does all of this have to do with the self, again?

We’ll get back to those questions in the next post.

This is the third post of the "a non-mystical explanation of insight meditation and the three characteristics of existence" series. The next post in the series will be posted on Friday, 22nd of May.

Discuss

### AI's shouldn't believe falsehoods about preferences; priors over preferences are not sufficient

Новости LessWrong.com - 15 мая, 2020 - 16:14
Published on May 15, 2020 1:14 PM GMT

I've always emphasised the constructive aspect of figuring out human preferences, and the desired formal properties of preference learning processes.

A common response to these points is something along the line of "have the AI pick a prior over human preferences, and update it".

However, I've come to realise that a prior over human preferences is of little use. The real key is figuring out how to update it, and that contains almost the entirety of the problem.

I've shown that you cannot deduce preferences from observations or facts about the world - at least, without making some assumptions. These assumptions are needed to bridge the gap between observations/facts, and updates to preferences.

For example, imagine you are doing cooperative inverse reinforcement learning[1] and want to deduce the preferences of the human H. CIRL assumes that H knows the true reward function, and is generally rational or noisily rational (along with a few other scenarios).

So, this is the bridging law:

• H knows their true reward function, and is noisily rational.

Given this, the AI has many options available to it, including the "drug the human with heroin" approach. If H is not well-defined in the bridging law, then "do brain surgery on the human" also becomes valid.

And not only are those approaches valid; if the AI wants to maximise the reward function, according to how this is defined, then these are the optimal policies, as they result in the most return, given that bridging law.

Note that the following is not sufficient either:

• H has a noisy impression of their true reward function, and is noisily rational.

Neither of the "noisy" statements are true, so if the AI uses this bridging law, then, for almost any prior, preference learning will come to a bad end.

Joint priors

What we really want is something like:

• H has an imperfect impression of their true reward function, and is biased.

And yes, that bridging law is true. But it's also massively underdefined. We want to know how H's impression is imperfect, how they are biased, and also what counts as H versus some brain-surgeried replacement of them.

Once we know that, we get a joint prior p over R×ΠH, the human reward functions and the human's policy[^huid]. Given that joint prior, then, yes, an AI can start deducing preferences from observations.

[^huid] And the human's identity, which we're implicitly modelling as part of the policy.

But such a "joint prior" is essentially exactly the same object as the assumptions needed to overcome the Occam's razor result.

Other areas

It seems to me that realisability has a similar problem: if the AI has an imperfect model of how they're embedded in the world, then they will "learn" disastrously wrong things.

1. This is not a criticism of CIRL; it does its task very well, but still requires some underlying assumptions. ↩︎

Discuss

### Distinguishing logistic curves: visual

Новости LessWrong.com - 15 мая, 2020 - 13:33
Published on May 15, 2020 10:33 AM GMT

I wrote a post about distinguishing between logistic curves, specifically for finding their turning points.

That post was highly mathematical; but here is a visual "proof" of the "theorem":

• Figuring out the turning point of a logistic curve before hitting the turning point is bloody hard, mate.

"Proof": The following is a plot two curves:

1. The logistic curve 1/(1+e−x) up to its turning point at x=0.
2. The exponential curve 0.51e0.69x, which never has any turning points.

So, if the data was noisy, could you distinguish between the curve that's reached its turning point, and the one that will never have one?

Things get even worse if we stop before the turning point; here's the plot of the logistic curve up to x=−log(3)≈−0.48, with the y=0.25 being half of the value at the turning point. This is plotted against the exponential 0.64e0.85x:

Discuss

### What are Michael Vasser's beliefs?

Новости LessWrong.com - 15 мая, 2020 - 09:15
Published on May 15, 2020 6:15 AM GMT

I've heard Michael Vasser's name mentioned a few times within the community. Why is he so well-known and what are his main ideas? I am particularly interested in the ideas that seem to have made him community famous.

Discuss

### Song for a Red Planet

Новости LessWrong.com - 15 мая, 2020 - 08:30
Published on May 15, 2020 5:30 AM GMT

This is the first original filk song I’ve written, it’s not derived from anything. I’ll be recording myself singing it if I can find a decent microphone because I don’t really think my webcam mic is up to the task. ( I might do a webcam mic version anyway just to get the basic idea, we’ll see). It’s sung as a duet and features call and response, although it can also be sung alone.

Verse 1:
I am just a son of men,  I walk upon the earth
I am just a boy trying to – prove – my – wo – rth
I am just a passenger aboard a ship without a berth
but if there’s one thing that I know it’s that I know i’m leaving earth

Ar – ca – dia – plan – i – ti- a – a – a
That’s where – I’ll lay my he – ad
Where sunsets are blue
And the domes are too – oo – oo
And the skies, are pai- n – t – ed re – ed

Chorus:
Ar – ca – dia – plan – i – ti- a – a – a
That’s where – I want to be
To red rock plains
I’ll sail my shi – i – i – p
Across – the highest se – as

Verse 2:
I am just a girl born to a ship without a har – bor
I am just a passenger please take –  me – far – ther
I am just a messenger and I am here to say,
Although I was born upon the earth on earth I cannot stay

Ar – ca – dia – plan – i – ti- a – a – a
Is where – I’ll make – my – ho – o – me
Through jet black space
That highest pla -a – a – ce
Is where –  I want – to – ro – o – am

Chorus:
Ar – ca – dia – plan – i – ti- a – a – a
That’s where – I want to be
To red rock plains
I’ll sail my shi – i – i – p
Across – the highest se – as

Bridge:
I don’t wanna go (don’t wanna go)
To hot bangkok (to hot bangkok)
I don’t want to go to the ei – f – fel – to – wer’s – top
I don’t wanna swim (don’t wanna swim)
In the gulfstream waters (in the gulfstream waters)
I want a land – not – of – our – fa – ther – s
I don’t wanna go (don’t wanna go)
To New York City (to New York City)
I want to be in the mariner valley.

Ar – ca – dia – plan – i – ti- a – a – a
A plea, – Please take me the – e -re
We’ll lay a course – We’ll board our shi – i – i – p
Soaring on a wing – and – a – pra – a – yer

Chorus:
Ar – ca – dia (Ar – ca – dia ) Plan – i – ti- a – a – a (Plan – i – ti- a – a – a)
That’s where (that’s where) – I want to be
To red rock plains (to red rock plains)
I’ll sail my shi (I’ll sail my shi – i – i – p)
Across – the highest se – as

Ar – ca – dia (Ar – ca – dia ) Plan – i – ti- a – a – a (Plan – i – ti- a – a – a)
That’s where (that’s where) – I want to be
To red rock plains (to red rock plains)
I’ll sail my shi (I’ll sail my shi – i – i – p)
Across – the highest se – as

Discuss

### What was your reasoning for deciding whether to raise children?

Новости LessWrong.com - 15 мая, 2020 - 06:53
Published on May 15, 2020 3:53 AM GMT

Discuss

### "It's Okay", Instructions, Focusing, Experiencing and Frames

Новости LessWrong.com - 15 мая, 2020 - 03:49
Published on May 15, 2020 12:49 AM GMT

Epistemic Status: Interpreting someone else's work will always be speculative

Valentine has described his experience of a Kensh&#x14D; (or moment of understanding) from which he took away a lesson that could roughly be summarised by "It's okay". I've read that post and the follow up comments far many times as there were elements of it that I struggled to understand. Kaj Sotala has already written a quite good explication, but I still feel there is more to explore there. If you have time, I strongly recommend reading Valentine's post first, even though you might be able to understand it quicker by just reading this post. I think there's value in grappling with confusion as you'll learn more about how to deal with confusion in the future.

Valentine describes his insight more fully as:

“I’m okay. You’re okay. Everything is fundamentally okay. Whatever happens, it will be fine and good. Even our worry and pain is okay. There is something deeply sad about someone dying… and their death is okay. Obliteration of humanity would be tragic, but the universe will go on, and it’s okay.”

He then highlights two possible misunderstandings:

1) Some thought I was saying that nothing matters and that all outcomes are equally good.
2) Some thought I was claiming that you’ll feel good no matter what if you’re enlightened.

Both these misunderstandings were propositional claims, but Valentine clarifies that he meant this as an instruction for practise. He instructs:

The world is real in your immediate experience before you think about it. Set aside your interpretations and just look.

However, even having said all these words, he still worries that "people's thinking systems can grab statements like this and try to interpret them". He states that we have misunderstood if we simply understand him to be saying that the map is not the territory; ie. that our mental model of the world doesn't necessarily directly respond to reality. He also clarifies that it isn't the same as Seeing with Fresh Eyes, which is thinking through an idea as though you are hearing it for the first time, without any assumptions that you've attached to it along the way.

Valentine repeatedly worries that people will misunderstand him by taking what he is saying and squeezing it into the closest conceptual bucket that they have (this corresponds to his refrain: "you are still looking at your phone"). Since they will now believe they understand his point, it'll be much harder to explain the concept to them.

Instructions vs. Epistemic Claims

There's quite a few challenges with understanding Valentine's post. Valentine a) doesn't clarify that he is talking about an instruction rather than a propositional belief until late in the piece b) doesn't provide an example of an instruction c) uses an analogy where someone provides an instruction and it is misunderstood as another instruction. To avoid falling into these traps, we'll begin by talking about instructions, making sure that we provide an example and our example will also exhibit a better analogy.

We'll start with an example that is so simple that it is almost boring, telling someone to "be confident". If we really tried we could misinterpret that as an epistemic statement telling someone that their level of confidence is too low given their level of ability. But of course, we aren't telling someone what to believe, but what to do. Maybe their level of confidence is properly calibrated with their level of ability, so that increasing their confidence would make them miscalibrated, but it might be good advice to tell them to be confident anyway. This might help them achieve the right mental state for optimal performance.

Another way to misinterpret this would be to take it as a claim that they'd perform better if they were confident. This is implied, but "be confident" is a command, not a mere statement of fact. It's one thing to think that confidence would make you perform better and quite another to actually have this confidence. It would also be a mistake to take it as a command to just think a proposition like, "It would be correct for me to be more confident" or "I would perform better if I have more confidence" really, really hard. Merely thinking these thoughts won't necessarily bring along any additional confidence. This seems to be the same kind of mistake Valentine identifies when he explains that he doesn't mean things are neither good nor bad.

Focusing vs Forcing

Hopefully, it should be clear by now what it means for "Be Confident" or "It's Okay" to be instructions, even if they are seen as too vague to be useful. The former is discussed more in the appendix, but for the later, we are fortunate in that Valentine provides more detail:

The world is real in your immediate experience before you think about it. Set aside your interpretations and just look.

The clear implication is that if we just look, we'll see that "It's okay", but not in a propositional sense. We'll get to that in a minute. But first notice, that he hasn't said that your interpretations of the world are wrong and that they should be discarded, just that we should put them to the side for the moment and pay attention to what the world looks like without them. I suspect this is a key part why his phone analogy involves trying to explain how to "Look Up".

It's very common for optimists to suggest that we focus on the positives. Some people may interpret this to mean intentionally biasing ourselves to believe that good thing happen and althogh this may occur as a result, I wouldn't say that's what's being suggested at all. Instead, we're just been told that the more time we think about the positive aspects of a situation, the more happiness we'll experience and the more time we focus on the negative aspects, the more sadness we'll experience.

Of course, there are complexities here. Merely repeating, "Everything is Awesome, Everything is Cool" again and again won't make you any happier. You need to actually focus on things that make you feel good and avoid focusing on things that make you feel bad and thoughts that are nominally good may make you feel bad if you know they aren't true. The strategy isn't to try force yourself to think positive thoughts, but rather to just pay attention to the positives that come naturally to you.

When Valentine talks about setting aside interpretations, I assume he is referring to the medative practise of non-judgement. It's a shame that he didn't mention it explicitly, as mentioning judgements would have turned our attention more toward interpretations of value. The technique of setting aside interpretations or judgements is also a shift in focus, but less likely to bias us than positive thinking. Instead of focusing on interpretations, we focus on the raw sensations and often realise that suffering which we thought was a result of the sensations was a result of our interpretations. Again, we aren't forcing ourselves to think that everything is equally good, but rather shifting our thoughts away from the judgements.

I should clarify that I don't mean that judgements are necessarily bad. On the contrary, they often provide useful information, but once we are aware of this information, spending more time focusing on them often just hurts us.

Experiencing vs. Spotting

Let's examine this instruction further:

The world is real in your immediate experience before you think about it. Set aside your interpretations and just look.

Part of what is confusing is that we haven't been told what to look for, which suggests that it must be something hard to describe.

Suppose someone tells you to look at a tree. You ask why, but they just tell you to look. Maybe eventually after a while you see a brown snake that is camoflaged due to its color. We will call this spotting.

Suppose instead you look at the tree, as as far as you can tell, it looks normal. You can't see any snakes, or carved messages, or hidden jaguars or anything else unusual. Eventually you give up on finding it and just look at the tree as a whole. You notice that the tree is creepy, but you didn't notice it because you thought you were looking for a thing, not a feeling or experience. If you'd seen a picture of a tree in an art gallery instead, you would have noticed it, since it is generally understood that paintings are in galleries to trigger emotions.

Bringing it back, it is easy to assume that we need to spot some thing or combination in what is immediately accessible that we haven't notcied before, when what we actually need to do is undergo the experience of focusing on what is immediately present and compare it to the experience of focusing on our judgements. If my interpretations is correct, then the reason why Valentine can't describe what to Look for is that what is important isn't what we are Looking at, but the experience of Looking itself.

Frames vs. Frameworks

One reason why Valentine might have found this hard to explain was that he was thinking of explaining it terms of Fake Frameworks (hacky epistemic models) rather than Frames (a much broader concept covering differences in thinking, seeing, feeling, intuiting or communicating.

If he had said it was a Fake Framework, people might have assumed that he was making an epistemic proposition. On the other hand, "It's Okay" and Being Confident seems to be way of existing in the world. You go for a job interview and you know you aren't as qualified as you'd like for the position, but you don't get caught worrying about it and just focus on doing the best you can. Or your girlfriend breaks up with you, but at the moment you just need to make dinner, so you focus on that. It's still the case that you probably won't have a job or that you'll experience significantly less happiness over the coming months, it's just that you aren't letting your awareness of this led you down certain thought paths that aren't productive. So frameworks are about how you model the world, while frames also include all aspects of your response to the world, including how you respond internally.

The confusing thing about the more emotional frames is that thoughts can often play a role, but without being the most crucial part. For example, why think "It's okay" if it's not meant to be taken as a literal statement of truth? Well, even if the proposition itself isn't the active component and can still be the trigger in adopting a partcular state where you are focused on the present, instead of something that is causing you suffering.

When describing "It's okay" Valentine claims "it’s already always here". This is confusing because it makes it sound like you've already been taught what to do when you haven't. So what does he mean then?

One additional detail he provides is that the already know is "that small quiet part of us that nudges us to notice that we're wrong while in a fight with a loved one". So rather than meaning that we've already been taught it, he seems to be suggesting that we already have the appropriate instinct and that we just have to notice it and follow what it tells us to do. I haven't experience it so I don't know what he is talking about.

Appendix 2: How useful are commands like Be Confident?

An obvious retort to an instruction like "Be confident" is that it is far too vague to be of any use. It tells you the end goal, but not how to get there. These critiques are valid, but that doesn't mean that the statement is necessarily useless by itself. We'll distinguish four different ways of filling this out. First, sometimes a person will already know techniques for shifting their mental state and only need a reminder to make use of them. Second, sometimes merely knowing that you want to inhabit a particular mental state opens up the opportunity to shift more into it. Third, sometimes a person simply needs social permission in order to feel confident being confident. Fourth, sometimes simply thinking "Be confident" or "I am confident" is enough, although it can also backfire and as we've already noted, he isn't thinking about this technique. Fifth, sometimes it's useful to know what the goal is, even if you don't have techniques to get there yet. Sixth, someone may just be providing you with a high level description of what you should do and be able to provide you with further instructions.

Discuss

### Covid 5/21: Limbo Under

Новости LessWrong.com - 15 мая, 2020 - 01:30
Published on May 14, 2020 10:30 PM GMT

Previous weekly reports: Covid-19 5/7: Fighting LimboCovid-19 4/30: Stuck in Limbo

Slowly, a nation partially reopens. Is it too much, too soon? It’s too early to know for sure, because of lags, but so far we’ve only seen extraordinary good news. If you don’t think what we saw this past week was good news, either we disagree about how to read the data quite a lot, or you had what I consider highly realistic expectations.

Remarkably little has happened in the last week. There are weeks when decades happen, and previous weeks have felt like that. This one didn’t. This felt like waiting for things to happen and nothing happening except good numbers. Yes, we got some other news, but did any of it matter or surprise much? I would say that it did not.

Last time I did a bunch of analysis and a bit of editorializing alongside the numbers. This time I’ll keep it brief. This is mostly to get the charts out. I’ll get the other stuff out there in distinct posts if it’s worth saying.

The Data

Deaths:

WEST MIDWEST SOUTH NE ex-NY NY Mar 19-Mar 25 116 67 111 84 203 Mar 26-Apr 1 347 477 502 454 1340 Apr 2-8 639 1335 1150 1783 3939 Apr 9-15 895 2106 1472 3261 5345 Apr 16-22 1008 2369 1730 5183 3994 Apr 23-29 1135 2500 1684 4285 2810 Apr 30-May 6 991 2413 1737 5349 2007 May 7-May 13 1044 2344 1679 4014 ~1500*

*- New York includes ~700 reclassified deaths that did not take place this past week, but were instead from past weeks and added to the death count on 5/7. I’ve subtracted them out for purposes of this chart.

Positive Tests:

WEST MIDWEST SOUTH NE ex-NY NY Mar 19-Mar 25 5744 6293 7933 8354 28429 Mar 26-Apr 1 15684 20337 24224 34391 52901 Apr 2-8 19455 31148 39618 56772 65604 Apr 9-15 16291 29267 35570 61921 64463 Apr 16-22 20065 34130 33932 64669 43437 Apr 23-29 21873 42343 33773 62189 42475 Apr 30-May 6 23424 49205 37880 51693 24287 May 7-May 13 22615 43264 37591 40209 16683

Overall test counts:

USA tests Positive % NY tests Positive % Mar 19-Mar 25 347577 16.2% 88882 32.0% Mar 26-Apr 1 728474 20.2% 117401 45.1% Apr 2-8 1,067,220 19.8% 144273 45.5% Apr 9-15 1,039,790 20.1% 160859 40.1% Apr 16-22 1,253,535 15.7% 143970 30.2% Apr 23-29 1,480,101 13.7% 202499 21.0% Apr 30-May 6 1,733,601 10.6% 183446 13.2% May 7-May 13 2,215,060 7.4% 202980 8.2%

The chart tell the story. We expanded testing dramatically and positive counts dropped in every region. New York continues to improve at a much faster clip than elsewhere, and some localities are seeing things get worse, but the overall trend is unmistakable. Positive counts fell in all regions while testing once again expanded substantially.

Deaths are not falling much yet, but they are a lagging indicator. Things are improving.

I don’t feel that much more confident in my priors than I did previously, but then I haven’t felt the need to update them either, in much of any direction.

New York will start its phase 1 reopening this coming week in four of its ten regions. It will likely expand that to seven to ten of them within two weeks. Restaurants come back in phase 3, which is four weeks in. It feels like normal is well on its way. I’m starting to feel much less paranoid, as my true estimate of infections per day in the state drops to 12,478 today, down from 20,200 a week ago. And also with my increased confidence in my infection modeling.

We also get baseball around July 4, assuming the players and owners can agree on how much the players get paid. That’s tricky in the best of times, but ultimately is probably settled one way or another. All theater from here, we hope.

I’m working on some other posts that will hopefully cover other angles.

Discuss

### Adding a Housemate Under Covid

Новости LessWrong.com - 15 мая, 2020 - 00:30
Published on May 14, 2020 9:30 PM GMT

In mid-March when we decided to isolate, one of our housemates was interested in having her partner move in for the duration. The house talked about it and was ok with that, but he ended up deciding to stay where he was. Two months later, with this lasting longer than we were expecting at the time, he decided he did want to switch houses. Here's how we decided to handle it:

• Before moving in, he made an appointment for a test. Starting in late April it's been possible to get a test here even if you don't have symptoms.

• We divided the unit in two with plastic sheeting. They would have two rooms, one bathroom, and the back exit.

• They had a microwave, cooler, and shelf stable food.

• We would bring them a hot meal at dinner time, and other things as needed. Things that came back out either sat in the basement for three days or got washed with soap and water.

• If the test came back negative, and no one in his former house had started having symptoms, we would end the isolation. If not, we'd deal with it then.

He moved in on Saturday, had his testing appointment on Monday, and the test came back negative this morning. We had a celebratory removal of the sheeting:

Since there can be false negatives the risk isn't zero, but it's a level we feel comfortable with, and that we think is responsible.

It will be nice having another housemate!

Discuss

Новости LessWrong.com - 14 мая, 2020 - 22:40
Published on May 14, 2020 7:37 PM GMT

After reading an introductory post on game theory, specifically regarding Schelling Points, I noticed a paradox and I wonder if anyone here can shed some light on the subject.

The scenario I will use to illustrate this is the inverse of a frequently used example to describe popular Schelling points.

The Scenario

Suppose, in a dystopian future, you are on a game show with a partner. A list of numbers is presented to you:

[2, 5, 9, 25, 69, 73, 82, 96, 100, 126, 150]

You and your partner have to pick one number from the list each, without communicating with each other. You both know that if you pick the same number, a terrible fate looms. If you pick different numbers, you are both set free and live long and happy lives.

Assume that you are both human and computer unaided and therefore cannot choose truly random positions on the list.

Naturally, you would want to avoid Schelling points (special numbers or numbers in special positions in the list) to minimise the chance of picking matching numbers. In this case, the Schelling points are numbers which you would think your partner would be more likely to pick, for whatever reason. However, if you both rule out Schelling points, you make the list of numbers to choose from smaller, thus increasing the chance of you both picking the same number significantly. Therefore, if you both actively pick numbers which you think your partner is least likely to pick, assuming you both think rationally, you inadvertently increase the chance of picking the same number. Thus the notion of the Schelling point has become the numbers that are especially insignificant, and the cycle continues. This is the paradox.

A real life example:

Which bar do i choose to drink at on a Friday night if I want to avoid my ex (assuming she's actively avoiding me too)?

The point of this post:

Using game theory, what logical strategy would you employ in two-player avoidance games similar to the one above?

I'm new to the site and if any of this is convoluted or has been covered before, I apologise in advance.

Discuss

### Utility need not be bounded

Новости LessWrong.com - 14 мая, 2020 - 21:10
Published on May 14, 2020 6:10 PM GMT

A few months ago I disagreed with Sniffnoy about whether the theorem regarding Savage's axioms for probability and utility, that utility must be bounded, is a good reason for believing that utility must be bounded. Sniffnoy said yes, because it follows from the axioms, I said no, therefore there is a flaw in the axioms. (The fact that the theorem does follow from the axioms is not an issue.) I concluded that conversation by saying I'd have to think about it.

This I have done. I have followed Jaynes' dictum that when infinities lead to problems, one must examine the limiting process by which those infinities were arrived at, which almost invariably dissolves the problem. The flaw in Savage's system is easy to find, easy to describe, and easy to rectify. I have devised a new set of axioms such that:

• Probability and utility are constructed from the preference relation by the same method as Savage.
• Every model of Savage's axioms is a model of these axioms and constructs the same probability measure and utility function.
• The new axioms also have models with acts and outcomes of unbounded utility.
• Acts of infinite utility (such as the St. Petersburg game) are admitted as second-class citizens, in much the same way that measurable functions with infinite integral are in measure theory.
• More pathological infinite games (such as St. Petersburg with every other payout in the series reversed in sign) are excluded from the start, but without having to exclude them by any criterion involving utility. (Utility is constructed from the axioms, so cannot be mentioned within them.) Like measurable functions that have no integral, well, that's just what they are. There's no point in demanding that they all should.

This removes all force from the argument that because Savage's axioms imply bounded utility, utility must be bounded. (There are other axiom systems that have that consequence, but I believe that my construction would apply equally to them all.) If one prefers Savage's axioms because they have that consequence, one must have some other reason for believing that utility must be bounded, or the argument is circular.

There are a few details of proofs still to be filled in, but I don't think there will be any problems there. Any expert on measure theory could probably dispose of them with a theorem off the shelf. Because of this I don't want to stick it on arXiv yet, but I would welcome interested readers. Anyone interested, ask me for a copy and give me a way of sending you a PDF.

Despite the title of this post, the only argument for bounded utility I am addressing here is the argument that it follows from various axiom systems. For other, more informal reasons people have for believing in bounded utility, Eliezer (an unbounded fun theorist) has had plenty to say about that in the past, so I'll just refer people to the Fun Theory Sequence. Because they are informal, you can chew over them forever, which I find an un-fun activity.

Discuss

### What newsletters are you subscribed to, and why?

Новости LessWrong.com - 14 мая, 2020 - 17:47
Published on May 14, 2020 2:47 PM GMT

I've found myself increasingly feeling the need for curated newsletters on the topics I care about from interesting people in the field, but I have little clue where to look for them.

I currently follow TLDR newsletter(in Telegram because that’s more convenient than in email), because I find the news it delivers worth my while, and I haven’t found a better alternative. I also use rss feeds to see posts with +500 scores from Hackernews and the top posts of lobste.rs.

Discuss

### Estimating logistic curves

Новости LessWrong.com - 14 мая, 2020 - 15:41
Published on May 14, 2020 12:41 PM GMT

This post will attempt to formalise the intuition that "it's hard to figure out the turning point of a logistic curve, at least until after that turning point".

Logistic curves

The logistic curves look like this:

Logistic curves can be specified by three parameters, c, 0">l>0, and 0">k>0. Their equation is then:

Fc,l,k(x)=lekxek(x−c)+1.

Note that this l is different from that in this article. The turning point of this curve is at x=c (where it takes the value of lekc/2) while its supremum is lekc; it tends to this value as x→∞. Take the limit as c→∞ as being the exponentials:

F∞,l,k(x)=lekx.

Figuring out the right curve

We'll imagine a simple Bayesian setup. An analyst of logistic curves is seeing data from one distribution, and has two hypotheses about it: FC,L,K, for values C, L, and K, and Fc,l,k with values c, l, and k. We'll designate FC,L,K by F and Fc,l,k by f.

Now, the true distribution is F, but the analyst doesn't know that. The question we're asking is thus:

• Starting from an equal prior on F and f, how much of what kind of observation will the analyst need to establish that F is the true underlying distribution?
Noisy Sampling

If the analyst can sample noiselessly from the curve, then three samples should generally suffice to fully establish F, and one sample should generally suffice to distinguish F from f. So we'll consider the (realistic) situation where there is noise in the samples.

So assume the analyst samples n points, at →x=(x1,x2,…,xn). In return, it gets n values, →y=(y1,y2,…,yn); these are sampled independently from N(F(xi),σ2i). This is a normal distribution with mean F(xi) and standard deviation σi.

The analyst is assumed to know the vector →σ=(σ1,…σn), and indeed everything about this setup, with one exception: whether the means of these normal distributions are F(xi) or f(xi).

Let Pa be the analyst's probability distribution. Their prior gives equal weight to both hypotheses: Pa(F)=Pa(f)=1/2. Let O→y→x be the analyst observing →y after sampling from →x; their posterior is then Pa(f∣O→y→x).

Note that, from our perspective, Pa(F∣→x) is a random variable whose distribution we know. Say that:

• →x establishes the difference between F and f if the expectation of Pa(F∣→x) is less than 1/16.

We could choose other criteria, and this a relatively loose one. It only assumes three bits of information in favour of F over f. Note that since Pa≥0, we can get probability bounds on Pa as well, from this result; for instance:

• If E[Pa(f∣→x)]≤q/4, then with probability at least 3/4, Pa(f∣→x)≤q.

So, for instance, our criteria above ensures that with probability at least 3/4, Pa(f∣→x)≤1/4. Conversely, since Pa≤1, probability bounds on Pa translate into expectation bounds, making the two approaches loosely equivalent. We'll use expectation bounds, as they are more natural for this random variable.

Bounding results

Our first result, proven in later sections, is a lower bound on the expectation of Pa(f∣→x):

E[Pa(f∣→x)]≥12∏ni=1[1−erf(δiσi2√2)].(1)

Here erf is the error function and δi is the absolute difference between F(xi) and f(xi). We can then get the slightly looser but more easily computable bound:

E[Pa(f∣→x)]≥12[1−∑ni=1δiσi√2π].(2)

How to sample Sampling very large positive or negative values

Note that:

0≤Fc,l,k=lekxek(x−c)+1≤lekx.

Hence we can bound the δi via:

δi=|F(xi)−f(xi)|≤max(LeKxi,lekxi).

Let m(xi)=max(LeKxi,lekxi); note this is an increasing function, exponential for very negative xi.

Assume we sample n′ different xi values below a very negative X; then if σ− is the minimum of all the σi for xi≤X, the contribution of these n′ points to the expectation bound is at most 0 and at least:

−n′1σ−√2πm(X).

This gives our result for very negative values:

• If noise is irreducible below σ−, then sampling below a very negative X will have very little impact on the analyst's posterior. To get a better result, increasing the X (exponential effect) is generally more powerful than decreasing σ− (inverse linear effect), and much more powerful than getting more samples (linear effect).

The behaviour for large positive xi is also clear: unless lekc=LeKC, f and F must have different asymptotes. So as long as there is an upper bound σ+ on the noise, sampling the curve at large values will cause the expectation of Pa(f∣→x) to converge to 0. For large xi, this is essentially trying to distinguish N(lekc,σ+) from N(LeKC,σ+), so each extra sample applies a multiplicative factor to the expected value of Pa(f∣→x). So, for large samples, the probability of the wrong function converges geometrically to zero in the number of samples.

Finding (any) turning point

So, distinguishing F and f for very low samples is very hard, but distinguishing them for very high samples is generally not very useful. But enough about asymptotic behaviour. The question is: what happens in between, closer to the turning points C and c of F and f?

We can make some scaling and translation choices to simplify F, setting c=0 and l=k=1. So the turning point is at 0 (y value 1/2) and the supremum is 1:

F(x)=exex+1=11+e−x.

Assume now that the noise σi is a constant σ. We want f to have a different turning point, so that can see how easy it is to identify this turning point. Let's choose the worst possible scenario: f is an exponential function with no turning point:

f(x)=lekx.

So, how can the analyst sample so that they have the greatest possible chance of distinguishing between a true function with a turning point at 0, and a false function with no turning point at all?

We have two free variables: the k and l of f, and we typically want to see how well we can do when sampling below a given X. For constant σ, the elements of the bound are given by:

δiσ√2π=|F(xi)−f(xi)|σ√2π=∣∣ ∣∣le(k+1)x+lekx−ex(ex+1)√2π∣∣ ∣∣1σ.

Define d(l,k)(x) as this function, without the σ term. We'll now consider X=0, ie we are sampling at any point before the turning point. Then some experimentation allows us to minimize d(l,k)(x) for negative values, by setting l=0.51 and k=0.69; given these values, d(l,k)(x) is bounded above by 0.007:

Consequently we can use equation (2) to get a bound:

E[Pa(f∣→x)]≥12[1−nσ0.007]

To establish the difference between F and f, we need this below 1/16. Consequently, we need nσ0.007≥78, or

nσ≥125.

So if the noise is 1/200, ie 1% of the value at the turning point, a single data point might suffice. But if the noise is 10% of the value at the turning point, then at least seven samples are needed - and this only works if the values are sampled independently close to the peak of the d(l,k) function. If the values are not independent - as values sampled close to each other tend not to be - then more must be sampled, and the same goes if the values are sampled away from the peaks.

The other issue is that, here, we've first optimised l and k for minimal peak of d(l,k), then assumed the best xi were sampled. We need to consider the opposite situations, too: given the sampled xi, optimise l and k. So, even if n samples are enough to distinguish F from this specific f, there are other exponential functions F∞,l,k that would be harder to distinguish from F.

Anyway, that's all the way to the turning point; what about if X is chosen so that the value F(X) is 1/3 (two thirds of the value at the turning point) or 1/4 (a half of the value at the turning point)? To get these, we need X=−log(2) and X=−log(3), respectively. We'll also look at past the turning point, X=log(2) and log(3).

Similarly optimising l and k gives, for all five values:

• For X=−log(3), nσ0.0015≥78 or nσ≥583.
• For X=−log(2), nσ0.0027≥78 or nσ≥324.
• For X=0, nσ0.007≥78 or nσ≥125.
• For X=log(2), nσ0.015≥78 or nσ≥58.
• For X=log(3), nσ0.021≥78 or nσ≥41.

Using equation (1) tends to stronger bonds, especially for small σ.

Proof

This section will prove the bounds in equation (1) and (2).

By Bayes rule:

Pa(f∣O→y→x)=Pa(O→y→x∣f)Pa(f)Pa(O→y→x)=Pa(O→y→x∣f)Pa(f)Pa(O→y→x∣f)Pa(f)+Pa(O→y→x∣F)Pa(F)=Pa(O→y→x∣f)Pa(O→y→x∣f)+Pa(O→y→x∣F),

since the prior probabilities are equal. Since the analyst knows the true variances, Pa(O→y→x∣f)=P(O→y→x∣f) and similarly for F: we can replace the analyst's probabilities with the true probabilities. So, contracting P(O→y→x∣F) as p1(→y) and P(O→y→x∣f) as p2(→y), we get:

Pa(f∣O→y→x)=p2(→y)p2(→y)+p1(→y))=11+p1(→y)p2(→y)−1.

To get the true expectation of this Pa, we need to integrate over the possible values of →y, weighted by the true probability P(O→y→x∣ F)=p1(→y) of this happening:

E[Pa(f∣→x)]=∫11+p1(→y)p2(→y)−1p1(→y)d→y=∫1p1(→y)−1+p2(→y)−1d→y.

Note that p1(→y) and p2(→y)−1 are both (strictly) positive, and that 1/(p1(→y)−1+p2(→y)−1) is half the harmonic mean of the two.

The harmonic mean of any number of positive elements is bounded below by the minimum value of its arguments. Hence: E[Pa(f∣→x)]≥12∫min(p1(→y),p2(→y))d→y.

Now, since the noise is independent, pj(→y)=∏ni=1pj(yi) where p1(yi)=P(yi∣xi,F) and p2(yi)=P(yi∣xi,f). For positive elements, the minimum of two products is greater than or equal to the product of minimums, so

E[Pa(f∣→x)]≥12∫∞−∞∏ni=1min(p1(yi),p2(yi))d→y≥12∏ni=1∫∞−∞min(p1(yi),p2(yi))dyi.

The expressions min(p1(yi),p2(yi)) can be expressed analytically. If φ is the probability density function of N(0,1), the normal distribution with mean 0 and variance 1, then

p1(yi)=1σiφ(F(xi)−yiσi),p2(yi)=1σiφ(f(xi)−yiσi).

So the two curves are normal curves with the same variance and means F(xi) and f(xi). Assume, without loss of generality, that F(xi)≤f(yi). Then the two functions will be equal at the midpoint μi=(F(xi)+f(xi))/2, and for yi≤μi, p1(yi) is higher, while for yi≥μi, p2(yi) is higher.

Thus min(p1(yi),p2(yi))=⎧⎪⎨⎪⎩1σiφ(f(xi)−yiσi), yi≤μi,1σiφ(F(xi)−yiσi), yi≥μi.

If δi=|F(xi)−f(xi)| is the distance between the two peaks, this becomes: min(p1(yi),p2(yi))=⎧⎪⎨⎪⎩1σiφ(μi+δi/2−yiσi), yi≤μi,1σiφ(μi−δi/2−yiσi), yi≥μi.

Since the integral of φ is 1/2[1+erf(y/√2)], for erf the error function, we can bound the expected probability by:

E[Pa(f∣→x)]≥12∏ni=1∫∞−∞min(p1(yi),p2(yi))dyi≥12∏ni=1(∫μi−∞1σiφ(μi+δi/2−yiσi)dyi+∫∞μi1σiφ(μi−δi/2−yiσi)dyi)≥12∏ni=1(∫−δi/2−∞1σiφ(−yiσi)dyi+∫∞δi/21σiφ(−yiσi)dyi)≥12∏ni=1(12[1+erf(−δi/2σi√2)]+12[1+erf(−δi/2σi√2)])≥12∏ni=1[1−erf(δiσi2√2)].

For positive values, the error function is concave, and it has derivative 2/√π at the origin, so

erf(δiσi2√2)≤2√πδiσi2√2=δiσi√2π.

Consequently

E[Pa(f∣→x)]≥12∏ni=1[1−δiσi√2π].

Using the fact that for x,y positive, (1−x)(1−y)≥1−(x+y), we get a final bound:

E[Pa(f∣→x)]≥12[1−∑ni=1δiσi√2π]

Discuss

### Small Data

Новости LessWrong.com - 14 мая, 2020 - 07:29
Published on May 14, 2020 4:29 AM GMT

Probabilistic reasoning starts with priors and then updates them based off of evidence. Artificial neural networks take this to the extreme. You start with deliberately weak priors, then update them with a tremendous quantity of data. I call this "big data".

In this article, I use "big data" to mean the opposite of "small data". By this, "big data" refers to situations with so much training data you can get away with weak priors. Autonomous cars are an example of big data. Financial derivatives trading is an example of small data.

The most powerful recent advances in machine learning, such as neural networks, all use big data. Machine learning is good at fields where data plentiful, such as in identifying photos of cats, or where data can be cheaply manufactured, such as in playing videogames. "Plentiful data" is a relative term. Specifically, it's a measurement of the quantity of training data relative to the size (complexity) of the search space.

Do you see the problem?

Physical reality is an upper bound on data collection. Even if "data" is just a number stored a CPU's register there is a hard physical limit to how much we can process. In particular, our data will never scale faster than O(x4) where x is the maximum diameter of our computer in its greatest spacetime dimension. O(x4) is polynomial time.

Machine learning search spaces are often exponential or hyperexponential. If your search space is exponential and you collect data polynomially then your data is sparse. When you have sparse data, you must compensate with strong priors. Big data uses weak priors. Therefore big data approaches to machine learning cannot, in general, handle small data.

Statistical Bias

Past performance is no guarantee of future results.

Suppose you want to estimate the mean variance σ2 of a Gaussian distribution. You could sample n points and then compute the mean variance of them.

σ=√∑ni=1(xi−¯x)2n

If you did you'd be wrong. In particular, you'd underestimate the mean variance by a factor of nn−1. The equation for standard deviation s corrects for this and uses n−1 in the denominator.

s=√∑ni=1(xi−¯x)2n−1

An estimate of the variance of a Gaussian distribution based solely on historical data, without adjusting for statistical bias bias will underestimate the mean variance.

σ2=n−1ns

Underestimating mean variance by a factor of nn−1 is not a big deal because a factor of nn−1 vanishes as n approaches infinity. Other learning environments are not so kind.

Divergent Series

Big data uses weak priors. Correcting for bias is a prior. Big data approaches to machine learning therefore have no built-in method of correcting for bias[1]. Big data thus assumes that historical data is representative of future data.

To state this more precisely, suppose that we are dealing with a variable xt where t∈Z+=[1,2,…,∞). In order to predict x=limt→∞xt from past performance ∑ni=1xtn, it must be true that such a limit limt→∞xt exists.

Sometimes no such limit exists. Suppose xt equals 1 for all positive integers whose most significant digit (in decimal representation) is odd and 0 for all positive integers whose most significant digit (in decimal representation) is even.

xt={1if MSD(xt)∉2Z+0if MSD(xt)∈2Z+

Suppose we want to predict the probability that an integer's first significant digit is odd.

The average limn→∞∑ni=1xtn never converges. The average oscillates from ½ up to just over ¾ and back. You cannot solve this problem by minimizing your error over historical data. Insofar as big data minimizes an algorithm's error over historical results, domains like this will be forever out-of-bounds to it.

Big data compensates for weak priors by minimizing an algorithm's error over historical results. Insofar as this is true, big data cannot reason about small data.

Small Data

Yet, human beings can predict once-per-century events. Few of us can do it, but it can be done. How?

Transfer learning. Human beings use a problem's context to influence our priors.

So can we just feed all of the Internet into a big data system to create a general-purpose machine learning algorithm? No. Because when you feed in arbitrary data it's not just the data the increases in dimensionality. Your search space of relationships between input data increases even faster. Whenever a human being decides what data to feed into an artificial neural network, we are implicitly passing on our own priors about what constitutes relevant context. This division of labor between human and machine has enabled recent developments in machine learning like self-driving cars.

To remove the human from the equation, we need a system that can accept arbitrary input data without human curation for relevance. The problem is that feeding "everything" into a machine is close to feeding "nothing" into a machine, like how a fully connected graph contains exactly as much information as a fully disconnected graph.

Similar, but not equal. Cosider Einstein. He saw beauty in the universe and then created the most beautiful theory that fit a particular set of data.

Beauty

Consider the sequence {1,2,3,…}. What comes next?

• It could be {1,2,3,1,2,3,1,2,3,…}
• It could be {1,2,3,4,5,6,7,8,9,…}
• It could be {1,2,3,5,8,13,21,34,55,…}
• It could be {1,2,3,4,I,declare,a,thumb,war,…}

You could say the answer[2] depends on one's priors. That wouldn't be wrong per se. But the word "priors" gets fuzzy around the corners when we're talking about transfer learning. It would be more precise to say this depends on your sense of "beauty".

The "right" answer is whichever one has minimal Kolmogorov complexity i.e. whichever sequence is described by the shortest computer program. But for sparse data, Kolmogorov complexity depends more on your choice of programming language than the actual data. It depends on the sense of beauty of whoever designed the your development environment.

The most important thing in a programming language is what libraries you have access to. If the Fibonacci sequence is a standard library function and the identity operator is not then the Fibonacci sequence has lower Kolmogorov complexity than the identity operator y=x∀x,y∈Z+.

The library doesn't even have to be standard. Any scrap of code lying around will do. In this way, Kolmogorov complexity, as evaluated in your local environment, is a subjective definition of beauty.

This is a flexible definition of "beauty", as opposed to big data where "beauty" is hard-coded as the minimization of an error function over historical data.

Programming languages like Lisp let you program the language itself. System-defined macros are stored in the same hash table as user-defined macros. A small data system needs the same capability.

No algorithm without the freedom to self-alter its own error function can operate unsupervised on small data.

―Lsusr's Second Law of Artificial Intelligence

To transcend big data, a computer program must be able to alter its own definition of beauty.

1. Cross-validation corrects for overfitting. Cross-validation cannot fully eliminate statistical bias because the train and test datasets both constitute "historical data". ↩︎

2. The answer is {1,2,3,4,−1,12,2π,∞,“asinglenoncomputablenumber”,…}. ↩︎

Discuss