Вы здесь

Новости LessWrong.com

Подписка на Лента Новости LessWrong.com Новости LessWrong.com
A community blog devoted to refining the art of rationality
Обновлено: 42 минуты 40 секунд назад

Coherent behaviour in the real world is an incoherent concept

11 февраля, 2019 - 20:00
Published on February 11, 2019 5:00 PM UTC

Rohin Shah has recently criticised Eliezer’s argument that “sufficiently optimised agents appear coherent”, on the grounds that any behaviour can be rationalised as maximisation of the expectation of some utility function. In this post I dig deeper into this disagreement, concluding that Rohin is broadly correct, although the issue is more complex than he makes it out to be. Here’s Eliezer’s summary of his original argument:

Violations of coherence constraints in probability theory and decision theory correspond to qualitatively destructive or dominated behaviors. Coherence violations so easily computed as to be humanly predictable should be eliminated by optimization strong enough and general enough to reliably eliminate behaviors that are qualitatively dominated by cheaply computable alternatives. From our perspective this should produce agents such that, ceteris paribus, we do not think we can predict, in advance, any coherence violation in their behavior.

First, we need to clarify what Eliezer means by coherence. He notes that there are many formulations of coherence constraints: restrictions on preferences which imply that an agent which obeys them is maximising the expectation of some utility function. I’ll take the standard axioms of VNM utility as one representative set of constraints. In this framework, we consider a set O of disjoint outcomes. A lottery is some assignment of probabilities to the elements of O such that they sum to 1. For any pair of lotteries, an agent can either prefer one to the other, or to be indifferent between them; let P be the function (from pairs of lotteries to a choice between them) defined by these preferences. The agent is incoherent if P violates any of the following axioms: completeness, transitivity, continuity, and independence. Eliezer gives several examples of how an agent which violates these axioms can be money-pumped, which is an example of the “destructive or dominated” behaviour he mentions in the quote above. And any agent which doesn’t violate these axioms has behaviour which corresponds to maximising the expectation of some utility function over O (a function mapping the outcomes in O to real numbers).

It’s crucial to note that, in this setup, coherence is a property of an agent’s preferences at a single point in time. The outcomes that we are considering are all mutually exclusive, so an agent’s preferences over other outcomes are irrelevant after one outcome has already occurred. In addition, preferences are not observed but rather hypothetical: since outcomes are disjoint, we can’t actually observe the agent choosing a lottery and receiving a corresponding outcome (more than once).¹ But Eliezer’s argument above makes use of a concept of coherence which differs in two ways: it is a property of the observed behaviour of agents over time. VNM coherence is not well-defined in this setup, so if we want to formulate a rigorous version of this argument, we’ll need to specify a new definition of coherence which extends the standard instantaneous-hypothetical one. Here are two possible ways of doing so:

  • Definition 1: Let O be the set of all possible “snapshots” of the state of the universe at a single instant (which I shall call world-states). At each point in time when an agent chooses between different actions, that can be interpreted as a choice between lotteries over states in O. Its behaviour is coherent iff the set of all preferences revealed by those choices is consistent with some coherent preference function P over all pairs of lotteries over O AND there is a corresponding utility function which assigns values to each state that are consistent with the relevant Bellman equations. In other words, an agent’s observed behaviour is coherent iff there’s some utility function such that the utility of each state is some fixed value assigned to that state + the expected value of the best course of action starting from that state, and the agent has always chosen the action with the highest expected utility.²
  • Definition 2: Let O be the set of all possible ways that the entire universe could play out from beginning to end (which I shall call world-trajectories). Again, at each point in time when an agent chooses between different actions, that can be interpreted as a choice between lotteries over O. However, in this case no set of observed choices can ever be “incoherent” - because, as Rohin notes, there is always a utility function which assigns maximal utility to all and only the world-trajectories in which those choices were made.

To be clear on the difference between them, under definition 1 an outcome is a world-state, one of which occurs every timestep, and a coherent agent makes every choice without reference to any past events (except insofar as they provide information about its current state). Whereas under definition 2 an outcome is an entire world-trajectory (composed of a sequence of world-states), only one of which ever occurs, and a coherent agent’s future actions may depend on what happened in the past in arbitrary ways. To see how this difference plays out in practice, consider the following example of non-transitive travel preferences: an agent which pays $50 to go from San Francisco to San Jose, then $50 to go from San Jose to Berkeley, then $50 to go from Berkeley to San Francisco (note that the money in this example is just a placeholder for anything the agent values). Under 2, this isn’t evidence that the agent is incoherent, but rather just an indication that it assigns more utility to world-trajectories in which it travels round in a circle than to other available world-trajectories. Since Eliezer uses this situation as an example of incoherence, he clearly doesn’t intend to interpret behaviour as a choice between lotteries over world-trajectories. So let’s examine definition 1 in more detail. But first note that there is no coherence theorem which says that an agent’s utility function needs to be defined over world-states instead of world-trajectories, and so it’ll take additional arguments to demonstrate that sufficiently optimised agents will care about the former instead of the latter. I’m not aware of any particularly compelling arguments for this conclusion - indeed, as I’ll explain later, I think it’s more plausible to model humans as caring about the latter.

Okay, so what about definition 1? This is a more standard interpretation of having preferences over time: requiring choices under uncertainty to move between different states makes this setup very similar to POMDPs, which are often used in reinforcement learning. It would be natural to now interpret the non-transitive travel example as follows: let F, J and B be the states of being in San Francisco, San Jose and Berkeley respectively. Then paying to go from F to J to B to F demonstrates incoherent preferences over states (assuming there’s also an option to just stay put in any of those states).

First problem with this argument: there are no coherence theories saying that an agent needs to maintain the same utility function over time. In fact, there are plenty of cases where you might choose to change your utility function (or have that change thrust upon you). I like Nate Soares’ example of wanting to become a rockstar; other possibilities include being blackmailed to change it, or sustaining brain damage. However, it seems unlikely that a sufficiently intelligent AGI will face these particular issues - and in fact the more capable it is of implementing its utility function, the more valuable it will consider the preservation of that utility function.³ So I’m willing to accept that, past a certain high level of intelligence, changes significant enough to affect what utility function a human would infer from that AGI’s behaviour seem unlikely.

Here’s a more important problem, though: we’ve now ruled out some preferences which seem to be reasonable and natural ones. For example, suppose you want to write a book which is so timeless that at least one person reads it every year for the next thousand years. There is no single point at which the state of the world contains enough information to determine whether you’ve succeeded or failed in this goal: in any given year there may be no remaining record of whether somebody read it in a previous year (or the records could have been falsified, etc). This goal is fundamentally a preference over world-trajectories.⁴ In correspondence, Rohin gave me another example: a person whose goal is to play a great song in its entirety, and who isn’t satisfied with the prospect of playing the final note while falsely believing that they’ve already played the rest of the piece.⁵ More generally, I think that virtue-ethicists and deontologists are more accurately described as caring about world-trajectories than world-states - and almost all humans use these theories to some extent when choosing their actions. Meanwhile Eric Drexler’s CAIS framework relies on services which are bounded in time taken and resources used - another constraint which can’t be expressed just in terms of individual world-states.

There’s a third issue with this framing: in examples like non-transitive travel, we never actually end up in quite the same state we started in. Perhaps we’ve gotten sunburned along the journey. Perhaps we spent a few minutes editing our next blog post. At the very least, we’re now slightly older, and we have new memories, and the sun’s position has changed a little. So really we’ve ended up in state F’, which differs in many ways from F. You can presumably see where I’m going with this: just like with definition 2, no series of choices can ever demonstrate incoherent revealed preferences in the sense of definition 1, since every choice actually made is between a different set of possible world-state outcomes. (At the very least, they differ in the agent’s memories of which path it took to get there.⁶ And note that outcomes which are identical except for slight differences in memories should sometimes be treated in very different ways, since having even a few bits of additional information from exploration can be incredibly advantageous.)

Now, this isn’t so relevant in the human context because we usually abstract away from the small details. For example, if I offer to sell you an ice-cream and you refuse it, and then I offer it again a second later and you accept, I’d take that as evidence that your preferences are incoherent - even though technically the two offers are different because accepting the first just leads you to a state where you have an ice-cream, while accepting the second leads you to a state where you both have an ice-cream and remember refusing the first offer. Similarly, I expect that you don’t consider two outcomes to be different if they only differ in the precise pattern of TV static or the exact timing of leaves rustling. But again, there are no coherence constraints saying that an agent can’t consider such factors to be immensely significant, enough to totally change their preferences over lotteries when you substitute in one such outcome for the other.

So for the claim that sufficiently optimised agents appear coherent to be non-trivially true under my first definition of coherence, we’d need to clarify that such coherence is only with respect to outcomes when they’re categorised according to the features which humans consider important, except for the ones which are intrinsically temporally extended. But then the standard arguments from coherence constraints no longer apply. At this point I think it’s better to abandon the whole idea of formal coherence as a predictor of real-world behaviour, and replace it with Rohin’s notion of “goal-directedness”, which is more upfront about being inherently subjective, and doesn’t rule out any of the goals that humans actually have.

Thanks to Tim Genewein, Ramana Kumar, Victoria Krakovna and Rohin Shah for discussions which led to this post, and helpful comments.

[1] Disjointedness of outcomes makes this argument more succinct, but it’s not actually a necessary component, because once you’ve received one outcome, your preferences over all other outcomes are allowed to change. For example, having won $1000000, the value you place on other financial prizes will very likely go down. This is related to my later argument that you never actually have multiple paths to ending up in the “same” state.

[2] Technical note: I’m assuming an infinite time horizon and no discounting, because removing either of those conditions leads to weird behaviour which I don’t want to dig into in this post. In theory this leaves open the possibility of states with infinite expected utility, as well as lotteries over infinitely many different states, but I think we can just stipulate that neither of those possibilities arises without changing the core idea behind my argument. The underlying assumption here is something like: whether we model the universe as finite or infinite shouldn’t significantly affect whether we expect AI behaviour to be coherent over the next few centuries, for any useful definition of coherent.

[3] Consider the two limiting cases: if I have no power to implement my utility function, then it doesn’t make any difference what it changes to. By comparison, if I am able to perfectly manipulate the world to fulfil my utility function, then there is no possible change in it which will lead to better outcomes, and many which will lead to worse (from the perspective of my current utility function).

[4] At this point you could object on a technicality: from the unitarity of quantum mechanics, it seems as if the laws of physics are in fact reversible, and so the current state of the universe (or multiverse, rather) actually does contain all the information you theoretically need to deduce whether or not any previous goal has been satisfied. But I’m limiting this claim to macroscopic-level phenomena, for two reasons. Firstly, I don’t think our expectations about the behaviour of advanced AI should depend on very low-level features of physics in this way; and secondly, if the objection holds, then preferences over world-states have all the same problems as preferences over world-trajectories.

[5] In a POMDP, we don’t usually include an agent’s memories (i.e. a subset of previous observations) as part of the current state. However, it seems to me that in the context of discussing coherence arguments it’s necessary to do so, because otherwise going from a known good state to a known bad state and back in order to gain information is an example of incoherence. So we could also formulate this setup as a belief MDP. But I prefer talking about it as a POMDP, since that makes the agent seem less Cartesian - for example, it makes more sense to ask what happens after the agent “dies” in a POMDP than a belief MDP.

[6] Perhaps you can construct a counterexample involving memory loss, but this doesn’t change the overall point, and if you’re concerned with such technicalities you’ll also have to deal with the problems I laid out in footnote 4.



Discuss

How important is it that LW has an unlimited supply of karma?

11 февраля, 2019 - 04:41
Published on February 11, 2019 1:41 AM UTC

Question

LessWrong users can up/downvote posts and comments, which then receive a karma boost (capped by the voters own karma). There is no limit to how many different posts and comments one can do this to. In this sense there is an unlimited supply of karma to be handed out. (This is also the case for Facebook, YouTube, Instagram, HackerNews(?), Medium, ...)

Is this important? That is, does it have non-trivial medium or long-term effects on the “LessWrong economy" -- the kind and amount of content that gets produced?

Rough Thoughts

Here are some quick thoughts I wrote down. I publish this question despite them being unfinished, instead of letting them wither deep in my Google Drive.

Under the current system…

  • Over time, it’s not clear whether karma is inflationary or deflationary. It depends at least on whether the rate of growth of content is slower or faster than the rate of increase of karma production.
  • The only way to get a large amount of karma is to produce content that appeals to many users, or produce a large amount of content that appeals to at least some users. One cannot get high karma by producing a small amount of content that a small number of users likes a lot.
    • If the real economy was like this, there wouldn’t exist businesses like SpaceX, Palantir or Boeing.
      • Something seems very broken about LW, if, were the big world to run on LW principles, people wouldn’t be able to fly as a means of travel. Lots of people want to fly. But very few are able to pay for the construction of a 747. So we only have airtravel because there can exist intermediaries who can make that payment, and in turn get rewarded by collecting all the little flight desires of very many people kind-of-keen to fly.
      • Currently, there cannot be any such intermediaries on LessWrong. A concrete example of a LessWrong Boeing might be something like: CFAR really wants someone to write a 40-page literature review of X. No one else really cares, apart from the fact that were CFAR to get that review, their workshops would improve pretty significantly for most attendees.
  • There are fewer free-rider problems. Despite content being non-excludable and publicly available, users have an incentive to upvote things, because Alice doing so instead of Bob does not cost Alice anything (we’re assuming they both end up consuming the content, so attention and time costs are the same).
    • This seems very important, and like something that could offset the "Boeing problem" mentioned above.

If instead of the current system each karma point given was taken from your own score, then…

  • One could not indefinitely keep up/down-voting content without producing new content oneself. In practice, one could do this if one had created one beacon of amazing work in the past.
  • Over time, as the same amount of karma gets spread across more and more content, the value of a karma point increases (because the opportunity cost of what else that karma point could have been used for increases).
  • There might be even more deflationary pressure on karma if users produce great content but then leave the site.
  • There is a disconnect between content karma and user karma. A user who has produced much high-quality content might not have a corresponding amount of karma, having given it away.

Some uncertainties

  • A salient implementation is that an upvote costs exactly the amount of karma that’s being awarded to the content. But how much karma should downvotes cost?
  • It is unclear how a limited karma supply interfaces with a limited maximal upvote size
  • There might be lessons from macroeconomics and monetary policy relevant to this. I don’t know, because I know something-that-rounds-to-nothing about those fields.


Discuss

Perfecting The Motion

11 февраля, 2019 - 03:33
Published on February 11, 2019 12:33 AM UTC

See also: Life can be better than you think.

“How we spend our days is, of course, how we spend our lives.”

— Annie Dillard

“To finish the moment, to find the journey’s end in every step of the road, to live the greatest number of good hours, is wisdom.”

— Ralph Waldo Emerson

The way I see it, the aim of life is to maintain a sustainable and pleasant motion. The idea that happiness is not a destination, but a journey, although admittedly rather trite and cliche, strikes me as something to be taken very seriously, both at an individual level and at a societal level.

We lust for vacation, love, accomplishments, etc., partially because we’re driven by instinct to do so, and partially because those were the things that have made us happy in the past. But one major issue with that is that there is more variability in happiness between, rather than within, individuals, and the set of human experiences that is possible today is vastly larger than the set of what was possible throughout human evolution. We learn what makes us happy based on our past experiences — individually and as a species — and those past experiences are but a sliver of all possible human experiences.

(On top of that, one of the findings of happiness research I have found most interesting is that someone’s baseline mood correlates negatively with their variability in mood. Unhappy people have a better idea of what it is like to be happy than happy people have an idea of what it is like to be unhappy. For unhappy people, happiness is a fleeting experience, associated with love, accomplishments, etc., then quickly dissipating, whereas for happy people — in a certain way for most of us — it is merely a way of life.)

This post is partly inspired by an essay Arthur Schopenhauer wrote in the 19th century, The Emptiness of Existence. I was absolutely fascinated by it when I was a 15-year-old angsty teen, and now that I am older, happier, and don’t see much point in reading philosophy from the time when computers did not exist, the text does not fascinate me nearly as much — but it still seems to me to be the case that Schopenhauer, both in that essay as well as in much of the rest of his work, points out a very crucial aspect of life. Namely, the pressing nature of entropy, and the necessity for continuous motion and restless striving. Quoting from the essay:

“Our existence is based solely on the ever-fleeting present. Essentially, therefore, it has to take the form of continual motion without there ever being any possibility of our finding the rest after which we are always striving. It is the same as a man running downhill, who falls if he tries to stop, and it is only by his continuing to run on that he keeps on his legs; it is like a pole balanced on one’s finger-tips, or like a planet that would fall into its sun as soon as it stopped hurrying onwards. Hence unrest is the type of existence. [...]“Looking at the matter a little closer, we see at the very outset that the existence of inorganic matter is being constantly attacked by chemical forces which eventually annihilates it. While organic existence is only made possible by continual change of matter, to keep up a perpetual supply of which it must consequently have help from without. Therefore organic life is like balancing a pole on one’s hand; it must be kept in continual motion, and have a constant supply of matter of which it is continually and endlessly in need. Nevertheless it is only by means of this organic life that consciousness is possible.”

I think that there is immense value in acknowledging that, in acknowledging the restlessness and continual motion of life, in acknowledging that there isn’t really a restful end. You just need to replace the hopelessness in the text with well-founded optimism, with a desire to bring about continuous improvement.

That is the reason why I am so interested in optimizing routine. Not only that, but also in doing the same at a broader time scale — optimizing the rhythm, the motion of life, so to say, of not only my days my also of my weeks, months, and years. That strikes me as a very, very important question in the search for happiness.

I am very far from being a know-it-all when it comes to optimal scheduling, and in fact, I began examining this issue in-depth only rather recently, after spending a few weeks in panic under the overwhelming pressure of all the things that I wanted to do. But I am going to share some of my preliminary thoughts.

In life, we face a certain tradeoff. The modern world is significantly different from the environment of evolutionary adaptedness, and so our instincts are no longer a guide for success. Humans do not feel naturally inclined to sit on a desk for eight hours a day, and yet many of us ought to do just that, apart from having to pay bills, make calls, think far into the future, make complex decisions, eat healthy, go to the doctor, visit relatives, etc.

Doing all of those things right can require quite a lot of thinking and planning. Unfortunately, that can be detrimental for our happiness. There’s often more authenticity, as well as more joy, in living mostly in execution-mode, in living in the present moment without thinking much about either the past or the future. Mind-wandering has a substantially large negative relationship with mood.

Thankfully, I think we can use computers to outsource the scheduling we need to keep our lives in motion, in such a way as to allow ourselves to spent long stretches of time being silly, fun and happy in the moment, or in the wonderful flow of deep work — states associated with good mood, and with getting things done — while simultaneously knowing that we don’t need to bother ourselves with the distraction of the million of issues in our lives, knowing that we are going to deal with whatever other issues that involve what we are not doing at the moment at a later time.

That is possible because there is a certain predictability, a certain cyclicality to life’s duties. At approximately fixed intervals of time, you must perform the very same actions, or analogous actions. It is a good exercise to summarize what all those actions are and what is the optimal timing for their execution. Working, studying, paying rent, getting a haircut, going to the doctor, talking to your relatives, taking your relatives to the doctor, etc. all follow a certain predictable regularity. Furthermore, although at different stages in your life the set of things you need to do in a frequent basis will be different, there will be patterns, and the more clearly you notice those patterns, the less you risk being caught off-guard by a bad life event.

I think it’s very important to develop that trust in yourself — the trust that yes, you can just enjoy yourself now, or that you can just keep working on this one thing for several hours, and not have worry about the million of other issues in your life. It’s important to show to yourself that you deserve that trust, moreover, by repeatedly succeeding at such time management.

I myself use Wunderlist in order to schedule actions to be taken at certain times in the future; that allows me to do wonderful superhuman things like signing up for 7-day free trials of software and cancelling the subscription in time and remembering to cancel before being billed. (Recently I’ve been slowly but steadily developing the necessity for something more complex than that, so I hope to work more closely on this issue later in the future.)

I think the Computer Science concept of batch processing is an important component of an ideal motion for life. Brian Christian and Tom Griffiths describes that and the closely related concept of interrupt coalescing in Algorithms to Live By (wonderful book, I recommend reading it, or at least checking out the 80000hours podcast episode with the authors and Robert Wiblin.). Batch processing and interrupt coalescing basically come down to scheduling the things you have to do in a regular basis in a manner so as to minimize the instances of context-switching, so as to maximize the amount of time spent on one task uninterruptedly. I really like that idea, as it seems to make it much more easy to achieve a state of flow, and to be fully immersed in what one is doing. Brian Christian's practical advice is synthesized well in this paragraph:

“The moral is that you should try to stay on a single task as long as possible without decreasing your responsiveness below the minimum acceptable limit. Decide how responsive you need to be—and then, if you want to get things done, be no more responsive than that.”

That, perhaps, may also help make work and study more enjoyable. I don’t know if this would work as well for others, but I myself have grown to deeply enjoy quite a few tasks that I used to find irksome merely by immersing myself deeply in them. It’s much more fun to become the Calculus Queen for sixteen hours at a time than to begrudgingly study a little bit per day because I have to.

Relatedly, when I lived with my parents I remember I routinely had to leave home for a few hours, and it always killed me inside. Spending three hours outside of home means way more than three hours of work lost. Tellingly, I learned to code very quickly one month after I started living on my own and spending more time alone than I had ever been able to before. Finding joy in work and immersing yourself in it seems to be a wise way of dealing with the endless striving that characterizes life, both in efficiency and in valence and batch processing seems to help with that.

“Life presents itself next as a task, the task, that is, of subsisting.”

— Arthur Schopenhauer

“Now, here, you see, it takes all the running you can do, to keep in the same place. If you want to get somewhere else, you must run at least twice as fast as that!”

— Lewis Carrol

“There is only one law of Nature — the second law of thermodynamics.”

—Arthur Eddington, in The Nature of The Physical World

I don’t know what a maximally good world would look like. What I do know, however, is that stillness is not an option.

Continuous work is necessary in order to avert entropy. That’s largely why I like the sentence “the meadows of heaven await harvest,” and have it as my personal website’s subtitle. It encourages optimism, but simultaneously emphasizes that heaven is not static, that it is not an idle, restful state, but rather something that is to be continuously harvested: it encourages a definite optimism. Due to the nature of entropy, that seems to me very much appropriate.

Just like while building our routine we need to understand that the very process is in a certain way an end-in-itself, and there’s no such thing as a permanent destination, the same concept applies to axiology.

Schopenhauer is far from being the only person to have noticed the importance of entropy. Steven Pinker does a great job at describing the importance of entropy in Enlightenment Now:

“How is entropy relevant to human affairs? Life and happiness depend on an infinitesimal sliver of orderly arrangements of matter amid the astronomical number of possibilities. Our bodies are improbable assemblies of molecules, and they maintain that order with the help of other improbabilities: the few substances that can nourish us, the few materials in the few shapes that can clothe us, shelter us, and move things around to our liking. Far more of the arrangements of matter found on Earth are of no worldly use to us, so when things change without a human agent directing the change, they are likely to change for the worse.

“Not only does the universe not care about our desires, but in the natural course of events it will appear to thwart them, because there are so many more ways for things to go wrong than for them to go right. Houses burn down, ships sink, battles are lost for want of a horseshoe nail. […]“Poverty, too, needs no explanation. In a world governed by entropy and evolution, it is the default state of humankind. Matter does not arrange itself into shelter or clothing, and living things do everything they can to avoid becoming our food. As Adam Smith pointed out, what needs to be explained is wealth.”

(I’ll preemptively apologize for citing Pinker; I am fully aware and upset with his refusal to acknowledge existential risk, but it still seems to me that 95%+ of his contributions are valuable.)

There is a reason why, despite being so fundamental, entropy seems to be underrated. As Peter Thiel describes in Zero to One, technological progress has made people assume that things get better by default:

“Indefinite optimism has dominated American thinking ever since 1982, when a long bull market began and finance eclipsed engineering as the way to approach the future. To an indefinite optimist, the future will be better, but he doesn’t know how exactly, so he won’t make any specific plans. He expects to profit from the future but sees no reason to design it concretely.“The strange history of the Baby Boom produced a generation of indefinite optimists so used to effortless progress that they feel entitled to it. Whether you were born in 1945 or 1950 or 1955, things got better every year for the first 18 years of your life, and it had nothing to do with you. Technological advance seemed to accelerate automatically, so the Boomers grew up with great expectations but few specific plans for how to fulfill them.”

(I thank Scott Alexander for having recently reviewed the book; it made me read it, and I really enjoyed it. in retrospect I should’ve read it much sooner.)

One issue with this view, which Peter Thiel pointed out, is that it is inherently unsustainable. “How can the future get better if no one plans for it?” he asks in the book. “To a definite optimist,” on the other hand, “the future will be better than the present if he plans and works to make it better,” which to me makes more sense. As Steven Pinker pointed out, things don’t get better by default. Definite optimism sounds to me like combining Schopenhauer’s harsh realism with a fruitful can-do attitude.

Another issue with underestimating the relevance of entropy is that doing so severely taints our moral intuitions.

“Many people imagine some future that won’t be much fun—and it doesn’t even seem to occur to them to try and change it.”

Eliezer Yudkowsky

As Ozy describes it,

“One very common critique of hedonic utilitarianism is the wireheading objection. If you try to fill the universe with beings experiencing as much pleasure as possible, then the perfect world would consist of nothing but rats– a larger or more intelligent animal would use up resources better spent on new morally relevant beings– with a steady drip of heroin into their systems, and the infrastructure necessary to keep them alive and drugged. (If you don’t happen to think animals are morally relevant, feel free to replace “rats’ with “humans.’) This seems, to put it lightly, counterintuitive.”

The rats on heroin meme is nothing new to EAs, every once in a while showing up in the Dank EA Memes Facebook group. Last year Scott Alexander wrote:

“Utilitarianism agrees that we should give to charity and shouldn’t steal from the poor, because Utility, but take it far enough to the tails and we should tile the universe with rats on heroin. [...]“This is why I feel like figuring out a morality that can survive transhuman scenarios is harder than just finding the Real Moral System That We Actually Use. There’s a potentially impossible conceptual problem here, of figuring out what to do with the fact that any moral rule followed to infinity will diverge from large parts of what we mean by morality.

The post includes this lovely chart:

But it seems to me that in large part, such divergence is an illusion; it seems to me that there isn’t a disagreement there. Intelligence, as well as lucidity and self-awareness, seem to me altogether necessary for infinite bliss, in a world where it takes all the running you can do to keep in the same place, in a world where life is only made possible by continual change of matter and must be kept in continual motion. Perhaps intelligence wouldn’t be used to its fullest 100% of the time in 100% of the locations of the universe, but contingent on the Second Law of Thermodynamics, its availability and readiness would be necessary. The infrastructure needed to maintain beings experiencing infinite euphoria throughout the entire universe which Ozy briefly mentioned would need to be extremely complex.

So the usage of that phrase might gratuitously and actively harm the image of utilitarianism (especially hedonic utilitarianism) by making it repulsive to the most well-meaning and altruistic of people wanting to make the world a better place, and by making many people confused about what it is that they truly value.

Granted, it could be that a superintelligence would eventually completely beat entropy (and therefore time itself(?)), thereby rendering that constraint unnecessary. But it seems to me that a world devoid of entropy would be so unimaginably different from ours that there is a burden of proof to claiming that the concept of “rats on heroin” would even make any sense at all. What would it mean for entropy not to exist? Would valence even be possible in such a world?

And, granted, I guess you could say that that phrase is just a metaphor, or a theoretical ideal. But at some point someone will have to investigate the minute engineering details. At some point someone will have to take the definite optimism approach to this whole “making the world a better place” thing.



Discuss

Minimize Use of Standard Internet Food Delivery

10 февраля, 2019 - 22:50
Published on February 10, 2019 7:50 PM UTC

Epistemic Status: Public service announcement. Confident and not sponsored.

Today, I went to one of my favorite local restaurants to find it was closed.

This is not an uncommon occurrence. About a month prior, I lost perhaps my favorite place in the world to go for a nice meal, BLT Prime. Today, I learned I’d lost my favorite Indian place, Old Monk. The list goes on. This has become frequent enough that I’m going to work on a list of places I’m afraid will close, so I can encourage others to help them keep the lights on.

The best way to help keep everyone’s lights on is simple. If you like the restaurant and want those working there to earn a living, and the place to continue to exist, do not order via online services like SeamlessWeb, GrubHub, Delivery.com or Caviar, if there is another way to contact the restaurant. Period.

This is because they take mindbogglingly huge fees out of every order. We’re talking on the order of 20%. I am not one to begrudge a middle man or market creator their reasonable fee. This is not a reasonable fee.

But because customers don’t know, and the store is forced to eat the entire cost or lose the order since customers have been trained by small conveniences and bribes to use the apps and websites, the fees continue to be collected, and the cycle continues. The few places that pass the cost along look super greedy and lose business.

If you would cost your local place $5 to save the cost of a fifteen second phone call, make no mistake. You are defecting. You are playing zero-sum games with those who should be your allies. You are bad, and you should feel bad.

This is way, way, way worse than not tipping where tipping is expected. Not tipping is shirking on the price and pocketing the money. Here you don’t even get the money.

If you are super rich and your time is that valuable, you can tip them 50% (or 500%) and make up for it. In that case, go for it. For the rest of us, seek out the restaurant’s website or if necessary, at least once you know they’re legit, pick up the damn phone. Talking to a human is a small price to pay to support what you get value from.

That’s why the promotions they bombard me with are so rich. How can they give me such deep discounts on almost every order I make? Now I know. They aren’t even always losing money on those orders. The bastards.

In New York City, the pizza places are fighting back using an app called Slice. Slice is essentially the same as the other apps, except it is run by and for pizza places. Thus it only offers local pizza and not other cuisines, but it allows pizza places to avoid the giant fees. As a bonus, they exclude horrible chains from your delivery options. They once sent me a hilarious promotion accusing (very, very guilty) chain pizza stores of ‘crimes against pizza.’

If you can, use Slice. I hope there’s more of these for other types of places in the future. Or better yet, I hope they already exist, in which case tell me in the comments and I’ll update the post.

There are larger principles in play. They are important. But first, be concrete. Start here.



Discuss

Propositional Logic, Syntactic Implication

10 февраля, 2019 - 21:12
Published on February 10, 2019 6:12 PM UTC

Propositional LogicSome Definitions

Propositional logic is a simplified model of what it means to prove things. The statements that propositional logic can work with are called propositions. Propositions consist of the primitives .mjx-chtml {display: inline-block; line-height: 0; text-indent: 0; text-align: left; text-transform: none; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; word-wrap: normal; word-spacing: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0; min-height: 0; border: 0; margin: 0; padding: 1px 0} .MJXc-display {display: block; text-align: center; margin: 1em 0; padding: 0} .mjx-chtml[tabindex]:focus, body :focus .mjx-chtml[tabindex] {display: inline-table} .mjx-full-width {text-align: center; display: table-cell!important; width: 10000em} .mjx-math {display: inline-block; border-collapse: separate; border-spacing: 0} .mjx-math * {display: inline-block; -webkit-box-sizing: content-box!important; -moz-box-sizing: content-box!important; box-sizing: content-box!important; text-align: left} .mjx-numerator {display: block; text-align: center} .mjx-denominator {display: block; text-align: center} .MJXc-stacked {height: 0; position: relative} .MJXc-stacked > * {position: absolute} .MJXc-bevelled > * {display: inline-block} .mjx-stack {display: inline-block} .mjx-op {display: block} .mjx-under {display: table-cell} .mjx-over {display: block} .mjx-over > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-under > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-stack > .mjx-sup {display: block} .mjx-stack > .mjx-sub {display: block} .mjx-prestack > .mjx-presup {display: block} .mjx-prestack > .mjx-presub {display: block} .mjx-delim-h > .mjx-char {display: inline-block} .mjx-surd {vertical-align: top} .mjx-mphantom * {visibility: hidden} .mjx-merror {background-color: #FFFF88; color: #CC0000; border: 1px solid #CC0000; padding: 2px 3px; font-style: normal; font-size: 90%} .mjx-annotation-xml {line-height: normal} .mjx-menclose > svg {fill: none; stroke: currentColor} .mjx-mtr {display: table-row} .mjx-mlabeledtr {display: table-row} .mjx-mtd {display: table-cell; text-align: center} .mjx-label {display: table-row} .mjx-box {display: inline-block} .mjx-block {display: block} .mjx-span {display: inline} .mjx-char {display: block; white-space: pre} .mjx-itable {display: inline-table; width: auto} .mjx-row {display: table-row} .mjx-cell {display: table-cell} .mjx-table {display: table; width: 100%} .mjx-line {display: block; height: 0} .mjx-strut {width: 0; padding-top: 1em} .mjx-vsize {width: 0} .MJXc-space1 {margin-left: .167em} .MJXc-space2 {margin-left: .222em} .MJXc-space3 {margin-left: .278em} .mjx-test.mjx-test-display {display: table!important} .mjx-test.mjx-test-inline {display: inline!important; margin-right: -1px} .mjx-test.mjx-test-default {display: block!important; clear: both} .mjx-ex-box {display: inline-block!important; position: absolute; overflow: hidden; min-height: 0; max-height: none; padding: 0; border: 0; margin: 0; width: 1px; height: 60ex} .mjx-test-inline .mjx-left-box {display: inline-block; width: 0; float: left} .mjx-test-inline .mjx-right-box {display: inline-block; width: 0; float: right} .mjx-test-display .mjx-right-box {display: table-cell!important; width: 10000em!important; min-width: 0; max-width: none; padding: 0; border: 0; margin: 0} .MJXc-TeX-unknown-R {font-family: monospace; font-style: normal; font-weight: normal} .MJXc-TeX-unknown-I {font-family: monospace; font-style: italic; font-weight: normal} .MJXc-TeX-unknown-B {font-family: monospace; font-style: normal; font-weight: bold} .MJXc-TeX-unknown-BI {font-family: monospace; font-style: italic; font-weight: bold} .MJXc-TeX-ams-R {font-family: MJXc-TeX-ams-R,MJXc-TeX-ams-Rw} .MJXc-TeX-cal-B {font-family: MJXc-TeX-cal-B,MJXc-TeX-cal-Bx,MJXc-TeX-cal-Bw} .MJXc-TeX-frak-R {font-family: MJXc-TeX-frak-R,MJXc-TeX-frak-Rw} .MJXc-TeX-frak-B {font-family: MJXc-TeX-frak-B,MJXc-TeX-frak-Bx,MJXc-TeX-frak-Bw} .MJXc-TeX-math-BI {font-family: MJXc-TeX-math-BI,MJXc-TeX-math-BIx,MJXc-TeX-math-BIw} .MJXc-TeX-sans-R {font-family: MJXc-TeX-sans-R,MJXc-TeX-sans-Rw} .MJXc-TeX-sans-B {font-family: MJXc-TeX-sans-B,MJXc-TeX-sans-Bx,MJXc-TeX-sans-Bw} .MJXc-TeX-sans-I {font-family: MJXc-TeX-sans-I,MJXc-TeX-sans-Ix,MJXc-TeX-sans-Iw} .MJXc-TeX-script-R {font-family: MJXc-TeX-script-R,MJXc-TeX-script-Rw} .MJXc-TeX-type-R {font-family: MJXc-TeX-type-R,MJXc-TeX-type-Rw} .MJXc-TeX-cal-R {font-family: MJXc-TeX-cal-R,MJXc-TeX-cal-Rw} .MJXc-TeX-main-B {font-family: MJXc-TeX-main-B,MJXc-TeX-main-Bx,MJXc-TeX-main-Bw} .MJXc-TeX-main-I {font-family: MJXc-TeX-main-I,MJXc-TeX-main-Ix,MJXc-TeX-main-Iw} .MJXc-TeX-main-R {font-family: MJXc-TeX-main-R,MJXc-TeX-main-Rw} .MJXc-TeX-math-I {font-family: MJXc-TeX-math-I,MJXc-TeX-math-Ix,MJXc-TeX-math-Iw} .MJXc-TeX-size1-R {font-family: MJXc-TeX-size1-R,MJXc-TeX-size1-Rw} .MJXc-TeX-size2-R {font-family: MJXc-TeX-size2-R,MJXc-TeX-size2-Rw} .MJXc-TeX-size3-R {font-family: MJXc-TeX-size3-R,MJXc-TeX-size3-Rw} .MJXc-TeX-size4-R {font-family: MJXc-TeX-size4-R,MJXc-TeX-size4-Rw} .MJXc-TeX-vec-R {font-family: MJXc-TeX-vec-R,MJXc-TeX-vec-Rw} .MJXc-TeX-vec-B {font-family: MJXc-TeX-vec-B,MJXc-TeX-vec-Bx,MJXc-TeX-vec-Bw} @font-face {font-family: MJXc-TeX-ams-R; src: local('MathJax_AMS'), local('MathJax_AMS-Regular')} @font-face {font-family: MJXc-TeX-ams-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_AMS-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_AMS-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_AMS-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-B; src: local('MathJax_Caligraphic Bold'), local('MathJax_Caligraphic-Bold')} @font-face {font-family: MJXc-TeX-cal-Bx; src: local('MathJax_Caligraphic'); font-weight: bold} @font-face {font-family: MJXc-TeX-cal-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-R; src: local('MathJax_Fraktur'), local('MathJax_Fraktur-Regular')} @font-face {font-family: MJXc-TeX-frak-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-B; src: local('MathJax_Fraktur Bold'), local('MathJax_Fraktur-Bold')} @font-face {font-family: MJXc-TeX-frak-Bx; src: local('MathJax_Fraktur'); font-weight: bold} @font-face {font-family: MJXc-TeX-frak-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-BI; src: local('MathJax_Math BoldItalic'), local('MathJax_Math-BoldItalic')} @font-face {font-family: MJXc-TeX-math-BIx; src: local('MathJax_Math'); font-weight: bold; font-style: italic} @font-face {font-family: MJXc-TeX-math-BIw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-BoldItalic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-BoldItalic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-BoldItalic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-R; src: local('MathJax_SansSerif'), local('MathJax_SansSerif-Regular')} @font-face {font-family: MJXc-TeX-sans-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-B; src: local('MathJax_SansSerif Bold'), local('MathJax_SansSerif-Bold')} @font-face {font-family: MJXc-TeX-sans-Bx; src: local('MathJax_SansSerif'); font-weight: bold} @font-face {font-family: MJXc-TeX-sans-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-I; src: local('MathJax_SansSerif Italic'), local('MathJax_SansSerif-Italic')} @font-face {font-family: MJXc-TeX-sans-Ix; src: local('MathJax_SansSerif'); font-style: italic} @font-face {font-family: MJXc-TeX-sans-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-script-R; src: local('MathJax_Script'), local('MathJax_Script-Regular')} @font-face {font-family: MJXc-TeX-script-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Script-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Script-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Script-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-type-R; src: local('MathJax_Typewriter'), local('MathJax_Typewriter-Regular')} @font-face {font-family: MJXc-TeX-type-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Typewriter-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Typewriter-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Typewriter-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-R; src: local('MathJax_Caligraphic'), local('MathJax_Caligraphic-Regular')} @font-face {font-family: MJXc-TeX-cal-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-B; src: local('MathJax_Main Bold'), local('MathJax_Main-Bold')} @font-face {font-family: MJXc-TeX-main-Bx; src: local('MathJax_Main'); font-weight: bold} @font-face {font-family: MJXc-TeX-main-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-I; src: local('MathJax_Main Italic'), local('MathJax_Main-Italic')} @font-face {font-family: MJXc-TeX-main-Ix; src: local('MathJax_Main'); font-style: italic} @font-face {font-family: MJXc-TeX-main-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-R; src: local('MathJax_Main'), local('MathJax_Main-Regular')} @font-face {font-family: MJXc-TeX-main-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-I; src: local('MathJax_Math Italic'), local('MathJax_Math-Italic')} @font-face {font-family: MJXc-TeX-math-Ix; src: local('MathJax_Math'); font-style: italic} @font-face {font-family: MJXc-TeX-math-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size1-R; src: local('MathJax_Size1'), local('MathJax_Size1-Regular')} @font-face {font-family: MJXc-TeX-size1-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size1-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size1-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size1-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size2-R; src: local('MathJax_Size2'), local('MathJax_Size2-Regular')} @font-face {font-family: MJXc-TeX-size2-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size2-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size2-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size2-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size3-R; src: local('MathJax_Size3'), local('MathJax_Size3-Regular')} @font-face {font-family: MJXc-TeX-size3-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size3-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size3-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size3-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size4-R; src: local('MathJax_Size4'), local('MathJax_Size4-Regular')} @font-face {font-family: MJXc-TeX-size4-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size4-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size4-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size4-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-R; src: local('MathJax_Vector'), local('MathJax_Vector-Regular')} @font-face {font-family: MJXc-TeX-vec-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-B; src: local('MathJax_Vector Bold'), local('MathJax_Vector-Bold')} @font-face {font-family: MJXc-TeX-vec-Bx; src: local('MathJax_Vector'); font-weight: bold} @font-face {font-family: MJXc-TeX-vec-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Bold.otf') format('opentype')} ⊥,p1,p2,p3,⋯ as well as anything of the form (X⟹Y) where X and Y are propositions. Note for pedants, this description allows the existence of propositions that can't be broken down into a finite number of primitives, we don't want such propositions.

Let A be the set of all propositions formed by taking any arbitrary propositions X, Y and Z, in any of the following three arrangements.

1) (X⟹(Y⟹X))

2) ((X⟹(Y⟹Z))⟹((X⟹Y)⟹(X⟹Z)))

3) (((X⟹⊥)⟹⊥)⟹X)

The set A is the set of axioms, and the three forms shown above are the infinite axiom schema. For example, taking X=p1 and Y=(p2⟹⊥) then applying the first schema gets (p1⟹((p2⟹⊥)⟹p1))∈A. So this is an axiom.

We say that a set of propositions S implies another proposition p, when there exists a proof of p from S. Write this as S⊢p.

A proof of a proposition p form S is a list L of propositions of length n∈N such that Ln=p and for each i≤n, Li fulfills at least one of the following 3 conditions.

1) Li∈S

2) Li∈A

3) ∃j,k<i:Lj=(Lk⟹Li) This is called Modus ponus

Example Proof

For example, here is a proof of (p⟹p)

L1=((pX⟹((p⟹p)Y⟹pZ))⟹((pX⟹(p⟹p)Y)⟹(pX⟹pZ)))

Formed using the second axiom schema.

L2=(pX⟹((p⟹p)Y⟹pX))

Is then created using the first axiom schema

L3=((p⟹(p⟹p))⟹(p⟹p))

Can be produced by modus ponus because L1=(L2⟹L3).

L4=(pX⟹(pY⟹pX))

Is allowed by using axiom schema 1 again.

L5=(p⟹p)

Follows by modus ponus again because L3=(L4⟹L5).

Philosophically Important Takeaways

The notion of mathematical proof is fully formalized. There is an entirely mechanical, and fairly fast, method of verifying proofs. Proofs can be generated in a finite amount of time, by brute force search, but this can be much slower.

The proof framework only believes things that you have actually proved in it. From our point of view outside the framework, the proof of (p⟹p) also shows that (q⟹q), but to actually prove that within the framework, you have to go through all the steps again.



Discuss

One week of CBT journaling: Fighting the allure of depressive realism

10 февраля, 2019 - 19:46
Published on February 10, 2019 4:46 PM UTC

Epistemic tragic backstory: Personal.

Earlier this week, on my first post on this site, shminux commented to the value of cognitive-behavioral therapy to get out of what I called "depression philosophising".

I was worried about trying it. One claim of CBT is that depressed people are negatively biased and by correcting their thinking errors they are gradually brought to being both happier and more accurate about the world around them.

However this may not actually be the case. The phenomenon known as "depressive realism" suggests that the ordinary person might be positively biased and that depressed people might need to correct their "errors" by forming less accurate, but happier patterns of cognition. Sort of like a really, really weak Nozick machine.

Now, this question doesn't actually undermine CBT itself that much. A movement from (depressed, inaccurate) to (not depressed, accurate) is pretty much as good, in terms of what any anti-depression therapy is trying to do, as one from (depressed, accurate) to (not depressed, inaccurate).

But if it's the latter, our least convenient likely world, we face the classic question: "Should we optimize more for epistemic or instrumental rationality?" This was a hurdle I had to get over before I could convince myself to use CBT. I had a few false starts, but eventually came up with some good convincing arguments that even if this is the case CBT is well worth it.

I decided to treat it as a decision on the margin, remembering my Econ 101. That turned out to be such an obviously right fit to the problem that I felt ridiculous for not having thought of it instantly. The tradeoff of a small amount of epistemic rationality (= losing the benefit of depressive realism) for a high chance of a moderate, potentially large amount of instrumental rationality (= all the time, energy, and general life pleasure I get from treating my depression) is one almost any sane person should make.

After that I also realized that there was an argument from symmetry here. Would I advise someone looking to improve their epistemic rationality to become depressed? Of course not. We have all of these tools already to improve our epistemics - the wisdom of crowds, prediction markets, good old fashioned education and reaching out to experts on whatever topics seem pertinent. But you will very likely lose at least some of the energy and motivation to pursue these much better strategies if you take the nuclear option first. Even people as already successful as Rob Wiblin think this is a good idea. The costs far, far outweigh the benefits.

Then we move to the specific case: Would I advise someone who is already using CBT successfully to treat their depression who wants to improve their epistemic rationality to stop using it and slip back into depression? Again, no, for all the reasons above.

And that general -> specific move makes me realize: These arguments are more about depression in general, not CBT in particular. While my fear started from thinking about starting CBT specifically, that's not where it actually was. It was in losing the benefits of depressive realism.

I was never scared about starting therapy. I was scared of losing part of my identity.

Throughout my life I've felt keenly aware that most people seem happy for no good reason. That they make decisions without having good evidence for why they do it. And I felt that I had to serve as some sort of counterbalance to that, that I had to be the person to bring everyone back down to reality. That was part of who I was. But those two things don't go hand in hand nearly as much as I've been telling myself. You can be happy most of the time, and you can also be aware of the human tendencies to overestimate and adjust accordingly in yourself. Life doesn't have to be this zero sum game where only the sad are wise and only the wise know enough to be sad.

I remembered a post by Natália Mendonça. I remembered that I had a whole community of people I could turn to for advice about when I was overestimating my chances on anything large enough to be worth more than 2 or 3 minutes of thought. I remembered that letting myself be depressed so I could be more realistic is making the classic mistake of trying to change human nature itself, rather than trying to change the environment to suit imperfect human nature. And, going off of that, I remembered that humans don’t owe society anything. We were here first.

I had let my desire to think accurately in every single domain in my life overpower me, and ironically cause me to think very inaccurately about the nature of my mental illness.

So I picked up a pen, and I printed out some ABC forms, and I got to work.



Discuss

Rationality Retreat in Cologne Area, Germany, Spring 2019

10 февраля, 2019 - 17:50
Published on February 10, 2019 2:50 PM UTC

I am excited to announce that I am hosting a rationality retreat near Cologne, Germany this spring. This is an opportunity for up to 20 people to meet up, share their thoughts and have a wonderful time together.

Most important information in brief, details below:

  • Date: From 30.05.2019, ~17:00h to 01.06.2019, ~11:00h
  • Location: Private house near Cologne, Germany
  • Cost: 25€ - 50€ (Food and accommodation included)
  • Format: Somewhere between an unconference and a group of friends just spending a vacation together
  • Limited to 20 participants. Apply here: https://goo.gl/forms/5fs4HLL9wAaTtDRh2
When

Thursday, May 30th (Ascension, public holiday in many countries) - Sunday, June 2nd

Cost

50€ full price, 25€ reduced price. Meals, snacks, drinks and sleeping accommodations are included in the ticket.

Who

Anyone who wants to spend a few days in the company of other aspiring rationalists is welcome.

Format

The retreat will be a participant-driven event for up to 20 people. It will begin with an opening session on Thursday afternoon but beyond that it is ultimately up to the participants to decide on how to use the time. What I imagine would work well is having two dedicated time slots on Friday and Saturday each for presentations / workshops with the remaining time being free for unstructured activities such as socializing, swimming, playing games etc. Departure will be Sunday after breakfast.

Location

My parents have generously offered to let us use their house for the weekend. The house is in the countryside, reachable within 1 - 1.5 hours from Cologne central station or Cologne/Bonn airport with public transit or half an hour by car from Cologne central station. Because this is my parents' private home I will not post the exact location publicly.

My parents' house has a large garden with a pool and a campfire site. It is located in the beautiful Bergisches Land with opportunities for hikes right outside the door. Two cuddly cats also live in the house.

Sleeping accommodations will be a mix of beds, mattresses, and air mattresses, with two to six people per room. We can also pitch a tent in the garden.

Food

All food (except for the cat food) will be vegan. We will cook meals ourselves.

Organizer

I'm Johannes, also called Joe. I was a co-organizer of the local EA group in Magdeburg for over a year and am now assisting for the second time to organize the European LessWrong Community Weekend. At the Community Weekend in 2017 there was a session on "What to do if you don't live in a rationality hub" and I got the idea that organizing a small retreat is something I could do mostly on my own and since the event lasts several days it's worth it to travel a few hours.

Application

As I am expecting the number of people interested to be higher than the number of available slots the slots will be allocated based on an application. Fill out the form here to apply. Application will be open until March 3rd. I might schedule a short call with you. Confirmations will be sent out at the latest on March 31st.


Feel free to share with anyone who might be interested in attending. Any questions? Leave them below or contact me at johannesgaetjen ⟨at⟩ gmail ⟨dot⟩ com.



Discuss

Structured Concurrency Cross-language Forum

10 февраля, 2019 - 12:20
Published on February 10, 2019 9:20 AM UTC

There are structured-concurrency-related efforts going on for different programming languages but the entire effort is kind of scattered, without people being aware of each other and speaking to each other.

If we had a common forum, we could share the use cases, the problems, the ideas and the solutions. Each of us, irrespective of which language they are working with, could benefit from this common pool of knowledge…

And here we go!

The forum now exists. Thanks to Nathaniel Smith for setting it up!

And here's my kick-off post on the forum.

Enjoy!

Feb 10th, 2019

by martin_sustrik



Discuss

Probability space has 2 metrics

10 февраля, 2019 - 03:28
Published on February 10, 2019 12:28 AM UTC

A metric is technically defined as a function from pairs of points to the non negitive reals. .mjx-chtml {display: inline-block; line-height: 0; text-indent: 0; text-align: left; text-transform: none; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; word-wrap: normal; word-spacing: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0; min-height: 0; border: 0; margin: 0; padding: 1px 0} .MJXc-display {display: block; text-align: center; margin: 1em 0; padding: 0} .mjx-chtml[tabindex]:focus, body :focus .mjx-chtml[tabindex] {display: inline-table} .mjx-full-width {text-align: center; display: table-cell!important; width: 10000em} .mjx-math {display: inline-block; border-collapse: separate; border-spacing: 0} .mjx-math * {display: inline-block; -webkit-box-sizing: content-box!important; -moz-box-sizing: content-box!important; box-sizing: content-box!important; text-align: left} .mjx-numerator {display: block; text-align: center} .mjx-denominator {display: block; text-align: center} .MJXc-stacked {height: 0; position: relative} .MJXc-stacked > * {position: absolute} .MJXc-bevelled > * {display: inline-block} .mjx-stack {display: inline-block} .mjx-op {display: block} .mjx-under {display: table-cell} .mjx-over {display: block} .mjx-over > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-under > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-stack > .mjx-sup {display: block} .mjx-stack > .mjx-sub {display: block} .mjx-prestack > .mjx-presup {display: block} .mjx-prestack > .mjx-presub {display: block} .mjx-delim-h > .mjx-char {display: inline-block} .mjx-surd {vertical-align: top} .mjx-mphantom * {visibility: hidden} .mjx-merror {background-color: #FFFF88; color: #CC0000; border: 1px solid #CC0000; padding: 2px 3px; font-style: normal; font-size: 90%} .mjx-annotation-xml {line-height: normal} .mjx-menclose > svg {fill: none; stroke: currentColor} .mjx-mtr {display: table-row} .mjx-mlabeledtr {display: table-row} .mjx-mtd {display: table-cell; text-align: center} .mjx-label {display: table-row} .mjx-box {display: inline-block} .mjx-block {display: block} .mjx-span {display: inline} .mjx-char {display: block; white-space: pre} .mjx-itable {display: inline-table; width: auto} .mjx-row {display: table-row} .mjx-cell {display: table-cell} .mjx-table {display: table; width: 100%} .mjx-line {display: block; height: 0} .mjx-strut {width: 0; padding-top: 1em} .mjx-vsize {width: 0} .MJXc-space1 {margin-left: .167em} .MJXc-space2 {margin-left: .222em} .MJXc-space3 {margin-left: .278em} .mjx-test.mjx-test-display {display: table!important} .mjx-test.mjx-test-inline {display: inline!important; margin-right: -1px} .mjx-test.mjx-test-default {display: block!important; clear: both} .mjx-ex-box {display: inline-block!important; position: absolute; overflow: hidden; min-height: 0; max-height: none; padding: 0; border: 0; margin: 0; width: 1px; height: 60ex} .mjx-test-inline .mjx-left-box {display: inline-block; width: 0; float: left} .mjx-test-inline .mjx-right-box {display: inline-block; width: 0; float: right} .mjx-test-display .mjx-right-box {display: table-cell!important; width: 10000em!important; min-width: 0; max-width: none; padding: 0; border: 0; margin: 0} .MJXc-TeX-unknown-R {font-family: monospace; font-style: normal; font-weight: normal} .MJXc-TeX-unknown-I {font-family: monospace; font-style: italic; font-weight: normal} .MJXc-TeX-unknown-B {font-family: monospace; font-style: normal; font-weight: bold} .MJXc-TeX-unknown-BI {font-family: monospace; font-style: italic; font-weight: bold} .MJXc-TeX-ams-R {font-family: MJXc-TeX-ams-R,MJXc-TeX-ams-Rw} .MJXc-TeX-cal-B {font-family: MJXc-TeX-cal-B,MJXc-TeX-cal-Bx,MJXc-TeX-cal-Bw} .MJXc-TeX-frak-R {font-family: MJXc-TeX-frak-R,MJXc-TeX-frak-Rw} .MJXc-TeX-frak-B {font-family: MJXc-TeX-frak-B,MJXc-TeX-frak-Bx,MJXc-TeX-frak-Bw} .MJXc-TeX-math-BI {font-family: MJXc-TeX-math-BI,MJXc-TeX-math-BIx,MJXc-TeX-math-BIw} .MJXc-TeX-sans-R {font-family: MJXc-TeX-sans-R,MJXc-TeX-sans-Rw} .MJXc-TeX-sans-B {font-family: MJXc-TeX-sans-B,MJXc-TeX-sans-Bx,MJXc-TeX-sans-Bw} .MJXc-TeX-sans-I {font-family: MJXc-TeX-sans-I,MJXc-TeX-sans-Ix,MJXc-TeX-sans-Iw} .MJXc-TeX-script-R {font-family: MJXc-TeX-script-R,MJXc-TeX-script-Rw} .MJXc-TeX-type-R {font-family: MJXc-TeX-type-R,MJXc-TeX-type-Rw} .MJXc-TeX-cal-R {font-family: MJXc-TeX-cal-R,MJXc-TeX-cal-Rw} .MJXc-TeX-main-B {font-family: MJXc-TeX-main-B,MJXc-TeX-main-Bx,MJXc-TeX-main-Bw} .MJXc-TeX-main-I {font-family: MJXc-TeX-main-I,MJXc-TeX-main-Ix,MJXc-TeX-main-Iw} .MJXc-TeX-main-R {font-family: MJXc-TeX-main-R,MJXc-TeX-main-Rw} .MJXc-TeX-math-I {font-family: MJXc-TeX-math-I,MJXc-TeX-math-Ix,MJXc-TeX-math-Iw} .MJXc-TeX-size1-R {font-family: MJXc-TeX-size1-R,MJXc-TeX-size1-Rw} .MJXc-TeX-size2-R {font-family: MJXc-TeX-size2-R,MJXc-TeX-size2-Rw} .MJXc-TeX-size3-R {font-family: MJXc-TeX-size3-R,MJXc-TeX-size3-Rw} .MJXc-TeX-size4-R {font-family: MJXc-TeX-size4-R,MJXc-TeX-size4-Rw} .MJXc-TeX-vec-R {font-family: MJXc-TeX-vec-R,MJXc-TeX-vec-Rw} .MJXc-TeX-vec-B {font-family: MJXc-TeX-vec-B,MJXc-TeX-vec-Bx,MJXc-TeX-vec-Bw} @font-face {font-family: MJXc-TeX-ams-R; src: local('MathJax_AMS'), local('MathJax_AMS-Regular')} @font-face {font-family: MJXc-TeX-ams-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_AMS-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_AMS-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_AMS-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-B; src: local('MathJax_Caligraphic Bold'), local('MathJax_Caligraphic-Bold')} @font-face {font-family: MJXc-TeX-cal-Bx; src: local('MathJax_Caligraphic'); font-weight: bold} @font-face {font-family: MJXc-TeX-cal-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-R; src: local('MathJax_Fraktur'), local('MathJax_Fraktur-Regular')} @font-face {font-family: MJXc-TeX-frak-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-B; src: local('MathJax_Fraktur Bold'), local('MathJax_Fraktur-Bold')} @font-face {font-family: MJXc-TeX-frak-Bx; src: local('MathJax_Fraktur'); font-weight: bold} @font-face {font-family: MJXc-TeX-frak-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-BI; src: local('MathJax_Math BoldItalic'), local('MathJax_Math-BoldItalic')} @font-face {font-family: MJXc-TeX-math-BIx; src: local('MathJax_Math'); font-weight: bold; font-style: italic} @font-face {font-family: MJXc-TeX-math-BIw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-BoldItalic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-BoldItalic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-BoldItalic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-R; src: local('MathJax_SansSerif'), local('MathJax_SansSerif-Regular')} @font-face {font-family: MJXc-TeX-sans-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-B; src: local('MathJax_SansSerif Bold'), local('MathJax_SansSerif-Bold')} @font-face {font-family: MJXc-TeX-sans-Bx; src: local('MathJax_SansSerif'); font-weight: bold} @font-face {font-family: MJXc-TeX-sans-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-I; src: local('MathJax_SansSerif Italic'), local('MathJax_SansSerif-Italic')} @font-face {font-family: MJXc-TeX-sans-Ix; src: local('MathJax_SansSerif'); font-style: italic} @font-face {font-family: MJXc-TeX-sans-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-script-R; src: local('MathJax_Script'), local('MathJax_Script-Regular')} @font-face {font-family: MJXc-TeX-script-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Script-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Script-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Script-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-type-R; src: local('MathJax_Typewriter'), local('MathJax_Typewriter-Regular')} @font-face {font-family: MJXc-TeX-type-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Typewriter-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Typewriter-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Typewriter-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-R; src: local('MathJax_Caligraphic'), local('MathJax_Caligraphic-Regular')} @font-face {font-family: MJXc-TeX-cal-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-B; src: local('MathJax_Main Bold'), local('MathJax_Main-Bold')} @font-face {font-family: MJXc-TeX-main-Bx; src: local('MathJax_Main'); font-weight: bold} @font-face {font-family: MJXc-TeX-main-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-I; src: local('MathJax_Main Italic'), local('MathJax_Main-Italic')} @font-face {font-family: MJXc-TeX-main-Ix; src: local('MathJax_Main'); font-style: italic} @font-face {font-family: MJXc-TeX-main-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-R; src: local('MathJax_Main'), local('MathJax_Main-Regular')} @font-face {font-family: MJXc-TeX-main-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-I; src: local('MathJax_Math Italic'), local('MathJax_Math-Italic')} @font-face {font-family: MJXc-TeX-math-Ix; src: local('MathJax_Math'); font-style: italic} @font-face {font-family: MJXc-TeX-math-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size1-R; src: local('MathJax_Size1'), local('MathJax_Size1-Regular')} @font-face {font-family: MJXc-TeX-size1-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size1-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size1-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size1-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size2-R; src: local('MathJax_Size2'), local('MathJax_Size2-Regular')} @font-face {font-family: MJXc-TeX-size2-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size2-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size2-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size2-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size3-R; src: local('MathJax_Size3'), local('MathJax_Size3-Regular')} @font-face {font-family: MJXc-TeX-size3-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size3-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size3-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size3-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size4-R; src: local('MathJax_Size4'), local('MathJax_Size4-Regular')} @font-face {font-family: MJXc-TeX-size4-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size4-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size4-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size4-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-R; src: local('MathJax_Vector'), local('MathJax_Vector-Regular')} @font-face {font-family: MJXc-TeX-vec-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-B; src: local('MathJax_Vector Bold'), local('MathJax_Vector-Bold')} @font-face {font-family: MJXc-TeX-vec-Bx; src: local('MathJax_Vector'); font-weight: bold} @font-face {font-family: MJXc-TeX-vec-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Bold.otf') format('opentype')} d:X×X→[0,∞) With the properties that d(x,y)=d(y,x) and d(x,y)=0⟺x=y and d(x,y)+d(y,z)≥d(x,z).

Intuitively, a metric is a way of measuring how similar points are. Which points are nearby which others. Probabilities can be represented in several different ways, including the standard p∈(0,1) range and the log odds b∈(−∞,∞). They are related by b=log(p1−p) and eb−1=11−p and p=ebeb+1 (equations algebraically equivalent)

The two metrics of importance are the baysian metric B and the probability metric P.

B(b1,b2)=|b1−b2|=∣∣ ∣∣log(p1(1−p2)p2(1−p1))∣∣ ∣∣P(p1,p2)=|p1−p2|=∣∣∣1ep1+1−1ep2+1∣∣∣

Suppose you have a prior, b1 in log odds, for some proposition. Suppose you update on some evidence that is twice as likely to appear if the proposition is true, to get a posterior, b2 in log odds. Then B(b1,b2)=log(2). The metric B measures how much evidence you need to move between probabilities.

Suppose you have a choice of actions, the first action will make an event of utility u happen with probability p1, the other will cause the probability of the event to be p2. How much should you care. uP(p1,p2).

The first metric stretches probabilities near 0 or 1 and is uniform in log odds. The second squashes all log odds with large absolute value together, and is uniform in probabilities. The first is used for baysian updates, the second for expected utility calculations.

Suppose an imperfect agent reasoned using a single metric, something in between these two. Some metric function less squashed up than P but more squashed than B around the ends. Suppose it crudely substituted this new metric into its reasoning processes whenever one of the other two metrics was required.

In decision theory problems, such an agent would rate small differences in probability as more important than they really were when facing probabilities near 0 or 1. From the inside, the difference between no chance and 0.01, would feel far larger than the distance between probabilities 0.46 and 0.47.

The Allais Paradox

However, the metric is more squashed than B, so moving from a 10000:1 odds to 1000:1 odds seems to require less evidence than moving from 10:1 to 1:1. When facing small probabilities, such an agent would perform larger baysian updates than really necessary, based on weak evidence.

Privileging the Hypothesis

As both of these behaviors correspond to known human biases, could humans be using only a single metric on probability space?



Discuss

Some Thoughts on Metaphilosophy

10 февраля, 2019 - 03:28
Published on February 10, 2019 12:28 AM UTC

A powerful AI (or human-AI civilization) guided by wrong philosophical ideas would likely cause astronomical (or beyond astronomical) waste. Solving metaphilosophy is one way that we can hope to avoid this kind of disaster. For my previous thoughts on this topic and further motivation see The Argument from Philosophical Difficulty, Metaphilosophical Mysteries, Three AI Safety Related Ideas, and Two Neglected Problems in Human-AI Safety.

Some interrelated ways of looking at philosophy Philosophy as answering confusing questions

This was my starting point for thinking about what philosophy is: it's what we do when we try to answer confusing questions, or questions that we don't have any other established methodology for answering. Why do we find some questions confusing, or lack methods for answering them? This leads to my next thought.

Philosophy as ability to generalize / handle distributional shifts

ML systems tend to have a lot of trouble dealing with distributional shifts. (It seems to be a root cause of many AI as well as human safety problems.) But humans seem to have some way of (sometimes) noticing out-of-distribution inputs, and can feel confused instead of just confidently use their existing training to respond to it. This is perhaps most obvious in unfamiliar ethical situations like Torture vs Dust Specks or trying to determine whether our moral circle should include things like insects and RL algorithms. Unlike ML algorithms that extrapolate in an essentially random way when given out-of-distribution inputs, humans can potentially generalize in a principled or correct way, by using philosophical reasoning.

Philosophy as slow but general purpose problem solving

Philosophy may even be a fully general purpose problem solving technique. At least we don't seem to have reason to think that it's not. The problem is that it's painfully slow and resource intensive. Individual humans acting alone seem to have little chance of achieving justifiably high confidence in many philosophical problems even if they devote their entire lives to those problems. Worse, humanity has been collectively trying to solve some philosophical problems for hundreds or even thousands of years, without arriving at final solutions. The slowness of philosophy explains why distributional shifts remain a safety problem for humans, even though we seemingly have a general way of handling them.

Philosophy as meta problem solving

Given that philosophy is extremely slow, it makes sense to use it to solve meta problems (i.e., finding faster ways to handle some class of problems) instead of object level problems. This is exactly what happened historically. Instead of using philosophy to solve individual scientific problems (natural philosophy) we use it to solve science as a methodological problem (philosophy of science). Instead of using philosophy to solve individual math problems, we use it to solve logic and philosophy of math. Instead of using philosophy to solve individual decision problems, we use it to solve decision theory. Instead of using philosophy to solve individual philosophical problems, we can try to use it to solve metaphilosophy.

Philosophy as "high computational complexity class"

If philosophy can solve any problem within a very large class, then it must have a "computational complexity class" that's as high as any given problem within that class. Computational complexity can be measured in various ways, such as time and space complexity (on various actual machines or models of computation), whether and how high a problem is in the polynomial hierarchy, etc. "Computational complexity" of human problems can also be measured in various ways, such as how long it would take to solve a given problem using a specific human, group of humans, or model of human organizations or civilization, or whether and how many rounds of DEBATE would be sufficient to solve that problem either theoretically (given infinite computing power) or in practice.

The point here is that no matter how we measure complexity, it seems likely that philosophy would have a "high computational complexity class" according to that measure.

Philosophy as interminable debate

The visible aspects of philosophy (as currently done by humans) seem to resemble an endless (both in clock time and in the number of rounds) game of debate, where people propose new ideas, arguments, counterarguments, counter-counterarguments, and so on, and at the same time to try judge proposed solutions based on these ideas and arguments. People sometimes complain about the interminable nature of philosophical discussions, but that now seems understandable if philosophy is a "high computational complexity" method of general purpose problem solving.

In a sense, philosophy is the opposite of math: whereas in math any debate can be settled by producing a proof (hence analogous to the complexity class NP) (in practice maybe a couple more rounds is needed of people finding or fixing flaws in the proof), potentially no fixed number of rounds of debate (or DEBATE) is enough to settle all philosophical problems.

Philosophy as Jürgen Schmidhuber's General TM

Unlike standard Turing Machines, a General TM or GTM may edit their previous outputs, and can be considered to solve a problem even if it never terminates, as long as it stops editing its output after a finite number of edits and the final output is the correct solution. So if a GTM solves a certain problem, you know that it will eventually converge to the right solution, but you may have no idea when, or if what's on its output tape at any given moment is the right solution. This seems a lot of like philosophy, where people can keep changing their minds (or adjust their credences) based on an endless stream of new ideas, arguments, counterarguments, and so on, and you never really know when you've arrived at a correct answer.

What to do until we solve metaphilosophy? Protect the trajectory?

What would you do if you had a GTM that could solve a bunch of really important problems, and that was the only method you had of solving them? You'd probably try to reverse-engineer it and make a bunch of copies. But if you couldn't do that, then you'd want to put layers and layers of protection around it. Applied to philosophy, this line of thought seems to lead to the familiar ideas of using global coordination (or a decisive strategic advantage) to stop technological progress, or having AIs derive their terminal goals from simulated humans who live in a safe virtual environment.

Replicate the trajectory with ML?

Another idea is to try to build a good enough approximation of the GTM by training ML on its observable behavior (including whatever work tapes you have read access to). But there are two problems with this: 1. This is really hard or impossible to do if the GTM has internal state that you can't observe. And 2. If you haven't already reverse engineered the GTM, there's no good way to know that you've built a good enough approximation, i.e., to know that the ML model won't end up converging to answers that are different from the GTM.

A three part model of philosophical reasoning

It may be easier to understand the difficulty of capturing philosophical reasoning with ML by considering a more concrete model. I suggest we can divide it into three parts as follows:

  • A. Propose new ideas/arguments/counterarguments/etc. according to some (implicit) distribution.
  • B. Evaluate existing ideas/arguments/counterarguments/etc.
  • C. Based on past ideas/arguments/counterarguments/etc., update some hidden state that changes how one does A and B.

It's tempting to think that building an approximation of B using ML perhaps isn't too difficult, and then we can just search for the "best" ideas/arguments/counterarguments/etc. using standard optimization algorithms (maybe with some safety precautions like trying to avoid adversarial examples for the learned model). There's some chance this could work out well, but without having a deeper understanding of metaphilosophy, I don't see how we can be confident that throwing out A and C won't lead to disaster, especially in the long run. (For example, if the order in which we think of / encounter arguments is important to the eventual conclusions we reach, then straightforwardly optimizing for the "best" arguments won't reproduce our trajectory. Or suppose C is what will allow us to eventually be able to think about a very wide class of ideas and arguments.) But A and C seem very hard or impossible for ML to capture (A due to paucity of training data, and C due to the unobservable state).

Is there a way around this difficulty? What else can we do in the absence of a full white-box solution to metaphilosophy?



Discuss

The Argument from Philosophical Difficulty

10 февраля, 2019 - 03:28
Published on February 10, 2019 12:28 AM UTC

(I'm reposting this comment as a top-level post, for ease of future reference. The context here is a discussion about the different lines of arguments for the importance of AI safety.)

Here's another argument that I've been pushing since the early days (apparently not very successfully since it didn't make it to this list :) which might be called "argument from philosophical difficulty". It appears that achieving a good long term future requires getting a lot of philosophical questions right that are hard for us to answer. Given this, initially I thought there are only three ways for AI to go right in this regard (assuming everything else goes well with the AI):

  1. We solve all the important philosophical problems ahead of time and program the solutions into the AI.
  2. We solve metaphilosophy (i.e., understand philosophical reasoning as well as we understand mathematical reasoning) and program that into the AI so it can solve philosophical problems on its own.
  3. We program the AI to learn philosophical reasoning from humans or use human simulations to solve philosophical problems.

Since then people have come up with a couple more scenarios (which did make me slightly more optimistic about this problem):

  1. We all coordinate to stop technological progress some time after AI but before space colonization, and have a period of long reflection where humans, maybe with help from AIs, spend thousands or millions of years to solve philosophical problems.
  2. We program AIs to be corrigible to their users, some users care about getting philosophy correct so the AIs help keep them safe and get their "fair share" of the universe until philosophical problems are solved eventually, enough users care about this so that we end up with a mostly good future, and lack of philosophical knowledge doesn't cause disaster in the meantime. (My writings on "human safety problems" were in part a response to this suggestion, outlining how hard it would be to keep humans "safe" in this scenario.)

The overall argument is that, given human safety problems, realistic competitive pressures, difficulties with coordination, etc., it seems hard to end up in any of these scenarios and not have something go wrong along the way. Maybe another way to put this is, given philosophical difficulties, the target we'd have to hit with AI is even smaller than it might otherwise appear.



Discuss

Dojo on stress

10 февраля, 2019 - 01:50
Published on February 9, 2019 10:49 PM UTC

Original post: http://bearlamp.com.au/dojo-on-stress/

This (thinking) dojo came about because someone described their biggest problem as stress, relating to recent job-change events.  I ran this dojo in Melbourne and Sydney to an audience of ~10 people each time. (45mins-1hr long)

As the facilitator:

  1. Say yes.  Accept what people bring.  There’s no wrong answer. Each person is bringing the most valuable thing for themselves to the discussion.  
  2. There is a need to balance the group time and let everyone talk if they want to, but generally people are aware of that.  If needed, thank someone and bring their awareness to the fact that this is a group event and everyone needs to participate to grow.  The exercise is not about being right but about sharing and discussing stress.
  3. Make empty space, (time when nothing is said) both for people to think, and for less talkative people to step up and share. Speak slowly, there is no rush.
  4. There is no need to force participation, feel free to mention that anyone can pass at any time.  It might be healthy to model “pass” behaviour at the start by getting everyone to say out loud “pass”.  “What do you say if you don’t want to contribute and you want to pass?” (“PASS” duh)
1. Share a stressful experience.

Each person should share a personal or significant experience of stress that they have encountered in their life.  It doesn’t have to still be “alive”, but it has to be personally relevant to them being engaged in the discussion and get an internal sense of what “stress” is and the sorts of things we are talking about.  (if doing this on your own, write down your experience, spend a few minutes waiting with the memories to get a sense of how it felt to be in the body during that experience, feel free to write more than one down)

(1-2mins per person)

Briefly check for a common theme.  I.e. stress caused by interpersonal relationships or work.  Be mindful of that when continuing the exercise.

2. Causes of stress

Stress usually has a cause in life.  Each person is different, each person will know their own common causes of stress.  Make a list of personal and common causes of stress.

(2 mins alone making a list, 5-10 mins to discuss the general possible causes of stress and build a group list)  (if alone, you can google it, but with emphasis that this exercise is not about getting the right answer, but more about being aware of the parts of the stress problem and playing around with them in mind in one session.)

2. What are the signs of stress.

How would you know someone else was stressed?  How would you know you were stressed? Make a list as a group discussion. There’s a short list at the bottom AFTER you have made your own list of the relevant signs.

(2 mins by timer by ourselves then 10mins for group discussion)  (if alone, spend more time making the list)

Get specific to name a few instances if it helps people participate.  “This one time I was stressed about X and I kept having nightmares”. etc.

3. what do you do about stress?  How do you relieve that stress?

get specific about how to wind down, how to rest, how to relax, feel safe, distract, and more. (list at the bottom AFTER the exercise to compare notes)

(2 mins by timer by ourselves then 10mins for group discussion)  (if alone, spend more time making the list)

Being Strategic

Once we know what causes stress, what stress looks like, and what to do about stress when it comes up, the last thing left to do is to be mindful.  Notice the causes, notice the signs that come up and act appropriately. As long as I am aware of my body, my behaviour and my actions, I can effectively manage my own stress and the stress of the people around me.

4. Anything else we want to share about stress?

How to tell someone else they are stressed without the words coming across like a slap in the face:

  • “I feel like you are stressed”
  • “I noticed you keep pacing, are you stressed?”  
  • (instead of, “you are stressed, stop that”)

Share any other personal stress stories or thoughts that come up from the exercise.  If we are done, go to the conclusion. (10 mins)

Conclusion

Reflect on if this is helpful personally.  How can I tie this into my life. How can I notice the stress?  How can I grow to use this information. Consider reminding myself in a month to check if I still do this.  Consider how I can plan a “stress check” into my weekly routine. Consider how I can make use of this information.  (3-5mins on our own doing what is needed to carry this out) (share any particularly good ones 5 mins)

The following lists are incomplete, they are here for clues, feel free to make your own or ignore these.

Causes of stress
  • Relationships
  • Work
  • Emotions
  • Food/diet
  • Exercise
  • Sleep
  • Mistakes, accidents
  • Emergencies
  • Surprises
  • Major events or life milestones
  • Big projects
  • Moving house/city
  • Family changes
Stress signs
  • Body based sensations (tightness, sweat, feeling heavy, doom, heart rate)
  • behaviour changes (posture changes, pacing, staying out late, sleep changes)
  • Expressions (face stuff, behaviour, I know I am confused from the confusion expression)
  • Emotions (aaaaah!, Sad, scared, flustered, etc)
  • Energy (lethargy, overactive)
  • Actions (eating more, injury)
  • dreams…  
What do you do about stress?
  • Take a bath
  • Distract myself
  • Leave the room
  • Meditate
  • Play video games
  • Sleep
  • Talk to someone about it
  • Pamper myself
  • Let go of trying to control everything and make sure it goes well
  • Stop doing the thing (sometimes an option)
  • Eat something
  • Concrete checks (have I eaten, drank, slept, got sunlight, spoken to friends) (you feel like shit guide)

Thanks for participating, feel free to get in touch with feedback. (google doc for comments: https://docs.google.com/document/d/1tme_NC3tusCXhwvEz_CaATx9nVp0IN4i-Pf4m0DVA2I/edit#



Discuss

When should we expect the education bubble to pop? How can we short it?

10 февраля, 2019 - 00:39
Published on February 9, 2019 9:39 PM UTC

I won't attempt to summarise the case for there being an education bubble here (see links below for some pointers). Rather, my questions are:

1) assuming there is an education bubble, when will it -- as bubbles tend to do -- pop?

(This plausibly entails some disjunction of *hundreds of thousands to millions of students defaulting on their debt, *lower number of college applicants, *non-top-tier colleges laying off faculty, *substantial reductions the signalling value of obtaining a diploma, *substantial reductions in tuition fees, *reduction in the level of education required by various employers, and more)

2) Which assets will be more scarce/in demand as that happens? Are there currently available opportunities for "shorting" the education bubble and invest in ways which will yield profit when it pops?

(I hereby preface the comments by noting that nothing discussed there is investment advice and no users can be held liable for investment decisions based on it.)

Peter Thiel summarises the inside view of there being an "education bubble" well.

And here are some interesting numbers:



Discuss

The Case for a Bigger Audience

9 февраля, 2019 - 10:22
Published on February 9, 2019 7:22 AM UTC

How do people feel like LW 2.0 is going? I'm impressed with the number and quality of posts that are being made, especially relative to the baseline of what LW 1.0 was like right before the relaunch. But I miss the lively discussions in the comments from the "Golden Age" of LW 1.0. Consider the Craft and Community sequence, written right when Eliezer transitioned from writing for Overcoming Bias to writing for Less Wrong. Here are six posts from that sequence which were especially memorable. On average, they have 179 comments. Looking at the "Curated Content" on the homepage right now, the 3 curated posts average only 19 comments, even though they've all been up for at least 2 months.

A big audience lets your posts to have a greater impact. (Have any posts from LW 2.0 generated new conceptual handles for the community like "the sanity waterline"? If not, maybe it's because they just aren't reaching a big enough audience.) Sometimes your audience generates interesting new ideas you hadn't thought of. And there's a virtuous cycle: People will write more comments if they have a justified expectation of comment readership and replies.

I think the biggest risk to LW 2.0 at this point might be that authors who invest in making posts find that they don't seem to be getting significant readership, making a significant impact, or generating useful feedback. There are a lot of people making posts right now, but there's a risk those people will drift away. If we can get the virtuous cycle going, that risk is lessened.

There's always the fear of Eternal September, but I think the rest of the internet has gotten more addictive since LW 1.0, so just being a website where longform essays are posted already gets you an audience that's selected for having a long attention span. And of course, countering Eternel September is a huge part of the motivation for the new voting system.

Some promotional ideas to consider:

  • Make use of the Less Wrong Facebook and Twitter feeds to highlight new content. (Is there still an RSS feed going? If anyone is still using RSS, it's probably the rationalist crowd.)
  • Send a one-time email to old LW 1.0 users with high karma who haven't logged in since the relaunch, announcing LW 2.0's launch and reminding them that their high karma puts them at the top of the heap.
  • Get Scott Aaronson to mention the fact that LW 2.0 is a real life instance of eigendemocracy in one of his "announcements" posts. The credit is his for inspiring the new voting system.
  • Back in the old days of LW, Louie Helm made use of a program where Google offered free Adwords credits to nonprofits in order to drive traffic to LW. I don't remember which keywords he used, but I can ask if you like.
  • Advertise on the sidebar of Scott Alexander's blog. I'm not sure whether he charges money to nonprofits which advertise or not.

Mods, let us know if you're looking for more promotional ideas and we can spend more time brainstorming.



Discuss

Can someone design this Google Sheets bug list template for me?

9 февраля, 2019 - 09:55
Published on February 9, 2019 6:55 AM UTC

Project:

My local rationality group is going to perform a structured test of the 'Hammertime' sequence. We are making a "bug list" of various problems in our lives for this sequence. (For those familiar, this variant of the bug list is loosely inspired by the FMEA) We assign, on a scale of 1-5, three different values to each bug: "Impact" (on our life), "Frequency" (of occurrence, or need of occurrence for things we need to do), and "How cheap is the cost" (in time/effort/money, where 1 is most expensive, and 5 is cheapest). The products of "Impact" and "Frequency" will all be totaled together, resulting in our "RPN" ("Risk Priority Number"), and this is the number we will be testing before and after performing the sequence, to measure its success.

Goal:

As I adjust the assigned values of each bug, eliminate bugs, and add new ones, I want the spreadsheet to calculate the product of the first two values, and auto-sort the list by this number, giving me a live priority list for addressing the bugs. I also want the ability to sort the list by the product of all three numbers, allowing me to optionally factor in the difficulty. This will result in 6 columns:

1-Name of Bug

2-Impact

3-Frequency

4-"Cheap Cost?"

5-Product of 2 and 3 (default auto-sorting)

6-Product of 2, 3, and 4 (optional sorting)

As well as a final row (or column, or whatever) with the total sum of Column 5's values (the "RPN").

Problem:

I suck at spreadsheets, having never learned them. I already made an attempt, but failed. I could put the time and effort into learning this, but I don't foresee using this skill again, so it seems more optimal to use someone else's skills for this.



Discuss

San Francisco: journal

9 февраля, 2019 - 06:50
Published on February 9, 2019 3:50 AM UTC

Lately I moved to San Francisco. More on that later perhaps. I have also been writing a journal. Here is some of it, somewhat edited, mostly not written with public consumption in mind*:

Jan 22

An interesting thing that happened to me today: where I would usually be very encumbered around cooking things—and if anything, follow instructions awkwardly—I had an inclination to buy zucchinis and ginger and then to slice them and fry them up, with some old wedges, and salt, and then a bit of cream and white wine.. I did the whole thing in a kind of ‘mmm mmmm…and now a little bit of ginger… ’ way, not a, ‘Ok, I will read the recipe again. It says slice the zucchinis. I suppose that means I should locate a knife. Ok. Knife. Wait, let me read the recipe again…’ kind of way. And it didn’t take that long, and I enjoyed it, and the food was goodish. (I also added a salad of parsley and radishes).

I put this in a class of things where I have awkwardly done the thing in the past, and heard that it was good, and checked boxes, and been diligent, but where there is a different mode where the thing is ‘alive’ for you, and that is so much better, and makes it just happen.

What does this mean? I had thought it was to do with the end state being actually motivating. Like, I run because it really feels like it is improving matters. But another possibility is that the process itself is somehow better connected to your System 1. For cooking, it’s not clear that I’m more excited about the outcome in the new mode, or at least that doesn’t feel like the relevant thing. I think the relevant thing is more like: there is an intuition for the system that has the goal and the steps leading to it in. So perhaps more broadly, a sense that your current actions will lead to the good goal.

That sounds right in the abstract, but I would have agreed with it before. And proceeded to dutifully envisage meditation helping, or slicing zucchinis leading to a stir fry, but I’m not sure this would have helped find the thing I’m now trying to point at. It’s more like a visceral deliciousness in each moment perhaps? I’m not sure, it’s hard to tell the difference between my mind now and at other times, on this thing.

Today I felt like there was some relation between the thing where something hard cuts through something weaker or vaguer (e.g. reality destroying your notions, adrenaline destroying your sense of vague unease, a task at hand destroying your malaise, a point of focus or a sharp sensation cutting through your sensory overwhelm), and the thing where you make a judgment call, or otherwise add an extra step of deciding, rather than waiting for the world (wait, what was the other thing like that I was thinking of recently, before I decided that it generalized to everything? Maybe feeling fear? Or something like that.) Or was it a relationship between two other important things? I can’t remember, and I have trouble thinking about this now. They seem superficially similar, but not actually. They both have the sense of a sharp clear thing cutting off a vague interminable blech thing.

23 January

The thing that was similar to making a judgment call was perhaps the act of deciding to believe in a thing, instead of waiting forever for your System 1 to come around. This is not a well understood move for me, and I don’t mean the thing where you just force yourself to act in spite of not feeling it. More like, in the dance between explicitness and intuition, you can perhaps make an explicit decision that causes your S1 to be on board, or asks it to look for how the thing is true in each moment, rather than waiting for it to come around eventually. Like if you decide that a thing is safe, you aren’t just feeling like it is dangerous and acting otherwise, the decision in part orients S1, or collapses its vagueness on a particular idea.

24 January

A friend wanted wrapping paper, and I let her look in my art box and use some art, or some origami paper. She finds it pretty and is excited, and wants to take some of the art herself. This is pleasing to me. I wish she would take it all. But what is that world that I want? One where she does things that I don’t find worth doing? Perhaps one where it is worth doing for her, or where it is worth doing, but I don’t understand the value, because I am somehow blind, or incapable?

February 4

Being myself has been been different these past days. There is an idea of doing things, instead of not. There is a feeling that is a cousin to the sense of violin strings being played, and a friend to substantiveness. There is a choosing of one thing to do each day. There is a lack of shame for showing where I am. There is a new relief of not having to be selfish. There is a desire to write on the whiteboard. There is real thought. There is doing so in public, freely and rightly. There is debate. There are words on the whiteboard. There is a sense that I can gather up the parts, myself, and bring them into myself. There is the idea that one should cut through the dallying with getting the window hangings right, and at last do. At first, do. That one can have not forgotten what one came here for. There was a confluence of themes: one of not getting drawn into appraising the side tables that Wirecutter drove you to; one of ideas that bloomed directly in your mind, versus locally spoken notions of AI risk and intervention; one of liking from direct awe-filled appreciation, and liking from identification of broadly understood merit. Of some things coming from oneself, and some not. Some things being first seen clearly, and others just being thrust dead into one’s basket, not being rejected but never coming to life. And now the sense that one can look, and look again, and have one’s world filled with new unfolding—new pockets of the territory lit, perhaps.

February 6

So, with these kinds of thoughts in mind, what should I do? Take the best actions… but one needs a process for doing that. To build a good-action taking system, one needs to somehow contend with when actions can be taken, and what the alternatives are, and a selection process. I think I should think more about this, when I have the energy for it. At the moment, my attention is elsewhere. The ability to direct attention seems perhaps very powerful. But also it sounds a bit misleading, because when my attention is elsewhere, it doesn’t feel like I want to redirect it fully—part of the problem is that I want something else.

***

*If you are reading this on LW and think that LW should not contain random personal journaling of no particular intellectual merit, the makers of LW seem to disagree, but I’m open to arguments that I shouldn’t put this here.



Discuss

Reinforcement Learning in the Iterated Amplification Framework

9 февраля, 2019 - 03:56
Published on February 9, 2019 12:56 AM UTC

When I think about Iterated Amplification (IA), I usually think of a version that uses imitation learning for distillation.

This is the version discussed in the Scalable agent alignment via reward modeling: a research direction, as "Imitating expert reasoning", in contrast to the proposed approach of "Recursive Reward Modelling". The approach works roughly as follows

1. Gather training data from experts on how to break problems into smaller pieces and combine the results

2. Train a model to imitate what the expert would do at every step

3. Amplification: Run a collaboration of a large number of copies of the learned model.

4. Distillation: Train a model to imitate what the collaboration did.

5. Repeat steps 3 and 4, increasing performance at every step

However, Paul has also talked about IA using reinforcement learning (RL) to maximize the approval of the amplified model. What does this approach (RL-IA) look like? How does it relate to Imitation-IA and Recursive Reward Modelling?

Puzzling about RL-IA

To get an agent that takes good actions in an Atari game, we use Imitation-IA to build a system that answers the question "how good is it to take actions from this state", then train a reinforcement learner to "output the best action to take from a given state".

But there it seems like the improvement stops there - it's not clear how "ability to output the best action to take from a given state" could improve "ability to evaluate how good actions are good from a state" in any way that's different from running a traditional reinforcement learning algorithm (which usually involves taking some policy/value estimate and gradually improving it).

Clarifying what RL-IA does

Claim: There is a fairly straightforward correspondence between how Imitation-IA and RL-IA perform a task (given no computational limits). RL-IA does not change the class of tasks that Imitation-IA can perform or perform them in a radically different way.

Suppose we have a current version of the model M1 that takes questions and produces a distribution over answers. Let M2 be an amplified version of that model (ie. produced by running a number of copies of M1). Let Y be some question, with domain of answers D. We want to find the answer X* that is the answer in D which maximizes the approval of amplified overseer, M2("How good is answer X to Y?"). Y could be

  • "What action is best to take from this state in this atari game?" where D is a small discrete set of possible actions
  • "What answer of less than 100 characters should I give to this question?" where D is a large discrete set of possible answers
  • "What answer of unbounded length should I give to this question?" where D is an infinite discrete set
  • "What is probability that event E will happen tomorrow?" where D is the continuous space of probabilities

An update using imitation learning would have the form:

  • X* = M1(Y)
  • For: number of samples
    • Sample an answer X from D
    • Evaluate M2("How good is answer X to Y?")
    • If M2("How good is answer X to Y?") > M2("How good is answer X* to Y?"), then set X* = X
  • Perform gradient descent to maximize the probability of outputting X*, using gradient .mjx-chtml {display: inline-block; line-height: 0; text-indent: 0; text-align: left; text-transform: none; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; word-wrap: normal; word-spacing: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0; min-height: 0; border: 0; margin: 0; padding: 1px 0} .MJXc-display {display: block; text-align: center; margin: 1em 0; padding: 0} .mjx-chtml[tabindex]:focus, body :focus .mjx-chtml[tabindex] {display: inline-table} .mjx-full-width {text-align: center; display: table-cell!important; width: 10000em} .mjx-math {display: inline-block; border-collapse: separate; border-spacing: 0} .mjx-math * {display: inline-block; -webkit-box-sizing: content-box!important; -moz-box-sizing: content-box!important; box-sizing: content-box!important; text-align: left} .mjx-numerator {display: block; text-align: center} .mjx-denominator {display: block; text-align: center} .MJXc-stacked {height: 0; position: relative} .MJXc-stacked > * {position: absolute} .MJXc-bevelled > * {display: inline-block} .mjx-stack {display: inline-block} .mjx-op {display: block} .mjx-under {display: table-cell} .mjx-over {display: block} .mjx-over > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-under > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-stack > .mjx-sup {display: block} .mjx-stack > .mjx-sub {display: block} .mjx-prestack > .mjx-presup {display: block} .mjx-prestack > .mjx-presub {display: block} .mjx-delim-h > .mjx-char {display: inline-block} .mjx-surd {vertical-align: top} .mjx-mphantom * {visibility: hidden} .mjx-merror {background-color: #FFFF88; color: #CC0000; border: 1px solid #CC0000; padding: 2px 3px; font-style: normal; font-size: 90%} .mjx-annotation-xml {line-height: normal} .mjx-menclose > svg {fill: none; stroke: currentColor} .mjx-mtr {display: table-row} .mjx-mlabeledtr {display: table-row} .mjx-mtd {display: table-cell; text-align: center} .mjx-label {display: table-row} .mjx-box {display: inline-block} .mjx-block {display: block} .mjx-span {display: inline} .mjx-char {display: block; white-space: pre} .mjx-itable {display: inline-table; width: auto} .mjx-row {display: table-row} .mjx-cell {display: table-cell} .mjx-table {display: table; width: 100%} .mjx-line {display: block; height: 0} .mjx-strut {width: 0; padding-top: 1em} .mjx-vsize {width: 0} .MJXc-space1 {margin-left: .167em} .MJXc-space2 {margin-left: .222em} .MJXc-space3 {margin-left: .278em} .mjx-test.mjx-test-display {display: table!important} .mjx-test.mjx-test-inline {display: inline!important; margin-right: -1px} .mjx-test.mjx-test-default {display: block!important; clear: both} .mjx-ex-box {display: inline-block!important; position: absolute; overflow: hidden; min-height: 0; max-height: none; padding: 0; border: 0; margin: 0; width: 1px; height: 60ex} .mjx-test-inline .mjx-left-box {display: inline-block; width: 0; float: left} .mjx-test-inline .mjx-right-box {display: inline-block; width: 0; float: right} .mjx-test-display .mjx-right-box {display: table-cell!important; width: 10000em!important; min-width: 0; max-width: none; padding: 0; border: 0; margin: 0} .MJXc-TeX-unknown-R {font-family: monospace; font-style: normal; font-weight: normal} .MJXc-TeX-unknown-I {font-family: monospace; font-style: italic; font-weight: normal} .MJXc-TeX-unknown-B {font-family: monospace; font-style: normal; font-weight: bold} .MJXc-TeX-unknown-BI {font-family: monospace; font-style: italic; font-weight: bold} .MJXc-TeX-ams-R {font-family: MJXc-TeX-ams-R,MJXc-TeX-ams-Rw} .MJXc-TeX-cal-B {font-family: MJXc-TeX-cal-B,MJXc-TeX-cal-Bx,MJXc-TeX-cal-Bw} .MJXc-TeX-frak-R {font-family: MJXc-TeX-frak-R,MJXc-TeX-frak-Rw} .MJXc-TeX-frak-B {font-family: MJXc-TeX-frak-B,MJXc-TeX-frak-Bx,MJXc-TeX-frak-Bw} .MJXc-TeX-math-BI {font-family: MJXc-TeX-math-BI,MJXc-TeX-math-BIx,MJXc-TeX-math-BIw} .MJXc-TeX-sans-R {font-family: MJXc-TeX-sans-R,MJXc-TeX-sans-Rw} .MJXc-TeX-sans-B {font-family: MJXc-TeX-sans-B,MJXc-TeX-sans-Bx,MJXc-TeX-sans-Bw} .MJXc-TeX-sans-I {font-family: MJXc-TeX-sans-I,MJXc-TeX-sans-Ix,MJXc-TeX-sans-Iw} .MJXc-TeX-script-R {font-family: MJXc-TeX-script-R,MJXc-TeX-script-Rw} .MJXc-TeX-type-R {font-family: MJXc-TeX-type-R,MJXc-TeX-type-Rw} .MJXc-TeX-cal-R {font-family: MJXc-TeX-cal-R,MJXc-TeX-cal-Rw} .MJXc-TeX-main-B {font-family: MJXc-TeX-main-B,MJXc-TeX-main-Bx,MJXc-TeX-main-Bw} .MJXc-TeX-main-I {font-family: MJXc-TeX-main-I,MJXc-TeX-main-Ix,MJXc-TeX-main-Iw} .MJXc-TeX-main-R {font-family: MJXc-TeX-main-R,MJXc-TeX-main-Rw} .MJXc-TeX-math-I {font-family: MJXc-TeX-math-I,MJXc-TeX-math-Ix,MJXc-TeX-math-Iw} .MJXc-TeX-size1-R {font-family: MJXc-TeX-size1-R,MJXc-TeX-size1-Rw} .MJXc-TeX-size2-R {font-family: MJXc-TeX-size2-R,MJXc-TeX-size2-Rw} .MJXc-TeX-size3-R {font-family: MJXc-TeX-size3-R,MJXc-TeX-size3-Rw} .MJXc-TeX-size4-R {font-family: MJXc-TeX-size4-R,MJXc-TeX-size4-Rw} .MJXc-TeX-vec-R {font-family: MJXc-TeX-vec-R,MJXc-TeX-vec-Rw} .MJXc-TeX-vec-B {font-family: MJXc-TeX-vec-B,MJXc-TeX-vec-Bx,MJXc-TeX-vec-Bw} @font-face {font-family: MJXc-TeX-ams-R; src: local('MathJax_AMS'), local('MathJax_AMS-Regular')} @font-face {font-family: MJXc-TeX-ams-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_AMS-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_AMS-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_AMS-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-B; src: local('MathJax_Caligraphic Bold'), local('MathJax_Caligraphic-Bold')} @font-face {font-family: MJXc-TeX-cal-Bx; src: local('MathJax_Caligraphic'); font-weight: bold} @font-face {font-family: MJXc-TeX-cal-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-R; src: local('MathJax_Fraktur'), local('MathJax_Fraktur-Regular')} @font-face {font-family: MJXc-TeX-frak-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-B; src: local('MathJax_Fraktur Bold'), local('MathJax_Fraktur-Bold')} @font-face {font-family: MJXc-TeX-frak-Bx; src: local('MathJax_Fraktur'); font-weight: bold} @font-face {font-family: MJXc-TeX-frak-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-BI; src: local('MathJax_Math BoldItalic'), local('MathJax_Math-BoldItalic')} @font-face {font-family: MJXc-TeX-math-BIx; src: local('MathJax_Math'); font-weight: bold; font-style: italic} @font-face {font-family: MJXc-TeX-math-BIw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-BoldItalic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-BoldItalic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-BoldItalic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-R; src: local('MathJax_SansSerif'), local('MathJax_SansSerif-Regular')} @font-face {font-family: MJXc-TeX-sans-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-B; src: local('MathJax_SansSerif Bold'), local('MathJax_SansSerif-Bold')} @font-face {font-family: MJXc-TeX-sans-Bx; src: local('MathJax_SansSerif'); font-weight: bold} @font-face {font-family: MJXc-TeX-sans-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-I; src: local('MathJax_SansSerif Italic'), local('MathJax_SansSerif-Italic')} @font-face {font-family: MJXc-TeX-sans-Ix; src: local('MathJax_SansSerif'); font-style: italic} @font-face {font-family: MJXc-TeX-sans-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-script-R; src: local('MathJax_Script'), local('MathJax_Script-Regular')} @font-face {font-family: MJXc-TeX-script-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Script-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Script-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Script-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-type-R; src: local('MathJax_Typewriter'), local('MathJax_Typewriter-Regular')} @font-face {font-family: MJXc-TeX-type-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Typewriter-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Typewriter-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Typewriter-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-R; src: local('MathJax_Caligraphic'), local('MathJax_Caligraphic-Regular')} @font-face {font-family: MJXc-TeX-cal-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-B; src: local('MathJax_Main Bold'), local('MathJax_Main-Bold')} @font-face {font-family: MJXc-TeX-main-Bx; src: local('MathJax_Main'); font-weight: bold} @font-face {font-family: MJXc-TeX-main-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-I; src: local('MathJax_Main Italic'), local('MathJax_Main-Italic')} @font-face {font-family: MJXc-TeX-main-Ix; src: local('MathJax_Main'); font-style: italic} @font-face {font-family: MJXc-TeX-main-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-R; src: local('MathJax_Main'), local('MathJax_Main-Regular')} @font-face {font-family: MJXc-TeX-main-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-I; src: local('MathJax_Math Italic'), local('MathJax_Math-Italic')} @font-face {font-family: MJXc-TeX-math-Ix; src: local('MathJax_Math'); font-style: italic} @font-face {font-family: MJXc-TeX-math-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size1-R; src: local('MathJax_Size1'), local('MathJax_Size1-Regular')} @font-face {font-family: MJXc-TeX-size1-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size1-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size1-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size1-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size2-R; src: local('MathJax_Size2'), local('MathJax_Size2-Regular')} @font-face {font-family: MJXc-TeX-size2-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size2-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size2-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size2-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size3-R; src: local('MathJax_Size3'), local('MathJax_Size3-Regular')} @font-face {font-family: MJXc-TeX-size3-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size3-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size3-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size3-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size4-R; src: local('MathJax_Size4'), local('MathJax_Size4-Regular')} @font-face {font-family: MJXc-TeX-size4-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size4-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size4-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size4-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-R; src: local('MathJax_Vector'), local('MathJax_Vector-Regular')} @font-face {font-family: MJXc-TeX-vec-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-B; src: local('MathJax_Vector Bold'), local('MathJax_Vector-Bold')} @font-face {font-family: MJXc-TeX-vec-Bx; src: local('MathJax_Vector'); font-weight: bold} @font-face {font-family: MJXc-TeX-vec-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Bold.otf') format('opentype')} ∇pM(X∗)

An update using the REINFORCE policy gradient estimator would have the form:

  • sample X from a stochastic policy M1(Y)
  • Perform gradient descent using gradientM2("How good is answer X to Y?")∗∇log(pM(X))

If we have a perfect distillation algorithm, these both converge to argmaxX(M2(X)) in the limit of infinite computation.

Practical Differences

Outside of this idealized situation, circumstances could make one or the other a better update to use.

The imitation update could converge more quickly if we have a good initialization for M(Y) from human data, as it bypasses the need to explore. It could also be less surprising, using only processes that the humans originally demonstrated.

The REINFORCE update could converge more quickly if the human initialization is suboptimal, or if it's hard to exactly reproduce the human demonstration.

In general, it seems like the system could use an algorithm that combines reinforcement learning updates with imitation learning updates, ie. Deep Q Learning from Demonstrations.

Returning to the original puzzle

I think the solution is not necessarily that "ability to output good actions at this timestep" translates into "ability to evaluate which actions are good"? Rather, I think that it is the case that the decomposition of "evaluate which actions are good" contains some questions which might perform a search over an answer space, and the answers to these questions are improved by reinforcement learning, and this improves the evaluation of atari actions. This can produce a model which uses a mix of imitation learning and reinforcement learning.

For example:

"What is a good action to take from state S?" could be learned to maximize "How good is it to take action A from this state S?"

"How good is it to take action A from this state S?" could be learned by imitating an amplified reasoner that asks the subquestion "What is the most useful information to provide about the consequences of action A from state S?"

"What is the most useful information to provide about the consequences of action A from state S?" could be learned to maximize "How useful is information I about the consequences of action A in state S?"

A modified version of the question, "How good is it to take action A from this state S, and include an explanation of your reasoning?" could also be reinforcement learned to maximize "How good is the explanation of how good it is to take action A in state S?"

Concluding Thoughts

Indeed, I think we could see every question answerable by an IA system in the form of "select the answer to question Y that the overseer approves most of", and use both demonstrations from the amplified reasoner and the amplified reasoner's evaluation to improve the answer. This perspective allows the system to learn to decompose problems better than original humans. But it might also cause problems if we can make a series of updates that cause the learned answering system to behave very differently from the original human demonstrators. We might want to be careful about the degree to which an RL learned policy can differ from the original demonstration.

In terms of getting a system to be capable of doing some task, I'd be most optimistic about systems that could combine RL-IA and Imitation-IA depending on the situation. But I still think there's usefulness in thinking about the pure Imitation-IA perspective to try and reason about the alignment properties of the system.

(Thanks to Andreas Stuhlmüller and Owain Evans for feedback on a draft of this post)



Discuss

HCH is not just Mechanical Turk

9 февраля, 2019 - 03:46
Published on February 9, 2019 12:46 AM UTC

HCH, introduced in Humans consulting HCH, is a computational model in which a human answers questions using questions answered by another human, which can call other humans, which can call other humans, and so on. Each step in the process consists of a human taking in a question, optionally asking one or more subquestions to other humans, and returning an answer based on those subquestions. HCH can be used as a model for what Iterated Amplification would be able to do in the limit of infinite compute. HCH can also be used to decompose the question of "is Iterated Amplification safe" into “is HCH safe” and “If HCH is safe, will Iterated Amplification approximate the behaviour of HCH in a way that is also safe”.

I think there's a way to interpret HCH in a way that leads to incorrect intuitions about why we would expect it to be safe. Here, I describe three models of how one could think HCH would work, and why we might expect them to be safe.

Mechanical Turk: The human Bob, is hired on Mechanical Turk to act as a component of HCH. Bob takes in some reasonable length natural language question, formulates subquestions to ask other Turkers, and turns the responses from those Turkers into an answer to the original question. Bob only sees the question he is asked and thinks for a short period of time before asking subquestions or returning an answer. The question of "is HCH corrigible" is about "how does the corrigibility of Bob translate into corrigibility of the overall system"? To claim that HCH is safe in this scenario, we could point to Bob being well-intentioned, having human-like concepts and reasoning in a human-like way. Also, since Bob has to communicate in natural language to other humans, those communications could be monitored or reflected upon. We could claim that this leads the reasoning that produces the answer to stay within the space of reasoning that humans use, and so more likely to reflect our values and less likely to yield unexpected outcomes that misinterpret our values.

Lookup Table: An AI safety research team lead by Alice writes down a set of 100 million possible queries that they claim capture all human reasoning. For each of these queries, they then write out the subquestions that would need to be written, along with simple computer code that combines the answers to the subquestions into an answer to the original question. This produces a large lookup table, and the "human" in HCH is just a call to this lookup table. The question of "is HCH corrigible" is about "has Alice's team successfully designed a set of rules that perform corrigible reasoning"? To justify this, we point to Alice's team having a large body of AI safety knowledge, proofs of properties of the system, demonstrations of the system working in practice, etc.

Overseer's Manual: An AI safety research team lead by Alice has written a manual on how to corrigibly answer questions by decomposing them into subquestions. This manual is handed to Bob, who was hired to decompose tasks. Bob carefully studies the manual and applies the rules in it when he is performing his task (and the quality of his work is monitored by the team). Alice's team has carefully thought about how to decomposed tasks, and performed many experiments with people like Bob trying to decompose tasks. So they understand the space of strategies and outputs that Bob will produce given the manual. The "human" in HCH is actually a human (Bob), but in effect Bob is acting as a compressed lookup table, and is only necessary because the lookup table is too large to write down. An analogy is that it would take too much space and time to write down a list of translations of all possible 10 word sentences from English to German, but it is possible to train humans who, given any 10 word English sentence can produce the German translation. The safety properties are caused by Alice's team's preparations, which include Alice's team modelling how Bob would produce answers after reading the manual. To justify the safety of the system, we again point to Alice's team having a large body of AI safety knowledge, proofs of properties of the system, demonstrations of the system working in practice etc.

I claim that the Mechanical Turk scenario is incomplete about why we might hope for an HCH system to be safe. Though it might be safer than a computation without human involvement, I would find it hard to trust that this system would continue to scale without running into problems, like handing over control deliberately or accidentally to some unsafe computational process. The Mechanical Turk scenario leaves out the process of design that Alice’s team takes part in the Lookup Table and Overseer’s Manual scenarios, which can include at least some consideration of AI safety issues (though how much of this is necessary is an open question). I think this design process, if done right, is the thing that could give the system the ability to avoid these problems as it scales. I think that we should keep these stronger Lookup Table and Overseer’s Manual scenarios in mind when considering whether HCH might be safe.

(Thanks to Andreas Stuhlmüller and Owain Evans for feedback on a draft of this post)



Discuss

The Hamming Question

8 февраля, 2019 - 22:34
Published on February 8, 2019 7:34 PM UTC

This is a stub post, mostly existing so people can easily link to a post explaining what the Hamming question is. For now, I am stealing the words from Jacobian's event post

If you would like to write a real version of this post, ping me and I'll arrange to give you edit rights to this stub.

Mathematician Richard Hamming used to ask scientists in other fields "What are the most important problems in your field?" partly so he could troll them by asking "Why aren't you working on them?" and partly because getting asked this question is really useful for focusing people's attention on what matters.

CFAR developed the technique of "Hamming Questions" as different prompts to get your brain to (actually) think about the biggest problems, bottlenecks, and unspoken desires in your life.



Discuss

Make an appointment with your saner self

8 февраля, 2019 - 08:05
Published on February 8, 2019 5:05 AM UTC

Most people have had the experience of being able to articulate advice that they themselves do not follow, even though it applies to their situation as well. Usually this implies that there's some sort of internal conflict present—a competing commitment that gets in the way of doing the thing that the person might consider reasonable. I have written much on transcending and untangling internal conflict (see these posts) and I will write much more.

But transcending internal conflict can be a lengthy, complex, and non-monotonic process, and in the meantime you're still sitting around with a bunch of great advice you're not taking. A bunch of untapped potential.

There's a really straightforward technique that can help with this:

make an appointment with your saner self.

Put an event on your calendar, and treat it with the respect you'd give any other appointment. Which is to say: show up. Or, if for some reason it turns out you can't, then reschedule for the nearest appropriate time.

Then, when the time comes, take your own advice. You can do this literally—consider what advice you'd give a friend in your situation, then do that—or you can just do the obvious thing. You can do this with specific object-level situations, eg "I need to get around to submitting that application" or with more abstract things like "I really should take more time to reflect on my life."

Or perhaps you've got a technique that you know really helps you, whenever you do it, but you never seem to do it. "If I actually used the CFAR techniques, my life would be way better," said almost every CFAR alumnus ever. Well, make an appointment with your saner self (the one who does the techniques) and then show up and do them.

Ways this can fail (and some suggestions)

Make sure you're clear on what the appointment is. It's okay to leave it open-ended when you make the appointment, but once the appointment starts, don't take more than 5 minutes to figure out how you're going to spend it. Or decide "I'm going to spend it prioritizing". The key is not to let the time slip by while you wonder what the best way to spend it would be. Which of course you probably know on some level. The point of this technique is to tap into what you already know about how you can have a better life.

If you don't have enough self-trust to show up for an appointment if there isn't someone else who'll be left stood-up, then make an appointment with someone else. Feel free to arrange this in the comments below. I've done this with strangers and also old friends I hadn't talked to in years (which was cool!). I recommend just trying a half-hour skype call, with a minute or two of "Hi, this is what I'm going to work on," then a 25-minute focused work period (aka "pomodoro") then a minute or two of "Here's how it went." Then if both of you want, you can continue for more pomodoros, but you're not committing up front to doing it for hours.

Even better, you can make a calendar where people can schedule such calls with you, using Calendly or youcanbook.me, share it with your friends, and then little sanity blocks will just automatically appear on your calendar. I did this for awhile and it was great. Each time a call occurred, I just asked "oh, what's some thing I've been putting off?" and I would get started on it.

If you don't have enough self-trust to show up for an appointment if there isn't someone else who'll be left stood-up, but you can't/won't schedule with someone else, then you could also try making a self-trust bet on this. Make sure to set a reminder so the thing doesn't just slip by forgotten.

If you don't have a calendar or any other system that you can rely on at all... get one? Assuming you have a smartphone, you can get it to bug you at a time. You then just need to (a) pick a time that you're likely to be interruptible, and (b) when the timer goes off, actually shift into doing whatever it was you set out to do.

Let's go meta: maybe you already knew about this sort of technique. Maybe you've done it before, or maybe you've suggested it to other people. Do you use it as much as you imagine would be optimal? If not, apply it to itself! Make an appointment right now with your saner self, and use the time to try to set up a regular event, or a youcanbook.me like I described above.

If the thing feels burdensome, then... this may not be the technique for you. You want to find a way of thinking about it so that you feel excited to spend time with (i.e. as) your saner self. If you can't find a way to feel excited or at least engaged about it, then it's not worth yelling at yourself about it. That defeats the point. Go read my post on self-referential motivation instead, and see if that helps.

Conceptual scaffolds and logistical scaffolds

The past few years, I've been part of a team hosting an event called the Goal-Crafting Intensive. It was a five-hour online workshop on setting your goals for the year. Ostensibly, the main value of the workshop was the instruction: presentations I made about goal-setting & planning, an 50+ page handbook, and chat-based coaching. Certainly, few people would have paid money for such an event if all three of those aspects had been absent.

And yet... I have a suspicion that the main value of the event was the fact that each participant carved out five hours from their schedule and then actually spent it focused on setting goals for the year.

Which is to say, if I imagine two people...

  • Allie buys the presentation videos and the handbook plus the ability to get some chat-based coaching.
  • Barry buys a ticket to "Open Goal Setting Afternoon" which is just a 5-hour solo-work context of some sort.

Who would have a more goal-directed year?

My money is on Barry.

Why? Our goal-setting content is actually quite good, but Allie would probably never actually open the handbook at all, let alone watch the videos. And even if she did, she would be likely to read it partway and then say, "Hmm yeah I really should do these exercises" ...but still not actually do them.

Whereas Barry, who only has his own advice to take, is at least taking the time to do the best he knows how to do.

And that's what counts. That's why even though the Goal-Crafting Intensive is 5 hours long, only about 10-15 minutes of each hour is presentations. Then I mute my microphone, to give each participant the rest of the hour to focus on whatever seems most important to them—which could be the technique I just described, or it could be something totally different!

We're running the Goal-Crafting Intensive again this year. So if you think your year could be improved by taking 5 hours to set some goals and design some systems, then come join us on Feb 23rd or 24th, and we'll give you both good advice and time to take it.

And I'd love to hear below more techniques for tapping into ones' more rational self.

(Crossposted from malcolmocean.com)



Discuss

Страницы