## Вы здесь

### Preferences as an (instinctive) stance

Новости LessWrong.com - 6 августа, 2019 - 03:43
Published on August 6, 2019 12:43 AM UTC

User Veedrac recently commented:

You have shown that simplicity cannot distinguish (p,R) from (−p,−R), but you have not shown that simplicity cannot distinguish a physical person optimizing competently for a good outcome from a physical person optimizing nega-competently for a bad outcome.

This goes to the heart of an important confusion:

• "Agent A has preferences R" is not a fact about the world. It is a stance about A, or an interpretation of A. A stance or an interpretation that we choose to take, for some purpose or reason.

Relevant for us humans is:

• We instinctively take a particular preference stance towards other humans; and humans tend to take the same stance towards others and towards each other. This makes the stance feel "natural" and intrinsic to the world, when it is not.
The intentional stance

Daniel Dennett defined the intentional stance as follows:

Here is how it works: first you decide to treat the object whose behavior is to be predicted as a rational agent; then you figure out what beliefs that agent ought to have, given its place in the world and its purpose. Then you figure out what desires it ought to have, on the same considerations, and finally you predict that this rational agent will act to further its goals in the light of its beliefs. A little practical reasoning from the chosen set of beliefs and desires will in most instances yield a decision about what the agent ought to do; that is what you predict the agent will do.

In the physical stance, we interpret something as being made of atoms and following the laws of physics. In the intentional stance, we see it as being an agent and following some goal. The first allows for good prediction of the paths of planets; the second, for the outcome of playing AlphaZero in a game of Go.

The preference/(ir)rationality stance What it the intentional stance for?

In a sense, the intentional stance is exactly the same as a preference stance. Dennett takes an object and treats it as an agent, and splits it into preference and rationality. Ok, he assumes that the agent is "rational", but allows for us to "figure out what what beliefs the agent ought to have." That, in practice, allows us to model a lot of irrationality if we want to. And I'm fully convinced that Dennett takes biases and other lapses of rationality into account when dealing with other humans.

So, in a sense, Dennett is already taking a preference/(ir)rationality stance[1] towards the object. And he is doing so for the express purpose of better predicting the behaviour of that object.

What is the preference stance for?

Unlike the intentional stance, the preference stance is not taken for the purpose of better predicting humans. It is instead taken for the purpose of figuring out what the human preferences are - so that we could maximise or satisfy them. The Occam's razor paper demonstrates that, from the point of view of Kolomogorov complexity, taking a good preference stance is not at all the same thing as taking a good (predictive) intentional stance.

But it often feels as if it is; we seem to predict people better when we assume, for example, that they have specific biases. Why is this, and how does it seem to get around the result?

Rationality stance vs empathy machine

There are two preference stances that it is easy for humans to take. The first is to assume that an object is a rational agent with a certain preference. Then we can try and predict which action or which outcome would satisfy that preference, and then expect that action/outcome. We do this often when modelling people in economics, or similar mass models of multiple people at once.

The second is to use the empathy machinery that evolution has developed for us, and model the object as being human. Applying this to the weather and the natural world, we anthropomorphised and created gods. Applying to other humans (and to ourselves) gives us quite decent predictive power.

I suspect this is what underlies Veedrac intuition. For if we apply our empathy machine to fellow humans, we get something that is far closer to a "goodness optimiser", albeit a biased one, than to an "badness nega-optimiser".

But this doesn't say that the first is more likely, or more true, about our fellow humans. It say that the easiest stance for us to take is to treat other humans in this way. And this is not helpful, unless we manage to get our empathy machine into an AI. That is part of the challenge.

And this brings us back to why the empathy machine seems to make better predictions about humans. Our own internal goals, the goals that we think we have on reflection, and how we expect people (including us) to behave given those goals... all of those coevolved. It seems that it was easier for evolution to use our internal goals (see here for what I mean by these) and our understanding of our own rationality, to do predictions. Rather than to run our goals and our predictions as two entirely separate processes.

That's why, when you use empathy to figure out someone's goals and rationality, this also allows you to better predict them. But this is a fact about you (and me), not about the world. Just as "Thor is angry" is actually much more complex than electromagnetism, our prediction of other people via our empathy machine is simpler for us to do - but is actually more complex for an agent that doesn't already have this empathy machinery to draw on.

So assuming everyone is rational is a simpler explanation of human behaviour than our empathy machinery - at least, for generic non-humans.

Or, to quote myself:

A superintelligent AI could have all the world’s video feeds, all of Wikipedia, all social science research, perfect predictions of human behaviour, be able to perfectly manipulate humans... And still conclude that humans are fully rational.

It would not be wrong.

1. I'll interchangeably call it a preference or an (ir)rationality stance, since given preferences, the (ir)rationality can be deduced from behaviour, and vice versa. ↩︎

Discuss

### [AN #61] AI policy and governance, from two people in the field

Новости LessWrong.com - 6 августа, 2019 - 01:59
Published on August 5, 2019 5:00 PM UTC

Find all Alignment Newsletter resources here. In particular, you can sign up, or look through this spreadsheet of all summaries that have ever been in the newsletter. I'm always happy to hear feedback; you can send it to me by replying to this email.

Highlights

The new 30-person research group in DC investigating how emerging technologies could affect national security (Rob Wiblin and Helen Toner): This 80,000 Hours podcast with Helen Toner dives into details of AI policy, China and the new Center for Security and Emerging Technology (CSET). I'm only summarizing the parts I found most relevant.

Many of the analogies for AI are quite broken. AI is a very broad set of software technologies, unlike nuclear weapons which are very discrete. It's not feasible to use export controls to keep "AI" within the US. In addition, AI will affect war far more fundamentally than just creating lethal autonomous weapons -- Helen thinks that the biggest military impact might be on logistics. It's also weird to compare data to oil, because oil is a rival good (two people can't use the same oil), whereas data can easily be copied. In addition, one barrel of oil can replace any other barrel, but data is very specific to the particular application. Helen's preferred analogy is thinking of AI as electricity -- a very general purpose tool that will transform lots of aspects of society. However, this analogy can also break down -- for example, the AI research community seems pretty important, but there was no analog for electricity.

And now for a few random points, in no particular order. China "exports" around 50,000 inventors (patent holders) every year, while the US imports 190,000, far more than any other country, suggesting that the US is a global hub for talent. AI is hard to define, because many of its properties lie on a continuum -- for example, is a landmine a lethal autonomous weapon? The way to affect policy is to make small, targeted changes in proposed policies so that the government makes slightly better decisions -- it's far too difficult to execute on a grand plan to get the government to do some big thing. The main skills for engaging with government on technology issues: be able to speak both to scientists as well as bureaucrats, and be able to navigate the DC setting -- knowing what people are doing, what their incentives are, and how to get your thing done given their different incentives.

Rohin's opinion: I enjoyed the section on how analogies for AI are broken -- I don't usually think much about them, but they always felt a bit off, and Helen makes it very clear what the issues are. It was also interesting seeing how the perspectives on AI are quite different from those of us thinking about AGI accident risk -- we often think about single, generally intelligent AGI systems, whereas Helen emphasized how current technologies can be easily deployed in many application-specific contexts. While data for current systems is very application-specific as Helen mentioned, if you believe the unsupervised learning story data may be more interchangeable for AGI systems.

AI Alignment Podcast: On the Governance of AI (Lucas Perry and Jade Leung): Jade makes a lot of points in this podcast, some of which I've summarized here in no particular order.

GovAI works on lots of research topics, including analysis of the inputs to AI, understanding historical cases of competition, looking at the relationship between firms and governments, and understanding public opinion.

Governance is particularly difficult because in the current competitive environment it's hard to implement any form of "ideal" governance; we can only make changes on the margin. As a result, it is probably better if we could get to a state where we could take a long time to deliberate about what ideal governance would look like, without having to worry about competitive pressures.

The biggest risk for governments is that they will make hasty, ill-informed regulation. However, given how uncertain we are, it's hard to recommend any concrete actions right now -- but governance will happen anyway; it won't wait for more research. One useful action we can take is to correct or add nuance to inaccurate memes and information, such as the "race" between the US and China, or the performance-safety tradeoff. Plausibly we should engage with government more -- we may have been biased towards working with private organizations because they are more nimble and familiar to us.

Instead of thinking about short term vs. long term, we should be thinking about the stakes. Some issues, such as privacy or job loss, can be thought of as "short term" but their stakes could scale to be huge in the long term. Those would be good areas to think about.

Rohin's opinion: I don't have any particular thoughts on these topics, but I am glad for both this and the previous podcast, which give more of a birds-eye view of the AI governance landscape, which is hard to get from any single paper.

Technical AI alignmentTechnical agendas and prioritization

On the purposes of decision theory research (Wei Dai): In this post, Wei Dai clarifies that he thinks decision theory research is important because it can help us learn about the nature of rationality, philosophy, and metaphilosophy; it allows us to understand potential AI failure modes; we can better understand puzzles about intelligence such as free will, logical uncertainty, counterfactuals and more; and it could improve human rationality. It is not meant to find the "correct" decision theory to program into an AI, nor to create safety arguments that show that an AI system is free of "decision-theoretic" flaws.

Bridging Hamilton-Jacobi Safety Analysis and Reinforcement Learning (Jaime F. Fisac, Neil F. Lugovoy et al): Reinforcement learning is not great at enforcing constraints that hold at all times, because the agent would violate a constraint now if it would lead to higher reward later. In robust optimal control theory, we maximize the minimum of the constraint reward over time to avoid this. We can do this in the Bellman equation by taking a minimum between the current reward and estimated future value (instead of summing), but this does not uniquely define a fixed point. Just as in regular RL, we can use discounting to avoid the problem: in particular, if we interpret the discount as the probability that the episode continues, we can derive a Safety Bellman equation for which Q-learning is guaranteed to converge. They demonstrate their method in classic control environments as well as half-cheetah, with a range of RL algorithms including soft actor-critic (SAC).

Rohin's opinion: I really like how simple the change is here -- it should be a one-line change for many deep RL algorithms. Previously, we had to choose between unconstrained agents for high dimensional problems, or constrained agents for low dimensional problems -- I like that this work is making progress on constrained agents for high dimensional problems, similarly to Constrained Policy Optimization. While this work doesn't involve a performance reward, you could use the resulting safe policy in order to guide a process of safe exploration to learn a policy that safely optimizes a performance metric. Of course, this is all assuming a specification for the constraint to satisfy.

Miscellaneous (Alignment)

Modeling AGI Safety Frameworks with Causal Influence Diagrams (Tom Everitt, Ramana Kumar, Victoria Krakovna et al): This paper describes several AI safety frameworks using the language of causal influence diagrams (AN #49), in order to make it easy to compare and contrast them. For example, the diagrams make it clear that while Cooperative IRL and reward modeling (AN #34) are very similar, there are significant differences: in cooperative IRL, the rewards come directly from the underlying human preferences, whereas in reward modeling, the rewards come from a reward model that depends on human feedback, which itself depends on the underlying human preferences.

Rohin's opinion: I like these diagrams as a way to demonstrate the basics of what's going on in various AI safety frameworks. Sometimes the diagrams can also show the differences in safety features of frameworks. For example, in reward modeling, the agent has an incentive to affect the human feedback in order to affect the reward model directly. (Imagine getting the human hooked on heroin, so that future feedback causes the reward model to reward heroin, which could be easy to produce.) On the other hand, in cooperative IRL, the agent only wants to affect the human actions inasmuch as the actions affect the state, which is a normal or allowed incentive. (Imagine the agent causing the human to leave their house earlier so that they get to their meeting on time.)

AI strategy and policy

Information security careers for GCR reduction (Claire Zabel and Luke Muehlhauser): This post suggests that information security could be a good career path for people looking to reduce global catastrophic risks (GCRs). For AI in particular, such experts could help mitigate attacks by malicious or incautious actors to steal AI-related intellectual property. It also reduces the risk of destabilizing AI technology races. Separately, such experts could think about the potentially transformative impact of AI on cyber offense and defense, develop or advise on credible commitment techniques (see eg. model governance (AN #38)), or apply the security mindset more broadly.

An Interview with Ben Garfinkel (Joshua Monrad, Mojmír Stehlík and Ben Garfinkel): AI seems poised to be a very big deal, possibly through the development of AGI, and it's very hard to forecast what would happen next. However, looking at history, we can see a few very large trajectory shifts, such as the Agricultural Revolution and Industrial Revolution, where everything changed radically. We shouldn't assume that such change must be for the better. Even though it's hard to predict what will happen, we can still do work that seems robustly good regardless of the specific long-term risk. For example, Ben is optimistic about research into avoiding adversarial dynamics between different groups invested in AI, research into how groups can make credible commitments, and better forecasting. However, credible commitments are probably less tractable for AI than with nukes or biological weapons because AI systems don't leave a large physical footprint, can easily proliferate, and are not a clear category that can be easily defined.

Other progress in AIExploration

Self-Supervised Exploration via Disagreement (Deepak Pathak, Dhiraj Gandhi et al) (summarized by Cody): For researchers who want to build a reinforcement learning system that can learn to explore its environment without explicit rewards, a common approach is to have the agent learn a model of the world, and incentivize it to explore places where its model has the highest error, under the theory that these represent places where it needs to interact more to collect more data and improve its world model. However, this approach suffers in cases when the environment is inherently stochastic, since in a stochastic environment (think: sitting in front of a static TV and trying to predict the next frame), prediction error can never be brought to zero, and the agent will keep interacting even when its world model has collected enough data to converge as much as it can. This paper proposes an alternative technique: instead of exploring in response to prediction error, learn an ensemble of bootstrapped next-state prediction models and explore in response to variance or disagreement between the models. This has a few nice properties. One is that, in cases of inherent stochasticity, all models will eventually converge to predicting the mean of the stochastic distribution, and so even though they've not brought error down to zero, the variance among models will be low, and will correctly incentivize our agent to not spend more time trying to learn. Another benefit is that since the reward is purely a function of the agent's models, it can be expressed analytically as a function of the agent's choices and trained via direct backpropogation rather than "black box reward" RL, making it more efficient.

Cody's opinion: I found this approach really elegant and clever as a way of addressing the "static TV" problem in curiosity literature. I'd be curious to see more work that introduces even stronger incentives towards diversity among the ensemble models (different architectures, even more different datasets they're trained on), to see if that amplifies the cases of model disagreement.

Deep learning

Weight Agnostic Neural Networks (Adam Gaier et al) (summarized by Cody): Inspired by the ability of animals to perform some tasks at birth, before learning about the world, this paper tries to find network architectures that perform well over a wide range of possible model parameters. The idea here is that if an architecture performs well with different sampled weights and without training to update those weights, then the architecture itself is what's responsible for encoding the solution, rather than any particular weight configuration. The authors look for such architectures on both classification and reinforcement learning problems by employing NEAT, a evolutionary method from Neural Architecture Search that searches for the best-performing topologies within the space of possible node connections and activations. The authors find that they're able to construct architectures that do better than random on their test problems without training weights explicitly.

Cody's opinion: I appreciate the premise of this paper, and in general feel positively towards papers that delve into a better understanding of how much of modern neural network performance is attributable to (discrete) structural architectures vs particular settings of continuous weight parameters, and I think this paper does that in a clever way by essentially marginalizing over different weight values. The framing of this paper, implicitly comparing networks used without weight training to animals with innate abilities, did make me wonder whether the architecture vs weights analogy to evolution vs learning is a sound one. Because, while it's true that the weights weren't explicitly gradient-descent trained in this paper, the network did still perform optimization based on task performance, just over a set of discrete parameters rather than continuous ones. In that context, it doesn't really seem correct to consider the resulting architectures "untrained" in a way that I think that analogy would suggest. I'd be curious to see more work in this direction that blends in ideas from meta-learning, and tries to find architectures that perform well on multiple tasks, rather than just one.

Hierarchical RL

Unsupervised Discovery of Decision States for Transfer in Reinforcement Learning (Nirbhay Modhe et al)

Miscellaneous (AI)

Explainable AI, Sparse Representations, and Signals: So far, we have built AI systems that store knowledge symbolically or in a distributed fashion (with neural nets being the latter). While the distributed form allows us to learn knowledge and rules automatically, it is much harder to understand and interpret than symbolically represented knowledge. This post argues that the main difference is in the sparsity of the learned knowledge. Of course, with more "sparse" knowledge, it should be easier for us to understand the internal workings of the AI system, since we can ignore the pruned connections. However, the author also argues that sparse knowledge will help 'guide the search for models and agents that can be said to "learn" but also "reason"'. Given that AGI will likely involve finding good representations for the world (in the sense of unsupervised learning), then sparse learning can be thought of as a bias towards finding better bases for world models, that are more likely to be conceptually clean and more in line with Occam's razor.

In a postscript, the author considers arguments for AI risk. Notably, there isn't any consideration of goal-directedness or alignment failures; the worry is that we will start applying superhuman AI systems to superhuman tasks, and we won't know how to deal with these situations.

Rohin's opinion: Sparsity seems like a good objective to shoot for in order to ensure explainability. I'm less convinced that it's worthwhile for representation learning: I doubt humans have any sort of "sparse learning" bias; I think sparsity of knowledge is a natural consequence of having to understand a very complex world with a very small brain. (Whereas current ML systems only have to understand much simpler environments.)

News

Microsoft invests in and partners with OpenAI to support us building beneficial AGI (Greg Brockman): After moving to a capped-profit investment model (AN #52), Microsoft has invested $1 billion in OpenAI. This allows OpenAI to keep their focus on developing and sharing beneficial AGI: instead of having to create a product to cover costs, they can license their pre-AGI technologies, likely through Microsoft. Research Associate in Paradigms of Artificial General Intelligence and Their Associated Risk (José Hernández-Orallo): CSER is hiring a post-doctoral research assistant to inform the AGI safety agenda by looking at existing and possible kinds of agents; the deadline is August 26. Discuss ### How to navigate through contradictory (health/fitness) advice Новости LessWrong.com - 5 августа, 2019 - 23:58 Published on August 5, 2019 8:58 PM UTC I will start with a brief story, but the question can be generalized. Last year, I decided to do something for my body. I joined and regularly went to K. Training (abbreviated name), a large gym chain in german-speaking countries. The claimed philosophy is different from many gyms: there is no music, no proteine shakes to buy, mostly old people around, and insistence that it is about strength, not show-off, and that strength is what keeps your spine together etc. They have no cardio bikes, no barbell, only machines, and the high-intensity approach is that at each machine you do one continuous exercise for two minutes. If you reach the two minutes, increase the weight next time. It all seems very serious, there is an orthopedist you talk to when you become a member. It all has been in existence for some decades. The founder writes books, of course mentioning that his approach is the only one that works against pain, and that he is not heard by the mainstream. While at the same time they have contracts with many orthopedists and this is part of the marketing. Now, a back problem. I have seen several orthopedists in my life, but the one I talked to this year (after two GPs, both clueless) is the first who seems competent and also listens. His comment about K. Training: it's ok, but sometimes hard to leave the contract. You could just as well try Yoga or Pilates. Anyways, he gives me a prescription for physical therapy. Talking about this and that, the therapist speaks out against K Training, because no warming up / cardio (something the founder explicitly defends in his books), and Yoga/Pilates/etc is better anyways. Then I googled again. Seemingly, gym experts all have their own approach. Some agree to the high-intensity two minutes thing, others disagree. Then there is also Mr. L.-B., an anti-pain guru with a somewhat different approach I dont really understand, again against the "mainstream" but also against K. Training. And from a lecture of his that I watched on youtube, he seems like a snake-oil seller; but then, he (of course) has many fans. Now I could just randomize what to do; or try to really read about approaches, but ALL of them seem plausible, if you listen to them. The investment necessary for actual judgement would be studying medicine. So long story, short question: how do you actually handle such cases of pratically relevant epistemic learned helplessness? Discuss ### My recommendations for gratitude exercises Новости LessWrong.com - 5 августа, 2019 - 22:30 Published on August 5, 2019 7:04 PM UTC Gratitude has become an increasingly important part of my life. It has also been one of my greatest sources of improvement of well-being and one of the biggest factors in lifting me out of depression. How does this work? The short answer is that I keep a gratitude journal. The rest of this post is the long answer. I think there are some theoretical reasons why we should expect gratitude to be helpful or extremely helpful. Try to imagine a time when you were deprived of something that you now have. For example, try to imagine a time when you misplaced your wallet, phone, or passport only to later realize where it was. Think of the sense of relief you got from this. Now recognize that you could feel that way now about all of the things that you have that you could have lost. The hedonic treadmill refers to the phenomenon of us quickly getting accustomed to any new improvements that we’ve made so that we have to keep running to stay in the same place and maintain our happiness. I think one of the ways that the hedonic treadmill works is by us almost immediately taking everything for granted. If we can stop this process to some extent, through gratitude exercises, we might be able to make large improvements to our well-being. In my own case, this is particularly vivid. Some years ago I had very bad repetitive strain injuries and associated chronic pain. I did not know if I would ever be able to work again or do many other normal things with my arms. The prospect of improvement seemed dim and my life seemed to be utterly ruined. It seemed to me that if I could only get the use of my arms back, life would be perfect. At that time I thought to myself if things do you ever get better, if there is anything positive I can draw from this piece of hell, it is to remember that feeling, so that if I recover, I can always feel that my life is perfect. You might be able to leverage past tragedies in your life in this way as well. You might be able to turn that darkness into light. I still have some trouble with repetitive strain injuries and chronic pain, but my situation is now vastly improved. Do I feel perfect now? Well no, it’s hard to fight the hedonic treadmill, but I do feel a lot better because of gratitude exercises. I think one mistake people make when it comes to gratitude is thinking too small. While it’s helpful to feel grateful for a lot of different things, and I do write down small things in my gratitude journal, there are lots of big things that we could feel grateful for. We don’t feel grateful for these things because we’ve become accustomed to having them and thoroughly take them for granted. It can take some extra work to feel grateful for these things, but it’s worth it. Here is a short, and by no means exhaustive, list of some of the things you might want to try feeling grateful for: being alive at all, being alive at this time in history, having loved ones who are alive, being born a human, having functional limbs, being able to make a difference in the world, having access to godlike technology, and having access to a wider range of media for free than any library could hold. To feel grateful for some of these things you might have to try to vividly imagine being without them for a time. If you are deprived of some of these things for a time (or temporarily believe you are) you can also try to remember what that feels like, so that you can recapture it later when you have them again. The idea with these techniques is to help them become ingrained as habits. It is to train your mind to see more of the good things that you have and naturally feel more gratitude for them. You should also expect to feel good while doing the gratitude exercise and this should help reinforce the habit. I found this technique to be less effective when I’m feeling quite bad. However, I think practising this technique has made me feel bad less often. Of course, we could instead imagine things as they could be in some hypothetical utopian society 100 years from now when most forms of suffering are unknown. We could then make ourselves feel bad because things are so much worse than they would be in that society. I don’t think it makes sense to say that any of these comparisons are more correct or meaningful than any other. The only thing we can say is that some of these comparisons are more useful than others. Making comparisons that allow us to feel grateful can be useful in improving our lives. One fear I had when starting this practice was that feeling gratitude would lead to complacency. However, I think that with some care this can be avoided and we can draw from the practice in order to be more effectively altruistic. If we have the ability to more effectively control and improve our well-being through our own thoughts, without having to spend expensive resources on it, we can allow ourselves to contribute more energy and resources to improving the lives of others. This technique may also point to a way in which we can help others without expending too many resources, since it is an inexpensive means of improving mental health that can be taught. I think this technique may also be helpful in allowing us to reflect on the suffering of the world without being overwhelmed with grief about it, which allows us to be more motivated to improve it. Part of the practice is reflecting on the suffering, but feeling grateful that we are not going through this suffering now allows us to turn this darkness into light. People waste a huge amount of resources pursuing ever smaller amounts of happiness as they climb the social ladder. Gratitude promises to be a way of achieving this that doesn’t involve wasting these resources, which could be vastly more useful in improving the well-being of the less fortunate. Some people might find comparisons between other people to be insensitive or in bad taste. If this is the case for you, you can instead reflect on ‘different hypothetical versions of yourself’ in different states of deprivation. I do think the process can be done in an inoffensive way—you just have to have the right intentions and be tactful. Certainly there are bad ways of doing this, such as if you use the comparison to fuel a sense of superiority or if you use it to ‘lord’ what you have over others. The perspective I try to approach this from is one of solidarity with all other sentient beings. We may be lucky enough to have many more resources than others and if so should draw whatever we can from those resources to help others. I sometimes feel that using gratitude in this way is too ‘Pollyanna’ or too ‘sunshine and rainbows.’ If you feel this way, I suggest considering which life feels more lucid and clear eyed to you—one where you are preoccupied with minor details, like the last person who cut you off in traffic, or one where you are keenly aware of all that you have and all that could be taken away from you. I haven’t looked deeply at the empirical literature on the subject. I suspect that the method does have more promise than Bruce’sis indicated by the studies, because I suspect that many people given the task of gratitude journaling in studies may not be doing so as effectively as they could be. The tips I give in this post should be an improvement on that. In practice it will still probably not be a magic bullet or panacea, but I think the method holds a lot of promise. It’s possible that gratitude isn’t the word I should be using in this post. Appreciation might be a better word. The word gratitude carries at least a subtle suggestion that there is someone responsible and that person deserves praise, and this isn’t necessarily the case. In particular, God doesn’t exist, and if he did exist, I don’t think he would deserve our praise. Still, gratitude is the word that usually gets used in this context and it is emotionally punchier than ‘appreciation,’ so I’ve decided to keep using it. Discuss ### DC SSC Meetup Новости LessWrong.com - 5 августа, 2019 - 19:19 Published on August 5, 2019 4:19 PM UTC SSC meetup this Saturday, August 10th. Discuss ### DC SSC Meetup Новости LessWrong.com - 5 августа, 2019 - 19:16 Published on August 5, 2019 4:16 PM UTC Meetup this Saturday, August 10th. Discuss ### Do you do weekly or daily reviews? What are they like? Новости LessWrong.com - 5 августа, 2019 - 04:23 Published on August 5, 2019 1:23 AM UTC • What system(s) do you use to keep yourself organized and working toward your goals? I'm interested in technologies, but more interested in what ontology you use to organize your tasks, events, goals, and plans. • Did you build your system slowly over time, or adopt it all at once? • What feels most important about your system to you? Discuss ### Can we really prevent all warming for less than 10B$ with the mostly side-effect free geoengineering technique of Marine Cloud Brightening?

Новости LessWrong.com - 5 августа, 2019 - 03:12
Published on August 5, 2019 12:12 AM UTC

If we're to believe the Philosophical Transactions of the Royal Society, or the Copenhagen Consensus Center, or apparently any of the individual geoengineering researchers who've modelled it, it's possible to halt all warming by building a fleet of autonomous wind-powered platforms that do nothing more sinister than spraying seawater into the air, in a place no more ecologically sensitive than the open ocean, and for no greater cost than 10 billion USD.

If this works, no significant warming will be allowed to occur after the political static friction that opposes the use of geoengineering is broken.

Isn't any amount of mean warming bad? So shouldn't we deploy something like this as soon as possible? Shouldn't we have started deploying it years ago?

The only clear side effect I've seen mentioned is a potential reduction in rainfall in south america, but one analysis found that this could be avoided by simply not doing any brightening near the land in that region.

I want to make it clear how little support this needs in order to get done: 10 billion USD is less than a quarter of the tax income generated by the nation of just New Zealand one year. A single tiny oecd government could, in theory, do it alone. It wont need the support of a majority of the US. It probably wont require any support from the US at all.

What should we do with this information?

I do buy the claim that public support for any sort of emission control will evaporate the moment geoengineering is realised as a tolerable alternative. Once the public believe, there will never be a quorum of voters willing to sacrifice anything of their own to reduce emissions. I think we may need to start talking about it anyway, at this point. Major emitters have already signalled a clear lack of any real will to change. The humans will not repent. Move on. Stop waiting for humanity to be punished for its sin, act, do something that has some chance of solving the problem.

Could the taboos against discussing geoengineering delay the discovery of better, less risky techniques?

Could failing to invest in geoengineering research ultimately lead to the deployment of relatively crude approaches with costly side effects? (Even if cloud brightening is the ultimate solution to warming, we still need to address ocean acidification and carbon sequestration, and I'm not aware of any ideal solution to those problems yet, but two weeks ago I wasn't aware of cloud brightening, so for all I know the problem isn't a lack of investment, might just be a lack of policy discussion.)

Public funding for geoengineering research is currently non-existent in the US (All research in the US is either privately funded or funded by universities), and weak in China (3M USD. 14 members, no tech development, no outdoor experiments.)

Discuss

### [Resource Request] What's the sequence post which explains you should continue to believe things about a particle moving that's moving beyond your ability to observe it?

Новости LessWrong.com - 5 августа, 2019 - 01:31
Published on August 4, 2019 10:31 PM UTC

I think it's a good and legitimate use of the question system to ask people to help you locate posts you can't quite remember where it is, or request posts relevant to a topic you're interested in. If there are ever "too many" such questions, I'm sure we can find a suitable way to "filter" them to be seen only in suitable places.

The posts is something about a particle moving at the speed of light and that you should continue to believe in your models even though you can't observe it any more: moving at the speed of light + expanding universe (?).

Discuss

### AI Alignment Open Thread August 2019

Новости LessWrong.com - 5 августа, 2019 - 01:09
Published on August 4, 2019 10:09 PM UTC

This is an experiment in having an Open Thread dedicated to AI Alignment discussion, hopefully enabling researchers and upcoming researchers to ask small questions they are confused about, share very early stage ideas and have lower-key discussions.

Discuss

### Is there a user's manual to using the internet more efficiently?

Новости LessWrong.com - 5 августа, 2019 - 00:19
Published on August 4, 2019 6:51 PM UTC

I'd like to condition the responses by elaborating on what I currently do and why I think having a body of work to reference would be beneficial.

This all started a couple months ago when I realized that most of internet usage revolved around Reddit/Quora + Wikipedia + arXiv. I use the internet to have questions answered, to reference authoritative information, and to explore developments in various research interests.

This equilibrium was disturbed by the invention of GPT-2. I started a project of writing a journal in conjunction with this "super"-autocomplete algorithm and realized that part of the reason I thought it sounded so great was because I was prompting it with things you might find on Reddit/Wikipedia/arXiv. This is kind of hard to explain, basically I awoke to this gestalt that the way I was thinking was being affected by the content I was consuming. Obvious in hindsight, but at the time quite a shocker.

I researched some ways to start curating my own content and found Pocket. After that, things started taking off. I currently use a combination of Pocket/GPT-2/Google to manage my curation of internet content. To constrain information overload, I generally use a question -> hypothesis -> research/experiment workflow. Sometimes getting the right question is hard so I'll use GPT-2 to try and "super"-autocomplete my way into a phrase that has potential. After that I try to google the question/phrase that popped in the first step. However, GPT-2 really has been sending me all over the internet so it's quite difficult to relate things together or to evaluate the quality of the information I'm receiving.

I think having some sort of book organizing the different "dimensions" of internet usage would be useful. Being able to have a tangible organization layout of the available tools would help me select ones that useful for whatever I'm interested in. At the moment I have only a few; search, save, "super"-autocomplete. Any and all references are useful, but the more in depth the better. Thanks!

Discuss

### The Planning Problem

Новости LessWrong.com - 5 августа, 2019 - 00:15
Published on August 4, 2019 6:58 PM UTC

(Originally posted on my blog here. Not sure how mathematical this place is, but it was seamless to crosspost so I'll continue doing that and see what happens!)

It's fair to say that we spend a lot of our lives floating from task to task. Some are work and some are not. Nevertheless, they are tasks. Things that we want to do. Some are complicated are require planning. However, since planning is such an integral step in completing more complicated tasks it seems as though we might want to "meta-plan" or plan our planning. I became fascinated by the concept after I realized it should be possible to frame the whole thing as a sort of optimization problem. In practice, the problem remains complicated by the fact that optimization occurs in a necessarily online fashion. However, any reasonable method of planning is especially well behaved in the formalism I'll present. Moreover, even when the method is faulty due to human bias, as long as it is consistent we still end up being to prove that meta-planning, in a certain sense, is rational.

One particularly recursive aspect of the problem is that to carry out a task to completion we must already have a plan that we execute. The plan could be written in an ‘online’ manner, which is done in improvisation. Still, the line between planning and doing at this point becomes very blurry and quickly leads to a chicken/egg problem. Which came first, knowledge of a plan or how to execute it? I’ll bound the scope of this using a few concrete examples and then we’ll jump off into abstraction to step around the recursion.

A First Example

The intuition seems to be that there is a strategy that can balance the objectives of planning and executing optimally. However, over the years I’ve caught myself making the error of measuring the effectiveness of having a plan or strategy in terms of how much time it saves me in carrying out the task. Yet, I will commonly ignore the time spent on creating a strategy in the first place. For example, while I was writing this article I spent a significant amount of time crafting a uniqueness proof for the optimal strategy I’ll propose in the next section. Was that optimal? I had a picture of the proof done in under an hour. If you’d asked me right after I’d proved the theorem whether or not it was a good use of time I’d say "Yes, but it wasn’t necessary for the article." So it’s possible that deviations from this supposed optimal strategy might be part some other strategy for the optimization of a different quantity. However, the case remains that making the proof was not an optimal use of my time if I claim the purpose was for the completion of the article.

It's worth pointing out that people are notoriously bad at estimating how much time things will take (see planning fallacy). Moreover, we’re poor at estimating the amount of time we spend on tasks with variable time requirements. Thus, when we decide to create a plan we often neglect to estimate the cost of taking such an action and we often fail to measure whether or not the plan was successful. It’s a sort of recursive issue since usually, we are trying to use the plan to estimate the cost of taking another action. In some sense what I’m trying to do to facilitate analysis is contain the infinite-regress to a second-order meta-plan.

A formal statement

Given that some vague notions are surrounding the ideas of planning, executing, and optimality let's put things on a more formal footing before going into detail. Assume that there is an oracle that could tell you how long a task would take without a dedicated plan. Say that for the task of writing this article it returned seven hours. If I think that I planning could help reduce this time I can take a bet with this oracle. I would place a bet that the time it takes to make a plan plus the time it takes to execute the resulting plan is less than the time returned by oracle. If not, I lose the bet. For example, I might spend 6 hours creating a detailed plan and then spend two hours executing for a total of eight hours. In this case, I lose. Even though I was able to significantly reduce the execution time, I spent way too much time planning to make the strategy effective and ultimately spent more time on the task than if I had just started executing from the start.

Same as the above, but with the level of formality necessary to make a rigorous argument. Let's say you have to complete some task T. Suppose also that you have some sort of initial policy π0 that delineates all the steps you need to take to complete T. Moreover, you also have an oracle Φ that will produce an estimate of you how long it will take to execute π. Naturally, we’ll place a restriction on Φ(π) so that it’s bounded and always equal to or greater than zero. Now, say you can invest t-time and plan π0→πt in the sense that it’s always the case that Φ(π0)≥Φ(πt). Thus, with a slight abuse of notation we can write the result of planning π for t-time as Φ(t). Naturally, our goal is to minimize the total amount of time Γ(t)=t+Φ(t) it takes to execute the planning and the resulting policy.

In the case where everything is nice Φ∈C1 and is convex. We know that it must be the case that Φ(t)≥0. Thus, it should follow that limt→∞Φ(t) exists and that limt→∞Φ′(t)=0. The point is that condition for the minima is easy to state and is equivalent to when Φ′(t)=−1. Thus, as long as the benefit rate of planning is faster than the rate at which time progresses there will be an optimal planning time t′.

Examples a)

Let’s consider a semi-realistic example of a planning function. The most obvious requirement to try and model is that we should see diminishing returns on planning as Φ(t)→0. Thus, a crude model would be to model our planning function so that the reduction in execution time is proportional to the size of the current execution time.

∂Φ∂t=−λ⋅Φ ⇒Φ(t)=T0⋅e−λt

We’re using T0 to model the amount of time a task would take with zero planning and λ to model the rate of reduction. We can solve for the optimal t′ using our discussion above since Φ is convex and bounded. We end up with,

t′=1λlog(λT0) ,Γ=1λ(1+log(λT0))

Thus, the planning to executing ratio ranges from zero to one and depends solely on λT0. For λT0<1 we’ll have t′<0 which means that planning is an ineffective option.

It's worth noting a connection between this simplistic model and the simplistic model we usually use to make plans. Usually, we make plans by creating lists of sub-tasks or steps that need to be completed. Sometimes, we iterate this process if the sub-tasks are difficult to complete. In an idealized model, we'd model the creation of a task list using a branching process and then claim that the execution time of a task is reduced proportionally to the number of leaves in the resulting tree.

b)

There is an important quantity here that can be used to make a practical algorithm. From the first example, we showed that the task reduction quantity λT0 governs the effectiveness of planning. The requirement that 1">λT0>1 is the same as demanding that the rate of reduction from planning be greater than one.

For many common tasks, it’s immediately obvious whether or not a plan should be constructed. Assume that this decision rule is a statistic of λT0. It follows that we can estimate the reduction in total time by looking at the current plan for execution at each time step. Once this estimate of the total time reaches a minimum we should stop planning and start executing.

The pathological aspect of this algorithm is that there might be a critical threshold that needs to be reached before planning is effective. Consider that we estimate λT0, but the true model for Φ has a varying rate equation. However, even in the worst case, we’d only double the amount of time taken to complete the task. The reasoning is that for planning to be an effective strategy the minimum must be reached sometime before the estimated time for execution of the initial plan. If this assumption is violated our original hypothesis would be as well which means that are no guarantees about when or if a minimum might be reached.

It would seem as though we could layer more complexity onto these simple models to make them more effective. For instance, we can formalize what it means to determine whether or not our guess is correct by using statistical estimators. It also seems as though the convexity requirement on Φ is a bit strong for our purposes since planning is a somewhat complicated and somewhat arbitrary/random affair. Finally, the fact that we can create plans that have sub-plans means that we can recursively call our meta-plan for making plans. We'll come back to this last point in a bit.

General Strategy

As noted in the previous section, we’ve restricted Φ so severely that we might be concerned it would tell us nothing about whether or not it’s a good idea in general to try and optimize π. This is because we have no guarantee that our optima at t′ is better than t+Φ(t) for t∈[0,t′]. Now, lucky for us most of this has already been studied in the context of optimal persistent policy theory. The theory deals with optimal selection problems where the n-th observation costs c(n). Thus, we can use this tool and assume we can construct a sequence of plans {πt}t≥0 and incur unit cost for each plan we create.

Let V(π) denote the expected time of planning plus execution for continuing to compress π after the observation of π. It follows that V defines a stopping criteria for if V(π)<Φ(π) we should expect to do worse unless we stop planning and start executing. In practice, for each t we observe/estimate Φ(t) and pay a cost t to do so. To accommodate error in estimation we introduce a transition function to map πt→πt+1. It’s given as the probability of going to policy y starting from x and will be written as Px(y). All of our policies live in the decision space D. The decision to continue planning incurs unit cost. If our V was optimal for each π we’d have,

(1)V(π)=min(Φ(π),∫DV(y) dPx(y)+1)

Theorem 3.1. Suppose that Φ(x) is bounded on D. Then equation (1) has at least one solution V that satisfies infd∈DV(d)≥infd∈DΦ(d).

Proof. The trick is to define a sequence of partial solutions using recursion. Define truncations of V as,

V0(π)=Φ(π) V1(π)=min(Φ(π),∫πV0(π) dPx(y)+1) Vn(π)=min(Φ(π),∫DVn−1(π) dPx(y)+1)

⇒Vn(π)≥Vn+1(π)

This is saying that the expected execution will go down as we continue to refine our V. It also follows that there is a limit to how far we can push this refinement process as we must have,

⇒Vn(π)≥infπ∈D(Φ(π))∀n

The intuition here is that we can only do as good as the best plan available. It follows from the Lebesgue Convergence Theorem that,

V∗(π)=limn→∞Vn(π)

=min(Φ(π),limn→∞∫DVn−1(π) dPx(y)+1)

=min(Φ(π),∫DV∗(π) dPx(y)+1)

⇒infπ∈DV∗(π)≥infπ∈DΦ(π)

Okay, now we need to show that our solution is unique. The idea of the proof is that if you had more than a single solution, say V and U, there will have to be regions where one policy will go for continuing and the other won’t. If we consider a region where U<V this means that the advantage of continuing will be greater if you use U. On the other hand, over the region where the value of V is greater than U there will also have to be a maximum difference between the value functions. However, this would mean that the advantage function of U will need to be greater than V which is a contradiction.

Given the sketch, it sounds like we’ll have to demand that Φ(D) be compact. We’ll show after the proof that it’s possible to relax this condition to an assertion that it’s always possible to transition to a certain set of policies.

Theorem 3.2. Let Φ(D) be a compact topological space and for π∈D let 0">Px(A)>0 for all open sets in A∈D. Then there’s at most a single solution such that (i) V is continuous and (ii) infπ∈DV(π)=infπ∈DΦ(x).

Proof. For a function R(x) that measures execution time we define the advantage as γr(π)=R(x)−∫DR(y) dPx(y). Think of this as a measure of the change in expected execution time. If it’s positive the current state has an advantage over the next expected state. We define things this way so that if R is a solution of (1) then it follows that γr(x)≤1. To see this note that subtracting V from the left-hand side of (1) needs to give zero. Now, imagine we have two solutions U and V for (1). Let,

U(x) \rbrace, \quad S_3 = \lbrace x \mid V(x) < U(x) \rbrace ">S1={x∣V(x)=U(x)},S2={x∣V(x)>U(x)},S3={x∣V(x)<U(x)}

We’ll show that S2=S3=∅. Define W(x)=max(U(x),V(x)) then for x∈S1∪S2 we have W(x)=V(x) and for all x∈D,

∫DV(y) dPx(y)≤∫DW(y) dPx(y)

From this we conclude that γw(x)≤γv(x)≤1. Since U(x)<V(x) and inf(V(x))=inf(Φ(x)) we conclude that we have to have γu(x)=1 or else (ii) won’t be satisfied. We’re about to run into a problem. We know from the compactness of Φ(D) that that there exists x0 such that,

0">W(x0)−V(x0)=max(W(x)−V(x))>0

\int_D W(y) - V(y) \ dP_x(y)">⇒W(x0)−V(x0)>∫DW(y)−V(y) dPx(y)

\gamma_v(x)">⇒γw(x0)>γv(x)

This is a contradiction so it must be the case that S2=∅. By symmetry of U and V we can also eliminate S3.

Corollary 3.3. Let Φ(D) be a measurable space where there exists a non-empty Φ(D′) such that Φ(D′)⊂Φ(D) and \delta > 0">Px(D′)>δ>0 for all x∉D′. Then we have a single solution V that is measurable and bounded.

We can use the same argument as above. However, because we lost compactness we must look at the infimum which will have a value of say −M. We fix ϵ<Mδ(1+δ)−1 and then we can get a point xϵ inside of the set we took an infimum over. Now when we integrate over S we can extract Sϵ and everything will work out.

Redux

Does this information help illuminate anything? Returning to the article example given at the beginning, it would seem as though I spent too much time planning the mathematical portion of the writing. I'm about done with the article and I see that the mathematical portion grew in scope way beyond what was intended in the original plan. This deviation was significant due to me not realizing that the article would need to be formatted. I think that if I had realized how much time it would take to format the article with late it would've become readily apparent that I should've been more sparing in the amount of math I used. Overall, I spent about 14 hours on this article while my original estimate was about 7. I spent several hours making mathematical proofs when I really should've been reading up on how to format an article with latex correctly. I also think that I could've spent a bit more time proof-reading everything before submitting.

What ultimately happened was that while I created an initial plan for the article, the unexpected addition of mathematical arguments broke all my estimates. I should've realized that formatting latex would take a long time since it was my first attempt. It would seem as though the idea of clean separation between planning and execution is an abstraction. The reality is that we all switch between planning and executing whenever we engage with sufficiently complex tasks. I only planned the structure of the article for about an hour because I was overly optimistic about how long it would take to finish the whole thing. I fell prey to the planning fallacy.

Does this mean this idea failed? Not exactly. Recall the idealized model of planning. When we make plans we oftentimes will create tasks that themselves require further planning. It seems perfectly reasonable that we could simply "call" our meta-plan on these sub-tasks to reduce the effect of human bias. I was overly optimistic in thinking about how long it would take to format the math. This problem could've been heavily mitigated if I had a plan that made the task of making the mathematical section a sub-plan. If I had done this, I would've had a chance to look at the bigger picture and most likely would've proceeded differently.

Outlook

Ultimately, I think that the idea was a success because it was developed to enough generality to be useful in getting a concept handle on some pretty big questions surrounding the idea of meta-planning. First, what I am currently engaging in is the construction of a plan that when executed can manage the planning of plans. While I showed that for a broad class of tasks meta-planning is rational, for a much broader class of problems it isn't. Not all problems are compressible. This means that oftentimes the most effective plan will be to have no real plan at all. You can't plan for this either. Pursuing this line of reasoning is difficult as you quickly reach the limit of rationality due to the undecidability of the various questions involved. Perhaps this is a reason for less than perfect rationality in humans. Second, we're overly optimistic about how much time our plans will take to execute. However, when we realize that our plans were too optimistic we are more than capable of updating them because we naturally make hierarchical or tree-like plans. Using these three observations I conclude that the current model I've developed is sufficient to fine-tune itself for the plans that any systematic method could hope to be capable of tackling. There is no need to continue worrying about abstract details at the meta-level. I need to practice using what I came up with.

Discuss

### Inversion of theorems into definitions when generalizing

Новости LessWrong.com - 4 августа, 2019 - 20:44
Published on August 4, 2019 5:44 PM UTC

This post describes a pattern of abstraction that is common in mathematics, which I haven't seen described in explicit terms elsewhere. I would appreciate pointers to any existing discussions. Also, I would appreciate more examples of this phenomenon, as well as corrections and other insights!

Note on prerequisites for this post: in the opening example below, I assume familiarity with linear algebra and plane geometry, so this post probably won't make much sense without at least some superficial knowledge of these subjects. In the second part of the post, I give a bunch of further examples of the phenomenon, but these examples are all independent, so if you haven't studied a particular subject before, that specific example might not make sense, but you can just skip it and move on to the ones you do understand.

There is something peculiar about the dependency of the following concepts in math:

• Pythagorean theorem
• Law of cosines
• Angle between two vectors
• Dot product
• Inner product

In the Euclidean geometry of R2 (the plane) and R3 (three-dimensional space), one typically goes through a series of steps like this:

1. Using the axioms of Euclidean geometry (in particular the parallel postulate), we prove the Pythagorean theorem.
2. We take the right angle to have angle π/2 and calculate other angles in terms of this one.
3. The Pythagorean theorem allows us to prove the law of cosines (there are many proofs of the law of cosines, but this is one way to do it).
4. Now we make the Cartesian leap to analytic geometry, and start treating points as strings of numbers in some coordinate system. In particular, the Pythagorean theorem now gives us a formula for the distance between two points, and the law of cosines can also be restated in terms of coordinates.
5. Playing around with the law of cosines (stated in terms of coordinates) yields the formula x1y1+x2y2=∥(x1,x2)∥∥(y1,y2)∥cosθ, where (x1,x2) and (y1,y2) are two vectors and θ is the angle between them (and similarly for three dimensions), which motivates us to define the dot product (as being precisely this quantity).

In other words, we take angle and distance as primitive, and derive the inner product (which is the dot product in the case of Euclidean spaces).

But now, consider what we do in (abstract) linear algebra:

1. We have a vector space, which is a structured space satisfying some funny axioms.
2. We define an inner product ⟨v,w⟩ between two vectors v and w, which again satisfies some funny properties.
3. Using the inner product, we define the length of a vector v as ∥v∥=√⟨v,v⟩, and the distance between two vectors v and w as ∥v−w∥.
4. Using the inner product, we define the angle between two non-zero vectors v and w as the unique number θ∈[0,π] satisfying cosθ=⟨v,w⟩∥v∥∥w∥.
5. Using these definitions of length and angle, we can now verify the Pythagorean theorem and law of cosines.

In other words, we have now taken the inner product as primitive, and derived angle, length, and distance from it.

Here is a shot at describing the general phenomenon:

• We start in a concrete domain, where we have two notions A and B, where A is a definition and B is some theorem. (In the example above, A is length/angle and B is the inner product, or rather, B is the theorem which states the equivalence of the algebraic and geometric expressions for the dot product.)
• We find some abstractions/generalizations of the concrete domain.
• We realize that in the abstract setting, we want to talk about A and B, but it's not so easy to see how to talk about them (because the setting is so abstract).
• At some point, someone realizes that instead of trying to define A directly (as in the concrete case), it's better to generalize/"find the principles" that make B tick. We factor out these principles as axioms of B.
• Finally, using B, we can define A.
• We go back and check that in the concrete domain, we can do this same inverted process.

Here is a table that summarizes this process:

Notion Concrete case Generalized case A primitive; defined on its own terms defined in terms of B B a theorem defined axiomatically

In what sense is this pattern of generalization "allowed"? I don't have a satisfying answer here, other than saying that generalizing in this particular way turned out to be useful/interesting. It seems to me that there is a large amount of trial-and-error and art involved in picking the correct theorem to use as the B in the process. I will also say that explicitly verbalizing this process has made me more comfortable about inner product spaces (previously, I just had a vague feeling that "something is not right").

Here are some other examples of this sort of thing in math. In the following examples, the step of using B to define A does not take place (in this sense, the inner product case seems exceptional; I would greatly appreciate hearing about more examples like it).

• Metric spaces: in Euclidean geometry, the triangle inequality is a theorem. But in the theory of metric spaces, the triangle inequality is taken as part of the definition of a metric.
• Sine and cosine: in middle school, we define these functions in terms of angles and ratios of side lengths of a triangle. Then we can prove various things about them, like the power series expansion. When we generalize to complex inputs, we then take the series expansion as the definition.
• Probability: in elementary probability, we define the probability of an event as the number of successful outcomes divided by the number of all possible outcomes. Then we notice that this definition satisfies some properties, namely: (1) the probability is always nonnegative; (2) if an event happens for certain, then its probability is 1; (3) if we have some mutually exclusive events, then we can find the probability that at least one of them happens by summing their individual probabilities. When we generalize to cases where the outcomes are crazy (namely, countably or uncountably infinite), instead of defining probability as a ratio, we take the properties (1), (2), (3) as the definition.
• Conditional probability: when working with finite sets, we can define the conditional probability as P(A∣B):=|A∩B||B|. We then see that if Ω is the (finite) sample space, we have P(A∣B)=|A∩B||B|=|A∩B|/|Ω||B|/|Ω|=P(A∩B)P(B). But now when we move to infinite sets, we just define the conditional probability as P(A∣B):=P(A∩B)P(B).
• Convergence in metric spaces: in basic real analysis in R, we say that limn→∞an=L if the sequence (an)∞n=0 satisfies some epsilon condition (this is the definition). Then we can prove that limn→∞an=L if and only if limn→∞|an−L|=0. Then in more general metric spaces, we define "limn→∞an=L" to mean that limn→∞d(an,L)=0. (Actually, this example is a little cheating, since we can just take the epsilon condition and swap in d(an,L) for |an−L|.)
• Differentiability: in single-variable calculus, we define the derivative to be f′(x0):=limx→x0f(x)−f(x0)x−x0 if this limit exists. We can then prove that f′(x0)=L if and only if limx→x0|f(x)−(f(x0)+L(x−x0))||x−x0|=0. This latter limit is an expression that makes sense in the several-variable setting, and is what we use to define differentiability.
• Continuity: in basic real analysis, we define continuity using an epsilon–delta condition. Later, we prove that this is equivalent to some statement involving open sets. Then in general topology we take the open sets statement as the definition of continuity.
• (Informal.) Arithmetic: in elementary school arithmetic, we "intuitively apprehend" the rational numbers. We discover (as theorems) that two rational numbers a/b and c/d are equal if and only if ad=bc, and that the rationals have the addition rule a/b+c/d=(ad+bc)/(bd). But in the formal construction of number systems, we define the rationals as equivalence classes of pairs of integers (with second coordinate is non-zero), where (a,b)∼(c,d) iff ad=bc, and define addition on the rationals by (a,b)+(c,d):=(ad+bc,bd). Here we aren't really even generalizing anything, just formalizing our intuitions.
• (Somewhat speculative.) Variance: if a random variable X has a normal distribution, its probability density can be parametrized by two parameters, μ and σ2, which have intuitive appeal (by varying these parameters, we can change the shape of the bell curve in predictable ways). Then we find out that σ2 has the property σ2=E[(X−μ)2]. This motivates us to define the variance as E[(X−μ)2] for other random variables (which might not have such nice parametrizations).

Discuss

### Cephaloponderings

Новости LessWrong.com - 4 августа, 2019 - 19:45
Published on August 4, 2019 4:45 PM UTC

Hello all. This is Jacob’s wife, and while Jacob is off chilling in the fjords, I’m staying home and have volunteered to write a guest post.

Instead of trying to match the usual Putanumonit fare, I chose to write about something I find very exciting. I’m a biologist, and while I normally work on research involving the sense of touch in tiny roundworms, I love to read about interesting animal biology that I come across through pop-sci media. Yes, I am one of those people you might find spouting “weird animal sex facts” at parties. But seriously, the diversity of life out there is incredible, and I really can’t get enough of learning and thinking about it. So here’s a bit about a creature that’s been capturing my fascination recently.

Consider the octopus. But first, you might want to consider considering the octopus. Why consider the octopus? I think they’re incredibly interesting creatures, and upon reflecting I think that’s probably because I’m so confused by them. They wouldn’t be very interesting to me if everything I knew about them fit well into my world view without jostling the content or structure of surrounding information, even if I had reached the point of knowing enough to know how much I didn’t know. Working in research, I’ve been whacked upon the head several times with the fact that just because information isn’t known or a problem isn’t solved, that doesn’t automatically mean that anyone cares about learning the information or solving the problem.

That goes for myself as well: I don’t know how many teeth the average walrus has, but until now it’s never occurred to me to look it up because it really doesn’t make much of a difference to me whether it’s 26 or 32. Once I started thinking about it, I had to Google the answer (it’s 18), but I’m happy to stop there instead of asking the same question about every other toothed animal I can think to name. It’s slightly more interesting for me to consider what walruses use their tusks for (Google says mostly for gripping the ice to haul themselves out of the water), but why should that question be more interesting? Walrus tusks are weird, and weirdness contributes to interestingness.

Whaaaat is going on?

For me, having the initial question answered actually spawns more questions. If tusks are so useful for pulling oneself up onto the ice, why are there seals without tusks? Now I’ve added confusion on top of the weirdness, and I’m even more interested. The same thing happened to me regarding octopuses.

Exhibit A: Octopuses are weird.

Octopuses and other cephalopods (the clade including octopuses, squid, and cuttlefish) can change the color of their skin by expanding and contracting pigment-filled bags to show or conceal their contents. Changes in colors and patterns are associated with certain behaviors, like hunting, resting, or defensive maneuvering. Octopuses can also change the texture of their skin by extending projections called papillae, which go from being small bumps to tall spikes. Feel free to spend a few minutes watching this in actionon YouTube.

Nothing but a bit of coral here

Another particularly strange facet of these creatures: octopuses have an unusually complex system for RNA editing, so that the same stretch of DNA can give rise to a large diversity of RNA codes (basic bio review: DNA is transcribed into RNA which is translated into proteins). In most animals, evolution is thought to occur mostly through changes to DNA, but cephalopod evolution and adaptationappears to have occurred mostly through RNA editing, with their DNA genomes changing relatively little over millennia.

Exhibit B: Octopuses are confusing.

Octopuses appear to be very intelligent. They get up to all sorts of shenanigans in captivity, like escaping their tanks, disassembling plumbing, or purposefully messing with electrical equipment for their own amusement. They’ve also been trained to solve puzzles and mazes, and seem to be able to recognize individual humans. However, they lack a lot of characteristics that we usually associate with intelligent animals: they aren’t social, they aren’t long-lived, and they don’t engage in post-hatching parental care.

Some intelligent animal species have something akin to culture, where certain adaptive behaviors must be taught. Cetacean (whale and dolphin) hunting techniques are one example of this, and since octopuses are known to use complex hunting strategies it would make sense for them to use their intelligence for intergenerational information exchange. But octopus parental care only happens up until egg-hatching, whereupon the octopus mom will go slink off and die instead of teaching her babies anything.

That brings me to something I find particularly confusing about octopuses: why do octopuses die after reproducing? It isn’t unusual for animals to die after reproducing once, but its certainly worth asking why that should be the case. What’s particularly interesting about the way octopuses die after reproducing is that it seems to occur by active self-destruction rather than by total expenditure of available resources.

Despite the adage of sperm being cheap and eggs being expensive, even male octopuses will often die after reproducing once. Some go to lengths to avoid being cannibalized by their mate, like detaching their mating arm (which is effectively like a penis for them) and leaving it stuck inside the female.

But without their penises it’s not as if they’re escaping with their lives in order to mate again (although some species have been found to have more than one mating arm). Instead they go off to become senile and die from their own carelessness within a few weeks. Usually that means being eaten by some other predator, so it would actually make more sense in my mind for those male octopuses to allow their mate to eat them, providing her with energy to take care of the fertilized eggs instead of just giving up their bodies to whatever random carnivore. Males have been observed mating with multiple female octopuses, but only for a relatively short period of time before they wander off and die alone.

While female octopuses do spend a lot of time and energy taking care of their fertilized eggs and will stop hunting while doing so, it still doesn’t seem that their deaths are absolutely energetically necessary. Female octopuses in captivity have been observed self-mutilating in apparent effort to hasten their own deaths after mating, smashing themselves into walls and eating off the tips of their own tentacles. Also, it seems that if a certain part of the octopus brain (called the optic gland) is removed, the females won’t engage in this behavior and instead go on to live, eat, and even mate again after reproducing. The optic gland was also found to release hormone-like signals that initiate programmed cell death elsewhere in the octopus’s body. With this being the case it seems that something more than malnourishment causes the female octopus to die after reproduction.

It’s important to note that these experiments were done on octopuses living in captivity, so perhaps a wild female octopus would have little chance of survival after spending so much energy taking care of her eggs, but if that were really the case, why would the animals need to have a self-destruct program installed?

Brooding octopus mom

All this points to orphaned octopus babies having an advantage over those with living parents, which is easy enough to fathom. Without mom and dad around, the kids are left with more food and territory for themselves. However I’m still confused. Wouldn’t a given gene be more likely to replicate itself residing in an individual that reproduced multiple times in addition to being in half the octopus babies it helped produce?

Here’s a try at coming up with a possible explanation: maybe octopuses are like single-use consumer products, think plastic forks. Plastic forks are so cheap that people are willing to buy them, eat with them once, toss them in the trash, and then buy new plastic forks for the next picnic that comes around. In order to reuse the forks, you’d have to wash them, which takes effort, but they can also be flimsy enough that you might already have a few bent prongs after a single round of use.

I don’t think I’d describe octopuses as particularly “cheap” though: while a single brood contains thousands of eggs, the female spends months guarding and cleaning the unhatched eggs without leaving to hunt and eat. After that, the offspring take months or even years to reach sexual maturity, and on average, you’d only expect two individuals from a brood to make it that far and successfully reproduce (assuming a stable population size). To continue with the fork analogy, it could be that they’re annoying to wash, here meaning that it’s difficult for them to make the transition from mating mode to growth and maintenance mode. Seeing as how the octopuses with their optic glands removed seemed to do okay at that, I’m not so sure, but again those octopuses were presumably being kept safe and fed with minimal effort on their part.

Is it that octopuses are flimsy like plastic fork prongs? Well, you’ve got the males that detach their mating arms to keep from being eaten, and generally they really can’t mate again after that (but again I’m super confused about why they bother trying to avoid being cannibalized if they don’t mate again afterwards). And you’ve got the females that are malnourished and weak after taking care of their eggs for months without hunting (but again I’m super confused as to why the self-destruct mode is necessary if they’re so likely to die right afterwards). So “flimsy” seems to fit but in a way that doesn’t make a lot of sense.

There is at least one octopus species that doesn’t tend to die after reproducing for the first time, called the Larger Pacific Striped Octopus (let’s just say LPSO). After reaching sexual maturity, LPSO females will brood for up to 8 months, repeatedly mating and spawning new eggs. The species’ unusual behavior life history was first recorded in the 1970s, but was rejected for publication because it seemed way too strange (in the sense of being so different from other octopuses) and reviewing scientists didn’t trust the observations. After that it took about 30 years for the observations to be confirmed and published.

LPSO kisses

What I consider to be the most striking difference about LPSOs is that they seem to be much more social than other octopus species. Instead of living solitary lives as almost all other octopuses do, they’ve been found living in large groups and sharing dens and hunting grounds. It seems like they’re capable of recognizing individuals among their species, and from what I could find they haven’t been observed cannibalizing each other. Perhaps most octopus parents have to die simply because they’re way too likely to eat their own offspring?

I think the biggest obstacle preventing us from understanding octopuses is that they’re hard for us to observe. We can capture them and raise them in captivity, but that doesn’t tell us much about how they’re adapted to the ecosystem they live in. We can stick video cameras underwater and make graduate students take careful notes on any octopus that swims by, but that only works for species that live in shallow water where light can get through. How to effectively study octopuses will prove to be quite the puzzle, but I’m looking forward to additional insights.

I hope you’re very confused about octopuses at this point, and I hope you’re happy about it. Science occurs in areas where believing you understand what’s going on tends to mean you’re deluding yourself, and NOT UNDERSTANDING has to be a default state that can sometimes be chipped into bits of UNDERSTANDING A LITTLE. But you know you might be in for a really good time when you come across a chunk of I AM SO CONFUSED BY THIS SEEMINGLY CONTRADICTORY INFORMATION, because that’s where you might find an interesting story that other people will care to hear.

Discuss

### Беседы

События в Кочерге - 4 августа, 2019 - 16:00
Формат встреч, где каждый участник может предложить вопрос для обсуждения, а дальше голосованием выбираются самые интересные для всех присутствующих вопросы. Приходите, если хотите обсудить бессмертие или телепорт, теорему Ауманна или уровни футурошока, GPT-2 или новости Neuralink, рациональность, экономику, трансгуманизм или что-то другое.

### LessWrong Community Weekend 2019 – Last 10 Spots

Новости LessWrong.com - 4 августа, 2019 - 12:20
Published on August 4, 2019 9:20 AM UTC

TL;DR: 10 remaining spots! Apply here: tiny.cc/lwcw2019_signup Keynote by Dr. Wanja Wiese: “From Predictive Engines to Conscious Machines and Uploading?” Already joining us in Berlin? Tell your best fellow human to join us When? Fr 30.08 noon - Mo 02.09 Where? http://jh-wannsee.de How much? €200

Less than 28 days left until the LessWrong Community and aspiring rationalists take off together for an exciting long weekend of sharing, exploration, connection and celebration. In the last few months we have already gotten many inspiring applications and filled all but the very last 10 spots! Apply now and join us for this special event.

From Friday August 30th to Monday 2nd September aspiring rationalists from across Europe will gather for 4 days of socializing, fun and intellectual exploration. The majority of the content will be unconference style and participant driven. Yet, we are very delighted to welcome Dr. Wanja Wiese as our keynote speaker and his take on predictive engines, conscious machines, and uploading. Find the full abstract for the keynote attached below.

On Friday afternoon we will put up four big daily planners and before you can spell “epistemology” the attendees will fill them up with 50+ workshops, talks and activities of their own devising, such as sessions about rationality techniques, acrobatic yoga and authentic relating; you can learn all about new hyper-cost-effective altruistic interventions, whether a dragon could hover and much much more.

This is our 6th year and we feel that the atmosphere and sense of community at these weekends is something really special. If that sounds like something you would enjoy and you have some exciting ideas or skills to contribute, do come along and get involved. This year is the biggest one yet and it’s an entire day longer than previous years!

The ticket price of €200 includes accommodation for 3 nights, on-site meals (breakfast, lunch, dinner) and snacks, as well as a tasty welcome lunch at 12:00 on Friday, and a shuttle bus from the restaurant to the venue. On Monday, we checkout by 10:00, but can continue to use some of the conference rooms for coworking and socializing until 15:00.

We still have spots available! Apply here: tiny.cc/lwcw2019_signup and make sure to let us know what experience and ideas you may contribute to this event: tiny.cc/lwcw2019_contribution.

If you would not attend due to financial constraints or if you have any questions, please email us at lwcw.europe@gmail.com.

Looking forward to seeing you there, The Community Weekend organizers and LessWrong Deutschland e.V.

From Predictive Engines to Conscious Machines and Uploading? Predictive processing approaches continue to play an influential role in cognitive neuroscience and philosophy of cognitive science. According to predictive processing, perception and action are underpinned by inference processes on sensory signals, based on an internal model of the world. Since predictive processing is a type of computation, it can also be implemented in artificial, silicon-based systems. But would this endow artificial systems with the same types of mental properties that intelligent biological systems, such as ourselves, possess? A lot hinges on whether the neural mechanisms underpinning consciousness and cognition can be regarded as implementations of predictive processing. If implementing certain forms of predictive processing is sufficient for consciousness, non-biological conscious machines will be possible. In principle, this would also enable us to upload our minds, thereby transcending the limits of mortal biological organisms. But would an uploaded version of myself really be me? And to what extent am I real in the first place, especially if what I experience as ‘me’ is the result of an evolved inference process?

Discuss

### Zeno walks into a bar

Новости LessWrong.com - 4 августа, 2019 - 10:00
Published on August 4, 2019 7:00 AM UTC

Zeno walks into a bar.

"I have a problem," he said.

"What is it?" said the bartender.

"Well, it has to do with the movement of physical bodies," said Zeno.

"Talk to my friend Max," said the bartender. He gestured toward a German man wearing round spectacles.

"Sir," said Zeno, "I wonder if you could help me with a problem."

"What's the problem?" said Max.

"Naturally," said Max.

"And before the arrow reaches point C1 it must reach a point C2 midway between points C1 and A," continued Zeno.

"I see," said Max.

"And before the arrow reaches point C2 it must reach a point C3 midway between points C2 and A," continued Zeno.

"Wait a minute," said Max. "How far apart are points A and B?"

"10 meters," said Zeno.

"Then yes," said Max. "I understand your situation."

"And before the arrow reaches point C3 it must reach a point C4 midway between points C3 and A," continued Zeno. "Do you see the impasse?"

"Nope," said Max, "I think we're getting somewhere. How long is the arrow?"

"One meter," said Zeno.

"The distance between points C3 and C4 is one sixteenth of a meter," said Max. "A one-meter-long arrow can be at point C3 and C4 at the same time."

"Let's consider the tip of the arrow then," said Zeno. "Before the tip of the arrow reaches point C3 it must reach a point C4 midway between points C3 and A."

They talked deep into the night.

"And before the high-energy particle reaches point C118 it must reach a point C119 midway between points C118 and A," continued Zeno.

"Hold on," said Max. "How far apart are points A and C119?"

"1.5×10−35 meters" said Zeno.

"That's shorter than 1.6×10−35 meters," said Max. "The uncertainty in the position of a particle must always exceed 1.6×10−35 meters, because of space-time equivalence and the quantum-mechanical velocity operator's non-commutation with position. Even theoretically, the wave function of a particle can't ever occupy a space smaller than 1.6×10−35 meters."

"Thanks," said Zeno.

"By the way," said Max, "What brought you to this question in the first place?"

"I wanted to know how to define the momentum of a particle at an instantaneous moment of time," said Zeno.

"You could have just asked," said Max. "The probability distribution of a particle's momentum is determined by the instantaneous phase and magnitude of its wave."

Discuss

### Alleviating Bipolar with meditation

Новости LessWrong.com - 4 августа, 2019 - 08:15
Published on August 4, 2019 5:15 AM UTC

Original post: Alleviating Bipolar with meditation

I was asked on the slack, about bipolar and what might help from a meditation standpoint.  I have my own experiences to share. (standard non-medical advice disclaimer applies here, i’m not qualified to give professional advice and you should probably confirm with a professional if you have doubts about trying any of this.)

Here’s a list of things that might help with the subjective mood swinging of bipolar experience.

1. A broadening of awareness and contexts.

For about 6 months of time when I was really focused on moods (and 10 years before that), I felt like I didn’t have moods, moods had me (moods distinct from emotions which can be had from moment to moment, moods are more like background, the colour of the day). I would wake up and find out today was “miserable” or “excited”.

I worked on a specific type of meditation practice that is called broadening of awareness (there are 2 different instructions for methods).  I got lucky that this helped me and I wasn’t expecting it. When moods had me, it felt like things “just are” miserable. Now my awareness is broader than the moods and “I”* contain them.  (*meditative “I” and “self” are a rabbit hole)

Instructions: Most people have their sense of their self boundary in line with their skin barrier. “I” end at my skin. But it’s possible to expand that boundary, and shift it to larger. Particularly the “kinetic sphere”, the area where one might be able to reach outside the body, and then further to the whole room size. Holding this “barrier” thing at the size of the room means that I’m “anchored” metaphorically to more solid things than my own body. Obviously “I’m” still the same but my ground is the actual stationary room. Which does not feel moods like my body does. (*explanation of why it helps may be entirely irrelevant, fact is, anecdata: it helped me)

There’s space in my new expanded “me” to find the body being a certain mood but also to find stillness out there in the room which doesn’t get dragged around like the moods do.  I felt the pull of daily moods dry up. Obviously my body is still in grump but “I’m not” mentally trapped in that experience. From there, there’s a new, deeper breathing pattern that supports the broader awareness practice and that’s to be discovered and also hinted at.  I would encourage trying it for a few minutes a day and then going for a permanent shift into what is sometimes described as “spaciousness”.

Instructions 2: awareness specifically in the visual field can be expanded out the peripheral. Start by picking an object straight ahead to look at and focus on. Now expand the awareness to the peripheral of the visual field. Hold there for 30 seconds, then push on towards expanding the peripheral. this works well looking up at the sky, or the ocean because of the broadness of the visual object in the visual field. push the “awareness” beyond the visual field until there’s a sense of spidey-sense tingling to what’s outside the visual field. Hold a broadness of awareness to the visual area and the spidey sense. Try to engage this broad sense regularly and through the day, try to live in this broad-sense of the world around you. Notice that a “mood” is within this sense, not fully covering the whole space. If you work at the broadness, that sense comes.

2. Stages of insight

At the same time as trying that practice, I was cycling through (technical meditation term – can be read about in MCTB2 book) “the stages of insight“. As I would cycle I would hit sensation like fear, and it would call up involuntary intrusive memories about things I feared, then I would the next day have a “when will it end” feeling and wrestle with that one.

For 2, what became important is forming a relationship with the memories that I didn’t like. Due to lots of meditation, I was pretty clear what was normal and what was an intrusive visit from my past. I started asking the question, “why is this here?” and that question eventually turned into, “how is this here to help?” or “what do I need to still learn from this memory?” and that was a huge shift.

After those questions were hard ingrained into my attitude, within a week, shitty memories stopped showing up. Possibly because I got so good at relating to them that I was never calling them, “shitty memories”, and possibly because I never felt shit again about them, I’d just appreciate the lesson that I was to learn.  And from that I stopped cycling nearly as hard. I still notice bits of cycling but I’m above the cycle, not in it.

3 Greater bodily awareness.

a few days ago I wanted a photo of myself, so I put on a fancy shirt and got out of bed to take the photo.  3 minutes later I found myself eating things. When I asked myself what’s going on, because I wasn’t hungry, I noticed that I was cold and I was using food to stop feeling cold. An interesting discovery. I made my way back to warm things.

It’s bodily awareness that helps with the moods and actions. I can feel where in my body (or not) I’m feeling depressed or angry and I can alleviate it via movement or internal sensation and not by outwardly being moody or suffering mood swings.

For this I’ve done a lot of meditation and body scan attention work. Any sensation is relevant, itching the head, the knot in the stomach, the tingle in the toes. It’s all relevant to the way I think.

It’s a rat rationality thing to assume that these sensation experiences are noise but they are not. All sensation is relevant.

Some combination of the 3 have helped me to the point where I doubt I have bipolar any more.  I was fairly confident at one point and now it seems unlikely to be a useful diagnosis.

And if there’s a 4 and 5 it’s, watch sleep and social life and make sure to get enough of both, as well as being aware of instability in both which can start a cycle of instability.  This is from Interpersonal Social Rhythm Therapy IPSRT – the only therapy designed for bipolar. Fixing my sleep made a big difference, and fixing my mood first thing in the morning did too.

Shoutout to Bipolar Awakenings for being more on the odd-strange-spiritual side of meditative practice towards progress on alleviating bipolar.

Discuss

### Встреча YC Startup School

События в Кочерге - 3 августа, 2019 - 18:00
Обсуждаем лекции Startup School - бесплатный 10-недельный курс от YCombinator, самого известного стартап-акселератора в мире - и устраиваем очные group sessions. Вы могли что-то слышать про YCombinator, потому что их основатель Пол Грэм написал известное в рацио-кругах эссе “Keep your identity small”; или потому что Сэм Альтман, второй президент YCombinator, теперь руководит проектом OpenAI; или потому что YCombinator хостят известный форум Hacker News; или потому, что среди их выпускников - Dropbox, Airbnb и 80,000 часов. Если вы хотите больше проникнуться стартап-культурой Кремниевой долины (как в сериале, только всерьёз), получить знание для будущих проектов, и повысить уровень амбициозности - приходите обсуждать лекции и свои проекты.

### Proposed algorithm to fight anchoring bias

Новости LessWrong.com - 3 августа, 2019 - 07:07
Published on August 3, 2019 4:07 AM UTC

Anchoring is a classic cognitive bias which has been discussed on Less Wrong before. Anchoring seems very difficult to avoid. Experiments have found that warning subjects about anchoring, or giving them cash incentives, doesn't solve the problem.

Here's an algorithm to fight anchoring that I would like to see a researcher test, based on binary search:

1. Tell subjects to think of a number which is clearly too high for the quantity they want to estimate (an upper bound).
2. Tell subjects to think of a number which is clearly too low (a lower bound).
3. Tell subjects to find the midpoint of the upper bound and the lower bound and figure out whether it's too high or too low.
4. The midpoint has now been judged as an upper/lower bound. Combined with the original lower/upper bound, we have a new, narrower range to explore. If this range is narrow enough, report its midpoint; otherwise go to step 3.

You could have two experimental conditions: one condition where subjects think of a number which is clearly too high first (the steps are in the order above), and another condition where subjects think of a number which is clearly too low first (steps 1 & 2 are swapped). If estimates from the two conditions are similar, the technique is successful.

Discuss