Вы здесь

Сборщик RSS-лент

Limiting an AGI's Context Temporally

Новости LessWrong.com - 17 февраля, 2019 - 22:29
Published on February 17, 2019 3:29 AM UTC

Okay, so I have a proposal for how to advance AI safety efforts significantly.

Humans experience time as exponential decay of utility. One dollar now is worth two dollars some time in the future, which is worth eight dollars even further in the future, and so forth. This is the principle behind compound interest. Most likely, any AI entities we create will have a comparable relationship with time.
So: What if we configured an AI's half-life of utility to be much shorter than ours?


Imagine, if you will, this principle applied to a paperclip maximizer. "Yeah, if I wanted to, I could make a ten-minute phone call to kick-start my diabolical scheme to take over the world and make octillions of paperclips. But that would take like half a year to come to fruition, and I assign so little weight to what happens in six months that I can't be bothered to plan that far ahead, even though I could arrange to get octillions of paperclips then if I did. So screw that, I'll make paperclips the old-fashioned way."
This approach may prove to be a game-changer in that it allows us to safely make a "prototype" AGI for testing purposes without endangering the entire world. It improves AGI testing in two essential ways:

  1. Decreases the scope of the AI's actions, so that if disaster happens it might just be confined to the region around the AGI rather than killing the entire world. Makes safety testing much safer on a fundamental level.
  2. Makes the fruits of the AI more obvious more quickly, so that iteration time is shortened drastically. If an AI doesn't care about any future day, it will take no more than 24 hours to come to a conclusion as to whether it's dangerous in its current state.

Naturally, finalized AGIs ought to be set so that their half-life of utility resembles ours. But I see no reason why we can't gradually lengthen it over time as we grow more confident that we've taught the AI to not kill us.

(Note: There are 6x10^27 grams of matter on Earth. Throw in a couple orders of magnitude for the benefit of being undisturbed and this suggests that taking over the world represents a utility bonus of roughly 10^30. This is pretty close to 2^100, which suggests that an AGI will not take over the world if its fastest possible takeover scheme would take more than 100 half-lives. Of course, this is just Fermi estimation here, but it still gives me reason to believe that an AGI with a half-life of, say, one second, won't end human civilization.)



Discuss

Can We Place Trust in Post-AGI Forecasting Evaluations?

Новости LessWrong.com - 17 февраля, 2019 - 22:20
Published on February 17, 2019 7:20 PM UTC

TLDR
Think "A prediction market, where most questions are evaluated shortly after an AGI is developed." We could probably answer hard questions more easily post-AGI, so delaying them would have significant benefits.

Motivation

Imagine that select pre-AGI legal contracts stay valid post-AGI. Then a lot of things are possible.

There are definitely a few different scenarios out there for economic and political consistency post-AGI, but I believe there is at least a legitimate chance (>20%) that legal contracts will exist for what seems like a significant time (>2 human-experiential years.)

If these contracts stay valid, then we could have contracts set up to ensure that prediction evaluations and prizes happen.

This could be quite interesting because post-AGI evaluations could be a whole lot better than pre-AGI evaluations. They should be less expensive and possibly far more accurate.

One of the primary expenses now with forecasting setups is the evaluation specification and execution. If these could be pushed off while keeping relevance, that could be really useful.

Idea

What this could look like is something like a Prediction Tournament or Prediction Market where many of the questions will be evaluated post-AGI. Perhaps there would be a condition that the questions would only be evaluated if AGI happens within 30 years, and in those cases, the evaluations would happen once a specific threshold is met.

If we expect a post-AGI world to allow for incredible reasoning and simulation abilities, we could assume that it could make incredibly impressive evaluations.

Some example questions:

  • To what degree is each currently-known philosophical system accurate?
  • What was the expected value of Effective Altruist activity Y, based on the information available at the time to a specific set of humans?
  • How much value has each Academic field created, according to a specific philosophical system?
  • What would the GDP of the U.S. have been in 2030, conditional on them doing policy X in 2022?
  • What were the chances of AGI going well, based on the information available at the time to a specific set of humans?
Downsides

My guess is that many people would find this quite counterintuitive. Forecasting systems are already weird enough.

There's a lot of uncertainty around the value systems and epistemic l states of authoritative agencies, post-AGI. Perhaps they would be so incredibly different to us now that any answers they could give us would seem arcane and useless. Similar to how it may become dangerous to extrapolate one's volition "too far", it may also be dangerous to be "too smart" when making evaluations defined by less intelligent beings.

That said, the really important thing isn't how the evaluations will actually happen, but rather what forecasters will think of it. Whatever evaluation system motivates forecasters to be as accurate and useful as possible (while minimizing cost) is the one to strive for.

My guess is that it's worth trying out, at least in a minor capacity. There should, of course, be related forecasts for things like, "In 2025, will it be obvious that post-AGI forecasts are a terrible idea?"

Questions for Others

This all leaves a lot of questions open. Here are a few specific ones that come to mind:

  • What kinds of legal structures could be most useful for post-AGI evaluations?
  • What, in general, would people think of post-AGI evaluations? Could any prediction community take them seriously and use them for additional accuracy?
  • What kinds of questions would people want to see forecasted, if we could have post-AGI evaluations?
  • What other factors would make this a good or bad thing to try out?


Discuss

Cambridge SSC Meetup

Новости LessWrong.com - 17 февраля, 2019 - 21:28
Published on February 17, 2019 6:28 PM UTC

This is the monthly Cambridge SSC meetup.



Discuss

Cambridge SSC Meetup

Новости LessWrong.com - 17 февраля, 2019 - 21:27
Published on February 17, 2019 6:27 PM UTC

This is the monthly Cambridge SSC meetup.



Discuss

Клуб чтения цепочек

События в Кочерге - 17 февраля, 2019 - 21:20
Пятница, 22 февраля, 19:30

Воркшоп по прикладной рациональности

События в Кочерге - 17 февраля, 2019 - 18:00
Суббота, 16 марта, 11:00

Extraordinary ethics require extraordinary arguments

Новости LessWrong.com - 17 февраля, 2019 - 17:59
Published on February 17, 2019 2:59 PM UTC

Previous post: Fighting the allure of depressive realism

My blog entries are about a personal battle against depression and anxiety, from the point of view of someone who has been immersed in rationalist/LW ideas and culture for a few years now.

I want to illustrate a particular, recurring battle I have with scrupulosity. (I'm not the best at dialogues, so bear with me for a moment.)

Me: Alright, it's time to be productive and get to my homework.???: Hold on! How can you possibly justify that if you haven't solved ethics yet?Me: What? Who are you?SD: Allow me to introduce myself, I'm your Skeptic Demon. I whisper ethical concerns in your ear to make sure you're always doing the right thing.Me: That sounds more like a daemon than a demon to me.SD: Demon. Trust me.Me: Solving ethics can't possibly be expected of a single person, demon.SD: Right you are! But you've looked around the world enough to know that everything you do could have ripple effects that might be disastrous. So how can you possibly feel good about working on your homework without accounting for that?Me: What? It's just homework.SD: Oh, no it isn't. Doing well on homework means sending a better signal to employers means more people want to hire you down the line, including for unscrupulous activities. And you've done not-great things before, so we can't be sure you'll resist. In fact the existence of first-, second-, third-, and nth-order effects implies you might not even realize when you're being offered such.Me: Erm... Well, it's true that things have unintended consequences, but--SD: No "buts"! You want to be a good person, right? So we gotta reason this out.Me: I guess you have a point...SD: Alright. So let's get started.(hours pass)SD: Okay. You're on shaky ice with some of these considerations. I'm not totally convinced you won't be tempted by the money to go and do something net harmful yourself, but I will give you a one-time pass. You may proceed to start your assignment .Me: I'm exhausted and I just want to go to sleep now.SD: Then my work is done here. *disappears in a puff of shaky logic*

This kind of conversation happens to me all the time. Why?

On one level, it's easy to see what the skeptic demon is doing. He's trolling. He's keeping me from doing the actual productive work I want to do, and very curiously never pops up to ask whether my watching TV or even whether my eating meat is ill-advised.

But he's trolling with a legitimate issue - the fact that we can't actually predict all of the possible consequences of our actions. It feels wrong to say that someone should be held ethically responsible for the sum total of that butterfly effect, but it feels equally wrong to deny they have any stake in it whatsoever. Trolls are worst when they find an issue that is near and dear to your heart and poke at it.

What to do? I'd like to at least justify why I think it's okay to ignore this little guy.

I think we can get a lot of mileage here out of the old Carl Sagan heuristic, "Extraordinary claims require extaordinary evidence." Here, it changes to extraordinary ethics require extraordinary arguments. And the idea that I should sabotage my own career out of the fear that I might accidentally harm someone down the line due to my own weakness is one heck of an extraordinary ethic.

For one, this ethic immediately fails my pop-philosophy understanding of the categorical imperative. If everyone acted like this, modern society and all of its woes would crumble, but so would its many, many, many benefits.

It also fails my understanding of why we usually give self-interest a seat at the table in ethics, even if we worry about its excesses: A world in which everyone spends all of their energy trying to make other people happy but never take time for themselves is a world where everyone runs themselves ragged and is uniformly miserable.

We could make the argument that people are far less morally responsible for second-, third-, etc. order effects from many different angles, one of my favorites being local validity. And so on.

I'm not sure how far I can take this heuristic before it breaks, but I think it's a very wise starting point to begin with when it comes to issues of scrupulosity.



Discuss

Major Donation: Long Term Future Fund Application Extended 1 Week

Новости LessWrong.com - 17 февраля, 2019 - 02:30
Published on February 16, 2019 11:30 PM UTC

(Crossposted to the EA Forum)

The Long-Term Future EA Funds application round is extended for one more week (until Midnight (PST) Sunday 24th of February).

The LTF Fund received a major donation such that the amount of money available for granting has more than doubled, to a total of ~$1.2 million, making our available funds greater than all money that the LTF fund has ever given away historically.

This means we can consider:

  • Grants of much larger ambition and scope.
  • Grants with longer time-horizons (who will need funding in future years)
  • Grants to larger teams
  • Seed funding for new organisations
  • Many more grants to smaller projects and individuals

I think this means applying this round is a particularly good idea, as well as forwarding the application to anyone else you know who might want to apply.

Apply here.

More info on the LTF Fund can be found here.

If you’re uncertain about whether we’re interested in funding a project, you can ask me in a comment, or email me and the other fund managers at ealongtermfuture@gmail.com.



Discuss

Games in Kocherga club: Fallacymania, Tower of Chaos, Scientific Discovery

Новости LessWrong.com - 17 февраля, 2019 - 01:29
Published on February 16, 2019 10:29 PM UTC

Welcome to Moscow LW community makeshift games! In that games, some rationality skills are involved, so you can practise while you playing!
* Fallacymania: it is a game where you guess logical fallacies in arguments, or practise using logical fallacies yourself (depending on team in which you will be).
* Tower of Chaos: funny game with guessing the rules of human placement on a Twister mat.
* Scientific Discovery: modified version of Zendo with simultaneous turns for all players.
Details about the games: https://bit.ly/2J2T5o8
Come to antikafe "Kocherga", ul.B.Dorogomilovskaya, 5-2. The map is here: https://kocherga-club.ru/#contacts
Games begin at 19:40, the length is 3 hours.



Discuss

Heuristics for decentralised coordination?

Новости LessWrong.com - 16 февраля, 2019 - 23:39
Published on February 16, 2019 8:39 PM UTC

Robin Hanson has argued that those who believe AI Risk to be a primary concern for humanity, are suffering from a bias toward thinking that concentration of power is always more efficient than a decentralised system (think government versus anarchy).

The obvious counterargument is that the most effective and scalable means of coordination that humanity has discovered has in fact been distributed markets.

Peter McCluskey puts it clearly:

Maybe there’s a useful analogy to markets – maybe people underestimate [Eric Drexler's models] because very decentralized systems are harder for people to model. People often imagine that decentralized markets are less efficient that centralized command and control, and only seem to tolerate markets after seeing lots of evidence (e.g. the collapse of communism).

When I initially read Robin's original post, my 'debiasing' detector went off. While humans are riddled with biases, from the perspective of a human, it is prudent to specify the heuristic we're running that was previously adaptive, identify why it is no longer adaptive, and then begin practising a new set of heuristics better optimised for our present-day environment.

It seems true to me that humans commonly assume centralisation to be more efficient, and that they're sometimes right, and sometimes very wrong.

So, for the cases where we get it wrong, the question is for people to do any of the following three things.

  • 1) Specify the heuristic you're (natively) running, with concrete examples of using it.
  • 2) Give a parsimonious explanation for why you would've evolved/learned this heuristic.
  • 3) Specify a new heuristic(s) that should be used in the modern world, and give a few concrete examples.

(State in your answer which of the three you're responding to.)



Discuss

Is there a way to hire academics hourly?

Новости LessWrong.com - 16 февраля, 2019 - 17:21
Published on February 16, 2019 2:21 PM UTC

Seems there should be but I'm not finding it. Craigslist and Upwork don't seem to, but I was (am actually; any takers?) looking for someone with expertise in geography and global mythology and it seems pretty clear to me the value to me is probably higher than they'd charge with those specialties, so I assume it's just my very weak googling skills getting in the way.

Thanks in advance, and if it doesn't exist and you want to build it I expressly release any claim to a cut of the profits,

Charlie



Discuss

Уличная эпистемология. Тренировка

События в Кочерге - 16 февраля, 2019 - 15:40
Вторник, 19 февраля, 19:30

Ненасильственное общение. Тренировка

События в Кочерге - 16 февраля, 2019 - 15:40
Четверг, 21 февраля, 19:30

Graceful Shutdown

Новости LessWrong.com - 16 февраля, 2019 - 14:30
Published on February 16, 2019 11:30 AM UTC

Within a structured concurrency world, graceful shutdown is a little understood, little addressed and confusing topic.

I've been fighting with it myself for almost a year (longer than with any other problem I can recall) and it haven't been a pleasant experience. If felt like running in circles. Too many contradictory requirements, to much complexity and too often a solution I've came up with turned out to be one of the solution I've already rejected in disguise.

Nathaniel Smith on structured concurrency forum helpfully pointed me to the discussion about graceful shutdowns in Trio and oh boy, does that discussion sound familiar

Anyway, finally I've came up with a solution that's simple and works for all use cases. But it turns out it's one of those ideas that seem not to make sense until something clicks in one's mind. From there on though, they seem perfectly obvious, up to the point where one wonders how the others can possibly not get it.

In this post I'll try to explain.

The use case

The classic use case for graceful shutdown is a web server. It's handling many connections from the clients. When it's being shut down, you want it to happen in a controlled manner so that clients don't end up with half-loaded pages and such. Specifically, you want to stop the accept loop to make sure that no new connections are being made. Then you want to wait for say 10 seconds to give existing connections time to finish their work. Finally, when the time is up, you want to cancel any remaining connections and exit the process.

Of course, the nature of the workload may vary. Web server needs to shut down connections. CRUD application needs to shut down its connection to database. Yet different application may want to save data to disk before exiting.

What's graceful shutdown, anyway?

Graceful shutdown intuitively seems to be some kind of cancellation. Here, I would like to compare it to the classic hard cancellation and argue that graceful shutdown isn't anything like it.

Let's have a look at hard coroutine cancellation first. The code of the coroutine looks like this

if(rc == ECANCELED) return;
rc = recv(s, &msg);
if(rc == ECANCELED) return;
rc = sleep(60); i
f(rc == EANCELED) return;

When parent coroutine hard-cancels a child coroutine, a blocking call within the coroutine will immediately return an error (or, in high-level languages, it will throw an exception) and the coroutine will exit.

Think of hard cancellation as "asking the coroutine to return immediately". In reality, it may do some bookkeeping tasks before exiting (freeing the allocated memory etc.) but form the perspective of this article, thinking of hard cancellation as eliciting immediate "return" in the coroutine is a pretty good approximation.

Now have a look at graceful shutdown.

It's very different. By saying "shut down gracefully" we are asking the coroutine to finish whatever it was doing and start doing different kind of work, namely it's shutdown business logic. So, for example, while during its normal lifetime it may be receiving data from the network and writing them to the database, after graceful shutdown request it may want to exchange terminal handshakes with the network peers and start flushing any in-memory data to the database.

Not only is graceful shutdown not immediate. It can be arbitrarily long or even infinite. Consider the case when you are trying to exchange a terminal handshake with a network peer. If the peer is not responding, the graceful shutdown will never finish.

More on the nature of graceful shutdown: In-band vs. out of band

Consider a case where the parent coroutine sends the child coroutine a stream of tasks to process. It does so via a channel. It may send DoA, DoB and DoC requests. Then it hard-cancels the child. Hard cancellation is supposed to be immediate and so the child coroutine exits even though DoA, DoB and DoC haven't been processed yet.

Does the graceful shutdown work is a similar way? No. Graceful shutdown means "finish whatever you are doing but take your time and be careful; don't break the business logic". During graceful shutdown you definitely don't want to drop the requests on the floor. So, instead of sending an out-of-band signal, like hard cancel did, you just write an application-defined STOP message to the channel and leave the child coroutine to do it's work. After it processes DoA, DoB and DoC, it will eventually receive the STOP message. At that point it will do the terminal handshakes, it will flush data to disk and so on. Finally, it will exit.


The point I am trying to make here is that while hard-cancel signal travels necessarily out-of-band, the graceful shutdown signal must be, equally necessarily, passed in-band.

And when you think about it a little bit more, "out-of-band" means that the communication channel is created and managed, invisibly to the user, by the language runtime (or a structured concurrency library, such as Trio or libdill) while "in-band" means that the communication channel is business-logic-specific and that it is defined and managed by the application itself.

In consequence, language runtime (or the library) has no way of knowing how to send a graceful shutdown to the child coroutine, even if it wanted to. The coroutine is listening on an application-specific channel. It may be channel A or channel B. It may be listening on both. Or maybe it's listening on channel A up to some point in its lifecycle and then starts listening on channel B. Or maybe it even gets the graceful shutdown request over the network (the latter is common during terminal handshake in network protocols). The language runtime just doesn't know.

Ergo, sending a graceful shutdown request cannot possibly be a feature of the language. It must be done manually by the application.

Even more on the nature of graceful shutdown: Cancellation points

The rules of hard cancellation are strict: Once a coroutine has been hard-canceled the very next blocking call will immediately return ECANCELED. In other words, every single blocking call is a cancellation point.

Graceful shutdown is different. It allows the coroutine to exit only at special points in the workflow where the business logic is in consistent state.

Consider the case of an RPC server. It reads a message from the peer, does some computation, then sends a reply to the peer.

Graceful shutdown can terminate the coroutine only when it is idling and waiting for a new request.

It we allowed graceful shutdown to terminate the coroutine while it is trying to send the reply, it would mean that a request could go unanswered. And that doesn't deserve to be called "graceful shutdown".

Let's summarize

What we learned so far is that hard cancellation:

  1. Is triggered via an invisible communication channel created by the language runtime.
  2. It manifests itself inside the target coroutine as an error code (ECANCELED in libdill) or an exception (Cancelled in Trio).
  3. The error (or the exception) can be returned from any blocking call.
  4. In response to it, the coroutine is not expected to do any application-specific work. It should just exit.

Graceful shutdown, on the other hand:

  1. Is triggered via an application-specific channel.
  2. Manifests itself inside the target coroutine as a plain old message.
  3. The message may only be received at specific, application-defined points in the coroutine.
  4. In response to it, the coroutine can do arbitrary amount of application-specific work.

Well, that seems to be it.

Hard cancellation is fully managed by the language. Graceful shutdown is fully managed by the application. From the language designer's perspective there's nothing to see here. We should just move on and forget about the entire business.

Or should we?

Keeping graceful shutdown in check

Well, it turns out that the parent coroutine often doesn't want graceful shutdown to take forever. It's willing to give it ten seconds to finish, but if that doesn't happen, so be it, c'est la vie, it will just hard-cancel it.

void main() {
ch = channel();
b = bundle(); // a.k.a. scope or nursery
bundle_go(b, worker(ch));
...
send(ch, STOP); // ask the child to shut down gracefully
sleep(10);
bundle_cancel(b); // hard-cancel the child if it haven't
finished yet
}

You may see the problem with this code. It works OK, but it always waits for 10 seconds, even when the child exits immediately. And that sounds like a waste.

So, after all, the language can do something to help with the graceful termination. It can provide a primitive to avoid the ugly non-negotiable sleep() step.

The proposal

First, let me be clear. I don't propose to add any new concept to the existing structured concurrency model. Not even the concept of "graceful shutdown". We'll do with the existing concepts of coroutine, bundle (scope, nursery) and plain old hard cancellation.

Second, I propose to add a timeout parameter to the traditional bundle_cancel() method:

bundle_cancel(b, 10);

The semantics are as follows: The function will wait for 10 seconds for all coroutines in the bundle to finish. If they all do before the timeout expires, the function will exit immediately as the last coroutine finishes. If they don't, it will hard-cancel all the remaining coroutines when the time is up.

The function also happens to be a blocking function and thus it behaves as any other blocking function. When the parent coroutine itself is hard-canceled bundle_cancel() hard-cancels all the coroutines in the bundle and returns ECANCELED error to the caller.

NOTE: With timeout set to zero, this is equivalent to the plain old hard cancellation. This has some pretty obvious implications for backward compatibility.

The examples

I am aware that such a trivial construct doesn't look like it's up to the task of dealing with the hydra of graceful shutdown.

So, to convince you, let us look how it fares in the wild.

The trivial case

In this scenario the child coroutine does some work and exits.

coroutine void worker(socket_t s) {
int rc = send(s, "Hello, world!");
if(rc == ECANCELED) return;
message_t msg;
rc = recv(s, &msg);
if(rc == ECANCELED) return;
return;
}

int main(void) {
socket_t s = create_connected_socket();
bundle_t b = bundle();
bundle_go(b, worker(s));
sleep(60);
bundle_cancel(b, 10);
return 0;
}

Note how the worker coroutine knows nothing about graceful shutdown. It looks exactly the same way as it would if there was only hard cancellation. Yet the parent

If the worker manages to finish its work in 10 seconds, the program will exit immediately. If it doesn't manage to finish in time, say if it is stuck in the recv() call, it will get hard-canceled after 10 seconds and the program will exit.

Infinite loop

This example deals with the question of what happens if the child coroutine never finishes.

coroutine void worker(socket_t s) {
while(1) {
int rc = send(s, "Hello, world!");
if(rc == ECANCELED) return;
message_t msg;
rc = recv(s, &msg);
if(rc == ECANCELED) return;
}
}

int main(void) {
socket_t s = create_connected_socket();
bundle_t b = bundle();
bundle_go(b, worker(s));
sleep(60);
bundle_cancel(b, 10);
return 0;
}

The code is identical to the previous example, except for the forever loop that was added to the worker. And once again, the child doesn't do anything specific to enable graceful shutdown.

You may feel like this example is a bit pointless. The worker coroutine runs at full speed throughout the entire grace period and then it gets hard-cancelled. It's hardly the graceful shutdown of your dreams.

But while that may be true, I am trying to make a point here. And the point is: The parent doesn't have to care.

The parent doesn't have to care whether the coroutine supports a graceful shutdown or not. It doesn't have to worry about how long the shutdown would take. It doesn't have to care whether there are infinite loops or not. It doesn't care if the worker behaves in a pointless way as shown in the example above.

The programmer doesn't have to go and inspect all the coroutines he is using to find out how the graceful shutdown would behave.

He just calls bundle_close(b, 10) and he is guaranteed that the coroutine will be dead in at most 10 seconds.

Graceful shutdown-aware coroutine

As already mentioned, the language has no concept of graceful shutdown.

Therefore, the parent coroutine has to send the graceful shutdown signal to the worker coroutine manually. It does so via a channel.

coroutine void worker(socket_t s, channel_t ch) {
while(1) {
message_t msg;
int rc = recv_from_socket_or_channel(s, ch, &msg);
if(rc == ECANCELED) return;
if(rc == FROM_CHANNEL) goto graceful_shutdown;
if(rc == FROM_SOCKET) {
// process msg here
}
rc = send(s, "Hello, world!");
if(rc == ECANCELED) return;
}
graceful_shutdown:
... // do the entire graceful shutdown thing here
}

int main(void) {
socket_t s = create_connected_socket();
channel_t ch = channel();
bundle_t b = bundle();
bundle_go(b, worker(s, ch));
sleep(60);
send(ch, "STOP"); // ask for graceful shutdown
bundle_cancel(b, 10); // give it at most 10 seconds to finish
return 0;
}

You may have noticed that I've been cheating here. recv_from_socket_or_channel() function hides a lot of complexity and to be fair, I am not aware of a language that has good API for this kind of thing. However, it is unrelated to the topic of graceful shutdown, so I'll let it be for now.

That being said, what I wanted to show here is how the cancellation points for graceful shutdown checkpoints are application-defined. In particular there's only one place in the workflow where STOP message can be received.

The application is basically an RPC server, waiting for a message from a client and then sending a "Hello, world!" reply. And we certainly don't want the client to miss their hello world message! Therefore, the only valid point to deal with the graceful shutdown is when the coroutine is idling, waiting for a new request.

If we checked for STOP before the send call, the coroutine would exit and RPC would end up unanswered. It would a large stretch to call that "graceful shutdown".

Graceful shutdown triggered by a network peer

I've already mentioned that it's entirely up to the application to decide how the graceful shutdown signal is delivered to the coroutine. One specific case of this is when the signal comes from the outside.

It's a common scenario with network protocols. Each peer can ask for the connection to be shut down. For example, with WebSockets, each party can send a special CLOSE message to inform the peer they want to shut down. The peer should respond by sending CLOSE message of its own.

The following code is a server that responds to CLOSE message from the client.

coroutine void worker(socket_t s) {
while(1) {
message_t msg;
int rc = recv(s, &msg);
if(rc == ECANCELED) return;
if(msg == "CLOSE") break;
rc = send(s, "Hello, world!");
if(rc == ECANCELED) return;
}
int rc = send(s, "CLOSE");
if(rc == ECANCELED) return;
return;
}

int main(void) {
socket_t s = create_connected_socket();
bundle_t b = bundle();
bundle_go(b, worker(s));
sleep(60);
bundle_cancel(b, 10);
return 0; }

Note how this interacts with the server-initiated graceful shutdown. The worker coroutine replies to CLOSE messages in a straightforward way. On the other hand, when the server is being shut down it stops accepting new connection (left as an exercise for the reader) and gives existing connections 10 seconds to exit. If they don't they'll get hard-canceled.

There's a missing piece of course. When server initiates the graceful shutdown, the worker should send unsolicited CLOSE message to the client to inform it that it wants to shut down the connection and then it should wait for CLOSE in reply.

So here's the full implementation:

coroutine void worker(socket_t s, channel_t ch) {
while(1) {
message_t msg;
int rc = recv_from_socket_or_channel(s, ch, &msg);
if(rc == ECANCELED) return;
if(rc == FROM_CHANNEL) goto server_initiated_shutdown;
if(rc == FROM_SOCKET) {
if(msg == "CLOSE") goto peer_initiated_shutdown;
}
rc = send(s, "Hello, world!");
if(rc == ECANCELED) return;
}
server_initiated_shutdown:
int rc = send(s, "CLOSE");
if(rc == ECANCELED) return; // drop all incoming messages
until CLOSE is encountered
while(1) {
message_t msg;
rc = recv(s, &msg);
if(rc == ECANCELED) return;
if(msg == "CLOSE") break;
}
return;
peer_initiated_shutdown:
int rc = send(s, "CLOSE");
if(rc == ECANCELED) return;
return;
}

int main(void) {
socket_t s = create_connected_socket();
channel_t ch = channel();
bundle_t b = bundle();
bundle_go(b, worker(s, ch));
sleep(60);
send(ch, "STOP"); // ask for graceful shutdown
bundle_cancel(b, 10); // give it at most 10 seconds to finish
return 0;
}Graceful shutdown is composable

Now that we are done with examples, let's look at the thing from a conceptual point of view.

I've already mentioned that parent coroutine doesn't care about specifics of the shutdown mechanism of the child coroutine. It doesn't care if the child has infinite loops, it doesn't care whether it has application-level support for graceful shutdown and so on.

But there's one more thing it doesn't care about. Guess what. It doesn't care if the child has children of its own.

It also doesn't care whether it is already in the process of graceful shutdown, initiated, for example, by a remote peer, by an internal failure or maybe by some kind of coroutine-wide timeout.

This enforces strict isolation between coroutines and makes the graceful shutdown composable.

Let's illustrate the point on a concrete example:

coroutine void nested_worker(message_t msg) {
// process the message here
}

coroutine void worker(socket_t s, channel_t ch) {
bundle_t b = bundle();
while(1) {
message_t msg;
int rc = recv_from_socket_or_channel(s, ch, &msg);
if(rc == ECANCELED) goto hard_cancellation;
if(rc == FROM_CHANNEL) goto graceful_shutdown;
if(rc == FROM_SOCKET) {
bundle_go(b, nested_worker(msg));
}
rc = send(s, "Hello, world!");
if(rc == ECANCELED) goto hard_cancellation;
}
graceful_shutdown:
rc = bundle_cancel(b, 20); // cancel the nested workers with 20
second grace period
if(rc == ECANCELED) return;
return;
hard_cancellation:
rc = bundle_cancel(b, 0); // cancel the nested worker
immediately
if(rc == ECANCELED) return;
return;
}

int main(void) {
socket_t s = create_connected_socket();
channel_t ch = channel();
bundle_t b = bundle();
bundle_go(b, worker(s, ch));
sleep(60);
send(ch, "STOP"); // ask for graceful shutdown
bundle_cancel(b, 10); // give it at most 10 seconds to finish
return 0;
}

Note that when the server is being shut down it allows only for 10 second grace period. However, worker(), when being gracefully shut down gives its children 20 second grace period. That sounds like it's going to be a headache, but it turns out it is not.

When the 10 second grace period granted by the main() function expires it hard-cancels the worker(). The worker may be still waiting for its children to finish. But we don't care. Hard-cancelling the worker() means that "bundle_cancel(b, 20)" will immediately hard-cancel all the nested workers and exit with ECANCELED error (double-check the semantics of bundle_cancel() as proposed above).

All in all, we get the intuitive behaviour: The server shuts down in 10 seconds and hard-cancels all the children and grand-children that were still running at the time.

Implications for higher level languages

Tt's obvious that in higher level languages a lot of the boilerplate code can be avoided. For example, with "Canceled" exception all the "if(rc == ECANCELED) return;" lines just go away.

However, how to wrap the entire thing into a higher-level level constructs, such as automatic scopes, is a problem yet to be solved.



Discuss

Why didn't Agoric Computing become popular?

Новости LessWrong.com - 16 февраля, 2019 - 09:19
Published on February 16, 2019 6:19 AM UTC

I remember being quite excited when I first read about Agoric Computing. From the authors' website:

Like all systems involving goals, resources, and actions, computation can be viewed in economic terms. This paper examines markets as a model for computation and proposes a framework--agoric systems--for applying the power of market mechanisms to the software domain. It then explores the consequences of this model and outlines initial market strategies.

Until today when Robin Hanson's blog post reminded me, I had forgotten that one of the authors of Agoric Computing is Eric Drexler, who also authored Comprehensive AI Services as General Intelligence, which has stirred a lot of recent discussions in the AI safety community. (One reason for my excitement was that I was going through a market-maximalist phase, due to influences from Vernor Vinge's anarcho-captalism, Tim May's crypto-anarchy, as well as a teacher who was a libertarian and a big fan of the Austrian school of economics.)

Here's a concrete way that Agoric Computing might work:

For concreteness, let us briefly consider one possible form of market-based system. In this system, machine resources-storage space, processor time, and so forth-have owners, and the owners charge other objects for use of these resources. Objects, in turn, pass these costs on to the objects they serve, or to an object representing the external user; they may add royalty charges, and thus earn a profit. The ultimate user thus pays for all the costs directly or indirectly incurred. If the ultimate user also owns the machine resources (and any objects charging royalties), then currency simply circulates inside the system, incurring computational overhead and (one hopes) providing information that helps coordinate computational activities.

When later it appeared as if Agoric Computing wasn't going to take over the world, I tried to figure out why, and eventually settled upon the answer that markets often don't align incentives correctly for maximum computing efficiency. For example, consider an object whose purpose is to hold onto some valuable data in the form of a lookup table and perform lookup services. For efficiency you might have only one copy of this object in a system, but that makes it a monopolist, so if the object is profit maximizing (e.g., running some algorithm that automatically adjusts prices so as to maximize profits) then it would end up charging an inefficiently high price. Objects that might use its services are incentivized to try to do without the data, or to maintain an internal cache of past data retrieved, even if that's bad for efficiency.

Suppose this system somehow came into existence anyway. A programmer would likely notice that it would be better if the lookup table and its callers were merged into one economic agent which would eliminate the inefficiencies described above, but then that agent would itself still be a monopolist (unless you inefficiently maintained multiple copies of it) so then they'd want to merge that agent with its callers, and so on.

My curiosity stopped at that point and I went on to other interests, but now I wonder if that is actually a correct understanding of why Agoric Computing didn't become popular. Does anyone have any insights to offer on this topic?



Discuss

Pedagogy as Struggle

Новости LessWrong.com - 16 февраля, 2019 - 05:12
Published on February 16, 2019 2:12 AM UTC

[See also: On Giving Advice and Recognizing vs Generating]

So this quarter, I’ve been tutoring for an undergraduate computer science course, and one interesting thing that I’ve revised is how I think about teaching/learning.

Let me try to illustrate. Here’s a conversation that happened between some staff and the professor one day; it’s about what advice to give students when testing their code.

Tutor A: “I think that we should remind students to be very thorough about how they approach testing their code. Like, so far, I’ve been telling them that every line of code they write should have a test.”Tutor B: “Hmmm, that might backfire. After all, lots of those lines of code are in functions, and we really just want them to make sure that their functions are working as intended. It’s not really feasible to test lines of code inside a function.”Professor: “Actually…that seems like a fine outcome. We want students to be thinking about testing, and I’d actually be very excited if someone came up to me and asked how to test for what goes on inside a function…”

This was surprising to me because most discussions I’ve previously had about student learning had focused on how to reduce confusion for the students. But in this scenario, the professor was fine with it happening; if anything, they seemed pleased that the concepts, when taken to their extremes, incited more questions.

And this general concept, of giving students something that's Not The Answer, in an effort to move them closer to The Answer seems to show up in several other areas.

For example, from my shallow understanding of how kōans work in Zen Buddhism, there’s a similar mechanic going on. The point of a kōan isn’t to develop a fully satisfactory answer to the question it asks, but to wrestle with the strangeness / paradoxical nature of the kōan. The auxiliary things that happen along the way, en route to the answer is really what the kōan is about.

To be clear, the thing I’m trying to point at isn’t just giving people practice problems, the way that we already do for math or physics.

Rather, I’m imagining things like people purposefully writing incorrect / confusing material, such that it prompts students to ask more questions. I guess there’s already a good amount of people in the rationality space who write abstrusely, but I wonder how many are doing so for pedagogical reasons? Seeing as noticing confusion is an oft-cited useful skill, I think that there’s more to do here, especially if you’re upfront about how some of the material is going to be incomplete, and maybe sometimes even wrong.

It seems like there’s a slew of related useful skills here. Several times, in math class, for example, I’ve had my instructor make an error while doing some proof. And now I’m confused about how we got from step N to step N+1. Sometimes I figure that they’re just wrong, and I write what I think is correct. And sometimes I get scared that I’m the one who doesn’t understand.

But this whole process raises good questions. What if I hadn’t noticed that something was wrong? What does that say about my understanding? When I do notice that something is wrong, how do I know if it’s me or the other person?

This all seems applicable outside of a pedagogical context.



Discuss

How the MtG Color Wheel Explains AI Safety

Новости LessWrong.com - 16 февраля, 2019 - 02:42
Published on February 15, 2019 11:42 PM UTC

Duncan Sabien has a post titled How the ‘Magic: The Gathering’ Color Wheel Explains Humanity. Without the context of that post, or other experience with the MtG color wheel, this post will probably not make sense. This post may not make sense anyway. I will use a type of analysis that is sometimes used to talk about humans (and often criticized even when used for humans), but rarely used in any technical subjects. I will abstract so far that everything will start to look like (almost) everything else. I will use wrong categories and stretch facts to make them look like they fit into my ontology.

I will describe 5 clusters of ideas in AI and AI safety, which correspond to the 5 Magic the Gathering colors. Each color will also come along with a failure mode. For each failure mode, the two opposing colors (on the opposite side of the pentagon) form a collection of tools and properties that might be useful for fighting that failure mode.

Mutation and Selection

So I want to make an AI that can accomplish some difficult task without trying to kill me (or at least without succeeding in killing me). Let's consider the toy task of designing a rocket. First, I need a good metric of what it means to be a good rocket design. Then, I need to search over all the space of potential rocket designs, and find one that scores well according to my metric. I claim that search is made of two pieces: Mutation and Selection, Exploration and Optimization, or Babble and Prune.

Mutation and Selection are often thought of as components of the process of evolution. Genes spin off slightly modified copies over time through mutation, and selection repeatedly throws out the genes that score badly according to a fitness metric (that is itself changing over time). The result is that you find genes that are very fit for survival.

However, I claim that mutation and selection are much more general than evolution. Gradient descent is very close to (a speed up of) the following process. Take an initial point called your current best point. Sample a large number of points within an epsilon ball of the current best point. Select the best of the sampled points according to some metric. Call the selected point the new current best point, and repeat.

I am not trying to claim that machine learning is simple because it is mostly just mutation and selection. Rather, I am trying to claim that many of the complexities of machine learning can be viewed as trying to figure out how to do mutation and selection well.

Goodharting is a problem that arrises when extreme optimization goes unchecked, especially when the optimization is much stronger than the process that chose the proxy that was being optimized for.

Similarly, unchecked exploration can also lead to problems. This is especially true for systems that are very powerful, and can take irreversible actions that have not been sufficiently optimized. This could show up as a personal robot accidentally killing a human user when it gets confused, or as a powerful agent exploring into taking actions that destroy themselves. I will refer to this problem as irreversible exploration.

Consequentialism

The process described above is how I want to find my rocket design. The problem is that this search is not stable or robust to scale. It is setting up an internal pressure for consequentialism, and if that consequentialism is realized, it might interfere with the integrity of the search.

By consequentialism, I am not talking about the moral framework. It is similar to the moral framework, but it should not have any connotations of morality. Instead I am talking about the process of reasoning about the consequences of potential actions, and choosing actions based on those consequences. Other phrases I may use for to describe this process include agency, doing things on purpose, and back-chaining.

Let's go back to the evolution analogy. Many tools have evolved to perform many subgoals of survival, but one of the most influential tools to evolve was the mind. The reason the mind was so useful was because it was able to optimize on a tighter feedback cycle than evolution itself. Instead of using genes that encode different strategies for gathering food and keeping the ones that work, the mind can reason about different strategies for gathering food, try things, see which ways work, and generalize across domains, all within a single generation. The best way for evolution to gather food is to create a process that uses a feedback loop that is unavailable to evolution directly to improve the food gathering process. This process is implemented using a goal. The mind has a goal of gathering food. The outer evolution process need not learn by trial and error. It can just choose the minds, and let the minds gather the food. This is more efficient, so it wins.

It is worth noting that gathering food might only be a subgoal for evolution, and it could still be locally worthwhile to create minds that reason terminally about gathering food. In fact, reasoning about gathering food might be more efficient than reasoning about genetic fitness.

Mutation and selection together form an outer search process that finds things that score well according to some metric. Consequentialism is a generic way to score well on any given metric: choose actions on purpose that score well according to that metric (or a similar metric). It is hard to draw the line between things that score well by accident, and things that score well on purpose, so when we try to search over things that score well by accident, we find things that score well on purpose. Note that the consequentialism might itself be built out of a mutation and selection process on a different meta level, but the point is that it is searching over things to choose between using a score that represents the consequences of choosing those things. From the point of view of the outer search process, it will just look like a thing that scores well.

So a naive search trying to solve a hard problem may find things that are themselves using consequentialism. This is a problem for my rocket design task, because I was trying to be the consequentialist, and I was trying to just use the search as a tool to accomplish my goal of getting to the moon. When I make consequentialism without being very careful to ensure it is pointed in the same direction as I am, I create a conflict. This is a conflict that I might lose, and that is the problem. I will refer to consequentialism arising within a powerful search process as inner optimizers or daemons.

Boxing and Mildness

This is where AI safety comes in. Note that this is a descriptive analysis of AI safety, and not necessarily a prescriptive one. Some approaches to AI safety attempt to combat daemons and irreversible exploration through structure and restrictions. The central example in this cluster is AI boxing. We put the AI in a box, and if it starts to behave badly, we shut it off. This way, if a daemon comes out of our optimization process, it won't be able to mess up the outside world. I obviously don't put too much weight in something like that working, but boxing is a pretty good strategy for dealing with irreversible exploration. If you want to try a thing that may have bad consequences, you can spin up a sandbox inside your head that is supposed to model the real world, you can try the thing in the sandbox, and if it messes things up in your sandbox, don't try it in the real world. I think this is actually a large part of how we can learn in the real world without bad consequences. (This is actually a combination of boxing and selection together fighting against irreversible exploration.)

Other strategies I want to put in this cluster include formal verification, informed oversight and factorization. By factorization, I am talking about things like factored cognition and comprehensive AI services. In both cases, problems are broken up into small pieces by a trusted system, and the small pieces accomplish small tasks. This way, you never have to run any large untrusted evolution-like search, and don't have to worry about daemons.

The main problem with things in this cluster is that they likely won't work. However, if I imagine they worked too well, and I had a system that actually had these types of restrictions making it safe throughout, there is still a (benign) failure mode which I will refer to as lack of algorithmic range. By this, I mean things like making a system that is not Turing complete, and so can't solve some hard problems, or a prior that is not rich enough to contain the true world.

Mildness is another cluster of approaches in AI safety, which is used to combat Daemons and Goodhart. Approaches in this cluster include Mild Optimization, Impact Measures, and Corrigibility. They are all based on the fact that the world is already partially optimized for our values (or vice versa), and too much optimization can destroy that.

A central example of this is quantilization, which is a type of mild optimization. We have a proxy which was observed to be good in the prior, unoptimized distribution of possible outcomes. If we then optimize the outcome according to that proxy, we will go to a single point with a high proxy value. There is no guarantee that that point will be good according to the true value. With quantilization, we instead do something like choose a point at random, according to the unoptimized distribution from among the top one percent of possible outcomes according to the proxy. This allows us to transfer some guarantees from the unoptimized distribution to the final outcome.

Impact measures are similarly only valuable because the do-nothing action is special in that it is observed to be good for humans. Corrigibility is largely about making systems that are superintelligent without being themselves fully agentic. We want systems that are willing to let human operators fix them, in a way that doesn't result in optimizing the world for being the perfect way to collect large amounts of feedback from microscopic humans. Finally, note that one way to stop a search from creating an optimization daemon is to just not push it too hard.

The main problem with this class of solutions is a lack of competitiveness. It is easy to make a system that doesn't optimize too hard. Just make a system that doesn't do anything. The problem is that we want a system that actually does stuff, partially because it needs to keep up with other systems that are growing and doing things.



Discuss

Some disjunctive reasons for urgency on AI risk

Новости LessWrong.com - 15 февраля, 2019 - 23:50
Published on February 15, 2019 8:43 PM UTC

(This has been sitting in my drafts folder since August 2017. Robin Hanson's recent How Lumpy AI Services? made me think of it again. I'm not sure why I didn't post it back then. I may have wanted to add more reasons, details and/or citations, but at this point it seems better to just post it as is. Apologies to those who may have come up with some of these arguments earlier.)

Robin Hanson recently wrote, "Recently AI risk has become something of an industry, with far more going on than I can keep track of. Many call working on it one of the most effectively altruistic things one can possibly do. But I’ve searched a bit and as far as I can tell that foom scenario is still the main reason for society to be concerned about AI risk now." (By "foom scenario" he means a local intelligence explosion where a single AI takes over the world.) In response, I list the following additional reasons to work urgently on AI alignment.

  1. Property rights are likely to not hold up in the face of large capability differentials between humans and AIs, so even if the intelligence explosion is likely global as opposed to local, that doesn't much reduce the urgency of working on AI alignment.

  2. Making sure an AI has aligned values and strong controls against value drift is an extra constraint on the AI design process. This constraint appears likely to be very costly at both design and run time, so if the first human level AIs deployed aren't value aligned, it seems very difficult for aligned AIs to catch up and become competitive.

  3. AIs' control of the economy will grow over time. This may happen slowly in their time frame but quickly in ours, leaving little time to solve value alignment problems before human values are left with a very small share of the universe, even if property rights hold up.

  4. Once we have human-level AIs and it's really obvious that value alignment is difficult, superintelligent AIs may not be far behind. Superintelligent AIs can probably find ways to bend people's beliefs and values to their benefit (e.g., create highly effective forms of propaganda, cults, philosophical arguments, and the like). Without an equally capable, value-aligned AI to protect me, even if my property rights are technically secure, I don't know how I would secure my mind.



Discuss

So you want to be a wizard

Новости LessWrong.com - 15 февраля, 2019 - 18:43
Published on February 15, 2019 3:43 PM UTC

This is a link post to a recent talk by Julia Evans, popular programming blogger, called "So you want to be a wizard". I'm linking it on LW because I think it's one of the better takes I've seen on how to "apply rationality" to programming. A few interesting examples of things she discusses that I think will appeal to the LW audience:

  • How she built a game to get better at Fermi estimate of how long different computer operations should take.
  • How believing that she can achieve a gears-level understanding of things she doesn't yet know much about makes her a better programmer and debugger.
  • How she uses "premortem"s to predict what will go wrong with projects.

While I suspect none of the above will be knew to much of the LW audience, I think she does a good job of connecting each strategy she discusses to real-world examples that help you understand what it would actually look like to apply them.



Discuss

Cooperation is for Winners

Новости LessWrong.com - 15 февраля, 2019 - 17:58
Published on February 15, 2019 2:58 PM UTC

Cross-posted from Putanumonit.

My last post was about learning to compete well from sports. But why compete at all? After all, I’m the one who wrote that winning is for losers, and we should avoid getting sucked into zero-sum contests.

Competing well doesn’t necessarily mean having to compete more, for one thing. Instead, it does allow you to choose where to compete. More importantly, sports-like competitions create prestige hierarchies (as opposed to dominance hierarchies). Climbing those is not intended to make you a fearsome boss but a valuable ally, one that others want to cooperate with. Getting to cooperation often requires winning competitions.

Wei Dai commented on the previous post:

More seriously, these days I think of competition as more of a problem than a solution. Some of the most important x-risks (e.g., advanced AI) are x-risks mainly because of competitive dynamics. If people weren’t competing for the prestige/power/money of being first to create AGI or to make advances in AI in general, we’d be able to solve AI safety problems at leisure.

How does one get to solve AI safety problems at their leisure, in a cooperative environment? Obviously, by working at the Machine Intelligence Research Institute.

But to work at MIRI one must first get a job at MIRI, and since MIRI has few spots and many applicants this means outcompeting other candidates for the job. MIRI itself is funded by donations, so it has to outcompete other non-profits for grants.

Hopefully, both competitions are conducted in a sportsmanlike way by demonstrating research achievements instead of by sabotaging competitors. But it’s still a competition – applying for jobs or grans is stressful and entails a high risk of failure, often due to circumstances outside one’s control.

Organizations have to manage very carefully the process of directing the winners of a competitive selection process to an ultimately cooperative endeavor. Here’s a wonderful post looking at this issue in two almost identical situations: chicken coops and academia.

So in the late 70s and early 80s there were active breeding programs to produce hens who laid more eggs. Take the highest egg layers in each generation and let them be the only ones that breed to produce the next generation. […]Only one problem. High productivity egg-laying is associated with aggression – indeed the highest egg layers are basically the ones that beat up the other hens in the coop with them and capture the most resources. This led to a choice between two routes. If nothing was done, total egg-laying went down as there were too many injuries and too much mortality. The alternative was to continue the selection process but to supplement it by engaging in what many would regard as barbaric practices – beak trimming, declawing etc to prevent injuries from the aggression.Then in the 1980s people got the idea to use group selection. Instead of picking individuals that were most productive, they selected entire hen houses that were most productive to produce the next generation. […]Now, when I tell fellow faculty this story, many immediately respond all on their own “that’s like the selection being imposed for high-performing individual faculty members at the cost of their departments and the larger mission”. Are Dean’s and higher ups committing the same error as hen breeders and relying too much on a one-dimensional individual level selection that results in destroying overall productivity? Are we producing professors who need to be declawed and have their beaks trimmed to maintain productivity (or less figuratively is departmental collegiality suffering as a result)?

Having to compete for the opportunity to cooperate is how dating works too. Your goal is to reach cooperation (for a night or for a lifetime) with the person you’re pursuing. But you’re competing with everyone else: competing to be noticed, to go on a date, to earn their commitment. A lot of people I know miss out on relationships because they’re afraid of losing the competition – getting rejected.

They are ways to sneak around this fear, but they’re not optimal. Guys can use Tinder, which is set up to make “not being swiped right on” feel as little as possible like rejection. But Tinder is superficial, has poor norms around it, and only really works for the 20% of men who are most attractive.

Women can just choose not to ask anyone out at all, whether online or in physical spaces. But that limits a woman’s choices to her OkCupid inbox or the men who hit on her, which selects for men who are aggressive flirters instead of the other attributes that are much more important in a boyfriend. I think that most women can do much better by reaching out themselves to better guys than their inbox provides.

Getting over the fear of rejection is hard, but here again, sports can show the way. Losing a tennis match is less bad but has a lot in common with getting turned down in a romantic setting. In the moment, it feels terrible. But after a while, you reflect that losing the match/date is not a wholesale condemnation of your worth as a person or your social standing. It tells you where your expectations should be set in the future and, most crucially, how you should improve.

There are competitive elements even in things that don’t look like a competition at all, like blogging. I enjoy reading all across the rationalist blogosphere. We build on each other’s ideas, link to each other’s posts, and sustain a shared community. But nobody (with one exception) has time to read all the blogs, so we are also competing for reader attention, links, and space on Scott’s blogroll.

This doesn’t mean that I post with the intention of stealing readers away from John or Sarah. But it implies that I think that my blog is competitive with theirs, that it’s worth someone’s time to read Putanumonit on their restroom break. Blogging also means I’m willing to take an occasional L: writing something stupid or political or just poorly written and being told by people that I wasted their restroom break and they’re not getting it back. Hopefully, these losses make Putanumonit better.

Only 20% of LessWrong participants active enough to fill out a survey have ever written a post. When I ask my friends who are in the remaining 80%, they don’t say it’s because they have literally nothing to contribute. They’re afraid that what they write will not be good, and if it’s good it won’t be original, and if it’s good and original then Scott will write about the same thing better in a few weeks anyway. They don’t like the karma system of upvotes and downvotes. In short: they know it’s a competition, and they don’t want to compete.

No one gets to hear their ideas, enjoy their writing, or correct their mistakes. If cooperation is happening, they’re not a part of it.

Winning is for Losers was about one sort of mistake people can make: they see a hierarchy of points and rankings (e.g., the entire American education system) and dedicating all their resources to winning instead of asking whether the game is even worth it. But I’m seeing the opposite mistake more and more, people seeing competition and flinching away instinctively. They tell themselves that they’re the good guys, cooperating instead of fighting.

But if winning is for losers, cooperation is for the winners.



Discuss

Страницы

Подписка на LessWrong на русском сбор новостей