Вы здесь

Новости LessWrong.com

Подписка на Лента Новости LessWrong.com Новости LessWrong.com
A community blog devoted to refining the art of rationality
Обновлено: 1 час 7 минут назад

Paper on qualitative types or degrees of knowledge, with examples from medicine?

15 июня, 2019 - 03:31
Published on June 15, 2019 12:31 AM UTC

There's a paper (or essay or something similar) I can remember reading that I've been struggling to find for some time now. It described different qualitative types of knowledge and used various examples from medicine, e.g. the difference between various kinds of trauma or emergency care where we know exactly what a problem is and can fix it (almost) exactly/completely versus something like obesity where we basically don't know (at least in the sense of commonly shared knowledge) much of anything about what exactly the problem even is.

Any help in uncovering the paper would be greatly appreciated!



Discuss

Recommendation Features on LessWrong

15 июня, 2019 - 03:23
Published on June 15, 2019 12:23 AM UTC

Today, we're rolling out several new beta features on the home page, which display recommended posts to read. The first is a Continue Reading section: if you start reading a multi-post sequence, it will suggest that you read another post from that sequence. The second is a From the Archives section, which recommends highly-rated posts that you haven't read, from all of LessWrong's history.

To use these features, please ensure you are logged-in.


Continue Reading

Sequences are a mechanism on LessWrong for organizing collections of related posts. Anyone can create a Sequence from the library page. If you write a series of posts and add them to a Sequence, they will have Previous and Next links at the top and bottom; if you create a Sequence out of posts by other authors, they will have Previous and Next links for readers who came to them via the Sequence.

When you are logged in and read a post from a Sequence, the first unread post from that sequence will be added as a recommendation in the Continue Reading section, like this:

If you decide not to finish, hover over the recommendation and click the X button to dismiss the recommendation. For logged-out users, the Continue Reading section is replaced with a Core Reading section, which suggests the first posts of Rationality: A-Z, The Codex, and Harry Potter and the Methods of Rationality.


From the Archives

The home page now has a From the Archives section, which displays three posts randomly selected from the entire history of LessWrong. Currently, a post can appear in this section if:

(1) you've never read it while logged in (including on old-LessWrong),

(2) it has a score of at least 50, and

(3) it is not in the Meta section, or manually excluded from recommendation by moderators. (We manually exclude posts if they aged poorly in a way that wouldn't be captured by votes at the time -- for example, announcements of conferences that have already happened, and reporting of studies that later failed to replicate.)

Currently, if a post is eligible to appear in the From the Archives section, it will appear with probability proportional to the cube of its score.


Why These FeaturesLessWrong as a Repository for "long content"

Gwern’s about page has influenced me a lot in thinking about the future of LessWrong. Gwern uses the following quote:

The Internet is self destructing paper. A place where anything written is soon destroyed by rapacious competition and the only preservation is to forever copy writing from sheet to sheet faster than they can burn. If it’s worth writing, it’s worth keeping. If it can be kept, it might be worth writing…If you store your writing on a third party site like Blogger, Livejournal or even on your own site, but in the complex format used by blog/wiki software du jour you will lose it forever as soon as hypersonic wings of Internet labor flows direct people’s energies elsewhere. For most information published on the Internet, perhaps that is not a moment too soon, but how can the muse of originality soar when immolating transience brushes every feather?-- Julian Assange (“Self destructing paper”, 5 December 2006)

Most of the content on the internet is designed to be read and forgotten in a very short period of time. Existing discussion platforms like Reddit and many forums even close threads automatically after a certain period of time to ensure that all discussion centers around the most recent activity on the site.

One of the goals I have with LessWrong is to be a place where we can build on each other's ideas over the course of multiple decades, and where if you show up, you engage with the content on the site in a focused way, more similar to a textbook than a normal internet forum. And like our best textbooks, good introductions into core topics tend to stand the test of time quite well (e.g. the Feynman Lectures, which is still one of, if not the best introduction to physics even 60 years later).

The Continue Reading system is a key part of that goal, because it makes it much easier to use the site as a tool for focused study, since continuing to read the sequences you started is now one of the core actions on the site.

The recommendation system is also a key part of that goal, because it creates a way to discover content from the complete history of LessWrong, instead of just the last week, which strikes me as a necessary component to make collective intellectual progress that can span multiple decades. The best things for almost anyone to read have very likely not been written in the past week.


LessWrong as a Nudge

Continue Reading is a nudge to encourage reading a few long things, rather than a lot of short things. Longer writing allows topics to be explored in greater depth, and also enables more explicit decisions about what to read, since making one decision per sequence is a lot less work than making one decision per post.

From the Archives is a nudge to read better posts. When we choose what to read, there is often a recency bias; the best writing of the past ten years will be better, on average, than the best writing of the past week, but active conversations will focus on the most recent things. A good information diet contains a mix of recent writing and of timeless classics; by putting From the Archives on the home page, we are saying, on the margin, read more of the past.

I also think that From the Archives will have a positive effect on what people write on LessWrong. There are many good ideas in LessWrong's archives, waiting to be built upon, which haven't received attention recently; my hope is that recommendations of older posts will inspire more good writing.


Caveat: Addictiveness

Reading the latest posts on LessWrong is finite; you will run out of interesting-seeming recent posts, which creates a natural limit on time spent. Reading posts from the archives is effectively infinite; LessWrong's archives are deep enough that you probably won't ever run out of things to read. These new recommendation features therefore offer an opportunity to spend a lot of time by accident. We'd rather you make a deliberate decision about what and how much to read on LessWrong.

If you find you're spending more time reading LessWrong's recommended posts than you want, or expect that you would spend more time than you want to, you can turn off the Continue Reading and/or From the Archives sections by clicking the gear icon. (This requires that you be logged in to save the setting.)


These Are Beta

These features are beta, and probably have bugs. The From the Archives post selection algorithm we're currently using (based on post scores) seems to work okay for now, but scores are heavily affected by post visibility as well as quality, so some posts (especially imported posts) aren't being recommended that should be, and post scores will suffer a positive-feedback effect where being recommended causes posts to be recommended more. So, we expect to rely less on the raw post score in the future, and more on other evaluation mechanisms such as asking users for retrospective evaluations of posts they've previously read, read-completion and clickthrough rates, vote-to-views ratios, etc. The recommendation algorithm is likely to become too complex to straightforwardly explain, though its workings will always be knowable to those willing to dive into the source code.



Discuss

Discourse Norms: Moderators Must Not Bully

15 июня, 2019 - 02:22
Published on June 14, 2019 11:22 PM UTC

One of the absolute worst things that can happen to a civic/public community online is for moderators to be bullies or for moderators to take the side of the bullies. Once that happens, the community is at grave risk of ceasing to be a public community and instead embracing cliquism. If the moderators enforce the will of their friends rather than good discussion norms, the space is no longer going to be a space for good discussion but rather one for a certain friend group.

The most common way I've seen this happen goes something like this. A newcomer with locally unusual ideas joins the community. Conflict between their ideas and the more established norms arises. Because these ideas are unpopular, people push back against them, often in mean or uncharitable ways. If left unchecked, the newcomer may soon become a target of bullying and sniping. [1]

At this point, moderators need to intervene in favor of the newcomer, because mean and uncharitable behavior shouldn't be allowed to stand in a civic/public space, even if it's towards ideas that are locally unpopular. Moderation is needed to rein in the attacks and keep things civil and productive. However, in practice what often ends up happening is that the moderators intervene against the newcomer, enforcing the local social hierarchy rather than good discussion norms.

This is toxic to a civic/public space and, if left unchecked, drives out views or discussion styles other than those that are locally popular.

One potential antidote to this sort of behavior is holding moderators to significantly higher standards than users. If a moderator and a user are in an angry, insulting argument with one another, the moderator should be removed from moderation or at minimum recuse themselves. If a moderator posts insults against another user - especially someone who isn't popular - they are at fault and should apologize or be removed from moderation.

Yes, this is a harsh standard. Yes, this means that being a moderator limits what you can say in some circumstances. But that's what you need to do to keep the bullies at bay, and ultimately, being a moderator shouldn't be a position of power but rather a position of responsibility.

Lastly, I want to point out that it's totally fine for a space to exist for a friend group or for those who agree with certain perspectives - and for those sorts of spaces, it's entirely fine for moderators to enforce local social norms or locally popular opinions! However, there's a big difference between that and a civic/public space, and if you're going for civic/public norms a higher standard is needed of moderators.


[1] This obviously doesn't apply to Nazis and the like, which should IMO be banned outright.

[2] Note that footnote [1] should not be construed as an excuse to go around calling everyone you don't like a Nazi in hopes of getting them banned, and such rules should be clearly articulated beforehand - the intent is merely to point out that you can have a civic/public space that still prevents certain objectionable content.





Discuss

SSC Sacramento Meetup

15 июня, 2019 - 02:08
Published on June 14, 2019 11:08 PM UTC



Discuss

Welcome to LessWrong!

14 июня, 2019 - 22:42
Published on June 14, 2019 7:42 PM UTC

LessWrong was founded in 2009 and relaunched in 2018 with a new codebase and full-time team.

We are a community dedicated to improving our reasoning and decision-making. We seek to hold true beliefs and to be effective at accomplishing our goals. More generally, we work to develop and practice the art of human rationality.[1]

To that end, LessWrong is a place to 1) develop and train rationality, and 2) apply one’s rationality to real-world problems.

LessWrong serves these purposes with its library of rationality writings, community discussion forum, open questions research platform, and community page for in-person events.

To get a feel for what LessWrong is about, here’s a selection of LessWrong posts which might appeal to you:

Check out this footnote[2] below the fold for samples of posts about AI, science, philosophy, history, communication, culture, self-care, and more.

If LessWrong seems like a place for you, we encourage you to become familiar with LessWrong’s philosophical foundations. Our core readings can be be found on the Library page.

We especially recommend:

Find more details about these texts in this footnote[3]

For further getting started info, we direct you to LessWrong’s FAQ. Lastly, we suggest you create an account so you can vote, comment, save your reading progress, get tailored recommendations, and subscribe to our latest and best posts. Once you've done so, please say hello on our latest welcome thread!

 

Related Pages

LessWrong FAQ

A Brief History of LessWrong

Team

  1. Rationality is a term which can have different meanings to different people. On LessWrong, we mean something like the following:  

    • Rationality is thinking in ways which systematically arrive at truth.
    • Rationality is thinking in ways which cause you to systematically achieve your goals.
    • Rationality is trying to do better on purpose.
    • Rationality is reasoning well even in the face of massive uncertainty.
    • Rationality is making good decisions even when it’s hard.
    • Rationality is being self-aware, understanding how your own mind works, and applying this knowledge to thinking better.

    See also: What Do We Mean By "Rationality"?, Why Spock is Not Rational, What are the open problems in Human Rationality? ↩︎

  2. More sample posts from LessWrong:  

    ↩︎
  3. More details about our core readings:

    Rationality: A-Z is a deep exploration of how human minds can come to under the world they exist in - and all the reasons they so commonly fail to do. The comprehensive work:

    The Codex collects Scott Alexander's writings on good reasoning, what we can learn from the institution of science, and the different ways society could be and has been organized. Exemplary essays include:

    Harry Potter and the Methods of Rationality started as a side project of Eliezer’s and grew to be one of the most highly rated Harry Potter fanfictions of all time and an an excellent primer on rationality. Eliezer imagined an alternate-universe Harry Potter who grew up with loving adopted parents, one of them an Oxford scientist. He enters the wizarding world with a scientific and rationalist mindset. Click here to read HPMOR through LessWrong or try the audiobook. ↩︎



Discuss

On Freud's examination of the Uncanny in Literature

14 июня, 2019 - 20:40
Published on June 14, 2019 5:40 PM UTC


It is often mentioned that Freud’s Psychoanalytical theory has influenced literature very considerably. Regarding stories that aspire to cause dread, his article on the Uncanny in Literature provides good insight – particularly his interpretation of “The Sandman”,  E.T.A. Hoffmann’s bleak and magical story of fiery circles, monsters and doppelgangers,  is worth mentioning...

The Sandman, the eponymous fiend of this story, is both a magical entity (a flying beaked demon that abducts little children and uses their eyes so as to feed his own offspring) and a dyad of mysterious men: an ominously looking old lawyer, Coppelius, and a merchant of eyeglasses, called Giuseppe Coppola. Both names etymologically are derived from the Italian word for “eye”, and their links to the monstrous Sandman of legend do not stop there.

The Lawyer Coppelius

Coppelius is a very unpleasant-looking man, who seems to despise children. The protagonist of the story, the student Nathaniel, recalls how he hated and feared Coppelius ever since he was a boy. While at first the antipathy was caused by disgust at how the old man looked, as well as due to the antagonistic attitude he consistently demonstrated when invited to Nathaniel's family home by his father, later on Nathaniel comes to regard Coppelius as the actual murderer of his father – whose death took place during a chemical, possibly alchemy-related, experiment.  

Coppelius had already been fused in Nathaniel's mind with the mythical Sandman – the child’s governess was reckless enough to fill his mind with tales of horror, and it should be noted that she was the one who first suggested that his father’s night-time guest was the Sandman. Nathaniel always loved stories of mystery and the macabre, so the repugnant and terrifying figure of the winged and beaked Sandman soon assumed a central position in his personal pantheon of ghouls.

Coppelius manages to escape after the apparent accident that killed Nathaniel's father, and Nathaniel will only see him again – or at any rate believe he saw him – years later, while studying far away from his home city.

The optometrist Giuseppe Coppola

Coppola first appears to the student Nathaniel wishing to sell him some of his wares – lenses, small telescopes and eyeglasses. Nathaniel immediately feels repulsed, because the visage of this merchant is uncannily similar to the lawyer Coppola's. At length, he decides to buy one of the elegant lenses, which he will soon put to use so to have a better look at the object of his desires: the university professor’s beautiful daughter, Olympia.

Unfortunately for Nathaniel the gained ability to have a closer look at Olympia - a girl normally isolated and confined to her room and only making her appearance by the window - results to dangerous infatuation and the dreaded sense that something is not quite right with her... For the rest of the story he will persistently attempt to negate his worries, despite the fact that they are consequently fueled by rumors circulating among the students, according to which Olympia is bizarrely wooden and barely ever speaks. Nathaniel is enamored and distances himself from his old friends as well as his old fiancee who stayed back at their hometown.

Two fathers, two father-killers and two sons

Hoffmann uses doppelgangers in most of his works. In the Sandman there are at least two notable pairs: Nathaniel has two fathers, the one who died during the alchemy experiment and his university professor (who wishes Nathaniel to marry Olympia, his daughter, and thus become his son-in-law). There are also two killers of the father figure: Coppelius (said to have caused the death of the father) and Coppola, who comes to fight with the university professor over ownership of the wooden automaton known as Olympia and mortally wounds him…

There are also, according to Freud, far more crucially two sons:

Freud does make a very convincing case when he argues that Olympia, the life-like piece of machinery, appears to be in reality part of Nathaniel. Indeed, the reader should note that while Nathaniel lost his father, Olympia is virtually next to her father all of the time, and whereas Nathaniel was scared by Coppola/Coppelius and the Sandman, Olympia is perfectly fine with being restricted and ordered around, docile and well-behaved. Freud argues that Olympia alludes to what the child, threatened by a potentially dangerous father, created as a means to avoid friction with the source of dread. Olympia can never antagonize the father, whereas Nathaniel keeps getting into considerable trouble in his attempt to come to terms with the various splits of the father-image.

In the end, Nathaniel only wishes to become one with Olympia, and if Freud’s analysis is correct then this wish is only one for self-completion. The split part of the youth can no longer stay away, it cannot be pushed away to another city and live in perpetual exile. Of course Nathaniel himself is not aware of the special tie to Olympia, yet everything about the story leads to the conclusion that this uncanny dance of living and wooden forms is orchestrated as an unwitting ceremony in honor of the father-image: Nathaniel and Olympia risk losing their very eyes – with Freud referring his reader to the psychoanalytic theory that links fear of losing one’s eyes to fear of castration.

In conclusion

E.T.A. Hoffmann’s The Sandman is, arguably, one of the most elegant works of dark romanticism. An uncanny story, presented masterfully – and one where the deeper meaning is allowed to remain hidden, so that the reader is free to be dazzled, surprised, horrified and indeed take part mentally in this macabre dance of hidden emotions and repressed memories. It was certainly a poignant decision by Freud to focus on this work in his article, since its use of the uncanny in high literature is paradigmatic.

By Kyriakos Chalkopoulos (https://www.patreon.com/posts/27630785)



Discuss

Unknown Unknowns in AI Alignment

14 июня, 2019 - 08:07
Published on June 14, 2019 5:07 AM UTC

It seems to me that no matter how many problems from different research agendas we solve, there is always the possibility that some 'unknown unknown' misalignment scenario can occur. I can imagine an approach of building model-agnostic, environment-agnostic minimal assumption alignment guarantees (which seems to be super hard), but I feel like things can go wrong in myriad other ways, even then.

Has there been any discussion about how we might go about this?



Discuss

SlateStarCodex Madison Meetup: Mental Health and Moloch

14 июня, 2019 - 06:13
Published on June 14, 2019 2:59 AM UTC



Discuss

Spiracular's Shortform Feed

13 июня, 2019 - 23:36
Published on June 13, 2019 8:36 PM UTC

I was just thinking about how to work myself up to posting full-posts, and this seemed like exactly the right level of difficulty and exposure for what I'm currently comfortable with. I'm glad that a norm for this exists!

This is mostly going to consist of old notebook-extracts regarding various ideas I've chewed on over the years.



Discuss

Storytelling and the evolution of human intelligence

13 июня, 2019 - 23:13
Published on June 13, 2019 8:13 PM UTC

This is a notice of a recent paper that may be of interest here, "The storytelling arms race: origin of human intelligence and the scientific mind" by Enrico Coen. It is a more specific working out of the Machiavellian Intelligence hypothesis, taking storytelling as the means through which the deception arms race operated, and which (it suggests) also raised up language alongside intelligence. It also discusses the relationship between storytelling (where, as in war, truth is the first casualty) and science (where truth is the goal).

Disclosure: I work closely with the author, although not on the subject of this paper.



Discuss

Real-World Coordination Problems are Usually Information Problems

13 июня, 2019 - 21:21
Published on June 13, 2019 6:21 PM UTC

Let’s start with a few examples of very common real-world coordination problems.

  • The marketing department at a car dealership posts ads for specific cars, but the salespeople don’t know which cars were advertised, causing confusion when a customer calls in asking about a specific car. There’s no intentional information-hoarding, it’s just that the marketing and sales people don’t sit next to each other or talk very often. Even if the info were shared, it would need to be translated to a format usable by the salespeople.
  • Various hard problems in analysis of large-scale biological data likely have close analogues in econometrics. The econometricians have good methods to solve the problems, and would probably be quite happy to apply those methods to biological data, and the bio experimentalists would love some analytic help. But these people hardly ever talk to each other, and use different language for the same things anyway.
  • When the US invaded Grenada in the ‘80’s, the marines occupied one side of the island and the army occupied the other. Their radios were not compatible, so if an army office needed to contact their counterpart in the marines, they had to walk to the nearest pay phone and get routed through Fort Bragg on commercial telephone lines.
  • Various US intelligence agencies had all of the pieces necessary to stop the 9/11 attacks. There were agencies which knew something was planned for that day, and knew who the actors were. There were agencies which knew the terrorists were getting on the planes. There were agencies which could have moved to stop them, but unfortunately the fax(!) from the agencies which knew what was happening wasn’t checked in time.
  • There are about 300 million people in the US. If I have a small company producing doilies, chances are there are plenty of people in the US alone who’d love my doilies and be happy to pay for them. But it’s hard to figure out exactly which people those are, and even once that’s done it’s hard to get them a message showing off my product. And even if all that works out, if the customers really want a slightly different pattern, it’s hard for them to communicate back to me what they want - even if I’d be happy to make it.
  • Just yesterday I was looking for data on turnover time of atherosclerotic plaques. I know plaques increase with age, but is it the same plaques in the same places growing, or is it an increase in equilibrium number of plaques (each appearing and dissipating quickly)? There’s probably thousands of people who can easily answer that question and would be happy to do so, yet finding a clear answer is still nontrivial.

Obviously these are all specific examples of problems which happen all the time.

To some extent, coordination problems are universal and have always been with us. But humans evolved to solve coordination problems in Dunbar’s-number-sized groups (plus or minus an order of magnitude) regularly talking face-to-face. Even just two hundred years ago, most people operated in relatively small communities. It’s only since the rise of cheap long-distance communication that large-scale coordination problems have crept into everyday life. The cheaper and more ubiquitous long-distance communication becomes, the more coordination problems are going to be a bottleneck. Not all coordination problems look like this, but these are the sort of coordination problems which we’d expect to become more common over time. (See “From Personal to Prison Gangs” for a more fleshed-out version of this argument, and related problems.)

Look over the list of coordination problems above, and a few major themes jump out:

  • Matching problems: Doily-makers know there are customers out there who want their product. Bio experimentalists know there are analysts out there who want their data. I know there’s someone out there who can answer my plaques question. But finding those people, in a world of 6 billion, is a needles-in-haystack problem. Just figuring out who to talk to is hard.
  • Lack of communication channels: Cheap, fast communication channels just don’t exist between company and customer or between departments of an organization. Even if you know who to talk to, you still need a way to talk to them.
  • Language difficulties: Econometricians and biologists use different language or even different frameworks for similar systems. Different departments use different data formats. The army and the marines had incompatible radios. Even when you know who to talk to and have a channel, communication can still be hard.

Standard discussions of coordination problems tend to focus on cases where a dictator could easily solve the problem. Need to meet up with someone in New York City at a specific place and time without communicating in advance? The dictator can declare “Empire State building at noon is the official meet-up spot and time”, and there we go, we’re done. But the harder sorts of real-world coordination problems usually aren’t that easy. Having a designated dictator on hand doesn’t help a doily company find enthusiastic customers, or help a biologist and an econometrician realize they should collaborate, or help translate data from one format to another (assuming they do in fact need different formats).

The biggest problem is that there’s a combinatorially huge space of possible coordination problems, and any particular coordination problem won’t happen many times. How many people have asked my exact question about atherosclerotic plaques? In order to be useful, a coordination mechanism has to address a very wide class of coordination problems in one fell swoop - e.g. the question-answering site Quora. But simply declaring “this is the canonical question-answering site” doesn’t solve the problem - in order for it to actually work, we still need a good matching engine, so that askers and answerers can find each other without having to search through the haystack themselves.

A combinatorially huge space of problems directly leads to a more insidious issue: humans have limited processing ability, so there will inevitably be coordination problems where nobody involved even knows what’s possible. The biologist and the econometrician don’t know that their fields complement each other. In order to solve that sort of problem, a third party has to proactively look for opportunities to coordinate. Once the opportunity is found, actually connecting people is the relatively easy part - lots of academics are interested in opportunities to collaborate across fields (I hear grantmakers love that stuff).



Discuss

On Having Enough Socks

13 июня, 2019 - 18:15
Published on June 13, 2019 3:15 PM UTC



Discuss

Learning Over Time for AI and Humans and Rationality

13 июня, 2019 - 16:23
Published on June 13, 2019 1:23 PM UTC

https://www.lesswrong.com/posts/pLZ3bdeng4u5W8Yft/let-s-talk-about-convergent-rationality-1

make the observation that natural and artificial intelligence will learn over time. Given the post's title that got me thinking about the two learning settings and how that might apply to concepts of rationality.

AI will presumably not face the same life time limitation that humans currently do. The implication is that learning will be different in the two settings. Human learning is very much dependent on prior generation and various social/group type aspects.

But I've generally thought of rationality (as generally understood) bound to a single mind, as it were. The idea of applying many rules about rational thought or behavior to aggregates, such as "the mob" largely a misapplication.

I wonder if the learning process based on knowledge learned (including the potential for mis-knowledge transmission) via a larger social process performs better in both general learning and development of rational though processes or if the AI single, long lived "mind" has some advantage.

I might also wonder a bit about how this might apply to questions such as that asked regarding why China did not develop science.

I suspect various aspects of my though have been discussed here -- or at least can inform on the though but there are lots of post to search.



Discuss

Some Ways Coordination is Hard

13 июня, 2019 - 16:00
Published on June 13, 2019 1:00 PM UTC

Response to (Raymond Arnold at Less Wrong): The Shilling Choice is Rabbit, Not Stag and by implication Duncan’s Open Problems in Group Rationality

Stag Hunt is the game whereby if everyone gets together, they can hunt the Stag and win big. Those who do not hunt Stag instead hunt Rabbit, and win small. But if even one person instead decides to hunt Rabbit rather than hunt Stag, everyone hunting Stag loses rather than wins. This makes hunting Stag risky, which makes Stag risky (since others view it as risky, and thus might not do it, and view others as viewing it as risky, making it even more likely they won’t do it, and so on). Sometimes this can be overcome, and sometimes it can’t.

Raymond claims that the Shilling point of this game, by default, is Rabbit, not Stag.

Whether or not this is true depends on the exact rules of the game and the exact game state, what one might call the margin of coordination. 

If you haven’t yet, click through to at least to Raymond’s article and his quote from Duncan’s original description, and consider reading Duncan’s full article.

Questions
  1. What is the risk versus reward on the stag hunt? How often must it work to be worth trying?
  2. Can we afford to fail? Can others? For how long?
  3. Can the stag hunt fail even if everyone chooses stag?
  4. Do we have time to iterate or communicate to establish future coordination, and will our actions now act as a signal?
  5. How many people need to cooperate before the stag hunt is worthwhile? Can we afford to lose anyone? Is there a lesser stag we can go after instead with a smaller group?
  6. Is there a particular player who has reason to worry, or might cause others to worry?
  7. Are the players known to be aware of the stag hunt? If there are multiple possible stag hunts, do we all know which one we’re going for?
  8. Are the players confident others know the payoff and agree it is known to be there for everyone? Does everyone even know what stag is and what rabbit is?
  9. Does this group have a history of going on stag hunts? Is going on the stag hunt praiseworthy or blameworthy if no one follows you, or not enough people do?
  10. Do the rabbit hunts have network effects such that failure to coordinate on them is bad for everyone, not only for the person going stag?
  11. Are there players who value relative success rather than absolute success, and want others to fail?
  12. Do we trust other players to behave in their own best interests and trust them to trust others in this way? Do we trust them to act in the best interest of the group knowing the group rewards that?
  13. Do we trust others to do what they say they are going to do?
  14. Are people capable of overcoming small short-term personal incentives to achieve or stand for something worthwhile, to help the group or world, or to cultivate and encourage virtue? Do they realize that cooperation is a thing and is possible at all? Do they realize that It’s Not the Incentives, It’s You?
  15. Is the stag hunt even worth asking all these questions?

Note that most of these require common knowledge. We need everyone to know, and for everyone to know that everyone knows that everyone knows that everyone knows for however many levels it takes. Alternatively, we need habits of stag hunting that are self-sustaining.

Man, coordination is complicated. And hard. So many questions! Such variation.

Duncan’s Example

In Duncan’s example, we have full common knowledge of the situation, and full agreement on payoffs, which is very good. Alas, we still suffer severe problems.

We have a very bad answer to questions two, five and six. And also fourteen. If we lose even one person, the stag hunt fails, and there is no alternative hunt with fewer people. And we have players who have reason to worry, because they can ill afford a failed stag hunt. One of them will be stranded without the ability to even hunt rabbit, should the stag hunt fail.

It seems that everyone involved is reasoning as if each member is looking out mostly or entirely for themselves and their short-term success, and expecting all others to do so as well, even when this is an obviously bad medium-term plan.

The result will often be cascading loss of faith, resulting in full abandonment of the stag hunt. Certainly the stag hunt is no longer the shilling point of selection, given the risks everyone sees. You wouldn’t do this implicitly, without everyone agreeing to it first, and you’d only do it with an explicit agreement if you had common knowledge of everyone being trustworthy to follow through.

Or, if there was a long history that the moment everyone had the resources to do it, they coordinate on the stag hunt. But that still only works with common knowledge of the full game state, so getting it without explicit communication is still super rare.

The obvious thing this group can do, in this situation, is to explicitly agree to go on the stag hunt. 

But they’re explicitly already trying to do that, and finding it won’t work, because these people do not trust each other. They fail question thirteen.

What are some other things this group might try? With so many advantages, it seems a shame to give up.

Solutions in Duncan’s Example

D1. Alexis gives one utility to Elliot (solve question two)

This actually should be enough! Alexis gives one of their 15 banked resources to Elliot. Now everyone has at least 6 banked resources, and will be able to choose rabbit even if the hunt fails. This makes the situation clear to everyone, and removes the worry that Elliot will need to choose rabbit.

D2. Wait until next hunt (solve question two)

Even simpler than option D1. If everyone hunts rabbit now, everyone goes up in stored resources, and next time has enough buffer to coordinate on a stag hunt.

Both point to the principle of slack that Raymond reminds us about, and extend this to the whole group. Don’t only keep your own slack high, don’t ask anyone else to fully deplete theirs under normal circumstances, even if it means everyone doing less efficient things for a bit.

D3. Build trust (solve question thirteen)

Note that if the group is sufficiently unreliable, that alone will prevent all stag hunts no matter what else is done. If the group could trust each other to follow through, knew that their words had meanings and promises meant something, then they could coordinate reliably despite their other handicaps here. With sufficient lack of trust, the stag hunt isn’t positive expectation to participate in, so there’s no point and everyone hunts rabbits until this is fixed.

D4. Use punishment or other incentives

A solution for any game-theoretic situation is to change the rules of the game, by coordinating to reallocate blame and resources based on actions. This is often the solution within the game, but can also happen by extending the situation outside the game. Raymond’s example shows that the game of Stag Hunt often inevitably causes punishment to take place. Using enough of it, reliably enough, predictably enough, should work, at least for the failures in Duncan’s example.

Improving any of the other answers would also help tilt the scales. Stag hunts are threshold effects at their core, so helping the situation in any way improves your chances more than one might think, and any problem causes more issues than you’d naively predict.

Solutions in Raymond’s Example

Raymond’s scenario faces different problems than Duncan’s. Where Duncan had problems with questions 2, 5, 6 and 13, Raymond faced a problem with questions 3, 7 and especially 8. He thought that staying focused had the big payoff, while his coworker thought that staying narrowly focused was a failure mode.

Coordination is especially hard if some of the people coordinating think that the result of coordination would be bad. What are the solutions now?

R1. Talk things over and create common knowledge (solve seven and eight)

R2. Propose a different coordinated approach designed to appeal to all participants

Once Raymond and his colleague talked things over and had common knowledge of each others’ preferences for conversation types, coordination became possible. It became clear that Raymond’s preferred approach didn’t count as a stag hunt, because it didn’t benefit all parties, and there was a disagreement about whether it was net beneficial slash efficient to do it. Instead, a hybrid approach seemed likely to be better.

In cases where there is a clearly correct approach, making that clear to everyone, and knowing this has happened, makes it much more likely that coordination can successfully take place. In cases where there turns out not to be such an approach, this prevents those who previously thought such an approach existed from having false expectations, and minimizes conflict and frustration.

R3. Bid on what approach to take

Often coordination on any solution is better than failure to coordinate. Some level of meandering versus focus that all parties agree to is better than fighting over that ratio and having the meeting collapse, provided the meeting is worthwhile. Thus, if the participants can’t agree on what’s best, or find a solution that works well enough for everyone to prompt coordination, then a transfer of some kind can fix that.

Doing this with dollars in social situations is usually a terrible idea. That introduces bad dynamics, in ways I won’t go into here. Instead, one should strive to offer similar consideration in other coordination questions or trade-offs in the future. The greater the social trust, the more implicit this can be while still working. This then takes the form of an intentionally poorly specified number of amorphous points, that then can be cashed in at a future time. The points matter. They don’t need to balance, but they can’t be allowed to get too imbalanced.

The Facebook Exodus Example

A while back, I realized I was very much Against Facebook. The problem was that the entire rationalist community was doing most of their discourse and communication there, as was a large portion of my other friend groups. I’d failed to find a good Magic: The Gathering team that didn’t want to do its (if you can still call it) coordination on Facebook. This was a major factor in ending my Magic comeback.

Many, but far from all, of those I queried agreed that Facebook was terrible and wished for a better alternative. But all of them initially despaired. The problem looked too hard. The network effects were too strong. Even if we could agree Facebook was bad, what was the alternative? What could possibly meet everyone’s needs as well as Facebook was doing, even if it was much better at not ruining lives and wasting time? Even if a good alternative was found, could we get everyone to agree on it?

Look at that list of questions. Consider that success depends to a large extent on common knowledge of the answers to those questions.

  1. What is the risk versus reward on the stag hunt? How often must it work to be worth trying?
  2. Can we afford to fail? Can others? For how long?
  3. Can the stag hunt fail even if everyone chooses stag?
  4. Do we have time to iterate or communicate to establish future coordination, and will our actions now act as a signal?
  5. How many people need to cooperate before the stag hunt is worthwhile? Can we afford to lose anyone? Is there a lesser stag we can go after instead with a smaller group?
  6. Is there a particular player who has reason to worry, or might cause others to worry?
  7. Are the players known to be aware of the stag hunt? If there are multiple possible stag hunts, do we all know which one we’re going for?
  8. Are the players confident others know the payoff and agree it is known to be there for everyone? Does everyone even know what stag is and what rabbit is?
  9. Does this group have a history of going on stag hunts? Is going on the stag hunt praiseworthy or blameworthy if no one follows you, or not enough people do?
  10. Do the rabbit hunts have network effects such that failure to coordinate on them is bad for everyone, not only for the person going stag?
  11. Are there players who value relative success rather than absolute success, and want others to fail?
  12. Do we trust other players to behave in their own best interests and trust them to trust others in this way? Do we trust them to act in the best interest of the group knowing the group rewards that?
  13. Do we trust others to do what they say they are going to do?
  14. Is the stag hunt even worth asking all these questions?

Which ones were problems?

Most of them.

We had (1) uncertain risk versus reward of switching to another platform or set of platforms, (3) even if coordination on the switch was successful, with (2) continuous and potentially painful social consequences and blameworthiness for being ‘out of the loop’ even temporarily. To be better off, often (5) the entire group would have to agree to the new location and method, with (6)(8) some people who would dislike any given proposal, or like staying with Facebook because they didn’t agree with my assessments, or because they’d need to coordinate elsewhere. (10) The attempt would hurt our network effects and cause non-trivial communication interruptions, even if it eventually worked. (7) Getting the word out would be a non-trivial issue, as this would include common knowledge that the word was out and the coordination was on, when it likely wasn’t going to be on at any given time.

(12) Facebook has many addictive qualities, so even many people who would say they ‘should’ quit or even were quitting would often fail. (13) Even when people agreed to switch and announced this intention, they’d often find themselves coming back.

There was a lot of excusing one’s actions because of (14) Network Effects and The Incentives.

A lot of people (15) reasonably didn’t want to even think about it, under these circumstances.

The good news is we had (4) plenty of time to make this work, and (9) even most of those who didn’t think the switch was a good idea understood that it was a noble thing to attempt and would make sense under some scenarios. And no one was (11) thinking of their relative social media position. And of course, that the stag hunt wouldn’t automatically or fully fail if one person didn’t cooperate, and if we got enough cooperation, critical mass would take over.

But that’s still 11 out of 14 problems that remained importantly unsolved.

The better news was we had one other important advantage. I hated hunting rabbit. Rabbit hunting was not a productive exercise for me, and I’d rather be off hunting stag unsuccessfully on my own, than hunting rabbit. It’s not a great spot, certainly there would be better uses of time, but that was a great advantage – that I didn’t feel too bad about failures. Otherwise, the whole thing would never have had a chance.

It also helped that many others are increasingly making similar cases, for a wide variety of reasons, some of which I don’t even agree with or I don’t think are big deals.

The solution I went with was three–fold.

F1. First, to explain why I believed Facebook was awful, in order to help create common knowledge. Starting with an article, then continuing to make the case.

F2. Second, go out and start stag hunting on my own, and make it clear I wasn’t going anywhere. This does not work when stag hunts are all-or-nothing with a fixed ‘all’ group, but this is rare. More often, stag hunts work if those who people count on, do their jobs rather than everyone who might show up in theory shows up and does their job. That’s a crucial distinction, and a dramatic drop in difficulty.

F3. Third, to reward with engagement, praise and when helpful direct encouragement and assistance those who wanted to make the switch to blogs or other less toxic communication forms. To some extent there was shaming of Facebook participation, but that’s not much use when everyone’s doing it.

Without the effort to first create common knowledge, the attempt would have had no chance of success. And of course, a combination of factors helped out, from the emergence of Less Wrong 2.0 to a variety of others waking up to the Facebook problem at about the same time, for their own reasons.

The solution of ‘be the coordination you want to see in the world even if it doesn’t make sense on its own’ is very powerful. That’s how Jason Murray kick-started the New York rationalist group, and how many other groups have started – show up at the same time and same place, even if you end up eating alone, to ensure others can come. Doing it for a full-on stag hunt with fixed potential participation is more expensive, but it is still a very strong signal that can encourage a cascade. We need to accept that coordination is hard, and therefore solutions that are expensive are worth considering.

Solving coordination problems is not only expensive, it’s also often highly unpleasant and non-intuitively difficult work, that superficially doesn’t look like the ‘real work.’ Thus, those who solve coordination problems often end up resented as those who didn’t do the real work and are pulling down the big bucks, as something that one should obviously be able to get along without, discouraging and lowering reward and thus discouraging this hard and risky work. Often many people correctly say ‘oh, the problem is people can’t coordinate’ go out to solve it, and make things worse, because the problems are indeed hard, and competition to solve them makes them harder.

If everyone required to successfully hunt a stag can agree on common knowledge of what the stag is, that the stag would be well worth hunting, and how and when the stag is hunted, one could argue that’s a lot more than half of the battle. The rest is still hard, but relatively hard part really, really should be over. Ninety percent of life after that, one could say, is the virtue of reliably showing up.

 

 

 

 

 

 



Discuss

Can we use ideas from ecosystem management to cultivate a healthy rationality memespace?

13 июня, 2019 - 15:38
Published on June 13, 2019 12:38 PM UTC

Background: ecosystems management practices for improving community memespaces

One can model individual human minds, as well as a community of minds, as an “ecosystem” of “memes”. These memes might be things like:

  • Bayesian epistemology
  • a habit of checking Facebook when one wakes up
  • wiggling one’s fingers to indicate agreement with a statement
  • prefacing things one says with “My model of this is that...”
  • Doing calibration training
  • Referring to blog posts in conversation
  • Taking silent pauses to think mid-conversation

Etc. etc.

Calling this set an “ecosystem”, seems to me to be mechanistically very close to what’s actually going on. At least, this is because:

  1. Memes mutate as they are transmitted between minds
  2. Memes undergo selection pressure as they are transmitted
  3. The underlying topology/geography of social, cultural and geographical networks of people influence their spread
  4. Memes can be in equilibrium with other memes
  5. Memes can act as “invasive species”

Now there is an emerging European rationality community, largely driven by efforts from the Prague rationalists. This community imports many memes from the Bay area rationality community. For a high-level, historical examination of this memespace, see Julia Galef’s map of bay area memespace.

At a recent CFAR workshop, we discussed how we can ensure that this interaction is successful.

Five of us (Ales Flidr, Elizabeth Garrett, Nora Ammann, Adam Scholl, Jacob Lagerros), felt that the ecosystem model carried sufficient mechanistic similarity to the actual situation that it would be helpful to read up on things like: protocols for deliberate introductions of new species into new environments, invasive species regulations and protection programs, pest control, and more.

Collection of background notes

We spent 1h researching this. Now the outside view predicts that if we were to leave it at that, the 16-page Google doc would never be used again. Hence we’re experimenting with releasing our notes together with a LessWrong question, in order to allow others to benefit from and build on our progress.

You can find our notes here.

These notes are provided “as is”. I (jacobjacob) briefly went through them to make them more readable, but apart from that this should not be interpreted as something the authors endorse as being true, and despite originating at a CFAR workshop it is not official CFAR content.

Open questions

We’d be interested in using further research to answer questions such as:

  • What are warning signs of a memespace/ecosystem being harmed?
  • What are best practices for introducing a new meme into a memespace, and what can we learn from actual ecosystems?
  • What are some useful models for thinking about this problem?


Discuss

Episode 6 of Tsuyoku Naritai (the 'becoming stronger' podcast)

13 июня, 2019 - 02:31
Published on June 12, 2019 11:31 PM UTC

Sorry for the delay; I fell victim to the planning fallacy. Again.

https://anchor.fm/tsuyokunaritai/episodes/Episode-6---Design-e4arua



Discuss

What kind of thing is logic in an ontological sense?

13 июня, 2019 - 01:28
Published on June 12, 2019 10:28 PM UTC

The existence of logic seems somewhat mysterious. It's this thing that seems to exist, but unlike other things that exist, it doesn't seem to exist anywhere in specific or in any tangible form. Further, while it is easy to mock Plato for mysticism when he posits perfect forms existing in some kind of mysterious Platonic Realm, that's actually uncomfortably close to a description of logic.



Discuss

Let's talk about "Convergent Rationality"

13 июня, 2019 - 00:54
Published on June 12, 2019 9:53 PM UTC

What this post is about: I'm outlining some thoughts on what I've been calling "convergent rationality". I think this is an important core concept for AI-Xrisk, and probably a big crux for a lot of disagreements. It's going to be hand-wavy! It also ended up being a lot longer than I anticipated.

Abstract: Natural and artificial intelligences tend to learn over time, becoming more intelligent with more experience and opportunity for reflection. Do they also tend to become more "rational" (i.e. "consequentialist", i.e. "agenty" in CFAR speak)? Steve Omohundro's classic 2008 paper argues that they will, and the "traditional AI safety view" and MIRI seem to agree. But I think this assumes an AI that already has a certain sufficient "level of rationality", and it's not clear that all AIs (e.g. supervised learning algorithms) will exhibit or develop a sufficient level of rationality. Deconfusion research around convergent rationality seems important, and we should strive to understand the conditions under which it is a concern as thoroughly as possible.

I'm writing this for at least these 3 reasons:

  • I think it'd be useful to have a term ("convergent rationality") for talking about this stuff.
  • I want to express, and clarify, (some of) my thoughts on the matter.
  • I think it's likely a crux for a lot of disagreements, and isn't widely or quickly recognized as such. Optimistically, I think this article might lead to significantly more clear and productive discussions about AI-Xrisk strategy and technical work.

Outline:

  • Characterizing convergent rationality
  • My impression of attitudes towards convergent rationality
  • Relation to capability control
  • Relevance of convergent rationality to AI-Xrisk
  • Conclusions, some arguments pro/con convergent rationality
Characterizing convergent rationality

Consider a supervised learner trying to maximize accuracy. The Bayes error rate is typically non-0, meaning it's not possible to get 100% test accuracy just by making better predictions. If, however, the test data(/data distribution) were modified, for example to only contain examples of a single class, the learner could achieve 100% accuracy. If the learner were a consequentialist with accuracy as its utility function, it would prefer to modify the test distribution in this way in order to increase its utility. Yet, even when given the opportunity to do so, typical gradient-based supervised learning algorithms do not seem to pursue such solutions (at least in my personal experience as an ML researcher).

We can view the supervised learning algorithm as either ignorant of, or indifferent to, the strategy of modifying the test data. But we can also this behavior as a failure of rationality, where the learner is "irrationally" averse or blind to this strategy, by construction. A strong version of the convergent rationality thesis (CRT) would then predict that given sufficient capacity and "optimization pressure", the supervised learner would "become more rational", and begin to pursue the "modify the test data" strategy. (I don't think I've formulated CRT well enough to really call it a thesis, but I'll continue using it informally).

More generally, CRT would imply that deontological ethics are not stable, and deontologists must converge towards consequentialists. (As a caveat, however, note that in general environments, deontological behavior can be described as optimizing a (somewhat contrived) utility function (grep "existence proof" in the reward modeling agenda)). The alarming implication would be that we cannot hope to build agents that will not develop instrumental goals.

I suspect this picture is wrong. At the moment, the picture I have is: imperfectly rational agents will sometimes seek to become more rational, but there may be limits on rationality which the "self-improvement operator" will not cross. This would be analogous to the limit of ω which the "add 1 operator" approaches, but does not cross, in the ordinal numbers. In other words, order to reach "rationality level" ω+1, it's necessary for an agent to already start out at "rationality level" ω. A caveat: I think "rationality" is not uni-dimensional, but I will continue to write as if it is.

My impression of attitudes towards convergent rationality

Broadly speaking, MIRI seem to be strong believers in convergent rationality, but their reasons for this view haven't been very well-articulated (TODO: except the inner optimizer paper?). AI safety people more broadly seem to have a wide range of views, with many people disagreeing with MIRI's views and/or not feeling confident that they understand them well/fully.

Again, broadly speaking, machine learning (ML) people often seem to think it's a confused viewpoint bred out of anthropomorphism, ignorance of current/practical ML, and paranoia. People who are more familiar with evolutionary/genetic algorithms and artificial life communities might be a bit more sympathetic, and similarly for people who are concerned with feedback loops in the context of algorithmic decision making.

I think a lot of people with working on ML-based AI safety consider convergent rationality to be less relevant than MIRI does, because 1) so far it is more of a hypothetical/theoretical concern, whereas we've done a lot of and 2) current ML (e.g. deep RL with bells and whistles) seems dangerous enough because of known and demonstrated specification and robustness problems (e.g. reward hacking and adversarial examples).

In the many conversations I've had with people from all these groups, I've found it pretty hard to find concrete points of disagreement that don't reduce to differences in values (e.g. regarding long-termism), time-lines, or bare intuition. I think "level of paranoia about convergent rationality" is likely an important underlying crux.

Relation to capability control

A plethora of naive approaches to solving safety problems by limiting what agents can do have been proposed and rejected on the grounds that advanced AIs will be smart and rational enough to subvert them. Hyperbolically, the traditional AI safety view is that "capability control" is useless. Irrationality can be viewed as a form of capability control.

Naively, approaches which deliberately reduce an agent's intelligence or rationality should be an effective form of capability control method (I'm guessing that's a proposal in the Artificial Stupidity paper, but I haven't read it). If this were true, then we might be able to build very intelligent and useful AI systems, but control them by, e.g. making them myopic, or restricting the hypothesis class / search space. This would reduce the "burden" on technical solutions to AI-Xrisk, making it (even) more of a global coordination problem.

But CRT suggests that these methods of capability control might fail unexpectedly. There is at least one example (I've struggled to dig up) of a memory-less RL agent learning to encode memory information in the state of the world. More generally, agents can recruit resources from their environments, implicitly expanding their intellectual capabilities, without actually "self-modifying".

Relevance of convergent rationality to AI-Xrisk

Believing CRT should lead to higher levels of "paranoia". Technically, I think this should lead to more focus on things that look more like assurance (vs. robustness or specification). Believing CRT should make us concerned that non-agenty systems (e.g. trained with supervised learning) might start behaving more like agents.

Strategically, it seems like the main implication of believing in CRT pertains to situations where we already have fairly robust global coordination and a sufficiently concerned AI community. CRT implies that these conditions are not sufficient for a good prognosis: even if everyone using AI makes a good-faith effort to make it safe, if they mistakenly don't believe CRT, they can fail. So we'd also want the AI community to behave as if CRT were true unless or until we had overwhelming evidence that it was not a concern.

On the other hand, disbelief in CRT shouldn't allay our fears overly much; AIs need not be hyperrational in order to pose significant Xrisk. For example, we might be wiped out by something more "grey goo"-like, i.e. an AI that is basically a policy hyperoptimized for the niche of the Earth, and doesn't even have anything resembling a world(/universe) model, planning procedure, etc. Or we might create AIs that are like superintelligent humans: having many cognitive biases, but still agenty enough to thoroughly outcompete us, and considering lesser intelligences of dubious moral significance.

Conclusions, some arguments pro/con convergent rationality

My impression is that intelligence (as in IQ/g) and rationality are considered to be only loosely correlated. My current model is that ML systems become more intelligent with more capacity/compute/information, but not necessarily more rational. If this is true, is creates exciting prospects for forms of capability control. On the other hand, if CRT is true, this supports the practice of modelling all sufficiently advanced AIs as rational agents.

I think the main argument against CRT is that, from an ML perspective, it seems like "rationality" is more or less a design choice: we can make agents myopic, we can hard-code flawed environment models or reasoning procedures, etc.The main counter-arguments arise from VNMUT, which can be interpreted as saying "rational agents are more fit" (in an evolutionary sense). At the same time, it seems like the complexity of the real world (e.g. physical limits of communication and information processing) makes this a pretty weak argument. Humans certainly seem highly irrational, and distinguishing biases and heuristics can be difficult.

A special case of this is the "inner optimizers" idea. The strongest argument for inner optimizers I'm aware of goes like: "the simplest solution to a complex enough task (and therefor the easiest for weakly guided search, e.g. by SGD) is to instantiate a more agenty process, and have it solve the problem for you". The "inner" part comes from the postulate that a complex and flexible enough class of models will instantiate such a agenty process internally (i.e. using a subset of the model's capacity). I currently think this picture is broadly speaking correct, and is the third major (technical) pillar supporting AI-Xrisk concerns (along with Goodhart's law and instrumental goals).

The issues with tiling agents also suggest that the analogy with ordinals I made might be stronger than it seems; it may be impossible for an agent to rationally endorse a qualitatively different form of reasoning. Similarly, while "CDT wants to become UDT" (supporting CRT), my understanding is that it is not actually capable of doing so (opposing CRT) because "you have to have been UDT all along" (thanks to Jessica Taylor for explaining this stuff to me a few years back).

While I think MIRI's work on idealized reasoners has shed some light on these questions, I think in practice, random(ish) "mutation" (whether intentionally designed or imposed by the physical environment) and evolutionary-like pressures may push AIs across boundaries that the "self-improvement operator" will not cross, making analyses of idealized reasoners less useful than they might naively appear.

This article is inspired by conversations with Alex Zhu, Scott Garrabrant, Jan Leike, Rohin Shah, Micah Carrol, and many others over the past year and years.



Discuss

Cryonics before natural death. List of companies?

12 июня, 2019 - 21:53
Published on June 12, 2019 6:53 PM UTC

I really don't want to wait those who are dear to me or myself to get Alzheimer's and only then, after our natural death, to be cryopreserved.

I want to choose the moment when I'm preserved. However, it seems like it is not allowed by Alcor, Cryonics institute, and many other companies due to legislation. I cannot even kill myself and be preserved right afterwards.

Serge Fague wrote something along the lines "I'd rather preserve myself right after I'm diagnosed with this", so I think there are companies (and countries) that can preserve you if you ask them.

My question is - do such companies exist, and if yes, can you name them?



Discuss

The Drifter (a short story)

12 июня, 2019 - 19:09
Published on June 12, 2019 4:09 PM UTC

https://www.youtube.com/watch?v=GyYxmGQbhuk


Have a look :) Audio of a 1700-word story. Somewhat psychological.



Discuss

Страницы