Вы здесь

Сборщик RSS-лент

Should it be a research paper or a blog post?

Новости LessWrong.com - 24 сентября, 2020 - 11:09
Published on September 24, 2020 8:09 AM GMT

The answer to this question may seem obvious to some, but let's see.

 

My impression is that some people write long blog posts about things that they perceive as innovative new ideas in philosophy, AI research, or whatever. So how do you decide that your idea should be a blogpost (here or on some other site) instead of submitting it to a journal?

Is the university research system just too closed? Or do you think it is superfluous?

Or do you think that you can do it faster for a blog? (This may not be unambiguously good. My impression is that there is less literature research for some blog articles than I would like to see, which often creates noise.)

Or do you want to have feedback for a blogpost first and submit later?

(Note that the question can be generalized to other media and forms, e.g. magazine essay vs. journal article.)



Discuss

I have discovered a new kind of unemployment.

Новости LessWrong.com - 24 сентября, 2020 - 08:43
Published on September 24, 2020 1:04 AM GMT

Hello everybody, I have an article that explains why since 2000: business investment has been weak; the fall in the U.S net labor share; the decline in the prime age U.S labor participation rate vs large gains elsewhere; the rise in deaths of despair. The article is called Skill Stalagmites, Technology Stalactites and can be found here. I have split the piece into two parts: a 1500 word article for the general reader and a longer piece for the more sophisticated reader. There is a link to the latter at the end of the first piece.

The punchline to the article is that the 4-5% gap in the lfpr between the U.S and peer economies is a form of disguised unemployment. And this is a novel kind of unemployment, which is not caused by a fall in aggregate demand. 

The actual cause is that firms are imposing higher effort levels on workers. I can summarize the argument you will find in the main article; it goes like this:

  1. Firms impose higher effort demands on workers; workers have to complete more tasks (for a higher wage) or be fired.
  2. The higher wage does not compensate workers for their lost work leisure; thus workers look for less demanding job positions (or refuse to move up to more senior roles).
  3. If one imagines a skill ladder, then all workers attempt to drop down a rung. This is easy for higher skilled workers, but what happens to workers at the bottom?
  4. The lowest skilled workers compete for job openings with somewhat more skilled workers. Firms prefer to hire the more skilled worker, resulting in the lowest skilled workers being pushed out of employment altogether.
  5. This assumes that employers can always identify the highest skilled worker from their pool of applicants. This won't always be the case; if the higher skilled worker has a bad interview or the weaker candidate has positive chemistry with the interviewer, then the objectively weaker candidate can win a job offer.
  6. Thus provided the lowest skill workers are willing to keep searching for jobs they will eventually obtain a job offer and regain employment.
  7. This means though that workers on the second lowest skill rung will be unable to drop down to the lowest rung unless they also increase their job search activity. And in turn this forces the workers above them to increase their job search.
  8. Any person wanting a job now has to apply to many more job positions before they can get their first job offer. But after a string of failures, job seekers become discouraged and temporarily withdraw from the search process. It is this temporary withdrawal that is responsible for the drop in lfpr. For those who are the main breadwinners, the period of withdrawal will be short - perhaps only a few months. But for workers who are more marginally attached to the labor force, it could be years or forever.
  9. Evidence for higher effort in the U.S can be found in the higher U.S productivity growth since 2000 vs peer economies.
  10. Evidence of higher job search can be found in the elevated duration of unemployment, which in 2019 was still equal to recessionary levels. The American Time Use Survey also shows higher than normal time spent on job search.

The questions of why this is happening post 2000 and not before, and why only in the U.S and not elsewhere, are taken up in the full article.

If you have any questions of your own, please ask away.

Best, 

Nathan.

P.S The article is published on Seeking Alpha, but don't let that put you off. Though I don't have a formal background in economics, I do keep up with the relevant literature.



Discuss

The rationalist community's location problem

Новости LessWrong.com - 23 сентября, 2020 - 21:39
Published on September 23, 2020 6:39 PM GMT

The Problem

Basically ever since the first rationalists settled in Berkeley, people have been saying, “Why do you live in Berkeley, Berkeley sucks! You should all move to Location X instead, it’s so much better.” The problem has always been that no one agrees on what Location X is. Some common candidates for Location X:

  • A smaller, cheaper, friendlier US city
  • NYC
  • Australia
  • Canada
  • Somewhere with very low cost of living (often in Southeast Asia or Latin America)
  • London
  • Oxford
  • Blackpool
  • Prague
  • A castle
  • A private island

and of course

  • Wherever the speaker is from

In the past I've brushed off all such suggestions, because it was just too hard a coordination problem to get multiple hundreds of rationalists to leave Berkeley, where they've gotten jobs, rented or even bought houses, established organizations, enrolled in schools, and established social circles. 

But we're in a unique time! Due to the pandemic, there's far less reason to stay in any one place - work and school are remote, expensive leases can be terminated, and you can't see your friends anyway. Most of the rationalist houses I know have moved or dissolved, and the former Berkeley rationalists are flung across all corners of the globe (yeah I know globes don't have corners). A fair number of us have stayed, but I think for most of us it's just because our friends are here, we're hoping that someday the rest of our friends come back, and we're not sure where else to go. 

So, if ever there were a time when we actually had the chance to move the physical locus of the rationalist community, it's now. Below, I'll lay out what I believe to be some of the most important general considerations for deciding on a new location. I encourage people to make their case for a specific location, either in comments or in their own posts. (Looking at you, Mikk!) 

Considerations for Location XPotential dealbreakers

Visas

In order to settle in a location, you have to be able to legally live there long-term. Most Berkeley rationalists are US citizens, and those who aren't have already paid the steep cost of acquiring US visas and learning US immigration law. This feels like a strong argument in favor of staying in the US somewhere, although it's possible there are places where this wouldn't actually be that much of an issue. In any case, it's certainly an argument against countries with strict immigration laws, like Switzerland.

Relatedly, organizations such as MIRI, CFAR, Open Phil, BERI, etc are registered in the US. I don't know how hard it would be for them to operate elsewhere and am unfamiliar with this domain in general.

Language 

Given that basically all rationalists speak English (since it's pretty hard to read the relevant material otherwise), we should settle somewhere English-speaking; it would be very costly if everyone had to deal with a language barrier every single day (or learn a new language). 

Notably this doesn't automatically disqualify all locations in e.g. continental Europe - Habryka points out that you can get along just fine in Berlin if you only know English. But somewhere like e.g. Japan looks like a much worse prospect on this metric.

National political environment / culture

The rationality community often attracts controversy, so it's important that we settle somewhere that protects freedom of thought and speech, and is generally friendly to weird ideas. We should definitely not move somewhere where political dissidents can be abducted willy nilly.

Some people are worried about unrest in the US, which might be reasonable, but on that metric it's still better to live here than, say, Mali or Afghanistan.

Local political environment / culture

Same basic considerations as the above. California may be an increasingly hostile environment for our community, but it's almost certainly still better to live here than in a town where people fly Confederate flags and openly carry guns. 

It's also really valuable to be near Silicon Valley. The Bay Area has a general culture of ambition and intellectual curiosity that's hard to find.

General infrastructure

People talk wistfully about private islands or about founding our own town, but my guess is that most of those people haven't actually thought those ideas through. A place needs SO MANY THINGS to sustain a modern human population: roads, electricity, water, laws, buildings, police, medicine, commerce, trash collection... and those are just the basic necessities! Despite the appeal of building something from the ground up and thus controlling every aspect of its development, it just seems way better to move to a place that already has this basic infrastructure in place.

Other important considerations

Cost of living

A major complaint about the Bay Area is rental prices, and justifiably so. Obviously cost of living interacts with a lot of other factors, but on the whole, it would feel pretty silly to leave the Bay only to move somewhere with equally high rent. 

Occupancy laws

Many municipalities, at least in the US, have laws prohibiting unrelated adults from sharing a home. This would render most group houses illegal.

Modern conveniences

Berkeley has fiber internet, 2-day Amazon delivery, a myriad of quick restaurant and grocery delivery options, and excellent coverage by Lyft, Uber, and bikeshares. I expect many would be reluctant to give up this level of convenience. This is a strike against private islands, remote castles, and developing countries, among others.

Walkability (/ bikeability / public transit)

Sparse suburban areas are terrible places to build community. In addition, driving is dangerous and owning a car is super annoying. We should settle somewhere where it's possible to all live close enough together that we can visit each other on foot, and also ideally where the city center is within walking distance of our homes. 

(Being able to bike safely and easily between homes and city center would also work. Sufficiently good public transit might also do the trick.)

Medical care

It's really important to have quick access to modern medicine – rationalists may largely be healthy 20-somethings, but healthy 20-somethings can still die of sepsis if they can't get antibiotics quickly. This is an argument against many locations in developing countries. It could also be construed as an argument against the US, where medical care is theoretically available but often avoided due to expense.

Additional things to consider

Crime

All else equal, less crime seems better. If that's not possible, property crime is better than violent crime. It's really unpleasant to have your bike or laptop stolen, but it's a lot worse when it happens at gunpoint (which happened to some of my friends when I lived in Chicago). 

(Aside: High-trust environments are great, but I would guess that in general they're also more insular, which might make it hard to pick up our ~300-person community, plop it down in an existing high-trust town, and have everyone maintain those high trust levels. No real action item here and I'm confused.)

Schools

Rationalists may be less likely than average to want kids, but that doesn't mean none of us are having them. I don't know if there's anywhere in the world that has truly non-terrible schools, but at least some schools are a lot less terrible than others.

Weather

A lot of people who live in California really hate extreme weather. A lot of people have SAD and don't want to live in a place that has winters. Natural disasters are bad too.

Call to Action

As I said above, I'd be excited for people to pitch their own favorite Location X! Write an essay making your case, or even just a bullet-pointed comment.

And please also let me know if there are additional considerations I missed.



Discuss

[AN #118]: Risks, solutions, and prioritization in a world with many AI systems

Новости LessWrong.com - 23 сентября, 2020 - 21:20
Published on September 23, 2020 6:20 PM GMT

[AN #118]: Risks, solutions, and prioritization in a world with many AI systems Alignment Newsletter is a weekly publication with recent content relevant to AI alignment around the world View this email in your browser Newsletter #118
Alignment Newsletter is a weekly publication with recent content relevant to AI alignment around the world. Find all Alignment Newsletter resources here. In particular, you can look through this spreadsheet of all summaries that have ever been in the newsletter.
Audio version here (may not be up yet). SECTIONS HIGHLIGHTS
TECHNICAL AI ALIGNMENT
        HANDLING GROUPS OF AGENTS
        FORECASTING
        MISCELLANEOUS (ALIGNMENT)
NEWS    HIGHLIGHTS

AI Governance: Opportunity and Theory of Impact (Allan Dafoe) (summarized by Rohin): What is the theory of change for work on AI governance? Since the world is going to be vastly complicated by the broad deployment of AI systems in a wide variety of contexts, several structural risks will arise. AI governance research can produce “assets” (e.g. policy expertise, strategic insights, important networking connections, etc) that help humanity make better decisions around these risks. Let’s go into more detail.

A common perspective about powerful AI is the “superintelligence” perspective, in which we assume there is a single very cognitively powerful AI agent. This leads people to primarily consider “accident” and “misuse” risks, in which either the AI agent itself “wants” to harm us, or some bad actor uses the AI agent to harm us.

However, it seems likely that we should think of an ecology of AI agents, or AI as a general purpose technology (GPT), as in e.g. CAIS (AN #40) or Age of Em. In this case, we can examine the ways in which narrow AI could transform social, military, economic, and political systems, and the structural risks that may arise from that. Concrete examples of potential existential structural risks induced by AI include nuclear instability, geopolitical turbulence, authoritarianism, and value erosion through competition.

A key point about the examples above is that the relevant factors for each are different. For example, for nuclear instability, it is important to understand nuclear deterrence, first strike vulnerability and how it could change with AI processing of satellite imagery, undersea sensors, cyber surveillance and weapons, etc. In contrast, for authoritarianism, relevant processes include global winner-take-all-markets, technological displacement of labor, and authoritarian surveillance and control.

This illustrates a general principle: unlike in the superintelligence perspective, the scope of both risks and solutions in the ecology / GPT perspectives is very broad. As a result, we need a broad range of expertise and lots of connections with existing fields of research. In particular, “we want to build a metropolis -- a hub with dense connections to the broader communities of computer science, social science, and policymaking -- rather than an isolated island”.

Another important aspect here is that in order to cause better decisions to be made, we need to focus not just on generating the right ideas, but also on ensuring the right ideas are in the right places at the right time (e.g. by ensuring that people with the right tacit knowledge are part of the decision-making process). Instead of the "product model" of research that focuses on generating good ideas, we might instead want a “field-building model”, which also places emphasis on improving researcher’s competence on a variety of issues, bestowing prestige and authority on those who have good perspectives on long-term risks, improving researcher’s networks, and training junior researchers. However, often it is best to focus on the product model of research anyway, and get these benefits as a side effect.

To quote the author: “I think there is a lot of useful work that can be done in advance, but most of the work involves us building our competence, capacity, and credibility, so that when the time comes, we are in position and ready to formulate a plan. [...] Investments we make today should increase our competence in relevant domains, our capacity to grow and engage effectively, and the intellectual credibility and policy influence of competent experts.”



Rohin's opinion: See the next summary. Note also that the author is organizing the Cooperative AI Workshop (AN #116) to tackle some of these issues.



Andrew Critch on AI Research Considerations for Human Existential Safety (Lucas Perry and Andrew Critch) (summarized by Rohin): This podcast discusses the recent ARCHES (AN #103) document, and several thoughts surrounding it. There’s a lot in here that I won’t summarize, including a bunch of stuff that was in the summary of ARCHES. I’m going to focus primarily on the (substantial) discussion of how to prioritize within the realm of possible risks related in some way to AI systems.

Firstly, let’s be clear about the goal: ensuring existential safety, that is, making sure human extinction never happens. Note the author means literal extinction, as opposed to something like “the loss of humanity’s long-term potential”, because the former is clearer. While it is not always clear whether something counts as “extinction” (what if we all become uploads?), it is a lot clearer than whether a scenario counts as a loss of potential.

Typical alignment work focuses on the “single-single” case, where a single AI system must be aligned with a single human, as in e.g. intent alignment (AN #33). However, this isn’t ultimately what we care about: we care about multi-multi existential safety, that is, ensuring that when multiple AI systems act in a world with multiple humans, extinction does not happen. There are pretty significant differences between these: in particular, it’s not clear whether multi-multi “alignment” even has meaning, since it is unclear whether it makes sense to view humanity as an agent to which an AI system could be “aligned”.

Nonetheless, single-single alignment seems like an important subproblem of multi-multi existential safety: we will be delegating to AI systems in the future; it seems important that we know how to do so. How do we prioritize between single-single alignment, and the other subproblems of multi-multi existential safety? A crucial point is that single-single work will not be neglected, because companies have strong incentives to solve single-single alignment (both in the sense of optimizing for the right thing, and for being robust to distributional shift). In contrast, in multi-multi systems, it is often the case that there is a complex set of interacting effects that lead to some negative outcome, and there is no one actor to blame for the negative outcome, and as a result it doesn’t become anybody’s job to prevent that negative outcome.

For example, if you get a huge medical bill because the necessary authorization forms hadn’t been filled out, whose fault is it? Often in such cases there are many people to blame: you could blame yourself for not checking the authorization, or you could blame the doctor’s office for not sending the right forms or for not informing you that the authorization hadn’t been obtained, etc. Since it’s nobody’s job to fix such problems, they are and will remain neglected, and so work on them is more impactful.

Something like transparency is in a middle ground: it isn’t profitable yet, but probably will be soon. So, if someone were indifferent between a bunch of areas of research, the author would advise for e.g. multi-stakeholder delegation over transparency over robustness. However, the author emphasizes that it’s far more important that people work in some area of research that they find intellectually enriching and relevant to existential safety.

The podcast has lots of other points, here is an incomplete quick selection of them:

- In a multi-multi world, without good coordination you move the world in a “random” direction. There are a lot of variables which have to be set just right for humans to survive (temperature, atmospheric composition, etc) that are not as important for machines. So sufficiently powerful systems moving the world in a “random” direction will lead to human extinction.

- One response to the multi-multi challenge is to have a single group make a powerful AI system and “take over the world”. This approach is problematic since many people will oppose such a huge concentration of power. In addition, it is probably not desirable even if possible, since it reduces robustness by creating a single point of failure.

- Another suggestion is to create a powerful AI system that protects humanity (but is still uncontrollable in that humanity cannot stop its operation). The author does not like the solution much, because if we get it wrong and deploy a misaligned uncontrollable AI system, then we definitely die. The author prefers that we instead always have control over the AI systems we deploy.



Rohin's opinion: Both this and the previous summary illustrate an increasingly common perspective:

1. The world is not going to look like “today’s world plus a single AGI agent”: instead, we will likely have a proliferation of many different AI systems specialized for different purposes.

2. In such a world, there are a lot of different challenges that aren’t standard intent alignment.

3. We should focus on these other challenges because [a variety of reasons].

If you have technical CS skills, how should you prioritize between this perspective and the more classical intent alignment perspective?

Importance. I’ve estimated (AN #80) a 10% chance of existential catastrophe via a failure of intent alignment, absent intervention from longtermists to address intent alignment. Estimates vary quite a lot, even among people who have thought about the problem a lot; I’ve heard as low as < 1% and as high as 80% (though these usually don’t assume “no intervention from longtermists”).

It’s harder to estimate the importance of structural risks and extinction risks highlighted in the two summaries above, but the arguments in the previous two posts seem reasonably compelling and I think I’d be inclined to assign a similar importance to it (i.e. similar probability of causing an existential catastrophe).

Note that this means I’m disagreeing with Critch: he believes that we are far more likely to go extinct through effects unique to multi-multi dynamics; in contrast I find the argument less persuasive because we do have governance, regulations, national security etc. that would already be trying to mitigate issues that arise in multi-multi contexts, especially things that could plausibly cause extinction.

Neglectedness. I’ve already taken into account neglectedness outside of EA in estimating the probabilities for importance. Within EA there is already a huge amount of effort going into intent alignment, and much less in governance and multi-multi scenarios -- perhaps a difference of 1-2 orders of magnitude; the difference is even higher if we only consider people with technical CS skills.

Tractability. I buy the argument in Dafoe’s article that for AI governance due to our vast uncertainty we need a “metropolis” model where field-building is quite important; I think that implies that solving the full problem (at today's level of knowledge) would require a lot of work and building of expertise. In contrast, with intent alignment, we have a single technical problem with significantly less uncertainty. As a result, I expect that currently in expectation a single unit of work goes further to solving intent alignment than to solving structural risks / multi-multi problems, and so intent alignment is more tractable.

I also expect technical ideas to be a bigger portion of "the full solution" in the case of intent alignment -- as Dafoe argues, I expect that for structural risks the solution looks more like "we build expertise and this causes various societal decisions to go better" as opposed to "we figure out how to write this piece of code differently so that it does better things". This doesn't have an obvious impact on tractability -- if anything, I'd guess it argues in favor of the tractability of work on structural risks, because it seems easier to me to create prestigious experts in particular areas than to make progress on a challenging technical problem whose contours are still uncertain since it arises primarily in the future.

I suspect that I disagree with Critch here: I think he is more optimistic about technical solutions to multi-multi issues themselves being useful. In the past I think humanity has resolved such issues via governance and regulations and it doesn’t seem to have relied very much on technical research; I’d expect that trend to continue.

Personal fit. This is obviously important, but there isn’t much in general for me to say about it.

Once again, I should note that this is all under the assumption that you have technical CS skills. I think overall I end up pretty uncertain which of the two areas I’d advise going in (assuming personal fit was equal in both areas). However, if you are more of a generalist, I feel much more inclined to recommend choosing some subfield of AI governance, again subject to personal fit, and Critch agrees with this.

   TECHNICAL AI ALIGNMENT
 HANDLING GROUPS OF AGENTS

Comparing Utilities (Abram Demski) (summarized by Rohin): This is a reference post about preference aggregation across multiple individually rational agents (in the sense that they have VNM-style utility functions), that explains the following points (among others):

1. The concept of “utility” in ethics is somewhat overloaded. The “utility” in hedonic utilitarianism is very different from the VNM concept of utility. The concept of “utility” in preference utilitarianism is pretty similar to the VNM concept of utility.

2. Utilities are not directly comparable, because affine transformations of utility functions represent exactly the same set of preferences. Without any additional information, concepts like “utility monster” are type errors.

3. However, our goal is not to compare utilities, it is to aggregate people’s preferences. We can instead impose constraints on the aggregation procedure.

4. If we require that the aggregation procedure produces a Pareto-optimal outcome, then Harsanyi’s utilitarianism theorem says that our aggregation procedure can be viewed as maximizing some linear combination of the utility functions.

5. We usually want to incorporate some notion of fairness. Different specific assumptions lead to different results, including variance normalization, Nash bargaining, and Kalai-Smorodinsky.

 FORECASTING

How Much Computational Power It Takes to Match the Human Brain (Joseph Carlsmith) (summarized by Asya): In this blog post, Joseph Carlsmith gives a summary of his longer report estimating the number of floating point operations per second (FLOP/s) which would be sufficient to perform any cognitive task that the human brain can perform. He considers four different methods of estimation.

Using the mechanistic method, he estimates the FLOP/s required to model the brain’s low-level mechanisms at a level of detail adequate to replicate human task-performance. He does this by estimating that ~1e13 - 1e17 FLOP/s is enough to replicate what he calls “standard neuron signaling” — neurons signaling to each other via using electrical impulses (at chemical synapses) — and learning in the brain, and arguing that including the brain’s other signaling processes would not meaningfully increase these numbers. He also suggests that various considerations point weakly to the adequacy of smaller budgets.

Using the functional method, he identifies a portion of the brain whose function we can approximate with computers, and then scales up to FLOP/s estimates for the entire brain. One way to do this is by scaling up models of the human retina: Hans Moravec's estimates for the FLOP/s of the human retina imply 1e12 - 1e15 FLOP/s for the entire brain, while recent deep neural networks that predict retina cell firing patterns imply 1e16 - 1e20 FLOP/s.

Another way to use the functional method is to assume that current image classification networks with known FLOP/s requirements do some fraction of the computation of the human visual cortex, adjusting for the increase in FLOP/s necessary to reach robust human-level classification performance. Assuming somewhat arbitrarily that 0.3% to 10% of what the visual cortex does is image classification, and that the EfficientNet-B2 image classifier would require a 10x to 1000x increase in frequency to reach fully human-level image classification, he gets 1e13 - 3e17 implied FLOP/s to run the entire brain. Joseph holds the estimates from this method very lightly, though he thinks that they weakly suggest that the 1e13 - 1e17 FLOP/s estimates from the mechanistic method are not radically too low.

Using the limit method, Joseph uses the brain’s energy budget, together with physical limits set by Landauer’s principle, which specifies the minimum energy cost of erasing bits, to upper-bound required FLOP/s to ~7e21. He notes that this relies on arguments about how many bits the brain erases per FLOP, which he and various experts agree is very likely to be > 1 based on arguments about algorithmic bit erasures and the brain's energy dissipation.

Lastly, Joseph briefly describes the communication method, which uses the communication bandwidth in the brain as evidence about its computational capacity. Joseph thinks this method faces a number of issues, but some extremely preliminary estimates suggest 1e14 FLOP/s based on comparing the brain to a V100 GPU, and 1e16 - 3e17 FLOP/s based on estimating the communication capabilities of brains in traversed edges per second (TEPS), a metric normally used for computers, and then converting to FLOP/s using the TEPS to FLOP/s ratio in supercomputers.

Overall, Joseph thinks it is more likely than not that 1e15 FLOP/s is enough to perform tasks as well as the human brain (given the right software, which may be very hard to create). And he thinks it's unlikely (<10%) that more than 1e21 FLOP/s is required. For reference, an NVIDIA V100 GPU performs up to 1e14 FLOP/s (although FLOP/s is not the only metric which differentiates two computational systems.)

Read more: Full Report: How Much Computational Power Does It Take to Match the Human Brain?



Asya's opinion: I really liked this post, although I haven't gotten a chance to get through the entire full-length report. I found the reasoning extremely legible and transparent, and there's no place where I disagree with Joseph's estimates or conclusions. See also Import AI's summary.

  MISCELLANEOUS (ALIGNMENT)

The "Backchaining to Local Search" Technique in AI Alignment (Adam Shimi) (summarized by Rohin): This post explains a technique to use in AI alignment, that the author dubs “backchaining to local search” (where local search refers to techniques like gradient descent and evolutionary algorithms). The key idea is to take some proposed problem with AI systems, and figure out mechanistically how that problem could arise when running a local search algorithm. This can help provide information about whether we should expect the problem to arise in practice.



Rohin's opinion: I’m a big fan of this technique: it has helped me notice that many of my concepts were confused. For example, this helped me get deconfused about wireheading and inner alignment. It’s an instance of the more general technique (that I also like) of taking an abstract argument and making it more concrete and realistic, which often reveals aspects of the argument that you wouldn’t have previously noticed.

   NEWS

The Open Phil AI Fellowship (summarized by Rohin): We’re now at the fourth cohort of the Open Phil AI Fellowship (AN #66)! Applications are due October 22.

Navigating the Broader Impacts of AI Research (summarized by Rohin): This is a workshop at NeurIPS; the title tells you exactly what it's about. The deadline to submit is October 12.

FEEDBACK I'm always happy to hear feedback; you can send it to me, Rohin Shah, by replying to this email. PODCAST An audio podcast version of the Alignment Newsletter is available. This podcast is an audio version of the newsletter, recorded by Robert Miles.
Subscribe here:

Copyright © 2020 Alignment Newsletter, All rights reserved.

Want to change how you receive these emails?
You can update your preferences or unsubscribe from this list.
 

Discuss

Petrov Day Ritual: Coronavirus Edition

Новости LessWrong.com - 23 сентября, 2020 - 19:41
Published on September 23, 2020 4:41 PM GMT

Ozy adapts the rationalist Petrov Day ritual for 2020.



Discuss

Dehumanisation *errors*

Новости LessWrong.com - 23 сентября, 2020 - 12:51
Published on September 23, 2020 9:51 AM GMT

In response to my post contrasting value learning with anthropomorphisation, steve2152 brought up the fact that dehumanisation can be seen as the opposite of anthropomorphisation.

I agree with this insight, but only when dehumanisation causes errors of interpretation. I was using empathy in the sense of "insight into the other agent", rather than "sympathy with the other agent".

In practice, dehumanisation does tend to cause errors. We see outgroups as more homogeneous, coherent, and organised than they actually are. Despite the suave psychopaths depicted in movies, psychopaths tend to be less effective at achieving their goals (as evidenced by the large number of psychopaths in prison). Torturers are less effective at extracting true information than classical interrogators.

Now, it's not a universal law by any means, but it does seem that dehumanisation can often lead to errors, and from that perspective can be seen as a failure of value learning.

The meaning of errors
  • "Objection! Hold on just a minute!" screams the convenient strawman I have just constructed to prove a point.

  • "You've claimed that 'agent's goals' are interpretations by the outside observer; that you can model a human as perfectly rational, without being wrong. You've claimed that this is 'tagged white box knowledge', which can't be deduced from the agent's policy or its algorithm."

  • "Given that, how can you claim that anyone 'fails' at interpreting the goals of others, or that they make 'errors'?"

This a very valid point, strawman, but I've also pointed out that human theory of mind/empathy is very similar from human to human, and tends to agree with how we interpret our own goals. Because of this, there is a rough "universal human theory of mind", ie a universal way of going from human policy to human preferences.

When I'm talking about errors, I'm talking about deviations from this ideal[1].

  1. Because human theories of mind do not agree perfectly, there will always be an irreducible level of uncertainty in this ideal, but there is agreement on the broad strokes of it. ↩︎



Discuss

The new Editor

Новости LessWrong.com - 23 сентября, 2020 - 05:25
Published on September 23, 2020 2:25 AM GMT

LookatthisglorioustableCelebrations! The new editor is finally here! 

Starting from today, all desktop users will by-default use the new visual editor that we've been testing for a while. While the primary goal of this is to have a better foundation on which to build future editor features, here are a number of things you can do starting from today: 

  • Insert tables! A heavily requested feature.
  • Copy-paste LaTeX without everything breaking!
  • Nest bullet lists and other block elements in blockquotes! (still no nested blockquotes though, though if enough people want that, it would be easy to change)
  • Much less jankyness and brokenness!

Let us know what you think about the new editor. We've been testing it for a while and have been pretty happy with it (and users who had opted into beta features also had predominantly positive feedback). You can also use the old editor if you run into any problems by checking the "Restore the previous WYSIWYG editor" checkbox in your user settings.



Discuss

New Tagging Power Tools: Dashboard + Upgraded Tagging Editing Experience

Новости LessWrong.com - 23 сентября, 2020 - 04:49
Published on September 23, 2020 1:49 AM GMT

Following the strong response of taggers to calls for help, plus lessons learned from our earlier designs, we are launching new and improved power tools for maintaining and improving the tag/wiki (twiki?) corpus.

We are first applying these tools to completing the underway Wiki Import campaign, and when that's done we'll reconfigure them for long-term use.

There are several connected parts to the new tools:

  1. Tag Flags
  2. New Twiki Dashboard (www.lesswrong.com/tags/dashboard)
  3. Upgraded Tag Editing
  4. Tag Discussion Sections
Tag Flags

At the heart of the new Tagging Power Tools are tag flags. These are a limited and generally fixed set of flags that people can set on tags when they require work, and then remove when the work is done. 

The Special Tag Flags for the Wiki Import 
The numbers indicate how many tags have this flag set.

Temporarily, the tag flags are designed specifically for the import (those from the deprecated spreadsheet). Long-term they'll be general things like "needs a description", "should have more posts tagged", "is out of date". The tag flags will be managed by admins, of course, with input from taggers. This is largely because we want to keep the number to something 5 main ones and maybe another 5-10 minor ones. Keep it manageable.

We are deprecating the Tag Grading Scheme.

New Twiki Dashboard

You can get lists of tags with each kind of tag-flag on the new dashboard. By default, you get a list of all tags with any flag applied. If you click on a flag in the header list, the page will be filtered for only tags with that tag. The selected Tag Filter displays in black. You can remove the filter with the Reset Filter button.

If you hover of the flag, you'll see a description describing what it means and how to address it.

 

Hover-Over explains the Tag FlagEditing Tags

Once you've found tags you want to edit, you have two options for how to do that.

  1. Twiki Dashboard On-Page Editing
  2. Full-Edit on Tag Page

On the Twiki Dashboard page, if you click the Edit button for a tag, it will open it up for editing on-page.

On-page editing.

As you see, here you can toggle the flags. Don't forget to click submit! If a removed flag was part of your filter, the tag probably won't disappear from the list until you refresh.

Upgraded Tag Editing

Continuing from the above, you can alternatively start editing tags in full-edit mode by clicking on the text body on the dashboard (rather the Edit button).

Click anywhere this region on a tag to open the tag page in editing mode.

The new full tag-editing experience looks like this:

The Editing Mode for Tags looks just like Published Page

Things to note:

  • Opens in edit mode, ready to go. Remember to click submit!
  • Tag flags can be set.
  • "Next Tag" [in filter list] button.
  • Tag Discussion Section
  • Posts List (can add posts from here)
  • For imported old wiki pages, there are links to the original on the old site, and the latest revision of the import.
    • These can be used to compare with the pre-import page for formatting issues, etc., and for copying/merging text. Copy from the revision page rather than the old wiki site since it has the links fixed up.

It's everything you need to edit and improve tags with minimal clicks.

Tag Flag Playlists

If you've clicked through to editing a tag via a filter on the dashboard, instead of going back to the dashboard, you can move onto the next tag with a handy next tag button. This takes you to the next from the tag flag list you've selected.

 

Tag Discussion

A week or two ago, we added a Discussion section for each tag/wiki. You can get to them by clicking the Discussion button on the top right of tag pages.

We've also placed the same Discussion section beneath the tag text when editing. This is so people can leave notes and ask questions about a tag while editing it, super conveniently. We'll work on further improving the visibility of these comments.

The Tag Discussion section, now visible when you're editing a tag.Feedback, please!

That's it, folks, I hope you like it. Please try it out and let us know what you think.

As always, you can join the Tagger Slack to chat live with other taggers and the LW team.



Discuss

AI Advantages [Gems from the Wiki]

Новости LessWrong.com - 23 сентября, 2020 - 01:44
Published on September 22, 2020 10:44 PM GMT

During the LessWrong 1.0 Wiki Import we (the LessWrong team) discovered a number of great articles that most of the LessWrong team hadn't read before. Since we expect many others to also not have have read these, we are creating a series of the best posts from the Wiki to help give those hidden gems some more time to shine.

The original wiki article was fully written by Kaj Sotala, who I've added as a coauthor to this post. Thank you for your work on the wiki!

AI advantages are various factors that might favor AIs in case there was ever a conflict between them and humans. These can be classified as hardware advantages, self-improvement capabilities, co-operative advantages, and human handicaps.

Hardware advantages
  • Superior processing power: Having more serial processing power would let an AI think faster than humans, while having more parallel processing power and more memory would let it think about more things at once.
Self-improvement capabilities

An AI with access to its source code may directly modify the way it thinks, or create a modified version of itself. An AI can intentionally be built in a manner that is easy to understand and modify, and may even read its own design documents. Self-improvement capabilities may enable recursive self-improvement to occur, thereby triggering an intelligence explosion.

  • Improving algorithms: An AI may modify its existing algorithms, e.g. making them faster, to consume less memory, or to rely on fewer assumptions.
  • Designing new mental modules: A mental module is a part of a mind that specializes in processing a certain kind of information. An AI could create entirely new kinds of modules, custom-tailored for specific problems.
  • Modifiable motivation systems: Humans frequently suffer from problems such as procrastination, boredom, mental fatigue, and burnout. A mind which did not become bored or tired with its work would have a clear advantage over humans.
Co-operative advantages
  • Copyability: A digital mind can be copied very quickly, and doing so has no cost other than access to the hardware required to run it.
  • Perfect co-operation: Minds might be constructed to lack any self-interest. Such entities minds could share the same goal system and co-operate perfectly with one another.
  • Superior communication: AIs could communicate with each other at much higher bandwidths than humans, and modify themselves to understand each other better.
  • Transfer of skills: To the extent that skills can be modularized, digital minds could create self-contained skill modules to be shared with others.
Human handicaps

Humans frequently reason in biased ways. AIs might be built to avoid such biases.

  • Biases from computational limitations or false assumptions: Some human biases can be seen as assumptions or heuristics that fail to reason correctly in a modern environment, or as satisficing algorithms that do the best possible job given human computational resources.
  • Human-centric biases: People tend to think of the capabilities of non-human minds, such as God or an artificial intelligence, as if the minds in question were human. This tendency persists even if humans are explicitly instructed to act otherwise.
  • Biases from socially motivated cognition: It has also been proposed that humans have evolved to acquire beliefs which are socially beneficial, even if those beliefs weren't true.
References

Discuss

Sunday September 27, 12:00PM (PT) — talks by Alex Flint, Alex Zhu and more

Новости LessWrong.com - 23 сентября, 2020 - 00:59
Published on September 22, 2020 9:59 PM GMT

This Sunday at 12pm (PT), we're running another session of "lightning talks" by curated LessWrong authors (see here for previous weeks' transcripts).

  • For the first hour, we will have a series of lightning talks each lasting about 5 minutes followed by discussion. The talks will be short and focus on presenting one core idea well, rather than rushing through a lot of content.
  • From 1PM to 2PM, we will hangout in Gather.town. Link will be sent out closer to the event. 
  • We want to give top LessWrong writers an interesting space to discuss their ideas, and have more fruitful collaboration between users. Think of it like a cross between an academic colloquium and some friends chatting by a whiteboard.

If you're a curated author and interested in giving a 5-min talk at a future event, which will then be transcribed and edited, sign up here.

SpeakersDetails

When? Sunday September 27, 12:00PM (PT)

Where? https://us02web.zoom.us/j/87259855821



Discuss

Comparative Advantage is Not About Trade

Новости LessWrong.com - 22 сентября, 2020 - 21:43
Published on September 22, 2020 6:43 PM GMT

Braudel is probably the most impressive historian I have read. His quantitative estimates of premodern populations and crop yields are exactly the sort of foundation you’d think any understanding of history would be based upon. Yet reading his magnum opus, it became steadily clearer as the books progressed that Braudel was missing some fairly fundamental economic concepts. I couldn’t quite put my finger on what was missing until a section early in book 3:

... these deliberately simple tautologies make more sense to my mind than the so-called ‘irrefutable’ pseudo-theorem of David Ricardo (1817), whose terms are well known: that the relations between two given countries depend on the “comparative costs” obtaining in them at the point of production

Braudel, apparently, is not convinced by the principle of comparative advantage. What is his objection?

The division of labor on a world scale (or on world-economy-scale) cannot be described as a concerted agreement made between equal parties and always open to review… Unequal exchange, the origin of the inequality in the world, and, by the same token, the inequality of the world, the invariable generator of trade, are longstanding realities. In the economic poker game, some people have always held better cards than others…

It seems Braudel is under the impression that comparative advantage is only relevant in the context of “equal” exchange or “free” trade or something along those lines.

If an otherwise impressive economic historian is that deeply confused about comparative advantage, then I expect other people are similarly confused. This post is intended to clarify.

The principle of comparative advantage does not require that trade be “free” or “equal” or anything of the sort. When the Portugese or the British seized monopolies on trade with India in the early modern era, those trades were certainly not free or equal. Yet the monopolists would not have made any profit whatsoever unless there were some underlying comparative advantage.

For example, consider an oversimplified model of the salt trade. People historically needed lots of salt to preserve food, yet many inland areas lack local sources, so salt imports were necessary for survival. Transport by ship was historically orders of magnitude more efficient than overland, so a government in control of a major river could grab a monopoly on the salt trade. Since the people living inland could not live without it, the salt monopolist could charge quite high prices - a “trade” arguably not so different from threatening inland farmers with death if they did not pay up. (An exaggeration, since there were other ways to store food and overland smuggling became viable at high enough prices, but I did say it’s an oversimplified example.)

Notice that, in this example, there is a clear underlying comparative advantage: the inland farmers have a comparative disadvantage in producing salt, while the ultimate salt supplier (a salt mine or salt pan) has a comparative advantage in salt production. If the farmer could produce salt with the same opportunity cost as the salt mine/pan, then the monopolist would have no buyers. If the salt mine/pan had the same opportunity cost for obtaining salt as the farmers, then the monopolist would have no supplier. Absent some underlying comparative advantage between two places, the trade monopolist cannot make any profit.

Another example: suppose I’m a transatlantic slave trader, kidnapping people in Africa and shipping them to slave markets in the Americas. It’s easy to see how the kidnapping part might be profitable, but why was it profitable to move people across the Atlantic? Why not save the transportation costs, and work the same slaves on plantations in Africa rather than plantations in the Americas? Or why not use native American slaves entirely, rather than importing Africans? Ultimately, the profits were because the Americas had a lot lower population density - there was more land, and fewer people to work it. Thus, labor was worth more in the Americas (and that same comparative advantage drove not just the slave trade, but also immigration and automation). Without a comparative advantage, enslaving people might still have been profitable, but there would be no reason to ship them across the Atlantic.

Let’s take it a step further. This argument need not involve any trade at all.

Suppose I’m the dictator of some small archipelago. I have total ownership and control over the country’s main industries (bananas and construction), and there’s an international embargo against trade with my little country, so there’s no trade to worry about either internally or externally. Let’s say I just want to maximize construction output - although I will still need to order some banana-growing in order to keep my construction workers fed.

The question is: who and where do I order to grow bananas, and who and where do I order to build things? To maximize construction, I will want to order people with the largest comparative advantage in banana-growing to specialize in banana-growing, and I will want to order those bananas to be grown on the islands with the largest comparative advantage in banana-growing. (In fact, this is not just relevant to maximization of construction - it applies to pareto-optimal production in general.) There’s no trade; I’m just using comparative advantage to figure out how best to deploy my own resources.

Takeaway: comparative advantage is not a principle of trade, it’s a principle of optimization. Pareto-optimal production means specialization by comparative advantage.



Discuss

The Haters Gonna Hate Fallacy

Новости LessWrong.com - 22 сентября, 2020 - 15:20
Published on September 22, 2020 12:20 PM GMT

Occasionally I see people doing what I think of as the “Haters Gonna Hate Fallacy”.

The HGHF says something like: “People are going to misinterpret you no matter how carefully you word things. Therefore, there’s no point wasting time wording things carefully.”

An example:

“I think [term X] in your post is going to cause misunderstandings, I’d suggest phrasing it differently.”
“Oh, haters are gonna hate, there’s no amount of rephrasing that’s going to prevent this from being misinterpreted if people want to.”

Now there’s obviously a grain of truth in this. It is impossible to phrase something in a way that would always be interpreted correctly, and for pretty much any message there are people who are hostile to it and who will twist it in the most uncharitable possible way.

The fallacy is in assuming that if you cannot avoid all misunderstandings, there is no point in avoiding any misunderstandings. Maybe 5% of your audience will dismiss the message no matter what, but 30% will dismiss the old phrasing while being receptive to the new phrasing.

This is most obvious if you take it to an extreme:

“Hey maybe you shouldn’t start your essay by saying that all of your readers are idiots who deserve to be shot.”
“Eh, if that upsets them then they wouldn’t like me explaining the theory of general relativity anyway.”

Communication is hard and – importantly – contextual. Most of your readers will be reasonable people and assume you to use words to mean things they’re used to them meaning. If they’re used to word X being used differently than how you mean it, that doesn’t make them haters.

When I’ve fallen into something like the fallacy myself, it has often been motivated by an unwillingness to put in work. Other people should just understand me right away! “It’s beneath me to waste my time on doing other people’s interpretative work for them!” It’s dangerous to psychoanalyze others, but I have seen at least one person communicate unclearly, have that pointed out to them, then argue for why it was right for them to be unclear… only to later on admit that they were enjoying the frustration of being misunderstood.

Now avoiding misunderstandings is a lot of work, and it’s totally valid not to bother! It’s alright to just focus on a particular target audience who understands you. I’m not saying that you should always put in maximal effort into being understood – I certainly don’t.

But I do suggest owning up to it if you are choosing to write something in a way that is going to cause misunderstandings that could have been avoided.

Cross-posts: Twitter, Facebook.



Discuss

Anthropomorphisation vs value learning: type 1 vs type 2 errors

Новости LessWrong.com - 22 сентября, 2020 - 13:46
Published on September 22, 2020 10:46 AM GMT

The Occam's razor paper showed that one cannot deduce an agent H.mjx-chtml {display: inline-block; line-height: 0; text-indent: 0; text-align: left; text-transform: none; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; word-wrap: normal; word-spacing: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0; min-height: 0; border: 0; margin: 0; padding: 1px 0} .MJXc-display {display: block; text-align: center; margin: 1em 0; padding: 0} .mjx-chtml[tabindex]:focus, body :focus .mjx-chtml[tabindex] {display: inline-table} .mjx-full-width {text-align: center; display: table-cell!important; width: 10000em} .mjx-math {display: inline-block; border-collapse: separate; border-spacing: 0} .mjx-math * {display: inline-block; -webkit-box-sizing: content-box!important; -moz-box-sizing: content-box!important; box-sizing: content-box!important; text-align: left} .mjx-numerator {display: block; text-align: center} .mjx-denominator {display: block; text-align: center} .MJXc-stacked {height: 0; position: relative} .MJXc-stacked > * {position: absolute} .MJXc-bevelled > * {display: inline-block} .mjx-stack {display: inline-block} .mjx-op {display: block} .mjx-under {display: table-cell} .mjx-over {display: block} .mjx-over > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-under > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-stack > .mjx-sup {display: block} .mjx-stack > .mjx-sub {display: block} .mjx-prestack > .mjx-presup {display: block} .mjx-prestack > .mjx-presub {display: block} .mjx-delim-h > .mjx-char {display: inline-block} .mjx-surd {vertical-align: top} .mjx-mphantom * {visibility: hidden} .mjx-merror {background-color: #FFFF88; color: #CC0000; border: 1px solid #CC0000; padding: 2px 3px; font-style: normal; font-size: 90%} .mjx-annotation-xml {line-height: normal} .mjx-menclose > svg {fill: none; stroke: currentColor} .mjx-mtr {display: table-row} .mjx-mlabeledtr {display: table-row} .mjx-mtd {display: table-cell; text-align: center} .mjx-label {display: table-row} .mjx-box {display: inline-block} .mjx-block {display: block} .mjx-span {display: inline} .mjx-char {display: block; white-space: pre} .mjx-itable {display: inline-table; width: auto} .mjx-row {display: table-row} .mjx-cell {display: table-cell} .mjx-table {display: table; width: 100%} .mjx-line {display: block; height: 0} .mjx-strut {width: 0; padding-top: 1em} .mjx-vsize {width: 0} .MJXc-space1 {margin-left: .167em} .MJXc-space2 {margin-left: .222em} .MJXc-space3 {margin-left: .278em} .mjx-test.mjx-test-display {display: table!important} .mjx-test.mjx-test-inline {display: inline!important; margin-right: -1px} .mjx-test.mjx-test-default {display: block!important; clear: both} .mjx-ex-box {display: inline-block!important; position: absolute; overflow: hidden; min-height: 0; max-height: none; padding: 0; border: 0; margin: 0; width: 1px; height: 60ex} .mjx-test-inline .mjx-left-box {display: inline-block; width: 0; float: left} .mjx-test-inline .mjx-right-box {display: inline-block; width: 0; float: right} .mjx-test-display .mjx-right-box {display: table-cell!important; width: 10000em!important; min-width: 0; max-width: none; padding: 0; border: 0; margin: 0} .MJXc-TeX-unknown-R {font-family: monospace; font-style: normal; font-weight: normal} .MJXc-TeX-unknown-I {font-family: monospace; font-style: italic; font-weight: normal} .MJXc-TeX-unknown-B {font-family: monospace; font-style: normal; font-weight: bold} .MJXc-TeX-unknown-BI {font-family: monospace; font-style: italic; font-weight: bold} .MJXc-TeX-ams-R {font-family: MJXc-TeX-ams-R,MJXc-TeX-ams-Rw} .MJXc-TeX-cal-B {font-family: MJXc-TeX-cal-B,MJXc-TeX-cal-Bx,MJXc-TeX-cal-Bw} .MJXc-TeX-frak-R {font-family: MJXc-TeX-frak-R,MJXc-TeX-frak-Rw} .MJXc-TeX-frak-B {font-family: MJXc-TeX-frak-B,MJXc-TeX-frak-Bx,MJXc-TeX-frak-Bw} .MJXc-TeX-math-BI {font-family: MJXc-TeX-math-BI,MJXc-TeX-math-BIx,MJXc-TeX-math-BIw} .MJXc-TeX-sans-R {font-family: MJXc-TeX-sans-R,MJXc-TeX-sans-Rw} .MJXc-TeX-sans-B {font-family: MJXc-TeX-sans-B,MJXc-TeX-sans-Bx,MJXc-TeX-sans-Bw} .MJXc-TeX-sans-I {font-family: MJXc-TeX-sans-I,MJXc-TeX-sans-Ix,MJXc-TeX-sans-Iw} .MJXc-TeX-script-R {font-family: MJXc-TeX-script-R,MJXc-TeX-script-Rw} .MJXc-TeX-type-R {font-family: MJXc-TeX-type-R,MJXc-TeX-type-Rw} .MJXc-TeX-cal-R {font-family: MJXc-TeX-cal-R,MJXc-TeX-cal-Rw} .MJXc-TeX-main-B {font-family: MJXc-TeX-main-B,MJXc-TeX-main-Bx,MJXc-TeX-main-Bw} .MJXc-TeX-main-I {font-family: MJXc-TeX-main-I,MJXc-TeX-main-Ix,MJXc-TeX-main-Iw} .MJXc-TeX-main-R {font-family: MJXc-TeX-main-R,MJXc-TeX-main-Rw} .MJXc-TeX-math-I {font-family: MJXc-TeX-math-I,MJXc-TeX-math-Ix,MJXc-TeX-math-Iw} .MJXc-TeX-size1-R {font-family: MJXc-TeX-size1-R,MJXc-TeX-size1-Rw} .MJXc-TeX-size2-R {font-family: MJXc-TeX-size2-R,MJXc-TeX-size2-Rw} .MJXc-TeX-size3-R {font-family: MJXc-TeX-size3-R,MJXc-TeX-size3-Rw} .MJXc-TeX-size4-R {font-family: MJXc-TeX-size4-R,MJXc-TeX-size4-Rw} .MJXc-TeX-vec-R {font-family: MJXc-TeX-vec-R,MJXc-TeX-vec-Rw} .MJXc-TeX-vec-B {font-family: MJXc-TeX-vec-B,MJXc-TeX-vec-Bx,MJXc-TeX-vec-Bw} @font-face {font-family: MJXc-TeX-ams-R; src: local('MathJax_AMS'), local('MathJax_AMS-Regular')} @font-face {font-family: MJXc-TeX-ams-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_AMS-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_AMS-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_AMS-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-B; src: local('MathJax_Caligraphic Bold'), local('MathJax_Caligraphic-Bold')} @font-face {font-family: MJXc-TeX-cal-Bx; src: local('MathJax_Caligraphic'); font-weight: bold} @font-face {font-family: MJXc-TeX-cal-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-R; src: local('MathJax_Fraktur'), local('MathJax_Fraktur-Regular')} @font-face {font-family: MJXc-TeX-frak-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-B; src: local('MathJax_Fraktur Bold'), local('MathJax_Fraktur-Bold')} @font-face {font-family: MJXc-TeX-frak-Bx; src: local('MathJax_Fraktur'); font-weight: bold} @font-face {font-family: MJXc-TeX-frak-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-BI; src: local('MathJax_Math BoldItalic'), local('MathJax_Math-BoldItalic')} @font-face {font-family: MJXc-TeX-math-BIx; src: local('MathJax_Math'); font-weight: bold; font-style: italic} @font-face {font-family: MJXc-TeX-math-BIw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-BoldItalic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-BoldItalic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-BoldItalic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-R; src: local('MathJax_SansSerif'), local('MathJax_SansSerif-Regular')} @font-face {font-family: MJXc-TeX-sans-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-B; src: local('MathJax_SansSerif Bold'), local('MathJax_SansSerif-Bold')} @font-face {font-family: MJXc-TeX-sans-Bx; src: local('MathJax_SansSerif'); font-weight: bold} @font-face {font-family: MJXc-TeX-sans-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-I; src: local('MathJax_SansSerif Italic'), local('MathJax_SansSerif-Italic')} @font-face {font-family: MJXc-TeX-sans-Ix; src: local('MathJax_SansSerif'); font-style: italic} @font-face {font-family: MJXc-TeX-sans-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-script-R; src: local('MathJax_Script'), local('MathJax_Script-Regular')} @font-face {font-family: MJXc-TeX-script-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Script-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Script-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Script-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-type-R; src: local('MathJax_Typewriter'), local('MathJax_Typewriter-Regular')} @font-face {font-family: MJXc-TeX-type-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Typewriter-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Typewriter-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Typewriter-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-R; src: local('MathJax_Caligraphic'), local('MathJax_Caligraphic-Regular')} @font-face {font-family: MJXc-TeX-cal-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-B; src: local('MathJax_Main Bold'), local('MathJax_Main-Bold')} @font-face {font-family: MJXc-TeX-main-Bx; src: local('MathJax_Main'); font-weight: bold} @font-face {font-family: MJXc-TeX-main-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-I; src: local('MathJax_Main Italic'), local('MathJax_Main-Italic')} @font-face {font-family: MJXc-TeX-main-Ix; src: local('MathJax_Main'); font-style: italic} @font-face {font-family: MJXc-TeX-main-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-R; src: local('MathJax_Main'), local('MathJax_Main-Regular')} @font-face {font-family: MJXc-TeX-main-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-I; src: local('MathJax_Math Italic'), local('MathJax_Math-Italic')} @font-face {font-family: MJXc-TeX-math-Ix; src: local('MathJax_Math'); font-style: italic} @font-face {font-family: MJXc-TeX-math-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size1-R; src: local('MathJax_Size1'), local('MathJax_Size1-Regular')} @font-face {font-family: MJXc-TeX-size1-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size1-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size1-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size1-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size2-R; src: local('MathJax_Size2'), local('MathJax_Size2-Regular')} @font-face {font-family: MJXc-TeX-size2-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size2-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size2-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size2-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size3-R; src: local('MathJax_Size3'), local('MathJax_Size3-Regular')} @font-face {font-family: MJXc-TeX-size3-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size3-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size3-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size3-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size4-R; src: local('MathJax_Size4'), local('MathJax_Size4-Regular')} @font-face {font-family: MJXc-TeX-size4-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size4-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size4-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size4-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-R; src: local('MathJax_Vector'), local('MathJax_Vector-Regular')} @font-face {font-family: MJXc-TeX-vec-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-B; src: local('MathJax_Vector Bold'), local('MathJax_Vector-Bold')} @font-face {font-family: MJXc-TeX-vec-Bx; src: local('MathJax_Vector'); font-weight: bold} @font-face {font-family: MJXc-TeX-vec-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Bold.otf') format('opentype')} 's reward function (RH - using the notation from that paper) or their level of rationality (pH) by observing their behaviour or even by knowing their policy (πH). Subsequently, in a LessWrong post, it was demonstrated that even knowing the agent's full algorithm (call this aH) would not be enough to deduce either RH or pH individually.

In an online video, I argued that the reason humans can do this when assessing other humans, is because we have an empathy module/theory of mind EH, that allows us to model the rationality and motives of other humans. These EH are, crucially, quite similar from human to human, and when we turn them on ourselves, the results are similar to what happens when others assess us. So, roughly speaking, there is an approximate 'what humans want', at least in typical environments[1], that most humans can agree on.

I struggled to convince people that, without this module, we would fail to deduce the motives of other humans. It is hard to imagine what we would be like if we were fundamentally different.

But there is an opposite error that people know very well: anthropomorphisation. In this situation, humans attribute motives to the behaviour of the wind, the weather, the stars, the stock market, cute animals, uncute animals...

So the same module that allows us to, somewhat correctly, deduce the motivations of other humans, also sets us up to fail for many other potential agents. If we started 'weakening' EH, then we would reduce the number of anthropomorphisation errors we made, but we'd start making more errors about actual humans.

So our EH can radically fail at assessing the motivations of non-humans, and also sometimes fails at assessing the motivations of humans. Therefore I'm relatively confident in arguing that EH is not some "a priori" object, coming from pure logic, but is contingent and dependent on human evolution. If we met an alien race, they we would likely assess their motives in ways they would find incorrect - and they'd assess our motives in ways we would find incorrect, no matter how much information either of us had.

  1. See these posts for how we can and do extend this beyond typical environments. ↩︎



Discuss

General Intelligence is a Fractal Representation

Новости LessWrong.com - 22 сентября, 2020 - 11:00
Published on September 22, 2020 8:00 AM GMT

Throwing more data and compute at a today's neural network architectures will not spit out an artificial general intelligence (AGI).

An AGI is a machine with human-equialent intelligence. Computers are already superior to human beings at memory, recall, scalability, compute speed and math. Humans are superior to computers at small data.

Today's artificial neural networks (ANNs) will not solve the problem of small data because today's ANNs are based off of the multilayer perceptron and the multilayer perceptron is too data hungry; it scales badly. No realistic quantity of training data plus physical compute hardware can compensate for a sufficiently bad scaling factor.

We know the multilayer perceptron must scale badly because it has fractal dimension 1.

Data Compression

"Entropy" is the minimum message length necessary to encode information. The combined entropy SA+B.mjx-chtml {display: inline-block; line-height: 0; text-indent: 0; text-align: left; text-transform: none; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; word-wrap: normal; word-spacing: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0; min-height: 0; border: 0; margin: 0; padding: 1px 0} .MJXc-display {display: block; text-align: center; margin: 1em 0; padding: 0} .mjx-chtml[tabindex]:focus, body :focus .mjx-chtml[tabindex] {display: inline-table} .mjx-full-width {text-align: center; display: table-cell!important; width: 10000em} .mjx-math {display: inline-block; border-collapse: separate; border-spacing: 0} .mjx-math * {display: inline-block; -webkit-box-sizing: content-box!important; -moz-box-sizing: content-box!important; box-sizing: content-box!important; text-align: left} .mjx-numerator {display: block; text-align: center} .mjx-denominator {display: block; text-align: center} .MJXc-stacked {height: 0; position: relative} .MJXc-stacked > * {position: absolute} .MJXc-bevelled > * {display: inline-block} .mjx-stack {display: inline-block} .mjx-op {display: block} .mjx-under {display: table-cell} .mjx-over {display: block} .mjx-over > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-under > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-stack > .mjx-sup {display: block} .mjx-stack > .mjx-sub {display: block} .mjx-prestack > .mjx-presup {display: block} .mjx-prestack > .mjx-presub {display: block} .mjx-delim-h > .mjx-char {display: inline-block} .mjx-surd {vertical-align: top} .mjx-mphantom * {visibility: hidden} .mjx-merror {background-color: #FFFF88; color: #CC0000; border: 1px solid #CC0000; padding: 2px 3px; font-style: normal; font-size: 90%} .mjx-annotation-xml {line-height: normal} .mjx-menclose > svg {fill: none; stroke: currentColor} .mjx-mtr {display: table-row} .mjx-mlabeledtr {display: table-row} .mjx-mtd {display: table-cell; text-align: center} .mjx-label {display: table-row} .mjx-box {display: inline-block} .mjx-block {display: block} .mjx-span {display: inline} .mjx-char {display: block; white-space: pre} .mjx-itable {display: inline-table; width: auto} .mjx-row {display: table-row} .mjx-cell {display: table-cell} .mjx-table {display: table; width: 100%} .mjx-line {display: block; height: 0} .mjx-strut {width: 0; padding-top: 1em} .mjx-vsize {width: 0} .MJXc-space1 {margin-left: .167em} .MJXc-space2 {margin-left: .222em} .MJXc-space3 {margin-left: .278em} .mjx-test.mjx-test-display {display: table!important} .mjx-test.mjx-test-inline {display: inline!important; margin-right: -1px} .mjx-test.mjx-test-default {display: block!important; clear: both} .mjx-ex-box {display: inline-block!important; position: absolute; overflow: hidden; min-height: 0; max-height: none; padding: 0; border: 0; margin: 0; width: 1px; height: 60ex} .mjx-test-inline .mjx-left-box {display: inline-block; width: 0; float: left} .mjx-test-inline .mjx-right-box {display: inline-block; width: 0; float: right} .mjx-test-display .mjx-right-box {display: table-cell!important; width: 10000em!important; min-width: 0; max-width: none; padding: 0; border: 0; margin: 0} .MJXc-TeX-unknown-R {font-family: monospace; font-style: normal; font-weight: normal} .MJXc-TeX-unknown-I {font-family: monospace; font-style: italic; font-weight: normal} .MJXc-TeX-unknown-B {font-family: monospace; font-style: normal; font-weight: bold} .MJXc-TeX-unknown-BI {font-family: monospace; font-style: italic; font-weight: bold} .MJXc-TeX-ams-R {font-family: MJXc-TeX-ams-R,MJXc-TeX-ams-Rw} .MJXc-TeX-cal-B {font-family: MJXc-TeX-cal-B,MJXc-TeX-cal-Bx,MJXc-TeX-cal-Bw} .MJXc-TeX-frak-R {font-family: MJXc-TeX-frak-R,MJXc-TeX-frak-Rw} .MJXc-TeX-frak-B {font-family: MJXc-TeX-frak-B,MJXc-TeX-frak-Bx,MJXc-TeX-frak-Bw} .MJXc-TeX-math-BI {font-family: MJXc-TeX-math-BI,MJXc-TeX-math-BIx,MJXc-TeX-math-BIw} .MJXc-TeX-sans-R {font-family: MJXc-TeX-sans-R,MJXc-TeX-sans-Rw} .MJXc-TeX-sans-B {font-family: MJXc-TeX-sans-B,MJXc-TeX-sans-Bx,MJXc-TeX-sans-Bw} .MJXc-TeX-sans-I {font-family: MJXc-TeX-sans-I,MJXc-TeX-sans-Ix,MJXc-TeX-sans-Iw} .MJXc-TeX-script-R {font-family: MJXc-TeX-script-R,MJXc-TeX-script-Rw} .MJXc-TeX-type-R {font-family: MJXc-TeX-type-R,MJXc-TeX-type-Rw} .MJXc-TeX-cal-R {font-family: MJXc-TeX-cal-R,MJXc-TeX-cal-Rw} .MJXc-TeX-main-B {font-family: MJXc-TeX-main-B,MJXc-TeX-main-Bx,MJXc-TeX-main-Bw} .MJXc-TeX-main-I {font-family: MJXc-TeX-main-I,MJXc-TeX-main-Ix,MJXc-TeX-main-Iw} .MJXc-TeX-main-R {font-family: MJXc-TeX-main-R,MJXc-TeX-main-Rw} .MJXc-TeX-math-I {font-family: MJXc-TeX-math-I,MJXc-TeX-math-Ix,MJXc-TeX-math-Iw} .MJXc-TeX-size1-R {font-family: MJXc-TeX-size1-R,MJXc-TeX-size1-Rw} .MJXc-TeX-size2-R {font-family: MJXc-TeX-size2-R,MJXc-TeX-size2-Rw} .MJXc-TeX-size3-R {font-family: MJXc-TeX-size3-R,MJXc-TeX-size3-Rw} .MJXc-TeX-size4-R {font-family: MJXc-TeX-size4-R,MJXc-TeX-size4-Rw} .MJXc-TeX-vec-R {font-family: MJXc-TeX-vec-R,MJXc-TeX-vec-Rw} .MJXc-TeX-vec-B {font-family: MJXc-TeX-vec-B,MJXc-TeX-vec-Bx,MJXc-TeX-vec-Bw} @font-face {font-family: MJXc-TeX-ams-R; src: local('MathJax_AMS'), local('MathJax_AMS-Regular')} @font-face {font-family: MJXc-TeX-ams-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_AMS-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_AMS-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_AMS-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-B; src: local('MathJax_Caligraphic Bold'), local('MathJax_Caligraphic-Bold')} @font-face {font-family: MJXc-TeX-cal-Bx; src: local('MathJax_Caligraphic'); font-weight: bold} @font-face {font-family: MJXc-TeX-cal-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-R; src: local('MathJax_Fraktur'), local('MathJax_Fraktur-Regular')} @font-face {font-family: MJXc-TeX-frak-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-B; src: local('MathJax_Fraktur Bold'), local('MathJax_Fraktur-Bold')} @font-face {font-family: MJXc-TeX-frak-Bx; src: local('MathJax_Fraktur'); font-weight: bold} @font-face {font-family: MJXc-TeX-frak-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-BI; src: local('MathJax_Math BoldItalic'), local('MathJax_Math-BoldItalic')} @font-face {font-family: MJXc-TeX-math-BIx; src: local('MathJax_Math'); font-weight: bold; font-style: italic} @font-face {font-family: MJXc-TeX-math-BIw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-BoldItalic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-BoldItalic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-BoldItalic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-R; src: local('MathJax_SansSerif'), local('MathJax_SansSerif-Regular')} @font-face {font-family: MJXc-TeX-sans-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-B; src: local('MathJax_SansSerif Bold'), local('MathJax_SansSerif-Bold')} @font-face {font-family: MJXc-TeX-sans-Bx; src: local('MathJax_SansSerif'); font-weight: bold} @font-face {font-family: MJXc-TeX-sans-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-I; src: local('MathJax_SansSerif Italic'), local('MathJax_SansSerif-Italic')} @font-face {font-family: MJXc-TeX-sans-Ix; src: local('MathJax_SansSerif'); font-style: italic} @font-face {font-family: MJXc-TeX-sans-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-script-R; src: local('MathJax_Script'), local('MathJax_Script-Regular')} @font-face {font-family: MJXc-TeX-script-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Script-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Script-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Script-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-type-R; src: local('MathJax_Typewriter'), local('MathJax_Typewriter-Regular')} @font-face {font-family: MJXc-TeX-type-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Typewriter-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Typewriter-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Typewriter-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-R; src: local('MathJax_Caligraphic'), local('MathJax_Caligraphic-Regular')} @font-face {font-family: MJXc-TeX-cal-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-B; src: local('MathJax_Main Bold'), local('MathJax_Main-Bold')} @font-face {font-family: MJXc-TeX-main-Bx; src: local('MathJax_Main'); font-weight: bold} @font-face {font-family: MJXc-TeX-main-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-I; src: local('MathJax_Main Italic'), local('MathJax_Main-Italic')} @font-face {font-family: MJXc-TeX-main-Ix; src: local('MathJax_Main'); font-style: italic} @font-face {font-family: MJXc-TeX-main-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-R; src: local('MathJax_Main'), local('MathJax_Main-Regular')} @font-face {font-family: MJXc-TeX-main-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-I; src: local('MathJax_Math Italic'), local('MathJax_Math-Italic')} @font-face {font-family: MJXc-TeX-math-Ix; src: local('MathJax_Math'); font-style: italic} @font-face {font-family: MJXc-TeX-math-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size1-R; src: local('MathJax_Size1'), local('MathJax_Size1-Regular')} @font-face {font-family: MJXc-TeX-size1-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size1-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size1-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size1-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size2-R; src: local('MathJax_Size2'), local('MathJax_Size2-Regular')} @font-face {font-family: MJXc-TeX-size2-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size2-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size2-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size2-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size3-R; src: local('MathJax_Size3'), local('MathJax_Size3-Regular')} @font-face {font-family: MJXc-TeX-size3-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size3-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size3-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size3-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size4-R; src: local('MathJax_Size4'), local('MathJax_Size4-Regular')} @font-face {font-family: MJXc-TeX-size4-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size4-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size4-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size4-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-R; src: local('MathJax_Vector'), local('MathJax_Vector-Regular')} @font-face {font-family: MJXc-TeX-vec-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-B; src: local('MathJax_Vector Bold'), local('MathJax_Vector-Bold')} @font-face {font-family: MJXc-TeX-vec-Bx; src: local('MathJax_Vector'); font-weight: bold} @font-face {font-family: MJXc-TeX-vec-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Bold.otf') format('opentype')} of two blocks of random information A,B equals the sum of the two entropies SA,SB because random information is uncompressible.

SA+B=SA+SB (uncompressible data)

Suppose instead you have two blocks of non-random information C,D such that C,D are related. Then the combined entropy SC+D is less than the sum of the individual entropies.

SC+D<SC+SD (compressible data)

In practice, part of this message can be though of as encoding ontologies (generalizations) and part of the message can be thought of as encoding specifics.

For example consider principal component analysis where you calculate the eigenvectors of a dataset, throw away all eigenvectors with small eigenvalues, and then project your original dataset into this lower-dimensional space. The eigenvectors with large eigenvalues can be through of as encoding ontologies and the projection of your dataset into this new basis can be thought of as encoding specifics.

As you compress more and more information, the uncompressed ratio of ontologies to specifics increases. This is the fundamental principal behind transfer learning and the last conceptual hurdle between today's technology and human-equivalent artificial intelligence.

Compressible Data is Fractal

The word "fractal" describes a self-similar mathematical structure with a fractal dimension Df greater than its topological dimension Dt.

Compressible data meets both requirements.

  • Compressed data is trivially self-similar.
  • The topological length lt of a block of information is equal to its raw uncompressed length. The fractal length lf of a block of information is equal to its entropy. D_t">Df>Dt follows from SC+D<SC+SD.

Compressed data is fractal too because compressed data is isomorphic to compressible data.

Generalizing a dataset and compressing it are the same thing. An artificial general intelligence equals a general-purpose compression algorithm. If a general purpose compression algorithm is to scale to arbitrary levels of complexity then it must encode data fractally.

  • It is no coincidence human brainwaves exhibit a fractal structure. General intelligence, including AGI, is necessarily fractal.
  • A multilayer perceptron scales badly on hierarchically complex data because the multilayer perceptron's fractal dimension 1 equals its topological dimension 1.


Discuss

Emptiness and Form

Новости LessWrong.com - 22 сентября, 2020 - 07:20
Published on September 22, 2020 4:20 AM GMT

Translation note: There is no English equivalent to the Sanskrit words शून्यता śūnyatā and रूप rūpa. By convention, शून्यता is translated "emptiness" and रूप is translated "form". I follow this convention. My use of the words "emptiness" and "form" in this post have little to do with the English words "emptiness" and "form"; they are placeholders for Sanskrit.

Consider a cat. From the perspective of fundamental physics, the cat is a collection of particles no more special than any other collection of particles. There is no clear line between "cat" and "non-cat". Everything is quantum fields. The "cat" is a representation created by the human mind. It is a trick of human perspective. From the perspective of an omniscient unbiased observer, the cat is just a scoop of water in a limitless ocean.

Cats are real.

The perspective "cats are real" is called "form". The perspective "cats are an arbitrary ontology with no well-defined meaning amongst the fundamental laws of the universe" is called "emptiness". There is no conflict between form and emptiness just as there is no conflict between quantum mechanics and classical mechanics. They are different ways interpreting the same thing at different scales.

Classical mechanics can be more practical than quantum mechanics even though quantum mechanics is more fundamental than classical mechanics. Similarly, emptiness is more fundamental than form yet form is a more useful model of the world than emptiness. Emptiness and form are neither equally true nor equally practical.

Maps ≠ Form & Emptiness ≠ Territory

You could say "form" roughly corresponds to "maps" and "emptiness" roughly corresponds to "territory". That would constitute a better translation from the original Sanskrit than "form" and "emptiness". But the form-emptiness dichotomy draws its line in a slightly different place than the map-territory dichotomy.

The map-territory dichotomy draws the line between reality and models of reality. The map-territory dichotomy distinguishes between reality and one's simplified models of reality. In this way, the map-territory dichotomy is a materialist perspective.

The form-emptiness dichotomy is an informatic perspective. If there is no difference between a map and a territory then—mathematically—the map and the territory are isomorphic respresentations of the same group.

Ontologies

"Emptiness" describes a shared quality between the reductionist nature of objective reality and the raw sensory data coming into a mind. In both cases, our Bayesian priors bucket high-dimensional data into into an ontology called "form".

In other words, form is a byproduct of subjectivity. All ontologies dissolve under the scrutiny of theoretical physics.

The duality between emptiness and form is fundamental to general intelligence.

Discreteness and Differentiability

Big data is easy. The hard problem of general intelligence concerns small data. Small data is all about transfer learning. Transfer learning is all about ontologies.

An intelligent system with hard-coded ontologies is conceptually adaptable and therefore not a general intelligence. A general intelligence's ontologies must be emergent from its input data. But ontologies are discrete and the only way to navigate a high-dimensional input data is via the gradient descent algorithm. But the gradient descent algorithm requires a continuous representation. Can a representation be both continuous and discrete?

In theory, no. In practice, yes.

Consider the sigmoid function in the multilayer perceptron.

output=tanh(input)=11+einput.mjx-chtml {display: inline-block; line-height: 0; text-indent: 0; text-align: left; text-transform: none; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; word-wrap: normal; word-spacing: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0; min-height: 0; border: 0; margin: 0; padding: 1px 0} .MJXc-display {display: block; text-align: center; margin: 1em 0; padding: 0} .mjx-chtml[tabindex]:focus, body :focus .mjx-chtml[tabindex] {display: inline-table} .mjx-full-width {text-align: center; display: table-cell!important; width: 10000em} .mjx-math {display: inline-block; border-collapse: separate; border-spacing: 0} .mjx-math * {display: inline-block; -webkit-box-sizing: content-box!important; -moz-box-sizing: content-box!important; box-sizing: content-box!important; text-align: left} .mjx-numerator {display: block; text-align: center} .mjx-denominator {display: block; text-align: center} .MJXc-stacked {height: 0; position: relative} .MJXc-stacked > * {position: absolute} .MJXc-bevelled > * {display: inline-block} .mjx-stack {display: inline-block} .mjx-op {display: block} .mjx-under {display: table-cell} .mjx-over {display: block} .mjx-over > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-under > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-stack > .mjx-sup {display: block} .mjx-stack > .mjx-sub {display: block} .mjx-prestack > .mjx-presup {display: block} .mjx-prestack > .mjx-presub {display: block} .mjx-delim-h > .mjx-char {display: inline-block} .mjx-surd {vertical-align: top} .mjx-mphantom * {visibility: hidden} .mjx-merror {background-color: #FFFF88; color: #CC0000; border: 1px solid #CC0000; padding: 2px 3px; font-style: normal; font-size: 90%} .mjx-annotation-xml {line-height: normal} .mjx-menclose > svg {fill: none; stroke: currentColor} .mjx-mtr {display: table-row} .mjx-mlabeledtr {display: table-row} .mjx-mtd {display: table-cell; text-align: center} .mjx-label {display: table-row} .mjx-box {display: inline-block} .mjx-block {display: block} .mjx-span {display: inline} .mjx-char {display: block; white-space: pre} .mjx-itable {display: inline-table; width: auto} .mjx-row {display: table-row} .mjx-cell {display: table-cell} .mjx-table {display: table; width: 100%} .mjx-line {display: block; height: 0} .mjx-strut {width: 0; padding-top: 1em} .mjx-vsize {width: 0} .MJXc-space1 {margin-left: .167em} .MJXc-space2 {margin-left: .222em} .MJXc-space3 {margin-left: .278em} .mjx-test.mjx-test-display {display: table!important} .mjx-test.mjx-test-inline {display: inline!important; margin-right: -1px} .mjx-test.mjx-test-default {display: block!important; clear: both} .mjx-ex-box {display: inline-block!important; position: absolute; overflow: hidden; min-height: 0; max-height: none; padding: 0; border: 0; margin: 0; width: 1px; height: 60ex} .mjx-test-inline .mjx-left-box {display: inline-block; width: 0; float: left} .mjx-test-inline .mjx-right-box {display: inline-block; width: 0; float: right} .mjx-test-display .mjx-right-box {display: table-cell!important; width: 10000em!important; min-width: 0; max-width: none; padding: 0; border: 0; margin: 0} .MJXc-TeX-unknown-R {font-family: monospace; font-style: normal; font-weight: normal} .MJXc-TeX-unknown-I {font-family: monospace; font-style: italic; font-weight: normal} .MJXc-TeX-unknown-B {font-family: monospace; font-style: normal; font-weight: bold} .MJXc-TeX-unknown-BI {font-family: monospace; font-style: italic; font-weight: bold} .MJXc-TeX-ams-R {font-family: MJXc-TeX-ams-R,MJXc-TeX-ams-Rw} .MJXc-TeX-cal-B {font-family: MJXc-TeX-cal-B,MJXc-TeX-cal-Bx,MJXc-TeX-cal-Bw} .MJXc-TeX-frak-R {font-family: MJXc-TeX-frak-R,MJXc-TeX-frak-Rw} .MJXc-TeX-frak-B {font-family: MJXc-TeX-frak-B,MJXc-TeX-frak-Bx,MJXc-TeX-frak-Bw} .MJXc-TeX-math-BI {font-family: MJXc-TeX-math-BI,MJXc-TeX-math-BIx,MJXc-TeX-math-BIw} .MJXc-TeX-sans-R {font-family: MJXc-TeX-sans-R,MJXc-TeX-sans-Rw} .MJXc-TeX-sans-B {font-family: MJXc-TeX-sans-B,MJXc-TeX-sans-Bx,MJXc-TeX-sans-Bw} .MJXc-TeX-sans-I {font-family: MJXc-TeX-sans-I,MJXc-TeX-sans-Ix,MJXc-TeX-sans-Iw} .MJXc-TeX-script-R {font-family: MJXc-TeX-script-R,MJXc-TeX-script-Rw} .MJXc-TeX-type-R {font-family: MJXc-TeX-type-R,MJXc-TeX-type-Rw} .MJXc-TeX-cal-R {font-family: MJXc-TeX-cal-R,MJXc-TeX-cal-Rw} .MJXc-TeX-main-B {font-family: MJXc-TeX-main-B,MJXc-TeX-main-Bx,MJXc-TeX-main-Bw} .MJXc-TeX-main-I {font-family: MJXc-TeX-main-I,MJXc-TeX-main-Ix,MJXc-TeX-main-Iw} .MJXc-TeX-main-R {font-family: MJXc-TeX-main-R,MJXc-TeX-main-Rw} .MJXc-TeX-math-I {font-family: MJXc-TeX-math-I,MJXc-TeX-math-Ix,MJXc-TeX-math-Iw} .MJXc-TeX-size1-R {font-family: MJXc-TeX-size1-R,MJXc-TeX-size1-Rw} .MJXc-TeX-size2-R {font-family: MJXc-TeX-size2-R,MJXc-TeX-size2-Rw} .MJXc-TeX-size3-R {font-family: MJXc-TeX-size3-R,MJXc-TeX-size3-Rw} .MJXc-TeX-size4-R {font-family: MJXc-TeX-size4-R,MJXc-TeX-size4-Rw} .MJXc-TeX-vec-R {font-family: MJXc-TeX-vec-R,MJXc-TeX-vec-Rw} .MJXc-TeX-vec-B {font-family: MJXc-TeX-vec-B,MJXc-TeX-vec-Bx,MJXc-TeX-vec-Bw} @font-face {font-family: MJXc-TeX-ams-R; src: local('MathJax_AMS'), local('MathJax_AMS-Regular')} @font-face {font-family: MJXc-TeX-ams-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_AMS-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_AMS-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_AMS-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-B; src: local('MathJax_Caligraphic Bold'), local('MathJax_Caligraphic-Bold')} @font-face {font-family: MJXc-TeX-cal-Bx; src: local('MathJax_Caligraphic'); font-weight: bold} @font-face {font-family: MJXc-TeX-cal-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-R; src: local('MathJax_Fraktur'), local('MathJax_Fraktur-Regular')} @font-face {font-family: MJXc-TeX-frak-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-B; src: local('MathJax_Fraktur Bold'), local('MathJax_Fraktur-Bold')} @font-face {font-family: MJXc-TeX-frak-Bx; src: local('MathJax_Fraktur'); font-weight: bold} @font-face {font-family: MJXc-TeX-frak-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-BI; src: local('MathJax_Math BoldItalic'), local('MathJax_Math-BoldItalic')} @font-face {font-family: MJXc-TeX-math-BIx; src: local('MathJax_Math'); font-weight: bold; font-style: italic} @font-face {font-family: MJXc-TeX-math-BIw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-BoldItalic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-BoldItalic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-BoldItalic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-R; src: local('MathJax_SansSerif'), local('MathJax_SansSerif-Regular')} @font-face {font-family: MJXc-TeX-sans-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-B; src: local('MathJax_SansSerif Bold'), local('MathJax_SansSerif-Bold')} @font-face {font-family: MJXc-TeX-sans-Bx; src: local('MathJax_SansSerif'); font-weight: bold} @font-face {font-family: MJXc-TeX-sans-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-I; src: local('MathJax_SansSerif Italic'), local('MathJax_SansSerif-Italic')} @font-face {font-family: MJXc-TeX-sans-Ix; src: local('MathJax_SansSerif'); font-style: italic} @font-face {font-family: MJXc-TeX-sans-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-script-R; src: local('MathJax_Script'), local('MathJax_Script-Regular')} @font-face {font-family: MJXc-TeX-script-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Script-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Script-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Script-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-type-R; src: local('MathJax_Typewriter'), local('MathJax_Typewriter-Regular')} @font-face {font-family: MJXc-TeX-type-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Typewriter-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Typewriter-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Typewriter-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-R; src: local('MathJax_Caligraphic'), local('MathJax_Caligraphic-Regular')} @font-face {font-family: MJXc-TeX-cal-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-B; src: local('MathJax_Main Bold'), local('MathJax_Main-Bold')} @font-face {font-family: MJXc-TeX-main-Bx; src: local('MathJax_Main'); font-weight: bold} @font-face {font-family: MJXc-TeX-main-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-I; src: local('MathJax_Main Italic'), local('MathJax_Main-Italic')} @font-face {font-family: MJXc-TeX-main-Ix; src: local('MathJax_Main'); font-style: italic} @font-face {font-family: MJXc-TeX-main-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-R; src: local('MathJax_Main'), local('MathJax_Main-Regular')} @font-face {font-family: MJXc-TeX-main-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-I; src: local('MathJax_Math Italic'), local('MathJax_Math-Italic')} @font-face {font-family: MJXc-TeX-math-Ix; src: local('MathJax_Math'); font-style: italic} @font-face {font-family: MJXc-TeX-math-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size1-R; src: local('MathJax_Size1'), local('MathJax_Size1-Regular')} @font-face {font-family: MJXc-TeX-size1-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size1-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size1-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size1-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size2-R; src: local('MathJax_Size2'), local('MathJax_Size2-Regular')} @font-face {font-family: MJXc-TeX-size2-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size2-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size2-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size2-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size3-R; src: local('MathJax_Size3'), local('MathJax_Size3-Regular')} @font-face {font-family: MJXc-TeX-size3-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size3-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size3-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size3-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size4-R; src: local('MathJax_Size4'), local('MathJax_Size4-Regular')} @font-face {font-family: MJXc-TeX-size4-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size4-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size4-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size4-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-R; src: local('MathJax_Vector'), local('MathJax_Vector-Regular')} @font-face {font-family: MJXc-TeX-vec-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-B; src: local('MathJax_Vector Bold'), local('MathJax_Vector-Bold')} @font-face {font-family: MJXc-TeX-vec-Bx; src: local('MathJax_Vector'); font-weight: bold} @font-face {font-family: MJXc-TeX-vec-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Bold.otf') format('opentype')}

If we zoom in on this function we can see it is continuously differentiable.

But when we zoom out it appears as a discrete step function.

The sigmoid function illustrates the scale-dependence of emptiness and form. When we zoom in we see continuity (emptiness), which is a prerequisite for gradient descent. When we zoom out, we see a discrete system (form), which is necessary for the emergence of ontologies. Emptiness and form work together to produce emergent ontologies.



Discuss

Forecasting Thread: Existential Risk

Новости LessWrong.com - 22 сентября, 2020 - 06:44
Published on September 22, 2020 3:44 AM GMT

This is a thread for displaying your probabilities of an existential catastrophe that causes extinction or the destruction of humanity’s long-term potential.

Every answer to this post should be a forecast showing your probability of an existential catastrophe happening at any given time.

For example, here is Michael Aird’s timeline:

The goal of this thread is to create a set of comparable, standardized x-risk predictions, and to facilitate discussion on the reasoning and assumptions behind those predictions. The thread isn’t about setting predictions in stone – you can come back and update at any point!

 

How to participate

  1. Go to this page
  2. Create your distribution
    • Specify an interval using the Min and Max bin, and put the probability you assign to that interval in the probability bin.
    • You can specify a cumulative probability by leaving the Min box blank and entering the cumulative value in the Max box.
    • To put probability on never, assign probability above January 1, 2120 using the edit button to the right of the graph. Specify your probability for never in the notes, to distinguish this from putting probability on existential catastrophe occurring after 2120.
  3. Click 'Save snapshot' to save your distribution to a static URL
    • A timestamp will appear below the 'Save snapshot' button. This links to the URL of your snapshot.
    • Make sure to copy it before refreshing the page, otherwise it will disappear.
  4. Click ‘Log in’ to automatically show your snapshot on the Elicit question page
    • You don’t have to log in, but if you do, Elicit will:
      • Store your snapshot in your account history so you can easily access it.
      • Automatically add your most recent snapshot to the x-risk question page under ‘Show more’. Other users will be able to import your most recent snapshot from the dropdown.
    • We’ll set a default name that your snapshot will be shown under – if you want to change it, you can do so on your profile page.
    • If you’re logged in, your snapshots for this question will be publicly viewable.
  5. Copy the snapshot timestamp link and paste it into your LessWrong comment
    • You can also add a screenshot of your distribution in your comment using the instructions below.

 

How to add an image to your comment

  1. Take a screenshot of your distribution
  2. Then do one of two things:
    1. If you have beta-features turned on in your account settings, drag-and-drop the image into your comment
    2. If not, upload it to an image hosting service like imgur.com, then write the following markdown syntax for the image to appear, with the url appearing where it says ‘link’: ![](link)
  3. If it worked, you will see the image in the comment before hitting submit.

 

If you have any bugs or technical issues, reply to Ben from the LW team or Amanda (me) from the Ought team in the comment section, or email me at amanda@ought.org.

 

Questions to consider as you're making your prediction

  • What definitions are you using? It’s helpful to specify them.
  • What evidence is driving your prediction?
  • What are the main assumptions that other people might disagree with?
  • What evidence would cause you to update?
  • How is the probability mass allocated amongst x-risk scenarios?
  • Would you bet on these probabilities?


Discuss

How often do series C startups fail to exit?

Новости LessWrong.com - 22 сентября, 2020 - 06:22
Published on September 21, 2020 7:37 PM GMT

How often do series C startups really fail? By fail I mean never have an acquisition or IPO. Internet says 80% (see https://medium.com/journal-of-empirical-entrepreneurship/dissecting-startup-failure-by-stage-34bb70354a36) but this seems very high to me.

Most Series C companies are worth in the 100-200M range, the one I'm at is worth 270M. How does all the value just evaporate? What happens to the companies that "fail"?

Asking to decide whether to exercise my options. I only need my company to exit at 41M to break even. I am bearish on the company but with around 40M in ARR it is hard to imagine it not exiting.



Discuss

What AI companies would be most likely to have a positive long-term impact on the world as a result of investing in them?

Новости LessWrong.com - 22 сентября, 2020 - 02:41
Published on September 21, 2020 11:41 PM GMT

Ever since GPT-3 was unveiled, I've been thinking pretty heavily about increasing my investment in AI-related companies. My first thoughts were to invest in Microsoft and Alphabet (Google) - Microsoft because they are partnered with OpenAI, and Alphabet since they have big AI reseach projects of their own. But in the process of thinking about investing in these companies, I started wondering about the long-term impacts such investments would have on the world - investing in the right or wrong company could dramatically change how the world looks 20 years from now, and whether it is a place I'd want to live in - the worst case scenario would be all humans dead, or even worse; best case scenario is... too amazing to put into words. And then there's plenty of room in between those for how things can go, depending on who makes the important decisions, and how good the decisions they make will be. (While I'm only a single person with modest funds to invest in companies, I also consider that my actions are acausally correlated with those of others sufficiently similar to me, which means the acausal results of any investment I make will be multiplied by an amount that makes my decisions have non-trivial impact on the world).

So the important question is, do I expect Microsoft and Alphabet to do better or worse, in regards to alignment and ethical issues, compared to other actors who will develop AGI in their lieu? (I do expect someone will develop AGI in their lieu) I can think of actors who I expect will likely do worse than Microsoft or Alphabet - the government of basically any country, or firms based in a country with more totalitarian ethics than the US - wheras I can only think of alternative actors who I expect to do roughly as good as Microsoft or Alphabet, but not neccesarily to do better than them. I trust MIRI, but I also don't perceive MIRI as being actively involved in the development of working AI systems; it seems to me that they are laying the important theoretical groundwork for getting things right, but aren't in position to be the ones who actually do the work that needs to be gotten right.

So my main problem here is a lack of knowledge - there almost certainly are other firms who, if I had the relevant information, I would expect would do better on alignment and ethical issues than Microsoft or Alphabet, but I also don't know who those firms are, or why I should expect them to do so. So my question is, for an investor looking to make a AGI-sized profit off of AGI, but also cares about what the future looks like as a result of such investment, what companies will be most likely to result in a good long-term future for humanity?

Note that I'm not asking which company will make the most profit - as long as I reasonably expect that a company will make an AGI-sized profit, that's all I care about on that front. What matters is the impact it has on the desirablity of the future world it will lead to. I'm also not asking about organizations to donate to, because while that is important, that's not the problem I'm chewing over right now.



Discuss

Prepare for COVID-19 Human Challenge Trials — A Petition in Canada (and soon, the UK)

Новости LessWrong.com - 21 сентября, 2020 - 22:57
Published on September 21, 2020 7:38 PM GMT

Canadians— sign the petition here.

TL;DR

COVID-19 human challenge trials could save tens of thousands of lives by quickly narrowing the field of promising candidates, and there is strong reason to believe that signaling clear public support for these trials via an official petition could meaningfully accelerate preparation.

Why COVID-19 Human Challenge Trials?

In a COVID-19 human challenge trial, willing participants would receive the vaccine candidate and, once the vaccine takes effect, be deliberately exposed to live coronavirus. The ability to observe participants closely and gather samples while tracing the progress of infection in real time, knowing exactly when they were infected and with what dose, and being able to follow up over a long period, would offer an unprecedented level of scientific and medical insight into an unfamiliar virus. It would also help us test vaccines far faster. If a challenge trial brings us one day closer to the development of an additional vaccine that could avert just 25% of daily COVID-19 deaths, it would save 1,250 lives. If a challenge trial brings us a month closer, it’d save 37,500 lives.

To learn more about COVID-19 human challenge trials:

  • Watch this Vox video
  • Read this piece by Dr. Sayantan Banerjee in The Telegraph
  • Read this paper in the Journal of Clinical Infectious Diseases
Why Is It Cost-Effective To Sign The Petition?

For one, it is remarkably easy (takes around 20 seconds), so even a very small chance that your signature makes a difference tips the scale in any cost-effectiveness calculation on the margin.

More broadly, though, signaling a groundswell of public support for COVID-19 human challenge trials has directly led to faster preparation for these trials. In May, a NIH document noted that their consideration of challenge trials “has been driven almost entirely by the altruism of potential volunteer advocates and the intense considerations of bioethicists.”

1Day Sooner, which has worked systematically to include volunteers in the public conversation about challenge trials, launched an open letter in support of COVID-19 challenge trials on July 15 that was signed by over 100 academics and experts as well as 2,000 potential challenge trial volunteers. A week later, the Washington Post Editorial Board wrote in favor of challenge trial preparation. Within a few weeks, Reuters reported that the National Institutes of Health were preparing a coronavirus strain for a COVID-19 challenge trial, in part due to “pressure from advocacy groups such as 1Day Sooner.”

The logic behind the effectiveness of public advocacy for challenge trials is that vaccine developers want assurance that their decision to deliberately infect people with a dangerous virus won’t prompt public backlash. By making clear that the public is actually on board with these trials, stakeholders have a safety net to move forward.

We are now launching a Canada and UK petition campaign because Oxford’s Jenner Institute and several Canadian MPs have signaled interest in conducting a COVID-19 human challenge trial. By showing broad support for these trials, we hope to make it easier for more stakeholders to come out in favor of these trials.



Discuss

Клуб чтения цепочек

События в Кочерге - 21 сентября, 2020 - 19:30
Якорение, прайминг, контаминация, кеширование мыслей… Как бороться с проявлениями эвристики доступности в собственному мозгу и принимать взвешенные решения? На следующей встрече клуба чтения цепочек приступим к «Свежему взгляду на вещи» а также подведем итоги обсуждения цепочки «Против двоемыслия». Встреча состоится в понедельник, 21-го сентября, в 19:30 по Москве. Присоединяйтесь поучаствовать в обсуждении или послушать, а также добавляйте ваши мысли и вопросы в общий конспект.

Страницы

Подписка на LessWrong на русском сбор новостей