Вы здесь

Новости LessWrong.com

Подписка на Лента Новости LessWrong.com Новости LessWrong.com
A community blog devoted to refining the art of rationality
Обновлено: 25 минут 56 секунд назад

Covid-19: New York’s Antibody Tests 2

6 мая, 2020 - 20:40
Published on May 6, 2020 5:40 PM GMT

Previously: On New York’s Antibody Tests

New York continues to be the only place even trying to do antibody tests in a way that involves releasing information on the population.

What do we now know? How should we update? What comes next?

As a reminder, the method of data collection seems to be to set up in a randomly selected set of geographically distributed grocery stores, with volunteers who are then told their status afterwards.

Here are the third wave slides.

Cuomo expressed confusion that these numbers, in some places, were moving in the ‘wrong’ direction. For any given area or group, infection numbers, and thus the percentage of the population with antibodies, must (of course) go up over time rather than down. Any numbers that declined from 4/22 to 4/27, or from 4/27 t o 5/2, represent some sort of contradiction or error.

The type of contradictions/errors we see, and their magnitude, tells us a lot about how this data was gathered, and how those errors and contradictions came to be. They also tell us, in some ways, better information than if we had only the average of the results from all three days.

We still do have a big mystery on our hands. The numbers dropped a lot when they should have increased. It’s not easy to explain it away.

Time Is Marching On

What should we expect to see in relational terms between results on 4/22, 4/27 and 5/2, assuming a short delay between testing and reporting? Five days should allow for some growth in the net number of infections, and in the number of those who have recovered and how have antibodies. As with last time, I’ll assume two weeks from infection to start testing positive on an antibody test. I will use my current best projections of the true infected rate, which can be found on the “Scrapwork” tab of my Covid-19 information spreadsheet, to estimate ratios of total infected. Even if I’m wrong about levels, the ratios shouldn’t be too far off.

My best previous guess for infected in New York State on 4/6, two weeks plus two testing and reporting days before 4/22, is 2.36 million. On 4/11, sixteen days before 4/27, it is 2.86 million. On 4/16, it is 3.25 million. That’s 21% growth followed by 13% growth (or 38% combined growth). That’s a lot. Changing the lag time can cut these numbers a lot. If the results had no delay from infection to reporting a positive result, we’d only expect 11% growth. So perhaps we’re overestimating the delay factor, but most of that delay is based on physical things that can’t change much. And the effect definitely runs the ‘wrong’ way for the drop going to 5/2.

The law of large numbers says that very little of this is individual tests randomly being positive or negative, either from error or from people happening to be positive or negative. This is not random chance at the individual level. Something systematic happened.

Break It Down Sex

Men were 32.5% more likely than women to be positive in the initial sample, 29% more likely in the second sample, and 14% more likely in the third sample.

Geographic variation shouldn’t be able to make a big difference here. If you set up in different grocery stores in different locations, these ratios should mostly stay the same. What could account for such a big difference?

One possibility is that some stores in some areas more male or female focused. This theory has the added advantage that it could help explain the overall numbers, by shifting from more male-focused stores in New York City to more female-focused stores in New York City, or from times of the day or week that favor males to those that favor females. That would make the sexes look more equal and could also lower overall infection rates, if these effects were big enough. However, samples overall remain 52% female, and this seems like the kind of thing that would get noticed, so that all seems unlikely.

A related possibility is that different areas have different male-female patterns of infection. One major source of infection is prisons and courts, which is predominantly male. We can imagine other similar cases. The extent to which such cases dominate in the area could matter. Again, if we had enough different collection sites chosen randomly this would cancel out, but it would still be nice to either do this truly randomly or have good cross-tabs and see what the data says within.

It seems more likely that this is related to methods of recruitment and participation. The samples are self-selecting in some sort of way that has changing bias over time. Perhaps some policies and procedures involved in collection changed? That could explain other changes, as well. Perhaps people’s overall attitudes changed over time as overall conditions changed and the tests became more public in general. That seems less plausibly big enough, but possible.

The 4/27 tests might have been collected on weekends, but the 4/22 tests then likely weren’t, so that doesn’t work either.

Another possibility is that the tests aren’t the same, and the error rates were different? I presume we would know if the tests were importantly different, but perhaps not, and perhaps needing to run so many tests could change the effective error rates. But that doesn’t give us a way to get from a 3.8% difference to a 1.6% difference, unless error rates for men and women were very differently changed, and that would be super weird.

Another approach might be, which ratio makes more sense? Consider this (source) from April 10:

Data from around 1.5 million tests done in the U.S. show that the majority of people tested, 56%, were women. Of those women, 16% tested positive for the virus. In contrast, only 44% of the tests were done on men. And 23% of them tested positive.

And also consider this New York Times report from April 7:

To date, there have been nearly 43 Covid-19 deaths for every 100,000 men in the city, compared with 23 such deaths for every 100,000 women, according to figures reported by the city’s health department. And men are being hospitalized with severe disease at higher rates.

While there were 712 cases per 100,000 women in the city, there were 932 cases for every 100,000 men.

That’s a ratio of 31% additional cases in early New York City. Men, I would assume, if anything seek and get less health care than women, which the other data point is consistent with. So such a close ratio of positive antibody test rates seems to me like an artifact. I’m therefore more inclined to think the new data has more of the data issues.

Regions

It’s worth noting that North County has a 1.2% positive rate in both time periods. In the previous post we estimated a 2.1% false positive rate, but if we have a region with 1.2% positive rates, that doesn’t seem possible. That means the false positive rate is mostly constant over time, and has an obvious upper bound of 1.2%. I’m going to revise my guess to about 0.8% false positives. Still have no idea what the false negative rate might be.

In New York City, rates went from 24.7% to 19.9%. That’s a 19.5% decrease.

For New York State, rates went from 14.9% to 12.3%. That’s an 18.5% decrease.

I notice that the slides we have here don’t mention Long Island or Westchester/Rockland. Presumably they show the same phenomenon as New York City or even larger, since the declines in upstate are net 10% or less and they’re a third of the state. There also might be some small compositional changes that weren’t accounted for.

So the question becomes, why is the new method showing lower rates in the city, but not elsewhere?

One hypothesis is that there was a behavior change after the first results, that likely was driven by the antibody results.

Does anyone else remember the trending hashtag “21 Percent of NYC” on Twitter?

Before those results, mainstream reporting was giving insane low rates of infection for New York. They reported positive test results in a world where most people can’t get tested, and think a few percent are infected in the city. I estimated that the real numbers were fifteen times higher. 

Then the antibody test results come out at 21% infected for the city. There were lots of reasons to put huge error bars on that result, but it was the New Official Number of sorts, and not of the same order of magnitude.

It was no longer possible to pretend that huge numbers of people weren’t infected.

People actually overestimated the number of infected. They started thinking intuitively that one person in five out there has Covid-19. Whereas what was actually found was that one person in five has had Covid-19, which is very different once you’re in the downward part of the curve. The number walking around a grocery store potentially symptomatic was at the time already at most 10% of the general positive rate (if you give people a five day window to be out and infectious, which is generous), likely smaller than that, and continuing to shrink.

But suddenly, we drove the concept home. People may have started taking things more seriously, in various ways. And that might change who shows up in grocery stores, perhaps? I don’t have a good way to check such data myself, but others could.

It could also be a general trend of changing behavior.

From 4/26 to 5/5, the rate of positive test results in New York State was cut roughly in half in only nine days. The previous halving took roughly nineteen days. R0 dropped a lot and now is plausibly south of 0.75 – we’ll know more every day, as we always do, and I keep slowly revising downward. Which means some combination of people changing behavior, increases in effective herd immunity, or the composition of who is infected changing in interesting ways, or most likely some combination thereof.

Breaking things down by borough, I’m actually suspicious of Manhattan being so close in infection rates to the other boroughs other than The Bronx, given what I know about the physical situation. Perhaps a lot of the infection happened very early, before people fled. Perhaps the people who did flee were mostly negative, and that helped equalize rates.

Race and Ethnicity

The numbers here jump around more than they ‘should’ from random chance. That tells me that some of this story is some sort of systematic change. The Latino positive rate changes are especially glaring. This seems to tell a strong story that the exact data gathering locations are having a strong influence on what subpopulations are giving us data, and thus the samples on different days are not as comparable as we would like them to be.

And again, the fact that we don’t have more of the details that were doubtless collected is getting in the way of getting a better understanding.

The white infection rate is down from 9% to 7% in the new sample. That’s a huge decline, bigger than the overall decline, despite the white share of upstate (where things didn’t change much) being much bigger than the white share of downstate (where measured rates cratered). What do we make of this? At a minimum, it probably means that we didn’t get the decline by shifting from non-white to white areas of New York City. It’s very confusing.

Age

If we aggregate the first two data sets and compare to the third, we see a decline in rates across all age groups except the youngest. We see 35-44 being less infected than the group on either side, in all three samples, which is super weird. It’s like the younger people get more infected because they don’t care as much, and older people are more vulnerable but don’t change behavior enough to compensate? Or perhaps people in the 35-44 range are exactly old enough to have stay at home jobs, whereas 45-54s grew up before the relevant technologies and are more likely to have to venture out?

But mostly these are Just So Stories, made up after seeing the data. They don’t count for much. And not to be too morbid, but the higher death rates and higher complication rates at higher age groups (65-74 and 75+) likely fully explain their measured infection rates being lower.

How Much to Update?

We still have huge potential sources of bias in both directions. The new data doesn’t change that. The questions are, what do we believe, and how can we learn more?

The second test gave us about the results we’d have expected after the first test. The third test gave us much lower results than we would have expected. We can suspect various things but that doesn’t get us off the hook. Our false positive rate estimate dropped, which compensates somewhat but not enough. We need to revise estimates for rates of infection downward some amount, compared to if we didn’t have this model.

The question is how much? Does the implied antibody test model drop enough that we can no longer consider it compatible with our other intuitions? Is this a bigger effect than the dramatic drops recently in R0 and positive test rates, which imply more herd immunity – or at least something making New York different from other areas more than we otherwise would expect?

We should have seen 10%+ growth in test results between the first and third samples. Instead, we saw about a 12% decline. That’s a huge difference of 20%+, although nothing like as huge as the difference between these tests and someone who naively trusts positive test counts with only small modifications. That would triple the distance between the antibody test result, and my other estimates. That could be discounted against the decline in our estimate of false positives, from 2.1% to 0.8%, which can potentially adjust our final result half as much in the other direction.

By contrast, the second sample showed about as much increase as you would have expected sample over sample, which makes the third one even stranger.

My gut answer is that the various biases going on are still potentially more than big enough to account for potential differences, so I notice myself not inclined to update much from this one additional set of data until we can get more details and figure out why it makes sense, especially since the data now looks somewhat more likely to be more biased. That’s likely partially bad reasoning, and I’m probably under-adjusting for the new data, but I think the effect should still be small.

Next Questions

What are the best next steps? This could be asked from the perspective of outsiders like me, or from the perspective of Governor Cuomo or others who can cause New York to collect or release/tabulate better data.

From the outsider perspective, the next questions become when and where additional data will become available, and whether we can find more ways to extract information from the data we do have.

If we do get our hands on more detailed versions of the data, then suddenly we can learn a lot more. And that therefore feeds back into the question of what New York should do to help make this and additional future data more useful.

I can think of a few distinct categories.

  1. Disclose More Already Known Details

This is the easy one. Give us a write up with every detail that does not cause a privacy violation. Give us the list of locations and a breakdown of results and demographics, and results for each demographic, at each location. Tell us the exact timing of when samples were collected, and how that changes results. Show us how various demographic subsections combine. As sample sizes increase, smaller subsections of the data become more reliable and useful.

2. Choose More Representative Samples

There’s no reason we can’t use census or phone book style sampling to get a truly random set of people, modulo the refusal rate, other than that it’s more expensive per test because you have to find them rather than setting up a table and waiting. But if we already did over 20,000 tests, why do we want quantity over quality? The law of large numbers is most definitely in play. So let’s stop wondering about all these weird biases the easiest way possible, which is by avoiding them entirely.

3. Choose More Unrepresentitive Samples

Testing samples of essential workers was a very good idea. Now let’s take samples from lots of other places and groups, and see what happens. The more different groups, the better. If we want to find out if the subway is a menace or innocent scapegoat, offer tests at subway stations. If we are worried about take-out and delivery, do the same at restaurants, and offer to test the delivery drivers and also the staff that handles the food. And so on. Combine this with the survey questions below and we’ll really get somewhere.

The stakes almost couldn’t be higher. The price of information is super low. There’s nothing stopping us. And of course, we can also do the obvious next thing:

4. Ask Survey Questions

This is the big one. It didn’t occur to me until two days ago. Once it did, it seemed obvious. We already get people’s contact information, because we give them their results. So we can go back and ask them questions, if they’re willing to answer. Given how much is at stake, we can happily pay them enough to make it worth their while, even if they aren’t willing to help in order to help. My guess is most would answer.

If we had answers, and we could match them with who was positive and who was negative, we’d suddenly have a very much improved understanding of what is going on.

We could correct for ways in which the sample we took was biased, and we could figure out which factors create risk of infection.

We can ask about types of exposures and contacts. We can ask about symptoms and when someone felt sick and what suspicion they had that they’d had Covid-19 (ideally before they know their results), and of course whether they were treated or tested.

We could ask whether someone was taking various precautions. Are they sanitizing their packages? Getting take out? Wearing a mask? Seeing anyone socially at all and to what extent? Doing how much walking outside? Taking how much public transit? When did they start modifying their behaviors? Et cetra, et cetra. Even better, if we get their permission, we can download their cell phone data, and see what places they visited.

Test and trace is great for finding new cases. Tracing after antibody testing is great for figuring out where cases come from. Right now, that is perhaps job one. If we can’t run experiments, because they’re banned (I like the metaphor that it’s a trolley problem where someone actively wants to jump in front of the trolley that’s about to run over thousands of people, they are 99.9% to survive the jump, and we are physically restraining them so they can’t do it, and also We Fail Philosophy Forever, also ethics).

Let’s take this whole thing to the next stage. What’s stopping us?

 

 

 

 

 

 

 

 



Discuss

[AN #98]: Understanding neural net training by seeing which gradients were helpful

6 мая, 2020 - 20:10
Published on May 6, 2020 5:10 PM GMT

[AN #98]: Understanding neural net training by seeing which gradients were helpful Alignment Newsletter is a weekly publication with recent content relevant to AI alignment around the world View this email in your browser Newsletter #98
Alignment Newsletter is a weekly publication with recent content relevant to AI alignment around the world. Find all Alignment Newsletter resources here. In particular, you can look through this spreadsheet of all summaries that have ever been in the newsletter.
Audio version here (may not be up yet). SECTIONS HIGHLIGHTS
TECHNICAL AI ALIGNMENT
        ITERATED AMPLIFICATION
        LEARNING HUMAN INTENT
        VERIFICATION
        FORECASTING
        MISCELLANEOUS (ALIGNMENT)
NEAR-TERM CONCERNS
        FAIRNESS AND BIAS
OTHER PROGRESS IN AI
        REINFORCEMENT LEARNING
NEWS    HIGHLIGHTS

LCA: Loss Change Allocation for Neural Network Training (Janice Lan et al) (summarized by Robert): This paper introduces the Loss Change Allocation (LCA) method. The method's purpose is to gain insight and understanding into the training process of deep neural networks. The method calculates an allocation of the change in overall loss (on the whole training set) between every parameter at each training iteration, which is iteratively refined until the approximation error is less than 1% overall. This loss change allocation can be either positive or negative; if it's negative, then the parameter is said to have helped training at that iteration, and if it's positive then the parameter hurt training. Given this measurement is per-parameter and per-iteration, it can be aggregated to per-layer LCA, or any other summation over parameters and training iterations.

The authors use the method to gain a number of insights into the training process of several small neural networks (trained on MNIST and CIFAR-10).

First, they validate that learning is very noisy, with on average only half of the parameters helping at each iteration. The distribution is heavier-tailed than a normal distribution, and is fairly symmetrical. However, parameters tend to alternate between helping and hurting, and each parameter only tends to help approximately 50% of the time.

Second, they look at the LCA aggregated per-layer, summed over the entire training process, and show that in the CIFAR ResNet model the first and last layers hurt overall (i.e. have positive LCA). In an attempt to remedy this and understand the causes, the authors try freezing these layers, or reducing their learning rate. The first layer can't be fixed (freezing makes it's LCA 0, but later layers' LCA is increased in turn so the overall final loss stays the same). However, for the last layer, freezing or reducing the learning rate increases the overall performance of the network, as the last layer's LCA is decreased more than all the other layer's LCAs are increased. They also hypothesize that by reducing the momentum for the last layer, they can give it fresher information and make it more likely to learn. They find that this does work, though in this setting previous layers’ LCA increases to compensate, leaving overall performance unchanged.

Finally, the authors show that learning seems to be synchronised across layers; layers get local LCA minima at the same training iterations, in a statistically significant way. They show this must be a combination of parameter motion and the gradient, as neither on their own explains this phenomenon.



Robert's opinion: I really liked this paper. The method is simple (although computationally expensive), and gives novel insights. I think understanding how deep learning training works is important as it can help us design better training processes, not just for better performance but for other properties we want the training process to induce. I think there's a lot of future work which could be done with this method, in making it more efficient and then applying it to larger models in domains other than vision. I'd also be interested in seeing if this can be used to understand which parts of the training set help and hurt training; for example seeing whether there's any correlation between the points of synchronised learning and the datapoints in the minibatch at that training iteration. Note: I'd recommend reading the paper (including the appendices) to see the graphs and visualisations the authors produced to demonstrate their arguments, as they're much easier to understand than a textual description.

Rohin's opinion: I also really enjoyed this paper, it has great empirical evidence about how neural networks work. I'd be inclined to analyze the results somewhat differently. In particular, suppose that when calculating LCA, we made the following changes:

1. We used the loss on the training batches instead of the full training set.

2. We didn't improve the approximation error (i.e. we just used the point estimate of the gradient calculated during training).

3. We trained using stochastic gradient descent (SGD) (as opposed to say Adam or Momentum-SGD).

Then all LCA values would be negative (explanation in this comment). So, when the paper shows experiments where LCA values are positive (i.e. the parameters / layers are anti-learning), we can attribute those effects to some combination of these three factors.

Take the observation that learning is very noisy. I would guess that this is primarily because of the first point: there are many many ways to improve the loss on a tiny little minibatch, but only a tiny fraction of those are capturing "real effects" that would improve the loss on the full large training dataset. Likely in the large majority of cases, the update doesn't capture a "real effect", and so it's a coin flip whether or not it will help with the loss on the full training dataset. A large probability of a coin flip + a small probability of a "real effect" gets you to an improvement slightly over half the time. This explanation applies across parameters, iterations, layers, etc.

Similarly, they find that learning is synchronized across layers. I think this is also primarily because of the first point. My guess is that there are some batches of data that are more "canonical" than others, that are easiest to learn from. In the case where we see synchronization for each class, this could be as simple as that particular training batch having more examples of that class than other training batches.

I’d be interested in seeing experiments in which we start with the version of LCA where everything is negative, and made only one of the changes. This would allow us to narrow down which particular change causes a given effect, kind of like an ablation study.

   TECHNICAL AI ALIGNMENT
 ITERATED AMPLIFICATION

How does iterated amplification exceed human abilities? (Issa Rice)

 LEARNING HUMAN INTENT

Shared Autonomy via Hindsight Optimization (Shervin Javdani et al) (summarized by Rohin): This paper considers a shared autonomy task in which a user controls a robot to achieve some goal, and the robot learns to assist the user, without knowing the goal in advance. They formalize this as a POMDP in which the state includes the user's goal, which the robot does not get to observe. However, the POMDP observation model assigns higher probability to user actions that better achieve the goal (a standard Boltzmann rationality model), and this allows the agent to reason about what the goal must be. In practice, for computational tractability, rather than choosing optimal actions in the overall POMDP, the robot chooses optimal actions using a technique called hindsight optimization, which assumes that the robot will never learn more information about the user's goal.



Rohin's opinion: The formulation of a POMDP with uncertainty over the goal is remarkably similar to the formulation of Cooperative Inverse Reinforcement Learning (AN #69) (and predates it), with the main difference being that there is only one actor (the robot hardware).



Imitation Learning via Off-Policy Distribution Matching (Ilya Kostrikov et al) (summarized by Zach): One way to view imitation learning is as a distribution matching problem. In other words, the agent is rewarded based on how well it can imitate the state-distribution induced by the expert. In recent years, distribution matching via adversarial methods such as GAIL has become a popular approach to imitation learning. However, one weakness of these methods is that they require on-policy samples which means they require the agent to interact with the environment. In this paper, the authors present an off-policy method for distribution matching which can work without environment interaction. They do this by building on the prior work of DualDICE, a policy-agnostic method to estimate distribution ratios between agent and expert which can then be used to provide a reward to the agent. This allows the optimal policy to be estimated directly from demonstrations without any need for agent interaction. The authors run a few experiments and show that the method has comparable performance to behavioral cloning in the off-policy setting and adversarial methods in the on-policy setting.

Prerequisities: DualDICE

Read more: GAIL



Zach's opinion: This is a cool application of density-estimation via DualDICE. While the experiments are a bit weak, the fact that an off-policy method exists to do distribution-matching is interesting in its own right. Moreover, the method seems able to compete with both BC and GAIL-like methods which is intriguing.

  VERIFICATION

Ethical Mission Definition and Execution for Maritime Robots Under Human Supervision (Don Brutzman et al) (summarized by Rohin) (H/T Jon Rodriguez): While underwater robots can perform missions that humans cannot, they cannot be held liable for their actions. Our society requires that someone be responsible for (and can be held liable for) the actions of any such robot, leading to a form of the specification problem: how do we program robots such that it is reasonable to hold their operators accountable for their actions?

This paper divides mission execution into three main parts: the execution level (hardware control), the tactical level (low-level behaviors), and the strategic level (what the robot should do). It proposes that, at the strategic level, we use formal methods to specify what the robot should do. The language should be expressive enough to be useful, while still keeping it sufficiently limited to allow exhaustive testing. They propose using state machines augmented with constraints. The constraints can be used to specify things like "the robot must stay at least 10m away from obstacles". The state machine decides which behaviors to execute, and each such behavior can have three results: success, failure, or exception (in the case that a constraint would have been violated had the behavior continued operating).



Rohin's opinion: It's interesting to see other groups also aiming to have what are essentially robustness guarantees, but motivated instead from the perspective of responsibility and liability. The actual method seems reasonable for the impoverished systems we have today, where we must specify everything that we want the system to do.

  FORECASTING

FLI Podcast: On Superforecasting (Lucas Perry and Robert de Neufville)

 MISCELLANEOUS (ALIGNMENT)

Formal Metaethics and Metasemantics for AI Alignment (June Ku) (summarized by Rohin): This website presents in great detail a process by which an agent might use data from human brains in order to infer a utility function for a single human (also spelling out what assumptions need to be made along the way), and then how it could combine the utility functions from different humans to arrive at "a fully technical ethical goal function". Emphasis is placed on solving the philosophical problems of metaethics and mental content. Quoting the website, they "suppose that unlimited computation and a complete low-level causal model of the world and the adult human brains in it are available".

Approaches to Deploying a Safe Artificial Moral Agent (Olivier Couttolenc) (summarized by Rohin): This post investigates which of the current moral theories would most reduce existential risk if we programmed it into an AI system, and settles on Aristotelian virtue ethics (over utilitarianism and Kant's categorical imperative).

   NEAR-TERM CONCERNS
 FAIRNESS AND BIAS

Algorithmic Fairness from a Non-ideal Perspective (Sina Fazelpour et al) (summarized by Rohin): The field of fairness has aimed to develop objective metrics of fairness, which can then be optimized for in order to produce a just AI system. Unfortunately, many intuitively desirable fairness metrics are fundamentally incompatible, and cannot be simultaneously achieved except in special circumstances. Should we lose all hope for fairness?

This paper argues that the problem was that we were building idealized theories, referring to a conception from political philosophy of ideal and non-ideal modes of theorizing. An ideal theory is one that describes an optimal, ideal world, and then identifies injustices by searching for discrepancies between the real world and the idealized one. This leads to three major flaws:

1. It can lead to systematic neglect of some injustices and distortions of our understanding of other injustices. For example, group parity metrics of fairness applied to college admissions would identify east Asian students as privileged relative to white students despite historical and institutional discrimination.

2. It does not offer sufficient practical guidance about what should be done, sometimes leading to misguided mitigation strategies. Consider college admissions again. A disparate learning process aims to be blind to protected characteristics (like gender) while still achieving demographic parity. This forces the model to penalize features that correlate with being male. As a result, we end up rewarding women who go into female-dominated fields, and penalize women who go into male-dominated fields! This was presumably not what we wanted.

3. It does not make clear who among decision-makers is responsible for intervening to correct specific injustices.

The authors suggest that the research community move towards a non-ideal mode of theorizing, in which there is more emphasis on having a deep empirical understanding of the problem (including the various causal factors, rather than summary statistics), and using empirically-informed choices of treatments, rather than modifying ML algorithms to optimize a mathematically defined metric.



Rohin's opinion: I really enjoyed this paper, and my summary doesn't do it justice -- it makes several other good points. I feel similarly about alignment: I feel relatively pessimistic about formal definitions of concepts like goal-directedness (AN #35) or safe exploration (AN #76), and feel much better about schemes that don't assume a formal definition of concepts and instead learn them from humans (or don't require them at all).

Another thing that jumped out at me was that their description of the non-ideal mode of theorizing focuses a lot on understanding what exactly is going on, which is very similar to the concepts of interpretability and universality (AN #81) in alignment.

   OTHER PROGRESS IN AI
 REINFORCEMENT LEARNING

The Ingredients of Real World Robotic Reinforcement Learning (Henry Zhu, Justin Yu, Abhishek Gupta et al) (summarized by Rohin): Suppose we wanted to train a robot to perform a task in the real world, and we didn't want to deal with the headache of sim-to-real transfer. Typically, since all of our experience must be collected in the real world, we would need a human to reset the robot to its initial state. The key idea of this paper is that the point of resets is to ensure that the robot explores a diversity of states causing it to learn a robust policy; this can be achieved by learning a perturbation policy whose objective is to take the robot to states it hasn't visited before. They then combine this with representation learning (so that they can learn from pixels) and use a classifier that distinguishes goal states from non-goal states as the reward function, to get a fully automated setup where once you start the robot's training, it trains itself without any human in the loop.

Read more: Paper: The Ingredients of Real World Robotic Reinforcement Learning



Rohin's opinion: This is a cool proof of concept, but the learned perturbation policy can only take you so far -- no learned perturbation policy is going to allow you to e.g. pick up an object after it is dropped, as you would want if you're training a robot to manipulate a Rubik's cube (AN #70). It seems hard to overcome this sort of problem in a fully automated and learned way (though perhaps you could use more classical techniques to have a "hardcoded" but still automated reset policy).

   NEWS

CLR Open Positions: Researchers and Summer Research Fellows (summarized by Rohin): The Center on Long-Term Risk is looking for researchers and summer research fellows to work on high-quality research relevant to s-risks, including on (among other areas) multiagent systems. The application deadline is May 13.

FEEDBACK I'm always happy to hear feedback; you can send it to me, Rohin Shah, by replying to this email. PODCAST An audio podcast version of the Alignment Newsletter is available. This podcast is an audio version of the newsletter, recorded by Robert Miles.
Subscribe here:

Copyright © 2020 Alignment Newsletter, All rights reserved.

Want to change how you receive these emails?
You can update your preferences or unsubscribe from this list.
 

Discuss

Introduction to forecasting worksheet

6 мая, 2020 - 16:54
Published on May 6, 2020 1:54 PM GMT

I've written this interactive notebook in Foretold prediction platform. It is meant to be completely beginner friendly and takes about 2 hours to go through. I've used it as the basis for a workshop, and the accompanying slides can be found at the bottom of the notebook.

From the notebook:

In this interactive notebook, our goal is to actively try out forecasting and learn several basic tools. After this, you will be able to more easily use forecasts in your daily life and decision making, understand broadly how forecasters go about predicting stuff, and you should know if this is something you want to dive into deeper and how to go about that. We have 5 sections:

  1. We will start immediately with several examples.
  2. Then go on to understand how probabilities feel like, and how to be more calibrated.
  3. Work on the technique of outside view and inside view reasoning.
  4. Briefly discuss several interesting techniques - research, combining models and changing scope.
  5. Try out some actual forecasts from start to finish!


Discuss

AudioWorklet Latency: Firefox vs Chrome

6 мая, 2020 - 16:50
Published on May 6, 2020 1:50 PM GMT

Reading the Firefox 76 Release Notes I noticed that it now supports AudioWorklet. I tested out my whistle-controlled bass synthesizer, and it felt much better than in Chrome, though still not as responsive as the native version. I decided to take some latency measurements.

I connected the computer to an external speaker and microphone, set the browser to run a simple AudioWorklet-based echo, and set my phone to record. I made a sharp tapping sound, and looked at the recording to see the delay from when the original sound is recorded to when the echo plays it back.

I tested Chrome and Firefox. I would have liked to compare to Safari, since Apple has historically taken audio latency very seriously, but they haven't implemented the AudioWorklet spec yet. I also tested Reaper and PortAudio [1] for comparison.

Here's what this looked like with Reaper, Firefox, and Chrome all set to echo:


(mp3)

I made twelve clicks in a row, and saw Reaper at 15ms, Firefox at 62ms, and Chrome at 124ms. I learned which echo was which by turning off each in turn.

You would expect that latency would be completely consistent, because you can't change the length of the delay without getting ugly artifacts, but in Chrome it was not consistent. Earlier I made a much longer recording (mp3) and taking the two sections where you can clearly hear Chrome echos (mp3, mp3) I get 78ms, 78ms, 77ms, 94ms, 94ms, 93ms, 93ms, 93ms and then later 88ms, 88ms, 88ms, 88ms, 88ms, 93ms, 109ms, 109ms, 114ms, 114ms, 120ms, 147ms, 152ms, 152ms, 152ms, 152ms, 152ms, 152ms, 152ms, 152ms, 152ms, 163ms, 153ms, 153ms. Since one of the projects I'd like to do with browser audio requires completely consistent latency, I'm not sure it would work at all in Chrome.

PortAudio compared very similarly to Firefox, about 3ms faster. I'm not sure what Reaper is doing to get much lower latency than PortAudio and Firefox? If I was going to look into this further my next step would be to log which system calls they're making and try to figure out what they're doing differently.

Overall, I'm very impressed with Firefox's AudioWorklet implementation's performance, and I hope Firefox pushes Chrome to do better as well.

(Disclosure: I work for Google, which makes Chrome. I don't work on the browser, though, and I'm speaking only for myself.)


[1] I used paex_read_write_wire.c with FRAMES_PER_BUFFER set to 64 and suggestedLatency set to defaultLowInputLatency and defaultLowOutputLatency. I installed portaudio (brew install portaudio) and then ran gcc paex_read_write_wire.c -o paex_read_write_wire -lportaudio && ./paex_read_write_wire.



Discuss

Assessing Kurzweil predictions about 2019: the results

6 мая, 2020 - 16:36
Published on May 6, 2020 1:36 PM GMT

.mjx-chtml {display: inline-block; line-height: 0; text-indent: 0; text-align: left; text-transform: none; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; word-wrap: normal; word-spacing: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0; min-height: 0; border: 0; margin: 0; padding: 1px 0} .MJXc-display {display: block; text-align: center; margin: 1em 0; padding: 0} .mjx-chtml[tabindex]:focus, body :focus .mjx-chtml[tabindex] {display: inline-table} .mjx-full-width {text-align: center; display: table-cell!important; width: 10000em} .mjx-math {display: inline-block; border-collapse: separate; border-spacing: 0} .mjx-math * {display: inline-block; -webkit-box-sizing: content-box!important; -moz-box-sizing: content-box!important; box-sizing: content-box!important; text-align: left} .mjx-numerator {display: block; text-align: center} .mjx-denominator {display: block; text-align: center} .MJXc-stacked {height: 0; position: relative} .MJXc-stacked > * {position: absolute} .MJXc-bevelled > * {display: inline-block} .mjx-stack {display: inline-block} .mjx-op {display: block} .mjx-under {display: table-cell} .mjx-over {display: block} .mjx-over > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-under > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-stack > .mjx-sup {display: block} .mjx-stack > .mjx-sub {display: block} .mjx-prestack > .mjx-presup {display: block} .mjx-prestack > .mjx-presub {display: block} .mjx-delim-h > .mjx-char {display: inline-block} .mjx-surd {vertical-align: top} .mjx-mphantom * {visibility: hidden} .mjx-merror {background-color: #FFFF88; color: #CC0000; border: 1px solid #CC0000; padding: 2px 3px; font-style: normal; font-size: 90%} .mjx-annotation-xml {line-height: normal} .mjx-menclose > svg {fill: none; stroke: currentColor} .mjx-mtr {display: table-row} .mjx-mlabeledtr {display: table-row} .mjx-mtd {display: table-cell; text-align: center} .mjx-label {display: table-row} .mjx-box {display: inline-block} .mjx-block {display: block} .mjx-span {display: inline} .mjx-char {display: block; white-space: pre} .mjx-itable {display: inline-table; width: auto} .mjx-row {display: table-row} .mjx-cell {display: table-cell} .mjx-table {display: table; width: 100%} .mjx-line {display: block; height: 0} .mjx-strut {width: 0; padding-top: 1em} .mjx-vsize {width: 0} .MJXc-space1 {margin-left: .167em} .MJXc-space2 {margin-left: .222em} .MJXc-space3 {margin-left: .278em} .mjx-test.mjx-test-display {display: table!important} .mjx-test.mjx-test-inline {display: inline!important; margin-right: -1px} .mjx-test.mjx-test-default {display: block!important; clear: both} .mjx-ex-box {display: inline-block!important; position: absolute; overflow: hidden; min-height: 0; max-height: none; padding: 0; border: 0; margin: 0; width: 1px; height: 60ex} .mjx-test-inline .mjx-left-box {display: inline-block; width: 0; float: left} .mjx-test-inline .mjx-right-box {display: inline-block; width: 0; float: right} .mjx-test-display .mjx-right-box {display: table-cell!important; width: 10000em!important; min-width: 0; max-width: none; padding: 0; border: 0; margin: 0} .MJXc-TeX-unknown-R {font-family: monospace; font-style: normal; font-weight: normal} .MJXc-TeX-unknown-I {font-family: monospace; font-style: italic; font-weight: normal} .MJXc-TeX-unknown-B {font-family: monospace; font-style: normal; font-weight: bold} .MJXc-TeX-unknown-BI {font-family: monospace; font-style: italic; font-weight: bold} .MJXc-TeX-ams-R {font-family: MJXc-TeX-ams-R,MJXc-TeX-ams-Rw} .MJXc-TeX-cal-B {font-family: MJXc-TeX-cal-B,MJXc-TeX-cal-Bx,MJXc-TeX-cal-Bw} .MJXc-TeX-frak-R {font-family: MJXc-TeX-frak-R,MJXc-TeX-frak-Rw} .MJXc-TeX-frak-B {font-family: MJXc-TeX-frak-B,MJXc-TeX-frak-Bx,MJXc-TeX-frak-Bw} .MJXc-TeX-math-BI {font-family: MJXc-TeX-math-BI,MJXc-TeX-math-BIx,MJXc-TeX-math-BIw} .MJXc-TeX-sans-R {font-family: MJXc-TeX-sans-R,MJXc-TeX-sans-Rw} .MJXc-TeX-sans-B {font-family: MJXc-TeX-sans-B,MJXc-TeX-sans-Bx,MJXc-TeX-sans-Bw} .MJXc-TeX-sans-I {font-family: MJXc-TeX-sans-I,MJXc-TeX-sans-Ix,MJXc-TeX-sans-Iw} .MJXc-TeX-script-R {font-family: MJXc-TeX-script-R,MJXc-TeX-script-Rw} .MJXc-TeX-type-R {font-family: MJXc-TeX-type-R,MJXc-TeX-type-Rw} .MJXc-TeX-cal-R {font-family: MJXc-TeX-cal-R,MJXc-TeX-cal-Rw} .MJXc-TeX-main-B {font-family: MJXc-TeX-main-B,MJXc-TeX-main-Bx,MJXc-TeX-main-Bw} .MJXc-TeX-main-I {font-family: MJXc-TeX-main-I,MJXc-TeX-main-Ix,MJXc-TeX-main-Iw} .MJXc-TeX-main-R {font-family: MJXc-TeX-main-R,MJXc-TeX-main-Rw} .MJXc-TeX-math-I {font-family: MJXc-TeX-math-I,MJXc-TeX-math-Ix,MJXc-TeX-math-Iw} .MJXc-TeX-size1-R {font-family: MJXc-TeX-size1-R,MJXc-TeX-size1-Rw} .MJXc-TeX-size2-R {font-family: MJXc-TeX-size2-R,MJXc-TeX-size2-Rw} .MJXc-TeX-size3-R {font-family: MJXc-TeX-size3-R,MJXc-TeX-size3-Rw} .MJXc-TeX-size4-R {font-family: MJXc-TeX-size4-R,MJXc-TeX-size4-Rw} .MJXc-TeX-vec-R {font-family: MJXc-TeX-vec-R,MJXc-TeX-vec-Rw} .MJXc-TeX-vec-B {font-family: MJXc-TeX-vec-B,MJXc-TeX-vec-Bx,MJXc-TeX-vec-Bw} @font-face {font-family: MJXc-TeX-ams-R; src: local('MathJax_AMS'), local('MathJax_AMS-Regular')} @font-face {font-family: MJXc-TeX-ams-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_AMS-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_AMS-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_AMS-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-B; src: local('MathJax_Caligraphic Bold'), local('MathJax_Caligraphic-Bold')} @font-face {font-family: MJXc-TeX-cal-Bx; src: local('MathJax_Caligraphic'); font-weight: bold} @font-face {font-family: MJXc-TeX-cal-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-R; src: local('MathJax_Fraktur'), local('MathJax_Fraktur-Regular')} @font-face {font-family: MJXc-TeX-frak-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-B; src: local('MathJax_Fraktur Bold'), local('MathJax_Fraktur-Bold')} @font-face {font-family: MJXc-TeX-frak-Bx; src: local('MathJax_Fraktur'); font-weight: bold} @font-face {font-family: MJXc-TeX-frak-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-BI; src: local('MathJax_Math BoldItalic'), local('MathJax_Math-BoldItalic')} @font-face {font-family: MJXc-TeX-math-BIx; src: local('MathJax_Math'); font-weight: bold; font-style: italic} @font-face {font-family: MJXc-TeX-math-BIw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-BoldItalic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-BoldItalic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-BoldItalic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-R; src: local('MathJax_SansSerif'), local('MathJax_SansSerif-Regular')} @font-face {font-family: MJXc-TeX-sans-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-B; src: local('MathJax_SansSerif Bold'), local('MathJax_SansSerif-Bold')} @font-face {font-family: MJXc-TeX-sans-Bx; src: local('MathJax_SansSerif'); font-weight: bold} @font-face {font-family: MJXc-TeX-sans-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-I; src: local('MathJax_SansSerif Italic'), local('MathJax_SansSerif-Italic')} @font-face {font-family: MJXc-TeX-sans-Ix; src: local('MathJax_SansSerif'); font-style: italic} @font-face {font-family: MJXc-TeX-sans-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-script-R; src: local('MathJax_Script'), local('MathJax_Script-Regular')} @font-face {font-family: MJXc-TeX-script-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Script-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Script-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Script-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-type-R; src: local('MathJax_Typewriter'), local('MathJax_Typewriter-Regular')} @font-face {font-family: MJXc-TeX-type-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Typewriter-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Typewriter-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Typewriter-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-R; src: local('MathJax_Caligraphic'), local('MathJax_Caligraphic-Regular')} @font-face {font-family: MJXc-TeX-cal-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-B; src: local('MathJax_Main Bold'), local('MathJax_Main-Bold')} @font-face {font-family: MJXc-TeX-main-Bx; src: local('MathJax_Main'); font-weight: bold} @font-face {font-family: MJXc-TeX-main-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-I; src: local('MathJax_Main Italic'), local('MathJax_Main-Italic')} @font-face {font-family: MJXc-TeX-main-Ix; src: local('MathJax_Main'); font-style: italic} @font-face {font-family: MJXc-TeX-main-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-R; src: local('MathJax_Main'), local('MathJax_Main-Regular')} @font-face {font-family: MJXc-TeX-main-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-I; src: local('MathJax_Math Italic'), local('MathJax_Math-Italic')} @font-face {font-family: MJXc-TeX-math-Ix; src: local('MathJax_Math'); font-style: italic} @font-face {font-family: MJXc-TeX-math-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size1-R; src: local('MathJax_Size1'), local('MathJax_Size1-Regular')} @font-face {font-family: MJXc-TeX-size1-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size1-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size1-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size1-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size2-R; src: local('MathJax_Size2'), local('MathJax_Size2-Regular')} @font-face {font-family: MJXc-TeX-size2-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size2-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size2-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size2-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size3-R; src: local('MathJax_Size3'), local('MathJax_Size3-Regular')} @font-face {font-family: MJXc-TeX-size3-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size3-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size3-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size3-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size4-R; src: local('MathJax_Size4'), local('MathJax_Size4-Regular')} @font-face {font-family: MJXc-TeX-size4-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size4-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size4-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size4-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-R; src: local('MathJax_Vector'), local('MathJax_Vector-Regular')} @font-face {font-family: MJXc-TeX-vec-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-B; src: local('MathJax_Vector Bold'), local('MathJax_Vector-Bold')} @font-face {font-family: MJXc-TeX-vec-Bx; src: local('MathJax_Vector'); font-weight: bold} @font-face {font-family: MJXc-TeX-vec-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Bold.otf') format('opentype')}

Thanks to all my brave assessors, I now have the data about Kurzweil's 1999 predictions about 2019.

This was a follow up to a previous assessment about his predictions about 2009, which showed a mixed bag. Roughly evenly divided between right and wrong, which I found pretty good for ten-year predictions:

So, did more time allow for trends to overcome noise or more ways to go wrong? Pause for a moment to calibrate your expectations.

Methods and thanks

So, for the 2019 predictions, I divided them into 105 separate statements, did a call for volunteers, with instructions here; the main relevant point being that I wanted their assessment for 2019, not for the (possibly transient) current situation. I got 46 volunteers with valid email addresses, of which 34 returned their predictions. So many thanks, in reverse alphabetical order, to Zvi Mowshowitz, Zhengdong Wang, Yann Riviere, Uriel Fiori, orthonormal, Nuño Sempere, Nathan Armishaw, Koen Holtman, Keller Scholl, Jaime Sevilla, Gareth McCaughan, Eli Rose and Dillon Plunkett, Daniel Kokotajlo, Anna Gardiner... and others who have chosen to remain anonymous.

The results

Enough background; what did the assessors find? Well, of the 34 assessors, 24 went the whole hog and did all 105 predictions; on average, 91 predictions were assessed by each person, a total of 3078 individual assessments[1].

So, did more time allow for more perspective or more ways to go wrong? Well, Kurzweil's predictions for 2019 were considerably worse than those for 2009, with more than half strongly wrong:

Interesting details

The (anonymised) data can be found here[2], and I encourage people to download and assess it themselves. But some interesting results stood out to me:

Predictor agreement

Taking a single prediction, for instance the first one:

  • 1: Computers are now largely invisible. They are embedded everywhere--in walls, tables, chairs, desks, clothing, jewelry, and bodies.

Then we can compute the standard deviation of the predictors' answer for that prediction. This gives an impression of how much disagreement there was between predictors; in this case, it was 0.84.

Perfect agreement would be a standard deviation of 0; maximum disagreement (half find "1", half find "5") would be a standard deviation of 2. Perfect spread - equal numbers of 1s, 2s, 3s, 4s, and 5s - would have a standard deviation of $\sqrt[2] \approx$1.4.

Across the 105 predictions, the maximum standard deviation was 1.7, the minimum was 0 (perfect agreement), and the average was 0.97. So the predictors had a medium tendency to agree with each other.

Most agreement/falsest predictions

There was perfect agreement on five predictions; and on all of these, the agreed prediction was always "5": "False".

These predictions were:

  • 51: "Phone" calls routinely include high-resolution three-dimensional images projected through the direct-eye displays and auditory lenses.
  • 55: [...] Thus a person can be fooled as to whether or not another person is physically present or is being projected through electronic communication.
  • 59: The all-enveloping tactile environment is now widely available and fully convincing.
  • 62: [...] These technologies are popular for medical examinations, as well as sensual and sexual interactions with other human partners or simulated partners.
  • 63: [...] In fact, it is often the preferred mode of interaction, even when a human partner is nearby, due to its ability to enhance both experience and safety.

As you can see, Kurzweil suffered a lot from his VR predictions. This seems a perennial thing: Hollywood is always convinced that mass 3D is just around the corner; technologists are convinced that VR is imminent.

Most accurate prediction:

With a mean score of 1.3, the prediction deemed most accurate was:

  • 83: The existence of the human underclass continues as an issue.

Now this might seem a trivial prediction, especially in retrospect, but I want to defend Kurzweil here - it was not at all certain in 1999, with many utopian changes foreseen and expected, that this would still be an issue.

The next prediction deemed most accurate (mean of 1.4), is:

  • 82: People attempt to protect their privacy with near-unbreakable encryption technologies, but privacy continues to be a major political and social issue with each individual's practically every move stored in a database somewhere.

This is truly non-trivial for 1999, and I do give Kurzweil credit for that.

Least agreement

With a standard deviation of 1.7, the predictors disagreed the most on this prediction:

  • 37: Computation in general is everywhere, so a student's not having a computer is rarely an issue.

This may have to do with different judgement over the extent of "everywhere" and "rarely an issue", or over who might or might not find this to be an issue.

Most "Cannot Decide"

This prediction had more than 46% of predictors choosing "Cannot Decide":

  • 20: "It is now fully recognized that the brain comprises many specialized regions, each with its own topology and architecture of interneuronal connections."

Maybe the ambiguity in "fully recognized" made this hard to assess.

A question of timeline?

It's been suggested that Kurzweil's predictions for 2009 are mostly correct in 2019. If this is the case - Kurzweil gets the facts right, but the timeline wrong - it would be interesting to revisit these predictions in 2029.

In conclusion

Again, thanks to all the volunteers who assessed these predictions and thanks to Kurzweil who, unlike most prognosticators, had the guts and the courtesy to write down his predictions and give them a date.

I strongly suspect that most people's 1999 predictions about 2019 would have been a lot worse.

  1. Five assessments of the 3078 returned question marks; I replaced these with "3" ("Cannot Decide"). Four assessors of the 34 left gaps in their predictions, instead of working through the randomly ordered predictions; to two significant figures, excluding these four didn't change anything, so I included them all. ↩︎

  2. Each column is an individual predictor, each row an individual prediction. ↩︎



Discuss

Maths writer/cowritter needed: how you can't distinguish early exponential from early sigmoid

6 мая, 2020 - 12:41
Published on May 6, 2020 9:41 AM GMT

It's well known in FHI and similar circles, that it's impossible to distinguish an exponential (growth going up wildly) from a sigmoid/logistic curve (exponential growth until a turning point - an S shape) - until well after the turning point.

Which means we can't effectively predict that turning point. And so can't distinguish when a sigmoid will have a turning point, even when we know it must have one.

But this doesn't seem to exist in the statistics literature; and it would be very useful to have such a paper or textbook to point to.

We don't have time to write a full paper ourselves, but is there someone on this list with statistical experience who would like to write or co-write such a paper?

Since this result is important and as yet unpublished, it's plausible that such a publication may get an extremely high number of citations.

Cheers!



Discuss

How can nonprofits gain the advantages of the for-profit model?

6 мая, 2020 - 04:15
Published on May 6, 2020 1:15 AM GMT

In my last post I described the advantages of for-profit models over nonprofit models, including scalable revenue, incentives and metrics to drive effectiveness and efficiency, and incentives to fund high-risk, high-reward experiments.

But not everything can be for-profit. How can nonprofit organizations get some of these advantages? Here are five ideas:

  • Nonprofits that generate revenue primarily through products & services, rather than through charitable donations, gain some of the advantages of for-profits: they survive only to the extent that they can deliver a product to the market that people are willing to buy, out-compete alternatives, and keep their costs below their prices. To the extent that a nonprofit’s paid services are subsidized by donations (as is the case with universities, museums, and opera houses, among others), this requirement is weakened but not destroyed.
  • In the case of charity, I wonder if the most effective form of it is simply giving money directly to beneficiaries, with the goods and services themselves provided by for-profit businesses. This would seem to let free-market capitalism work to the maximum extent. There is some research to support this idea.
  • If enough people promote the idea of donating based primarily on demonstrated impact, the world might slowly shift towards more strategic nonprofits driven by output metrics and other clear indication of delivered benefits. For instance, when reporting on a contribution from a major donor, the news media could focus more on the impact or potential impact of the contribution, rather than the amount of money given or the percent of their wealth that represents.
  • To drive innovation, perhaps we should be putting more of our resources into prizes or mechanisms like advance market commitments, rather than grants. Tyler Cowen summarizes: “The case for prizes is stronger when you don’t know who is likely to make the breakthrough, you value the final output more than the process, there is an urgency to solutions (talent development is too slow), success is relatively easy to define, and efforts and investments are likely to be undercompensated.” It seems to me that most of those conditions apply to a lot of breakthrough scientific and technological R&D. Indeed, one of the earliest and most famous prizes, the Longitude Prize, had exactly the effect of uncovering an unexpected solution from an unlikely innovator: while most of the scientific community was looking for astronomy-based methods, John Harrison addressed the problem with a highly robust and accurate clock—and he wasn’t even trained as a clockmaker. Why don’t we have more prizes for grand challenge problems today?
  • Beyond this, I think we need more mechanisms to give credit for being right early, for being the first backer of a risky experiment that has transformative effects. Who were the donors who gave small amounts of money to Howard Florey’s lab around 1940 when they were inventing penicillin? The world should know their names. A special award or Hall of Fame could be created for these bold bets (perhaps with a sophisticated scorekeeping mechanism).


Discuss

Modeling naturalized decision problems in linear logic

6 мая, 2020 - 03:15
Published on May 6, 2020 12:15 AM GMT

The following is a model of a simple decision problem (namely, the 5 and 10 problem) in linear logic. Basic familiarity with linear logic is assumed (enough to know what it means to say linear logic is a resource logic), although knowing all the operators isn't necessary.

The 5 and 10 problem is, simply, a choice between taking a 5 dollar bill and a 10 dollar bill, with the 10 dollar bill valued more highly.

While the problem itself is trivial, the main theoretical issue is in modeling counterfactuals. If you took the 10 dollar bill, what would have happened if you had taken the 5 dollar bill? If your source code is fixed, then there isn't a logically coherent possible world where you took the 5 dollar bill.

I became interested in using linear logic to model decision problems due to noticing a structural similarity between linear logic and the real world, namely irreversibility. A vending machine may, in linear logic, be represented as a proposition "$1 → CandyBar", encoding the fact that $1 may be exchanged for a candy bar, being consumed in the process. Since the $1 is consumed, the operation is irreversible. Additionally, there may be multiple options offered, e.g. "$1 → Gumball", such that only one option may be taken. (Note that I am using "→" as notation for linear implication.)

This is a good fit for real-world decision problems, where e.g. taking the $10 bill precludes also taking the $5 bill. Modeling decision problems using linear logic may, then, yield insights regarding the sense in which counterfactuals do or don't exist.

First try: just the decision problem

As a first try, let's simply try to translate the logic of the 5 and 10 situation into linear logic. We assume logical atoms named "Start", "End", "$5", and "$10". Respectively, these represent: the state of being at the start of the problem, the state of being at the end of the problem, having $5, and having $10.

To represent that we have the option of taking either bill, we assume the following implications:

TakeFive : Start → End ⊗ $5

TakeTen : Start → End ⊗ $10

The "⊗" operator can be read as "and" in the sense of "I have a book and some cheese on the table"; it combines multiple resources into a single linear proposition.

So, the above implications state that it is possible, starting from the start state, to end up in the end state, yielding $5 if you took the five dollar bill, and $10 if you took the 10 dollar bill.

The agent's goal is to prove "Start → End ⊗ $X", for X as high as possible. Clearly, "TakeTen" is a solution for X = 10. Assuming the logic is consistent, no better proof is possible. By the Curry-Howard isomorphism, the proof represents a computational strategy for acting in the world, namely, taking the $10 bill.

Second try: source code determining action

The above analysis is utterly trivial. What makes the 5 and 10 problem nontrivial is naturalizing it, to the point where the agent is a causal entity similar to the environment. One way to model the agent being a causal entity is to assume that it has source code.

Let "M" be a Turing machine specification. Let "Ret(M, x)" represent the proposition that M returns x. Note that, if M never halts, then Ret(M, x) is not true for any x.

How do we model the fact that the agent's action is produced by a computer program? What we would like to be able to assume is that the agent's action is equal to the output of some machine M. To do this, we need to augment the TakeFive/TakeTen actions to yield additional data:

TakeFive : Start → End ⊗ $5 ⊗ ITookFive

TakeTen : Start → End ⊗ $10 ⊗ ITookTen

The ITookFive / ITookTen propositions are a kind of token assuring that the agent ("I") took five or ten. (Both of these are interpreted as classical propositions, so they may be duplicated or deleted freely).

How do we relate these propositions to the source code, M? We will say that M must agree with whatever action the agent took:

MachineFive : ITookFive → Ret(M, "Five")

MachineTen : ITookTen → Ret(M, "Ten")

These operations yield, from the fact that "I" have taken five or ten, that the source code "M" eventually returns a string identical with this action. Thus, these encode the assumption that "my source code is M", in the sense that my action always agrees with M's.

Operationally speaking, after the agent has taken 5 or 10, the agent can be assured of the mathematical fact that M returns the same action. (This is relevant in more complex decision problems, such as twin prisoner's dilemma, where the agent's utility depends on mathematical facts about what values different machines return)

Importantly, the agent can't use MachineFive/MachineTen to know what action M takes before actually taking the action. Otherwise, the agent could take the opposite of the action they know they will take, causing a logical inconsistency. The above construction would not work if the machine were only run for a finite number of steps before being forced to return an answer; that would lead to the agent being able to know what action it will take, by running M for that finite number of steps.

This model naturally handles cases where M never halts; if the agent never executes either TakeFive or TakeTen, then it can never execute either MachineFive or MachineTen, and so cannot be assured of Ret(M, x) for any x; indeed, if the agent never takes any action, then Ret(M, x) isn't true for any x, as that would imply that the agent eventually takes action x.

Interpreting the counterfactuals

At this point, it's worth discussing the sense in which counterfactuals do or do not exist. Let's first discuss the simpler case, where there is no assumption about source code.

First, from the perspective of the logic itself, only one of TakeFive or TakeTen may be evaluated. There cannot be both a fact of the matter about what happens if the agent takes five, and a fact of the matter about what happens if the agent takes ten. This is because even defining both facts at once requires re-using the Start proposition.

So, from the perspective of the logic, there aren't counterfactuals; only one operation is actually run, and what "would have happened" if the other operation were run is undefinable.

On the other hand, there is an important sense in which the proof system contains counterfactuals. In constructing a linear logic proof, different choices may be made. Given "Start" as an assumption, I may prove "End ⊗ $5" by executing TakeFive, or "End ⊗ $10" by executing TakeTen, but not both.

Proof systems are, in general, systems of rules for constructing proofs, which leave quite a lot of freedom in which proofs are constructed. By the Curry-Howard isomorphism, the freedom in how the proofs are constructed corresponds to freedom in how the agent behaves in the real world; using TakeFive in a proof has the effect, if executed, of actually (irreversibly) taking the $5 bill.

So, we can say, by reasoning about the proof system, that if TakeFive is run, then $5 will be yielded, and if TakeTen is run, then $10 will be yielded, and only one of these may be run.

The logic itself says there can't be a fact of the matter about both what happens if 5 is taken and if 10 is taken. On the other hand, the proof system says that both proofs that get $5 by taking 5, and proofs that get $10 by taking 10, are possible.

How to interpret this difference? One way is by asserting that the logic is about the territory, while the proof system is about the map; so, counterfactuals are represented in the map, even though the map itself asserts that there is only a singular territory.

And, importantly, the map doesn't represent the entire territory; it's a proof system for reasoning about the territory, not the territory itself. The map may, thus, be "looser" than the territory, allowing more possibilities than could possibly be actually realized.

What prevents the map from drawing out logical implications to the point where it becomes clear that only one action may possibly be taken? Given the second-try setup, the agent simply cannot use the fact of their source code being M, until actually taking the action; thus, no amount of drawing implications can conclude anything about the relationship between M and the agent's action. In addition to this, reasoning about M itself becomes harder the longer M runs, i.e. the longer the agent is waiting to make the decision; so, simply reasoning about the map, without taking actions, need not conclude anything about which action will be taken, leaving both possibilities live until one is selected.

Conclusion

This approach aligns significantly with the less-formal descriptions given of subjective implication decision theory and counterfactual nonrealism. Counterfactuals aren't real in the sense that they are definable after having taken the relevant action; rather, an agent in a state of uncertainty about which action it will take may consider multiple possibilities as freely selectable, even if they are assured that their selection will be equal to the output of some computer program.

The linear logic formalization increases my confidence in this approach, by providing a very precise notion of the sense in which the counterfactuals do and don't exist, which would be hard to make precise without similar formalism.

I am, at this point, less worried about the problems with counterfactual nonrealism (such as global accounting) than I was when I wrote the post, and more worried about the problems of policy-dependent source code (which requires the environment to be an ensemble of deterministic universes, rather than a single one), such that I have updated towards counterfactual nonrealism as a result of this analysis, although I am still not confident.

Overall, I find linear logic quite promising for modeling embedded decision problems from the perspective of an embedded agent, as it builds critical facts such as non-reversibility into the logic itself.

Appendix: spurious counterfactuals

The following describes the problem of spurious counterfactuals in relation to the model.

Assume the second-try setup. Suppose the agent becomes assured that Ret(M, "Five"); that is, that M returns the action "Five". From this, it is provable that the agent may, given Start, attain the linear logic proposition 0, by taking action "Ten" and then running MachineTen to get Ret(M, "Ten"), which yields inconsistency with Ret(M, "Five"). From 0, anything follows, e.g. $1000000, by the principle of explosion.

If the agent is maximizing guaranteed utility, then they will take the $10 bill, to be assured of the highest utility possible. So, it cannot be the case that the agent can be correctly assured that they will take action five, as that would lead to them taking a different action.

If, on the other hand, the agent would have provably taken the $5 bill upon receiving the assurance (say, because they notice that taking the $10 bill could result in the worst possible utility), then there is a potential issue with this assurance being a self-fulfilling prophecy. But, if the agent is constructing proofs (plans for action) so as to maximize guaranteed utility, this will not occur.

This solution is essentially the same as the one given in the paper on UDT with a known search order.



Discuss

A non-mystical explanation of insight meditation and the three characteristics of existence: introduction and preamble

5 мая, 2020 - 22:09
Published on May 5, 2020 7:09 PM GMT

Introduction

Insight meditation, enlightenment, what’s that all about?

The sequence of posts starting from this one is my personal attempt at answering that question. It grew out of me being annoyed about so much of this material seeming to be straightforwardly explainable in non-mysterious terms, but me also being unable to find any book or article that would do this to my satisfaction. In particular, I wanted something that would:

  • Explain what kinds of implicit assumptions build up our default understanding of reality and how those assumptions are subtly flawed. It would then point out aspects from our experience whose repeated observation will update those assumptions, and explain how this may cause psychological change in someone who meditates.
  • It would also explain how the so-called “three characteristics of existence” of Buddhism - impermanence, no-self and unsatisfactoriness - are all interrelated and connected with each other in a way your average Western science-minded, allergic-to-mysticism reader can understand.

I failed to find a resource that would do this in the way I had in mind, so then I wrote one myself.

From the onset, I want to note that I am calling this a non-mystical take on the three characteristics, rather than the non-mystical take on the three characteristics. This is an attempt to explain what I personally think is going on, and to sketch out an explanation of how various experiences and Buddhist teachings could be understandable in straightforward terms. I don’t expect this to be anything like a complete or perfect explanation, but rather one particular model that might be useful.

The main intent of this series is summarized by a comment written by Vanessa Kosoy, justifiably skeptical of grandiose claims about enlightenment that are made without further elaboration on the actual mechanisms of it:

I think that the only coherent way to convince us that Enlightenment is real is to provide a model from a 3rd party perspective. [...] The model doesn't have to be fully mathematically rigorous: as always, it can be a little fuzzy and informal. However, it must be precise enough in order to (i) correctly capture the essentials and (ii) be interpretable more or less unambiguously by the sufficiently educated reader.Now, having such a model doesn't mean you can actually reproduce Enlightenment itself. [...] However, producing such a model would give us the enormous advantages of (i) being able to come up with experimental tests for the model (ii) understanding what sort of advantages we would gain by reaching Enlightenment (iii) being sure that your are talking about something that is at least a coherent possible world even if we are still unsure whether you are describing the actual world.

I hope to at least put together a starting point for a model that would fulfill those criteria.

Note that these articles are not saying “you should meditate”. Getting deep in meditation requires a huge investment of time and effort, and is associated with its own risks [1 2 3 4]. My intent is merely to discuss some of the mechanisms involved in meditation and the mind. Whether one should get direct acquaintance with them is a separate question that goes beyond the scope of this discussion.

Briefly on the mechanisms of meditation

In a previous article, A Mechanistic Model of Meditation, I argued that it is possible in principle for meditation to give people an improved understanding of the way their mind operates.

To briefly recap my argument: we know it is possible for people to train their senses, such as learning to notice more details or make more fine-grained sensory discriminations. One theory is that those details have always been processed in the brain, but the information has not made it to the higher stages of the processing hierarchy. As you repeatedly focus your attention to a particular kind of pattern in your consciousness, neurons re-orient to strengthen that pattern and build connections to the lower-level circuits from which it emerges. This re-encodes the information in those circuits in a format which can be represented in consciousness.

This means at least some kinds of sensory training are training in introspection - learning to better access information which already exists in your brain. This implies you can also learn to strengthen other patterns in your consciousness, especially if you have some source of feedback that you can use to guide the training.

I gave an example of experiential forms of therapy doing exactly this, and then described how a particular style of meditation used one’s awareness of the breath as an objective feedback signal for developing increased “introspective awareness” of one’s own mind.

That post was mostly describing the ways in which meditation can be used to become more aware of the content of your thoughts. However, in observing the content, it is hard to avoid noticing at least some of the structure of the thought process as well.

For example, you might try to follow your breath and think you are doing a good job. In this case, there are at least two kinds of content in your mind: the actual sensory experience of the breath, and thoughts about how badly or well you are doing. The latter might take the form of e.g. mental dialogue that says things like "I'm still managing to follow my breath". Now, since you may find it rewarding to just think that you are meditating well, that thought may start to become rewarded, and you may find yourself repeatedly thinking that you are successfully following the breath... even as the thought of “I am meditating well” has become self-sustaining and no longer connected to whether you are following the breath or not.

Eventually you will realize that you have actually been thinking about following the breath rather than actually following it. This is a minor insight into the way that your thought processes are structured, revealing it is possible for sensations and thoughts about sensations to become mixed up.

It is also possible to practice meditation in a way which explicitly focuses on investigating structure. We can make an analogy to looking at a painting. (Thanks to Alexei Andreev for suggesting this analogy.) Seen from some distance, a painting has "content": it depicts things like people, buildings, boats and so forth. But when you get closer to it and look carefully, you can see that all the content is composed of things like brush strokes, individual colored shapes, paint of varying thickness, and so on. This is "structure". While all types of meditation are going to reveal something about structure, there are also types of meditation which are specifically aimed at exploring it. Meditation which focuses on investigating structure is commonly called insight meditation.

Investigating the mind vs. investigating reality

Now, it is worth noting that these practices are not always framed in terms of "investigating the structure of the mind", nor does the actual experience of doing them necessarily feel like that. Rather, the framing and experience is commonly that of investigating the nature of reality.

For example, in an earlier article trying to explain insight meditation, I mentioned I had once had the thought that I could never be happy. When I paid closer attention to why I thought that, I noticed that my mental image of a happy person included strong extraversion, which conflicted with the self-image that I had of myself as an introvert. After I noticed the happiness-extraversion connection, it became apparent that I could be happy even as an introvert, and the original thought disappeared. (Although I didn't know it at the time, it is common for emotional beliefs to change when they become explicit enough for the brain to notice them being erroneous.)

Essentially, I had originally believed “I can never be happy”, and this belief about me didn’t feel like a “belief”. It felt like a basic truth of what I was, the kind of truth that you just know - in the same way that you might look at an apple and just know you are having the experience of seeing an apple. But when I investigated the details of that experience, I realized that this wasn’t actually a fact about me. Rather it was just a belief that I had.

In a similar way, there are many aspects of our subjective experience that feel like just facts about reality, but upon doing insight practices and investigating them closer, we can come to see that they are not so.

The philosopher Daniel Dennett has coined the term "heterophenomenology" to refer to a particular approach to the study of consciousness. In this approach, we assume that people are correctly describing how things seem to them and treat this as something that needs to be explained. However, the actual mechanism of why things seem like that to them, may be different from what they assume.

If I see an apple, it typically feels to me like I am seeing reality as it is. From a scientific point of view, this is mistaken: the sight of an apple is actually a complex interpretation my brain has created. Likewise, if I have the experience that I can never be happy, then this also feels like a raw fact while actually being an interpretation. In either case, if I manage to do practices which reveal my interpretation to be flawed, they will subjectively feel like I am investigating reality... while from a third-person perspective, we would rather say that I am investigating the way my mind builds up reality.

It is valid to stick to just the first-person experience of investigating reality directly. Many of these practices are framed solely in those terms, because a stance of curiosity and having as few assumptions as possible is the best mindset for actually doing the practices. But if one says that meditation investigates the nature of reality, then it becomes hard to test the claim from a third-person perspective. A common criticism is that meditation certainly changes how people experience the world, but it might just as well be loosening their grasp on reality.

On the other hand, if we provisionally assume that meditation works by revealing how the mind structures its model of reality, then we can check whether the kinds of insights that people report are compatible with what science tells us about the brain. If it turns out that meditators doing insight practices are coming up with experiences that match our understanding of actual brain mechanisms, then the practices might actually provide insight rather than delusion. In cases where no scientific evidence is yet available, it should at least be possible to construct a model that could be true and compatible with the third-person evidence.

In previous posts, I have explored some scientifically-informed models of the brain, which I think are naturally linked to the kinds of discoveries made in insight meditation. This article will more explicitly connect concepts from the theory of meditation to those kinds of models.

It is also worth noting that I think both claims about meditative insights are true: some things you can do with meditation do give you a better insight into reality, while some other things do just break your brain and reduce your contact with reality. (A fact responsible meditation teachers also warn about.) This makes it important to have third-person models of what could be a genuine insight and what is probably delusion, to help avoid the dangerous territory.

My multiagent model of mind

I have been calling my interpretation of those models a “multiagent model of mind”. One of the main ideas of the multiagent model is that the brain contains a number of different subsystems operating in parallel, each focusing on their own responsibilities. They share information on a subconscious level, but also through conscious thought. The content of consciousness roughly corresponds to information which is being processed in a “global workspace” - a “brain web” of long-distance neurons, which link multiple areas of the brain together into a densely interconnected network.

The global workspace can only hold a single piece of information at a time. At any given time, multiple different subsystems are trying to send information into the workspace, or otherwise modify its contents. Experiments show that a visual stimuli needs to be shown for about 50 milliseconds for it to be consciously registered, suggesting that the contents of consciousness might be updated at least 20 times per second. Whatever information makes it into consciousness will then be broadcast widely throughout the brain, allowing many subsystems to synchronize their processing around it.

The exact process by which this happens is not completely understood, but involves a combination of top-down mechanisms (e.g. attentional subsystems trying to strengthen particular signals and keep those in the workspace) as well as bottom-up ones (e.g. emotional content getting a priority). For example, if you are listening to someone talk in a noisy restaurant, both their words and the noise are bottom-up information within the workspace, while a top-down process tries to pick up on the words in particular. If a drunk person then suddenly collides with you, you are likely to become startled, which is a bottom-up signal strong enough to grab your attention (dominate the workspace), momentarily pushing away everything else.

There is also a constant learning process going on, where the brain learns which subsystems should be given access in which circumstances, while the subsystems themselves also undergo learning about what kind of information to send to consciousness.

When I talk about “subsystems” sending content into consciousness, I mean this as a very generic term, which includes all of the following:

  • Literal subsystems, e.g. information from the visual, auditory, and other sensory systems
  • Subpatterns within larger subsystems, e.g. a particular neuronal pattern encoding a specific memory or habit
  • Emotional schemas which trigger in particular situations and contain an interpretation of that situation and a response
  • Working memory buffers associated with type 2 (“System 2”) reasoning, helping chain the outputs of several different subsystems together

In some cases, I might talk about there being two separate subsystems, when one could argue that this would be better described as something like two separate pieces of data within a single subsystem. For example, I might talk about two different memories as two different subsystems, when one could reasonably argue that they are both contained within the same memory subsystem. Drawing these kinds of distinctions within the brain seems tricky, so rather than trying to figure out what term to use when, I will just talk about subsystems all the time.

Epistemic statusBuddhist theories of the mind are based on textual traditions that purport to record the remembered word of the Buddha, on religious and philosophical interpretations of those texts, and on Buddhist practices of mental cultivation. The theories aren’t formulated as scientific hypotheses and they aren’t scientifically testable. Buddhist insights into the mind aren’t scientific discoveries. They haven’t resulted from an open-ended empirical inquiry free from the claims of tradition and the force of doctrinal and sectarian rhetoric. They’re stated in the language of Buddhist metaphysics, not in an independent conceptual framework to which Buddhist and non-Buddhist thinkers can agree. Buddhist meditative texts are saturated with religious imagery and language. Buddhist meditation isn’t controlled experimentation. It guides people to have certain kinds of experiences and to interpret them in ways that conform to and confirm Buddhist doctrine. The claims that people make from having these experiences aren’t subject to independent peer review; they’re subject to assessment within the agreed-upon and unquestioned framework of the Buddhist soteriological path. [...] I’m not saying that Buddhist meditative techniques haven’t been experientially tested in any sense. Meditation is a kind of skill, and it’s experientially testable in the way that skills are, namely, through repeated practice and expert evaluation. I have no doubt that Buddhist contemplatives down through the ages have tested meditation in this sense. I’m also not saying that meditation doesn’t produce discoveries in the sense of personal insights. (Psychoanalysis can also lead to insights.) Rather, my point is that the experiential tests aren’t experimental tests. They don’t test scientific hypotheses. They don’t provide a unique set of predictions for which there aren’t other explanations. The insights they produce aren’t scientific discoveries. [...]I’m also not trying to devalue meditation. On the contrary, I’m trying to make room for its value by showing how likening it to science distorts it. Meditation isn’t controlled experimentation. Attention and mindfulness aren’t instruments that reveal the mind without affecting it. Meditation provides insight into the mind (and body) in the way that body practices like dance, yoga, and martial arts provide insight into the body (and mind). Such mind-body practices—meditation included—have their own rigor and precision. They test and validate things experientially, but not by comparing the results obtained against controls.-- Evan Thompson, Why I Am Not A Buddhist

I think it is reasonable to believe that meditation can give us genuine insights into the way the mind functions. The meditative techniques and practices which I am drawing upon in this article have been developed within Buddhist traditions, and I make frequent references to the theory developed within those traditions.

At the same time, while I am drawing upon theories developed within these traditions, I am treating those as a source of inspiration to be critically examined, rather than as sources of authority.

For one, there are many different Buddhist theories and schools that disagree with each other, many of them claiming to teach what the Buddha really meant. And as e.g. Evan Thompson’s book discusses, one cannot cleanly separate Buddhist meditative techniques from Buddhist religious teaching. People who meditate using those techniques - myself included - do so while being guided by an existing conceptual framework, framing their experiences in light of their framework. Practitioners who use different kinds of techniques and frameworks end up drawing different conclusions: e.g. some frameworks end up at the conclusion that no selves exist, while others end up believing that everything is self. (The extent to which this difference in framing actually leads to a different experience is unclear.) Many of these frameworks also draw upon supernatural elements, such as claims of rebirth and remembering past lives.

Still, many meditation teachers also say things along the lines of “you should not take any of this on faith, just try it out and see for yourself”. Personally I started out skeptical of many claims, dismissing them as pre-scientific folk-psychological speculation, before gradually coming to believe in them - sometimes as a result of meditation which hadn’t even been aimed at investigating those claims in particular, but where I thought I was doing something completely different. And it seems to me that many of the meditative techniques actively require you to suspend your expectations in order to work properly, requiring you to look at what’s present rather than at the thing you expect to see.

So, like many others, I simultaneously believe that i) meditative techniques point at genuine insights and also produce them in the minds of people who meditate and also that ii) we should not put excess faith in the claims of the existing meditation traditions. As many teachers encourage exactly this line of thought - as in the comment of taking nothing on faith - this feels like an appropriate spirit for approaching these matters.

Rather than trying to be authentically Buddhist, this article is concerned with building a model of the neural and psychological mechanisms I think the three characteristics are pointing at, even if that model ends up sharply deviating from the original theories. I heavily draw on my own experiences and the experiences and theories of other people whose reports I have reason to trust. I proceed from the assumption that regardless of whether the original frameworks are true or false, they do systematically produce similar effects and insights in the minds of the people following them, and that is an observation which needs to be explained.

In fact, I am happy to mix and match examples, exercises, interpretations and results drawn from all of the contemplative traditions that I happen to have any familiarity with, with current-day Western psychology and psychotherapy thrown in for good measure. They may have different approaches, but to the extent that they share commonalities, those commonalities tell us something about what human minds might have in common. And to the extent that they differ, one tradition might be pointing out aspects about the human mind that the others have neglected and vice versa, as in the fable of the blind men and the elephant.

Thank you to Alexei Andreev, David Chapman, Eliot Re, Jacob Spence, James Hogan, Magnus Vinding, Max Daniel, Matthew Graves, Michael Ashcroft, Romeo Stevens, Santtu Heikkinen, and Vojtěch Kovařík for valuable comments. Additional special thanks to Maija Haavisto. None of the people in question necessarily agree with all the content in this or the upcoming posts.

This is the first post of the "a non-mystical explanation of insight meditation and the three characteristics of existence" series. The next post in the series, "Self and No-Self", will be posted on Friday, 8th of May.



Discuss

Competitive safety via gradated curricula

5 мая, 2020 - 21:11
Published on May 5, 2020 6:11 PM GMT

Epistemic status: brainstorming some speculative research directions. Not trying to thoroughly justify the claims I’m making.

One way to think about the AI safety problem: there’s a spectrum of methods which each represent a different tradeoff between safety and ease of training an AGI, and unfortunately the two are anticorrelated. In particular, consider four regimes in which the bulk of training might occur (perhaps with additional fine-tuning afterwards):

  1. Training a language model to answer questions correctly.
  2. Training a RL agent on a range of limited tasks (e.g. games).
  3. Training a RL agent on general tasks in large-scale simulations for long time periods.
  4. Training a RL agent in competitive multi-agent environments.

I claim (but won’t fully defend here) that these are in order from safest but most difficult, to easiest but most dangerous:

  1. Regime 1 will produce a question-answering system which has no experience taking actions in the world, and which may not be goal-directed at all. But many researchers expect that it’ll be much easier to create an AGI which can answer difficult questions by training it to interact with a simulated world, so that its concepts are “grounded” by experience.
  2. Regime 2 is likely to produce an agent whose goals are bounded, and whose concepts are grounded; but which might only do well on the specific tasks it had been trained on. If so, building AGI in this regime would require a very sophisticated curriculum, if it’s possible at all.
  3. Regime 3 provides a rich environment for an agent to learn quite general skills and concepts. However, now the agent will also be rewarded for developing large-scale goals, which might make it dangerous.
  4. Regime 4 additional provides an “autocurriculum” via competition, the training signal from which could accelerate the development of general intelligence (as it did in humans). However, the agent could learn harmful skills and motivations (such as deception, manipulation or aggression) from competing with other agents, which it might then apply to interactions with humans.

This is a problem - but it’s also an opportunity. If we accept the claims I’ve made about this spectrum, then it might be much easier to train a relatively safe and non-agentic AGI if we start training in less safe regimes, and then gradually transition the training of that AGI into safer regimes. More specifically, I’m proposing a training curriculum in which an agent is trained in regime 4 until it displays a given level of competence; then moved to regime 3 until it again displays a significant amount of progress; then regime 2; then regime 1. The specific regimes used are not vital; some could be removed or replaced by others that I haven’t thought of. Neither is it essential to keep using exactly the same agent; it’d need to be retrained to use different observation and action spaces, and perhaps have its architecture modified during transitions. (In particular, it might be useful to incorporate a pre-trained language model at some point to kick-start its understanding of language.) The main point is that as training progresses, we increasingly use safer training regimes even though we expect it to be much more difficult to train an AGI solely using those regimes.

The key hypothesis is that it’s not uniformly harder to train AGIs in the safer regimes - rather, it’s primarily harder to get started in those regimes. Once an AI reaches a given level of intelligence, then transitioning to a safer regime might not slow down the rate at which it gains intelligence very much - but might still decrease the optimisation pressure in favour of that AI being highly agentic and pursuing large-scale goals.

I have some intuitions in favour of this hypothesis (although I’m still pretty uncertain). Here are three:

  1. Language-based tasks or limited-domain tasks can be made almost arbitrarily hard, we won’t run out of ways to challenge the agent (although this might require more work than continuing training in other regimes).
  2. Exploration is much easier for an agent that’s already quite intelligent, and so reward sparsity matters less. This would also make it easier to manually design tasks, since a wider range of difficulties would be acceptable.
  3. Once an agent has had some amount of interaction with a (simulated or real) world, it can interpret language as referring to objects in that world. Whereas without any interaction, it seems far more difficult to develop a solid understanding of objects and concepts. (Note that although quite a few AI researchers seem to believe that such grounding is important, I haven’t seen much justification for it in the context of modern machine learning. So I wanted to flag this as a particularly intuition-driven claim.)

One example which might help illustrate the overall hypothesis is that of Helen Keller, who developed a sophisticated understanding of the world (and wrote 12 books), despite becoming deaf and blind before the age of 2. Compare two options: either trying to train an AGI (from random initialisation to human-level intelligence) which receives the inputs that a typical human has, or else trying to do the same thing giving it only the inputs that Helen had (primarily touch, and language initially communicated via taps on her hand). I think the latter would be significantly more difficult, because having much less data imposes many fewer constraints on what the structure of the world could be like. And yet Helen did not learn many times more slowly than most people; in fact she earned a degree from Harvard in four years. My explanation for this: the learning problem Helen faced was much harder than what most of us face, but because her brain architecture had already been “trained” by evolution, she could make use of those implicit priors to match, and then surpass, most of her contemporaries.

The second crucial hypothesis is that the AGI doesn’t also retain dangerous characteristics from earlier training regimes. Again, I’m very uncertain about this hypothesis - it might be that, once the system has started training in a highly competitive environment, it will continue to have competitive and highly agentic motivations until we actively prevent it from doing so. Yet there are ways to mitigate this. In particular: to the extent that we can identify which part of the AGI is responsible for goal-directed cognition, we can remove that before continuing to train the rest of the network in the new regime. This would rely on interpretability techniques - but techniques which could be much cruder than those required to understand what an AGI is thinking or planning at any given moment. As an illustration, I think we already understand the human brain well enough to identify the parts most responsible for goal-directed cognition and motivation (although I don’t know much neuroscience, so corrections on this point would be very welcome). After removing the analogous sections of an AGI, it’d need to develop a new motivational system in subsequent training regimes - but I’m hoping that those later regimes only incentivise limited goal-directedness, towards bounded goals.

Right now, I think it’s pretty likely that there’ll be some transitions between different training regimes when developing AGI - e.g. for language models it’s already common to start with unsupervised pre-training, and then do supervised or RL fine-tuning. But note that this transition goes in the opposite direction (from more limited tasks to larger-scale tasks) compared with the ones I discussed above. So my proposal is a little counterintuitive; but while I’m not confident that it’ll be useful (perhaps because I’m still quite confused about agency and goal-directedness), I think it’s worth evaluating. One empirical test which could already be done is to see whether pre-training in a simulated environment is advantageous for developing better language models. Another, which might only be viable in a few years, is to investigate how separable goal-directed cognition is from world-modelling and other aspects of intelligence, in deep reinforcement learners with a range of neural architectures.


* Given that I have no experience of being deaf or blind, and have not looked into it very much, my intuitions on this point are not very well-informed; so I wanted to explicitly flag it as quite speculative.



Discuss

Writing Causal Models Like We Write Programs

5 мая, 2020 - 21:05
Published on May 5, 2020 6:05 PM GMT

Clunc

We’ll start with a made-up programming language called Clunc. The distinguishing feature of clunc is that it combines classes and functions into a single type, called a clunc. It looks like this:

quad = Clunc {
x = 4
constant = 3
linear = 2*x + constant
result = x*x + linear
}

We could then go to a terminal:

>>> quad.result
27
>>> quad.linear
11

In order to use this clunc like a function, we apply the do() operator. For instance,

>>> quad3 = do(quad, x=2)

… creates a new clunc which is just like quad, except that x is 2 rather than 4:

>>> quad3
Clunc {
x = 2
constant = 3
linear = 2*x + constant
result = x*x + linear
}

When we query fields of quad3, they reflect the new x-value:

>>> quad3.result
11
>>> quad3.linear
7

There’s no designated “input” or “output”; we can use the do() operator to override any values we please. For instance

>>> quad_zero_linear = do(quad, linear=0)
>>> quad_zero_linear
Clunc {
x = 4
constant = 3
linear = 0
result = x*x + linear
}
>>> quad_zero_linear.result
16

A few quick notes:

  • Clunc is purely clunctional: everything is immutable, and each variable can only be written once within a clunc. No in-place updates.
  • Clunc is lazy.
  • Variables can be set randomly, e.g. “x = random.normal(0, 1)”.
  • The do() operator creates a new clunc instance with the changes applied. If there are any random variables, they are re-sampled within the new clunc. If we want multiple independent samples of a randomized clunc M, then we can call do(M) (without any changes applied) multiple times.

To make this whole thing Turing complete, we need one more piece: recursion. Cluncs can “call” other cluncs, including themselves:

factorial = Clunc {
n = 4
base_result = 1
recurse_result = do(factorial, n=n-1).result
result = (n == 0) ? base_result : n * recurse_result
}

… and that’s where things get interesting.

Causal Models

Hopefully the mapping from clunc to probabilistic causal models is obvious: any clunc with random variables in it is a typical Pearl-style causal DAG, and the .mjx-chtml {display: inline-block; line-height: 0; text-indent: 0; text-align: left; text-transform: none; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; word-wrap: normal; word-spacing: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0; min-height: 0; border: 0; margin: 0; padding: 1px 0} .MJXc-display {display: block; text-align: center; margin: 1em 0; padding: 0} .mjx-chtml[tabindex]:focus, body :focus .mjx-chtml[tabindex] {display: inline-table} .mjx-full-width {text-align: center; display: table-cell!important; width: 10000em} .mjx-math {display: inline-block; border-collapse: separate; border-spacing: 0} .mjx-math * {display: inline-block; -webkit-box-sizing: content-box!important; -moz-box-sizing: content-box!important; box-sizing: content-box!important; text-align: left} .mjx-numerator {display: block; text-align: center} .mjx-denominator {display: block; text-align: center} .MJXc-stacked {height: 0; position: relative} .MJXc-stacked > * {position: absolute} .MJXc-bevelled > * {display: inline-block} .mjx-stack {display: inline-block} .mjx-op {display: block} .mjx-under {display: table-cell} .mjx-over {display: block} .mjx-over > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-under > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-stack > .mjx-sup {display: block} .mjx-stack > .mjx-sub {display: block} .mjx-prestack > .mjx-presup {display: block} .mjx-prestack > .mjx-presub {display: block} .mjx-delim-h > .mjx-char {display: inline-block} .mjx-surd {vertical-align: top} .mjx-mphantom * {visibility: hidden} .mjx-merror {background-color: #FFFF88; color: #CC0000; border: 1px solid #CC0000; padding: 2px 3px; font-style: normal; font-size: 90%} .mjx-annotation-xml {line-height: normal} .mjx-menclose > svg {fill: none; stroke: currentColor} .mjx-mtr {display: table-row} .mjx-mlabeledtr {display: table-row} .mjx-mtd {display: table-cell; text-align: center} .mjx-label {display: table-row} .mjx-box {display: inline-block} .mjx-block {display: block} .mjx-span {display: inline} .mjx-char {display: block; white-space: pre} .mjx-itable {display: inline-table; width: auto} .mjx-row {display: table-row} .mjx-cell {display: table-cell} .mjx-table {display: table; width: 100%} .mjx-line {display: block; height: 0} .mjx-strut {width: 0; padding-top: 1em} .mjx-vsize {width: 0} .MJXc-space1 {margin-left: .167em} .MJXc-space2 {margin-left: .222em} .MJXc-space3 {margin-left: .278em} .mjx-test.mjx-test-display {display: table!important} .mjx-test.mjx-test-inline {display: inline!important; margin-right: -1px} .mjx-test.mjx-test-default {display: block!important; clear: both} .mjx-ex-box {display: inline-block!important; position: absolute; overflow: hidden; min-height: 0; max-height: none; padding: 0; border: 0; margin: 0; width: 1px; height: 60ex} .mjx-test-inline .mjx-left-box {display: inline-block; width: 0; float: left} .mjx-test-inline .mjx-right-box {display: inline-block; width: 0; float: right} .mjx-test-display .mjx-right-box {display: table-cell!important; width: 10000em!important; min-width: 0; max-width: none; padding: 0; border: 0; margin: 0} .MJXc-TeX-unknown-R {font-family: monospace; font-style: normal; font-weight: normal} .MJXc-TeX-unknown-I {font-family: monospace; font-style: italic; font-weight: normal} .MJXc-TeX-unknown-B {font-family: monospace; font-style: normal; font-weight: bold} .MJXc-TeX-unknown-BI {font-family: monospace; font-style: italic; font-weight: bold} .MJXc-TeX-ams-R {font-family: MJXc-TeX-ams-R,MJXc-TeX-ams-Rw} .MJXc-TeX-cal-B {font-family: MJXc-TeX-cal-B,MJXc-TeX-cal-Bx,MJXc-TeX-cal-Bw} .MJXc-TeX-frak-R {font-family: MJXc-TeX-frak-R,MJXc-TeX-frak-Rw} .MJXc-TeX-frak-B {font-family: MJXc-TeX-frak-B,MJXc-TeX-frak-Bx,MJXc-TeX-frak-Bw} .MJXc-TeX-math-BI {font-family: MJXc-TeX-math-BI,MJXc-TeX-math-BIx,MJXc-TeX-math-BIw} .MJXc-TeX-sans-R {font-family: MJXc-TeX-sans-R,MJXc-TeX-sans-Rw} .MJXc-TeX-sans-B {font-family: MJXc-TeX-sans-B,MJXc-TeX-sans-Bx,MJXc-TeX-sans-Bw} .MJXc-TeX-sans-I {font-family: MJXc-TeX-sans-I,MJXc-TeX-sans-Ix,MJXc-TeX-sans-Iw} .MJXc-TeX-script-R {font-family: MJXc-TeX-script-R,MJXc-TeX-script-Rw} .MJXc-TeX-type-R {font-family: MJXc-TeX-type-R,MJXc-TeX-type-Rw} .MJXc-TeX-cal-R {font-family: MJXc-TeX-cal-R,MJXc-TeX-cal-Rw} .MJXc-TeX-main-B {font-family: MJXc-TeX-main-B,MJXc-TeX-main-Bx,MJXc-TeX-main-Bw} .MJXc-TeX-main-I {font-family: MJXc-TeX-main-I,MJXc-TeX-main-Ix,MJXc-TeX-main-Iw} .MJXc-TeX-main-R {font-family: MJXc-TeX-main-R,MJXc-TeX-main-Rw} .MJXc-TeX-math-I {font-family: MJXc-TeX-math-I,MJXc-TeX-math-Ix,MJXc-TeX-math-Iw} .MJXc-TeX-size1-R {font-family: MJXc-TeX-size1-R,MJXc-TeX-size1-Rw} .MJXc-TeX-size2-R {font-family: MJXc-TeX-size2-R,MJXc-TeX-size2-Rw} .MJXc-TeX-size3-R {font-family: MJXc-TeX-size3-R,MJXc-TeX-size3-Rw} .MJXc-TeX-size4-R {font-family: MJXc-TeX-size4-R,MJXc-TeX-size4-Rw} .MJXc-TeX-vec-R {font-family: MJXc-TeX-vec-R,MJXc-TeX-vec-Rw} .MJXc-TeX-vec-B {font-family: MJXc-TeX-vec-B,MJXc-TeX-vec-Bx,MJXc-TeX-vec-Bw} @font-face {font-family: MJXc-TeX-ams-R; src: local('MathJax_AMS'), local('MathJax_AMS-Regular')} @font-face {font-family: MJXc-TeX-ams-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_AMS-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_AMS-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_AMS-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-B; src: local('MathJax_Caligraphic Bold'), local('MathJax_Caligraphic-Bold')} @font-face {font-family: MJXc-TeX-cal-Bx; src: local('MathJax_Caligraphic'); font-weight: bold} @font-face {font-family: MJXc-TeX-cal-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-R; src: local('MathJax_Fraktur'), local('MathJax_Fraktur-Regular')} @font-face {font-family: MJXc-TeX-frak-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-B; src: local('MathJax_Fraktur Bold'), local('MathJax_Fraktur-Bold')} @font-face {font-family: MJXc-TeX-frak-Bx; src: local('MathJax_Fraktur'); font-weight: bold} @font-face {font-family: MJXc-TeX-frak-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-BI; src: local('MathJax_Math BoldItalic'), local('MathJax_Math-BoldItalic')} @font-face {font-family: MJXc-TeX-math-BIx; src: local('MathJax_Math'); font-weight: bold; font-style: italic} @font-face {font-family: MJXc-TeX-math-BIw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-BoldItalic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-BoldItalic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-BoldItalic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-R; src: local('MathJax_SansSerif'), local('MathJax_SansSerif-Regular')} @font-face {font-family: MJXc-TeX-sans-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-B; src: local('MathJax_SansSerif Bold'), local('MathJax_SansSerif-Bold')} @font-face {font-family: MJXc-TeX-sans-Bx; src: local('MathJax_SansSerif'); font-weight: bold} @font-face {font-family: MJXc-TeX-sans-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-I; src: local('MathJax_SansSerif Italic'), local('MathJax_SansSerif-Italic')} @font-face {font-family: MJXc-TeX-sans-Ix; src: local('MathJax_SansSerif'); font-style: italic} @font-face {font-family: MJXc-TeX-sans-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-script-R; src: local('MathJax_Script'), local('MathJax_Script-Regular')} @font-face {font-family: MJXc-TeX-script-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Script-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Script-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Script-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-type-R; src: local('MathJax_Typewriter'), local('MathJax_Typewriter-Regular')} @font-face {font-family: MJXc-TeX-type-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Typewriter-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Typewriter-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Typewriter-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-R; src: local('MathJax_Caligraphic'), local('MathJax_Caligraphic-Regular')} @font-face {font-family: MJXc-TeX-cal-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-B; src: local('MathJax_Main Bold'), local('MathJax_Main-Bold')} @font-face {font-family: MJXc-TeX-main-Bx; src: local('MathJax_Main'); font-weight: bold} @font-face {font-family: MJXc-TeX-main-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-I; src: local('MathJax_Main Italic'), local('MathJax_Main-Italic')} @font-face {font-family: MJXc-TeX-main-Ix; src: local('MathJax_Main'); font-style: italic} @font-face {font-family: MJXc-TeX-main-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-R; src: local('MathJax_Main'), local('MathJax_Main-Regular')} @font-face {font-family: MJXc-TeX-main-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-I; src: local('MathJax_Math Italic'), local('MathJax_Math-Italic')} @font-face {font-family: MJXc-TeX-math-Ix; src: local('MathJax_Math'); font-style: italic} @font-face {font-family: MJXc-TeX-math-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size1-R; src: local('MathJax_Size1'), local('MathJax_Size1-Regular')} @font-face {font-family: MJXc-TeX-size1-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size1-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size1-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size1-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size2-R; src: local('MathJax_Size2'), local('MathJax_Size2-Regular')} @font-face {font-family: MJXc-TeX-size2-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size2-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size2-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size2-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size3-R; src: local('MathJax_Size3'), local('MathJax_Size3-Regular')} @font-face {font-family: MJXc-TeX-size3-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size3-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size3-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size3-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size4-R; src: local('MathJax_Size4'), local('MathJax_Size4-Regular')} @font-face {font-family: MJXc-TeX-size4-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size4-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size4-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size4-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-R; src: local('MathJax_Vector'), local('MathJax_Vector-Regular')} @font-face {font-family: MJXc-TeX-vec-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-B; src: local('MathJax_Vector Bold'), local('MathJax_Vector-Bold')} @font-face {font-family: MJXc-TeX-vec-Bx; src: local('MathJax_Vector'); font-weight: bold} @font-face {font-family: MJXc-TeX-vec-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Bold.otf') format('opentype')} do() operator works exactly like it does for causal models. The “clunc” is really a model, given by structural equations. The one big change is the possibility of recursion: causal models “calling” other models or other instances of themselves.

To get some practice with this idea, let’s build a reasonably-involved analogue model of a ripple-carry adder circuit.

We’ll start at the level of a NAND gate (levels below that involve equilibrium models, which would require a bunch of tangential explanation). We’ll assume that we have some model MNAND, and we use do(MNAND,a=...,b=...).result to get the (noisy) output voltage of the NAND gate in terms of the input voltages a and b. Since we’re building an analogue model, we’ll be using actual voltages (including noise), not just their binarized values.

We’ll take MNAND as given (i.e. assume somebody else built that model). Building everything out of NAND gates directly is annoying, so we’ll make an XOR as well: Mxor=Model{a=0.0b=0.0intermediate=do(MNAND,a=a,b=b).resultleft=do(MNAND,a=a,b=intermediate).resultright=do(MNAND,a=intermediate,b=b).resultresult=do(MNAND,a=left,b=right).result}

This looks like a program which performs an XOR using NAND gates. But really, it’s a Pearl-style causal DAG model which uses a lot of NAND-submodels. We can write out the joint probability distribution P[a=a∗,b=b∗,intermediate=i∗,left=l∗,right=r∗,result=result∗|Mxor] via the usual method, with each line in the model generating a term in the expansion: P[a=a∗|MXOR]=I[a∗=0]P[b=b∗|MXOR]=I[b∗=0]P[intermediate=i∗|MXOR,a=a∗,b=b∗]=P[result=i∗|do(MNAND,a=a∗,b=b∗)]P[left=l∗|MXOR,a=a∗,intermediate=i∗]=P[result=l∗|do(MNAND,a=a∗,b=i∗)]P[right=r∗|MXOR,intermediate=i∗,b=b∗]=P[result=r∗|do(MNAND,a=i∗,b=b∗)]P[result=result∗|MXOR,left=l∗,right=r∗]=P[result=result∗|do(MNAND,a=l∗,b=r∗)] The full distribution is the product of those terms.

That’s just the first step. Next, we need a full adder, a circuit block which computes the sum and carry bits for one “step” of binary long addition. It looks like this:

Mfull_adder=Model{a=0b=0c=0sab=do(MXOR,a=a,b=b).results=do(MXOR,a=sab,b=c).resultcarryab=do(MNAND,a=a,b=b)carryc=do(MNAND,a=sab,b=c)carry=do(MNAND,a=carryab,b=carryc)}

As before, we can write out the components of the joint distribution line-by-line. I’ll just do a few this time:

P[a=a∗|Mfull_adder]=I[a∗=0]...P[sab=s∗ab|Mfull_adder,a=a∗,b=b∗]=P[result=s∗ab|MXOR,a=a∗,b=b∗]P[s=s∗|Mfull_adder,sab=s∗ab,c=c∗]=P[result=s∗|MXOR,a=s∗ab,b=c∗]...

Notice that some of these involve probabilities on the model MXOR, which we could further expand using the joint distribution of MXOR variables from earlier.

Finally, we can hook up a bunch of full adders to make our 32-bit ripple-carry adder:

Mrc=Model{a=[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]b=[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]f0=do(Mfull_adder,a=a[0],b=b[0])fi=do(Mfull_adder,a=a[i],b=b[i],c=fi−1.carry)∀i∈1,…,31result=[f0,…,f31]}

The components of the joint distribution for this one are left as an exercise for the reader.

Why Is This Useful?

Classes/functions let us re-use code; we don’t have to repeat ourselves. Likewise, clunc-ish causal models let us re-use submodels; we don’t have to repeat ourselves.

Obviously this has many of the same advantages as in programming. We can modularize our models, and fiddle with the internals of one submodel independently of other submodels. We can “subclass” our models via the do()-operator, to account for different contexts. Different people can work on different submodels independently - we could even imagine libraries of submodels. An electrical engineer could write a probabilistic causal model representing the low-level behavior of a chip; others could then import that model and use it as a reference when designing things which need to work with the chip, like packaging, accessories, etc.

From a more theoretical perspective, when we write programs with unbounded runtime, we have to have some way to re-use code: there’s only so many lines in the program, so the program must visit some of the lines multiple times in the course of execution. Some lines must be re-used. Likewise for probabilistic models: if we want to define large models - including unbounded models - with small/finite definitions, then we need some way to re-use submodels. We could do that by writing things like “∀i:<submodeli>”, but if we want Turing completeness anyway, we might as well go for recursion.

From a pure modelling perspective, the real world contains lots of repeating structures. If we’re modelling things like cars or trees, we can re-use a lot of the information about one car when modelling another car. We think of cars as variations on a template, and that’s exactly what the do() operator provides: we give it some “template” model, and apply modifications to it. The corresponding inverse problem then says: given a world full of things which are variations on some templates, find the templates and match them to the things - i.e. learn to recognize and model “cars” and “trees”. Clunc-ish causal models are a natural fit for this sort of problem; they naturally represent things like “corvette with a flat tire”.

Finally, the main reason I’ve been thinking about this is to handle abstraction. Clunc-ish models make layers of abstraction natural; lower-level behaviors can be encapsulated in submodels, just as we saw above with the ripple-carry adder. If we want to write abstraction-learning algorithms - algorithms which take in raw data and spit out multi-level models with layers of abstraction - then clunc-ish models are a natural form for their output. This is what multi-level world models look like.



Discuss

I'd Like To Maximizing Profit and Happiness but My Happiness conflicts a lot with making profit...

5 мая, 2020 - 20:50
Published on May 5, 2020 5:54 AM GMT

I have a history of changing and trying different things. I can learn things relatively quickly than the average person however there comes a point where I reassess what I am doing and most of the time change my course of action. A huge goal of my life is to be extremely wealthy. However, many of the businesses I chose or that was choosen for me "up for debate" has been slow growth businesses. The business don't have much rooms to scale very quickly because I prefer the own and rent model. As it doesn't take up much time so I can spend learning new things. The two businesses are real estate and car rentals. I finally realized that the majority of the value in the business is from the capital built up and management. So I decided to go into the arts and businesses that I create value for business. My most recent Art business is music. I tried painting and not that interested or also its not really scalable. I also worked on fashion design however I ended up outsourcing the entire design process and without sufficient knowledge of the design process my value to the business was pretty much zero. Anyways I feel like I have gaps in my knowledge and understanding to get to the next level of personal development and financial success. Any suggestions to help speed up this process?

Thank you



Discuss

"AI and Efficiency", OA (44✕ improvement in CNNs since 2012)

5 мая, 2020 - 19:32
Published on May 5, 2020 4:32 PM GMT



Discuss

Страницы