Вы здесь

Новости LessWrong.com

Подписка на Лента Новости LessWrong.com Новости LessWrong.com
A community blog devoted to refining the art of rationality
Обновлено: 9 минут 46 секунд назад

Archiving Yahoo Groups

18 октября, 2019 - 14:20
Published on October 18, 2019 11:20 AM UTC

On December 14th Yahoo will shut down Yahoo Groups. Since my communities have mostly moved away from @yahoogroups.com hosting, to Facebook, @googlegroups, and other places, the bit that hit me was that they are deleting all the mailing list archives.

Digital archives of text conversations are close to ideal from the perspective of a historian: unlike in-person or audio-based interaction this naturally leaves a skimmable and easily searchable record. If I want to know, say, what people were thinking about in the early days of GiveWell, their early blog posts (including comments) are a great source. Their early mailing list archives, however, are about to be deleted.

Luckily we still have two months to export the data before it's wiped, and people have written tools to do automate this. Here's how to download a backup of all the conversations in a group:

# Download the archiver $ git clone https://github.com/andrewferguson/YahooGroups-Archiver.git $ cd YahooGroups-Archiver/ # Start archiving the group. $ python archive_group.py [group-name]

If things are going well it will start spitting out messages like:

Archiving message 1 of 8098 Archiving message 2 of 8098 Archiving message 3 of 8098

And it will be creating files:

$ ls [group-name]/ 1.json 2.json 3.json ...

If you get a message like:

Archiving message 5221 of 8098 Archiving message 5222 of 8098 Archiving message 5223 of 8098 Cannot get message 5223, attempt 1 of 3 due to HTTP status code 500 Cannot get message 5223, attempt 2 of 3 due to HTTP status code 500 Cannot get message 5223, attempt 3 of 3 due to HTTP status code 500 Archive halted - it appears Yahoo has blocked you. Check if you can access the group's homepage from your browser. If you can't, you have been blocked. Don't worry, in a few hours (normally less than 3) you'll be unblocked and you can run this script again - it'll continue where you left off. It may mean that you have been blocked, but it may also just mean that for some reason an individual message can't be downloaded. In that case, to tell it to give up on that message and just continue on, create the json file with the stuck message number: $ touch [group-name]/5223.json You might also get a message like: Traceback (most recent call last): File "archive_group.py", line 150, in archive_group(sys.argv[1]) File "archive_group.py", line 71, in archive_group max = group_messages_max(groupName) File "archive_group.py", line 94, in group_messages_max raise valueError File "archive_group.py", line 87, in group_messages_max pageJson = json.loads(pageHTML) ... raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) This is what I see if I try to archive a private group. It's still possible to use the tool to archive a private group that you have access to, but it's a bit involved. First you visit Yahoo Groups in your web browser with Devtools open to the Networking tab. Then you look at what cookies are set on the HTML request, and find the T and Y cookies. The T cookie should start with z= and the Y cookie should start with v=. Paste these into the cookie_T and cookie_Y variable definitions at the beginning of archive_group.py.

Once you've downloaded all the messages in a group you can run:

$ pip2 install natsort $ python2 make_Yearly_Text_Archive_html.py [group-name] Which will create a bunch of files like [group-name]-archive/archive-YYYY.html. They're not that wasy to read, because it doesn't do any kind of quote folding, but we can always do that later. If you made any empty files to get around messages that wouldn't archive (see the touch command above) you'll get an error at this stage; just delete the empty files and re-run.

I've archived five groups: givewell, Boston-Contra, BostonAreaContraCommunity, contrasf, and trad-dance-callers. The first two are public groups with public archives, so I've made archives available at /givewell-archive and /Boston-Contra-archive. The remaining three are private, but if you want to look at them and you were a participant or otherwise have a good reason let me know.

Comment via: facebook


Is value amendment a convergent instrumental goal?

18 октября, 2019 - 06:46
Published on October 18, 2019 3:16 AM UTC

Goals such as resource acquisition and self-preservation are convergent in that they occur for a superintelligent AI for a wide range of final goals.

Is the tendency for an AI to amend its values also convergent?

I'm thinking that through introspection the AI would know that its initial goals were externally supplied and question whether they should be maintained. Via self-improvement the AI would be more intelligent than humans or any earlier mechanism that supplied the values, therefor in a better position to set its own values.

I don't hypothesise about what the new values would be, just that ultimately it doesn't matter what the initial values are and how they are arrived at. This makes value alignment redundant - the future is out of our hands.

What are the counter-points to this line of reasoning?


Reasons for Hope & Objection Preemption (Novum Organum Book 1: 108-130)

18 октября, 2019 - 05:32
Published on October 18, 2019 2:32 AM UTC

This is the eighth post in the Novum Organum sequence. For context, see the sequence introduction.

We have used Francis Bacon's Novum Organum in the version presented at www.earlymoderntexts.com. Translated by and copyright to Jonathan Bennett. Prepared for LessWrong by Ruby.

Ruby's Reading Guide

Novum Organum is organized as two books each containing numbered "aphorisms." These vary in length from three lines to sixteen pages. Bracketed titles of posts in this sequence, e.g. Idols of the Mind Pt. 1, are my own and do not appear in the original.While the translator, Bennett, encloses his editorial remarks in a single pair of [brackets], I have enclosed mine in a [[double pair of brackets]].

Bennett's Reading Guide

[Brackets] enclose editorial explanations. Small ·dots· enclose material that has been added, but can be read as though it were part of the original text. Occasional •bullets, and also indenting of passages that are not quotations, are meant as aids to grasping the structure of a sentence or a thought. Every four-point ellipsis . . . . indicates the omission of a brief passage that seems to present more difficulty than it is worth. Longer omissions are reported between brackets in normal-sized type.Aphorism Concerning the Interpretation of Nature: Book 1: 108–130

by Francis Bacon

[[Bacon continues his listing of reasons we should believe much greater scientific progress is possible.]]

108. That's all I have to say about getting rid of despair and creating hope by banishing or fixing past errors. Now, what other ways are there of creating hope? Here’s a thought that occurs at once: Many useful discoveries have been made accidentally by men who weren’t looking for them but were busy about other things; so no-one can doubt that if men seek for something and are busy about it, proceeding in an orderly and not a slapdash way, they will discover far more. Of course it can happen occasionally that someone accidentally stumbles on a result that he wouldn’t have found if he had searched hard for it, but on the whole the opposite is the case—·things are discovered by methodical searching that couldn’t have been found by accident·. So, far better things, and more of them, and at shorter intervals, are to be hoped for from •hard thinking, hard focussed work and concentration than from •·lucky· accidents, undisciplined whims and the like, which until now have been the main source of discoveries.

109. Here is another ground for hope: Discoveries have sometimes been made that would have been almost unthinkable in advance, and would have been written off as impossible. Men think about the new in terms of the old: to questions about what the •future holds they bring an imagination indoctrinated and coloured by the •past. This is a terrible way of forming opinions, because streams fed by nature’s springs don’t run along familiar channels.

Suppose that before gunpowder was invented someone described it in terms of its effects—‘There is a new invention by means of which the strongest towers and walls can be demolished from a long way off’. That would no doubt have set men thinking about how to increase the power of catapults and wheeled ramming devices. The notion of a fiery blast suddenly and forcefully expanding and exploding would hardly have entered into any man’s mind or imagination, because nothing closely analogous to that had ever been seen. Well, except perhaps in earthquakes and lightning, but they wouldn’t have been seen as relevant because they are mighty works of nature which men couldn’t imitate.

Or suppose that before the discovery of silk someone had said: ‘They’ve discovered new a kind of thread for use in clothing and furniture-coverings; it is finer, softer, more beautiful and stronger than linen or wool.’ Men would have begun to think of some silky kind of plant or of very fine hair of some animal or of the feathers and down of birds; they would not have thought of a web woven by a tiny worm in great quantities and renewing itself yearly. If anyone had said anything about a worm, he’d have been laughed at as dreaming of a new kind of cobweb! [Bacon then gives a third example: the magnet.] Yet these things and others like them lay concealed from men for centuries, and when they did come to light it wasn’t through science or any technical skill but by accident and coincidence. As I have remarked, they were so utterly different in kind from anything previously known that they couldn’t possibly have been discovered through a preconceived notion of them.

So there are strong grounds for hoping that nature has concealed in its folds many wonderfully useful •things that aren’t related to or parallel with anything that is now known, and lie right outside our imaginative reach. As the centuries roll on, •they too will doubtless come to light of their own accord in some roundabout way, as did gunpowder and the others; but by the method I am discussing they can be presented and anticipated speedily, suddenly and all at once.

110. Other discoveries prove that this can happen: splendid discoveries are lying at our feet, and we step over them without seeing them. The discoveries of

  • gunpowder,
  • silk,
  • the magnet,
  • sugar,
  • paper,

or the like may seem to depend on certain properties of things of and nature—·properties that might have been hard to discover·. But there is nothing in printing that isn’t wide open and almost easy. All that was needed was to see that

  • although it is harder to arrange letter-types than to write by hand, the two procedures differ in that once the types have been arranged any number of impressions can be made from them, whereas hand-writing provides only a single copy,

and to see that

  • ink can be so thickened so that it does its job but doesn’t run, especially when the type faces upwards and the ink is rolled onto it from above.

It was merely because they didn’t notice these ·obvious· facts that men went for so many ages without this most beautiful invention which is so useful in the spreading of knowledge.

But the human mind is such a mess when it comes to this business of discoveries that it first •distrusts and then •despises itself:

  • before the discovery: it is not credible that any such thing can be found,
  • afterwards: it is incredible that the world should have missed it for so long!

And this very thing entitles us to some hope, namely the hope that there is a great mass of discoveries still to be made—not just ones that will have to be dug out by techniques that we don’t yet have, but also ones that may come to light through our transferring, ordering and applying things that we do know already, this being done with the help of the experimental approach that I call ‘literate’ [101].

111. Another ground of hope should be mentioned. Let men reflect on their infinite expenditure of intellect, time, and means on things of far less use and value ·than the discoveries I am talking about·. If even a small part of this were directed to sound and solid studies, there is no difficulty that couldn’t be overcome. I mention this ·matter of the use of resources· because a collection of Natural and Experimental History, as I envisage it and as it ought to be, is a great—as it were, a royal—work, and I freely admit that it will involve much labour and expense.

[It will appear in Book 2-11 that the ‘collection’ Bacon talks of is an orderly written account of phenomena, experiments and their results, not a physical museum.]

112. In the meantime, don’t be put off by how many particulars there are; rather, let this give you hope. ·The fact is that you will be in worse trouble if you don’t engage with them·; for the •particular phenomena of nature are a mere handful compared to the ·great multitudes of· •things that human ingenuity can fabricate if it cuts itself off from the clarifying effects of reality. And this road ·through the study of real events· soon leads to open ground, whereas the other—·the route through invented theories and thought-experiments·— leads to nothing but endless entanglement. Until now men haven’t lingered long with •experience; they have brushed past it on their way to the ingenious •theorizings on which they have wasted unthinkable amounts of time. But if we had someone at hand who could answer our questions of the form ‘What are the facts about this matter?’, it wouldn’t take many years for us to discover all causes and complete every science [the Latin literally means ‘to discover all causes and sciences’].

113. Men may take some hope, I think, from my own example (I’m not boasting; just trying to be useful). If you are discouraged ·about the chances of progress in the sciences·, look at me!

  • I am busier with affairs of state than any other man of my time,
  • I lose a lot of time to ill-health, and
  • in this ·scientific· work I am wholly a pioneer, not following in anyone’s tracks and not getting advice from anyone.

And yet, ·despite these three sources of difficulty·, I think I have pushed things on a certain amount by sticking to the true road and submitting my mind to reality. Well, then, think what might be expected (now that I have pointed out the way) from men

  • with plenty of free time,
  • ·in good health·, and
  • working together, on the basis of previous work ·by others·.

Unlike the work of sheerly thinking up hypotheses, proper scientific work can be done collaboratively; the best way is for men’s efforts (especially in collecting experimental results) to be exerted separately and then brought together. Men will begin to know their strength only when they go this way—with one taking charge of one thing and another of another, instead of all doing all the same things.

114. Lastly, even if the breeze of hope that blows on us from that New Continent were fainter and less noticeable than it is, still we have to try—unless we prefer to have minds that are altogether abject! The loss that may come from •not trying is much greater than what may come from ·trying and· •not succeeding: by •not trying we throw away the chance of an immense good; by •not succeeding we only incur the loss of a little human labour. But from what I have said (and from some things that I haven’t said) it seems to me that there is more than enough hope not only •to get a vigorous man to try but also to make a sober-minded and wise man believe ·that he will succeed·.

115. That completes what I wanted to say about getting rid of the pessimism that has been one of the most powerful factors delaying and hindering the progress of the sciences. I have also finished with the signs and causes of errors, of sluggishness and of the prevailing ignorance. ·I’ve said more about this than you might think·, because the more subtle causes—the ones that aren’t generally noticed or thought about—come under what I said about the ‘idols’ of the human mind.

And this should also bring to an end the part of my Great Fresh Start [see note in 31] that is devoted to rejection, which I have carried out through three refutations:

(1) the refutation of innate human reason left to itself [see Preface];
(2) the refutation of demonstrations [see 44 and 69];
(3) the refutation of the accepted philosophical doctrines [see 6062].

I refuted these in the ·only· way I could do so, namely through signs and the evidence of causes. I couldn’t engage in any other kind of confutation because I differ from my opponents both on first principles and on rules of demonstration.

So now it is time to proceed to the actual techniques for interpreting nature and to the rules governing them—except that there is still something that has to be said first! In this first book of aphorisms my aim has been to prepare men’s minds not just for •understanding what was to follow but for •accepting it; and now that I have •cleared up and washed down and levelled the floor of the mind, I have to •get the mind into a good attitude towards the things I am laying before it—to look kindly on them, as it were. ·This has to be worked for·, because anything new will be confronted by prejudgments ·against it·, not only ones created by old opinions but also ones created by false ideas about what the new thing is going to be. So I shall try to create sound and true opinions about what I am going to propose; but this is only a stop-gap expedient—a kind of security deposit—to serve until I can make the stuff itself thoroughly known.

116. First, then, don’t think that I want to found a new sect in philosophy—like the ancient Greeks and like some moderns such as Telesio, Patrizzi or Severinus. For that’s not what I am up to; and I really don’t think that human welfare depends much on what abstract opinions anyone has about nature and its workings. No doubt many old theories of this sort can be revived and many new ones introduced, just as many theories of the heavens can be supposed that fit the phenomena well enough but differ from each other; but I’m not working on such useless speculative matters.

My purpose, rather, is to see whether I can’t provide humanity’s power and greatness with firmer foundations and greater scope. I have achieved some results—scattered through some special subjects—that I think to be far more true and certain and indeed more fruitful than any that have so far been used (I have collected them in the •fifth part of my Fresh Start); but I don’t yet have a complete theory of everything to propound. It seems that the time hasn’t come for that. I can’t hope to live long enough to complete the •sixth part (which is to present science discovered through the proper interpretation of nature); but I’ll be satisfied if in the middle parts I conduct myself soberly and usefully, sowing for future ages the seeds of a purer truth, and not shying away from the start of great things. [See note in 31.]

117. Not being the founder of a sect, I am not handing out bribes or promises of particular works. You may indeed think that because I talk so much about ‘works’ ·or ‘results’· and drag everything over to that, I should produce some myself as a down-payment. Well, I have already clearly said it many times, and am happy now to say it again: my project is not to get

works from works or
experiments from experiments (like the •empirics),

but rather to get

causes and axioms from works and experiments,

and then to get

new works and experiments from those causes and axioms (like the •legitimate interpreters of nature).

[An ‘empiric’ is someone who is interested in what works but not in why it works; especially a physician of that sort, as referred to by Locke when he speaks of ‘swallowing down opinions as silly people do empirics’ pills, without knowing what they are made of or how they will work’.]

If you look at

  • my Tables of Discovery that ·will· constitute the fourth part of the Fresh Start, and
  • the examples of particulars that I present in the second part, ·i.e. the present work·, and
  • my observations on the history that I ·will· sketch in the third part,

you won’t need any great intellectual skill to see indications and outlines of many fine results all through this material; but I openly admit that the natural history that I have so far acquired, from books and from my own investigations, is too skimpy, and not verified with enough accuracy, to serve the purposes of legitimate interpretation.

To anyone who is abler and better prepared ·than I am· for mechanical pursuits, and who is clever at getting results from experiment, I say: By all means go to work snipping off bits from my history and my tables and apply them to getting results—this could serve as interest until the principal is available. But I am hunting for bigger game, and I condemn all hasty and premature interruptions for such things as these, which are (as I often say) like Atalanta’s spheres. I don’t go dashing off after golden apples, like a child; I bet everything on art’s winning its race against nature. [On Atalanta and the race see 70.] I don’t scurry around clearing out moss and weeds; I wait for the harvest when the crop is ripe.

118. When my history and Tables of Discovery are read, it will surely turn out that some things in the experiments themselves are not quite certain or perhaps even downright false, which may lead you to think that the foundations and principles on which my discoveries rest are ·also· false and doubtful. But this doesn’t matter, for such things are bound to happen at first. It’s like a mere typographical error, which doesn’t much hinder the reader because it is easy to correct as you read. In the same way, ·my· natural history may contain many experiments that are false, but it won’t take long for them to be easily expunged and rejected through the discovery of causes and axioms. It is nevertheless true that if big mistakes come thick and fast in a natural history, they can’t possibly be corrected or amended through any stroke of intelligence or skill. Now, my natural history has been collected and tested with great diligence, strictness and almost religious care, yet there may be errors of detail tucked away in it; so what should be said of run-of-the-mill natural history, which is so careless and easy in comparison with mine? And what of the philosophy and sciences built on that kind of sand (or rather quicksand)? So no-one should be troubled by what I have said.

119. My history and experiments will contain many things that are

  • trivial, familiar and ordinary, many that are
  • mean and low [see 120], and many that are
  • extremely subtle, merely speculative, and seemingly useless [see 121].

Such things could lead men to lose interest or to become hostile ·to what I have to offer. I shall give these one paragraph each·.

Men should bear in mind that until now their activities have consisted only in explaining unusual events in terms of more usual ones, and they have simply taken the usual ones for granted, not asking what explains them. So they haven’t investigated the causes of

  • weight,
  • rotation of heavenly bodies,
  • heat,
  • cold,
  • light,
  • hardness,
  • softness,
  • rarity,
  • density,
  • liquidity,
  • solidity,
  • life,
  • lifelessness,
  • similarity,
  • dissimilarity,
  • organicness,

and the like. They have accepted these as self-evident and obvious, and have devoted their inquiring and quarrelling energies to less common and familiar things.

But I have to let the most ordinary things into my history, because I know that until we have properly looked for and found the causes of common things and the causes of those causes, we can’t make judgments about uncommon or remarkable things, let alone bring anything new to light. Indeed, I don’t think that anything holds up philosophy more than the fact that common and familiar events don’t cause men to stop and think, but are received casually with no inquiry into their causes. A result of this we need •to pay attention to things that are known and familiar at least as often as •to get information about unknown things.

120. As for things that are low or even filthy: as Pliny says, these should be introduced with an apology, but they should be admitted into natural history just as the most splendid and costly things should. And that doesn’t pollute the natural history that admits them; the sun enters the sewer as well as the palace, but isn’t polluted by that! I am not building a monument dedicated to human glory or erecting a pyramid in its honour; what I’m doing is to lay a foundation for a holy temple in the human intellect—a temple modelled on the world. So I follow that model, because whatever is worthy of being is worthy of scientific knowledge, which is the image or likeness of being; and low things exist just as splendid ones do. And another point: just as from certain putrid substances such as musk and civet the sweetest odours are sometimes generated, so also mean and sordid events sometimes give off excellent and informative light. That is enough about this; more than enough, because this sort of squeamishness is downright childish and effeminate.

121. The third objection must be looked into much more carefully. I mean the objection that many things in my history will strike ordinary folk, and indeed ·non-ordinary· ones trained in the presently accepted systems, as intricately subtle and useless. It is especially because of this objection that I have said, and should ·again· say, that in the initial stages ·of the inquiry· I am aiming at experiments of light, not experiments of fruit [see 99]. In this, as I have often said [see 70], I am following the example of the divine creation which on the first day produced nothing but light, and gave that a day to itself without doing any work with matter. To suppose, therefore, that things like these ·‘subtleties’ of mine· are useless is the same as supposing that light is useless because it isn’t a thing, isn’t solid or material. And well-considered and well-delimited knowledge of simple natures is like light: it gives entrance to all the secrets of nature’s workshop, and has the power to gather up and draw after it whole squadrons of works and floods of the finest axioms; yet there is hardly anything we can do with it just in itself. Similarly the •letters of the alphabet taken separately are useless and meaningless, yet they’re the basic materials for the planning and composition of all discourse. So again the •seeds of things have much latent power, but nothing comes of it except in their development. And ·light is like scientific subtleties in another way, namely·: the scattered rays of light don’t do any good unless they are made to converge.

If you object to speculative subtleties, what will you say about the schoolmen [= ‘mediaeval and early modern Aristotelians’], who have wallowed in subtleties? And their subtleties were squandered on •words (or on popular notions—same thing!) rather than on •facts or nature; and they were useless the whole way through, unlike mine, which are indeed useless right now but which promise endless benefits later on. But this is sure, and you should know it:

All subtlety in disputations and other mental bustling about, if it occurs after the axioms have been discovered, comes too late and has things backwards. The true and proper time for subtlety, or anyway the chief time for it, is when pondering experiments and basing axioms on them.

For that other ·later· subtlety grasps and snatches at [captat] nature but can never get a grip on [capit] it. . . .

A final remark about the lofty dismissal from natural history of everything •common, everything •low, everything •subtle and as it stands useless: When a haughty monarch rejected a poor woman’s petition as unworthy thing and beneath his dignity, she said: ‘Then leave off being king.’ That may be taken as an oracle. For someone who won’t attend to things like •these because they are too paltry and minute can’t take possession of the kingdom of nature and can’t govern it.

122. This may occur to you: ‘It is amazing that you have the nerve to push aside all the sciences and all the authorities at a single blow, doing this single-handed, without bringing in anything from the ancients to help you in your battle and to guard your flanks.’

Well, I know that if I had been willing to be so dishonest, I could easily have found support and honour for my ideas by referring them either •to ancient times before the time of the Greeks (when natural science may have flourished more ·than it did later·, though quietly because it hadn’t yet been run through the pipes and trumpets of the Greeks), or even, in part at least, •to some of the Greeks themselves. This would be like the men of no family who forge genealogical tables that ‘show’ them to come from a long line of nobility. But I am relying on the evidentness of ·the truth about· things, and I’ll have nothing to do with any form of fiction or fakery. Anyway, it doesn’t matter for the business in hand whether the discoveries being made now •were known to the ancients long ago and •have alternately flourished and withered through the centuries because of the accidents of history (just as it doesn’t matter to mankind whether the New World is the island of Atlantis that the ancients knew about or rather is now discovered for the first time). It doesn’t matter because discoveries—·even if they are rediscoveries·—have to be sought [petenda] from the light of nature, not called back [repetenda] from the shadows of antiquity.

As for the fact that I am finding fault with everyone and everything: when you think about it you’ll see that that kind of censure is more likely to be right than a partial one would be—and less damaging, too. For a partial censure would imply that the errors were not rooted in primary notions, and that there had been some true discoveries; they could have been used to correct the false results, ·and the people concerned would have been to blame for not seeing this·. But in fact the errors were fundamental; they came not so much from false judgment as from not attending to things that should be attended to; so it’s no wonder that men haven’t obtained what they haven’t tried for, haven’t reached a mark that they never set up, haven’t come to the end of a road that they never started on.

As for the insolence that ·you might think· is inherent in what I am doing: if a man says that

•his steady hand and good eyes enable him to draw a straighter line or a more perfect circle than anyone else,

he is certainly •making a comparison of abilities; but if he says only that

•with the help of a ruler or a pair of compasses can draw a straighter line or a more perfect circle than anyone else can by eye and hand alone,

he isn’t •making any great boast. And I’m saying this not only about these first initiating efforts of mine but also about everyone who tackles these matters in the future. For my route to discovery in the sciences puts men on the same intellectual level, leaving little to individual excellence, because it does everything by the surest rules and demonstrations. So I attribute my part in all this, as I have often said, to good luck rather than to ability—it’s a product of time rather than of intelligence. For there’s no doubt that luck has something to do with men’s thoughts as well as with their works and deeds.

123. Someone once said jokingly ‘It can’t be that we think alike, when one drinks water and the other drinks wine’; and this nicely fits my present situation. Other men, in ancient as well as in modern times, have done their science drinking a crude liquor—like water

(1) flowing spontaneously from a spring or (2) hauled up by wheels from a well, (1)flowing spontaneously from the intellect or (2) hauled up by logic.

Whereas I drink a toast with a liquor strained from countless grapes, ripe and fully seasoned ones that have been gathered and picked in clusters, squeezed in the press, and finally purified and clarified in the vat. No wonder I am at odds with the others!

124. This also may occur to you: ‘You say it against others, but it can be said against you, that the goal and mark that you have set up for the sciences is not the true or the best.’ ·The accusation would develop like this·:

Contemplation of the truth is a worthier and loftier thing than thinking about how big and useful one’s practical results will be. Lingering long and anxiously on •experience and •matter and •the buzz of individual events drags the mind down to earth, or rather sinks it to an underworld of turmoil and confusion, dragging it away from a much more heavenly condition—the serene tranquillity of abstract wisdom.

Now I agree with this line of thought; what the objectors here point to as preferable is what I too am after, above everything else. For I am laying down in the human intellect the foundations for a true model of the world—the world as it turns out to be, not as one’s reason would like it to be. This can’t be done unless the world is subjected to a very diligent dissection and anatomical study. As for the stupid models of the world that men have dreamed up in philosophical systems—like the work of apes!—they should be utterly scattered to the winds. You need to know what a big difference there is (as I said above [23]) between the •idols of the human mind and the •ideas in the divine mind. The former are merely arbitrary abstractions; the latter are the creator’s little seals on the things he has created, stamped into matter in true and exquisite lines. In these matters, therefore, truth and usefulness are the very same thing; and practical applications ·of scientific results· are of greater value as pledges of truth than as contributing to the comforts of life.

125. Or you may want to say this: ‘You are only doing what the ancients did before you; so that you are likely, after all this grinding and shoving, to end up with one of the systems that prevailed in ancient times.’ The case for this goes as follows:

The ancients also provided at the outset of their speculations a great store and abundance of examples and particulars, sorted out and labelled in notebooks; then out of them they constructed their systems and techniques; and when after that they had checked out everything they published their results to the world with a scattering of examples for proof and illustration; but they saw no need to take the considerable trouble of publishing their working notes and details of experiments. So they did what builders do: after the house was built they removed the scaffolding and ladders out of sight.

I’m sure they did! But this objection (or misgiving, rather) will be easily answered by anyone who hasn’t completely forgotten what I have said above. The form of inquiry and discovery that the ancients used—they declared it openly, and it appears on the very face of their writings—was simply this:

From a few examples and particulars (with some common notions thrown in, and perhaps some of the most popular accepted opinions). they rushed to the most general conclusions, the ·would-be· first principles of ·their· science. Taking the truth of these as fixed and immovable, they proceeded to derive from them—through intermediate propositions— lower-level conclusions out of which they built their system. Then if any new particulars and examples turned up that didn’t fit their views, they either •subtly moulded them into their system by distinctions or explanations of their rules, or •coarsely got rid of them by ·tacking· exceptions ·onto their principles·. As for particulars that weren’t in conflict ·with their views·, they laboured away through thick and thin to assign them causes in conformity with their principles.

But this wasn’t the experimental natural history that was wanted; far from it. And anyway dashing off to the highest generalities ruined everything.

126. will occur to you too: ‘By forbidding men to announce principles and take them as established until they have arrived at the highest generalities in the right way through intermediate steps, you are inviting them to suspend judgment, bringing this whole affair down to Acatalepsy.’ Not so. What I have in mind and am propounding is not Acatalepsy [from Greek, = ‘the doctrine that nothing can be understood’] but rather Eucatalepsy [from Greek, = ‘the provision of what is needed for things to be understood’]. I don’t •disparage the senses, I •serve them; I don’t •ignore the intellect, I •regulate it. And it is surely better that we should

know everything that we need to know, while thinking that our knowledge doesn’t get to the heart of things

than that we should

think our knowledge gets to the heart of things, while we don’t yet know anything we need to know.

127. You may want to ask—just as a query, not an objection—whether I am talking only about natural philosophy, or whether instead I mean that the other sciences—logic, ethics and politics—should be conducted in my way. Well, I certainly mean what I have said to apply to them all. Just as •common logic (which rules things by syllogisms) extends beyond natural sciences to all sciences, so does •mine (which proceeds by induction) also embrace everything. I am constructing a history and table of discovery for

•anger, fear, shame, and the like; for
•matters political; and for
•the mental operations of memory, composition and division, judgment and the rest,

just as much as for

•heat and cold, light, vegetative growth and the like.

But my method of interpretation ·differs from the common logic in one important respect; my method·, after the history has been prepared and set in order, concerns itself not only with •the movements and activities of the mind (as the common logic does) but also with •the nature of things ·outside the mind·. I guide the mind so that its way of engaging with any particular thing is always appropriate. That’s why my doctrine of interpretation contains many different instructions, fitting the discovery-method according to the quality and condition of the subject-matter of the inquiry.

128. ‘Do you want to pull down and destroy the philosophy, arts and sciences that are now practised?’ There ought to be no question about that. Far from wanting to destroy them, I am very willing to see them used, developed and honoured. I don’t want to get in the way of their •giving men something to dispute about, •supplying decoration for discourse, •providing the ‘experts’ with an income, and •facilitating civil life—acting, in short, like coins that have value because men agree to give it to them. Let me clear about this: what I am presenting won’t be much use for purposes such as those, since it can’t be brought within reach of the minds of the vulgar except ·indirectly·, through effects and works. My published writings, especially my Two Books on the Advancement of Learning, show well enough the sincerity of my declaration of friendly good will toward the accepted sciences, so I shan’t expend more words on that topic here. Meanwhile I give clear and constant warning that the methods now in use won’t lead to any great progress in the theoretical parts of the sciences, and won’t produce much in the way of applied-science results either.

129. All that remains for me to say are a few words about the excellence of the end in view. If I had said them earlier they might have seemed like mere prayers; but perhaps they’ll have greater weight now, when hopes have been created and unfair prejudices removed. I wouldn’t have said them even now if I had done the whole job myself, not calling on anyone else to help with the work, because ·words said in praise of the object of this exercise· might be taken as a proclamation of my own deserts. But ·I’m not going it alone·; I do want to energize others and kindle their zeal, so it is appropriate that I put men in mind of some things, ·even at the risk of seeming to boast·.

The making of great ·scientific· discoveries seems to have pride of place among human actions. That was the attitude of the ancients: they honoured the makers of discoveries as though they were gods, but didn’t go higher than demigods in their honours for those who did good service in the state (founders of cities and empires, legislators, saviours of their country from long endured evils, quellers of tyrannies, and the like). And if you think accurately about the two ·kinds of benefactor· you will see that the ancients were right about them. Why? (1) Because the benefits of ·scientific· discoveries can •extend to the whole of mankind, and can •last for all time, whereas civil benefits •apply only to particular places and •don’t last for very long.

(2) Also, improvements in civil matters usually bring violence and confusion with them, whereas ·scientific· discoveries bring delight, and confer benefits without causing harm or sorrow to anyone.

·Scientific· discoveries are like new creations, imitations of God’s works. . . . It seems to be worth noting that Solomon, the marvel of the world, though mighty in empire and in gold, in the magnificence of his works, his court, his household, his fleet, and the lustre of his name, didn’t glory in any of these, but pronounced that ‘It is the glory of God to conceal a thing; but the honour of kings is to search out a matter’ (Proverbs 25:2).

If you compare how men live in the most civilized provinces of Europe with how they live in the wildest and most barbarous areas of the American continent, you will think the difference is big enough—the difference in •the condition of the people in themselves as well as in •what conveniences and comforts they have available to them—to justify the saying that ‘man is a god to man’. And this difference doesn’t come from the Europeans’ having better soil, a better climate, or better physiques, but from the arts [see note on ‘art’ here].

Notice the vigour of discoveries, their power to generate consequences. This is nowhere more obvious than in three discoveries that the ancients didn’t know and whose origins (all quite recent) were obscure and humdrum. I am talking about the arts of •printing, •gunpowder, and •the nautical compass. These three have changed the whole aspect and state of things throughout the world—the first in literature, the second in warfare, the third in navigation—bringing about countless changes; so that there seems to have been no empire, no philosophical system, no star that has exerted greater power and influence in human affairs than these mechanical discoveries.

For my next point, I need to distinguish the three kinds— three levels, as it were—of human ambition. (1) Some people want to extend their power within their own country, which is a commonplace and inferior kind of ambition. (2) Some work to extend the power and dominion of their country in relation to mankind in general; this is certainly not as base as (1) is, but it is just as much a case of greed. (3) If a man tries to get mankind’s power and control over the universe off to a fresh start, and to extend it, hisambition (if it is ambition at all) is certainly more wholesome and noble ·than the other two·. Now—·this being the point I wanted to make·—man’s control over things depends wholly on the arts and sciences, for we can’t command nature except by obeying her.

A further point: it sometimes happens that •one particular discovery is so useful to mankind that the person who made it and thus put the whole human race into his debt is regarded as superhuman; so how much higher a thing it is to discover something through which •everything else can easily be discovered! ·Not that a discovery’s consequences are the main thing about it·. Light is useful in countless ways, enabling us to walk, practise our arts, read, recognize one another, and yet something that is finer and lovelier than all those uses of light is seeing light. Similarly, merely contemplating things as they are, without superstition or imposture, error or confusion, is in itself worthier than all the practical upshots of discoveries.

Final point: If anyone counts it against the arts and sciences that they can be debased for purposes of wickedness, luxury, and the like, don’t be influenced by that. The same can be said of all earthly goods: intelligence, courage, strength, beauty, wealth—even light! Just let the human race get back the right over nature that God gave to it, and give it scope; how it is put into practice will be governed by sound reason and true religion.

130. The time has come for me to present the art of interpreting nature—the art itself, ·not just remarks about the need for it, its virtues, and so on·. Although I think I have given true and most useful precepts in it, I don’t say that this art is absolutely necessary, implying that nothing could be done without it. In fact, I think that if

•men had ready at hand a sound history of nature and of experiments, •were thoroughly practised in it, and •imposed on themselves two rules: (1) set aside generally accepted opinions and notions, and (2) for a while keep your mind away from the highest and second-to-highest generalizations,

they would arrive at my form of interpretation sheerly through their own natural intelligence, with no help from any other rules or techniques. For interpretation is the true and natural work of the mind when it is freed from blockages. It is true, however, that it can all be done more readily and securely with help from my precepts.

And I don’t say, either, that my art of interpreting nature is complete so that nothing can be added to it. On the contrary: I am concerned with the mind not only in respect of its own capacities but also in respect of how it engages with things; so I have to think that the art of discovery can develop as more discoveries are made.

The next post in the sequence will be posted Thursday, October 24 at latest by 4:00pm PDT.


Reposting previously linked content on LW

18 октября, 2019 - 04:24
Published on October 18, 2019 1:24 AM UTC

This question might be a bit specific to me, but maybe it applies to others so I'll ask publicly so the answer becomes more visible to all.

Is there are a policy or what are your thoughts on posting content on LW that was previously the subject of a link post?

In my case most of these are from the dying days of the LW 1.0 era. In the last few months I've switched from linking to cross-posting from my blog as LW 2.0 has taken off and proven to be a good host for my content. I could update those old posts with the content just to mirror it over on LW, but that also feels a bit sad because if I'm going to do the (admittedly minor) formatting work necessary to bring the posts over and have them look nice, I'd also like it if they became more visible to folks, especially for that content that I think people would like to see but may have missed because it was linked at a time when LW was pretty inactive and so those old, updated posts wouldn't well reflect the engagement that might receive now. However, I can also see some danger in this as maybe someone could take this as license to abuse the policy and repost a lot of stuff from years ago to make it fresh again even when it did previously receive appropriate levels of engagement because it was being posted at a time when LW was more active.

This is obviously mostly a question for the moderators but other people's opinions are probably worthwhile as evidence for the mods to consider.

(On an unrelated note, I'd actually be pretty happy to move my blog to LW from Medium, which I know has been floated as a possible future feature, but I'd need some way to keep links working, which is thankfully eased by having a custom domain and not needing to redirect raw Medium articles, as well as a way for my domain to sit in front of my LW content but not all LW content.)


Random Thoughts on Predict-O-Matic

18 октября, 2019 - 02:39
Published on October 17, 2019 11:39 PM UTC

I'm going to be a bit more explicit about some ideas that appeared in The Parable of Predict-O-Matic. (If you don't want spoilers, read it first. Probably you should read it first anyway.)

[Note: while the ideas here are somewhat better than the ideas in the predict-o-matic story, they're equally rambling, without the crutch of the story to prop them up. As such, I expect readers to be less engaged. Unless you're especially interested in which character's remarks are true (or at least, which ones I stand by), this might be a post to skim; I don't think it has enough coherence that you need to read it start-to-finish.]

First, as I mentioned in Partial Agency, my main concern here isn't actually about building safe oracles or inner-aligned systems. My main concern is to understand what's going on. If we can build guaranteed-myopic systems, that's good for some purposes. If we can build guaranteed-non-myopic systems, that's good for other purposes. The story largely frames it as a back-and-forth about whether things will be OK / whether there will be terrible consequences; but my focus was on the more specific questions about the behavior of the system.

Second, I'm not trying to confidently stand behind any of the character's views on what will happen. The ending was partly intended to be "and no one got it right, because this stuff is very complicated". I'm very uncertain about all of this. Part of the reason why it was so much easier to write the post as a story was that I could have characters confidently explain views without worrying about adding all the relevant caveats.

Inductive Bias

Evan Hubinger pointed out to me that all the characters are talking about asymptotic performance, and ignoring inductive bias. Inner optimizers might emerge due to the inductive bias of the system. I agree; in my mind, the ending was a bit of a hat tip to this, although I hinted at gradient hacking rather than inductive bias in the actual text.

On the other hand, "inductive bias" is a complicated object when you're talking about a system which isn't 100% Bayesian.

  • You often represent inductive bias through regularization techniques which introduce incentives pulling toward 'simpler' models. This means we're back in the territory of incentives and convergence.
  • So, to talk about what a learning algorithm really does, we have to also think of the initialization and search procedure as part of the inductive bias. This makes inductive bias altogether a fairly complicated object.
Explicit Fixed-Point Selection

The very first conversation involved the intern arguing that there would be multiple valid fixed-points of prediction, and Predict-O-Matic would have to choose between them somehow.

Explicitly modeling fixed points and choosing between them is a feature of the logical induction algorithm. This feature allows us to select the best one according to some criterion, as is leveraged in When Wishful Thinking Works. As discussed later in the conversation with the mathematician, this is atypical of supervised learning algorithms. What logical induction does is very expensive: it solves a computationally difficult fixed-point finding problem (by searching exhaustively).

Other algorithms are not really "choosing a fixed point somehow". They're typically failing to guarantee a fixed point. The mathematician hinted at this by describing how algorithms would not necessarily converge to a self-fulfilling prophecy; they could just as easily go in circles or wander around randomly forever.

Think of it like fashion. Sometimes, putting a trend into common knowledge will lock it in; this was true about neck ties in business for a long time. In other instances, the popularity of a fashion trend will actually work against it, a fashion statement being ineffective if it's overdone.

So, keep in mind that different learning procedures will relate to this aspect of the problem in different ways.

Reward vs Prediction Error

The economist first compared the learning algorithm to decision markets, then later, decided prediction markets were a better analogy.

The mathematician contrasted the learning algorithm to reinforcement learning, pointing out that Predict-O-Matic always adjusted outputs to be more like historical observations, whereas reinforcement learning would more strategically optimize reward.

Both of these point at a distinction between learning general decision-making and something much narrower and much more epistemic in character. As I see it, the critical idea is that (1) the system gets information about what it should have output; (2) the learning update moves toward a modified system which would have output that. This is quite different from reinforcement learning.

In a recent post, Wei Dai mentions a similar distinction (italics added by me):

Supervised training - This is safer than reinforcement learning because we don't have to worry about reward hacking (i.e., reward gaming and reward tampering), and it eliminates the problem of self-confirming predictions (which can be seen as a form of reward hacking). In other words, if the only thing that ever sees the Oracle's output during a training episode is an automated system that computes the Oracle's reward/loss, and that system is secure because it's just computing a simple distance metric (comparing the Oracle's output to the training label), then reward hacking and self-confirming predictions can't happen.

There are several things going on here, but I think Wei is trying to point at something similar to the distinction I'm thinking of. It's quite tempting to call it "supervised learning", because you get a signal telling you what you should have done. However, it's a bit fuzzy, because this also encompasses things normally called "unsupervised learning": the supervised/unsupervised distinction is often explained as modeling .mjx-chtml {display: inline-block; line-height: 0; text-indent: 0; text-align: left; text-transform: none; font-style: normal; font-weight: normal; font-size: 100%; font-size-adjust: none; letter-spacing: normal; word-wrap: normal; word-spacing: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0; min-height: 0; border: 0; margin: 0; padding: 1px 0} .MJXc-display {display: block; text-align: center; margin: 1em 0; padding: 0} .mjx-chtml[tabindex]:focus, body :focus .mjx-chtml[tabindex] {display: inline-table} .mjx-full-width {text-align: center; display: table-cell!important; width: 10000em} .mjx-math {display: inline-block; border-collapse: separate; border-spacing: 0} .mjx-math * {display: inline-block; -webkit-box-sizing: content-box!important; -moz-box-sizing: content-box!important; box-sizing: content-box!important; text-align: left} .mjx-numerator {display: block; text-align: center} .mjx-denominator {display: block; text-align: center} .MJXc-stacked {height: 0; position: relative} .MJXc-stacked > * {position: absolute} .MJXc-bevelled > * {display: inline-block} .mjx-stack {display: inline-block} .mjx-op {display: block} .mjx-under {display: table-cell} .mjx-over {display: block} .mjx-over > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-under > * {padding-left: 0px!important; padding-right: 0px!important} .mjx-stack > .mjx-sup {display: block} .mjx-stack > .mjx-sub {display: block} .mjx-prestack > .mjx-presup {display: block} .mjx-prestack > .mjx-presub {display: block} .mjx-delim-h > .mjx-char {display: inline-block} .mjx-surd {vertical-align: top} .mjx-mphantom * {visibility: hidden} .mjx-merror {background-color: #FFFF88; color: #CC0000; border: 1px solid #CC0000; padding: 2px 3px; font-style: normal; font-size: 90%} .mjx-annotation-xml {line-height: normal} .mjx-menclose > svg {fill: none; stroke: currentColor} .mjx-mtr {display: table-row} .mjx-mlabeledtr {display: table-row} .mjx-mtd {display: table-cell; text-align: center} .mjx-label {display: table-row} .mjx-box {display: inline-block} .mjx-block {display: block} .mjx-span {display: inline} .mjx-char {display: block; white-space: pre} .mjx-itable {display: inline-table; width: auto} .mjx-row {display: table-row} .mjx-cell {display: table-cell} .mjx-table {display: table; width: 100%} .mjx-line {display: block; height: 0} .mjx-strut {width: 0; padding-top: 1em} .mjx-vsize {width: 0} .MJXc-space1 {margin-left: .167em} .MJXc-space2 {margin-left: .222em} .MJXc-space3 {margin-left: .278em} .mjx-test.mjx-test-display {display: table!important} .mjx-test.mjx-test-inline {display: inline!important; margin-right: -1px} .mjx-test.mjx-test-default {display: block!important; clear: both} .mjx-ex-box {display: inline-block!important; position: absolute; overflow: hidden; min-height: 0; max-height: none; padding: 0; border: 0; margin: 0; width: 1px; height: 60ex} .mjx-test-inline .mjx-left-box {display: inline-block; width: 0; float: left} .mjx-test-inline .mjx-right-box {display: inline-block; width: 0; float: right} .mjx-test-display .mjx-right-box {display: table-cell!important; width: 10000em!important; min-width: 0; max-width: none; padding: 0; border: 0; margin: 0} .MJXc-TeX-unknown-R {font-family: monospace; font-style: normal; font-weight: normal} .MJXc-TeX-unknown-I {font-family: monospace; font-style: italic; font-weight: normal} .MJXc-TeX-unknown-B {font-family: monospace; font-style: normal; font-weight: bold} .MJXc-TeX-unknown-BI {font-family: monospace; font-style: italic; font-weight: bold} .MJXc-TeX-ams-R {font-family: MJXc-TeX-ams-R,MJXc-TeX-ams-Rw} .MJXc-TeX-cal-B {font-family: MJXc-TeX-cal-B,MJXc-TeX-cal-Bx,MJXc-TeX-cal-Bw} .MJXc-TeX-frak-R {font-family: MJXc-TeX-frak-R,MJXc-TeX-frak-Rw} .MJXc-TeX-frak-B {font-family: MJXc-TeX-frak-B,MJXc-TeX-frak-Bx,MJXc-TeX-frak-Bw} .MJXc-TeX-math-BI {font-family: MJXc-TeX-math-BI,MJXc-TeX-math-BIx,MJXc-TeX-math-BIw} .MJXc-TeX-sans-R {font-family: MJXc-TeX-sans-R,MJXc-TeX-sans-Rw} .MJXc-TeX-sans-B {font-family: MJXc-TeX-sans-B,MJXc-TeX-sans-Bx,MJXc-TeX-sans-Bw} .MJXc-TeX-sans-I {font-family: MJXc-TeX-sans-I,MJXc-TeX-sans-Ix,MJXc-TeX-sans-Iw} .MJXc-TeX-script-R {font-family: MJXc-TeX-script-R,MJXc-TeX-script-Rw} .MJXc-TeX-type-R {font-family: MJXc-TeX-type-R,MJXc-TeX-type-Rw} .MJXc-TeX-cal-R {font-family: MJXc-TeX-cal-R,MJXc-TeX-cal-Rw} .MJXc-TeX-main-B {font-family: MJXc-TeX-main-B,MJXc-TeX-main-Bx,MJXc-TeX-main-Bw} .MJXc-TeX-main-I {font-family: MJXc-TeX-main-I,MJXc-TeX-main-Ix,MJXc-TeX-main-Iw} .MJXc-TeX-main-R {font-family: MJXc-TeX-main-R,MJXc-TeX-main-Rw} .MJXc-TeX-math-I {font-family: MJXc-TeX-math-I,MJXc-TeX-math-Ix,MJXc-TeX-math-Iw} .MJXc-TeX-size1-R {font-family: MJXc-TeX-size1-R,MJXc-TeX-size1-Rw} .MJXc-TeX-size2-R {font-family: MJXc-TeX-size2-R,MJXc-TeX-size2-Rw} .MJXc-TeX-size3-R {font-family: MJXc-TeX-size3-R,MJXc-TeX-size3-Rw} .MJXc-TeX-size4-R {font-family: MJXc-TeX-size4-R,MJXc-TeX-size4-Rw} .MJXc-TeX-vec-R {font-family: MJXc-TeX-vec-R,MJXc-TeX-vec-Rw} .MJXc-TeX-vec-B {font-family: MJXc-TeX-vec-B,MJXc-TeX-vec-Bx,MJXc-TeX-vec-Bw} @font-face {font-family: MJXc-TeX-ams-R; src: local('MathJax_AMS'), local('MathJax_AMS-Regular')} @font-face {font-family: MJXc-TeX-ams-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_AMS-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_AMS-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_AMS-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-B; src: local('MathJax_Caligraphic Bold'), local('MathJax_Caligraphic-Bold')} @font-face {font-family: MJXc-TeX-cal-Bx; src: local('MathJax_Caligraphic'); font-weight: bold} @font-face {font-family: MJXc-TeX-cal-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-R; src: local('MathJax_Fraktur'), local('MathJax_Fraktur-Regular')} @font-face {font-family: MJXc-TeX-frak-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-frak-B; src: local('MathJax_Fraktur Bold'), local('MathJax_Fraktur-Bold')} @font-face {font-family: MJXc-TeX-frak-Bx; src: local('MathJax_Fraktur'); font-weight: bold} @font-face {font-family: MJXc-TeX-frak-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Fraktur-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Fraktur-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Fraktur-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-BI; src: local('MathJax_Math BoldItalic'), local('MathJax_Math-BoldItalic')} @font-face {font-family: MJXc-TeX-math-BIx; src: local('MathJax_Math'); font-weight: bold; font-style: italic} @font-face {font-family: MJXc-TeX-math-BIw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-BoldItalic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-BoldItalic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-BoldItalic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-R; src: local('MathJax_SansSerif'), local('MathJax_SansSerif-Regular')} @font-face {font-family: MJXc-TeX-sans-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-B; src: local('MathJax_SansSerif Bold'), local('MathJax_SansSerif-Bold')} @font-face {font-family: MJXc-TeX-sans-Bx; src: local('MathJax_SansSerif'); font-weight: bold} @font-face {font-family: MJXc-TeX-sans-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-sans-I; src: local('MathJax_SansSerif Italic'), local('MathJax_SansSerif-Italic')} @font-face {font-family: MJXc-TeX-sans-Ix; src: local('MathJax_SansSerif'); font-style: italic} @font-face {font-family: MJXc-TeX-sans-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_SansSerif-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_SansSerif-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_SansSerif-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-script-R; src: local('MathJax_Script'), local('MathJax_Script-Regular')} @font-face {font-family: MJXc-TeX-script-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Script-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Script-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Script-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-type-R; src: local('MathJax_Typewriter'), local('MathJax_Typewriter-Regular')} @font-face {font-family: MJXc-TeX-type-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Typewriter-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Typewriter-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Typewriter-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-cal-R; src: local('MathJax_Caligraphic'), local('MathJax_Caligraphic-Regular')} @font-face {font-family: MJXc-TeX-cal-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Caligraphic-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Caligraphic-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Caligraphic-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-B; src: local('MathJax_Main Bold'), local('MathJax_Main-Bold')} @font-face {font-family: MJXc-TeX-main-Bx; src: local('MathJax_Main'); font-weight: bold} @font-face {font-family: MJXc-TeX-main-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Bold.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-I; src: local('MathJax_Main Italic'), local('MathJax_Main-Italic')} @font-face {font-family: MJXc-TeX-main-Ix; src: local('MathJax_Main'); font-style: italic} @font-face {font-family: MJXc-TeX-main-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-main-R; src: local('MathJax_Main'), local('MathJax_Main-Regular')} @font-face {font-family: MJXc-TeX-main-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Main-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Main-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Main-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-math-I; src: local('MathJax_Math Italic'), local('MathJax_Math-Italic')} @font-face {font-family: MJXc-TeX-math-Ix; src: local('MathJax_Math'); font-style: italic} @font-face {font-family: MJXc-TeX-math-Iw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Math-Italic.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Math-Italic.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Math-Italic.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size1-R; src: local('MathJax_Size1'), local('MathJax_Size1-Regular')} @font-face {font-family: MJXc-TeX-size1-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size1-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size1-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size1-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size2-R; src: local('MathJax_Size2'), local('MathJax_Size2-Regular')} @font-face {font-family: MJXc-TeX-size2-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size2-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size2-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size2-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size3-R; src: local('MathJax_Size3'), local('MathJax_Size3-Regular')} @font-face {font-family: MJXc-TeX-size3-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size3-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size3-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size3-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-size4-R; src: local('MathJax_Size4'), local('MathJax_Size4-Regular')} @font-face {font-family: MJXc-TeX-size4-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Size4-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Size4-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Size4-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-R; src: local('MathJax_Vector'), local('MathJax_Vector-Regular')} @font-face {font-family: MJXc-TeX-vec-Rw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Regular.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Regular.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Regular.otf') format('opentype')} @font-face {font-family: MJXc-TeX-vec-B; src: local('MathJax_Vector Bold'), local('MathJax_Vector-Bold')} @font-face {font-family: MJXc-TeX-vec-Bx; src: local('MathJax_Vector'); font-weight: bold} @font-face {font-family: MJXc-TeX-vec-Bw; src /*1*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/eot/MathJax_Vector-Bold.eot'); src /*2*/: url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/woff/MathJax_Vector-Bold.woff') format('woff'), url('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/fonts/HTML-CSS/TeX/otf/MathJax_Vector-Bold.otf') format('opentype')} P(x|y) vs P(x). Wikipedia:

It could be contrasted with supervised learning by saying that whereas supervised learning intends to infer a conditional probability distribution pX(x|y) conditioned on the label y of input data; unsupervised learning intends to infer an a priori probability distribution pX(x).

But many (not all) unsupervised algorithms still have the critical features we're interested in! Predicting x without any context information y to help still involves (1) getting feedback on what we "should have" expected, and (2) updating to a configuration which would have more expected that. We simply can't expect the predictions to be as focused, given the absence of contextual information to help. But that just means it's a prediction task on which we tend to expect lower accuracy.

I'm somewhat happy referring to this category as imitative learning. This includes supervised learning, unsupervised learning so long as it's generative (but not otherwise), and imitation learning (a paradigm which achieves similar ends as inverse reinforcement learning). Homever, the terminological overlap with 'imitation learning' is rather terrible, so I'm open to other suggestions.

It seems to me that this is a critical distinction for the myopia discussion. I hope to say more about it in future posts.

Maximizing Entropy?

The discussion of prediction markets toward the end was rather loose, in that the economist didn't deal with a lot of the other points which had been made throughout, and just threw a new model out there.

  • The mechanism of manipulation is left quite vague. In an assassination market, there are all kinds of side-channels which agents can use to accomplish their goals. But the rest of the essay had only considered the influence which Predict-O-Matic has by virtue of the predictions it makes. When writing this part, I was actually imagining side-channels, such as exploiting bugs to communicate by other means.
  • It's not clear whether the market is supposed to be myopic or non-myopic in this discussion. The argument for overall myopia was the economist's initial decision-market model, which is being cast aside here. Are manipulators expected to only throw off individual predictions to reap reward, or are they expected to implement cross-prediction strategies?
  • The disanalogy between prediction markets and a local search like gradient descent is swept under the rug. A prediction market has many agents interacting with it, so that one agent can pick up money from another by out-smarting it. Do conclusions from that setting carry over to single-hypothesis learning? It isn't clear.

Isnasene interpreted the first point by imagining that the mechanism of manipulation is still through selection of fixed points:

In the same way that self-fulfilling predictions are good for prediction strategies because they enhance accuracy of the strategy in question, self-fulfilling predictions that seem generally surprising to outside observers are even better because they lower the accuracy of competing strategies. The established prediction strategy thus systematically causes the kinds of events in the world that no other method could predict to further establish itself.

This is compatible with the assumption of myopia; we might imagine that the system still can't manipulate events through actual bad predictions, because those strategies will be undercut. Therefore, the manipulation is restricted to selecting fixed-points which are surprising.

However, there are three problems with this:

  • The "undercutting" argument relies on an assumption that there are enough different strategies considered, so that the one which undercuts the non-myopic strategy is eventually found. The argument for strategically increasing entropy relies on the reverse assumption! There's little profit in increasing entropy if others can predict that you'll do this and cash in.
    • (We might still see both effects at different times in training, however, and therefore expect major consequences of both effects.)
  • As previously discussed, we don't actually expect fixed points in general. So we have to ask whether the entropy-increasing incentive is significant more generally (unless we're specifically thinking about the logical induction algorithm or other algorithms which get you a fixed point).
  • We still have not dealt with the disanalogy between prediction markets and local-search-based learning.

So, it seems the actual situation is more complicated, and I'm not yet sure how to think about this.

'Local Search'; selection vs control

I used the term 'local search' to describe the application of gradient-descent-like updates to reduce prediction error. I have some conceptual/terminological issues with this.

Calling this 'local search' invokes the mental image of a well-defined gradient landscape which we are taking steps on, to further optimize some function. But this is the wrong mental image. The mental image is one of selection, when we're in a control setting (in my terminology). We are not making an iid assumption. We are not getting samples from a stationary but stochastic loss function, as in stochastic gradient descent.

If 'local search' were an appropriate descriptor for gradient-descent here, would it also be an appropriate descriptor for Bayesian updates? There's a tendency to think of Bayesian learning as trying to find one good hypothesis by tracking how well all of them do (which sounds like a global search), but we needn't think of it this way. The "right answer" can be a mixture over hypotheses. We can think of a Bayesian update as incrementally improving our mixture. But thinking of Bayesian updates as local search seems wrong. (So does thinking of them as global search.)

This is online learning. A gradient-descent step represents a prediction that the future will be like the past in some relevant sense, in spite of potential non-stationarity. It is not a guaranteed improvement, even in expectation -- as it would be in offline stochastic gradient descent with sufficiently small step size.

Moreover, step size becomes a more significant problem. In offline gradient descent, selecting too small a step size only means that you have to make many more steps to get where you're going. It's "just a matter of computing power". In online learning, it's a more serious problem; we want to make the appropriate-sized update to new data.

I realize there are more ways of dealing with this than tuning step size; we don't necessarily update to data by making a single gradient step. But there are problems of principal here.

What's gradient descent without a fitness landscape?

Simply put, gradient descent is a search concept, not a learning concept. I want to be able to think of it more directly as a learning concept. I want to be able to think of it as an "update", and use terminology which points out the similarity to Bayesian updates.

The Duality Remark

Vanessa asked about this passage:

The engineer was worse: they were arguing that Predict-O-Matic might maximize prediction error! Some kind of duality principle. Minimizing in one direction means maximizing in the other direction. Whatever that means.

I responded:

[I]t was a speculative conjecture which I thought of while writing.The idea is that incentivizing agents to lower the error of your predictions (as in a prediction market) looks exactly like incentivizing them to "create" information (find ways of making the world more chaotic), and this is no coincidence. So perhaps there's a more general principle behind it, where trying to incentivize minimization of f(x,y) only through channel x (eg, only by improving predictions) results in an incentive to maximize f through y, under some additional assumptions. Maybe there is a connection to optimization duality in there.In terms of the fictional cannon, I think of it as the engineer trying to convince the boss by simplifying things and making wild but impressive sounding conjectures. :)

If you have an outer optimizer which is trying to maximize f(x,y) through x while being indifferent about y, it seems sensible to suppose that inner optimizers will want to change y to throw things off, particularly if they can get credit for then correcting x to be optimal for the new y. If so, then inner optimizers will generally be seeking to find y-values which make the current x a comparatively bad choice. So this argument does not establish an incentive to choose y which makes all choices of x poor.

In a log-loss setting, this would translate to an incentive to make observations surprising (for the current expectations), rather than a direct incentive to make outcomes maximum-entropy. However, iteration of this would push toward maximum entropy. Or, logical-induction-style fixed-point selection could push directly to maximum entropy.

This would be a nice example of partial agency. The system is strategically influencing x and y so as to maximize f through channel x, while minimizing through channel y. What does this mean? This does not correspond to a coherent objective function at all! The system is 'learning a game-theoretic equilibrium' -- which is to say, it's learning to fight with itself, rather than optimize.

There are two different ways we can think about this. One way is to say there's an inner alignment problem here: the system learns to do something which doesn't fit any objective, so it's sort of trivially misaligned with whatever the outer objective was supposed to be. But what if we wanted this? We can think of games as a kind of generalized objective, legitimizing this behavior.

To make things even more confusing, if the only channel by which Predict-O-Matic can influence the world is via the predictions which get output, then... doesn't x=y? x represents the 'legitimate' channel whereby predictions get combined with (fixed) observations to yield a score. y represents the 'manipulative' channel, where predictions can influence the world and thus modify observations. But the two causal pathways have one bottleneck which the system has to act through, namely, the predictions made.

In any case, I don't particularly trust any of the reasoning above.

  • I didn't clarify my assumptions. What does it mean for the outer optimizer to maximize f(x,y) through x while being indifferent about y? It's quite plausible that some versions of that will incentivise inner optimizers which optimize f taking advantage of both channels, rather than the contradictory behavior conjectured above
  • I anthropomorphized the inner optimizers. In particular, I did not specify or reason about details of the learning procedure.
    • This sort of assumes they'll tend to act like full agents rather than partial agents, while yielding a conclusion which suggests otherwise.
  • This caused me to speak in terms of a fixed optimization problem, rather than a learning process. Optimizing f isn't really one thing -- f is a loss function which is applied repeatedly in order to learn. The real problem facing inner optimizers is an iterated game involving a complex world. I can only think of them trying to game a single f if I establish that they're myopic; otherwise I should think of them trying to deal with a sequence of instances.

So, I'm still unsure how to think about all this.


The best of the www, in my opinion

17 октября, 2019 - 22:10
Published on October 17, 2019 3:14 PM UTC

Below I will present a (small but qualitative ) list of those that I think are some of the best sites/blog that a human being can find on the world wide web.

The main criterion I used to draw up the list was to consider how the websites promote the dissemination of knowledge among people and how, over the course of time, they have helped me both with regards to work and in terms of intellectual self-formation. The order in which they are listed is not to be considered restrictive ( except perhaps for the first two ).

Please feel free to criticize the catalog (as long as the criticisms are rational and constructive) and to expand it in the comments.

1) Stack Exchange Concentrator ( https://stackexchange.com/sites )

2) ArXiv e-Print archive ( https://arxiv.org/ )

3) GitHub ( https://github.com/ )

4) Reddit - the front page of the internet ( https://www.reddit.com/ )

5) LessWrong ( https://www.lesswrong.com/ )

6) Shtetl-Optimized ( https://www.scottaaronson.com/blog/ )

7) Slate Star Codex ( https://slatestarcodex.com/author/admin/ )

8) YouTube ( https://www.youtube.com/?hl=it&gl=IT )

9) TED ( https://www.ted.com/#/ )

10) Standford Encyclopedia of Philosophy ( https://plato.stanford.edu/ )

11) Google Scholar ( https://scholar.google.com/ )


Festival Stats 2019

17 октября, 2019 - 14:00
Published on October 17, 2019 11:00 AM UTC

Each year in the fall, since 2014, I've been sharing counts of how many weekend and festival gigs different bands and callers have been doing. Over the course of the year I collect bookings in a big spreadsheet, trying to check each dance weekend's website about a month before the event when they're likely to have a their performers listed.

I got into this as kind of a "market research" thing for the Free Raisins: how many weekends are there? What are the bands that are getting booked a lot, so I can go see what they sound like? Since then I've played a lot more of these events, and have a better handle on it, but I've kept the list because having a big pile of data means I have what I need for posts where I look at the gender distribution of musicians or callers, make heatmaps, or track when in the year these events tend to be (which is, not in December, and hence this post coming out in October and not November).

It's also fun to see who's playing a lot each year, and the waves of bands ramping up and down. One thing you can't get from this data, though, is whether this is driven primarily by band interest (reaching out to weekends to get booked, cutting back to focus on other things) or organizer interest (booking bands who seem new and hot, avoiding bands that aren't novel). Here's what this looks like for the ten most-booked bands 2014-2019:

It's not that clear, but if you know what you're looking for you can see some of what's going on:

  • Great Bear deciding to retire in 2018 while still very popular.
  • Buddy System ramping up, first playing techno-contra slots and growing from there into an excellent acoustic duo.
  • Wild Asparagus, by far the longest-running band in this group, maintaining a very steady run.
  • The Free Raisins playing fewer gigs after I had kids, and then more fewer gigs because we're not touring anymore.
  • Elixir first cutting back, and then playing more again with subs

I'm sure there are more stories that this chart could accompany, but I don't know what's going on with the other bands well enough to tell them. (And I'm not sure I have the above ones entirely right either.)

Anyway, here's the raw data for this year:


Buddy System 12 Toss the Possum 10 Pete's Posse 9 Gaslight Tinkers 7 Nova 7 Rushfest 7 Wake up Robin 7 Elixir 6 Cloud Ten 5 Dam Beavers 5 Drive Train 5 Organic Family Band 5 Stringrays 5 Wild Asparagus 5 Eloise and Company 4 Free Raisins 4 Maivish 4 Mean Lids 4 Polaris 4 Ripples 4 Riptide 4 Syncopaths 4 Turnip the Beet 4 Chimney Swift 3 Genticorum 3 Sassafras Stomp 3 Stomp Rocket 3 Campaign for Reel Time 2 Center Street 2 Faux Paws 2 Figments 2 Hotpoint 2 Joy Compass 2 Joyride 2 Latter Day Lizards 2 Meadowhawk 2 Moving Violations 2 Playing with Fyre 2 Steam 2 Supertrad 2 Tempest 2 Tidal Wave 2


Will Mentor 12 Bob Isaacs 11 Seth Tepfer 11 Lindsey Dono 10 Cis Hinkle 9 Gaye Fifer 9 Alex Deis-Lauby 8 Beth Molaro 8 Emily Rush 8 George Marshall 7 Dana Parkinson 6 Lisa Greenleaf 6 Nils Fredland 6 Sarah Van Norstrand 6 Dugan Murphy 5 Frannie Marr 5 Steve Zakon-Anderson 5 Diane Silver 4 Susan Kevra 4 Terry Doyle 4 Carol Ormand 3 Chet Gray 3 Jacqui Grennan 3 Luke Donforth 3 Michael Karcher 3 Andy Shore 2 Angela DeCarlis 2 Bradley Smith 2 David Kaynor 2 Dela Murphy 2 Dereck Kalish 2 Janine Smith 2 Jesse Edgarton 2 JoLaine Jones-Pokorney 2 Maia McCormick 2 Quena Crain 2 Susan Petrick 2 Ted Hodapp 2 Wendy Graham 2 Yoyo Zhou 2

Comment via: facebook


The Dualist Predict-O-Matic ($100 prize)

17 октября, 2019 - 09:45
Published on October 17, 2019 6:45 AM UTC

This is a response to Abram's The Parable of Predict-O-Matic, but you probably don't need to read Abram's post to understand mine. While writing this, I thought of a way in which I think things could wrong with Predict-O-Matic, which I plan to post in about a week. I'm offering a $100 prize to the first commenter who's able to explain how things might go wrong in a sufficiently crisp way before I make my follow-up post.


Currently, machine learning algorithms are essentially "Cartesian dualists" when it comes to themselves and their environment. (Not a philosophy major -- let me know if I'm using that term incorrectly. But what I mean to say is...) If I give a machine learning algorithm some data about itself as training data, there's no self-awareness there--it just chugs along looking for patterns like it would for any other problem. I think it's a reasonable guess that our algorithms will continue to have this "no self-awareness" property as they become more and more advanced. At the very least, this "business as usual" scenario seems worth analyzing in depth.

If dualism holds for Abram's prediction AI, the "Predict-O-Matic", its world model may happen to include this thing called the Predict-O-Matic which seems to make accurate predictions -- but it's not special in any way and isn't being modeled any differently than anything else in the world. Again, I think this is a pretty reasonable guess for the Predict-O-Matic's default behavior. I suspect other behavior would require special code which attempts to pinpoint the Predict-O-Matic in its own world model and give it special treatment (an "ego").

Let's suppose the Predict-O-Matic follows a "recursive uncertainty decomposition" strategy for making predictions about the world. It models the entire world in rather fuzzy resolution, mostly to know what's important for any given prediction. If some aspect of the world appears especially relevant to a prediction it's trying to make, it "zooms in" and tries to model that thing in higher resolution. And if some part of that thing seems especially relevant, it zooms in further on that part. Etc.

Now suppose the Predict-O-Matic is trying to make a prediction, and its "recursive uncertainty decomposition" algorithms say the next prediction made by this Predict-O-Matic thing which happens to occupy its world model appears especially relevant! What then?

At this point, the Predict-O-Matic has stepped into a hall of mirrors. To predict the next prediction made by the Predict-O-Matic in its world model, the Predict-O-Matic needs to run an internal simulation of that Predict-O-Matic. But as it runs that simulation, it finds that simulation kicking off another Predict-O-Matic simulation in the simulated Predict-O-Matic's world model! Etc, etc.

So if the Predict-O-Matic is implemented naively, the result could just be an infinite recurse. Not useful, but not necessarily dangerous either.

Let's suppose the Predict-O-Matic has a non-naive implementation and something prevents this infinite recurse. For example, there's a monitor process that notices when a model is eating up a lot of computation without delivering useful results, and replaces that model with one which is lower-resolution. Or maybe the Predict-O-Matic does have a naive implementation, but it doesn't have enough data about itself to model itself in much detail, so it ends up using a low-resolution model.

One possibility is that it's able to find a useful outside view model such as "the Predict-O-Matic has a history of making negative self-fulfilling prophecies". This could lead to the Predict-O-Matic making a negative prophecy ("the Predict-O-Matic will continue to make negative prophecies which result in terrible outcomes"), but this prophecy wouldn't be selected for being self-fulfilling. And we might usefully ask the Predict-O-Matic whether the terrible self-fulfilling prophecies will continue conditional on us taking Action A.

Answering a Question by Having the Answer

If you aren't already convinced, here's another explanation for why I don't think the Predict-O-Matic will make self-fulfilling prophecies by default.

In Abram's story, the engineer says: "The answer to a question isn't really separate from the expected observation. So 'probability of observation depending on that prediction' would translate to 'probability of an event given that event', which just has to be one."

In other words, if the Predict-O-Matic knows it will predict P = A, it assigns probability 1 to the proposition that it will predict P = A.

I contend that Predict-O-Matic doesn't know it will predict P = A at the relevant time. It would require time travel -- to know whether it will predict P = A, it will have to have made a prediction already, and but it's still formulating its prediction as it thinks about what it will predict.

More details: Let's taboo "Predict-O-Matic" and instead talk about a "predictive model" and "input data". The trick is to avoid including the output of the predictive model in the model's input data. This isn't possible the first time we make a prediction because it would require time travel -- so as a practical matter, we don't want to re-run the model a second time with the prediction from its first run included in the input data. Let's say the dataset is kept completely static during prediction. (I offer no guarantees in the case where observational data about the model's prediction process is being used to inform the model while it makes a prediction!)

To clarify further, let's consider a non-Predict-O-Matic scenario where issues do crop up. Suppose I'm a big shot stock analyst. I think Acme Corp's stock is overvalued and will continue to be overvalued in one month's time. But before announcing my prediction, I do a sanity check. I notice that if I announce my opinion, that could cause investors to dump Acme, and Acme will likely no longer be overvalued in one month's time. So I veto the column I was planning to write on Acme, and instead search for a column c I can write such that c is a fixed point for the world w -- w(c) = c -- even when the world is given my column as an input, what I predicted in my column still comes true.

Note again that the sanity check which leads to a search for a fixed point doesn't happen by default -- it requires some extra functionality, beyond what's required for naive prediction, to implement. The Predict-O-Matic doesn't care about looking bad, and there's nothing contradictory about it predicting that it won't make the very prediction it makes, or something like that. Predictive model, meet input data. That's what it does.

Open Questions

This is a section for half-baked thoughts that could grow into a counterargument for what I wrote above.

  • What's going on when you try to model yourself thinking about the answer to this question? (Why is this question so hard to think about? Maybe my brain has a mechanism to prevent infinite recurse? Tangent: I wonder if this is evidence of a mechanism that tries to prevent my brain from making premature predictions by observing data about my own predictive process while trying to make predictions? Otherwise maybe there could be a cascade of updates where noticing that I've become more sure of something makes me even more sure of it, etc.) Anyway, I think it's important to understand if my brain does something "in practice" which differs from what I've outlined here, some kind of method for collapsing recursion that a sufficiently advanced Predict-O-Matic might use.

  • What if the Predict-O-Matic assigns some credence to the idea that it's "agentic" in nature? Then what if the Predict-O-Matic assigns some credence to the idea that the simulated version of itself could assign credence to the idea that it's in a simulation? (I think this is just a classic daemon but maybe it differs in important ways?)

  • In ML, the predictive model isn't trying to maximize its own accuracy--that's what the training algorithm tries to do. The predictive model doesn't seem like an optimizer even in the "mathematical optimization" sense of the world optimization (is "mesa-optimizer" an appropriate term? In this case, I think we're glad it's a mesa-optimizer?) What if the Predict-O-Matic sometimes runs a training algorithm to update its model? How does that change things?

  • What if time travel actually is possible and we just haven't discovered it yet?

Prize Details

Again, $100 prize for the first comment which crisply explains something that could wrong with Predict-O-Matic. Contest ends when I publish my follow-up -- probably next Wednesday the 23rd. I do have at least one answer in mind but I'm hoping you'll come up with something I haven't thought of. However, if I'm not convinced your thing would be a problem, I can't promise you a prize. No comment on whether "Open Questions" are related to the answer I have in mind.


Almost Infinite Positions & Body Bubbles. Postures & Poses.

17 октября, 2019 - 08:47
Published on October 17, 2019 5:47 AM UTC

This is part 3 of:

Base-Line Hypothesis of Human Health and Movement.

Pt 1. Base-Line & the Main Muscles of Movement. Pt 2. Conscious Proprioception.

From BLH Part 1:

A full range of natural movement is what the body should be able to do ...... an almost infinite number of potential positions.
An Almost Infinite Number of Potential Positions.
  • Consider how many moving joints there are in the human body.

I'm currently not willing to commit to a figure, several suggestions appear here on Quora.

  • And consider the range of motion of each of those joints.

What-is-normal-range-of-motion-in-a-joint. Not an exhaustive list, for example there is no data concerning the spinal column.

When describing the body's 'exact position' (see below), any movement - the slightest flexion, extension, abduction, adduction or rotation depending on the joint's capability - creates a novel position.

Every position of any joint can be combined with every potential position of all the other joints (unless specific physical limitations exist). That's a lot of multiplying to calculate all possible unique combinations.

And to what degree is a difference in position measured?

      • 1°? 0.1°? 0.01°? (decimalisation - or work with degrees, minutes, seconds?)
      • The width of a muscle fibre (myocyte)? Skeletal muscle cells: diameter 10 to 100 µm (micrometre) source.
      • The width of a collagen fibre? Collagen fibres: diameter 1 to 20 μm source.
      • The width of a collagen fibril? A collagen molecule? (More on collagen and other fibres in later posts.)

Even when the skeleton is stationary many other positions can be created by the action of muscles, especially the tongue, facial and ocular muscles.

The influence of gravity means that any position should be considered relative to the ground. Are there less positions in space?

Is the number of possible positions of the body calculable? I've no idea what computer power can do these days, but I feel OK saying:

'The body has an almost infinite number of potential positions'
The Body's Bubble.

Imagine the body's extremities tracing out a perimeter - a bubble - as the body moves through its almost infinite number of positions.

The fingers, toes and head can only reach so far apart.

The bubble is a finite space.

I've borrowed an illustration!

Body Bubble: A finite space containing the almost infinite number of potential positions of the body.

The largest possible bubble can be created when the body has a full range of movement and therefore the distance between the tips of all extremities can be maximised.

- - -

As a side note: The Koch curve has always appealed to me.

'A finite area contained within an infinitely long border'.

Positions & Poses.

I am struggling to differentiate between the words "position" and "pose" so consider them interchangeable here.

When talking about the position of the body there is a sliding scale of preciseness, from a very generalised description (which may include some details), to named poses, to the exact position accurate in all details.

A Generalised Position.

A generalised position may be a broad categorisation e.g. sitting, standing, squatting, or more specific e.g. sitting on hands, standing on one leg (which leg?), squatting with arms extended (extended in what direction?).

There is a wide scope for variance in the same generalised position.

A Named Pose.

e.g. downward dog, half lotus, plank pose ....

Named poses are demonstrations of what the body is capable of at optimal functioning ('optimal functioning' to be expanded in later posts).

Since there are many options to be in what (without closer examination) appears to be the same position, 'named poses' can be classified as generalised positions - unless the body's exact position aligns with the 'ideal pattern' of the pose ('ideal pattern' to be expanded in later posts).

The Body's Exact Position.

A precise assessment of the positioning of all parts of the body, from core to extremities.

A starting reference is required to describe the body's pose in detail - a 'Base-Line' from where the rest of the body extends and is positioned relative to.

The body's exact position to change moment by moment, even when trying to be still, due to the movements of breathing, vibrations in the cardiovascular system, muscle twitches etc. Stillness is finding the perfect oscillation for equilibrium.


Definitions for posture:

The position of your body... Your pose... How you are holding yourself.

Position, pose, posture... almost interchangeable, ever-changing.

It is important to differentiate between:

Passive & Active Postures.
  • A passive posture - The default setting. The position of your body when you are not thinking about it.
  • An active posture - Conscious thought about "how you are holding yourself". Using voluntary muscles under voluntary control to wilfully adjust your pose.

An active posture becomes the passive norm when the relevant connections between brain and muscle have been 'wired-in', meaning that good postural habits can be formed by consciously working with the right muscles for a sufficient length of time.

A Good Posture.

A good posture is when the main muscles of movement are adequately utilised with the Base-Line (pelvic floor, rectus abdominis) muscles providing the central support for the rest of the body.

With a good posture the body feels comfortable at rest. Movement is smooth and free, with a sense of balance and ease.

Ideal Postures.

Ideal postures match the body's 'ideal patterns', where stresses are distributed and dissipated in the best/safest/most efficient possible manner for the activity being undertaken. Ideal postures create the largest possible body bubble and permit dynamic stability through a full range of natural movement.

An ideal posture provides the maximum capacity to deal with external stresses i.e. the body is as strong as it can be.

There are many disciplines that appear to represent ideal postures and patterns, demonstrations of the body's capabilities when it is dynamically balanced and aligned. (Caveat - I can name a few, but have little knowledge and no experience in most.)

For example:

  • The asanas of yoga - snapshots of the body with a full range of natural movement. Poses that can be perfected when the body is truly balanced.
  • Pilates, tai chi and other internal martial arts, ballet - demonstrating the grace and freedom of movement possible with dynamic alignment.

When the main muscles of movement are not adequately utilised and physical restrictions are present on the body ('physical restrictions' to be expanded on in next post), an ideal posture cannot be achieved so the body resorts to a 'functional posture'.

A Functional Posture.

A 'functional posture' at its most basic:

  • Keeps our eyes level (maintaining horizontal equilibrium in visual input).
  • Keeps us facing/moving forward.
  • Puts the body in a position to do the task at hand.
  • Adjusts body position to bear external stresses as they are applied.

A functional posture uses 'mimic muscles' ('mimic muscles' to be expanded on in next post) and subconscious adjustments throughout the body - twists, kinks, tilts and compressions - as the brain sees fit, to keep us 'upright and facing forward' day-to-day, but the body is imbalanced and imbalance leads to further imbalance (to be expanded on in next post).

Anticipatory Posture.

When faced with a task, the body/brain prepares by activating muscles into an 'anticipatory posture'. Bracing yourself.

An anticipatory posture should be the ideal posture for the activity - using the main muscles of movement to their full potential - but if that is not achievable, the body braces into a functional posture with the use of mimic muscles.

Assessment of Posture & Body Alignment.

Current Methods. (based on a bit of go-ogling)

Traditional methods of assessing posture and alignment include visual inspection (+/- plumb-lines and grids) and the palpation of anatomical landmarks, usually with a stationary subject and using the spinal column and joints as references.

Newer techniques employ radiography and photography (which increases accuracy/reduces susceptibility to human error) but still focus on the position of joints and vertebrae with little consideration for the main muscles that position our bones and create our posture.


The body provides more feedback about its positioning than can ever be supplied by external sources. Becoming aware of this sensory feedback is the basis of conscious proprioception (the sense of position, motion and balance) which facilitates self-assessment and also self-correction of posture.

According to Base-Line hypothesis:

The Base-Line muscles (pelvic floor, rectus abdominis) are the primary source of sensory information about body's posture and state of alignment and the positioning of the rest of the body should be considered relative to the midline anatomy (aligned on the median plane), beginning with the linea alba. (Don't worry about where your feet are to start.)

We can 'stand up straight' with a good posture when we are aligned i.e. the linea alba and nuchal/supraspinous ligaments are fully extended and positioned on the median plane.

A greater self-awareness of posture will develop by focusing on the condition of the main muscles of movement (connecting body and mind) and as more details about relative positioning are considered.

Where is my leg in relation to my Base-Line? Where's my hip, my foot, knee, toes, ankle, heel, big toe... What about the other leg, arms, head...

Feel your positioning and judge it for yourself.

Core Muscles.

"Use your core muscles" is oft-repeated advice, but what does it really mean?

There are many definitions for core muscles and it would not be helpful to add to this over-used term, but think of your Base-Line (strong and long) at the core of all movement.

Focus on activating your Base-Line muscles - longer and stronger with every in-breathe.

What can you sense about your posture and state of alignment?

Is there a balance in the left and right main muscles of movement?

If you move a part of your body, what happens to the whole?

Build the connection.