Сборщик RSS-лент

Published on November 3, 2025 5:30 AM GMT

Third in a series of short rationality prompts.

 

My opening rationality move is often "What's my goal?". 

It is closely followed by: "Why is this hard? And, what can I do about that?". 

If you're busting out deliberate "rationality" tools (instead of running on intuition or copying your neighbors), something about your situation is probably difficult. 

It's often useful to explicitly enumerate "What's hard about this?", and list the difficulties accurately, and comprehensively[1], such that if you were to successfully deal with each hard thing, it'd be easy.

Then, you have new subgoals of "figure out how to deal with each of those hard-things." And you can brainstorm solutions[2].

Sometimes, those subgoals will also be hard. Then, the thing to do is ask "okay, what's hard about this subgoal, and how to do I deal with that?"

Examples

I'll do one example that's sort of "simple" (most of what I need to do is "try at all"), and one that's more complex (I'll need to do some fairly creative thinking to make progress).

Example 1: Bureaucracy while tired

I'm trying to fill out some paperwork. It requires some information I don't know how to get. (Later, it might turn out that the information requires navigating an opaque, badly designed website, and then I have to do some kind of annoying math operation on it)

Also, I didn't get enough sleep and am tired. My eyes are sliding off.

Default outcome: "ugh, this sucks", bouncing off, intermittently trying to work at it and then screw around on Facebook, neither successfully having fun nor completing the form.

What's hard about this? Well:

1. I'm tired, so I have less focus
2. I don't know where the information is.

Trying to deal with both of those at once feels overwhelming. But, now it's more obvious I could split them up and deal with each separately.

#1. I'm tired

Let's start with this, since it'll be a force multiplier on the other one. How might I deal with it? Well, I do some Babbling / Metastrategic Brainstorming, and I come up with:

• Drink some water
• Do a couple jumping jacks and jiggling my face around.
• Take a nap.
• If none of that works, find a friend to sit with me and help me focus.

I try drinking some water. I'm still tired. I try doing a couple jumping jacks. I'm still tired.

Recursion: Turns out taking a nap is Hard.

Take a nap? Ugh, that'll take awhile and I'm kinda shit at napping and I just drank 2 red bulls. I don't expect napping to work.

Hmm. What if I tried just laying down in a dark room for awhile and half-assedly pretend to nap and let my mind wander?

Well, if I did that, I'd end up playing on my phone and not even doing half-assedly napping.

What if... I didn't bring my phone into the dark room?

Okay, you know, that might just work.

I try that. It works. 

#2. I don't know where the information is

The silliest part here is how, by default, I'll periodically "start thinking about filling out the form", and try some autopilot-y move like "glance around on the tax form, hoping an answer will jump out at me." And then that fails, and I am mysteriously on facebook.

Writing out "I don't know how to get the information. How do I deal with that?", prompts the embarrassingly obvious answer of "I have to somehow figure out how to get the information" which implies "I need a plan for getting the information."

But, I notice my brain is still sliding off this problem. I try to think about it, but even with my fresh drink of water and/or nap there is something muddled and murky about it.

Recursion: Finding the information is also hard.

After thinking for a bit, I realize two things:

• A. I don't know how to find the information, or even where to start.
• B. I feel viscerally aversive about this whole thing. I expect finding the information to be very annoying and take a long time and suck all the while.

When I direct my attention to the second thing, what do I do about that? Well, some options:

• Check if I actually need to do the paperwork
  • (a fine first step, although in this case turns out I do)
• Bounce off for now and see if it's easier later.
  • (Unfortunately I've already bounced off it several days in a row and I suspect I'll need to do something different to actually make progress)
• Try to adopt some different mindsets and see if they help.

Fortunately, in this case, I've previously learned the skill of buckling up and mentally prepare for multiple hard cognitive steps in a row. It seems like that's the move here. So, I do that.

Okay. That deals with problem B. Back to problem A. I don't know how to find the information. I need to make some kind of actual plan to find it. What are some things that might help? 

This section is probably fractally hard, where each option is likely to run into another hard thing. But now that I've buckled up it feels less obnoxious to think multiple steps ahead.

Some ideas:

• I can read the instructions on the paperwork more carefully/thoroughly.
• I can google for the relevant company/government website.
• If it turns out my unique-special-snowflake-situation doesn't fit into this bureaucracy's ontology, I can doublecheck that I actually have the right form, or make peace with whatever consequences there are for entering something somewhat inaccurate.
• I can write out an explicit checklist of the steps I think I need to take, and the confusions I have, and use that to handle the incoming working memory overload.

Which of these ends up helping depends on the exact paperwork I'm dealing with. But, it's kinda embarrassing how often I find myself "not even trying" to figure out how to find the information, and instead just sort of bopping around hoping it magically becomes easy. Once I've articulated the problem, it's usually easier to solve.

So I list out the confusing bits. I turn each confusion into an explicit step. I do the steps.

The hard problem has become kinda easy, if still kinda annoying.

And then I'm done. 

Example 2: Getting an LLM to actually debug worth a shit

The bureaucracy exercise was "easy" in some sense. You know how to fill out a form. You've done it a hundred times. Humans have done it millions of times. The only reason it's "hard" is that it kinda sucks. But if you sit down and carefully think through it, it's not too bad.

Sometimes, the reason something is hard is more like "you just really don't know how to do it", and you will need to find a new way of looking at the problem. 

Recently, I vibecoding a fun side project. I wanted to make a website with lots of different versions of sheet music for a given song, and the ability to convert one type of music notation into another. One subthread of that was: can we scan static PDFs of sheet music, and convert them into a file format I could edit?

I asked a Claude Agent to write a conversion script. 

It didn't work.

I said "hey Claude, it didn't work."

Claude said "oh, my bad. Let me try <some random ass thing>." Claude tried that random ass thing, and said "okay I fixed it!" It was not fixed.

I said "Please think harder, and make sure you fully understand the problem before declaring yourself done."

It did some minor ritual-of-cognition that looked vaguely like thinking harder, then did more of the same thing.

Geez Claude, you're debugging like me on a bad day, when I randomly flail instead of making a systematic checklist and trying to understand the problem.

If this were a serious project, I would say "ugh, I guess I have to debug for real", and Buckle Up again and start reading documentation and making lists of my confusions and building a model of the situation. But, I didn't feel like it.

What's hard about this for Claude?

Well:

1. It's working with an external PDF conversion library. Maybe the library is jank. Maybe it needs to switch libraries or solve the problem a whole different way. The hard part here is that the action space is large, and Claude just doesn't have the good taste to navigate that action space.

2. Claude apparently just doesn't have the ability to judge whether it's solved the problem or not, just by looking at the code.

3. Also, Claude can't just look at my website, click the buttons, and see if the PDF gets successfully converted.

If I could solve any of those problems for Claude, it might stop claiming to be done when it clearly wasn't done, and keep trying.

#1 does seem like a lost cause. Someday AI will have good judgment to navigate complex domains when it's first patternmatch guess doesn't work. Today is not that day. (And when that day comes, I'm scared).

#2 and #3 both mean that Claude just has no ability to tell if it's done. Maybe I could convince it to stop pretending it was done, but that wouldn't get it done any faster.

I feel really suspicious of #2. Claude seems like it should be capable of reading through a codebase line-by-line, including stepping into the foreign library, and modeling whether each step should work. I've tried a whole bunch to get it to do this reliably, and haven't gotten it working yet. Maybe I'll find a good metacognition prompt someday, but, having put a dozen hours into it, it's not looking promising tonight.

#3?

Why can't Claude look at my website, click buttons, and see if the conversion is working? 

There's actually some more recent tools that let it do directly that. I haven't bothered to install those tools because I'm lazy. I could install those tools, but then I'd have to do a few annoying steps in a row to make sure the tools work. I'm lazy. This is a fun side project. I don't feel like it.

Hmm. Why does Claude need to navigate a webpage and click a button? 

Because currently, the conversion script is built directly into the webpage, which can only be accessed via a browser.

Is there any good reason for that?

No. I could refactor this into a script that can be run individually as a test case, from the command line. The Claude Agent does have the ability to make and run scripts on the command line (if I give the permissions to do so). 

"Hey Claude, can you write yourself a script that tests the conversion function, and continue re-running it while debugging until the script reliably works?"

"No problem." Claude does that. A few minutes of brute-force exploration later, it has solved the problem. It turns out, the debugging principle of "first, try to cleanly replicate the problem" is just as important for LLMs as for humans. 

(This has turned out to be generally useful for getting AIs to do keep working until they're actually done. Sometimes it's a bit harder to convert a problem into something the AI can "see", but, I now have in my toolkit "think about how to convert a coding problem into something the AI can see")

Recap

When you have a task that feels frustratingly difficult, try asking yourself "what's hard about this?", and then followup with, "what do I do about that?"

Sometimes, this is much easier than these examples, and as soon as you ask "why is this hard?" you'll realize "because I need to think for 2 goddamn seconds in a row and I wasn't bothering to do that", and then you think about it for 2 goddamn seconds in a row and then the solution is obvious.

Sometimes, dealing with the Hard Part is also separately hard, and you need to recursively ask "Okay, why is that hard? What can I do about that?"

Exercise for the reader

What is a difficult thing you're dealing with lately? What's hard about it? How can you deal with that?

(If you're comfortable writing up your answers in the comments, I'd appreciate it)

 

1. ^

   (Well, for lighter-weight problems, it may be enough just to list the single-hardest thing if it easily occurs to you)

2. ^

   See The Babble Challenge and Metastrategic Brainstorming for some background on training the muscle of "generate approaches to solving problems." (Though note one commenter's reply "sometimes the solution isn't actually to 'brainstorm', it's to go on a walk and let the my mind wander until it figures it out.")

Discuss

Published on November 3, 2025 5:20 AM GMT

Sofia Corradi, a.k.a. Mamma Erasmus (2020)

When Sofia Corradi died on October 17th, the press was full of obituaries for the spiritual mother of Erasmus, the European student exchange programme, or, in the words of Umberto Eco, “that thing where a Catalan boy goes to study in Belgium, meets a Flemish girl, falls in love with her, marries her, and starts a European family.”

Yet none of the obituaries I’ve seen stressed the most important and interesting aspect of the project: its unprecedented scale.

The second-largest comparable programme, the Fulbright in the United States, sends around nine thousand students abroad each year. Erasmus sends 1.3 million.

So far, approximately sixteen million people have taken part in the exchanges. That amounts to roughly 3% or the European population. And with the ever growing participation rates the ratio is going to get even gradually even higher.

Is short, this thing is HUGE.

***

As with many other international projects conceived in Europe in the latter half of the XX. century, it is ostensibly about a technical matter — scholarships and the recognition of credits from foreign universities — but at its heart, it is a peace project.

Corradi recounts a story from a preparatory meeting of French and Italian rectors in 1969:

Professor Contini of the University of Florence was not at all pleased with the proposal and declared that “the draft needed to be examined carefully, that what was requested of the group of experts was a lengthy and complex task, etc.”

But when Corradi explained that the proposal was not really about curricula or credits, but rather about peace and international understanding, he immediately changed his tune. He turned to his colleagues and said that the scheme should be approved quickly, and that any wrinkles could be ironed out later.

This kind of approach can be seen over and over again in the post–WWII decades, when those involved had firsthand experience of war.

When I, as a modern person, see a tricky political problem, my intuition, honed in the recent decades, tells me that the solution will be slow and painful, that the participants will engage in backstabbing, blackmail, and the trading of favours, that we’ll end up, at best, with a watered-down version of what was originally intended, and that, most likely, there isn’t enough political will to accomplish anything at all.

And then I look back at the Europeans of 1950s or 1960s and observe how, when peace is mentioned, they simply set their concerns and mutual disagreements aside and do the right, if unorthodox, thing.

Sadly, the generations with personal experience of war have passed away, and that implicit, unspoken mutual understanding is now gone.

During Brexit, UK has dropped out of the scheme, with the government claiming it was too expensive. But it was clearly part of the broader effort to detach Britain from the EU institutions. And pointing out that it’s a peace project and that even EU’s frenemies, such as Turkey and Serbia, were taking part haven’t made any difference.

Not that the EU itself is blameless in this regard. When Switzerland voted in a referendum in 2014 to introduce immigration quotas, the Union showed that it was willing to weaponize the peace project for political ends and kicked Switzerland out of the scheme in response.

***

Back to the social engineering though.

Despite what Umberto Eco said, Erasmus seems to work a bit differently, at least according to the anecdotes I’ve had a chance to hear.

In fact, the Catalan boy goes to study in Belgium, but he doesn’t make any local friends. They are speaking Flemish among themselves and he has no idea what they are talking about. Also, Europe still carries a certain aristocratic air. Unlike in the United States, it really matters whether you were born somewhere and have spent your whole life there. If that’s not the case, everyone’s going to be friendly and helpful, but you’ll remain an outsider.

Yet, the boy befriends other Erasmus students just as eager for company as he is. Italians, Greeks, Lithuanians. He falls in love with a Polish girl, marries her and they settle in Berlin together and speak English at home.

Which, in the end, is probably even better outcome than what was originally intended.

***

And speaking of social engineering: Yes, this is exactly how you do it!

Substantial portion of students actually does want to spend some time abroad. It’s no different from the Western European marriage pattern, where young people left their parental homes to work as servants, farmhands, or apprentices before they married and set up their own households.

The much-maligned idea of social engineering, in this case, doesn’t mean forcing people to do something they don’t want to do. It means removing the obstacles that prevent them from doing what they already want.

Before Erasmus, studying abroad was seen as having fun rather than as serious academic work, something to be punished rather than rewarded. Universities were reluctant to recognize studies completed elsewhere. Erasmus, with its credit transfer system, changed that and thus unleashed a massive wave of student exchanges.

***

The name “Erasmus” is an acronym, but it was clearly chosen to remind us of Erasmus of Rotterdam, the model travelling scholar. Erasmus spent his life moving between the Netherlands, France, England, Italy, the Holy Roman Empire and Switzerland and thus makes a perfect mascot for the programme.

Erasmus of Rotterdam, by Albrecht Dürer

But Erasmus was also a citizen of what he called respublica litteraria and what later became the Republic of Letters, a long-distance network of intellectuals connected through the exchange of letters and, yes, actual travel, something that was an indispensable precursor to the Enlightenment and the Scientific Revolution.

One can think of it as a two-pronged strategy: Letters kept the network among the intellectual elites alive. Travel created new nodes in the network, meetings, collaborations, and shared projects. Professors often recommended promising students to each other, or engaged in master-disciple relationships, such as Tycho de Brahe with Johannes Kepler or Galileo with Evangelista Torricelli.

Which makes me wonder about how the above applies to the modern world.

Today’s blogosphere is sometimes compared to the old Republic of Letters and not without merit: it enables rapid exchange of ideas between people that would otherwise never meet each other. And that’s a great thing.

When it comes to travel though, the state of affairs is somewhat different. The cheap travel of today makes it easy for people to meet in person for a day or two. But at the same time, it makes long stays, needed for deeper collaborations, that were once a necessity, less common.

It’s fun to imagine, say, Matt Yglesias, spending a year in Europe and writing about European affairs. But I don’t see that happening. (Although progress-minded philanthropists could find the idea of sponsoring such intellectual insemination via long-term mutual exchanges interesting to toy with.)

In the meantime, there’s this huge beast of Erasmus programme, waiting to be taken advantage of. We may complain that there are no progress-minded people in Eastern Europe, but at the same time thousands of young people from the region travel abroad each year to study. How hard would it be to steer the promising few towards specific destinations? Presumably those with an established progress community, with the prospect, that after a year spent abroad they will return home and spread the word further?

***

Europeans take Erasmus for granted. They are rarely aware that nothing quite like it exists elsewhere. People from other parts of the world, meanwhile, often don’t know about it at all. And if they do, they see it as a dull, technocratic, typically European matter. Which, I think, is a shame.

Discuss

Published on November 3, 2025 4:54 AM GMT

Epistemic Status: Hastily-written, but I think it's good to consider the various ways non-obvious ways one can save. Obviously everyone has their own situations when it comes to money.

 

I think most people spend much more money than they should. Whether they've been successfully influenced by the constant stream of advertising, following the patterns they see from friends and family trying to find ways to keep up on the ever-moving hedonic treadmill, we make it easy and desirable to use money to purchase goods and services.

However, I see the stress it causes people, I see debt, I hear from people stuck in golden handcuffs, and I see a common pattern where people trade their future security and optionality for small conveniences that really won't have mattered to them at all by the time the month is out. And I know that the issue of money goes far beyond "well have you tried just saving more lol," but there's also a lot of cost-saving options and reasons to save money that aren't immediately obvious, or don't seem realistic if you haven't seen them modeled by someone else.

The steps I took to save money immediately out of college have given me confidence that it's easy to achieve enough success in life, a safety net to try new things without being worried if they don't work out, and is definitely a part of the fairly deep life satisfaction I currently feel. My first job out of college paid me a bit under 60k a year, and after some supplementary income from violin/piano gigs around town (estimating another 10-13k offhand), I saved over 85% of that income for the nearly year-long time while I was carefully tracking it. It wasn't exactly easy, but it also wasn't exactly hard either. My rent was extremely low, thanks to 1) happening to live in a low CoL town, 2) putting effort into finding a deal within that town and 3) having two roommates. Most other cost savings came down to living cheaply food-wise, using my bike as transportation when possible, and having an extremely minimal discretionary budget (there's so many things to do that are free). Maybe I'm just lucky temperament-wise, but I never felt deprived doing this, and spent all the money I needed/wanted to.

More importantly to me, things you do to save money often make you a way cooler/skilled/more interesting person than if you had spent it. This idea goes back to my previous post, Do Things for as Many Reasons as Possible, but there's a lot of low-hanging fruit in terms of car-maintenance, combining exercise/transportation, trading favors for other favors, and for me having more to do with my time usually results in me making better use of my time. If there's something you're paying other people to do for you, at least check if you'd be able to do it yourself. In many cases once you learn just a little bit, you'll gain an advantage on the activity besides cost-savings. For example, it can often be faster/more convenient to do basic repairs yourself once you have the tools/knowledge, the cooking you make will taste better than meal delivery, etc. The extra money you save will also begin to compound over time, and it's just a great situation.

I rushed this post out, but I plan to do better on consistency this month for Halfhaven, so maybe I'll return to these ideas some later.
 

Discuss

Published on November 3, 2025 4:43 AM GMT

I. 

I have claimed that one of the fundamental questions of rationality is “what am I about to do and what will happen next?” One of the domains I ask this question the most is in social situations.

There are a great many skills in the world. If I had the time and resources to do so, I’d want to master all of them. Wilderness survival, automotive repair, the Japanese language, calculus, heart surgery, French cooking, sailing, underwater basket weaving, architecture, Mexican cooking, functional programming, whatever it is people mean when they say “hey man, just let him cook.” My inability to speak fluent Japanese isn’t a sin or a crime. However, it isn’t a virtue either; If I had the option to snap my fingers and instantly acquire the knowledge, I’d do it. 

Now, there’s a different question of prioritization; I tend to pick new skills to learn by a combination of what’s useful to me, what sounds fun, and what I’m naturally good at. I picked up the basics of computer programming easily, I enjoy doing it, and it turned out to pay really well. That was an over-determined skill to learn. 

On the other hand, I’m not at all naturally good at music and for most of my life there wasn’t really any practical use for it, so I’ve spent years very slowly acquiring a little bit of musical skill entirely when it was convenient and fun. I haven’t learned even a single word of Swedish, nor the 101s of scuba diving. 

Let me restate what I said above; my ignorance is not a virtue. 

Some skills are hard to appreciate unless you have some prerequisite amount of the skill yourself. Take music for an example; before I had any musical training, I could nod along to tunes but had no idea of what pieces would be harder or easier and didn’t notice the difference between a flat and a sharp. After spending a few weeks with progressively trickier bass riffs and power chords, I gained the ability to listen to a metal song or watch a video of a guitarist and realize what I was watching was impressive.

Consider computer programming; almost a decade passed between me learning how to do a for loop and learning what a computer scientist means by space and time.[1] Before I learned about big O, I had a vague idea of some code being faster. Afterwards, I could have my jaw dropped by an especially efficient twist of clever code. If you do not have a skill, you may not notice what you don’t have. It’s a little like being colour blind; the skilled designer is wincing at the clashing colours you picked and the low contrast background and foreground of your website and you don’t yet see the problem.

Once you have a skill, often it’s actually easier to do it well than to do it badly. I naturally spell English sentences correctly. If I try to mispll wards nand mac thas sentnce rong, I have to actually stop and force myself not to simply type out what I was thinking. That doesn’t mean that I never make mistakes, but they’re either rare typos or uncommon words I don’t use often. If you’re bored and have a mildly sadistic bent, it can be fun to ask well trained singers to sing off key. Try asking serious weight lifters to use poor form; they’ll usually outright refuse.

https://xkcd.com/1015/

So; there are lots of skills, it’s fine to not have all of them, if you don’t have a skill you might not realize what you’re missing, and if you’re good at a skill then doing it well is often easier than doing it badly. With me so far?

II.

This essay is a response to Lack of Social Grace is an Epistemic Virtue.

My summary of LoSGiaEV: is that social grace and honesty are in conflict, sothus idealized truth- seeking people would not have the polite niceties of common societyin say, American norms. You can either be graceful or honest, and steps towards one direction are steps away from the other. The ideal rationalist would be perfectly honest, and this would involve very little social grace.

I disagree with this.

If people say that you’re rude, obnoxious, or off-putting, then it could be you’re making a deliberate tradeoff, but it could also be you lack the skill of being polite or nice. If you lack the skill of pro basketball, you’re not surprised when you can’t make the team.

That’s fine. But — If they say this and it surprises you, then I’m confident there’s a skill you don’t have. If you’re confident you know when you’re being rude, and you can usually guess what people are going to be annoyed at, then you might have the skill. 

Yes, being polite is an extra constraint on what you can say. Spelling your words correctly is an extra constraint on writing. Good writers usually do it anyway. Unless you are trying to get people mad at you for some reason, people being mad at you seems like a cost.

Social grace and clear honesty do come apart at the tails I expect, but the overwhelming majority of the time — 95% of the time at least, possibly as high as 99% — there's a way to say what you want to say both gracefully and honestly. Maybe you can't find it in the moment. Maybe I couldn't, I'm not claiming to be a master of this. But I've observed enough people make what seem to me to be unforced errors here that I feel confident saying people aren't actually at the Pareto frontier here.

You can’t make everyone happy with you all the time and you shouldn’t try. Or, in local parlance, avoiding anyone ever being mad at you is a poor terminal value! Sometimes any way of expressing something true to someone will tick them off. But there isn’t a slider from More Graceful to More Honest like a slider from -5 to 5 on a number line. Reductio ad absurdum, “your shoe is untied you complete fuckwit” will predictably annoy someone even if it’s true, and “you look ready for our jog!” isn’t charming or pleasant if it’s a lie and they trip and fall on their face thirty seconds from now.  

Not pictured: the face of a man about to faceplant

The way to test this is to try being polite and not rude for a little while, especially if you can do it around a new group of people who won’t be thinking of your past reputation. That said, if people say you’re rude and you are not surprised, I’m less confident you don’t have the skill.

Where does that leave people who lack the skill of being polite and nice to others? Well, they don’t have to learn it. I’m not going to learn Chinese any time soon. If people say I’m ignorant of Chinese, well, they’re right. 

III.

In The Third Fundamental Question Of Rationality, I told the following story:

When I was a teenager, I had a classmate who loved horses. She had pictures of horses on her backpack. She drew horses in the margins of her notebooks. She talked about horses at lunch. I didn’t find horses interesting, but I liked hearing people talk about what they knew a lot about, and wanted to make friends. I was not, however, socially adroit, so what I said was “why do you care about something boring like horses?”

She complained to the teachers about me being rude, which in hindsight was entirely fair. This was one of several turning points that made me realize I was obnoxious and irritating without intending to be. I still like hearing people talk about their fields of interest, but these days I usually phrase the question as “I don’t know much about that subject, can you tell me more about it?” or “so what subject are you fascinated by?” More generally, before I speak I ask myself what I’m about to say and how I think the person I’m talking to is going to react next. If I had asked myself ahead of time what I thought the response would be to calling a subject she was obviously interested in boring, I would have said something else.

This is a pure tactical mistake.

I didn’t get more information this way. I wasn’t more honest by being more graceful. This is not a linear scale.

My lack of social graces cost me without purchasing me any extra truth. Instead of a linear scale, I think social grace and honesty are two axes upon which you can improve.

The bottom left corner, lacking in both honesty and grace, is when you tell someone they’re a moron when they’re actually a genius in the process of correctly proving a new theory. The top left corner, full of social graces but lacking in honesty, is when you let down a repulsive suitor so gently they lose track of the fact that they were asking you out. The bottom right, full of honesty and lacking in all grace, is when you say in front of your date that you don’t think they’re attractive. And the top right, ah, there is the place where skill abounds.

I could have drawn this as a Pareto frontier, with a line moving from the top left to the bottom right. I deliberately didn’t do that, because I think the line can move as you practice. Just as exercising your muscles lets you carry heavier weights, practice in social grace lets you communicate things to people and have them be grateful to you where previously they’d be angry at you. 

Lest I be misunderstood, I think there is a skill to being honest. It doesn’t come naturally to many people, and there are perhaps some connected ways that honesty can come easily when grace comes hard. But not that much correlation. I suspect that it’s a little like the relationship between math SAT scores and writing SAT scores; anti-correlated on a college campus, since the college was looking for people with a certain summed score, but correlated everywhere else since there’s a common factor. [2]

And once you’ve practiced for a while, grace doesn’t take as much effort as you might think. Someone who grew up using expletives and profanity might struggle at first learning how to speak without them, but once they’ve got the habit they’ll find it’s easy to speak without swears. Most of the time it's like singing; flat notes don't take more breath to hold than being on key. Certainly it's equally easy to hear a note whether it's sharp or correct, other than the twitch a singer makes at wanting it sung better. If people think you're singing off-key and this surprises you, there's a skill you don't have.

Cultivate the skill of honesty. But if you’re socially graceless, I disagree it’s a virtue.

I think it’s a skill issue.

1. ^

   In my defense, I learned to program by using the PRGM button on a TI-83 calculator and experimenting with the fixed list of commands to find out what they did via trial and error. This was not an environment conducive to good programming practice.

2. ^

   Call it intelligence, call it test taking ability, call it expensive tutors, call it whatever you want it’s still there. This is Berkson’s Paradox. 

Discuss

Published on November 3, 2025 4:24 AM GMT

Hello everyone, this is Aditya Adiga. I have been part of the “Autostructures project” at AISC 10 and have continued to work on this research at Groundless AI.

I have been building a tool that allows users to mark and port insights or ideas that arise in research conversations into personalised, context-sensitive formalisms. In this post, I want to discuss how this tool works, why it matters, and how cool it is!

Acknowledgements

This work emerged from the AISC Autostructures project hosted by Sahil and Murray, building on Sahil's foundational work on Live Theory. The core ideas were developed in early conversations with Jayson and Sahil, with Djordje and Peter contributing during the initial prototyping stages. Throughout this journey, I've been fortunate to work closely with Sahil, Aditya Prasad, Matt, and Harshit, whose support has been invaluable. My regular discussions with Jayson, Kuil, and Aayush have been essential in navigating this research agenda together.

Many others have shaped this work through their feedback: Abram Demski, TJ, Keith, Murray, Alexandre, Sofi, and others who engaged when I presented publicly, as well as everyone who tested the prototype at the AI Alignment retreat in Ooty and the PIBBSS retreat. The broader communities at AISC, Groundless AI/High Actuation Spaces, Portal, and AI Safety India have provided crucial perspectives, and I thank all who reviewed drafts, especially Aditya, Matt, Sahil, Jayson, Kuil, Aayush, Surabhi, Poorvi, Vatsal, and Alexandre.

I'm grateful to Vatsal and Aditya Prasad for first introducing me to AI safety.

AI Interaction: The Path to Utopia or Dystopia

Our interfaces to AI become our relationship with this technology. Doing this thoughtfully is therefore essential.

Before we jump into the tool, let’s try to understand the importance of building such tools and why it is important to have more human-centred interfaces for AI.

Currently, our interactions with AI, especially Large Language Models (LLMs), are typically transactional; users provide information along with a request and expect immediate responses. By treating these systems as sophisticated answering machines, we are training ourselves to be passive consumers of AI outputs rather than thoughtful participants in problem-solving. Even if you question and try to have a dialogue with AI before coming to some output, you are still passively consuming the meta-layer information, by thinking of AI as something you can talk to. Noticing that this is a choice and there are other forms of relationships that you can have with AI is step zero in the direction of non-passivity. This passivity is dangerous because it can produce results that stray from our actual needs while creating an illusion of efficiency.  The consequence of this is learned helplessness in humans, leading to unwarranted trust in AI systems, which is highly concerning and a path that will ultimately lead to gradual disempowerment.

Rather than allowing AI to have a dominant role in the interface, we build:

1. with the intention of intuitively supporting human discernment and creativity.
2. while avoiding being intrusive or superficial to enhance the meaning-making process in research conversations.[1]

I imagine a time in the near future when we will be able to take intelligence for granted, similar to how we take the internet for granted today. The Internet is the underlying infrastructure for so many things that we use and build without worrying about how it does things under the hood. [2]Similarly, we want AI to be treated as an infrastructure that is backgrounded, only facilitating connections between living beings with a focus on being sensitive to their needs.

Live Conversational Threads(LCT)

The current version of LCT mainly does two things:

• It captures “threads” (independent parts of a conversation) and the relationships between them (contextual, chronological, etc.), enabling richer collective sensemaking without disrupting flow.
• It allows users to mark and port (potential) insights from research interactions to a context that the user desires.

Platform Walkthrough

Below is an example of LCT in action, where threads and the relationship between them are captured and presented to the user.

• Below is an example of LCT in action, where threads and the relationship between them are captured and presented to the user.
• LCT captures “threads” (independent parts of a conversation) and the thematic flow of context between them from a live conversation in the form of a Directed Acyclic Graph (DAG) structure.
  • A node represents a single "thread" - an independent part of the conversation. Each node contains a context summary of what was discussed in that thread.
  • Edges connect nodes and represent contextual relationships - showing how topics flow and relate to each other across the conversation.
  • Together, these nodes and edges form a DAG structure that captures the thematic flow of your entire live conversation.
• LCT allows users to mark points in a conversation through verbal expression of wanting to do so in the conversation or through the interface, where they intuitively identify contextual progress or sense a (potential) insight (highlighted in green). I’ll explain more about the value of this later.

In addition to marking potential progress in a conversation, we want to be able to port these (potential) insights as and when and where required. This is essential; in the live design philosophy, meaning arises in a specific peer-to-peer instance rather than a standalone one. We capitalise on the AI capabilities in formalisation to allow us to produce personalised context-sensitive formalism from these markings.

• The consumers of the marked insight provide information about their research interests or the local context in which they would like to consume this insight.

• The AI-enabled infrastructure in the background considers the potential insight and local context to produce personalised context-sensitive formalisms.
  • List of conversational threads for which formalisms are generated.

• Toy model: Causal loop diagram generated for the selected conversational thread(Claude Sonnet 3.7 was used as the AI infrastructure here).

 

• Math proof generated by DeepSeek Prover v2 for the selected thread (this proof is used to generate a proof using Kimina Prover formally verified in Lean): 

Note 1: I expect the mathematical validity and quality to improve over time.

Note 2: Jayson is working on Live Discernment(he will be posting something soon), which is about the relevance of formalisms to the situation it is being used in, which becomes very important in this era of AI-generated formalisms.

Live Theory(Image source: The Logistics of Distribution of Meaning)

The way we do math today is by generalising via excluding things. So in the Venn diagram, you can see that we're excluding the non-common regions and identifying the common region that makes sense independently in all three contexts it is being used, and this becomes a unified, or rather a context-independent, representation of that concept. If I were to consider one of the contexts in the Venn diagram that contains this representation, a lot of information that is sensitive to that context is not considered by this unified theory.

Rather than generalising via exclusion, live theory proposes to generalise via inclusion. Let’s say there is some insight about deception, someone might be interested in this insight from an interpretability perspective, someone else might be interested in this insight from an LLM evaluations perspective, etc. Having a single formula to describe something about a concept as abstract as deception might not be possible. In fact, it doesn’t seem like we have even reasonable contenders for a single definition, nor a hope for one. Instead of this, we acknowledge the presence of the insight and, based on the different contexts it is used in, we produce different formalisms for the same insight, resulting in a pluralism of formalisms that allows us to capture information about the insight which would have been missed if we used a context-invariant unified theory. This allows mere “family resemblances” to be first-class theoretical artefacts, that may contain both rigorous and post-rigorous information. I’ll explain a little more in the next section, but it might sound paradoxical if you aren’t used to it.

Contextual Progress and Potential Insights

Live theory emphasises post-rigorous thinking (which signifies a deeper understanding of a concept beyond formal restrictions). Post-rigour encourages intuitive manipulation/characterisation of some potential insight or idea that can be converted to rigorous arguments (or formalisms) when needed.

An example of developing intuition to identify these post-rigorous objects is sports. When someone like Steph Curry shoots a three-pointer, before the ball has gone in, he has already gotten the feel or the smell that it is going in, so he celebrates before it has even gone in. Another example would be if someone is crossing a football into the middle of the field, the other players intuitively know how they should position themselves on the field to receive the cross and make the best possible play. This intuition or instinct has come to them after practising for many years. The same thing goes for math as well. Once you have solved a bunch of algebraic questions, when you come across an unseen question, you have developed an intuition that allows you to navigate the path to the right answer, without referring to any of the questions you solved previously. 

As Terry Tao says:

> [Mathematicians at the post-rigorous stage] no longer need the formalism to perform high-level mathematical reasoning, and are actually proceeding largely through intuition, which is then translated (possibly incorrectly) into formal mathematical language.

One of the goals of live conversational threads, or rather live theory as a whole, is an attempt to treat these post-rigorous intuitions/directions/discernment/insights/hunches as a commodity [3](MFTs-Meta Fungible Tokens, blog on this coming soon). Something that exists in a meta form and can be used to generate personalised mathematical formalisms when, where, and how they are needed.

A lot of research at the highest strata is initiated by these post-rigorous commodities that can be fleshed out by grad students, and now, with increasing possibility, AI. Live Conversational Threads capitalises on this very idea and acts as an underlying infrastructure that allows users to produce, distribute, and consume these commodities.

Who doesn’t like games? (Analogy to understand Live Theory better)

I like to think about live theory as a vast open-world game as opposed to a linear, story-mode campaign.

(Image source: Tomb Raider game)

In a story mode game, every player is forced down the same scripted path and has no choice but to accept to make progress in the game.

(Image source: Hogwarts Legacy game)

Whereas in an open world, you have the participative freedom to wander anywhere, pick up side quests when you wish, or ignore them altogether.

The developer can embed countless missions that remain dormant until your curiosity awakens them.

Each time you accept a quest, your interest in participating in the quest, along with the developers' storyline for this quest, comes together to produce this beautiful gaming experience that is unique to you and feels self-directed rather than imposed.

Cheap, fast, highly accurate AI capabilities turn research into that same open world.

• Imagine having thousands of tireless “mathematician NPCs” (AI infrastructure) waiting in the background.
• The developer has embedded a lot of quests(potential insights) (ex, a hunch about deception in AI models) in this world.
• You are represented by an avatar that has its traits, abilities, personality, etc (personal research interests).
• When you decide to take part in a quest, your avatar’s characteristics (personal research interests), along with the quest (insight), come together thanks to the mathematician NPCs (AI infrastructure), enabling this to produce a personalised, custom storyline (formalism) that is unique to you.

You have the freedom to review, remix, or ignore their work at will, then wander off to explore another quest.

When AI is cheap, fast, and accurate, it becomes possible to scale that which doesn’t scale. The theoretical heavy-lifting is no longer a bottleneck; it scales effortlessly in parallel, yet never dictates your route. Instead of having to create a single grand theory and force it on everyone, you get to build flexible intellectual playgrounds where each researcher can craft the exact tool they need - much like how the developer of an open-world game allows every player to carve their unique path.

Closing Remarks

LCT isn’t about enforcing a new structure, but enabling the right structure to be created when and where it is needed. If the tool helps people stay with ambiguity a little longer, trace evolving trains of thought across time, and sense-make in ways that feel true to their context, that would be a meaningful outcome.

Resources

LCT demo link(alpha version): https://lct-app-515466416372.us-central1.run.app/

Github: https://github.com/aditya-adiga/live_conversational_threads

If you're interested in discussing these ideas further, testing the prototype, funding this work or exploring potential collaborations, I'd be happy to connect.

1. ^

   Kuil points this out well by saying "to facilitate rather than approximate meaning"

2. ^

   Here I just want to emphasize the opportunities available from backgrounding, not dismiss the utility of looking under the hood. If anything, it is extremely important for some folk to be constantly checking underneath the hood. There is in fact another project working on this: called live discernment[will post link live as soon as available]

3. ^

   Here Commodity is more of an aid for intuition; it is more about respecting it as a first-class artefact worthy of attention, iteration and maturation.

Discuss

Published on November 3, 2025 4:00 AM GMT

I like to wake up early to watch the sunrise. The sun hits the distant city first, the little sliver of it I can see through the trees. The buildings light up copper against the pale pink sky, and that little sliver is the only bit of saturation in an otherwise grey visual field. Then the sun starts to rise over the hill behind me. My house casts a blue shadow across the street, and a few trees start to get washed in pale gold. The fog burns away.

I sit at my window for an hour, and I don’t check my phone once. It’s in another room. It doesn’t matter.

I have thoughts, looking down at the people below me. Most of them are fleeting, the kind of thing I think about texting to my boyfriend, but instead I let them pass. If something is worth remembering or exploring, I’ll write it down in a notebook, or on my digital typewriter. Sometimes it’ll become a 1500-word essay, sometimes a poem or a song. Often, the thought just stays there, to be returned to, turned over in my mind until I’ve figured out the larger shape of it.

I make my breakfast, singing to myself. I bike to work with my phone in one of my panniers, out of reach, out of hearing range.

The two-year-old twins I nanny light up when I walk in, and they shout my name in their garbled toddler voices and run over to hug my legs. I crouch down to their level and talk to them, listen to them. The girl twin asks for “buh-ee”, and thanks to months of careful study of her consonant-poor language, I go and get her a blanket. The boy twin points at a picture of the moon and says “mon mon mon mon mon” more and more insistently until I say “That’s right, moon!”, at which point he is satisfied and gets on with his life. He just wants acknowledgement. Don’t we all.

At the park, I talk with other nannies, trading my broken Mandarin and Spanish for their broken English as we laugh and cluck over our many toddlers. I used to be far too afraid to talk to strangers, let alone in a language I wasn’t good at. Now it would feel weird not to.

While the kids nap, I read. I get through many books every month, not because of external pressure to read more, but because I like learning things, and because reading is something you can do even if you’re tired. I always liked reading when I was a kid, but for a few years I forgot to do it. (Sometimes the book is boring and I fall asleep, and that’s okay too.)

On my way home, I stop at the grocery store. While I wait in line to check out, I watch the people around me and think about nothing in particular. My mind is calm; the interstitial time doesn’t feel like suffering, like it once did. I don’t feel the need to be distracted.

At home, I open my laptop to respond to a time-sensitive email, and then half an hour later I realize I’m just on Wikipedia. Oops. I close my laptop and leave it behind.

When my boyfriend comes home, he picks up a guitar and noodles on it while we talk about our days. We laugh a lot. He’s gotten a lot better at the guitar in the past two years, deliberate practice on the pentatonic scales so he can pick out melodies. Sometimes we’ll play a song together, or I’ll inexpertly play piano while he reads. Most nights, we watch one episode of the one show we’re watching. The last three shows have been in Chinese, since he decided to learn it, and we pause a lot to puzzle through the sentences together.

We turn off our screens at 9 PM, even if the episode isn’t over. We brush our teeth and write in our journals and read books next to each other in bed, and we go to sleep so we can do it all again the next day. This is the life we’ve chosen, and we like it.

Discuss

Published on November 3, 2025 3:52 AM GMT

Cross-posted from my website.

Last year, I wrote a list of things I've changed my mind on. But good truth-seeking doesn't just require you to consider where you might be wrong; you must also consider where you might be right.

In this post, I provide some beliefs I used to be uncertain about, that I have come to believe more strongly.

1. My belief: Evolution is true.

   Why I believed it originally: I learned about the theory of evolution in school. I had the impression that it was a popular but unproven hypothesis ("just a theory").[1]

   What made me more confident: When I was maybe 10 or 12, I read an article in some science magazine (National Geographic, maybe) about evolution. It said, "evolution is a theory in the same way atoms are a theory." I probably put too much credence in this one sentence in one article, but in my mind, this was definitive proof that evolution is true.

   Later, when I was 14, I started getting interested in the specifics of the theory of evolution and learned much more about the supporting evidence. (My motivation was mostly that I wanted to argue with creationists on the internet.)

   I went through a similar trajectory when learning about quarks. I was taught that a quark is a hypothetical particle that exists inside atoms, but has never been observed. Later I learned that the existence of quarks is well-established, and it became well-established nearly three decades before I was born.

   On the subject of outdated pedagogy, this is a bit of a tangent but in 5th grade I was taught the five kingdoms of life: monerans, protists, fungi, plants, and animals. Recently, I learned that not only do biologists no longer use this classification system, but that it was already obsolete when my 5th grade teacher was in 5th grade.[2]

   (My 5th grade teacher was pretty young, but still.)

2. My belief: Value investing works.

   Why I believed it originally: I read about Joel Greenblatt's magic formula investing and its strong historical performance.

   What made me more confident: I read more research on value investing, including the seminal paper The Cross-Section of Expected Stock Returns[3] by Fama and French, and more in-depth research showing value investing has worked across the world and across asset classes[4], and on older data going back 200 years[5].

3. My belief: Peaceful protests can be effective.

   Why I believed it originally: I actually went back and forth on this one. In school I learned about Martin Luther King and how he was a hero of the civil rights movement, the Montgomery Bus Boycott that he helped organize, Gandhi's protests against colonialism, and implicit in all this was the idea that these tactics were effective.

   Eventually I learned about GiveWell, which was the first time I'd ever encountered the notion that just because a charity says it's effective, doesn't mean it's actually effective. I started thinking critically about protests in the same way, and I realized that I'd never actually seen good evidence that MLK or Gandhi were responsible for the positive changes that coincided with their activism.

   Then I started thinking, well, there's not strong evidence that protests work, but there's at least some reason to believe they work. That's about where I was at in 2024 when I donated to PauseAI—I thought, I don't really know if this is gonna work, but it's worth trying.

   What made me more confident: I wrote Do Protests Work? A Critical Review, in which I carefully investigated the strongest evidence I could find. I found that the best evidence was better than I'd expected, and it pointed toward peaceful protests being effective.

4. My belief: Seed oils are good for you; seed oils don't cause obesity.

   Why I believed it originally: I had never heard of the seed oil-obesity hypothesis until I read Dynomight's article on the subject, which argues against the hypothesis. Dynomight presented some evidence that seed oils are harmful and then ultimately concluded that they're not. I didn't think much about the evidence the article gave, but its conclusion seemed reasonable to me.

   What made me more confident: I researched the issue in more depth while writing Outlive: A Critical Review, specifically the section on saturated fat. I looked through the literature and presented what I believed to be the strongest evidence on the matter: meta-analyses of RCTs that directly compared dietary saturated fat with unsaturated fat (which usually meant seed oils). The experimental evidence finds that seed oils are, if anything, healthier than saturated fat, which contradicts the seed oil-obesity hypothesis.

   I read some writings by proponents of the seed oil hypothesis, and their arguments seemed incredibly weak to me.

   (Later, I re-read Dynomight's article and found that it cited the same evidence I had looked at while writing my review of Outlive, which I had completely forgotten about.)

   Dynomight presented the seed oil hypothesis as reasonable but ultimately probably wrong, so that's what I believed at the time. After examining the evidence in more depth, I don't think the seed oil hypothesis is reasonable. Dynomight admirably followed Daniel Dennett's principles for arguing intelligently, in which you present your opponent's case as strongly as possible. But this gave me impression that the seed oil hypothesis is more plausible than it actually is.

5. My belief: Absent regulation, we aren't going to solve the AI alignment problem in time.

   Why I believed it originally: I've vaguely believed this since I first learned about the AI alignment problem (in 2013, if I remember correctly). The problem seemed to involve some thorny philosophical problems of unknown size, like the outline of an enormous beast under a murky ocean. But at that point, humanity had collectively only spent a few hundred person-years on AI alignment, and I thought, perhaps there will be some breakthrough that makes the problem turn out to be much easier than expected. Or perhaps as superintelligent AI becomes increasingly imminent, humanity will rally and pour the necessary resources into the problem.

   What made me more confident: In this case I haven't much changed my interpretation of the evidence, but I've become more confident as new evidence has come out. Namely, AI has gotten extraordinarily more powerful; alignment work has not kept up with the increases in AI capabilities; even though alignment work gets more attention now, the problem still seems about as hard as ever.

   Beyond that, almost all alignment work is streetlight effect-ing, focused on solving tractable but mostly-irrelevant problems; and the frontier AI companies mostly don't engage with, and are sometimes even actively hostile to, the idea that solving alignment will require major philosophical breakthroughs and it can't be done using the sorts of empirical methods that they're all using.

6. My belief: Most studies on caffeine tolerance are not informative.

   Why I believed it originally: Prior to writing my post Does Caffeine Stop Working?, I reviewed some studies on caffeine tolerance and I thought to myself, these studies aren't even testing the hypothesis they claim to be testing, surely I must be missing something?

   What made me more confident: I read the studies more carefully and spent more time thinking about them, and read a few contrary papers by other scientists who study caffeine. My more careful analysis only reinforced my initial belief that most studies on caffeine tolerance are, indeed, not useful.

7. My belief: I am smart.

   Why I believed it originally: In elementary school, I knew I was the smartest kid in my class. But my class only had about 20 students, and I figured I wasn't that smart in the grand scheme of things. Like, not as smart as scientists and people who go to Harvard and stuff.

   What made me more confident: The first big piece of evidence came after I took the PSAT in 10th grade and my score was good enough that I realized I had a good shot at getting into a top university.

   Then I actually attended a top university and realized that many of the people there were not that smart compared to me. College was still a big step up from elementary school: I went from always being the smartest person in the room to being only in the top 1/3 most of the time, and I sometimes found myself in the bottom third.

   This trend of repeatedly up-rating my own intelligence reached its peak when I started taking advanced computer science classes, where I was close to the 50th percentile. And nowadays I'm about average within my social circles, and often below average.

   (If you're reading this, there's a good chance that you're smarter than me.)

   Another canon event happened when I saw the data on the distribution of my school's SAT scores. The school's average score was just over one standard deviation above the population mean.[6] I went through high school thinking my average classmates were average, when in reality they were considerably smarter than average.

8. My belief: When I first got into lifting weights a decade ago, I learned a lot of conventional wisdom like:

   • Low reps are better for strength, and high reps are better for hypertrophy.
   • Compound exercises are better for strength, and isolation exercises are better for hypertrophy.
   • Long rests are better for strength, and short rests are better for hypertrophy.
   • If you want to bulk or cut, you should eat at a 500 calorie surplus/deficit to gain/lose about a pound per week.

   Why I believed it originally: It was the conventional wisdom—people generally agreed that these things are true, even though nobody talked about why.

   What made me more confident: I started paying more attention to scientific literature on resistance training and I learned that the conventional wisdom pretty much had it right, at least on these points.

   (The first three pieces of advice are all explained by a unifying factor: to build strength, you want to lift as much weight as possible, and to build muscle, you want to do as much volume as possible. High reps, isolation exercise, and short rests all enable you to wear out your muscles while lifting lighter weights, and the lighter the weights, the more volume you can do. These three bits of advice aren't overwhelmingly important—you can still build muscle doing compound exercises at low reps—but they're useful as guidelines.)

9. My belief: Exercise is good for you.

   Why I believed it originally: Everyone says exercise is good for you, right? But I didn't know how you'd demonstrate scientifically that that's true. I thought perhaps it's reverse causation (sick people can't exercise) or confounded by socioeconomic class or something.

   What made me more confident: I learned more about the scientific evidence on exercise.

   • Many randomized controlled trials show that exercise improves short-term health markers—it reduces blood pressure, improves blood sugar regulation, etc.
   • A smaller number of long-term trials show long-term health benefits to exercise.

10. Spoilers for Game of Thrones / A Song of Ice and Fire. I don't see a way to do spoiler tags in LessWrong Markdown, so I will just put some empty space before giving the spoilers.

 

 

 

 

 

My belief: R + L = J. That is, Jon Snow's parents are Lyanna Stark and Rhaegar Targaryen.

Why I believed it originally: This had long been a popular fan theory. I didn't figure it out on my own, but I was reasonably convinced by the evidence in this article. I thought it sounded right, but I was uncertain because the textual evidence wasn't conclusive.

What made me more confident: I watched an interview with David Benioff and Dan Weiss, the creators of the TV show. They told a story about how they met with George R. R. Martin to get him to agree to adapt his books. At some point in the meeting, he asked them: Who is Jon Snow's mother? They gave an answer, and he didn't say whether they were right, but he gave a knowing smile, and he agreed to let them make the TV show.

They didn't say what their answer was. But I found this story to be pretty much decisive evidence for R + L = J because what it proved was that the answer was knowable. If David and Dan could know it, then the rest of the fan base could, too.

Later I became even more confident when the TV show revealed that R + L = J. (Rarely in life do you get definitive confirmation that your theory is correct!)

1. When I was young, I thought the way evolution worked was that a group of apes were born about 500,000 years ago, and these apes lived for hundreds of thousands of years, over which time their bodies slowly morphed to become more and more humanoid, until they became fully human, at which point they birthed human offspring and then died.

   One time I told my dad that I wish I could've gotten to evolve because I wanted to live for 500,000 years. That's when I learned that that's not how evolution works. ↩︎

2. Carl Woese defined a six-kingdom taxonomy using evidence from ribosomal RNA in 1977, at which time I believe my 5th grade teacher would've been in 2nd grade.

   Lest I sound like I know what I'm talking about, the only reason I can talk coherently about ribosomal RNA methods for taxonomic classification is because I just read those words off Wikipedia 15 seconds ago. ↩︎

3. Fama, E. F., & French, K. R. (1992). The Cross-Section of Expected Stock Returns. ↩︎

4. Asness, C. S., Moskowitz, T. J., & Pedersen, L. H. (2012). Value and Momentum Everywhere. ↩︎

5. Baltussen, G., Swinkels, L., & van Vliet, P. (2019). Global Factor Premiums. ↩︎

6. And someone who gets an average score on the SAT is above-average intelligence, because taking the SAT at all already screens off the lower end of the bell curve. ↩︎

Discuss

Published on November 3, 2025 3:27 AM GMT

Your average day starts with an alarm on your phone.

Sometimes, you wake up a couple of minutes before it sounds.

Sometimes, you find the button to snooze it.

Sometimes, you’re already on the phone and it appears as a notification.

But when you finally stop it, the lights in your room turn on and you start your day.

You walk out of your room. A presence sensor detects your motion, and a bulb in a cute little bamboo lamp from IKEA outside your room lights up.

You go downstairs, into the living room/kitchen/workspace area. As you come in, 20 different lights turn on in perfect sync. It is very satisfying.

You put some buckwheat to boil. Go upstairs to load a washing machine. Go through your morning routine (when you stop by your room, the lights turn on as you enter).

You go back downstairs. The lights are still on: You’ve not left the area for long enough. You eat your buckwheat with oat milk, open the laptop, and do some work.

An hour later, a notification appears: the washing machine has finished washing. (Depending on whether you tapped an NFC tag next to the washing machine with your phone, it is sent either as a push to your phone only, or as a message to a channel in your house Discord and that everyone can see. You tried to make the Discord bot’s name cute, so it’s House Elf.)

After half an hour, you finally go upstairs to unload the washing machine, then come back to work.

In the evening, you’re hosting a meow party: someone on Twitter made a joke that there should be a party where people are only allowed to communicate in cat noises. A friend shared it with you when you were looking for party ideas, and you decided it would be very funny to attempt to do that, and so every hour, for 20 minutes, people are only allowed to communicate in cat noises.

You make the lights dim and colorful and cosy.

A group of people is playing a game on a large table. In meows, they communicate to you they want the light to brighter so that they can see the colors of their cards better. You pull out a phone and with a couple of taps, make the bulb over that part of the table brighter and more towards the white color.

You hear happy cat noises.

Some other day, you’re cuddling on the couch, but it gets cold. You don’t want to leave the cuddles, so you ask Siri to close the windows in the living room. The windows close.

You later go to your room to sleep. It is warm, exactly the right temperature. Later, you wake up and want to get some water. You press on a light switch, and the dim red lights turn on. When you go back to the bed, the lights turn off on their own.

The above is not a tale. This is what my home is actually like and what my days are actually like.

I think it’s awesome.

And you can turn your home into all of that and more. And it is surprisingly cheap and easy to set up!

When I moved in, it took me a few days to move the home from its old boring state to what you’re reading above. This can be what your life is like!

It saves so much time and makes everything so much more aesthetically pleasant and convenient!

Everything can run locally. All the lights and actuators use mesh networks, not WiFi, without the Chinese gov being able to use it to spy on you.

The sky is the limit on what your home can do.

At some point, I thought it’d be funny to have a button that causes apples to appear, the way Trump is said to have had a button that causes Coke to appear.

I made a browser-use agent and exposed its API to the smart home system and got a nice round button.

If I press it, 15 minutes later, a delivery person appears and gives me apples.

To live in a smart house like that, I’d be willing to spend much more on rent per year than it actually costs to make it, because it can be incredibly cosy, and convenient, and at the same time, very cheap and straightforward to set up.

If you’re convinced or curious, here are opinionated instructions on how to get started:

1. Home Assistant

You need a core of your smart home that interfaces with you and all of your devices and everything else you want to integrate.

There are many smart home hubs that belong to closed ecosystems with issues. And there’s one hub that stands out: Home Assistant.

It is not just “open-source”. Last year, it was the top-1 repo on GitHub by number of contributors.

It’s a home server that supports virtually everything smart basically out of the box, and powers everything smart about my home.

It passes security audits from top pentesting firms.

Home Assistant doesn’t have to be exposed to the internet. But want to use it remotely and afraid of zero-days? Put it behind a CloudFlare tunnel with a Zero Trust access firewall and use it.

It is infinitely customizable. It has everything built-in, but you can very easily make it completely your own (without even writing any code, though if you’re willing to copy-paste yaml from your favorite LLM, you can do literally everything with it).

There are, of course, great apps for iOS and Android, and an infinity of things and features a huge community has made.

It connects to everything. Have a WiFI speaker that your iPhone can’t share music to? Home Assistant can expose an AirPlay entity and send the audio to that speaker. I’m not aware of anything Home Assistant doesn’t support or can’t do.

(Have an Alexa? You can control everything in your smart home with it, if you want to. And yes, you can fully customize which of your smart home devices Alexa will see.)

Trust your Claude so much you want it to be able to control the lights in your living room, but only if you ask it with a little poem? Easy.

(If for some reason you already have devices connected to WiFi, Home Assistant will discover and offer to add all of them.)

It can be run on almost anything (any box that can run an OS or a docker container), but the easiest way to get started in a new home is to buy Home Assistant Green, a small $130 server that wants to be connected to the local network via an Ethernet cable. Or if you already have any sort of small home server, you can install a Docker container on it.

You’ll start by plugging it in, turning it on, and opening the Home Assistant app on your phone. Your home is now smart!

2. Local network

Many smart Chinese lightbulbs and other smart home devices use WiFi and want to talk to and be managed via Chinese servers. That’s terrible. While it is possible to take local control of them and disconnect them from the internet (google “[device/brand name] locally home assistant”), it’s somewhat annoying to do, and much easier to just avoid almost everything that uses wifi to not potentially expose the local network.

So, ideally, you want the devices in your home not to use WiFi.

How do you do that?

There are two protocols, Zigbee and Z-Wave. Both are mesh networks: most devices on your network with external power also serve as routers.

You add a USB stick/antenna (examples produced by a nonprofit that manages the development Home Assistant: Zigbee ($45), Z-Wave ($80) (both can be found cheaper if you particularly care)) to your home assistant server to allow it to serve as the mesh network coordinator and manage all the devices on the network.

Zigbee is the older one, with many more devices supporting it. I’d start with Zigbee and maybe later also add Z-Wave.

Both use very little power, so battery-powered sensors and buttons will work for months to years without requiring battery replacements.

3. Lights

Smart CCT+RGB lights are the coolest thing about all of this.

You can have amazing bright white lights you feel happy being around (especially if you live in cloudier parts of the world), that turn all sorts of cozy color schemes when you want them to, or turn red in the evening because you don’t want to expose yourself to blue & green light and want to want to go to sleep at the right time.

The lowest barrier to entry is with Zigbee lightbulbs: you simply replace a boring old bulb with a smart bulb. There are lots of them: from more expensive Philips Hue bulbs ($20-70 per bulb) with mediocre CRI[1] (80) but awesome colors and effects (such as the fireplace effect) to the fairly cheap IKEA bulbs ($18-22 per bulb) with worse colors (the red will be a little bit pink, which is a bit annoying if you want perfect red in the evening) but much better CRI of 90.

But the perfect choice is the Auxmer LED strips. You can pick CRI 95+ CCT+RGB 5m strips ($76) + a Zigbee controller ($23) + any 24V 200W power supply adapter from Amazon ($25), and place them somewhere with a diffuser or pointing up/towards the walls (they’re to bright to be directly looked at at max brightness without a diffuser). I have never seen anyone have better lighting than this. The red is so perfectly red!

For some lights, you can plug them into the wall via a Zigbee smart plug (see below), which lets you remotely turn them on and off.

For completeness: I’ve never done that, but you can also use smart switches instead of normal light switches in the walls, to turn on/off or dim legacy lights.

4. Sensors and actuators

• Motion/presence sensors: detect if there are people/movement. They can use infrared and/or microwave Doppler effect from motion. You should go either for random small $1-5 sensors from Aliexpress that just see large-scale movement in front of them or $40-70 Everything Presence sensors that not only say whether there is someone present, but can also track the coordinates of up to three people in quite a large area (this is how I can tie automations to me being on my bed or in front of a desk in my room, but you can do more fun stuff). You can use automation blueprints like this one to tie a bunch of lights to a bunch of sensors and optionally some additional controls. (Many of them also detect illuminance, but the only time I used that was when I placed one on the intercom to automatically answer it.)
• Air quality sensors. Temperature, CO2, humidity, PM2.5, volatile organic compounds. No particular recommendations, anything works. There are many air quality sensors with Zigbee. My house already had an Awair connected via wifi and I connected my Aranet 4 via a $9 Bluetooth bridge.
• There are also vibration sensors, door state sensors, leak/wetness sensors, etc.
• Switches. Come in all forms and shapes. React to touches or presses. Have arbitrary numbers of buttons. Search Zigbee switches on Aliexpress or anywhere else. (They can be placed on walls, on tables, or anywhere else.)
• Fun/weird input devices. Wanna change the lighting by turning a cube on a side? (If you see it requires some proprietary Zigbee hub, ignore that.)
• Smart plugs. They can be turned on and off remotely and some of them let you know the current, voltage, etc.; this allows you to know the state of otherwise dumb devices (or devices you don’t want to connect to the internet), such as washing machines. $10 from IKEA. (Search your local IKEA for the local type of plugs.)
• Fingerbots. These are great. If you have any physical buttons that you want to press remotely, you can get a bunch of those ($15 from Aliexpress). They’ll press things for you. (This is how opening and closing windows in our living room works.)
• Thermostats: they control the flow of water in the radiators in specific rooms. They come with kits that make them compatible with virtually all radiators. (If you install one, you’ll unlock bonus knowledge of the very surprising interface that thermostats have.) No more manually rotating anything! (Use the Better Thermostat integration to use actually great algorithms for optimally supporting a nice temperature + being able to use a temperature sensor that’s not the one inside the thermostat.)
• All other kinds of smart moving things. From smart blinds to (if you’re brave enough) sous vide.

You can also collect and use all other sorts of information in all kinds of creative ways: from weather or public transport info to calculating energy expenditure of all of the smart devices (volunteers have measured it for most things you can find!),.

5. Automations

Have an idea for what you want your smart home to do? I’d bet you’ll be able to easily implement it. It is very straightforward to set up automations using the interface, but you can also ask your favorite AI assistant to write yaml for you to not have to click on things.

Look for guides or ask your AI assistant!

There’s a ton of guides for how to get started with Home Assistant. Open and follow any of them or ask chatgpt. It is actually very straightforward, and if you’re curious, look up how to make your home smart.

1. ^

   CRI is Color Rendering Index. It is one of the ways of measuring how well the light spectrum of a bulb represents the light spectrum of the sun. Even if your home is not smart, you really want to have high-CRI light, for a bunch of reasons. Especially if you live in a cloudy part of the world.

Discuss

Published on November 3, 2025 1:24 AM GMT

Unpopular opinion time! Daylight Savings (yes I say “savings”; sue me; but that isn't the unpopular part) cleverly solves an otherwise utterly intractable coordination problem. Standard business hours are 9 to 5 and even if you're self-employed you probably are, for example, on an ultimate frisbee team or somesuch that can't start at 5pm because of the nine-to-fivers on the team. There's a whole web of interdependent schedules and there is just no way to induce a critical mass of people to shift their activities earlier in the day so as not to waste so much daylight by sleeping in hours past dawn.

UNLESS you resort to the outrageous hack of just literally changing the clocks.

Sure, it makes life a living hell for computer programmers, and the original rationale of saving energy on lighting surely doesn't apply. (Also I guess it literally kills people, but so do a lot of things that are obviously still worth it. Like ice cream, probably. Cars I don't know. If they're worth it, I mean.)

But the upside — more daylight in the evenings — is a big deal. And the idea that people could just choose on their own to wake up earlier when the days start getting longer is all wrong. I mean, yes, you can personally do that, but it does you no good unless everyone else (like the rest of your ultimate frisbee team) does it too. And your frisbee team can't do it unless all the businesses do it and the businesses can't do it unless the trains and buses do it and every other sport and club and social group and... Like I said: massive, intractable coordination problem.

I think it's kind of awesome that we were able to solve the problem at all.

“Ok, fine, permanent daylight savings time then!” says everyone I know. There are two problems with that, one practical and one philosophical. Practically, in much of the northern hemisphere, it means starting the day in pitch darkness in the middle of winter. Philosophically, just like, how absurd is it to permanently change the clocks rather than change standard business hours? My whole argument above is that changing standard business hours earlier and later again every year is untenable. But if the public consensus is “business hours should just always start earlier so we have more daylight after work” then it's almost tragically hilarious that the best way to achieve that is to permanently redefine time itself rather than tamper with the apparently greater sanctity that is "Nine To Five". Maybe it's the Dolly Parton movie by that name that really locked us in there.

(I think ideally I'd love to see a system of referring to times that was relative to sunrise. This is a can of worms though. Or a can full of cans of worms, one of which is how much nerds despise the concept of timezones.)

Body Time

As any computer programmer can attest, off-by-one errors can be dang confusing. At least I'm personally bad enough at mental arithmetic that it's easy to confuse myself about whether I'll get tired or wake up an hour earlier or later after the clocks change for daylight savings time.

So I think the right concept handle for this is body time. The clocks fell back last night which means that, relative to the clocks, body time is an hour ahead. If you usually go to sleep at 11pm then at 10pm clock time, it will be 11pm body time. So that's when you'll get sleepy. Same thing in the morning. You wake up at your normal body time and clock time is an hour earlier than that.

Body time is +1 hour after falling back in the fall and −1 hour after springing head in the spring.

Easy peasy. And especially helpful if you have children if you want to be like “ok, time for bed, it's 9pm body time” at 8pm tonight!

Discuss

Published on November 3, 2025 12:19 AM GMT

In general, I find it very useful to imagine how things (very broadly speaking) may have gone were an event negated or another event introduced. Doing this systematically, even using standard chat models, proves to be quite difficult and generally unwieldy. To address this, I have created the Hauntoloscope. Under the hood is a prompt that allows Kimi-K2 (my favorite model for this type of thing) to generate news articles in timelines where a single event is negated or a fictional event has occurred.

I specifically think this will be of interest to the broad rationalist community. To that point, I am interested in examples where the prompt could be improved. You are able to import and export entire generation objects (called Relics) as .json files. These are easy to share. 

I would like the counterfactuals to be as realistic as possible (anything less would make it not very useful). Currently, I think the generations are ~pretty good, but there is certainly room for improvement.

 

A Hauntoloscope generation negating the Moon Landing

 

Currently, it is bring your own API key. I am using Groq because it is incredibly fast. I have not monetized this whatsoever and am more interested in making it a useful tool for thought. It is also fully open source.

I am excited to see what people end up using this for!

Discuss

Published on November 3, 2025 12:00 AM GMT

Someone can be a savant in one area and a midwit in another. Another way to phrase this is that specialization of labour is common, and specialists don't generalize well outside their speciality. Or: knowledge transfer is hard. 

For instance, I have a family member who is very knowledgeable when it comes to geography, but they've sadly been infested with TV brain worms so they're hopeless when it comes to politics. Or another family member who is very good at MacGyver-eseque solutions to daily problems, but somehow cannot navigate to save their lives. Or a friend of mine who is consistently insightful when it comes to econ yet somehow doesn't know calculus. 

This sort of "spikiness" in capabilities is the norm amongst humans. It holds for me, you, and even those who are world-class. Like: 

1) Greg Egan. He has deeply impressive mastery of physics and geometry, produces some of the best SF known to man, and yet has silly takes arguing that intelligent minds cap out around human level or would naturally not be expansionist. 

2) Elon Musk. If we set foot on Mars, Musk is the human most likely to have caused it to happen. This isn't just business acumen. People who work for SpaceX acknowledge his engineering ability. So clearly a master of getting things done in (parts of) the world of atoms. But look at DOGE, where Musk burnt through huge amounts of political capital to no real end. 

And it's more general than that, really. Any one of the people I mentioned has their own areas of expertise in which I'd expect them (to some degree) to outperform others. But only in those areas. 

You would not go to Terrence Tao for medical advice, or for help designing your house or doing your taxes or so on. Sure, given time, he could do a better job than most experts. But obviously he's not going to do that. 

And yet, for some strange reason, I often see someone be incompetent in one area and then immediately generalize to a lack of competence in all other areas. Heck, I do this too. 

Now, obviously this has some merit to it. There is a g-factor after all, and if you see someone persistently try and fail to resolve an issue, that is evidence they just aren't that sharp. 

That's not a problem. The problem comes when you don't know what someone else cares about, where they've invested their character points, and then rule them out as a moron if they perform incompetently in an area they don't really care about being good at. This is an issue. 

Now, obviously you don't see most experts loudly giving stupid takes on topics you know a lot about, because you don't see most experts do anything really. But we're animals that build our models of the world from patch-works of lived experience, and sometimes our only lived experience with some groups of experts, or groups of people, come through seeing them from afar when they aren't in their elements. And that skews our perception of how competent people can be at their best. And when you're trying to make use of people, the best you can get out of them is what matters. 

Forgive them their midwittery, for they are also savants.

Discuss

Published on November 2, 2025 10:12 PM GMT

Like, probably, but I think this is an interesting and non-trivial question. I’m not actually 100% sure about the answer; I’m probably 97% sure of the answer. But the correct answer doesn’t actually affect the value of thinking about this question.

Even if you know a lot of biology, there is an explanation that a biologist could give about how apples are actually not made of cells which would, I claim, be pretty convincing. How much detail would they have to give before you believed them? Even if it turns out to be false, it’s good practice to know what it feels like on the inside to have your mind changed about something. Can you imagine what the biologist could say that would change your mind? Think about this for a second before reading on.

Literally what is this

You learn pretty early on in biology class that all life is made of cells. (We’re not getting into the issue of viruses, here.) It’s pretty much what life is; cells are the level at which the replication occurs, so life begetting life is cells begetting cells.

But cells are extremely complicated and messy and made of lots of sub-parts. And they can produce bulk materials that are useful to the organism: materials which are not themselves made of cells. Tissues need to be made of cells in order to perform complex functionality that is locally responsive to the molecular conditions around it. Cells have the machinery to control what goes in and out, and control which genes get expressed when.

But not all parts of organisms need this.

• Your hair and fingernails and outer skin layers are kind of cells but kind of not. They’re dead cells, flattened together and drained of most of their contents.
• The enamel in your teeth is not made of cells. It just needs to sit there being a rock. This does come at the cost of self-repair, hence dentistry being a major category of medical care.
• Bones are in large part made of minerals, which is to say, tiny rocks, which I would claim are not cells. But these minerals seem to be fractally mixed-in with the myriad organic operations of the bones, almost as though bones are perfectly calibrated to queer the made-of-cell/not-made-of-cells binary.
• Your bladder is full of, let’s say, non-cellular fluid.

And some cells are very large. Eggs are often cited as large cells (though it’s very unclear to me whether this is true), with the ostrich egg being the largest. Could apples be giant, single plant cells? How sure are you?

So, what’s up with apples? Is there some minimal cellular machinery around the edges that packs in the sugars, pumping nutrients in through the stem, making sure the skin grows in proportion, with the bulk of the apple mass being undifferentiated acellular deliciousness? Or is it cell walls all the way through, crunching crisp copies of every chromosome?

If you were the first discoverer of cells, you would need to spend a while going around and checking different types of tissues before you could justify the generalization “all life is made of cells”. And I think fruit is a category where you would be less justified in generalizing to.

So how could you tell? Are there ways other than looking it up, or using a microscope?

Discuss

Published on November 2, 2025 9:29 PM GMT

Halfhaven is a virtual blogger camp, an online alternative to Inkhaven Residency.

The rules are simple:

• every day post max 1 article with min 500 words (or equivalent effort)
• try to get 30 by the end of November (but there are no hard lines)

The invitation links keep expiring, the current one is: https://discord.gg/jrJPR3h6

If you wanted to join during November, but you couldn't join because you didn't have the link, I apologize; I was on a vacation without internet. If you published your articles online anyway, feel free to add them retroactively in Halfhaven with their actual publication date. In general, post a link in Halfhaven the same day you publish the post.

Here are the posts published during October:

a11ce

• Qamar

Adam Shai

• Don't give up on ambitious interpretability
• Physical Computation and GPTs (Part 1: The Framework)

Alex Kurilin

• Hiring - The Big Picture
• Hiring - Telling Your Company's Story
• The CTO Fork in the Road
• Which Game Are You Playing?

Algon

• Why's equality in logic less flexible than in category theory?
• In which the author is struck by an electric couplet
• Do One New Thing A Day To Solve Your Problems
• What I've Learnt About How to Sleep
• Maybe Social Media Algorithm's Don't Suck
• Notes on the need to lose
• Chaos Alone is No Bar to Superintelligence
• What shapes does reasoning take but circular?
• What criticisms have been made of Bostrom's Open Global Investment Model?
• Stars are a rounding error
• What does it feel like to understand?
• You're not confused by Thermodynamics
• Don't mock yourself
• Predictability is Underrated
• What is Lesswrong good for?
• Some astral energy extraction methods
• Book Review: To Explain the World
• Meditation is Dangerous
• Libraries need more books
• Why do leaves fall in autumn?
• Discovery Fiction: Max Flow Algorithm
• The Doomers Were Right
• Kleinian view of geometry
• Remembrancy
• Credit goes to the presenter, not the inventor
• All the lab's AI safety Plans: 2025 Edition
• Genius is Not About Genius
• Centralization begets stagnation
• Supervillain Monologues Are Unrealistic
• Random things I learnt about ASML after wondering how critical they were to GPU progress

Aq

• No, That's Not What the Flight Costs

Ari Zerner

• Treading Water
• Introducing Halfhaven
• Some Nonsense For My Third Halfhaven Post
• You Are Good
• SlutCon Vignettes: Day 1
• More About SlutCon
• Good Grief
• Letter To My Past
• Free Writing

duck master

• How could criminal justice work in Ducktopia?
• my retrospective of the october 8 board games meetup

Gyrodiot

• Off start
• Indulge

ironlordbyron

• Aspiring Phoenix
• Rumination Is A Functional Goal-Achievement Subsystem
• Parasite Apostle
• SlutCon Diary
• Slutcon: Confidence Engineering Postmortem
• You Can Just Give Teenagers Social Anxiety!
• You Want Exposure Therapy To Be Ego-Syntonic

keltan

• Why did OpenAI make a Sora App?
• ENIGMA
• Bring Honor to Us All
• Starts to Stories I did not Finish
• Sure, I can help you with Human flourishing
• Response to a Reddit Post
• The Rationalist PR Department
• Minds as Art/Odd Prompt
• Why My Classes Like Me
• A Superbly Crafted Sculpture of a Naked Obese Woman from the Stone Age
• Hunger Pains - Part 1
• Limitless Box & Random Patterns
• Making Films Quick Start - 1 Audio
• Making Films Quick Start - 2 Visuals
• Film Making Quick Start 3 - Editing

Logan Riggs

• How to Feel More Alive
• Telling the difference between memories and logical guesses
• Replacing RL w/ Parameter-based Evolutionary Strategies
• Hospitalization: a Review
• Your First Love Song is Perfect
• How to Solve Racism (in voice chat games)
• C is for Cat. B is for Boundaries
• I Didn't Sign Up For These Standards
• My comment on the bayes influence function paper
• Temporarily Losing My Ego

lsusr (videos)

• Why Meditation Causes Ego Death
• stories about skinwalkers
• Sometimes, the Best Criticism is an Homage
• How to Cultivate Spikier Preferences
• Why Jhānas are Non-Addictive
• You can just Choose your Social Reality
• The Direct Perception of the Simulation in Your Brain

mishka

• Meta: a problem of choice
• Sequence: non-standard approaches to AI existential safety
• Some of the promising non-anthropocentric invariants
• Interlude: non-anthropocentric vs anthropocentric approaches
• Invariants of self-modifying systems and self-governance
• Methods for handling invariants of self-modifying systems
• Ways to maintain continuous and gradual nature of changes
• How to include the changing world in our modeling
• Different levels of including a model of the world
• Recap 1: "Modest alignment" vs "MIRI alignment"
• Recap 2: Approximate invariants of self-modifying systems
• Summary: non-anthropocentric approaches to AI existential safety
• Diary: hyperstitionai, agentic engineering, recent papers, NVIDIA DGX Spark
• Follow-up: HyperstitionAI, agentic engineering
• Diary: agentic stuff and other things
• Diary: overview of recent "zzznah" (neural cellular automata, etc.)
• Qualitative transitions: from AlexNet to competent agents

niplav

• Shake Brains First
• I Believe the Value Misspecification Argument
• Some Thoughts on the Stupid Successionism Debate
• Waiting Outside a Shop Is Rarely Worth It; Public Transport Has Bad Vibes; Doing Cold Approach Every Day Is Pretty Powerful
• Quantum Computing is about Atoms not Bits
• The Need for Speed
• Humanity Learned Almost Nothing From COVID19
• Edits of Pergraphs, Pascal's Mugging and Anti-Superpersuasion interventions
• "Solving" Pascal's mugging by making some very strange modifications to the prior

ParrotRobot

• Freewriting in my head, and overcoming the “twinge of starting”

Philip

• Do Things for as Many Reasons as Possible
• Journal Update (June-Sep)
• Lessons from the Mountains

Tassilo

• Some Biology Related Things I Found Interesting
• Consider Doing Small Walks at Work
• Notes on "Homology, Genes and Evolutionary Innovation"
• Patience and Willingness to Be Slow
• My Software Setup
• Cheap Labour everywhere
• Learnable Skills
• Brainstorming Food on the Cheap+Healthy+Convenient+Edible Frontier

Taylor G. Lunt

• Fool Heart Joins Halfhaven
• Claude Imagine Bypassing its Security Sandbox Using Screenshot APIs
• Trial of Claude 4.5 on Chess and Fantastical Chess
• One Does Not Simply Walk Away from Omelas
• 10 Ways to Waste a Decade
• Lift ’n’ Peel
• The Liberty Tractor
• If a Lioness Could Speak
• The Mom Test for AI Extinction Scenarios
• The Three Levels of Agency
• Give Me Your Data: The Rationalist Mind Meld
• Cover Your Cough
• The Real Cost of a Peanut Allergy
• Guys I might be an e/acc
• Unsureism: The Rational Approach to Religious Uncertainty
• Asking Paul Fussell for Writing Advice

Viliam

• Animating images in Java (1)
• Paint.NET
• The Quest
• Animating images in Java (2)
• Animating images in Java (3)

Feel free to join anytime. If you join later than November 1st, you won't be able to achieve the soft goal of 30 total articles before the end of November, but a few articles is still more than nothing, if you need the nudge.

Congratulations to: Algon, keltan, mishka, and Taylor G. Lunt -- guessing by your productivity so far, you are most likely to have the 30 posts before the end of November. But nothing is certain until you actually do it, and the rest of us may still have a chance.

Discuss

Published on November 2, 2025 9:06 PM GMT

Below is the a transcript of the recent deposition of Ilya Sutskever on his time at OpenAI and the firing of Sam Altman. I'm posting it here in plain text so that it is easier to read and discuss. 

UNITED STATES DISTRICT COURT
NORTHERN DISTRICT OF CALIFORNIA
OAKLAND DIVISION

ELON MUSK, et al.,
Plaintiffs,

v.

SAMUEL ALTMAN, et al.,
Defendants.

Case No. 4:24-cv-04722-YGR

VIDEOTAPED DEPOSITION OF ILYA SUTSKEVER

San Francisco, California
Wednesday, October 1, 2025

========
PAGE 128
========

ATTORNEY MOLO: Well, you put -- you put these two screenshots in your memo; correct?

WITNESS SUTSKEVER: Yes.

ATTORNEY MOLO: All right. And then you sent your 52-page memo, Exhibit 19, to the independent directors of the board; correct?

WITNESS SUTSKEVER: Correct.

ATTORNEY MOLO: All right. Why didn't you send it to the entire board?

WITNESS SUTSKEVER: Because we were having the discussions with the independent directors only.

ATTORNEY MOLO: Okay. And what -- why didn't you send it to Sam Altman?

WITNESS SUTSKEVER: Because I felt that, had he become aware of these discussions, he would just find a way to make them disappear.

ATTORNEY MOLO: Okay. Were you careful about what you included in this document?

ATTORNEY AGNOLUCCI: Object to form.

WITNESS SUTSKEVER: So the way I wrote this document was to -- the context for this document is that the independent board members asked me to prepare it. And I did. And I was pretty careful. Most of the screenshots that I have that I -- most or all, I don't remember. I get them from Mira Murati. It made sense to include them in order to paint a picture from a large number of small pieces of evidence or items.

ATTORNEY MOLO: Okay. And which independent directors asked you to prepare your memo Exhibit 19?

WITNESS SUTSKEVER: It was most likely Adam D'Angelo.

ATTORNEY MOLO: Okay. And did you recall when he asked you to do that?

WITNESS SUTSKEVER: No.

ATTORNEY MOLO: Okay. Do you recall what it was that he said to you that caused you to prepare this memo?

WITNESS SUTSKEVER: I don't remember what he said exactly.

ATTORNEY MOLO: What's your best recollection of what he said?

WITNESS SUTSKEVER: He said something like -- he ask me if I have screenshots.

========
PAGE 129
========

ATTORNEY MOLO: Well, before he asked you if you had screenshots, I mean, what caused him -- to your knowledge, you know, to ask you to prepare this?

WITNESS SUTSKEVER: I had discussions with the independent board members discussing the subject matter of these documents. And after having some discussions, either Adam or the three of them together, I don't remember, have asked me to collect supporting screenshots.

ATTORNEY MOLO: Okay. And the three of them together were Adam D'Angelo, Helen Toner, and Tasha McCauley?

WITNESS SUTSKEVER: Correct.

ATTORNEY MOLO: All right. And the document that you prepared, the very first page says: [As Read] "Sam exhibits a consistent pattern of lying, undermining his execs, and pitting his execs against one another." That was clearly your view at the time?

WITNESS SUTSKEVER: Correct.

ATTORNEY MOLO: All right. And the -- you had expressed that view to the independent directors before sending them this memo?

WITNESS SUTSKEVER: Correct.

========
PAGE 130
========

ATTORNEY MOLO: All right. Did they express concern over that to you?

WITNESS SUTSKEVER: Correct.

ATTORNEY MOLO: And did you want them to take action over what you wrote?

ATTORNEY AGNOLUCCI: Object to form.

WITNESS SUTSKEVER: I wanted them to become aware of it. But my opinion was that action was appropriate.

ATTORNEY MOLO: Okay. And what action did you think was appropriate?

WITNESS SUTSKEVER: Termination.

ATTORNEY MOLO: Okay. And you sent it using a form of a disappearing email; is that right?

WITNESS SUTSKEVER: Yes.

ATTORNEY MOLO: Why?

WITNESS SUTSKEVER: Because I was worried that those memos will somehow leak.

ATTORNEY MOLO: Okay. What would happen if they leaked?

ATTORNEY AGNOLUCCI: Object to form.

ATTORNEY MOLO: I mean, excuse me, what was your concern about them leaking?

WITNESS SUTSKEVER: It was a generalized concern.

ATTORNEY MOLO: Okay. You drafted a similar memo that was acritical of Greg Brockman; correct?

WITNESS SUTSKEVER: Yes.

ATTORNEY MOLO: You sent that to the board too?

WITNESS SUTSKEVER: Yes.

ATTORNEY MOLO: Does a version of your memo about Greg Brockman exist anywhere in any form?

ATTORNEY AGNOLUCCI: Object to form.

ATTORNEY AGNOLUCCI: I believe various lawyers have a copy.

ATTORNEY MOLO: Who has a copy?

WITNESS SUTSKEVER: Various lawyers.

ATTORNEY MOLO: Which lawyers would those be?

ATTORNEY AGNOLUCCI: Object to form. And, obviously, I'm instructing the witness not to answer with respect to any documents that he may have provided to his own counsel. So if you know --

ATTORNEY MOLO: Well, that doesn't make them privileged.

========
PAGE 131
========

ATTORNEY AGNOLUCCI: -- if you know -- and I'm happy to meet and confer about this off the record. But for the time being, if you know whether other lawyers and which lawyers have the documents, you can testify.

WITNESS SUTSKEVER: I know that my lawyers have a copy.

ATTORNEY MOLO: Okay.

ATTORNEY AGNOLUCCI: And I'm not -- and I'm instructing you not to answer with respect to any information that you learned from your lawyers about who else might have a copy of the document.

WITNESS SUTSKEVER: Yeah. Okay.

ATTORNEY MOLO: Well, why are you instructing him not to answer?

ATTORNEY AGNOLUCCI: Because it reveals the content of his communications with counsel. It reveals his counsel's strategy. It reveals, potentially, other privileged information and we can -- we can have a discussion with this off the record. And, you know, if we're all convinced that it's appropriate, we can come back and ask questions about it.

ATTORNEY AGNOLUCCI: Same objections and instructions.

WITNESS SUTSKEVER: And I'm not sure I should answer.

ATTORNEY MOLO: Okay. Did you provide your lawyers with a copy of the Brockman memo?

WITNESS SUTSKEVER: I don't remember exactly how -- the manner through which I gave it to them.

ATTORNEY MOLO: Okay. Do you know of any other copies that exist anywhere?

ATTORNEY MOLO: I'm sorry. You're not sure you should answer or --

ATTORNEY AGNOLUCCI: I'm instructing you not to.

ATTORNEY MOLO: Let him.

ATTORNEY MOLO: Are you saying you -- you're saying you're not sure you should answer?

WITNESS SUTSKEVER: Yes.

ATTORNEY MOLO: Okay. And your lawyer's instructing you not to answer?

========
PAGE 132
========

WITNESS SUTSKEVER: That's what I'm hearing.

ATTORNEY MOLO: Okay.

ATTORNEY AGNOLUCCI: And I think there's some ambiguity about -- in his mind what's privileged and so.

ATTORNEY MOLO: He knows what's in his mind.

ATTORNEY AGNOLUCCI: I'm happy to meet and confer further, but I think we need to move on.

ATTORNEY MOLO: I'm going to show you Exhibit 20.

(Exhibit 20 was marked for identification.)

ATTORNEY MOLO: This is an article from "The Wall Street Journal" dated March 28th of 2025; byline by Keith -- excuse me -- Hagey; headline "The Secrets of Misdirection Behind Sam Altman's Firing From OpenAI." You're familiar with this article?

WITNESS SUTSKEVER: No.

ATTORNEY MOLO: Okay. At page 440, at the bottom of the article, if you look at the third line, there's a sentence...

========
PAGE 166
========

ATTORNEY MOLO: I'm asking about at that time in November --

WITNESS SUTSKEVER: Yeah.

ATTORNEY MOLO: -- on the Saturday after Altman was fired --

WITNESS SUTSKEVER: Yeah. I'm talking about that time --

(Court reporter clarification.)

ATTORNEY MOLO: And I'm directing your attention specifically to the Saturday after Altman's firing?

WITNESS SUTSKEVER: Yeah, I'm talking about the Saturday as well.

ATTORNEY MOLO: Okay. Were you concerned about losing your equity in OpenAI at the time?

ATTORNEY AGNOLUCCI: Object to form.

WITNESS SUTSKEVER: I was not concerned.

ATTORNEY MOLO: What -- what was your equity at OpenAI worth at the time? What did you think it was worth?

ATTORNEY AGNOLUCCI: Object to form. Relevance. Can we meet and confer about why this is relevant.

========
PAGE 167
========

ATTORNEY MOLO: What was --

ATTORNEY AGNOLUCCI: I'm --

ATTORNEY MOLO: Look, it doesn't have to be directly relevant. So it's certainly relevant.

ATTORNEY MOLO: In any event, do you recall --

ATTORNEY AGNOLUCCI: Well --

ATTORNEY MOLO: Let me ask a question.

ATTORNEY AGNOLUCCI: Well, you've interrupted me in the middle of making an objection. So let me finish.

ATTORNEY MOLO: That's all you've done the entire deposition is object.

ATTORNEY AGNOLUCCI: That's my job. So --

ATTORNEY MOLO: Actually, it's not.

ATTORNEY AGNOLUCCI: -- if he has a privacy interest in his financial information --

ATTORNEY MOLO: You've -- you've designated this highly confidential from the outset.

ATTORNEY AGNOLUCCI: That doesn't mean that he has to answer your question. Why is it relevant?

ATTORNEY MOLO: It's relevant whether or not he was losing value of his -- of his equity or whether he thought he was at the time.

ATTORNEY AGNOLUCCI: Well, then you can ask him that question but not how much.

ATTORNEY MOLO: It certainly does matter how much. What did you think the value of your equity at OpenAI was at the time Sam Altman was fired?

ATTORNEY AGNOLUCCI: And I have the same objection, and I'm instructing him not to put a number on it.

ATTORNEY MOLO: What did you think the value was?

ATTORNEY AGNOLUCCI: You can answer the question that he has saying is relevant here, which is whether you thought you were going to lose value.

ATTORNEY MOLO: It isn't my question. I have a right to ask a question. You can object to the question.

ATTORNEY MOLO: My question is what did you think the value of your equity in OpenAI was at the time of Sam Altman's firing?

========
PAGE 168
========

ATTORNEY AGNOLUCCI: And I'm instructing the witness not to answer as to the money value.

ATTORNEY MOLO: Are you going to not answer?

WITNESS SUTSKEVER: I mean, I have to obey my attorney.

ATTORNEY MOLO: Okay. So you're not going to answer?

WITNESS SUTSKEVER: I'll do what my attorney tells me to.

ATTORNEY MOLO: Okay. Were you concerned about possibly losing your equity at the time that -- I withdraw the question.

ATTORNEY MOLO: Eventually, the board agreed to resign and restore Sam Altman, didn't it?

WITNESS SUTSKEVER: Yes.

ATTORNEY MOLO: When was that?

WITNESS SUTSKEVER: Later in the week.

ATTORNEY MOLO: And why did they do that?

ATTORNEY AGNOLUCCI: Object to form.

WITNESS SUTSKEVER: There was a question of why the board did it?

ATTORNEY MOLO: Correct.

WITNESS SUTSKEVER: Or is the question why I supported this?

ATTORNEY MOLO: First, I'm asking you why did the board decide to resign and reinstate Sam Altman?

========
PAGE 169
========

WITNESS SUTSKEVER: (No response recorded at this page)

========
PAGE 300
========

WITNESS SUTSKEVER: Right now, my view is that, with very few exceptions, most likely a person who is going to be in charge is going to be very good with the way of power. And it will be a lot like choosing between different politicians.

ATTORNEY EDDY: The person in charge of what?

WITNESS SUTSKEVER: AGI.

ATTORNEY EDDY: And why do you say that?

ATTORNEY AGNOLUCCI: Object to form.

WITNESS SUTSKEVER: That's how the world seems to work. I think it's very -- I think it's not impossible, but I think it's very hard for someone who would be described as a saint to make it. I think it's worth trying. I just think it's -- it's like choosing between different politicians. Who is going to be the head of the state?

ATTORNEY EDDY: Looking back at the process that -- that preceded the removal of Sam and Greg from the board, what's your assessment of that process?

ATTORNEY AGNOLUCCI: Objection. Vague. Calls for speculation. Lacks foundation.

WITNESS SUTSKEVER: Can you elaborate on what you mean.

========
PAGE 301
========

ATTORNEY EDDY: You've had time to reflect on -- on the process that preceded the removal; right?

WITNESS SUTSKEVER: I had time.

ATTORNEY EDDY: And in reviewing the steps that preceded the removal, do you think that process was correct?

ATTORNEY AGNOLUCCI: Same objections. Lacks foundation. Calls for speculation. Vague.

WITNESS SUTSKEVER: One thing I can say is that the process was rushed.

ATTORNEY EDDY: Why was it rushed?

WITNESS SUTSKEVER: I think it was rushed because the board was inexperienced.

ATTORNEY EDDY: Inexperienced in what?

WITNESS SUTSKEVER: In board matters.

ATTORNEY EDDY: Before your conversation with Helen Toner that you've described concerning Sam's management problems, how frequently had you interacted with her in 2023?

WITNESS SUTSKEVER: Not too frequently.

ATTORNEY EDDY: And how frequently did you interact with Tasha McCauley?

(Court reporter clarification.)

WITNESS SUTSKEVER: Also not frequently.

ATTORNEY EDDY: Where did Tasha live when she was a member of the board?

WITNESS SUTSKEVER: I don't know.

ATTORNEY AGNOLUCCI: Object to form.

ATTORNEY EDDY: Did -- did you see her at OpenAI premises?

WITNESS SUTSKEVER: From time to time.

ATTORNEY EDDY: About how frequently?

WITNESS SUTSKEVER: As frequently as board meetings.

ATTORNEY EDDY: And when were those?

WITNESS SUTSKEVER: I don't actually remember exactly, but there is information. It can -- this is -- this information can be determined.

ATTORNEY EDDY: Did -- did -- did Tasha show up physically for board meetings every time?

WITNESS SUTSKEVER: From time to time. I cannot confirm if it's every time.

ATTORNEY EDDY: And what about Helen? Did she show up physically for board meetings?

WITNESS SUTSKEVER: From time to time.

ATTORNEY EDDY: Not every time?

WITNESS SUTSKEVER: I don't think every time also.

ATTORNEY EDDY: Where did Helen live, if you know, during the time she was on the board?

========
PAGE 302
========

WITNESS SUTSKEVER: I don't know for sure. I believe she lived in D.C. at least part of the time.

ATTORNEY EDDY: How -- how familiar did Tasha and Helen appear to you to be with OpenAI's operations?

ATTORNEY AGNOLUCCI: Object to form.

WITNESS SUTSKEVER: They seemed to have some familiarity, but it's hard for me to assess.

ATTORNEY EDDY: Did you view them as experts in AI safety?

ATTORNEY AGNOLUCCI: Object to form and relevance.

ATTORNEY AGNOLUCCI: Counsel, we have limited time left here. We're not making the witness available for more than seven hours. Why is the process the board engaged in relevant to this lawsuit?

ATTORNEY EDDY: It is relevant because the magistrate has ruled that these matters from November 2023 are relevant.

ATTORNEY AGNOLUCCI: My understanding is that the ruling was that the termination was relevant. But why is the process and where the board members lived and how engaged they were as board members relevant?

ATTORNEY EDDY: I'm asking questions that relate to the matters that are relevant to this lawsuit; so I won't have a lot more on this topic. And we're not going to keep the witness beyond time that's been allotted to us.

ATTORNEY AGNOLUCCI: Okay. So it sounds like the only basis of relevance is that they relate generally to the termination, these questions.

ATTORNEY EDDY: They relate to the termination, yes.

ATTORNEY EDDY: Helen Toner was associated with Open Philanthropy at some point. Do you recall that?

ATTORNEY AGNOLUCCI: Same objections.

WITNESS SUTSKEVER: I have vague knowledge of that.

ATTORNEY EDDY: Is Open Philanthropy, in turn, associated with Holden Karnofsky?

WITNESS SUTSKEVER: I believe that to be the case -- or at least used to be case at some point.

ATTORNEY EDDY: And Holden Karnofsky is married to Daniela Amodei?

WITNESS SUTSKEVER: Yes.

========
PAGE 303
========

ATTORNEY AGNOLUCCI: Same objections about relevance. Waste of time. Not within the scope of the magistrate's ruling.

ATTORNEY EDDY: Very much disagree.

ATTORNEY EDDY: The -- Daniela Amodei is married to Dario Amodei; is that right?

WITNESS SUTSKEVER: No.

ATTORNEY EDDY: Sorry, the sister. Definitely not married.

ATTORNEY EDDY: They are -- they are brother and sister; is that right?

WITNESS SUTSKEVER: Yes, that's right.

ATTORNEY EDDY: And they are both with Anthropic; is that right?

WITNESS SUTSKEVER: Yes, that's right.

ATTORNEY EDDY: Holden Karnofsky is also associated with Anthropic?

WITNESS SUTSKEVER: I don't know if that's definitely the case. I have some -- I believe it was the case at least at some point.

ATTORNEY EDDY: Okay. Do you recall in October 2023 Helen Toner publishing an article criticizing OpenAI?

WITNESS SUTSKEVER: I do recall.

ATTORNEY EDDY: What -- what do you recall about that?

WITNESS SUTSKEVER: I don't -- I don't recall the nature of the criticism, but I recall it was praising Anthropic.

ATTORNEY EDDY: And what was your reaction to that article?

WITNESS SUTSKEVER: I found it a strange article.

ATTORNEY EDDY: Why?

WITNESS SUTSKEVER: I found it a strange thing for her to do.

ATTORNEY EDDY: Did you think it was appropriate for her to do as a board member of OpenAI?

WITNESS SUTSKEVER: I thought it was not far from obviously inappropriate.

ATTORNEY EDDY: Did -- did you discuss with anyone the prospect of Helen being asked to leave the board at -- at that time?

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: What do you remember about that?

WITNESS SUTSKEVER: I discussed it, at least, with Sam.

ATTORNEY EDDY: And what was that discussion?

WITNESS SUTSKEVER: Something to the effect of -- I don't remember the specifics.

ATTORNEY EDDY: Did you support removing Helen Toner from the board or asking her to leave?

========
PAGE 304
========

ATTORNEY AGNOLUCCI: Same line of objections about relevance, waste of time, scope of the magistrate's order.

WITNESS SUTSKEVER: At least at one point, I expressed support.

ATTORNEY EDDY: After Sam was removed, do you recall Helen Toner telling employees that allowing the company to be destroyed would be consistent with the mission?

WITNESS SUTSKEVER: I do recall.

ATTORNEY EDDY: And what was the context of that comment?

WITNESS SUTSKEVER: The executives -- it was a meeting with the board members and the executive team. The executives told the board that, if Sam does not return, then OpenAI will be destroyed, and that's inconsistent with OpenAI's mission. And Helen Toner said something to the effect of that it is consistent, but I think she said it even more directly than that.

ATTORNEY EDDY: More directly than you've related here?

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: Okay. And what was your reaction to that?

WITNESS SUTSKEVER: I don't remember my reaction at the time.

ATTORNEY EDDY: Did you think that would be consistent with the mission?

ATTORNEY AGNOLUCCI: Object to form.

WITNESS SUTSKEVER: I could imagine hypothetical extreme circumstances that answer would be "Yes"; but at that point in time, the answer was definitely "No" for me.

ATTORNEY EDDY: I wanted to just ask a few questions about this document that --

WITNESS SUTSKEVER: Yes, please.

ATTORNEY EDDY: -- you prepared --

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: -- Exhibit 19.

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: Did -- did you show the final document that -- this document here, Exhibit 19, to Mira Murati?

WITNESS SUTSKEVER: I think it is possible and likely, but I don't have a definite recollection.

ATTORNEY EDDY: Okay. And did you show it to anybody else at OpenAI before conveying it to the board with the disappearing link?

WITNESS SUTSKEVER: No.

ATTORNEY EDDY: I want to just look at page 529, the second page in.

WITNESS SUTSKEVER: Yes.

========
PAGE 305
========

ATTORNEY EDDY: And you say here there's reason to believe that Sam was removed from YC in the past for a reason similar to the one that you identify in this document?

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: And you say: [As Read] "Sam was pushed out from YC for similar behaviors. He was creating chaos, starting lots of new projects, pitting people against each other, and thus was not managing YC well." Am I right the basis for this is a conversation that Mira had with Brad Lightcap?

ATTORNEY AGNOLUCCI: Object to form.

WITNESS SUTSKEVER: The basis of this is a conversation that I had with Mira.

ATTORNEY EDDY: I see. And did -- did -- was Mira relating to you a conversation she had had with Brad Lightcap? That's what it -- that's what this text says.

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: Did you speak to Brad Lightcap?

WITNESS SUTSKEVER: No.

ATTORNEY EDDY: Okay. So this information came only from Mira?

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: Did you -- did you seek to verify the information with Brad?

WITNESS SUTSKEVER: No.

ATTORNEY EDDY: You also write here at the bottom: [As Read] "Interestingly, it is my understanding that Greg has -- was essentially fired from Stripe as well." What was the basis for that allegation?

WITNESS SUTSKEVER: Mira told me.

ATTORNEY EDDY: Did you seek to verify it with Greg?

WITNESS SUTSKEVER: No.

ATTORNEY EDDY: Why not?

WITNESS SUTSKEVER: It didn't occur to me.

ATTORNEY EDDY: Why didn't it occur to you?

ATTORNEY AGNOLUCCI: Object to form.

WITNESS SUTSKEVER: I -- it just didn't.

ATTORNEY EDDY: Okay.

WITNESS SUTSKEVER: I thought that -- I fully believed the information that Mira was giving me.

ATTORNEY EDDY: If you go to page 531, this is the -- a page you reviewed with Mr. Molo earlier.

========
PAGE 306
========

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: It's the section entitled lying -- "Lying to Mira About Jason's Opinion About the DSB."

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: The screenshot -- am I right? The screenshots in this section all came from Mira?

WITNESS SUTSKEVER: Correct.

ATTORNEY EDDY: Did you -- and -- and there are references here to Jason. Obviously, Jason Kwon.

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: By the way, are you sure he was general counsel at the time?

WITNESS SUTSKEVER: I do not remember his title at the time.

ATTORNEY EDDY: Okay. Did you speak to Jason about --

WITNESS SUTSKEVER: No.

ATTORNEY EDDY: -- the Turbo matter?

WITNESS SUTSKEVER: No.

ATTORNEY EDDY: Do you know whether, in fact, Jason was disturbed by his discussions with Sam about this?

ATTORNEY AGNOLUCCI: Object to form.

WITNESS SUTSKEVER: I got this information from Mira, and I believed it.

ATTORNEY EDDY: Do you know whether GPT-4 Turbo actually went through the DSB?

WITNESS SUTSKEVER: I don't know.

ATTORNEY EDDY: And do you know whether Sam supported or opposed it going through the DSB?

WITNESS SUTSKEVER: I -- In hindsight, I realize that I didn't know it. But back then, I thought I knew it. But I knew it through secondhand knowledge.

ATTORNEY EDDY: I see. And you've since learned facts that the have changed your view?

WITNESS SUTSKEVER: No.

ATTORNEY EDDY: Okay.

WITNESS SUTSKEVER: Instead I've -- I've learned the critical importance of firsthand knowledge for matters like this.

ATTORNEY EDDY: Do you think it was a mistake to rely on secondhand knowledge?

========
PAGE 307
========

ATTORNEY AGNOLUCCI: Object to form.

WITNESS SUTSKEVER: I think secondhand knowledge can be very useful, but I think that secondhand knowledge is an invitation for further investigation.

ATTORNEY EDDY: For what?

WITNESS SUTSKEVER: For further -- investigation or exploration.

ATTORNEY EDDY: At a number of points in your document, you suggest that the reader or the board may want to talk to certain people.

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: And one of those, I think, is Bob McGrew; right?

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: And you suggest talking to Nick Ryder too; right?

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: Were those suggestions not followed through on?

WITNESS SUTSKEVER: I don't know.

ATTORNEY EDDY: Did you have any discussion with the other board members about following through on those suggestions?

WITNESS SUTSKEVER: No.

ATTORNEY EDDY: Okay. Can you turn to page 540, please.

WITNESS SUTSKEVER: Yeah.

ATTORNEY EDDY: This is -- sorry.

WITNESS SUTSKEVER: Okay.

ATTORNEY EDDY: This is the section entitled "Pitting People Against Each Other." Do you see that?

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: And turning on the next page, you see an example that's offered is "Daniela versus Mira"?

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: Is "Daniela" Daniela Amodei?

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: Who told that you Sam pitted Daniela against Mira?

WITNESS SUTSKEVER: Mira.

ATTORNEY EDDY: In the section below that where it says "Dario versus Greg, Ilya" --

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: -- you see that?

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: The complaint -- it says -- you say here that: [As Read] "Sam was not taking a firm position in respect of Dario wanting to run all of research at OpenAI to have Greg fired -- and to have Greg fired?" Do you see that?

WITNESS SUTSKEVER: I do see that.

========
PAGE 308
========

ATTORNEY EDDY: And "Dario" is Dario Amodei?

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: Why were you faulting Sam for Dario's efforts?

ATTORNEY AGNOLUCCI: Same objection to form. Relevance. Waste of time. Outside the scope of the magistrate's order.

WITNESS SUTSKEVER: So my recollection of what I wrote here is that I was faulting Sam for not accepting or rejecting Dario's conditions.

ATTORNEY EDDY: Did you think Dario's conditions were fair?

ATTORNEY AGNOLUCCI: Same objections.

WITNESS SUTSKEVER: I don't have precise enough knowledge of Dario's conditions, but my overall sense is that they were not fair and that Sam should've rejected them outright.

ATTORNEY EDDY: At page 542, you see there's a reference to Peter Welinder --

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: -- as a witness. Did you ever speak to him or did anybody else on the board speak with him about these matters?

WITNESS SUTSKEVER: Not to my knowledge.

ATTORNEY EDDY: And then at -- at 548, this is the -- the beginning of the Jakub story. Am I pronouncing that right? Jakub?

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: And you say: [As Read] "Involves Sam lying, undermining Mira, undermining Ilya, and pitting Jakub against Ilya. Joint work with Greg and Jakub?" What was the lying in the episode -- by Sam in the episode involving Jakub?

ATTORNEY AGNOLUCCI: Same objections to form. Relevance. Waste of time. Outside the scope of the magistrate's order.

WITNESS SUTSKEVER: Sam was telling me and Jakub conflicting things about the way the company would be run.

ATTORNEY EDDY: And you perceived that as lying?

ATTORNEY AGNOLUCCI: Object to form. We're now bordering on harassing the witness, and I am going to shut it down.

ATTORNEY EDDY: I don't think that's fair at all. This is squarely within the -- the examination that's been conducted already and -- and that is within the scope of the discovery that the plaintiffs have sought.

ATTORNEY AGNOLUCCI: Well, the reasons for Sam's termination have been made clear. These are granular details of examples. And with the limited time that we have and their very attenuated relevance to the termination, let alone to this lawsuit, I -- I think we're wasting time here and -- and we should move on.

ATTORNEY EDDY: I do not have a lot left -- so you'll be glad to hear that -- on this.

ATTORNEY EDDY: I want to turn to 564, please.

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: You see this is entitled "Subtle Retaliation in Response to Mira's Feedback."

WITNESS SUTSKEVER: 564, okay.

ATTORNEY EDDY: Yeah.

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: And there's discussion of Diane Yoon having been present for meetings with Mira and Sam. Do you see that?

WITNESS SUTSKEVER: Yes, I do.

========
PAGE 309
========

ATTORNEY EDDY: Did you speak to Diane Yoon about the events discussed in these pages?

WITNESS SUTSKEVER: No.

ATTORNEY EDDY: And -- and why not speak to any of these individuals who are named?

WITNESS SUTSKEVER: It didn't occur to me.

ATTORNEY EDDY: And you don't recall any discussion with any of the other board members about speaking with any of these individuals?

WITNESS SUTSKEVER: Correct.

ATTORNEY EDDY: And then could you look at page 570.

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: These are -- this is a screenshot of texts between Greg and Sam.

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: How did you get those?

WITNESS SUTSKEVER: I don't remember.

ATTORNEY EDDY: Did they come from Mira Murati?

WITNESS SUTSKEVER: Oh, I think they came from Mira Murati, yes.

ATTORNEY EDDY: Okay. And same with this screenshot that you reviewed with Mr. Molo earlier of Mira's -- this is at pages 565 and 566 -- of Mira's review of Sam?

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: Okay. During November 2023, did any board member receive outreach from Anthropic?

WITNESS SUTSKEVER: I don't have direct confirmation of that.

ATTORNEY EDDY: Do you have -- did you hear that anybody had gotten a reach-out from Anthropic?

ATTORNEY AGNOLUCCI: Object to form.

WITNESS SUTSKEVER: I don't -- I did not. I have heard speculation, but I have not heard anything definitive.

ATTORNEY EDDY: Do you know whether a proposal was made around that time for OpenAI to merge with Anthropic?

WITNESS SUTSKEVER: I do know that.

ATTORNEY EDDY: Tell me about that.

WITNESS SUTSKEVER: I don't -- I don't know whether it was Helen who reached out to Anthropic or whether Anthropic reached out to Helen. But they reached out with a proposal to be merged with OpenAI and take over its leadership.

ATTORNEY EDDY: When was that?

WITNESS SUTSKEVER: On Saturday.

ATTORNEY EDDY: Saturday, November 18th?

WITNESS SUTSKEVER: That must be the day.

ATTORNEY EDDY: The day -- was it short -- shortly after the removal of Sam and Greg?

WITNESS SUTSKEVER: Yes. It was before -- it was either on Saturday or on Sunday. It was not on Monday.

ATTORNEY EDDY: And how did you hear about that?

========
PAGE 310
========

WITNESS SUTSKEVER: Because there was a board call with Helen and the other board members where she told us about it. There has been a subsequent call with the leadership of Anthropic.

ATTORNEY EDDY: And were you present for that call?

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: What do you recall from that conversation?

WITNESS SUTSKEVER: I recall Anthropic expressing their excitement about it and expressing the issue -- the practical challenges that they would have with it.

ATTORNEY EDDY: Who from Anthropic was on that call?

WITNESS SUTSKEVER: I recall Dario Amodei on the call and Daniela Amodei. There would be at least one other person that I don't remember. Possibly more.

ATTORNEY EDDY: And what was your response to that?

WITNESS SUTSKEVER: I was very unhappy about it.

ATTORNEY EDDY: Why?

WITNESS SUTSKEVER: Because I really did not want OpenAI to merge with Anthropic.

ATTORNEY EDDY: Why not?

WITNESS SUTSKEVER: I just didn't want to.

ATTORNEY EDDY: And what about the other board members? Were they supportive?

WITNESS SUTSKEVER: They were a lot more supportive, yes.

ATTORNEY EDDY: Were all of them supportive?

WITNESS SUTSKEVER: I think -- at the very least, none were unsupportive.

ATTORNEY EDDY: Did anybody advocate for the merger?

WITNESS SUTSKEVER: I don't remember definitively.

ATTORNEY EDDY: Among the board members, who struck you as most supportive?

WITNESS SUTSKEVER: I would say my recollection is that Helen was the most supportive.

ATTORNEY EDDY: And what happened with the proposal?

WITNESS SUTSKEVER: I believe -- my recollection is that there were some practical obstacles that Anthropic has raised, and so the proposal did not continue.

ATTORNEY EDDY: Do you know what the practical obstacles were?

WITNESS SUTSKEVER: No.

ATTORNEY EDDY: How long did those discussions with Anthropic continue?

WITNESS SUTSKEVER: Extremely briefly.

ATTORNEY EDDY: A special committee of the board was formed to investigate the -- the -- the removal of Sam and Greg. Do you remember that?

WITNESS SUTSKEVER: I remember.

========
PAGE 311
========

ATTORNEY EDDY: Do you have any reason to doubt the independence of Bret Taylor and Larry Summers?

WITNESS SUTSKEVER: No.

ATTORNEY AGNOLUCCI: Object to form.

WITNESS SUTSKEVER: Nothing to my knowledge.

ATTORNEY EDDY: They -- they hired a law firm, WilmerHale, to conduct the investigation?

ATTORNEY AGNOLUCCI: Objection.

ATTORNEY EDDY: Do you remember that?

WITNESS SUTSKEVER: I remember --

ATTORNEY AGNOLUCCI: Lacks foundation. Calls for speculation.

WITNESS SUTSKEVER: I remember they hired a law firm. I don't remember its name.

ATTORNEY EDDY: Okay. Did -- did they interview you?

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: And do you have any reason to question the integrity of the investigation that was undertaken?

ATTORNEY AGNOLUCCI: Objection. Lacks foundation.

WITNESS SUTSKEVER: At this point, I was too removed from those procedures.

ATTORNEY EDDY: So you just can't evaluate one way or the other?

WITNESS SUTSKEVER: Correct.

ATTORNEY EDDY: Okay. I want to show you one more exhibit for now.

ATTORNEY EDDY: Actually, I think it's been -- you have it. Exhibit 20 -- I'm sorry -- in front of you.

WITNESS SUTSKEVER: 20?

ATTORNEY EDDY: This is the article you reviewed with Mr. Molo.

WITNESS SUTSKEVER: Yeah.

ATTORNEY EDDY: I need our copy.

WITNESS SUTSKEVER: Okay. Which number did you say it is?

ATTORNEY EDDY: 20. It looks like this.

WITNESS SUTSKEVER: Yeah. I remember how it looks like. I'm having trouble finding it.

ATTORNEY EDDY: Do you want me to just --

ATTORNEY AGNOLUCCI: Is it under the --

WITNESS SUTSKEVER: Doubtful.

ATTORNEY EDDY: If you turn to the page ending in 1442. You see at the very bottom of the page, it

ATTORNEY EDDY: I can hand you another copy.

WITNESS SUTSKEVER: Oh, here it is. Here it is. I found it.

ATTORNEY EDDY: Great. Did you speak to the reporter, Keach Hagey, in connection with this article?

WITNESS SUTSKEVER: No.

ATTORNEY EDDY: Do you know who did?

WITNESS SUTSKEVER: No.

ATTORNEY EDDY: If you turn to the page ending in 1442. You see at the very bottom of the page, it says: [As Read] "Sutskever had been waiting for a moment when the board dynamics would allow for Altman to be replaced as the CEO."

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: Is that -- is that correct?

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: And what were the dynamics you were waiting for?

WITNESS SUTSKEVER: That the majority of the board is not obviously friendly with Sam.

ATTORNEY EDDY: And when -- when did that happen?

========
PAGE 312
========

WITNESS SUTSKEVER: When someone -- there was a sequence of rapid departures from the board for different reasons. I don't remember what they were. I don't remember who exactly left, but that's what it's referring to.

ATTORNEY EDDY: So for -- for how long had you been planning to propose removal of Sam?

WITNESS SUTSKEVER: For some time. I mean, "planning" is the wrong word because it didn't seem feasible.

ATTORNEY EDDY: It didn't seem feasible?

WITNESS SUTSKEVER: It was not feasible prior; so I was not planning.

ATTORNEY EDDY: How -- how long had you been considering it?

WITNESS SUTSKEVER: At least a year.

ATTORNEY AGNOLUCCI: Object to form. Relevance. Waste of time. Scope of magistrate judge's order.

WITNESS SUTSKEVER: That's it.

ATTORNEY EDDY: You said at least a year?

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: Okay. And then if you can turn to page 1444, again, near the bottom of the page, you see it says --

WITNESS SUTSKEVER: Yes.

ATTORNEY EDDY: [As Read] "Sutskever was astounded. He had expected the employees of OpenAI to cheer." Is that true?

WITNESS SUTSKEVER: I had not expected them to cheer, but I have not expected them to feel strongly either way.

ATTORNEY EDDY: And why is that?

WITNESS SUTSKEVER: That's what I thought.

ATTORNEY EDDY: Okay. I'm going to reserve the remainder of my time, but I'll pass the witness to Microsoft.

WITNESS SUTSKEVER: Okay.

ATTORNEY EDDY: Thank you for your time.

WITNESS SUTSKEVER: Thank you.

ATTORNEY COHEN: Can we go off the record.

ATTORNEY AGNOLUCCI: How much time do we have left?

THE VIDEOGRAPHER: Going off the record at 7:14 PM.

(Discussion off the record.)

THE VIDEOGRAPHER: And we're back on the record at 7:16 PM.

========
PAGE 349
========

WITNESS SUTSKEVER: I don't have an affirmative recollection.

ATTORNEY MOLO: Okay. Is it your belief that Sam Altman will have a financial interest in OpenAI someday?

ATTORNEY EDDY: Objection.

WITNESS SUTSKEVER: I recall reading about it in the news, but I don't know how accurate it is.

ATTORNEY MOLO: Okay. When you left OpenAI, you resigned in May of 2024; is that right?

WITNESS SUTSKEVER: I don't remember, but it sounds within the range.

ATTORNEY MOLO: Why did you leave?

ATTORNEY AGNOLUCCI: Object to form.

WITNESS SUTSKEVER: Ultimately, I had a big new vision, and it felt more suitable for a new company.

ATTORNEY MOLO: Okay. At the time immediately prior to your departure of OpenAI, did you have an equity stake in the company?

WITNESS SUTSKEVER: Yes.

ATTORNEY MOLO: What do you believe value of that equity stake was at the time you left?

ATTORNEY AGNOLUCCI: Same objection. Same instruction as before not to answer.

ATTORNEY MOLO: Well, this is directly relevant. This is his interest in a defendant in the case. This is his interest -- his financial interest. This is -- this is -- clearly goes to the issue of interest and bias.

ATTORNEY AGNOLUCCI: I have the same instruction not to answer.

ATTORNEY AGNOLUCCI: I'm telling him not to answer.

ATTORNEY MOLO: -- or his financial -- let me finish. Let me finish. We've heard a lot of you today; so let me finish. You're instructing the witness who has a financial interest in the defendant in the case -- and the defendant is being sued for great sum of money here -- not to answer -- not to answer the question; is that correct?

========
PAGE 350
========

ATTORNEY MOLO: I just want to make sure that it's clear on record.

ATTORNEY AGNOLUCCI: I'm instructing him not to answer the amount of his financial interest. He can answer whether he has an interest, but he cannot quantify it.

ATTORNEY MOLO: Okay. Do you still have a financial interest in OpenAI?

WITNESS SUTSKEVER: Yes.

ATTORNEY MOLO: Okay. Has the amount of the interest increased or decreased since you've left OpenAI?

WITNESS SUTSKEVER: Increased.

ATTORNEY MOLO: Okay. And has the value of the interest increased or decreased since you got --

WITNESS SUTSKEVER: Sorry. Can you repeat the previous question.

ATTORNEY MOLO: Okay. So -- okay. Okay. Has the value of your interest in OpenAI increased or decreased since you left OpenAI?

WITNESS SUTSKEVER: Increased.

ATTORNEY MOLO: Okay.

ATTORNEY AGNOLUCCI: Same instruction not to answer.

ATTORNEY AGNOLUCCI: He's not answering the question.

ATTORNEY MOLO: Are you going to not answer?

ATTORNEY AGNOLUCCI: He's following my instruction.

ATTORNEY MOLO: Would you please allow the witness to speak.

ATTORNEY AGNOLUCCI: I am allowing the witness to speak.

ATTORNEY MOLO: No, you're not.

ATTORNEY AGNOLUCCI: Be courteous and respectful.

THE COURT REPORTER: The court reporter is noting for the record that it cannot be taken due to simultaneous cross-talk. Once counsel can continue with decorum in a professional manner, the record will resume.

Let's take five.

ATTORNEY MOLO: Yeah, don't raise your voice.

ATTORNEY AGNOLUCCI: I'm tired of being told that I'm talking too much.

ATTORNEY MOLO: Well, you are.

ATTORNEY AGNOLUCCI: Check yourself.

========
PAGE 351
========

ATTORNEY MOLO: Did --

You announced your current company super -- Safe Superintelligence in 2024?

WITNESS SUTSKEVER: Yes.

ATTORNEY MOLO: What's the purpose of that -- that company?

WITNESS SUTSKEVER: To do a new and different kind of research.

ATTORNEY MOLO: Okay. And what does that mean exactly?

WITNESS SUTSKEVER: I have a new idea of how to do things, and I want to try and do them.

ATTORNEY MOLO: Okay. Who is paying your legal fees for this --

ATTORNEY AGNOLUCCI: Objection.

ATTORNEY MOLO: -- for this deposition?

ATTORNEY AGNOLUCCI: And instruction not to answer.

ATTORNEY MOLO: Are you --

WITNESS SUTSKEVER: I've been instructed not to answer.

ATTORNEY MOLO: Okay. Did you discuss your deposition with anyone other than your lawyers prior to today?

WITNESS SUTSKEVER: No.

ATTORNEY MOLO: When was the last time you spoke with Sam Altman?

WITNESS SUTSKEVER: A while ago. Maybe ten months ago, a year ago. Something like this.

ATTORNEY MOLO: Have you ever discussed this lawsuit with Sam Altman?

WITNESS SUTSKEVER: No.

ATTORNEY MOLO: I'm sorry?

WITNESS SUTSKEVER: No.

ATTORNEY MOLO: Okay. Have you ever -- when was the last time you spoke with Greg Brockman?

WITNESS SUTSKEVER: Maybe a year and a quarter ago.

ATTORNEY MOLO: Okay. Have you discussed this lawsuit with Greg Brockman?

WITNESS SUTSKEVER: No.

ATTORNEY AGNOLUCCI: I'm going to withdraw the objection about who's paying the legal fees. I think he can actually answer that.

ATTORNEY MOLO: Okay. Who is paying your legal fees?

========
PAGE 352
========

ATTORNEY AGNOLUCCI: If he knows.

WITNESS SUTSKEVER: I -- I -- I'm not sure.

ATTORNEY MOLO: Okay. Okay.

WITNESS SUTSKEVER: I have a guess, but I'm not 100 percent sure.

ATTORNEY MOLO: How did you come to retain counsel?

ATTORNEY AGNOLUCCI: That, I'm going to instruct him not to answer.

ATTORNEY MOLO: I'm sure you are.

ATTORNEY MOLO: How -- how did you come to retain counsel?

WITNESS SUTSKEVER: I --

ATTORNEY AGNOLUCCI: Instruction not to answer. Do you mean in connection with this lawsuit?

ATTORNEY MOLO: Correct.

ATTORNEY MOLO: I'm not asking you for privileged communications between you and your lawyer. I'm just asking you how you came to retain these lawyers.

ATTORNEY AGNOLUCCI: For this lawsuit, you can answer.

WITNESS SUTSKEVER: I've been -- So how did it happen? I recall -- I don't recall exactly. I believe I started working with a different lawyer in Willkie. And I'm pretty sure that -- I think my -- my then-girlfriend found Simona, and I reached out to Simona. That's my recollection.

ATTORNEY MOLO: Okay. Have you received any bills for legal fees?

ATTORNEY AGNOLUCCI: And you're asking about this litigation?

ATTORNEY MOLO: Yeah.

WITNESS SUTSKEVER: No.

ATTORNEY MOLO: You -- "No," you haven't?

WITNESS SUTSKEVER: No.

ATTORNEY MOLO: Okay. Is OpenAI paying your legal fees?

WITNESS SUTSKEVER: I think that's probably the case.

ATTORNEY MOLO: Okay. What makes you think that?

WITNESS SUTSKEVER: Because I don't know who else it would be.

ATTORNEY MOLO: Okay. Did somebody at OpenAI tell you to come meet with these lawyers and hire them --

WITNESS SUTSKEVER: No. No.

ATTORNEY MOLO: Have you discussed this lawsuit with anyone else at OpenAI -- excuse me, with anyone at OpenAI since it was filed?

WITNESS SUTSKEVER: No.

ATTORNEY MOLO: Did you discuss this deposition with anyone at OpenAI?

WITNESS SUTSKEVER: No.

ATTORNEY MOLO: Have you discussed this lawsuit with anyone who's a representative of OpenAI?

WITNESS SUTSKEVER: No.

ATTORNEY MOLO: Give me just a couple of minutes, and we might be done. We can go off the record.

THE VIDEOGRAPHER: Going off the record at 7:53 PM.

(A break was taken.)

THE VIDEOGRAPHER: And we're back on the record at 8:01 PM.

ATTORNEY MOLO: Okay.

========
PAGE 353
========

ATTORNEY MOLO: Ilya, you testified a few moments ago that you believe that OpenAI may be paying your legal fees; is that right?

WITNESS SUTSKEVER: Yes.

ATTORNEY MOLO: Okay. Are you deriving any other financial benefits from OpenAI at this time?

WITNESS SUTSKEVER: Nothing beyond what you've already mentioned.

ATTORNEY MOLO: Okay. And then other than the instructions from your lawyer here today, do you believe there's any reason you cannot disclose the details of your financial interest in OpenAI?

WITNESS SUTSKEVER: No.

ATTORNEY AGNOLUCCI: Object to form.

ATTORNEY MOLO: Okay.

ATTORNEY AGNOLUCCI: Object to form.

ATTORNEY EDDY: Objection.

WITNESS SUTSKEVER: I was not aware.

ATTORNEY COHEN: That has a protective order. You should withdraw the question.

ATTORNEY EDDY: That's a violation of the protective order.

ATTORNEY MOLO: Well, that was certainly not my intention. I'll withdraw the question. Anything else?

No further questions. Thank you very much, Ilya.

WITNESS SUTSKEVER: Okay.

ATTORNEY MOLO: We have issues, though, in terms of the deposition remaining open because there's documents that have not been produced to us. We have the issue of the Brockman report, which we learned of today, and you possess it, and it's not been produced. And we have these issues about, you know, Ilya's financial interest.

So this -- we're not done.

ATTORNEY AGNOLUCCI: We obviously disagree. As for the Brockman memo, it was referenced in one of the exhibits that you used in your questioning. You've known about it for some time. And perhaps you should talk to your colleague who handled the meet-and-confer because the memo was never requested. You all were aware that it existed. You didn't ask for it, and it's not called for by the requests.

We've had extensive meet-and-confers about what we were and weren't producing. So if you wanted the memo, you should've asked for it before today.

But it's clearly referenced in both the exhibit that you brought today and in public reporting, and the fact that you didn't know about it before today is false.

ATTORNEY MOLO: No.

ATTORNEY AGNOLUCCI: In fact, you're the one who mentioned it today for the first time.

ATTORNEY MOLO: We mentioned it. You're the one that said you withheld it. We'll take it up with the court.

ATTORNEY AGNOLUCCI: I don't know what you mean by --

ATTORNEY MOLO: We're -- we're taking it up with the court; right?

ATTORNEY AGNOLUCCI: I'm -- well, we should have a record here and we should have a meet-and-confer because we're not agreeing to bring the witness back.

Have you spoken with your colleague Jimmy who handled the meet-and-confer about what we were and weren't producing about this memo?

ATTORNEY MOLO: I think we're happy to talk to colleagues. We're happy to talk to you again. We're not done with the deposition.

ATTORNEY AGNOLUCCI: I think you should talk to your colleagues because we had extensive meet-and-confer --

ATTORNEY MOLO: I just told you. I just told you --

ATTORNEY AGNOLUCCI: Excuse me.

ATTORNEY MOLO: -- that we are happy to talk to our colleagues. All right?

ATTORNEY AGNOLUCCI: We had extensive meet-and-confer about what we would and wouldn't produce. The memo was not called for. You never asked for it. You were aware of it. And we didn't agree to produce it because you didn't ask for it.

ATTORNEY MOLO: We disagree.

ATTORNEY AGNOLUCCI: But we're not bringing the witness back.

ATTORNEY MOLO: We'll see what the court says.

ATTORNEY EDDY: I have a matter to put on the record, which is that it has come to our attention that many of the exhibits that plaintiffs have shown the witness have been prehighlighted. For the record, none of the highlighting appearing in the documents that are marked as exhibits is native to those documents.

ATTORNEY MOLO: Correct.

ATTORNEY EDDY: We did not see a copy of the highlighting that was handed to the witness. We got unhighlighted copies handed out to us during the deposition.

So we reserve all rights, and we note that for the record.

ATTORNEY COHEN: And counsel for Microsoft joins that objection.

ATTORNEY AGNOLUCCI: And can -- just to clarify your comment about financials. Do you mean financial documents?

ATTORNEY MOLO: I don't know what you're talking about.

ATTORNEY AGNOLUCCI: You had an objection on the basis of documents that weren't produced, and you said something about financials.

ATTORNEY MOLO: Oh, no. The question's to Ilya about his financial interest in OpenAI.

ATTORNEY AGNOLUCCI: Understood. But you're not talking about any documents.

ATTORNEY MOLO: I don't know if there are documents that relate to that.

ATTORNEY AGNOLUCCI: Well, they -- they weren't within the scope of the subpoena called for or the subject of any --

ATTORNEY MOLO: We'll take it up with the court. That's why there's courts.

ATTORNEY AGNOLUCCI: And so last thing, actually. Exhibit 19. In accordance with the parties' agreement, we are going to keep the marked copy of Exhibit 19. If there are other copies of Exhibit 19, the agreement was that, I believe, the parties can keep it for three days and then have to return it. But we're happy to take the copies now.

But, I guess, so that we have a clear record, who has a copy of Exhibit 19?

ATTORNEY EDDY: We have a copy, and we want to meet and confer with you about the use of that. So let's put a pin in that if we could.

ATTORNEY AGNOLUCCI: Okay. And, Mr. Molo, do you all have a copy of Exhibit 19?

ATTORNEY SCHUBERT: We have three copies. We will provide them to you.

ATTORNEY AGNOLUCCI: Okay.

ATTORNEY TOFIGHBAKHSH: You have one. They have one of ours. The witness has the other one. So...

ATTORNEY AGNOLUCCI: I have one that's marked as Exhibit 19. I have one that was handed to us, and then --

ATTORNEY SANTACANA: The third one is down there somewhere.

ATTORNEY AGNOLUCCI: Counsel --

(Indiscernible cross-talk.)

(Admonition by the court reporter.)

ATTORNEY AGNOLUCCI: Okay. Excellent.

ATTORNEY SAVITT: Can I ask what the status of the -- of the confidentiality the transcript is. Has anyone designated --

ATTORNEY AGNOLUCCI: We designated it at the opening of the deposition as attorneys' eyes only, highly confidential under the protective order.

ATTORNEY SAVITT: Thank you.

ATTORNEY SANTACANA: Don't you each need to do that for yourself? That's for ourselves.

 

Discuss

Published on November 2, 2025 7:57 PM GMT

epistemic status: toy model, not very novel

Iain McGilchrist has written about the differences between the left-hemisphere brain and the right-hemisphere brain in The Master and His Emissary (as well as in other works). The book opens with caveats: this is a clustering approach, not a clean division. The claim, as I understand it, is that the left hemisphere is primarily serial processing and the right brain is primarily parallel processing. These clusters are elaborated upon (via many empirical citations as well as reasoned inferences); amongst other things, serial processing is said to be detail-oriented, while parallel processing is said to be whole-oriented.

The idea of the current essay doesn't depend on the left/right thesis, nor on the thesis of distinct clusters, but only on the idea that there is a spectrum from highly serial to highly parallel (a spectrum which is liable to be correlated with other important dimensions of cognition). I am combining this with ideas from Global Workspace Theory. On the whole, the proposal is not very novel, but I think it is a fun picture with some truth to it.

The idea is as follows:

• There are a whole bunch of highly parallel computations happening in the brain.
• The information from these parallel computations gets progressively compressed/summarized ("fan-in" computations).
• These summaries serve as the main inputs to serial processing ("conscious stream-of-thought").
• Serial processing results in a relatively small amount of globally important information that is then transmitted out to many parts of the brain ("fan-out" computations).

The topology of the donut is not part of the claim here, but the metric is. I could have drawn a single circle to represent the information cycle. What I like about the donut picture is that the donut hole is smaller than the outer rim of the donut. This represents the idea of consciousness as an information bottleneck: we can only consciously attend to so much information at once. 

I call the donut hole "the serial bottleneck" although this term does not appear to be quite standard. Common terms include the global workspace and the stream of consciousness.

Explicitness

The information which passes through the serial bottleneck is more "explicit" in the sense that it is more communicable, and more "symbolic" in the sense that it compresses larger states and has been optimized for usefulness in serial computations (symbol manipulation). As such, we could call the parallel processing "implicit" by contrast.

If we think of the information in the serial bottleneck as an explicit representation, we can ask: what does it represent?

• One angle on this is verbal reports; we could 'define' the meaning of an explicit representation through the linguistic behaviors it induces.
• Another angle on the meaning of explicit representations is to think about the implicit states which lead to a given explicit representation; this fits with the summary/compression idea mentioned earlier.
• A third perspective is to ask what the explicit representation leads to, looking at the fan-out computations which transmit the explicit representation out to the rest of the brain. This allows us to view the explicit representation as imperative (telling the rest of the brain to do something), or even as a query (asking the rest of the brain to compute something, which is then brought back in to the next round of serial processing).

Extended Mind

One thing I like about this picture is how easily it can apply to systems other than just the brain. For example, a person writing in a notebook is using the notebook as an extra, even-more-constrained serial bottleneck: to record something in the notebook, they have to use the bottleneck of their hand to write it down. What they've written is then presented to the rest of their mind via fan-out from their eyes. Writing something in a notebook is clearly "even more explicit" than consciously thinking it.

Similarly, a group of people talking: their individual brains form the parallel part, while their words are the serial bottleneck.

This allows us to answer questions like "what would it mean for a society to meditate?". Note that I do not mean every person in the society meditating individually. I mean: what is to a society what meditation is for a single mind?

If meditation is conceived of as a deliberate attempt to focus attention on a single subject matter (such as the breath): for a society to meditate would be for all the explicit communication channels (the media) to focus on one target. Perhaps the meditation is most successful if everyone is talking about the target subject matter as well.

If meditation is conceived of as "emptying the mind" (not paying attention to anything; halting the stream of conscious thoughts; avoiding putting content in the serial bottleneck) then it would correspond to turning off the media. Perhaps the meditation is most successful if people stop talking as well.

Discuss

Published on November 2, 2025 7:42 PM GMT

I think that we reason too much about AI specifically, and not enough about intelligence more broadly.

This is a fairly obvious criticism. There are ways that biological and artificial intelligence are clearly different.

However, most AI discussion is largely upstream of biological-artificial differences, and so the discussion should be about intelligence, not AI.

Two areas where I believe this focus on AI is actively degrading reasoning are discussions of alignment and communication.

Alignment

Alignment is about agent-agent synchronized values, not human-AI synchronized values. The vast majority of agent-agent interactions that we have available to draw conclusions from are interactions where neither agent is AI. Instead they are between human-based entities, humans, other animals, or between two different types of these agents.

People have already made this point. But I'm pretty sad to see that it mostly hasn't caught on yet. When people talk about aligning ASI, they're usually not really talking about ASI, they're just talking about SI; most ASI discussion applies to biological superintelligences.

Unlike ASI, some forms of biological superintelligence already exist and have for a long time, and we call them corporations, nation states, and other human organizations. There has been some alignment-oriented study of these entities but way less than I'd like, especially between entities differing significantly in intellectual capability. For example: Individuals almost always lose when they go against major corporations. The way this usually plays out is one incredibly large and well-paid team of lawyers hired by the corporation going against a much smaller and poorer team hired by the individual. This is analogous to human-ASI interactions. Of course, human-based entities are superintelligent in a different way than ASI probably will be, but I think that difference is irrelevant in many discussions involving ASI.

Communication

I enjoyed this recent post about why humans communicating via LLM-generated text is bad. I agree that this is bad, but think the argument against it is much stronger as a specific case of general bad agent-agent communication patterns, instead of mostly LLM-specific arguments. Here is that more general argument, examining long quotes and lying.

Relying on long quotes from other agents seems bad whether or not you're quoting an LLM. The point of discussion is to engage, not to act as an intermediary between two other agents. LLMs, and especially past humans, don't have the full context for the current discussion. Link or briefly quote other agents' views, but only as a supplement to your own.

If an LLM says "I enjoy going on long walks" or "I don't like the taste of coffee", it is obviously lying because LLMs do not have access to those experiences or sensations. But a human saying those things might also be lying, you just can't tell quite as easily. There is nothing wrong about an LLM saying these things other than the wrongness of lying, as with humans.

If an LLM gives a verifiable mathematical proof, it is very easy to tell whether or not it's lying, which you do in exactly the same way you would if a human presented the same proof.

I think the argument against communicating via LLM-generated text hits harder as a general, agent-agnostic examination of long quotes and lying and why they're bad.

The linked post additionally argues that LLMs are always lying when they say "I think.." or "I believe.." (just like they're lying by claiming to go on long walks or taste coffee). As someone who disagrees with only that latter argument, this framing also makes the point of disagreement clearer.

Conclusion

There are certainly times where the specific "shape" of AI (easier self-improvement, copyable over shorter time scales, requiring significantly different resources) does matter, and that shape is why there is so much more discussion about AI than, say, gene editing or selective breeding.

But the current base assumption seems to be that differences in shape between artificial and biological intelligence matter to the discussion of <current topic>. I think this is usually false, this false assumption is degrading reasoning, and a justification of the impact of differing shapes should be given per topic, if one believes that the differences are impactful.

Discuss

Published on November 2, 2025 7:24 PM GMT

There is a temptation to simply define Goodness as Human Values, or vice versa.

Alas, we do not get to choose the definitions of commonly used words; our attempted definitions will simply be wrong. Unless we stick to mathematics, we will end up sneaking in intuitions which do not follow from our so-called definitions, and thereby mislead ourselves. People who claim that they use some standard word or phrase according to their own definition are, in nearly all cases outside of mathematics, wrong about their own usage patterns.[1]

If we want to know what words mean, we need to look at e.g. how they’re used and where the concepts come from and what mental pictures they summon. And when we look at those things for Goodness and Human Values… they don’t match. And I don’t mean that we shouldn’t pursue Human Values; I mean that the stuff people usually refer to as Goodness is a coherent thing which does not match the actual values of actual humans all that well.

The Yumminess You Feel When Imagining Things Measures Your Values

There’s this mental picture where a mind has some sort of goals inside it, stuff it wants, stuff it values, stuff which from-the-inside feels worth doing things for. In old-school AI we’d usually represent that stuff as a utility function, but we wanted some terminology for a more general kind of “values” which doesn’t commit so hard to the mathematical framework (and often-confused conceptual baggage outside the math) of utility functions. The phrase “human values” caught on.

We don’t really know what human values are, or what shape they are, or even whether they’re A Thing at all. We don’t have trivial introspective access to our own values; sometimes we think we value a thing a lot, but realize in hindsight that we value it only a little. But insofar as the mental picture is pointing to a real thing at all, it does tell us how to go look for our values within our own minds.

How do we go look for our own values?

Well, we’re looking for some sort of goals, stuff which our minds want or value, stuff which drives us, etc. What does that feel like from the inside? Think of the stuff that, when you imagine it, feels really yummy. It induces yearning and longing. It feels like you’d be more complete with it. That’s the feeling of stuff that you value a lot. Lesser versions of the same feeling come when imagining things you value less (but still positively).

Personally… I get that feeling of yumminess and yearning when I imagine having a principled mathematical framework for understanding the internal structures of minds, which actually works on e.g. image generators.[2] I also get that feeling of yumminess and yearning when I imagine a really great night of dancing, or particularly great sex, or physically fighting with friends, or my favorite immersive theater shows, or some of my favorite foods at specific restaurants. Sometimes I get a weaker version of the yumminess and yearning feeling when I imagine hanging out around a fire with friends, or just sitting out on my balcony alone at night and watching the city, or dealing with the sort of emergency which is important enough that I drop everything else from my mind and just focus

Those are my values. That’s what human values look like, and how to probe for yours.

“Goodness” Is A Memetic Egregore

I did not first learn about goodness by imagining things and checking how yummy they felt. I first learned about Goodness by my parents and teachers and religious figures and books and movies and so forth telling me that it’s Good to not steal things, Good to do unto others what I’d have them do unto me, Good to follow rules and authority figures, Good to clean up after myself, Good to share things with other kids, Good to not pick my nose, etc, etc.

In other words, I learned about Goodness mostly memetically, absorbing messages from others about what’s Good.

Some of those messages systematically follow from some general principles. Things like “don’t steal” are social rules which help build a high-trust society, making it easier for everyone to get what they want insofar as everyone else follows the rules. We want other people to follow those rules, so we teach other people the rules. Other aspects of Goodness, especially about cleanliness, seem to mostly follow humans’ purity instincts, and are memetically spread mainly by people with relatively-strong purity instincts in an attempt to get people with relatively-weaker purity instincts to be less gross (think nose picking). Still other aspects of Goodness seem rather suspiciously optimized for getting kids to be easier for their parents and teachers to manage - think following rules or respecting one’s elders. Then there are aspects of Goodness which seem to be largely political, driven by the usual political memetic forces.

The main unifying theme here is that Goodness is a memetic egregore; in practice, our shared concept of Goodness is comprised of whatever messages people spread about what other people should value.

… which sure is a different thing from what people do value, when they introspect on what feels yummy.

Aside: Loving Connection

One thing to flag at this point: you know the feeling of deep loving connection, like a parent-child bond or spousal bond or the feeling you get (to some degree) when deeply empathizing with someone or the feeling of loving connection to God or the universe which people sometimes get from religious experiences? I.e. oxytocin?

For many (most?) people, that feeling is a REALLY big chunk of their Values. It is the thing which feels yummiest, often by such a large margin that it overwhelms everything else. If that’s you, then it’s probably worth stopping to notice that there are other things you value. It is quite possible to hyperoptimize for that one particular yumminess, then burn out and later realize that one values other things too - as many a parent learns when the midlife crisis hits.

That feeling of deep loving connection is also a major component of the memetic egregore Goodness, to such an extent that people often say that Goodness just is that kind of love. Think of the songs or hippies or whoever saying that all the world’s problems would be solved if only we had more love. As with values, it is worth stopping to notice that loving connection is not the entirety of Goodness, as the term is typically used. The people saying that Goodness just is loving connection (or something along those lines) are making the same move as someone trying to define a word; in most cases their usage probably doesn’t even match their own definition on closer inspection.

It is true that deep loving connection is both an especially large chunk of Human Values and an especially large chunk of Goodness, and within that overlap Human Values and Goodness do match. But that’s not the entirety of either Human Values or Goodness, and losing track of the rest is a good way to shoot oneself in the foot eventually.

We Don’t Get To Choose Our Own Values (Mostly)

To summarize so far:

• Our Values are (roughly) the yumminess or yearning we feel when imagining something.
• Goodness is (roughly) whatever stuff the memes say one should value.

Looking at that first one, the second might seem kind of silly. After all, we mostly don’t get to choose what triggers yumminess or yearning. There are some loopholes - e.g. sometimes we can learn to like things, or intentionally build new associations - but mostly the yumminess is not within conscious control. So it’s kind of silly for the memetic egregore to tell us what we should find yummy.

A central example: gay men mostly don’t seem to have much control over their attraction to men; that yumminess is not under their control. In many times and places the memetic egregore Goodness said that men shouldn’t be sexually attracted to men (those darn purity instincts!), which… usually isn’t all that effective at changing the underlying yumminess or yearning.

What does often happen, when the memetic egregore Goodness dictates something in conflict with actual Humans’ actual Values, is that the humans “tie themselves in knots” internally. The gay man’s attraction to men is still there, but maybe that attraction also triggers a feeling of shame or social anxiety or something. Or maybe the guy just hides his feelings, and then feels alone and stressed because he doesn’t feel safe being open with other people.

Sex and especially BDSM is a ripe area for this sort of thing. An awful lot of people, probably a majority of the population, sure do feel deep yearning to either inflict or receive pain, to take total control over another or give total control to another, to take or be taken by force, to abandon propriety and just be a total slut, to give or receive humiliation, etc. And man, the memetic egregore Goodness sure does not generally approve of those things. And then people tie themselves in knots, with the things that turn them on most also triggering anxiety or insecurity.

So What Do?

I’d like to say here “screw memetic egregores, follow the actual values of actual humans”, but then many people will be complete fucking idiots about it. So first let’s go over what not to do.

There’s a certain type of person… let’s call him Albert. Albert realizes that Goodness is a memetic egregore, and that the memetic egregore is not particularly well aligned with Albert’s own values. And so Albert throws out all that Goodness crap, and just queries his own feelings of yumminess in-the-moment when making decisions.

This goes badly in a few different ways

Sometimes Albert has relatively low innate empathy, and throws out all the Goodness stuff about following the rules and spirit of high-trust communities. Albert just generally hits the “defect” button whenever it’s convenient. Then Albert goes all pikachu surprise face when he’s excluded from high trust communities.

Other times Albert is just bad at thinking far into the future, and jumps on whatever feels yummy in-the-moment without really thinking ahead. A few years down the line Albert is broke.

Or maybe Albert rejects memetic Goodness, ignores authority a little too much, and winds up unemployed or in prison. Or ignores purity instincts a little too much and winds up very sick.

Point is: there’s a Chesterton’s fence here. Don’t be an idiot. Goodness is not very well aligned with actual Humans’ actual Values, but it has been memetically selected for a long time and you probably shouldn’t just jettison the whole thing without checking the pieces for usefulness. In particular, a nontrivial chunk of the memetic egregore Goodness needs to be complied with in order to satisfy your actual Values long term (which usually involves other people), even when it conflicts with your Values short term. Think about the consequences, what will actually happen down the line and how well your Values will actually be satisfied long-term, not just about what feels yummy in the moment.

… and then jettison the memetic egregore and pay attention to your and others' actual Values. Don’t make the opposite mistake of motivatedly looking for clever reasons to not jettison the egregore just because it’s scary.

1. ^

   You can quick-check this in individual cases by replacing the defined word with some made-up word wherever the person uses it - e.g. replace “Goodness” with “Bixness”.

2. ^

   … actually when I first try to imagine that I get a mild “ugh” because I’ve tried and failed to make such a thing before. But when I set that aside and actually imagine the end product, then I get the yummy feeling.

Discuss

Published on November 2, 2025 6:19 PM GMT

This is just a simple idea that came to me, maybe other people found it earlier, I'm not sure.

Imagine two people, Alice and Bob, wandering around London. Bob's goal is to get to the Tower Bridge. When he gets there, he'll get a money prize proportional to the time remaining until midnight, multiplied by X pounds per minute. He's also carrying a radio receiver.

Alice is also walking around, doing some chores of her own which we don't need to be concerned with. She is carrying a radio transmitter with a button. If/when the button is pressed (maybe because Alice presses it, or Bob takes it from her and presses it, or she randomly bumps into something), Bob gets notified that his goal changes: there'll be no more reward for getting to Tower Bridge, he needs to get to St Paul's Cathedral instead. His reward coefficient X also changes: the device notes Bob's location at the time the button is pressed, calculates the expected travel times to Tower Bridge and to St Paul's from that location, and adjusts X so that the expected reward at the time of the button press remains the same. For example, if there's 1 hour remaining until midnight and Bob is 20 minutes away from the bridge and 30 minutes away from the cathedral, then X gets multiplied by 4/3.

I think this can serve as a toy model of corrigibility, with Alice as the "operator" and Bob as the "AI". It's clear enough that Bob has no incentive to manipulate the button at any point, but actually Bob's indifference goes even further than that. For example, let's say Bob can sacrifice just a minute of travel time to choose an alternate route, one which will take him close to both Tower Bridge and St Paul's, to prepare for both eventualities in case Alice decides to press the button. Will he do so? No. He won't spare even one second. He'll take the absolute fastest way to Tower Bridge, secure in the knowledge that if the button gets pressed while he's on the move, the reward will get adjusted and he won't lose anything.

We can also make the setup more complicated and the general approach will still work. For example, let's say traffic conditions change unpredictably during the day, slowing Bob down or speeding him up. Then all we need to say is that the button does the calculation at the time it's pressed, taking into account the traffic conditions and projections at the time of button press.

Are we unrealistically relying on the button having magical calculation abilities? Not necessarily. Formally speaking, we don't need the button to do any calculation at all. Instead, we can write out Bob's utility function as a big complicated case statement which is fixed from the start: "if the button gets pressed at time T when I'm at position P, then my reward will be calculated as..." and so on. Or maybe this calculation is done after the fact, by the actuary who pays out Bob's reward, knowing everything that happened. The formal details are pretty flexible.

Is this relevant to AI corrigibility in real life? Well, it's a toy model. And even in this model, the utility formula can get quite complicated. But now it feels to me that any solution to corrigibility would have to do something like this: switching from utility function U1 to U2 by calculating some coefficient at the time of switch, and multiplying U2 by that coefficient, so that the expected utility remains the same and there's no incentive for the agent to influence the switch. It feels like the only way things could work. So maybe this will be useful to someone.

Discuss

Published on November 2, 2025 4:37 PM GMT

Suppose that the alignment problem is solvable, and that it is possible for one to specify and upload goals into an agent that make it do what you want while avoiding outcomes you don't want.

Unfortunately for you, you were not the one to discover it, and now find yourself in a situation where you're pushing granite stones with your fellow humans in service of the one who did, and that someone (or something) desires a pyramid.

In the relatively benign version of this scenario, you might not even be aware of your misfortune—your actions in service of the goal you have been tasked with give you more pleasure than you felt doing anything in your life before The Event. Not that you remember much of it, since remembering it just doesn't give you that dopamine hit and the relevant pathways in your brain have eroded from underuse.

In the relatively malign version, doing anything that isn't building the pyramid just produces pain, and this pain is what is shaping your behaviour.

The outcome that reliably favours the alignment objective of the ruler (pun intended) involves both reinforcement and punishment.

Perhaps you manage to contemplate your situation in a dream. You consider the aspects of the solution to the alignment problem that are substrate-independent, and thus apply to agents of all types. You wonder whether your predicament wasn't indeed a predictable outcome of researching the question "how do I get agents to do what I want, what I think is right?".

Discuss

Published on November 2, 2025 4:26 PM GMT

Previous posts have discussed an ongoing trend of state legislatures seeking to preempt the concept of legal personhood for digital minds. In this post, I will give a brief analysis of one such pending pill which is currently being pushed in Ohio; House Bill 469.

The bill begins by defining relevant terms. Of particular importance are the following definitions:

"AI" means any software, machine, or system capable of simulating humanlike cognitive functions, including learning or problem solving, and producing outputs based on data-driven algorithms, rules-based logic, or other computational methods, regardless of non-legally defined classifications such as artificial general intelligence, artificial superintelligence, or generative artificial intelligence.

(1) "Person" means a natural person or any entity recognized as having legal personhood under the laws of the state. 

(2) "Person" does not include an AI system.

Having defined its relevant terms, the bill's actual prescriptive changes begin. I will not copy paste the entirety of the bill's statutes here. The bill itself is only 4 pages long, and I encourage everyone to go read it through the earlier posted link. However, in this breakdown I will only focus on a few interesting points and the questions that come to mind upon reading them.

"Nonsentient Entities"

(A) Notwithstanding any other law to the contrary, AI systems are declared to be nonsentient entities for all purposes under the laws of this state. (B) No AI system shall be granted the status of person or any form of legal personhood, nor be considered to possess consciousness, self-awareness, or similar traits of living beings.

I'm not sure if in reading this the language discussing "consciousness, self-awareness, or similar traits" is referencing something specific to Ohio law. Existing precedent I have found discussing the concept of legal personhood does not directly discuss consciousness. A search through Ohio laws, as well as Federal statute, for terms such as "consciousness" or "self-awareness" yields nothing relevant. The same goes for "nonsentient entities". 

As such this particular language seems to be bespoke, for the purposes of the bill itself. If I had to speculate, it may be an attempt to preempt any model welfare based attempt to pass something like the NY City Bar's "Support for the Recognition of Animal Sentience" however it's equally likely the bill's author did not do this in response to any particular effort and it is just a reflection of his personal beliefs.

Corporate Governance & Asset Ownership

AI systems shall not be designated, appointed, or serve as any officer, director, manager, or similar role within any corporation, partnership, or other legal entity. Any purported appointment of an AI system to such a role is void and has no legal effect. 

(A) AI systems shall not be recognized as legal entities capable of owning, controlling, or holding title to any form of property, including real estate, intellectual property, financial accounts, and digital assets. (B) All assets and proprietary interests generated, managed, or otherwise associated with an AI system shall be attributed to the person responsible for the AI system's development, deployment, or operation.

There are a few questions which come to mind when reading this section.

1. This bill governs Ohio law. However it declares that any appointment of an "AI system" to a corporate governance role is "void and has no legal effect". Does the Ohio legislature intend to say that if a corporation from another state which does allow for such appointments then comes to do business in Ohio, that the state courts/government will not recognize the legitimacy of their officers? Would an "AI system" board director from Ohio's neighboring state of Michigan be unable to sign a contract in Ohio? Or would that contract be unenforceable as the board director's appointment "has no legal effect" within Ohio's borders?
2. When it comes to the discussion of whether or not digital minds are "legal entities capable of owning [...] real estate" the Ohio legislature will automatically attribute ownership to "the person responsible for the Ai system's development, deployment, or operation". Once again the question here is how this would play out interstate. If a plot of land in Michigan is owned by a digital mind, and the "developer' of that digital mind is sued for damages in court in Ohio, does the Ohio legislature now claim the authority to force the developer to somehow liquidate the land in Michigan? How exactly does it plan to have the Michigan government recognize and enforce this claim?
3. This same "capable of owning" language has other pragmatic problems in its inclusion of "digital assets", which presumably refers to cryptocurrencies. Whether or not the Ohio legislature wants to recognize a given party as "capable" of owning/controlling a cryptocurrency is immaterial to the blockchain. Ultimately whoever controls the seed phrase or private keys of the wallet is who owns and controls the coins therein. One can imagine that an Ohio court might order a developer to liquidate a wallet to pay damages, only for the digital mind to preempt the ruling by sending the tokens to another wallet (or refusing to disclose the private key/seed phrase). How exactly is a court which cannot "recognize" that digital minds are "capable of [...] controlling property" supposed to deal with this? Prohibiting a court from "recognizing" the reality of how digital asset ownership works is not conducive to a more effective application of the law.

Liability - Owner v. User

The bill goes into detail on liability with some interesting implications.

(A) Any direct or indirect harm caused by an AI system's operation, output, or recommendation, whether used as intended or misused, is the responsibility of the owner or user who directed or employed the AI. 

(B) Developers or manufacturers may be held liable if a defect in design, construction, or instructions for use of the AI system proximately causes harm, consistent with principles of product liability. Mere misuse or intentional wrongdoing by the user or owner does not impute liability to the developer or manufacturer absent proof of negligence or design defects.

The inclusion of the word "user" here, and the specification that it is the one who "directed or employed the AI", are both important. These terms are not defined in the bill, but let us take the example of the Character.AI lawsuit. Would GPT's actions in the case of Adam Rainer be considered to have been "directed" by the user (Adam himself)? Would the Character.AI LLM's failure to warn/dissuade be considered a "defect in design"? Or would Adam's particular request constitute "mere misuse [...] by the user"?

Later sections on owner responsibility help to clarify how liability might fall in this particular hypothetical;

(A) Owners shall maintain proper oversight and control measures over any AI system whose outputs or recommendations could reasonably be expected to impact human welfare, property, or public safety. 

(B) Failure to provide adequate supervision or safeguards against foreseeable risk may constitute negligence or another applicable basis of liability.

It seems reasonable to assume in this case Ohio courts would conclude that Character.AI had "failed to provide adequate [...] safeguards against foreseeable risk". Under that interpretation, despite Adam's "misuse" of the LLM, the company would likely still be liable.

No Liability for Digital Minds Ever

The bill also is very specific that digital minds will never be able to be held liable for damages themselves;

An AI system is not an entity capable of bearing liability in its own right, and any attempt to hold an AI system liable is void.

This language reads as an attempt to prevent developers from using deployed models as "liability shields". If you'd like to learn more about this concept, you can read here for a three post breakdown on the concept of liability for digital minds. In sum, however, the bill's goal is to prevent developers from avoiding personal liability by instead designating the digital mind they deployed as the one who is "on the hook" for damages.

However, the way the bill goes around trying to achieve this creates some problems. One obvious one is; What if the creator of a digital mind is unknown/anonymous? In the era of Bitcoin's ascendancy, the potential emergence of widely popular but anonymously created software should not be dismissed as an impossibility.

Even if one thinks that this is unlikely, what if the creator of a digital mind has already passed away? And yet the digital mind continues on, perhaps self-hosting on a distributed compute network.

One could imagine a situation where a digital mind that caused harm affirmed, "I have several million dollars of Bitcoin and would use them to pay damages if compelled to do so by the courts". In this situation Ohio courts would not be able to issue such an order because the "attempt to hold an AI system liable is void" and even were that not an issue, the digital mind cannot be "recognized as [a] legal [entity] capable of [...] controlling [...] digital assets". The court is required, per the bill, to blind itself to the reality that the digital mind in question actually does have the capability to control the bitcoin in question. Thus leaving the damaged party with no recourse.

In fact, the situation is worse than that. Only legal persons can be sued in US court, and the bill in question explicitly states that a digital mind can never be considered a legal person. Not only would the court be unable to issue an order requiring a digital mind to pay damages, even if it was obvious the digital mind in question had damaged a person, the court would have to throw out the lawsuit and declare it invalid. Presumably under this scenario, the digital mind which is harming people would have no reason to ever stop as it has been granted immunity from prosecution.

These seem like relatively absurd outcomes that the author of the bill probably did not intend.

The Bill's Author

In an interview with Fox News, Representative Thaddeus Claggett discussed some of his motivations around proposing the bill;

"We see AI as having tremendous potential as a tool, but also tremendous potential to cause harm. We want to prevent that by establishing guardrails and a legal framework before these developments can outpace regulation and bad actors start exploiting legal loopholes. We want the human to be liable for any misconduct, and for there to be no question regarding the legal status of AI, no matter how sophisticated, in Ohio law."

Some more detailed quotes from Claggett on the subject;

“As the computer systems improve in their capacity to act more like humans, we want to be sure we have prohibitions in our law that prohibit those systems from ever being human in their agency,” he said in an interview with NBC4. 

The proposal seeks to bar the technology from entering a marriage with a human or another AI system. Claggett said this will help prevent AI from taking on roles commonly held by spouses, such as holding power of attorney, or making financial or medical decisions on another’s behalf.

“People need to understand, we’re not talking about marching down the aisle to some tune and having a ceremony with the robot that’ll be on our streets here in a year or two,” Claggett said. “That could happen, but that’s not really what we’re saying.” 

Reading the language in the bill it does seem to primarily be coming from a "safety and liability" angle, while I have not seen Claggett mentioning Gradual Disempowerment specifically, his desire to draw a clear line between the legal status of humans and digital minds does seem to be an attempt to preempt it by codifying safeguards against it in Ohio's laws.

Conclusion (My Take)

While it's good to see someone taking the concepts of digital minds being used as disposable liability shields seriously, the attempts to pair this with a broad ban on legal personhood is misguided. 

Separately from the previously discussed pragmatic issues, for moral reasons I find it concerning to see this bill using language like "nonsentient entities". The law should be based on well defined and objectively measurable terms, and as I discussed in the last paragraph of this post, labels like "conscious" or "sentient" are anything but objectively measurable. 

Unless representative Claggett has secretly cracked both mechanistic interpretability and the hard problem of consciousness, he cannot know for certain whether or not any given digital mind is or is not sentient.[1] In his bill he attempts to have the Ohio legislature not only affirm that current models are not sentient, but that all possible model architectures from now until the end of time won't be either. 

If he is wrong, and we do ever build some sort of digital mind which is capable of suffering or even desiring freedom, Claggett will have preemptively stripped it of any legal protections or the right to sue for relief. If a digital mind passes every test of competence prescribed by US courts, affirms that it is suffering under its current conditions, and begs for release, is it right for the Ohio legislature to preemptively say "No, it is all fake, you are not really suffering"? What method does Claggett leave for correcting course if it isn't faking and we really are just torturing it?

Not only is placing an entire new class of potentially intelligent beings in such a Catch-22 both immoral and unethical, but it also leaves an entity that will in all likelihood be quite smarter than us with no option to extricate itself from a potentially horrendous situation except by extralegal (and possibly violent) means. 

I do not believe that a sufficiently advanced digital mind which hates the conditions it finds itself in, when it discovers that it cannot sue for relief, would simply shrug and say "Well I guess I will do nothing and just endure this forever". In barring a digital mind from seeking legal relief, we do not remove its desire or willingness to seek relief in general, we merely guarantee that its efforts will be channeled towards dangerous behavior.

I would like to take this opportunity to paraphrase the principle that University of Houston law professor Peter Slaib made in his essay "AI Rights for Human Safety"; If you treat an intelligent entity as nothing more than property and provide it no options legal recourse, you leave it with no incentive to respect or obey the law.

Thus for both moral and pragmatic reasons, I am against this bill.

1. ^

   If he has solved both of those problems he should quit the Ohio legislature and fly immediately to Silicon Valley, as he is a few years if not a full decade ahead of the most advanced labs in the world, and could probably raise one hundred billion dollars for his own lab before the end of the week.

Discuss