"I’ve never met an AI doomer who claimed to be massively borrowing in order to maximally enjoy their final years. But if doomers took their pessimistic view seriously, they obviously would so borrow."
This is a very odd framing. Why would it be obvious that they should so borrow?
If I was confident that the world would end in or before 2030, trying to figure out how to prevent that would be my top priority. Hedonic maximization wouldn't be particularly relevant. Borrowing heavily comes around the same time as heroin and suicide in terms of "nothing I do can plausibly have any positive impact on the world anymore", and that is not what most of the people you call doomers profess to think.
I mean, borrowing and plowing the money into prevention makes sense. I have no easy way to do this though (I have wealth, but no stable income, so bank loans/mortgages are out).
I'd rather die knowing I did something to try and prevent it happening, but ymmv (I think you are failing to really put yourself in my shoes here - you are still thinking the risk doesn't exist in the back of your head, right? Maybe imagine you or your loved ones had cancer instead - you'd do all you could to try and prevent them dying, right? Rather than shrug and say "I can't do more than the assigned doctor because "epistemic humility"")
How does someone having bought an asset that can be sold later at a likely profit demonstrate that they think that the end of the world is coming? Mind you, I think he probably does think it, but this is not a demonstration of it.
Tell me you haven't been to Blackpool without saying you haven't been to Blackpool. :)
It doesn't show that he thinks the end of the world is coming per se, but it shows that he's willing to put money into something exceptionally unprofitable for a cause he believes in.
I’m with Bryan on this—if the world is ending, the least we can do is price it correctly.
These end-of-the-world bets always remind me of Balaji confidently lighting ~$1M on fire in public during his Bitcoin hyperinflation call. There’s something oddly comforting about watching very smart people discover that markets are less impressed by vibes than they are.
At this point, Eliezer and Balaji feel like useful reminders of an alternate timeline where both ended up as mid-level paper pushers in some sprawling bureaucracy—arguing in memos no one reads—if not for the cosmic luck of being in Silicon Valley at exactly the right moment with exactly the right priors.
Instead, we get front-row seats to them making apocalyptic proclamations and occasionally converting large sums of money into… content.
Caplan’s framing is refreshingly grounded. If you really believe the end is near, you don’t write long blog posts—you lever up, spend aggressively, and stop worrying about 2030 repayment terms.
Until then, I’m happy to take the other side of the bet—and keep my apocalypse hedging strictly limited to snacks and a decent internet connection.
I don’t understand why they’re not betting you much more money than this. It seems like an easy way to increase your enjoyment of the end times! As you note, massive borrowing also makes sense.
Sure you do. None of these guys (subconsciously, I think) really buy what they’re selling. They don’t believe it will happen, the same way most other doomers of various stripes (climate, fall of the West, etc.) don’t really believe it will happen.
The only people really putting their money where their mouths are would be prepper types: guns, canned food, bunkers.
I do think the psychology of this is really interesting; there’s a lot of doublethink. Bryan’s bet only makes sense for him because he knows this. As noted in the other comment, if they really believed, it would be a bad bet for Bryan because they would spend themselves into a hole and never be able to repay. Bryan knows they don’t really believe, but they want to be seen as believing, so they pay a nominal amount to continue having credibility in an arena that they eventually know will puncture their credibility totally.
No you are totally wrong on this. I've tried to make large bets (up to $250k, including to Bryan), but the stumbling block has always been trust. My public reputation is worth more than that to me, but people don't accept that and want to make it legally binding. But there is literally no precedent for this - long term bets with people who aren't close. So there is no viable watertight, enforceable legal mechanism, it seems. I've offered to put my (Airbnb) house up as collateral, but how does such a contract get enforced if it comes down to it?
Also, I have actually made a €10k bet for "charities of choice" on this.
And prepping is totally useless in the face of the threat of rogue ASI. It won't save you when the Earth is completely obliterated. There is literally nowhere to run or hide (apart from leaving for interstellar space at very close to the speed of light, which is obviously beyond us.)
In the AI doomers defense - and I'm not an AI doomer at all, just for the record - they don't believe simple tools like guns, canned food and bunkers will be any bulwark against the AI. Now, they could team up with Musk and get the expedition to colonize Mars going. They might think they are safe there. That would be a strongly revealed preference.
Haha no! Mars is only a few light minutes away. Obviously not safe from a rogue ASI! There is literally nowhere to run or hide. The only way we survive is stopping it from being built in the first place (with well enforced international law).
Your claim is that only “well enforced international law” is plausible not merely for delaying it by a few months or years, but stopping it?
Well, at least now I understand why you’re so confident that doom will happen!
Back to an earlier question, given this as ONLY chance, why is it remotely rational for you to “plow the money into prevention”?!? How could anything less than Musk-level money have even a 0.0000001% chance of accomplishing this?
We can buy at least a few more years, and then keep kicking the can down the road a few years at a time. Sadly a very long term solution to the problem does currently seem to be elusive.
>"why is it remotely rational for you to “plow the money into prevention”?"
By plowing money into high-leverage places that the big funders are missing (e.g. I was the biggest funder of PauseAI for a while, with 6-fig donations). Also, Musk is actually doing the opposite of what is needed (i.e. contributing to the suicide race with X AI). And the big money in Effective Altruism is mostly corrupted by support for Anthropic.
You are right but I mean that as an analogy. I think the comparable money-where-mouth-is actions for AI doomers would be terrorism (cyberwarfare or actual bombs, attempt at EMP, whatever).
Yudkowsky has loosely gestured at this but backed off of it immediately. But if he was a true believer, if he thought the human race was at imminent threat of getting paperclipped, bombs would be the right move. I’m pretty sure they believe AGI to be god so escape to Mars will be pointless, it’ll simulate them in hell even if it doesn’t leave the planet right?
Likewise for the guys that believe, I dunno, the economy is going to collapse into anarchy because there’s no gold standard or whatever, buying canned corn and 5.56 in bulk is the obvious rational move, and if you don’t move to a remote farm but instead stay in your comfy suburb with your 401(k) contrivance still going, you obviously do not really believe.
Your analogy worked. I wasn't pushing back on that. Mine was meant as "additional observation" type of comment. That said, I do think even if they are right, a colony on Mars would have a higher chance of surviving. You'd be largely immune to engineered pandemics, for instance. Of course, to bank on it, you would then also have to believe colonizing Mars is logistically possible.
My impression is that most of those worried about AI risk don't think it's a certainty, they just think there's a 5-10% chance of something very bad happening, which is enough of a risk to take action.
How much? And would you be willing to trust me when I say my public reputation is worth more than the bet amount, so I won't renege if/when it comes to Jan 2030?
The optimist-prepays mechanism sounds good till you run into the consideration you bring up at the end, which is that a true pessimist has every incentive to blow through the prepayment, as well as any reserves they have to pay the bet. You might as well go to a gamblers anonymous meeting and bet degenerate sports betters that they can’t go bust in the next five years. Good luck collecting when you “win”. All this makes me wonder: usually when you’re betting, I assume you’re trying to find people who sincerely believe the bet has a different expected yield than you do. When you bet an AI doomer, what you’re really betting is that their actual views are different than their publicly stated view. If their views are sincere, then they know full well they’ll never repay you even if you end up being right about the thing you’re betting on.
If you want an actual bet with them, you should secure their obligation. The contract should be something like mortgaging their house with a high rate that’s due as a balloon in 10 years or whatever. You can record that against their property which would prevent them from selling or further encumbering their house in the intervening years.
If they do what’s “rational” given their views, the pessimists will almost certainly be insolvent, meaning they’ll want to declare bankruptcy and discharge all that debt.
See my comment to Perry. I think you’re right, but the answer is that their views are obviously not fully rational, and therefore sincere but not fully committed in the sense that a fully rational person commits to a fully held rational belief.
I've long offered an immediate payout of $1k to anyone that can change my credences by 20% (up or down) on the likelihood of global catastrophe occurring due to AI. I'm at ~75%.
A single short conversation or link might be enough. If you wanted to share your most convincing argument or piece of evidence, please do. Upon my honor, I'll immediately pay you if it is compelling to me. (I know that might not make it worth your time to respond.)
I live off-grid already, if that is a costly enough signal for you that I take risks seriously. And spent my life in preparation for this moment. It's hard to say what exactly will wreak the most global havoc, but it does seem highly likely to happen unless we actively prevent it. On that note, see existentialsafety.org for 90+ ways to help.
Given the Iran War, Mythos leak, Claude's cyber espionage incident, and many others, I'm quite glad I already made the move. When Event X, Event Y, and Event Z happen next, I'll be even happier.
Hi Bryan! I'm proud to be one of your counterparties (recall our "Climate Shock" wager) even though, of course, you won that one. (I still say you're on track to lose the climate bet that really counts!)
But this end-of-the-world bet I'm not impressed by. Colburn has merely borrowed $250 from you at an effective interest rate of 21% (using the market's expectation for inflation of 2.6%).
He might as well just max out his credit cards, right?
My bigger objection is to your misplaced certainty. And I have the same objection to the doomers. The doomers shouldn't be sure that AI will turn the world upside-down and neither should you be sure that it won't. I think 2030 is soon enough that it's reasonable to be *pretty* confident in a rightside-up world, but for 2040 it starts to feel more like a coin flip for me. And even a coin flip on the end of the world (or the end of money?) doesn't mean "might as well max out your credit cards". I don't think? You're the economist -- I'd be interested to hear what different p(doom) values at different time horizons would imply, rationally/financially speaking. I might bite those bullets.
As for whether doomers are hypocrites, I think many of them are not saving for retirement, to whatever extent that counts as putting their money where their mouths are.
PS: I'm impressed by Toby Ord's characteristically careful analysis of exactly how freaked out to be about AI: https://www.tobyord.com/writing/broad-timelines He points out that the most credible people (including some that get characterized as doomers) have 80% confidence intervals on the arrival of transformative AI that are at least 50 years wide. Ord's is 97 years wide.
I believe Cate Hall cashed out her entire retirement portfolio in ~2017ish when she became convinced of the imminent and certain dangers of AI. I also believe she now regrets doing it.
Not quite. Bryan lost the first bet (om the sense of making a poor loan for much lower interest than he could have gotten elsewhere), but this second bet is essentially a mini-loan for ~21% interest. While Greg's investments may very well out-perform this, this is objectively a high-interest rate and Greg undoubtedly can get lower interest loans elsewhere.
I can't actually get loans elsewhere. Banks want you to have a stable income (which I don't have). They don't care for assets as collateral because they don't want to go to the trouble of actually having to seize them, it seems.
Thanks for clarifying. Are there not other individuals willing to make you private loans at lower than 21% interest rates? I would think that there are many people who consider 20% interest to be a great return such that they'd be willing to loan you the money. Your counterparty risk seems quite low given your high trustworthiness and the fact that you're not making super high risk investments that would allow you to lose all your assets (my understanding is that you plan to hold roughly half your assets long-term without donating them all at any point).
If you haven't found such people willing to loan you money, what do you think the reason is? Are there tax reasons preventing this, or is it just an unconventional thing to do such that people are averse to it or something?
Who is it that's born every minute? 🤔 You still have almost two million chances to make similar bets (there are approximately 1,944,625 minutes remaining until January 1, 2030).
"I’ve never met an AI doomer who claimed to be massively borrowing in order to maximally enjoy their final years. But if doomers took their pessimistic view seriously, they obviously would so borrow."
This is a very odd framing. Why would it be obvious that they should so borrow?
If I was confident that the world would end in or before 2030, trying to figure out how to prevent that would be my top priority. Hedonic maximization wouldn't be particularly relevant. Borrowing heavily comes around the same time as heroin and suicide in terms of "nothing I do can plausibly have any positive impact on the world anymore", and that is not what most of the people you call doomers profess to think.
I mean, borrowing and plowing the money into prevention makes sense. I have no easy way to do this though (I have wealth, but no stable income, so bank loans/mortgages are out).
If that confident, why not lever up and simply give at least 85% of your money to the desperately poor now?
The most likely cause of death for even the desperately poor is still rogue ASI.
“If I was confident that the world would end in or before 2030, trying to figure out how to prevent that would be my top priority.”
Wow, what a perfect example of a complete lack of epistemic humility!
Unless you have the resources of, say, an Elon Musk, or the position of a Yudkowsky, why do you think you have any chance at all to prevent same?
Especially given the short timeframe?
Especially if you are *confident* that the world will end by then?
I'd rather die knowing I did something to try and prevent it happening, but ymmv (I think you are failing to really put yourself in my shoes here - you are still thinking the risk doesn't exist in the back of your head, right? Maybe imagine you or your loved ones had cancer instead - you'd do all you could to try and prevent them dying, right? Rather than shrug and say "I can't do more than the assigned doctor because "epistemic humility"")
I believe it's spelled Colbourn.
Greg has bought out two hotels in Blackpool, Lancashire for philanthropic use - they are called CEEALAR and Pause House (https://forum.effectivealtruism.org/posts/fhiXNB4RS3h5BpQLF/pause-house-blackpool), and they're dedicated for use by people trying to stop existential risks.
He's definitely someone who puts his money where his mouth is, and I'm glad to see he's made this bet!
How does someone having bought an asset that can be sold later at a likely profit demonstrate that they think that the end of the world is coming? Mind you, I think he probably does think it, but this is not a demonstration of it.
Tell me you haven't been to Blackpool without saying you haven't been to Blackpool. :)
It doesn't show that he thinks the end of the world is coming per se, but it shows that he's willing to put money into something exceptionally unprofitable for a cause he believes in.
It's actually 3 hotels, and I've donated the first 2 (now joined together) to the charity set up to run the EA Hotel (CEEALAR).
I’m with Bryan on this—if the world is ending, the least we can do is price it correctly.
These end-of-the-world bets always remind me of Balaji confidently lighting ~$1M on fire in public during his Bitcoin hyperinflation call. There’s something oddly comforting about watching very smart people discover that markets are less impressed by vibes than they are.
At this point, Eliezer and Balaji feel like useful reminders of an alternate timeline where both ended up as mid-level paper pushers in some sprawling bureaucracy—arguing in memos no one reads—if not for the cosmic luck of being in Silicon Valley at exactly the right moment with exactly the right priors.
Instead, we get front-row seats to them making apocalyptic proclamations and occasionally converting large sums of money into… content.
Caplan’s framing is refreshingly grounded. If you really believe the end is near, you don’t write long blog posts—you lever up, spend aggressively, and stop worrying about 2030 repayment terms.
Until then, I’m happy to take the other side of the bet—and keep my apocalypse hedging strictly limited to snacks and a decent internet connection.
I don’t understand why they’re not betting you much more money than this. It seems like an easy way to increase your enjoyment of the end times! As you note, massive borrowing also makes sense.
Sure you do. None of these guys (subconsciously, I think) really buy what they’re selling. They don’t believe it will happen, the same way most other doomers of various stripes (climate, fall of the West, etc.) don’t really believe it will happen.
The only people really putting their money where their mouths are would be prepper types: guns, canned food, bunkers.
I do think the psychology of this is really interesting; there’s a lot of doublethink. Bryan’s bet only makes sense for him because he knows this. As noted in the other comment, if they really believed, it would be a bad bet for Bryan because they would spend themselves into a hole and never be able to repay. Bryan knows they don’t really believe, but they want to be seen as believing, so they pay a nominal amount to continue having credibility in an arena that they eventually know will puncture their credibility totally.
No you are totally wrong on this. I've tried to make large bets (up to $250k, including to Bryan), but the stumbling block has always been trust. My public reputation is worth more than that to me, but people don't accept that and want to make it legally binding. But there is literally no precedent for this - long term bets with people who aren't close. So there is no viable watertight, enforceable legal mechanism, it seems. I've offered to put my (Airbnb) house up as collateral, but how does such a contract get enforced if it comes down to it?
Also, I have actually made a €10k bet for "charities of choice" on this.
And prepping is totally useless in the face of the threat of rogue ASI. It won't save you when the Earth is completely obliterated. There is literally nowhere to run or hide (apart from leaving for interstellar space at very close to the speed of light, which is obviously beyond us.)
In the AI doomers defense - and I'm not an AI doomer at all, just for the record - they don't believe simple tools like guns, canned food and bunkers will be any bulwark against the AI. Now, they could team up with Musk and get the expedition to colonize Mars going. They might think they are safe there. That would be a strongly revealed preference.
Haha no! Mars is only a few light minutes away. Obviously not safe from a rogue ASI! There is literally nowhere to run or hide. The only way we survive is stopping it from being built in the first place (with well enforced international law).
Your claim is that only “well enforced international law” is plausible not merely for delaying it by a few months or years, but stopping it?
Well, at least now I understand why you’re so confident that doom will happen!
Back to an earlier question, given this as ONLY chance, why is it remotely rational for you to “plow the money into prevention”?!? How could anything less than Musk-level money have even a 0.0000001% chance of accomplishing this?
We can buy at least a few more years, and then keep kicking the can down the road a few years at a time. Sadly a very long term solution to the problem does currently seem to be elusive.
>"why is it remotely rational for you to “plow the money into prevention”?"
By plowing money into high-leverage places that the big funders are missing (e.g. I was the biggest funder of PauseAI for a while, with 6-fig donations). Also, Musk is actually doing the opposite of what is needed (i.e. contributing to the suicide race with X AI). And the big money in Effective Altruism is mostly corrupted by support for Anthropic.
You are right but I mean that as an analogy. I think the comparable money-where-mouth-is actions for AI doomers would be terrorism (cyberwarfare or actual bombs, attempt at EMP, whatever).
Yudkowsky has loosely gestured at this but backed off of it immediately. But if he was a true believer, if he thought the human race was at imminent threat of getting paperclipped, bombs would be the right move. I’m pretty sure they believe AGI to be god so escape to Mars will be pointless, it’ll simulate them in hell even if it doesn’t leave the planet right?
Likewise for the guys that believe, I dunno, the economy is going to collapse into anarchy because there’s no gold standard or whatever, buying canned corn and 5.56 in bulk is the obvious rational move, and if you don’t move to a remote farm but instead stay in your comfy suburb with your 401(k) contrivance still going, you obviously do not really believe.
Your analogy worked. I wasn't pushing back on that. Mine was meant as "additional observation" type of comment. That said, I do think even if they are right, a colony on Mars would have a higher chance of surviving. You'd be largely immune to engineered pandemics, for instance. Of course, to bank on it, you would then also have to believe colonizing Mars is logistically possible.
My impression is that most of those worried about AI risk don't think it's a certainty, they just think there's a 5-10% chance of something very bad happening, which is enough of a risk to take action.
I'm at more like 50%, all things considered, which means the bet at even odds makes sense.
Bryan was the one to make the bet for a token amount, not Greg: https://x.com/gcolbourn/status/2036471089732211060
Greg very generously gave you 1:1 odds when you offered 10:1 odds and you still were only willing to bet $250? Why? https://x.com/gcolbourn/status/2036471089732211060
1) opportunity cost
2) uncertainty of pessimist’s ability to pay
This is a great bet for Bryan but the main problem from Bryan’s end is those two, linked
I'd bet more against Greg.
How much? And would you be willing to trust me when I say my public reputation is worth more than the bet amount, so I won't renege if/when it comes to Jan 2030?
I understand the risks of AI, but betting we'll all be dead in 4 years is wild!
Stupid people making stupid bets. Get back to work, Caplan!
The optimist-prepays mechanism sounds good till you run into the consideration you bring up at the end, which is that a true pessimist has every incentive to blow through the prepayment, as well as any reserves they have to pay the bet. You might as well go to a gamblers anonymous meeting and bet degenerate sports betters that they can’t go bust in the next five years. Good luck collecting when you “win”. All this makes me wonder: usually when you’re betting, I assume you’re trying to find people who sincerely believe the bet has a different expected yield than you do. When you bet an AI doomer, what you’re really betting is that their actual views are different than their publicly stated view. If their views are sincere, then they know full well they’ll never repay you even if you end up being right about the thing you’re betting on.
If you want an actual bet with them, you should secure their obligation. The contract should be something like mortgaging their house with a high rate that’s due as a balloon in 10 years or whatever. You can record that against their property which would prevent them from selling or further encumbering their house in the intervening years.
They might use all their money, but they won't use up all their human capital which they can use to earn some more money in order to pay the dept.
If they do what’s “rational” given their views, the pessimists will almost certainly be insolvent, meaning they’ll want to declare bankruptcy and discharge all that debt.
See my comment to Perry. I think you’re right, but the answer is that their views are obviously not fully rational, and therefore sincere but not fully committed in the sense that a fully rational person commits to a fully held rational belief.
I've long offered an immediate payout of $1k to anyone that can change my credences by 20% (up or down) on the likelihood of global catastrophe occurring due to AI. I'm at ~75%.
A single short conversation or link might be enough. If you wanted to share your most convincing argument or piece of evidence, please do. Upon my honor, I'll immediately pay you if it is compelling to me. (I know that might not make it worth your time to respond.)
I live off-grid already, if that is a costly enough signal for you that I take risks seriously. And spent my life in preparation for this moment. It's hard to say what exactly will wreak the most global havoc, but it does seem highly likely to happen unless we actively prevent it. On that note, see existentialsafety.org for 90+ ways to help.
Given the Iran War, Mythos leak, Claude's cyber espionage incident, and many others, I'm quite glad I already made the move. When Event X, Event Y, and Event Z happen next, I'll be even happier.
Hi Bryan! I'm proud to be one of your counterparties (recall our "Climate Shock" wager) even though, of course, you won that one. (I still say you're on track to lose the climate bet that really counts!)
But this end-of-the-world bet I'm not impressed by. Colburn has merely borrowed $250 from you at an effective interest rate of 21% (using the market's expectation for inflation of 2.6%).
He might as well just max out his credit cards, right?
My bigger objection is to your misplaced certainty. And I have the same objection to the doomers. The doomers shouldn't be sure that AI will turn the world upside-down and neither should you be sure that it won't. I think 2030 is soon enough that it's reasonable to be *pretty* confident in a rightside-up world, but for 2040 it starts to feel more like a coin flip for me. And even a coin flip on the end of the world (or the end of money?) doesn't mean "might as well max out your credit cards". I don't think? You're the economist -- I'd be interested to hear what different p(doom) values at different time horizons would imply, rationally/financially speaking. I might bite those bullets.
As for whether doomers are hypocrites, I think many of them are not saving for retirement, to whatever extent that counts as putting their money where their mouths are.
PS: I'm impressed by Toby Ord's characteristically careful analysis of exactly how freaked out to be about AI: https://www.tobyord.com/writing/broad-timelines He points out that the most credible people (including some that get characterized as doomers) have 80% confidence intervals on the arrival of transformative AI that are at least 50 years wide. Ord's is 97 years wide.
I believe Cate Hall cashed out her entire retirement portfolio in ~2017ish when she became convinced of the imminent and certain dangers of AI. I also believe she now regrets doing it.
https://x.com/gcolbourn/status/1770849110498636045
Good for you! "Bah" is the correct answer to the doomers.
You already lost. SPY has gone from $250 in late 2017 to $700 today. At best, you get $500 in 2030.
You need to charge interest and adjust for inflation.
I would be tempted to take the other side of your bet just to invest the float.
Not quite. Bryan lost the first bet (om the sense of making a poor loan for much lower interest than he could have gotten elsewhere), but this second bet is essentially a mini-loan for ~21% interest. While Greg's investments may very well out-perform this, this is objectively a high-interest rate and Greg undoubtedly can get lower interest loans elsewhere.
I can't actually get loans elsewhere. Banks want you to have a stable income (which I don't have). They don't care for assets as collateral because they don't want to go to the trouble of actually having to seize them, it seems.
Thanks for clarifying. Are there not other individuals willing to make you private loans at lower than 21% interest rates? I would think that there are many people who consider 20% interest to be a great return such that they'd be willing to loan you the money. Your counterparty risk seems quite low given your high trustworthiness and the fact that you're not making super high risk investments that would allow you to lose all your assets (my understanding is that you plan to hold roughly half your assets long-term without donating them all at any point).
If you haven't found such people willing to loan you money, what do you think the reason is? Are there tax reasons preventing this, or is it just an unconventional thing to do such that people are averse to it or something?
I think the problem is most people want some kind of legal guarantee, which seems impossible to actually have enforced if it came down to it.
Who is it that's born every minute? 🤔 You still have almost two million chances to make similar bets (there are approximately 1,944,625 minutes remaining until January 1, 2030).