>One other way to see that it isn’t ad hoc is that these kinds of infinities cause problems almost across the board. There are many different paradoxes that arise from normalizable probability functions—but they all result from something else relevant growing faster than the probabilities drop off.
Indeed, the culprit is hypotheses involving random variables with infinite expectation. And that's bad, because in the real world there are always such hypotheses lurking in the background for every decision, however non-saliently. But it's even worse than that, because even if you rather bluntly choose to ignore any such hypothesis, your decisions with respect to the remaining better-behaved hypotheses won't be continuous in your priors unless total utility is finite, which effectively means you (as a bounded reasoner who inevitably works with approximations) should massively distrust all of your decision-theoretic calculations. To deal with this, you can either 1. abandon anything like utility maximization (which means abandoning fanatacism), or 2. go with bounded utilities.
The problem isn't just messiness, it's literally impossible failing infinite computational power! On unbounded utility maximization, there's no reason to think that spending Graham's number worth of years computing the best action to take in the real world will get you any closer to the right answer than just guessing, even when you're arbitrarily excluding infinite-EV hypotheses!
You're hungry and deciding whether to get food from your kitchen for or remain where you are. Let O be the ordinary hypothesis (with high probability) that you'll get the food you expect from the kitchen - say, one util's worth - and that nothing weird happens in any case. But now consider exotic hypotheses (you can use your imagination to concoct these) X_1, X_2, ..., where X_k rewards you with k utils for going to the kitchen and zero otherwise; and also a different set of exotic hypotheses Y_1, Y_2, ..., which do the same thing for not going to the kitchen.
The X_i's and Y_i's are competing with each other and with O. The exotic X_i's and Y_i's also all have extremely small probabilities, which necessarily decay to zero as i goes to infinity (otherwise, the probabilities couldn't sum to 1). Nevertheless, even assuming they're well-behaved enough that the expected utility E[U(go to kitchen)] is finite, the exact value of that quantity depends very sensitively on (among other things) *how fast* P(X_i) and P(Y_i) decay to zero.
You can numerically change each of the P(X_i)'s and P(Y_i)'s by arbitrarily tiny absolute amounts (say, you tweak each one by a different quantity that's always less than one in a googolplex), while drastically affecting the decay rate. So in order to get a handle on expected utility, you need to get a handle on the decay rate of all the probabilities of all the exotic hypotheses in question as a function of their reward.
Realistically, given that there's way more exotic hypotheses to deal with than the X_i's and Y_i's of infinitely dizzying variety and character, there's no computationally finite way to do this! Some of them will even involve explicitly uncomputable things, like God promising you BB(30,000) utils for doing something.
I don't see what's wrong with saying that though we can't enumerate all the available options, it seems like you starving to death would be bad for your capacities and thus the odds of realizing arbitrarily valuable scnearios overall.
That shouldn’t matter. For one, the exotic hypotheses might outweigh the utility of anything I might realistically hope to do if I do or don’t starve to death. For another, the exotic hypotheses can affect the probability that I starve to death depending on whether I go to the kitchen or not (they’re competing with the ordinary hypothesis O!), thus they’re already incorporated in this kind of thinking.
This piece resonated with an argument I’ve been developing about epistemic stakes: if certain discoveries about the fundamental nature of reality could radically change what counts as “good” or how we should live, then even tiny chances of making those discoveries might outweigh more certain but smaller goods. I’ve sketched it here if anyone's curious:
I’m interested in how much your theoretical conclusions here have really “sunk in” to your mind at the level of practical reasoning.
Suppose a genie really did appear to you and give you the offer to extend your life by a googolplex years with one in a quadrillion probability, and otherwise will kill you instantly.
Would you find it easy to accept the offer, do you think? Or do you still have animal instincts yet to be overcome by philosophy which you think might actually carry the day here, albeit incorrectly?
(I didn’t finish reading to this, for full disclosure.)
I’m a utilitarian in spirit, but doesn’t fanatical utilitarianism always falter on the requirement of quantifiability and pure additivity?
I would allow an infinitely large number of people (7 billion, 100 billion, 300 billion, 10 to the 100th power, whatever) to each get pricked by a small needle than let one person get tortured to death.
That's not related to fanaticism. You could be a fanatic or not and hold that judgment or not (full disclosure, I don't think that judgment is defensible). Fanaticism is about risk not comparing guaranteed outcomes.
For more on why I reject your judgment see this article (but replace shrimp torture with "prevent n people from getting dust specks in their eyes."
Most of this post is just reiterating that expected value is defined as the mean of a distribution, not the mode. By presenting thought experiments where we get to enjoy complete disassociation with the consequences of the problem, it essentially allows us to pretend we can make this decision an infinite number of times, in which case clearly the mean is what we care about.
In the real world, as you point out in Section 4, we often only get to make decisions a handful of times or perhaps even once. The more important the decision, the less frequently we tend to be able to make it, and the more dominant the actual outcome is on our life. In other words, the modal outcome becomes increasingly important and the tails decreasingly important.
Another commenter asked about St. Petersburg Paradox- indeed this is what you have to solve to argue for expected value fanaticism in the real world, and you haven't addressed it at all. An EV fanatic like Sam Bankman-Fried says to never stop flipping the coin, maximizing EV at the cost of ever-increasing chance of ruin, which is basically the exact same argument you are using here to say you would choose ever decreasing odds of saving ever increasing magnitudes of value.
One argument you make is that tail-discounting results in bizzare hypersensitivity, where at some point a .0000000001% drop in probability causes a massive discontinuity in value, which seems pretty odd. I am willing to admit this is true of tail discounting, and definitely true for my proposed strategy for these scenarios in my other comment (just draw an arbitrary line at a desired probability and defend it no matter what). I don’t think this is a fatal objection, and would argue it also occurs for the fanatic.
Let’s return to my earlier button hypothetical: 99.9999999999% chance 4 billion people die horribly, 0.0000000001% chance everyone goes to Heaven.
Under fanaticism, you simply see that a 0.0000000001% chance of infinite utility comes out to infinite expected value. Not only is this action defensible, it is *infinitely good* to do it. It is one of the most desirable possible actions to take.
And yet, if I subtract exactly 0.0000000001% from the probability that the button makes sure everyone goes to Heaven, suddenly the EV has changed quite a bit. Infinity * 0 = 0, so the EV is now extremely negative. A change of only 0.0000000001% probability has taken this action from one of the best possible actions one could ever take, to one of the worst. Sounds like a pretty bizzare hypersensitivity to me!
Does it really make that much more sense to have this discontinuity at exactly zero, rather than at some other number? I would say if an action is a terrible idea at 0% probability of success, an increase of .0000000001% *should probably not make it a good idea*.
So either fanaticism is fatally flawed, or bizarre hypersensitivity is not a fatal objection towards fanaticism’s competitors.
-"Now, what should you do when you conclude fanaticism is right? Well, a fanatic thinks that their sole aim should be to maximize the odds of infinite value. Keeping the world around probably does that, so you should donate to Longtermist charities (especially because they potentially increase the number of people ever born, thus giving more people a chance of getting into heaven). Maybe you should also try to become religious to increase your odds of getting into Heaven, but that’s more controversial."
Longtermist charities only give you a chance of infinite value if the heat death of the universe is false. So basically you are saying, fanaticism means we should assume that either the afterlife is real or the laws of thermodynamics are wrong, as the alternative has only finite value and thus should be ignored.
There is also the issue that if you base your decisions on value from the afterlife, there's no point in multiple people getting to heaven because infinity plus infinity equals infinity. You could try to build some theory where multiple infinities are better than one but it's not obvious that this doesn't lead to a fanaticism in favor of "infinitely many infinities", a process which could be repeated ad nauseum.
For the record, my solution is to assume that the laws of thermodynamics are not wrong and that the afterlife doesn't exist. I think that will make your hypothetical scenarios impossible but let me know if I missed something.
On one hand I want to be a fanatic because I made a rather simple argument that we should accelerate AI for the low probability at achieving eternal consciousness (on the assumption that eternal consciousness will result in infinite value). However, on reflection, on the other hand, I think I should not be a fanatic because the actual payouts trend towards zero as the payout approaches infinity. I think this is Huemer's point in Approaching Infinity, but I could be misguided in applying that problem here (or I could be wrong in interpreting his point as that, or both). I also think the mugger is a convincing reason to give up the wager, and I don't think I understand your reply as to why you should accept the mugging (or why if you shouldn't?).
Maybe this is no different than the mugger (or no different than what I believe is a similar example Huemer gives in Approaching Infinity), but assume a man approaches you while you are walking home one day and offers you one util for free. You go sure! Then the man says, but wait, if you let me cause -1 util of harm to you, then I will meet you here tomorrow same time and give you 2 utils! You accept, then tomorrow the man meets you and offers to instead meet you the next day same time and give you 4 utils if he can cause you -2 utils of harm at this time. This results in accepting indefinite suffering chasing an infinite payout. Now, the same reasoning for why I think one should reject this is back to (what I think is) Huemer's point, which is that the actual payouts will not be feasible or even possible after some point. However, that appears to leave one to also reject the speculative payout in the first place (like in Pascal's Wager). So rejecting one under this reasoning, rejects both, while accepting one, seems to accept the other.
> If you’re a fanatic, you should spend your life chasing infinite value.
One problem with this view is that if you accept that any decision has non-0 chance of infinite value, you seem committed to the view that *every* action has a non-0 chance of infinite value, which leaves you without any basis on which to make decisions (since the expected payoff of all actions is now infinite).
I don't see any problem with presuming in the real world, as the value of some_payoff approaches infinity, E[some_payoff] tends to approach 0. This seems consistent with our experience (if I were to actually Pascal's mug you, I imagine you'd find my offer more compelling if I were to offer you £10 tomorrow than if I were to offer you £10^100. If not, please let me know, and I'll send you payment details), and solves most of the problems you mention with expected utility maximisation for either infinities or very large numbers.
It requires a little trust in an underlying expectation-bounding mechanism we (or at least I) can't elucidate yet, but as bullets go that seems a better one to bite than allowing yourself to be trivially mugged out of arbitrary amounts of money.
I think the rational thing for Dave to do in the initial parable is to simply draw an arbitrary line of what probability he deems acceptable (whether it be 60, 75, 90, or some other percentage is left as an excercise for the reader) and stick to it no matter how much surface-level better the next step is.
This is because the next step isn’t just the next step, it’s committing yourself to all future deals in the chain which eventually lead to a near-certain probability of getting nothing. So the choice isn’t really between 60% chance of saving 100^100 people and a 59% chance of saving 100^100^100 people, it’s between the former and a 99.999999999999% chance of nothing, which is worse no matter what you get on the 0.00000000000000001% chance, because it’s overwhelmingly likely you get nothing.
The only way you avoid committing yourself to the fanatic chain is if you draw the arbitrary line and stick to it even if it seems irrational. You can debate where precisely that line should be (probably above 50% at the very least) but the person who picks a line and sticks to it has a good shot of keeping their winnings, whereas the fanatic is nearly guaranteed to lose everything.
I’m pretty sure this doesn’t violate transitivity: the premise of my argument is that the next step is actually just worse, because it’s not what it appears to be.
I also don’t think it’s timidity, because it doesn’t require you draw the line at 100%.
I’m sure it requires me to bite various bullets, but the button-press bullet from my comment the other day seems to be a vastly worse bullet to bite than anything my position requires.
An obvious counter to EV fanaticism, is that if we all became EV fanatics, civilization would collapse. The economy heavily depends on most people pursuing lower-EV, higher-certainty opportunities, if all economic activity became people chasing unicorns, very little would get done and civilization would quickly end. However, some amount of economic activity should be devoted to moonshots (the sort of things EV fanatics would do), and it would be interesting to calculate how much...
And charity too. EV fanaticism means we should stop trying to help out Africa, charitable dollars should all go to charitable moonshots, which runs into the same issues as the economy would.
The article is a beautiful reductio ad absurdum of its own premise. The entire argument hinges on taking a single, overly simplistic rule—expected utility maximization—as the ultimate criterion for rational decision-making. That's not a serious theory. Frankly, the issues the author grapples with are well-known, elementary problems in decision theory that show precisely why that single rule is flawed.
> The entire argument hinges on taking a single, overly simplistic rule—expected utility maximization—as the ultimate criterion for rational decision-making.
This article does the exact opposite of that, at rather insane length.
Your comment claimed his argument hinges on taking expected utility maximization as a rule and then deriving fanaticism. His article is instead about how a whole bunch of completely different, much weaker and more plausible principles than expected utility maximization imply fanaticism.
I'm going to apologize that my life situation doesn't allow me the focus to read the whole piece with the attention that it deserves — and thus should not ideally be posting a comment! — but am I correct in thinking that here, you're agreeing to something like Sam Bankman Fried's famous (infamous!) answer to the St. Petersburg question: that he'd continue to flip the coin ad infinitum? (Insofar as I'm asking a question that doesn't even make sense in this context, I apologize.)
That sounds like a very good question he asked though! From a expected value maximization/fanaticism standpoint, when exactly should you stop a Martingale/St. Petersburg situation, where you are being repeatedly given positive expected value bets? Such as $2^n for every tails you get (and losing it all at the first heads). It seems like fanaticism keeps you playing indefinitely because it cannot tell the step you should leave it at while invoking only expected value which is always positive.
>One other way to see that it isn’t ad hoc is that these kinds of infinities cause problems almost across the board. There are many different paradoxes that arise from normalizable probability functions—but they all result from something else relevant growing faster than the probabilities drop off.
Indeed, the culprit is hypotheses involving random variables with infinite expectation. And that's bad, because in the real world there are always such hypotheses lurking in the background for every decision, however non-saliently. But it's even worse than that, because even if you rather bluntly choose to ignore any such hypothesis, your decisions with respect to the remaining better-behaved hypotheses won't be continuous in your priors unless total utility is finite, which effectively means you (as a bounded reasoner who inevitably works with approximations) should massively distrust all of your decision-theoretic calculations. To deal with this, you can either 1. abandon anything like utility maximization (which means abandoning fanatacism), or 2. go with bounded utilities.
I had a section about how to apply probabilities in the real world. I agree it willbe pretty messy,
The problem isn't just messiness, it's literally impossible failing infinite computational power! On unbounded utility maximization, there's no reason to think that spending Graham's number worth of years computing the best action to take in the real world will get you any closer to the right answer than just guessing, even when you're arbitrarily excluding infinite-EV hypotheses!
Why?
You're hungry and deciding whether to get food from your kitchen for or remain where you are. Let O be the ordinary hypothesis (with high probability) that you'll get the food you expect from the kitchen - say, one util's worth - and that nothing weird happens in any case. But now consider exotic hypotheses (you can use your imagination to concoct these) X_1, X_2, ..., where X_k rewards you with k utils for going to the kitchen and zero otherwise; and also a different set of exotic hypotheses Y_1, Y_2, ..., which do the same thing for not going to the kitchen.
The X_i's and Y_i's are competing with each other and with O. The exotic X_i's and Y_i's also all have extremely small probabilities, which necessarily decay to zero as i goes to infinity (otherwise, the probabilities couldn't sum to 1). Nevertheless, even assuming they're well-behaved enough that the expected utility E[U(go to kitchen)] is finite, the exact value of that quantity depends very sensitively on (among other things) *how fast* P(X_i) and P(Y_i) decay to zero.
You can numerically change each of the P(X_i)'s and P(Y_i)'s by arbitrarily tiny absolute amounts (say, you tweak each one by a different quantity that's always less than one in a googolplex), while drastically affecting the decay rate. So in order to get a handle on expected utility, you need to get a handle on the decay rate of all the probabilities of all the exotic hypotheses in question as a function of their reward.
Realistically, given that there's way more exotic hypotheses to deal with than the X_i's and Y_i's of infinitely dizzying variety and character, there's no computationally finite way to do this! Some of them will even involve explicitly uncomputable things, like God promising you BB(30,000) utils for doing something.
I don't see what's wrong with saying that though we can't enumerate all the available options, it seems like you starving to death would be bad for your capacities and thus the odds of realizing arbitrarily valuable scnearios overall.
That shouldn’t matter. For one, the exotic hypotheses might outweigh the utility of anything I might realistically hope to do if I do or don’t starve to death. For another, the exotic hypotheses can affect the probability that I starve to death depending on whether I go to the kitchen or not (they’re competing with the ordinary hypothesis O!), thus they’re already incorporated in this kind of thinking.
This piece resonated with an argument I’ve been developing about epistemic stakes: if certain discoveries about the fundamental nature of reality could radically change what counts as “good” or how we should live, then even tiny chances of making those discoveries might outweigh more certain but smaller goods. I’ve sketched it here if anyone's curious:
https://heatdeathandtaxes.substack.com/p/find_purposeexe
I’m interested in how much your theoretical conclusions here have really “sunk in” to your mind at the level of practical reasoning.
Suppose a genie really did appear to you and give you the offer to extend your life by a googolplex years with one in a quadrillion probability, and otherwise will kill you instantly.
Would you find it easy to accept the offer, do you think? Or do you still have animal instincts yet to be overcome by philosophy which you think might actually carry the day here, albeit incorrectly?
Definitely still have instincts, overall judgment unclear.
(I didn’t finish reading to this, for full disclosure.)
I’m a utilitarian in spirit, but doesn’t fanatical utilitarianism always falter on the requirement of quantifiability and pure additivity?
I would allow an infinitely large number of people (7 billion, 100 billion, 300 billion, 10 to the 100th power, whatever) to each get pricked by a small needle than let one person get tortured to death.
That's not related to fanaticism. You could be a fanatic or not and hold that judgment or not (full disclosure, I don't think that judgment is defensible). Fanaticism is about risk not comparing guaranteed outcomes.
For more on why I reject your judgment see this article (but replace shrimp torture with "prevent n people from getting dust specks in their eyes."
https://benthams.substack.com/p/the-staggeringly-strong-case-that
A high-quality, through post as usual. I’ll set a reminder to line by line this with throwaway arguments when I wake up.
Most of this post is just reiterating that expected value is defined as the mean of a distribution, not the mode. By presenting thought experiments where we get to enjoy complete disassociation with the consequences of the problem, it essentially allows us to pretend we can make this decision an infinite number of times, in which case clearly the mean is what we care about.
In the real world, as you point out in Section 4, we often only get to make decisions a handful of times or perhaps even once. The more important the decision, the less frequently we tend to be able to make it, and the more dominant the actual outcome is on our life. In other words, the modal outcome becomes increasingly important and the tails decreasingly important.
Another commenter asked about St. Petersburg Paradox- indeed this is what you have to solve to argue for expected value fanaticism in the real world, and you haven't addressed it at all. An EV fanatic like Sam Bankman-Fried says to never stop flipping the coin, maximizing EV at the cost of ever-increasing chance of ruin, which is basically the exact same argument you are using here to say you would choose ever decreasing odds of saving ever increasing magnitudes of value.
One argument you make is that tail-discounting results in bizzare hypersensitivity, where at some point a .0000000001% drop in probability causes a massive discontinuity in value, which seems pretty odd. I am willing to admit this is true of tail discounting, and definitely true for my proposed strategy for these scenarios in my other comment (just draw an arbitrary line at a desired probability and defend it no matter what). I don’t think this is a fatal objection, and would argue it also occurs for the fanatic.
Let’s return to my earlier button hypothetical: 99.9999999999% chance 4 billion people die horribly, 0.0000000001% chance everyone goes to Heaven.
Under fanaticism, you simply see that a 0.0000000001% chance of infinite utility comes out to infinite expected value. Not only is this action defensible, it is *infinitely good* to do it. It is one of the most desirable possible actions to take.
And yet, if I subtract exactly 0.0000000001% from the probability that the button makes sure everyone goes to Heaven, suddenly the EV has changed quite a bit. Infinity * 0 = 0, so the EV is now extremely negative. A change of only 0.0000000001% probability has taken this action from one of the best possible actions one could ever take, to one of the worst. Sounds like a pretty bizzare hypersensitivity to me!
Does it really make that much more sense to have this discontinuity at exactly zero, rather than at some other number? I would say if an action is a terrible idea at 0% probability of success, an increase of .0000000001% *should probably not make it a good idea*.
So either fanaticism is fatally flawed, or bizarre hypersensitivity is not a fatal objection towards fanaticism’s competitors.
-"Now, what should you do when you conclude fanaticism is right? Well, a fanatic thinks that their sole aim should be to maximize the odds of infinite value. Keeping the world around probably does that, so you should donate to Longtermist charities (especially because they potentially increase the number of people ever born, thus giving more people a chance of getting into heaven). Maybe you should also try to become religious to increase your odds of getting into Heaven, but that’s more controversial."
Longtermist charities only give you a chance of infinite value if the heat death of the universe is false. So basically you are saying, fanaticism means we should assume that either the afterlife is real or the laws of thermodynamics are wrong, as the alternative has only finite value and thus should be ignored.
There is also the issue that if you base your decisions on value from the afterlife, there's no point in multiple people getting to heaven because infinity plus infinity equals infinity. You could try to build some theory where multiple infinities are better than one but it's not obvious that this doesn't lead to a fanaticism in favor of "infinitely many infinities", a process which could be repeated ad nauseum.
For the record, my solution is to assume that the laws of thermodynamics are not wrong and that the afterlife doesn't exist. I think that will make your hypothetical scenarios impossible but let me know if I missed something.
On one hand I want to be a fanatic because I made a rather simple argument that we should accelerate AI for the low probability at achieving eternal consciousness (on the assumption that eternal consciousness will result in infinite value). However, on reflection, on the other hand, I think I should not be a fanatic because the actual payouts trend towards zero as the payout approaches infinity. I think this is Huemer's point in Approaching Infinity, but I could be misguided in applying that problem here (or I could be wrong in interpreting his point as that, or both). I also think the mugger is a convincing reason to give up the wager, and I don't think I understand your reply as to why you should accept the mugging (or why if you shouldn't?).
Maybe this is no different than the mugger (or no different than what I believe is a similar example Huemer gives in Approaching Infinity), but assume a man approaches you while you are walking home one day and offers you one util for free. You go sure! Then the man says, but wait, if you let me cause -1 util of harm to you, then I will meet you here tomorrow same time and give you 2 utils! You accept, then tomorrow the man meets you and offers to instead meet you the next day same time and give you 4 utils if he can cause you -2 utils of harm at this time. This results in accepting indefinite suffering chasing an infinite payout. Now, the same reasoning for why I think one should reject this is back to (what I think is) Huemer's point, which is that the actual payouts will not be feasible or even possible after some point. However, that appears to leave one to also reject the speculative payout in the first place (like in Pascal's Wager). So rejecting one under this reasoning, rejects both, while accepting one, seems to accept the other.
> If you’re a fanatic, you should spend your life chasing infinite value.
One problem with this view is that if you accept that any decision has non-0 chance of infinite value, you seem committed to the view that *every* action has a non-0 chance of infinite value, which leaves you without any basis on which to make decisions (since the expected payoff of all actions is now infinite).
I don't see any problem with presuming in the real world, as the value of some_payoff approaches infinity, E[some_payoff] tends to approach 0. This seems consistent with our experience (if I were to actually Pascal's mug you, I imagine you'd find my offer more compelling if I were to offer you £10 tomorrow than if I were to offer you £10^100. If not, please let me know, and I'll send you payment details), and solves most of the problems you mention with expected utility maximisation for either infinities or very large numbers.
It requires a little trust in an underlying expectation-bounding mechanism we (or at least I) can't elucidate yet, but as bullets go that seems a better one to bite than allowing yourself to be trivially mugged out of arbitrary amounts of money.
I think the rational thing for Dave to do in the initial parable is to simply draw an arbitrary line of what probability he deems acceptable (whether it be 60, 75, 90, or some other percentage is left as an excercise for the reader) and stick to it no matter how much surface-level better the next step is.
This is because the next step isn’t just the next step, it’s committing yourself to all future deals in the chain which eventually lead to a near-certain probability of getting nothing. So the choice isn’t really between 60% chance of saving 100^100 people and a 59% chance of saving 100^100^100 people, it’s between the former and a 99.999999999999% chance of nothing, which is worse no matter what you get on the 0.00000000000000001% chance, because it’s overwhelmingly likely you get nothing.
The only way you avoid committing yourself to the fanatic chain is if you draw the arbitrary line and stick to it even if it seems irrational. You can debate where precisely that line should be (probably above 50% at the very least) but the person who picks a line and sticks to it has a good shot of keeping their winnings, whereas the fanatic is nearly guaranteed to lose everything.
I’m pretty sure this doesn’t violate transitivity: the premise of my argument is that the next step is actually just worse, because it’s not what it appears to be.
I also don’t think it’s timidity, because it doesn’t require you draw the line at 100%.
I’m sure it requires me to bite various bullets, but the button-press bullet from my comment the other day seems to be a vastly worse bullet to bite than anything my position requires.
An obvious counter to EV fanaticism, is that if we all became EV fanatics, civilization would collapse. The economy heavily depends on most people pursuing lower-EV, higher-certainty opportunities, if all economic activity became people chasing unicorns, very little would get done and civilization would quickly end. However, some amount of economic activity should be devoted to moonshots (the sort of things EV fanatics would do), and it would be interesting to calculate how much...
And charity too. EV fanaticism means we should stop trying to help out Africa, charitable dollars should all go to charitable moonshots, which runs into the same issues as the economy would.
The article is a beautiful reductio ad absurdum of its own premise. The entire argument hinges on taking a single, overly simplistic rule—expected utility maximization—as the ultimate criterion for rational decision-making. That's not a serious theory. Frankly, the issues the author grapples with are well-known, elementary problems in decision theory that show precisely why that single rule is flawed.
> The entire argument hinges on taking a single, overly simplistic rule—expected utility maximization—as the ultimate criterion for rational decision-making.
This article does the exact opposite of that, at rather insane length.
How exactly? The whole thing is an insanely long defence of fanaticism/expected utility maximisation...
Your comment claimed his argument hinges on taking expected utility maximization as a rule and then deriving fanaticism. His article is instead about how a whole bunch of completely different, much weaker and more plausible principles than expected utility maximization imply fanaticism.
I'm going to apologize that my life situation doesn't allow me the focus to read the whole piece with the attention that it deserves — and thus should not ideally be posting a comment! — but am I correct in thinking that here, you're agreeing to something like Sam Bankman Fried's famous (infamous!) answer to the St. Petersburg question: that he'd continue to flip the coin ad infinitum? (Insofar as I'm asking a question that doesn't even make sense in this context, I apologize.)
No I don't think that's right. If you keep flipping the coin forever you're guaranteed to get nothing, so you shouldn't do that!
That sounds like a very good question he asked though! From a expected value maximization/fanaticism standpoint, when exactly should you stop a Martingale/St. Petersburg situation, where you are being repeatedly given positive expected value bets? Such as $2^n for every tails you get (and losing it all at the first heads). It seems like fanaticism keeps you playing indefinitely because it cannot tell the step you should leave it at while invoking only expected value which is always positive.