I'm somewhat new to this blog, so forgive me if I point out things you've covered before, but:
- Doesn't the whole "future is huge, therefore give money now" point amount to little more than Pascal's mugging? The expected-value calculation can't work (or at least shouldn't compel action) when the expected value is absurdly high, no matter how conservative your calculations are.
- If we're deeply clueless about the sign of long-run effects of saving lives, shouldn't we should be equally (or more) clueless about the long-term effects of funding AI safety orgs, growing EA, or making political donations, all of which could have unpredictable negative second- and third-order consequences over cosmic timescales?
- Given just how uncertain the spatial argument is (do we get a long future? do we end up expanding into space? how do animals or digital minds fare there? do current donations change any of that?) it seems ridiculous to use it as a deciding factor between animal welfare and global health.
- Can you also rank the sub-cause areas? Or the sub-sub-cause areas? Shouldn't you also determine which specific AI safety intervention or insect-sentience weighing people should pursue? At some point, aren't you left with just one optimal action that every EA should pursue? Surely diversification---across sub-cause and cause areas---is good.
Ah, I hadn't connected that the Better Futures series makes growing EA more valuable (since we probably need steering towards the good de dicto). Thanks, that's useful.
Thanks for the post; I agree cross-cause comparison is important!
Some claims I’d make:
1. Digital minds are neglected relative to human-centric AI safety. Many more people are bought into arguments for aligning AI to humans than into the general idea of improving all sentient beings’ welfare. The digital minds research field is also far less developed than human-centric AI alignment.
2. Animal welfare is neglected and underrated relative to most longtermist causes, including AI safety. I agree AI safety is higher EV in the risk-neutral sense due to astronomical waste. But if we have any kind of risk aversion, including difference-making risk aversion or an emphasis on reducing suffering, allocating some of our resources to animal welfare makes a lot of sense. Animal welfare is by far the most cost-effective way to have high confidence we’re making the world a better place, and I think that merits much more than the <10% of resources EA currently allocates to it. This is why I think donations to animal welfare are probably better under risk aversion than most longtermist donations, excepting digital minds.
In my Longtermism piece I explain why I don't think that risk aversion is much help, and I think you need pretty extreme negative utilitarianism to avoid this. Agree with 1.
The overall goal of dissimilar comparisons is certainly important, but there are several issues with both time and uncertainty and effectiveness.
Many longtermism risks will be the same tomorrow as today. But things like animal welfare produced benefits now. So we might donate to animal welfare today, produce some tangible benefit, and then face the same choice tomorrow.
There's also discounting over time for distant future benefits. This is related to the first point, but not quite the same.
And there is uncertainty and effectiveness. Imagine that when we moved from the bronze age to the iron age, someone suggested that with our new knowledge of how different material combine, we might end up with a hugely powerful weapon one day that could destroy civilization and we should start working on some way of preventing this. There's not really much that someone could have done in those day to reduce nuclear proliferation. The same was true in 1900, when we were much closer. Only after the specifics became clear -- what a nuclear bomb was, how it worked, who had it, who was likely to get it -- could the problem be realistically addressed.
AI is here now and we can certainly do important things involving that. (Lots of room for debate on what those are.) But other longterm risks might be sufficiently far in the future with so much that could happened between now and then that it makes sense to wait until the nature of the problem and its solutions becomes clearer.
Disagree on Longtermism: I mean, one thing is that, of course, in the cases where Longtermist interventions prevent extinction, say, then if you waited for tomorrow you would be dead. So the precise cases where the Longtermist interventions do good are the ones where you can't wait. Also, now is an especially good time because it's the height of the AI stuff.
The worry with the bronze age analogy isn't for Longtermism, it's just that that's a particularly idiotic way for them to go about benefitting the long-term future. At that time, the best ways to improve it would be to generally advance civilizations. But now we have atom bombs and AI that can destroy civilization, and there's much that can be done to mitigate that.
While I do agree with this ranking I maintain my skepticism on digital minds being so important. For two reasons
1. Seems like marginal utility potential of minds may decrease exponentially with intelligence. We are much much much more intelligent than say a dog, but maybe only have a slightly greater capacity for utility.
2. Lots of really good reasons to want to control a digital mind intelligence explosion.
3. It seems really unlikely we would want lots of smaller digital minds as opposed to one or a few huge digital minds. Unless they suffer serious problems with multitasking but I don’t see why that would be the case. It seems like very large minds would overall be better. If that’s the case then it seems like they won’t have that much utility seeing point 1.
But that aside I think broadly that this underrates both politics as a separate contemporary frontier and as a longtermism concern. You have written a good article on the former so I’ll skip it. But the latter is under appreciated in my view and a motivating reason for me to go into academia.
But the case is this, political thinkers do change the world on long time horizons. Nationalism for example didn’t happen on its own, it didn’t exist for a long time, a world without nations today would be so different. It may take a long time for ideas to enter the practice, (it took a millennia to make a Locke, and a century for his ideas to forge governments for example(don’t go after the exact timeline on this I’m pulling it out of my memory)) but once it does it can topple empires and save or kill millions (in the long term billions or trillions). So it seems to me that if you can improve governments going forward by even a hundredth of a percent or enable a Locke in 300 years, you do a lot of difference (though maybe that’s me coping lol)
Im not entirely sure how exactly you translate this into an effective donation plan that being said. But I wanted to make a case for why this matters.
1. That wouldn't be decreasing exponentially but increasing (something like) logarithmically. Intelligence alone doesn't seem to automatically beget more welfare, but my guess is that the most efficient conduits for welfare have a lot of it! In any case, the argument just needs numerous digital minds--holds even if they have similar welfare to us.
2. Not sure what this has to do with anything? I agree that don't just want unfettered digital minds, but that's not responsive to the claim that most future welfare is digital.
3. Well presumably it would be good to create lots of small ones if this is the most efficient way to get welfare. Now, maybe it's unlikely we'll actually get that, but that's a reason to work on things that raise the odds of getting it.
1. You’re right that’s a basic mistake on part, clearly been too long since I did math. But yeah all of that seems fair.
2. Yeah it’s just a consideration against many digital mind scenarios
3. Yeah if it does create more utility but it depends on the advantages of consolidated intelligent I think. My suspicion is given limited compute it will be much to our advantage to consolidate.
Yeah I’m unsure if the politics thing can fit into ea since idk how you would make donations cost effective but if you can make it fit it should.
I am afraid that I find this question to have a questionable assumption.
It assumes that there is a single correct answer.
Consider the question, "What food should I eat?" The answer to the question is not a single food. It is a combination of foods each serving different purposes. And, one of the relevant questions to answer is "Which foods do you like?" Because, if there are multiple foods that serve a particular purpose, one might as well choose the option one likes.
Or the question, "Which profession is best?" We need different people involved in different professions. And one of the relevant questions to answer is, "Which profession do you want to be in?" Because, if there are multiple professions to choose from, one might as well choose the profession one likes and leave others to choose the professions they like.
Claiming that one has an obligation to do what is best and then searching for the one "best" food to eat or the one "best" profession to be involved in is not exactly the best way to be approaching the problem.
I agree that there won't automatically be some ranking between two cause areas. Some might just depend too much on the details. But in other cases there might be a ranking, if there are arguments for why even modest progress on one is much better than enormous progress on the other.
What do you think about long term suffering focused organisations like Center for Reducing Suffering and Center on Long Term Risk? They do research on how to mitigate s-risks.
I'm somewhat new to this blog, so forgive me if I point out things you've covered before, but:
- Doesn't the whole "future is huge, therefore give money now" point amount to little more than Pascal's mugging? The expected-value calculation can't work (or at least shouldn't compel action) when the expected value is absurdly high, no matter how conservative your calculations are.
- If we're deeply clueless about the sign of long-run effects of saving lives, shouldn't we should be equally (or more) clueless about the long-term effects of funding AI safety orgs, growing EA, or making political donations, all of which could have unpredictable negative second- and third-order consequences over cosmic timescales?
- Given just how uncertain the spatial argument is (do we get a long future? do we end up expanding into space? how do animals or digital minds fare there? do current donations change any of that?) it seems ridiculous to use it as a deciding factor between animal welfare and global health.
- Can you also rank the sub-cause areas? Or the sub-sub-cause areas? Shouldn't you also determine which specific AI safety intervention or insect-sentience weighing people should pursue? At some point, aren't you left with just one optimal action that every EA should pursue? Surely diversification---across sub-cause and cause areas---is good.
I talk about why I don't think risk discounting undermines the case for Longtermism in the linked Longtermism piece https://benthams.substack.com/p/strong-longtermism-is-simply-correct
Yes we're deeply clueless about their actual sign too. But what you should act on is something like expected value.
Re spatial argument, even if highly uncertain it is still by far the most important thing in expectation given how high the stakes could be.
Agree that diversification is good--I was describing how, in general, EAs should currently behave at the margins.
lol
Ah, I hadn't connected that the Better Futures series makes growing EA more valuable (since we probably need steering towards the good de dicto). Thanks, that's useful.
Thanks for the post; I agree cross-cause comparison is important!
Some claims I’d make:
1. Digital minds are neglected relative to human-centric AI safety. Many more people are bought into arguments for aligning AI to humans than into the general idea of improving all sentient beings’ welfare. The digital minds research field is also far less developed than human-centric AI alignment.
2. Animal welfare is neglected and underrated relative to most longtermist causes, including AI safety. I agree AI safety is higher EV in the risk-neutral sense due to astronomical waste. But if we have any kind of risk aversion, including difference-making risk aversion or an emphasis on reducing suffering, allocating some of our resources to animal welfare makes a lot of sense. Animal welfare is by far the most cost-effective way to have high confidence we’re making the world a better place, and I think that merits much more than the <10% of resources EA currently allocates to it. This is why I think donations to animal welfare are probably better under risk aversion than most longtermist donations, excepting digital minds.
In my Longtermism piece I explain why I don't think that risk aversion is much help, and I think you need pretty extreme negative utilitarianism to avoid this. Agree with 1.
The overall goal of dissimilar comparisons is certainly important, but there are several issues with both time and uncertainty and effectiveness.
Many longtermism risks will be the same tomorrow as today. But things like animal welfare produced benefits now. So we might donate to animal welfare today, produce some tangible benefit, and then face the same choice tomorrow.
There's also discounting over time for distant future benefits. This is related to the first point, but not quite the same.
And there is uncertainty and effectiveness. Imagine that when we moved from the bronze age to the iron age, someone suggested that with our new knowledge of how different material combine, we might end up with a hugely powerful weapon one day that could destroy civilization and we should start working on some way of preventing this. There's not really much that someone could have done in those day to reduce nuclear proliferation. The same was true in 1900, when we were much closer. Only after the specifics became clear -- what a nuclear bomb was, how it worked, who had it, who was likely to get it -- could the problem be realistically addressed.
AI is here now and we can certainly do important things involving that. (Lots of room for debate on what those are.) But other longterm risks might be sufficiently far in the future with so much that could happened between now and then that it makes sense to wait until the nature of the problem and its solutions becomes clearer.
Disagree on Longtermism: I mean, one thing is that, of course, in the cases where Longtermist interventions prevent extinction, say, then if you waited for tomorrow you would be dead. So the precise cases where the Longtermist interventions do good are the ones where you can't wait. Also, now is an especially good time because it's the height of the AI stuff.
On discounting, see https://pdf.stafforini.com/cowen-1992-social-discount-rate.pdf
The worry with the bronze age analogy isn't for Longtermism, it's just that that's a particularly idiotic way for them to go about benefitting the long-term future. At that time, the best ways to improve it would be to generally advance civilizations. But now we have atom bombs and AI that can destroy civilization, and there's much that can be done to mitigate that.
While I do agree with this ranking I maintain my skepticism on digital minds being so important. For two reasons
1. Seems like marginal utility potential of minds may decrease exponentially with intelligence. We are much much much more intelligent than say a dog, but maybe only have a slightly greater capacity for utility.
2. Lots of really good reasons to want to control a digital mind intelligence explosion.
3. It seems really unlikely we would want lots of smaller digital minds as opposed to one or a few huge digital minds. Unless they suffer serious problems with multitasking but I don’t see why that would be the case. It seems like very large minds would overall be better. If that’s the case then it seems like they won’t have that much utility seeing point 1.
But that aside I think broadly that this underrates both politics as a separate contemporary frontier and as a longtermism concern. You have written a good article on the former so I’ll skip it. But the latter is under appreciated in my view and a motivating reason for me to go into academia.
But the case is this, political thinkers do change the world on long time horizons. Nationalism for example didn’t happen on its own, it didn’t exist for a long time, a world without nations today would be so different. It may take a long time for ideas to enter the practice, (it took a millennia to make a Locke, and a century for his ideas to forge governments for example(don’t go after the exact timeline on this I’m pulling it out of my memory)) but once it does it can topple empires and save or kill millions (in the long term billions or trillions). So it seems to me that if you can improve governments going forward by even a hundredth of a percent or enable a Locke in 300 years, you do a lot of difference (though maybe that’s me coping lol)
Im not entirely sure how exactly you translate this into an effective donation plan that being said. But I wanted to make a case for why this matters.
1. That wouldn't be decreasing exponentially but increasing (something like) logarithmically. Intelligence alone doesn't seem to automatically beget more welfare, but my guess is that the most efficient conduits for welfare have a lot of it! In any case, the argument just needs numerous digital minds--holds even if they have similar welfare to us.
2. Not sure what this has to do with anything? I agree that don't just want unfettered digital minds, but that's not responsive to the claim that most future welfare is digital.
3. Well presumably it would be good to create lots of small ones if this is the most efficient way to get welfare. Now, maybe it's unlikely we'll actually get that, but that's a reason to work on things that raise the odds of getting it.
Agree shaping ideas is good Longtermistly.
1. You’re right that’s a basic mistake on part, clearly been too long since I did math. But yeah all of that seems fair.
2. Yeah it’s just a consideration against many digital mind scenarios
3. Yeah if it does create more utility but it depends on the advantages of consolidated intelligent I think. My suspicion is given limited compute it will be much to our advantage to consolidate.
Yeah I’m unsure if the politics thing can fit into ea since idk how you would make donations cost effective but if you can make it fit it should.
I am afraid that I find this question to have a questionable assumption.
It assumes that there is a single correct answer.
Consider the question, "What food should I eat?" The answer to the question is not a single food. It is a combination of foods each serving different purposes. And, one of the relevant questions to answer is "Which foods do you like?" Because, if there are multiple foods that serve a particular purpose, one might as well choose the option one likes.
Or the question, "Which profession is best?" We need different people involved in different professions. And one of the relevant questions to answer is, "Which profession do you want to be in?" Because, if there are multiple professions to choose from, one might as well choose the profession one likes and leave others to choose the professions they like.
Claiming that one has an obligation to do what is best and then searching for the one "best" food to eat or the one "best" profession to be involved in is not exactly the best way to be approaching the problem.
I agree that there won't automatically be some ranking between two cause areas. Some might just depend too much on the details. But in other cases there might be a ranking, if there are arguments for why even modest progress on one is much better than enormous progress on the other.
What do you think about long term suffering focused organisations like Center for Reducing Suffering and Center on Long Term Risk? They do research on how to mitigate s-risks.
Seems fine, probably not great though I don't really know. Largely downstream of different philosophical priorities.