Like many intuitions that run against consequentialist conclusions, the one giving people issues here seems to rely upon an ambiguity, in this case with respect to "replacing". The sense of "replace" relevant to the argument would involve the *theoretically previous* (not temporally previous!) person being substituted in their entirely, including all knowledge, memories and hopes others have concerning them. The sense of "replace" making the conclusion counterintuitive is the sense of one person simply disappearing and another appearing, with the grief, confusion and costly adjustments by others remaining as part of the picture.
Perhaps the problem here is a core difference people see between existence/nonexistence and utility/harm, sort of parallel to how people dont accept the conclusion that a certain amount of toe stubbings over a lifetime is a greater harm than death. If you value the existence of a person (in any state positive or negative) as a matter of first order moral significance you might object to this hypothetical in all but the most extreme pleasure/pain differentials, distorting the applicability to other scenarios.
Oh and change the subheading from “Carpar” to “Caspar” 😁
> For one, it denies the transitivity of active favoring. One actively prefers A to B if, were one unsure whether A or B was actual, they’d hope A was actual.
I don't see how the transitivity of active preferences is denied if favors every step until the one where Bob stops existing. I'm not sure what the argument is supposed to be for that.
It looks like you need some kind of linking premise which asserts that one's normal (not active) preferences should stand in some kind of relationship to their *active* preferences. Without such a linking premise, one can just say "Sure, if I didn't know what was actual, I would prefer P(n) to P(n-1) for all n and therefore (by transitivity) I would prefer P(n) to P(m) for all n > m. However, given that I know what is actual, I prefer P(n) to P(n-1) only if n doesn't breach the threshold determined by what is actual"
> Third, it seems plausible that personal identity is vague. There isn’t a precise fact about how many changes make one no longer the same person. But if it’s vague whether a person is replaced, and you shouldn’t replace a person, then there are a range of actions whose desirability is vague—where there isn’t a precise fact about it. But how should one act in such cases? If one accepts that there are precise facts about what one should do in various circumstances—which they should—then they can’t think that personal identity is both vague and determines what one should do.
If one accepts this, then this undermines much of the intuitive force for Minimal Benevolence (at least for non-Utilitarians). Much of the plausibility of that principle stems from how well it explains our intuitions about treating persons. Without the force of person-affecting intuitions, Minimal Benevolence is much less obvious.
I do however think that the vagueness of personal identity is a problem for one who believes both that (1) moral realism is true and (2) our treatment of persons is a fundamental morally relevant consideration, apart from just aggregate utility (e.g., if you think we should respect the rights of other persons, we should keep our promises to other persons, we should give persons what they deserve, we should be partial towards persons that we stand in special relationship with, we should give special priority to persons who are worse-off, we should care about how utility is distributed among persons, etc. -- in fact, I think almost all of the proposed non-Utilitarian morally relevant factors depend on persons in some way). If one thinks that vagueness doesn't exist objectively and/or thinks that the contours of personal identity depend on social conventions (as I do), then it's not clear why or how a non-objective factor (i.e. personal identity) would have objective fundamental moral relevance.
For every step of the morphing sequence, if you didn't know which was actual, you'd prefer the one that has more morphing. Thus, if active favoring is transitive then so too must be you prefer P(10000000000) to P(1) and thus will bring out P(10000000000) over P1.
I don't think it undermines the intuitive force of minimal benevolence. Even if persons are vague, there are still persons.
1. S actively prefers P(N) to P(1), i.e. S would prefer P(N) to P(1) if S were ignorant of the actual.
2. S does not actually prefer P(N) to P(1), because S is aware of the actual.
If one favors every step until the one where Bob stops existing, then that just commits them to 2, not 1. But 1 is required to violate the transitivity of active preferences.
> I don't think it undermines the intuitive force of minimal benevolence. Even if persons are vague, there are still persons.
I was responding to your claim that "If one accepts that there are precise facts about what one should do in various circumstances—which they should—then they can’t think that personal identity is both vague and determines what one should do." This claim undermines the intuitive force of MB, because the claim implies that treatment of persons specifically (as opposed to some non person-affecting consideration e.g. utility) is not morally relevant.
> Transitivity: if you prefer A to B and B to C you are rationally required to prefer A to C.
I don't think you accept this given your realist perspective. I believe you've said in the past that you take "one is rationally required to do X" to be analytically equivalent to "one should do X". If so, then I think you should mean something like this:
"If you SHOULD prefer A to B and B to C, then you should prefer A to C".
But you wouldn't say:
"If yo DO prefer A to B and B to C, then you should prefer A to C"
Unless you've had some fairly big change in your views recently.
> Rational People are Guided by their Preferences: if there are two options, A and B open to you and you prefer A to B, you will actualize A rather than B. For example, if you’d rather there be a gold bar in the middle of time square to a genocide, then if you are deciding between bringing about one of the two, you should pick the gold bar over the genocide.
Again, here presumably what you mean is "If you SHOULD prefer X, then you should do X". Otherwise, I don't think you accept this.
Then either one of the principles is not true or the principles are equivocating on the normative terms (e.g. "should", "rational", etc.). Here are the three principles again.
1. Transitivity
2. Rational People are Guided by their Preferences
3. Minimal Benevolence
Each of these principles makes some kind of normative claim (i.e. some claim about what one should prefer/do or is rationally required to prefer/do). There's two possibilities for how to interpret those normative claims. Either each normative claims is concerned with instrumental normativity or some of the normative claims are not concerned with instrumental normativity.
If each of the normative claims is concerned with instrumental normativity, then MB looks false, because some agents have preferences not fulfilled by MB. E.g. surely you don't think that these principles implies that instrumental rationality requires all agents to adopt utilitarian preferences/actions?
But if some of the normative claims are NOT concerned with instrumental normativity, then the principles equivocate on the normative terms ("should", "rational", etc.). So those principles alone cannot constitute premises of a valid argument for utilitarianism.
I think you're just mapping the surface form of natural language here directly to the intended logical structure of the claim, instead of recognizing the limitations of English syntax. The intended claim seems to be:
Should((you do prefer A to B & you do prefer B to C) => you do prefer A to C).
Not:
(You do prefer A to B & you do prefer B to C) => should(you do prefer A to C).
On the former statement, if someone shouldn't prefer A to C, then it follows that they shouldn't both prefer A to B and B to C for any value of B. Which is what we want.
I think vagueness of personal identity can also cause problems for Utilitarians. Many of the arguments for e.g. transitivity of preferences derive from money pumps--it's irrational to put yourself in a position to lose money while someone cycles through your preferences.
But if it's vague whether the person who faces the choice between Z and A at the end of a cycle is the same as the person who faced the choice between B and A at the beginning, the argument loses some of its force.
If it's someone else who loses the dollar to get back to the choice that I had initially, it's much less compelling than if it's *me specifically* who loses the dollar.
Another place this could bite is Pareto improvements by adding a new person to an existing population: usually the justification says that, since none of the existing people become worse off, as long as the new person has utility better than non existence, everyone is at least as well off the world with one extra person.
But if personal identity is vague, then it's also vague whether this is a genuine Pareto improvement--to the extent that each person in the original world regards the new person as potentially being identical to them, they should not only regard the addition of that new person as a Pareto improvement if the new person has utility at least as high as they have now.
One can imagine that the new addition is for example going to be a clone of one of the original people, who will have the original's memories up to cloning implanted in their head. But the clone will from then out have a worse life than the original (but still worth living).
The original might regard the clone as personally identical, and now object to the cloning on the basis that they will be made worse off.
This sort of reasoning reminds me strongly of the saturating counterparty relations that Michael St Jules has brought up on Richard's blog (I'm on my phone at lunch, so don't have time to look it up, and can't remember to whom I should actually be attributing the idea).
The idea sounds like a very natural way of applying the reasoning, "since personal identity is vague, for any two worlds A, and B, each person in A should regard each person in B as potentially identical with themselves, and hence should only consider the change to world B a Pareto improvement if it's an improvement for all people in A under all possible mappings of people in A to people in B".
Now, this can sound absurd since personal identity is only *vague*, not *arbitrary*, so it probably makes more sense to imagine some weighting of each injection from A into B by how reasonable it is to regard each person in A as identical with their image in B.
But already this suggests the necessity of a more complicated way of handling the vagueness of personal identity.
Great discussion! I agree with your conclusion re: non-identity cases (where neither A nor B is antecedently actual).
Self-binding is only "clearly irrational" if we should never do things that change our values/preferences, like falling in love. But if it's possible that you should do something that will change your preferences, then your (current) preferences could be better achieved by binding your future self. This would seem to make such self-binding rational (if your current preferences are themselves reasonable).
And, of course, wanting your child not to be replaced by a completely different person seems pretty reasonable, on its face.
To avoid implausibly sharp discontinuities, it may be that you should discount benefits to the counterfactual person the more different they become from your actual child. There will be some optimal point X that best balances benefits to your child vs changes to their psychological identity. After you adjust to that change, and come to care about the new person Child(X), you'll now be tempted to make further changes for *their* sake. But the prospect of this further change would make it *not* worth it, for Child(1)'s sake, to press X buttons in the first place. It will only be worth it if you can self-bind your future self to stop at X.
Imagine if a person said "intransitive preferences are only "clearly irrational" if we shouldn't prefer lots of dust specks to one torture. But clearly we should."
That you shouldn't prevent yourself from doing the right thing in the future is a plausible constraint on theories of rationality like transitivity. I agree that views according to which replacement is bad and Hare is wrong are views that require rejecting it, but that just seems like a counterexample.
Here's another way of seeing the intuition. I agree with your intuition that third parties should hope that people do the right thing. But it seems even more plausibly like you should hope you do the right thing. However, your view violates that constraint in two ways. First, if your values ought to shift, then there are actions that you should take only if you'll do the wrong thing in the future. But if you will only take some act if you will act wrongly in the future, then clearly you want yourself to act wrongly--you're trying to bring it about. For instance, I should only take the first morph+improvement if, in the future, I expect to act wrongly and not take the sequence of other morphs. It seems utterly loony to think an agent who is morally ideal at some time will think "I really hope I do the wrong thing in the future!"
Second, if your values are going to shift in the future, then on your current set of values, you'll want yourself to do the wrong thing. Based on your current value, taking the future morphs will be bad.
In regards to implausibly sharp discontinuities, I think this view is going to have some problems. For many years, I took ADHD medication that resulted in pretty dramatic shifts in my psychology. It doesn't seem like the mere fact that there was a change was even a bit bad for me.
I'd be curious to hear what you make of the points I make here. I don't want to overstate things, but to me, it seems like a pretty knockdown objection to your set of views (and I've recently been convinced by one good philosopher to describe my confidently held, potentially wrong views).
One of the things these pre-commitment arguments make me think of is hostage-taking. In such a case, it seems reasonable to me to pre-commit to actions that on first-order bring about wrong actions--because we believe that this will shape the incentives of others in a way that will be better over all. I commit to not ransoming hostages in advance, even though if a hostage taking arises I believe it's clearly better to save them, in order to *discourage the hostage-taking* to begin with.
Obviously such reasoning doesn't apply here, but I suspect our intuitions about these situations may be influenced by such considerations, and I'd be curious to see if someone can construct a personal-identity shift adversarial strategy that it would be reasonable to oppose by pre-committing to acting as if there is a threshold.
Ok, my first thought of such a case: a person who credibly promises to self-modify into a utility monster. I think it's plausible that we want to discourage the formation of utility monsters not on first-order grounds, but on higher-order grounds: if utility monsters are incentivized, or even not-disincetivized, we probably just end up in a protracted war between competing would-be-UMs that lowers utility for everyone.
But a gradual self-modification has no obvious point where the disincentives should be applied--obviously it's fine for people to try and gain a little more joy out of the ordinary things in their lives. But at the least for strategic reasons, we should have a threshold above which we will not regard a gradual change as better.
It doesn't seem self-evidently irrational to hope you do the wrong thing in the future. It could be that the world will go better for you if you're predictably irrational in some way than if you do your sincere best at each case to steer it correctly. Someone too dumb to defect at the end of an iterated prisoner's dilemma (and too dumb to successfully lie about it) might find themselves in better bargaining positions than a purely rational agent.
I also accept Hare's argument.
Like many intuitions that run against consequentialist conclusions, the one giving people issues here seems to rely upon an ambiguity, in this case with respect to "replacing". The sense of "replace" relevant to the argument would involve the *theoretically previous* (not temporally previous!) person being substituted in their entirely, including all knowledge, memories and hopes others have concerning them. The sense of "replace" making the conclusion counterintuitive is the sense of one person simply disappearing and another appearing, with the grief, confusion and costly adjustments by others remaining as part of the picture.
Perhaps the problem here is a core difference people see between existence/nonexistence and utility/harm, sort of parallel to how people dont accept the conclusion that a certain amount of toe stubbings over a lifetime is a greater harm than death. If you value the existence of a person (in any state positive or negative) as a matter of first order moral significance you might object to this hypothetical in all but the most extreme pleasure/pain differentials, distorting the applicability to other scenarios.
Oh and change the subheading from “Carpar” to “Caspar” 😁
Which part of the argument would one object to?
Fixed!
> For one, it denies the transitivity of active favoring. One actively prefers A to B if, were one unsure whether A or B was actual, they’d hope A was actual.
I don't see how the transitivity of active preferences is denied if favors every step until the one where Bob stops existing. I'm not sure what the argument is supposed to be for that.
It looks like you need some kind of linking premise which asserts that one's normal (not active) preferences should stand in some kind of relationship to their *active* preferences. Without such a linking premise, one can just say "Sure, if I didn't know what was actual, I would prefer P(n) to P(n-1) for all n and therefore (by transitivity) I would prefer P(n) to P(m) for all n > m. However, given that I know what is actual, I prefer P(n) to P(n-1) only if n doesn't breach the threshold determined by what is actual"
> Third, it seems plausible that personal identity is vague. There isn’t a precise fact about how many changes make one no longer the same person. But if it’s vague whether a person is replaced, and you shouldn’t replace a person, then there are a range of actions whose desirability is vague—where there isn’t a precise fact about it. But how should one act in such cases? If one accepts that there are precise facts about what one should do in various circumstances—which they should—then they can’t think that personal identity is both vague and determines what one should do.
If one accepts this, then this undermines much of the intuitive force for Minimal Benevolence (at least for non-Utilitarians). Much of the plausibility of that principle stems from how well it explains our intuitions about treating persons. Without the force of person-affecting intuitions, Minimal Benevolence is much less obvious.
I do however think that the vagueness of personal identity is a problem for one who believes both that (1) moral realism is true and (2) our treatment of persons is a fundamental morally relevant consideration, apart from just aggregate utility (e.g., if you think we should respect the rights of other persons, we should keep our promises to other persons, we should give persons what they deserve, we should be partial towards persons that we stand in special relationship with, we should give special priority to persons who are worse-off, we should care about how utility is distributed among persons, etc. -- in fact, I think almost all of the proposed non-Utilitarian morally relevant factors depend on persons in some way). If one thinks that vagueness doesn't exist objectively and/or thinks that the contours of personal identity depend on social conventions (as I do), then it's not clear why or how a non-objective factor (i.e. personal identity) would have objective fundamental moral relevance.
For every step of the morphing sequence, if you didn't know which was actual, you'd prefer the one that has more morphing. Thus, if active favoring is transitive then so too must be you prefer P(10000000000) to P(1) and thus will bring out P(10000000000) over P1.
I don't think it undermines the intuitive force of minimal benevolence. Even if persons are vague, there are still persons.
Both of the following can be true for a person S:
1. S actively prefers P(N) to P(1), i.e. S would prefer P(N) to P(1) if S were ignorant of the actual.
2. S does not actually prefer P(N) to P(1), because S is aware of the actual.
If one favors every step until the one where Bob stops existing, then that just commits them to 2, not 1. But 1 is required to violate the transitivity of active preferences.
> I don't think it undermines the intuitive force of minimal benevolence. Even if persons are vague, there are still persons.
I was responding to your claim that "If one accepts that there are precise facts about what one should do in various circumstances—which they should—then they can’t think that personal identity is both vague and determines what one should do." This claim undermines the intuitive force of MB, because the claim implies that treatment of persons specifically (as opposed to some non person-affecting consideration e.g. utility) is not morally relevant.
Some nitpicks on the first two principles:
> Transitivity: if you prefer A to B and B to C you are rationally required to prefer A to C.
I don't think you accept this given your realist perspective. I believe you've said in the past that you take "one is rationally required to do X" to be analytically equivalent to "one should do X". If so, then I think you should mean something like this:
"If you SHOULD prefer A to B and B to C, then you should prefer A to C".
But you wouldn't say:
"If yo DO prefer A to B and B to C, then you should prefer A to C"
Unless you've had some fairly big change in your views recently.
> Rational People are Guided by their Preferences: if there are two options, A and B open to you and you prefer A to B, you will actualize A rather than B. For example, if you’d rather there be a gold bar in the middle of time square to a genocide, then if you are deciding between bringing about one of the two, you should pick the gold bar over the genocide.
Again, here presumably what you mean is "If you SHOULD prefer X, then you should do X". Otherwise, I don't think you accept this.
I was talking about instrumental rationality.
Then either one of the principles is not true or the principles are equivocating on the normative terms (e.g. "should", "rational", etc.). Here are the three principles again.
1. Transitivity
2. Rational People are Guided by their Preferences
3. Minimal Benevolence
Each of these principles makes some kind of normative claim (i.e. some claim about what one should prefer/do or is rationally required to prefer/do). There's two possibilities for how to interpret those normative claims. Either each normative claims is concerned with instrumental normativity or some of the normative claims are not concerned with instrumental normativity.
If each of the normative claims is concerned with instrumental normativity, then MB looks false, because some agents have preferences not fulfilled by MB. E.g. surely you don't think that these principles implies that instrumental rationality requires all agents to adopt utilitarian preferences/actions?
But if some of the normative claims are NOT concerned with instrumental normativity, then the principles equivocate on the normative terms ("should", "rational", etc.). So those principles alone cannot constitute premises of a valid argument for utilitarianism.
I think you're just mapping the surface form of natural language here directly to the intended logical structure of the claim, instead of recognizing the limitations of English syntax. The intended claim seems to be:
Should((you do prefer A to B & you do prefer B to C) => you do prefer A to C).
Not:
(You do prefer A to B & you do prefer B to C) => should(you do prefer A to C).
On the former statement, if someone shouldn't prefer A to C, then it follows that they shouldn't both prefer A to B and B to C for any value of B. Which is what we want.
I think vagueness of personal identity can also cause problems for Utilitarians. Many of the arguments for e.g. transitivity of preferences derive from money pumps--it's irrational to put yourself in a position to lose money while someone cycles through your preferences.
But if it's vague whether the person who faces the choice between Z and A at the end of a cycle is the same as the person who faced the choice between B and A at the beginning, the argument loses some of its force.
If it's someone else who loses the dollar to get back to the choice that I had initially, it's much less compelling than if it's *me specifically* who loses the dollar.
Another place this could bite is Pareto improvements by adding a new person to an existing population: usually the justification says that, since none of the existing people become worse off, as long as the new person has utility better than non existence, everyone is at least as well off the world with one extra person.
But if personal identity is vague, then it's also vague whether this is a genuine Pareto improvement--to the extent that each person in the original world regards the new person as potentially being identical to them, they should not only regard the addition of that new person as a Pareto improvement if the new person has utility at least as high as they have now.
One can imagine that the new addition is for example going to be a clone of one of the original people, who will have the original's memories up to cloning implanted in their head. But the clone will from then out have a worse life than the original (but still worth living).
The original might regard the clone as personally identical, and now object to the cloning on the basis that they will be made worse off.
This sort of reasoning reminds me strongly of the saturating counterparty relations that Michael St Jules has brought up on Richard's blog (I'm on my phone at lunch, so don't have time to look it up, and can't remember to whom I should actually be attributing the idea).
The idea sounds like a very natural way of applying the reasoning, "since personal identity is vague, for any two worlds A, and B, each person in A should regard each person in B as potentially identical with themselves, and hence should only consider the change to world B a Pareto improvement if it's an improvement for all people in A under all possible mappings of people in A to people in B".
Now, this can sound absurd since personal identity is only *vague*, not *arbitrary*, so it probably makes more sense to imagine some weighting of each injection from A into B by how reasonable it is to regard each person in A as identical with their image in B.
But already this suggests the necessity of a more complicated way of handling the vagueness of personal identity.
Great discussion! I agree with your conclusion re: non-identity cases (where neither A nor B is antecedently actual).
Self-binding is only "clearly irrational" if we should never do things that change our values/preferences, like falling in love. But if it's possible that you should do something that will change your preferences, then your (current) preferences could be better achieved by binding your future self. This would seem to make such self-binding rational (if your current preferences are themselves reasonable).
And, of course, wanting your child not to be replaced by a completely different person seems pretty reasonable, on its face.
To avoid implausibly sharp discontinuities, it may be that you should discount benefits to the counterfactual person the more different they become from your actual child. There will be some optimal point X that best balances benefits to your child vs changes to their psychological identity. After you adjust to that change, and come to care about the new person Child(X), you'll now be tempted to make further changes for *their* sake. But the prospect of this further change would make it *not* worth it, for Child(1)'s sake, to press X buttons in the first place. It will only be worth it if you can self-bind your future self to stop at X.
Imagine if a person said "intransitive preferences are only "clearly irrational" if we shouldn't prefer lots of dust specks to one torture. But clearly we should."
That you shouldn't prevent yourself from doing the right thing in the future is a plausible constraint on theories of rationality like transitivity. I agree that views according to which replacement is bad and Hare is wrong are views that require rejecting it, but that just seems like a counterexample.
Here's another way of seeing the intuition. I agree with your intuition that third parties should hope that people do the right thing. But it seems even more plausibly like you should hope you do the right thing. However, your view violates that constraint in two ways. First, if your values ought to shift, then there are actions that you should take only if you'll do the wrong thing in the future. But if you will only take some act if you will act wrongly in the future, then clearly you want yourself to act wrongly--you're trying to bring it about. For instance, I should only take the first morph+improvement if, in the future, I expect to act wrongly and not take the sequence of other morphs. It seems utterly loony to think an agent who is morally ideal at some time will think "I really hope I do the wrong thing in the future!"
Second, if your values are going to shift in the future, then on your current set of values, you'll want yourself to do the wrong thing. Based on your current value, taking the future morphs will be bad.
In regards to implausibly sharp discontinuities, I think this view is going to have some problems. For many years, I took ADHD medication that resulted in pretty dramatic shifts in my psychology. It doesn't seem like the mere fact that there was a change was even a bit bad for me.
I'd be curious to hear what you make of the points I make here. I don't want to overstate things, but to me, it seems like a pretty knockdown objection to your set of views (and I've recently been convinced by one good philosopher to describe my confidently held, potentially wrong views).
One of the things these pre-commitment arguments make me think of is hostage-taking. In such a case, it seems reasonable to me to pre-commit to actions that on first-order bring about wrong actions--because we believe that this will shape the incentives of others in a way that will be better over all. I commit to not ransoming hostages in advance, even though if a hostage taking arises I believe it's clearly better to save them, in order to *discourage the hostage-taking* to begin with.
Obviously such reasoning doesn't apply here, but I suspect our intuitions about these situations may be influenced by such considerations, and I'd be curious to see if someone can construct a personal-identity shift adversarial strategy that it would be reasonable to oppose by pre-committing to acting as if there is a threshold.
Ok, my first thought of such a case: a person who credibly promises to self-modify into a utility monster. I think it's plausible that we want to discourage the formation of utility monsters not on first-order grounds, but on higher-order grounds: if utility monsters are incentivized, or even not-disincetivized, we probably just end up in a protracted war between competing would-be-UMs that lowers utility for everyone.
But a gradual self-modification has no obvious point where the disincentives should be applied--obviously it's fine for people to try and gain a little more joy out of the ordinary things in their lives. But at the least for strategic reasons, we should have a threshold above which we will not regard a gradual change as better.
It doesn't seem self-evidently irrational to hope you do the wrong thing in the future. It could be that the world will go better for you if you're predictably irrational in some way than if you do your sincere best at each case to steer it correctly. Someone too dumb to defect at the end of an iterated prisoner's dilemma (and too dumb to successfully lie about it) might find themselves in better bargaining positions than a purely rational agent.