Another one: the probabilities we should assign to theories depends on multiple factors, e.g. their simplicity, their explanatory power, etc. And it's hard to see how there could be facts about the precise exchange rates at which these factors trade off against each other. So multiple exchange rates are admissible, and these multiple exchange rates give rise to imprecise probabilities.
I think there might be multiple permissible credences that an agent could have, but that an agent should in fact have just one of them. So, when coming to a decision, even if a person could rationally have a credence anywhere between .5 and .7, you have to do your best to figure out which of those credences is likeliest.
Yeah that's another way to go and seems pretty plausible to me. One point against made by Brian Hedden is that it suggests agents can rationally think that their own beliefs are arbitrary, and maybe there's something iffy about that.
This is a big crux for me, and something I'm keen to dig into more. I think having arbitrary beliefs is (epistemically) bad, because, well, "arbitrary" means you've chosen these beliefs (over some imprecise beliefs) for _no reason_. Why does that not seem iffy to you?
Ah, gotcha, sorry. (That doesn't sound like "rationally thinking your own beliefs are arbitrary", then, so I might still be confused what your view is.)
Good stuff! I'm still figuring out where I come down on imprecise probabilities but here's my best case for them. The probabilities that go into calculations of ex ante value aren't subjective probabilities: the probabilities you in fact assign. They're epistemic probabilities: the probabilities that you ought to assign (where one possible view about the probabilities that you ought to assign is that they're the probabilities you would assign after ideal reflection on your evidence). And sometimes (including in cases of cluelessness) we don't know exactly what probabilities we ought to assign, in which case the normatively relevant probabilities are imprecise.
That's a good way of doing it. But having uncertainty about what your subjective credences should ideally be is, of course, compatible with having precise actual credences.
A few followups I'd be keen to discuss, either here or in another convo if you prefer:
1. Say you have credence p in some claim C that you’ve spent a lot of time thinking about. What’s the reason to make your decisions based on p precisely? Maybe p seems intuitively a bit more plausible than p - 0.01 or p + 0.01. And maybe this intuition tracks the credence your ideal self would assign to C, to some degree. But does it track this ideal credence _with perfect precision_? If not, why pin your credence down to p? (Also, why think your ideal self should have precise credences anyway?)
2. To spell out the point I explained kinda poorly at the end of the convo, given this: Suppose we’re forming a credence in a domain where we have very poor feedback loops, a history of crucial considerations piling up, etc. Presumably our intuitions in such a domain will be more poorly calibrated. So I’d think these intuitions track our “ideal” credence with less precision. Hence, more severely imprecise credences in this domain.
3. One “companion in guilt” to rejecting the reflection principle in opaque sweetening is: If we have an unbounded value function, we need to give up either a dominance principle or one of two other super plausible principles (Sec. 4 of https://www.globalprioritiesinstitute.org/wp-content/uploads/Beckstead-Thomas-A-Paradox-for-Tiny-Probabilities-and-Enormous-Values-Version-2.pdf). Bounded value functions with respect to welfare seem really unacceptably arbitrary, so I’m sympathetic to giving up that dominance principle. Likewise, since I find _epistemic_ arbitrariness pretty unacceptable, I’m sympathetic to giving up a pretty similar principle in this context.
Hmm, it's not clear to me why we can say "Infinity has deeply weird mathematical properties that make it possible to produce utterly bizarre results", but can't just as well say "Indeterminate values / incomplete orders have deeply weird mathematical properties that make it possible to produce utterly bizarre results". I might misunderstand your point though.
(Maybe you'd say: "Infinity is at least something we have scientific reasons to think actually exists." But if someone has a bounded value function, infinity doesn't feature in their values.)
I can't remember exactly what you said now but I remember not finding it very intuitive. States in decision theory are usually characterized as something like 'ways that -- for all the agent knows -- the world could be', so heads and tails are still possible states after the coin has been flipped. There's also special relativity stuff suggesting that it should make no difference whether the coin has been flipped.
Giving all the information I currently have about how lotteries A and B turned out, I prefer A to B, but I know that I don’t prefer how lottery A turned out to how lottery B turned out.
The Caspar Hare argument doesn't work against incompleteness induced by imprecise probabilities. If two options are incomparable in virtue of imprecise probabilities, they must be lotteries. That means that no violation of reflection is implied.
Wouldn't it still work so long as you just do a lottery over lotteries. So if, for instance, you say that 1,000 dollars given to ex risk reduction is incomparable to 1,000 dollars given to S-risk reduction, then you can do:
A coin is flipped.
Two lotteries:
Lottery 1: If heads, 1000 are given to ex risk reduction. If tails 1,000 are given to S-risk reduction.
Lottery 2: if heads, 1001 is given to S-risk reduction. If tails, 1,000 is given to ex-risk reduction.
That's what I was imagining, but that won't imply violations of Reflection understood as: ‘If I know that I won't prefer the outcome of A to the outcome of B conditional on any state, then I don't prefer A to B.’ It will still imply violations of an eventwise version of Reflection, but that's not quite as plausible.
The eventwise version of Reflection here would be:
'If I know that I won't prefer A to B conditional on heads, and I know that I won't prefer A to B conditional on tails, then I don't prefer A to B.'
The agent with imprecise probabilities violates that, but they won't necessarily violate the statewise version of Reflection, because they might prefer A to B conditional on some specific state within one of the events (where event = set of states). For example, they might prefer A to B conditional on heads and their $1000 counterfactually preventing an x-catastrophe.
I guess another way to put this point is: statewise reflection is the analogue of statewise dominance, and eventwise reflection is the analogue of the sure-thing principle.
You switched from talking about imprecise probabilities to incompleteness, but you can have imprecise probabilities without incompleteness, e.g. if you use a maximin rule to choose prospects.
Yeah that sounds right. Or at least for something like ‘cluelessness-induced permissivism.’ It seems like people use the word ‘cluelessness’ differently in different contexts, sometimes just referring to the phenomenon of having credence ~50% that your action will turn out objectively best.
Harvey Lederman's stuff on negative dominance is a better argument against incompleteness than the Caspar Hare one. Harvey shows that incompleteness -- together with some plausible principles -- implies that A can be better B even though *no* possible outcome of A is better than *any* possible outcome of B.
Another one: the probabilities we should assign to theories depends on multiple factors, e.g. their simplicity, their explanatory power, etc. And it's hard to see how there could be facts about the precise exchange rates at which these factors trade off against each other. So multiple exchange rates are admissible, and these multiple exchange rates give rise to imprecise probabilities.
I think there might be multiple permissible credences that an agent could have, but that an agent should in fact have just one of them. So, when coming to a decision, even if a person could rationally have a credence anywhere between .5 and .7, you have to do your best to figure out which of those credences is likeliest.
Yeah that's another way to go and seems pretty plausible to me. One point against made by Brian Hedden is that it suggests agents can rationally think that their own beliefs are arbitrary, and maybe there's something iffy about that.
Doesn't seem so iffy to me.
This is a big crux for me, and something I'm keen to dig into more. I think having arbitrary beliefs is (epistemically) bad, because, well, "arbitrary" means you've chosen these beliefs (over some imprecise beliefs) for _no reason_. Why does that not seem iffy to you?
I don't think it implies that. There might be a rational range of possible beliefs but some of them strike you as more plausible than others.
I think a person could rationally be a deontologist but I'm not one!
Ah, gotcha, sorry. (That doesn't sound like "rationally thinking your own beliefs are arbitrary", then, so I might still be confused what your view is.)
Anyway, even if some numbers in the credal interval [a, b] strike you as more plausible than others, this doesn't imply it's non-arbitrary to adopt a precise number from [a, b] _as your actual credence_. The precise degrees of plausibility you put on each number in [a, b], to get your all-things-considered precise credence, will themselves be arbitrary. I say more on this here: https://forum.effectivealtruism.org/posts/NKx8sHcAyCiKT723b/should-you-go-with-your-best-guess-against-precise#_Aggregating_our_representor_with_higher_order_credences_uses_more_information__
Good stuff! I'm still figuring out where I come down on imprecise probabilities but here's my best case for them. The probabilities that go into calculations of ex ante value aren't subjective probabilities: the probabilities you in fact assign. They're epistemic probabilities: the probabilities that you ought to assign (where one possible view about the probabilities that you ought to assign is that they're the probabilities you would assign after ideal reflection on your evidence). And sometimes (including in cases of cluelessness) we don't know exactly what probabilities we ought to assign, in which case the normatively relevant probabilities are imprecise.
That's a good way of doing it. But having uncertainty about what your subjective credences should ideally be is, of course, compatible with having precise actual credences.
But on this view your actual credences are not the normatively relevant ones. It's your epistemic credences that are normatively relevant.
Thanks again for the conversation!
A few followups I'd be keen to discuss, either here or in another convo if you prefer:
1. Say you have credence p in some claim C that you’ve spent a lot of time thinking about. What’s the reason to make your decisions based on p precisely? Maybe p seems intuitively a bit more plausible than p - 0.01 or p + 0.01. And maybe this intuition tracks the credence your ideal self would assign to C, to some degree. But does it track this ideal credence _with perfect precision_? If not, why pin your credence down to p? (Also, why think your ideal self should have precise credences anyway?)
2. To spell out the point I explained kinda poorly at the end of the convo, given this: Suppose we’re forming a credence in a domain where we have very poor feedback loops, a history of crucial considerations piling up, etc. Presumably our intuitions in such a domain will be more poorly calibrated. So I’d think these intuitions track our “ideal” credence with less precision. Hence, more severely imprecise credences in this domain.
3. One “companion in guilt” to rejecting the reflection principle in opaque sweetening is: If we have an unbounded value function, we need to give up either a dominance principle or one of two other super plausible principles (Sec. 4 of https://www.globalprioritiesinstitute.org/wp-content/uploads/Beckstead-Thomas-A-Paradox-for-Tiny-Probabilities-and-Enormous-Values-Version-2.pdf). Bounded value functions with respect to welfare seem really unacceptably arbitrary, so I’m sympathetic to giving up that dominance principle. Likewise, since I find _epistemic_ arbitrariness pretty unacceptable, I’m sympathetic to giving up a pretty similar principle in this context.
I basically don't think counterexamples involving infinites are real :) https://benthams.substack.com/p/why-im-a-fanatic
Hmm, it's not clear to me why we can say "Infinity has deeply weird mathematical properties that make it possible to produce utterly bizarre results", but can't just as well say "Indeterminate values / incomplete orders have deeply weird mathematical properties that make it possible to produce utterly bizarre results". I might misunderstand your point though.
(Maybe you'd say: "Infinity is at least something we have scientific reasons to think actually exists." But if someone has a bounded value function, infinity doesn't feature in their values.)
The coin having already been flipped makes no difference. You should still prefer A+/B. You're still uncertain about which state is actual.
What do you make of the intuition that I cite?
I can't remember exactly what you said now but I remember not finding it very intuitive. States in decision theory are usually characterized as something like 'ways that -- for all the agent knows -- the world could be', so heads and tails are still possible states after the coin has been flipped. There's also special relativity stuff suggesting that it should make no difference whether the coin has been flipped.
The core intuition is something like:
Giving all the information I currently have about how lotteries A and B turned out, I prefer A to B, but I know that I don’t prefer how lottery A turned out to how lottery B turned out.
But that's just a past-tense version of the previous intuition, right? That shift doesn't affect my judgements much.
The Caspar Hare argument doesn't work against incompleteness induced by imprecise probabilities. If two options are incomparable in virtue of imprecise probabilities, they must be lotteries. That means that no violation of reflection is implied.
Wouldn't it still work so long as you just do a lottery over lotteries. So if, for instance, you say that 1,000 dollars given to ex risk reduction is incomparable to 1,000 dollars given to S-risk reduction, then you can do:
A coin is flipped.
Two lotteries:
Lottery 1: If heads, 1000 are given to ex risk reduction. If tails 1,000 are given to S-risk reduction.
Lottery 2: if heads, 1001 is given to S-risk reduction. If tails, 1,000 is given to ex-risk reduction.
That's what I was imagining, but that won't imply violations of Reflection understood as: ‘If I know that I won't prefer the outcome of A to the outcome of B conditional on any state, then I don't prefer A to B.’ It will still imply violations of an eventwise version of Reflection, but that's not quite as plausible.
Wait sorry what's the difference?
The eventwise version of Reflection here would be:
'If I know that I won't prefer A to B conditional on heads, and I know that I won't prefer A to B conditional on tails, then I don't prefer A to B.'
The agent with imprecise probabilities violates that, but they won't necessarily violate the statewise version of Reflection, because they might prefer A to B conditional on some specific state within one of the events (where event = set of states). For example, they might prefer A to B conditional on heads and their $1000 counterfactually preventing an x-catastrophe.
I guess another way to put this point is: statewise reflection is the analogue of statewise dominance, and eventwise reflection is the analogue of the sure-thing principle.
You switched from talking about imprecise probabilities to incompleteness, but you can have imprecise probabilities without incompleteness, e.g. if you use a maximin rule to choose prospects.
Agree! But for cluelessness to go through you need incompleteness I think.
Yeah that sounds right. Or at least for something like ‘cluelessness-induced permissivism.’ It seems like people use the word ‘cluelessness’ differently in different contexts, sometimes just referring to the phenomenon of having credence ~50% that your action will turn out objectively best.
Only 13 minutes in, but it's a super dope conversation! Haven't been this enthusiastic about a podcast in a long time.
in the future could you cross post these to YouTube? sub stack doesn't have a way to play offline on Mobile afaict
Great episode! I now feel less clueless about cluelessness, but perhaps more clueless overall
Minor amendment to what I said about (imprecise) credences being primitives: Another view is that the primitives are our _comparative_ probability judgments, i.e., "A is [more likely than] / [less likely than] / [equally likely to] / [none of the above] B". And credences are representations of our sets of probability comparisons. More on this here and references therein: https://forum.effectivealtruism.org/posts/NKx8sHcAyCiKT723b/should-you-go-with-your-best-guess-against-precise#Appendix__Indeterminacy_for_ideal_agents
Harvey Lederman's stuff on negative dominance is a better argument against incompleteness than the Caspar Hare one. Harvey shows that incompleteness -- together with some plausible principles -- implies that A can be better B even though *no* possible outcome of A is better than *any* possible outcome of B.
Interesting, I'll check it out.
There's also the Dorr, Nebel, and Zuehl paper which you might like, particularly the ‘chains of incomparability’ argument.
I remember years ago reading their "the case for comparability." Is it in there?
Yep that's the one!