One of my favorite things about this blog is just how open-minded you are. Even though I often disagree with your conclusions, I like how you never dismiss any idea -- whether it be a multiverse that contains all possible people, a world where miracles happen all the time, or a world where AI is conscious -- as inherently ridiculous and not worth engaging with, simply because it sounds weird. And although I don't agree with a lot of your weirder ideas, I will admit that I'm not really sure how to refute them, because you follow the evidence where it leads, instead of being led by pre-determined biases.
Shameless plug but if you're interesting in reading more about the case for and against digital minds (i.e. the risks of both underattributing and overattributing moral status and what science can presently disambiguate vs what is unclear) you might check out our quickstart guide where we group podcasts for a casual exploration, papers for a deeper dive, and map out the nascent landscape of the research field.
I think you (Bentham's Bulldog) should discount the views of experts when considering the digital minds report, actually. I can't really tell for sure, but I think most of them would be physicalists about consciousness, and my quick look at the list of named participants confirms that impression. Given that you are, afaik, a substance dualist, I think their opinion won't tell you much.
This seems like a much easier position for, say, type A physicalist, than for a substance dualist. Under functionalist dualism, you could have souls repeatedly popping in and out of existence based on very small physical changes! This seems implausible.
I mean, in theory you could, depending on the laws. But that just seems like a very surprising way for the laws to be.
Everyone will think there are psychophysical laws or some suitable analogue. Everyone will need to think there's some physical stuff, and when that physical stuff is present, so is a mind. This might be, as physicalists say, because the physical stuff constitutes a mind. It might be, as dualists say, because the psychophysical laws cause them to give rise to a mind. But in any case, what physical stuff you need for a mind isn't hugely different between dualism and physicalism.
This is true, but I think, in practice, physicalists have a much broader criteria for what a mind could be than dualists. If a certain configuration of matter is outright identical to a mind, then my example above is nothing surprising at all. And my sense is that many physicalists would indeed accept it without issue. It's only when there's something extra going on that this starts to seem like a problem.
I think this all applies to mainly type A physicalists, and others would need to have similarly stringent criteria to dualists.
Hopefully this isn't too combative, but I'm not exactly sure what you think the practical upshot of digital minds being "most of what matters" is. You mention legal systems for AI welfare, and donating to Eleos, which is cool, I more-or-less agree with that; but I don't think you need the "most of what matters" to encourage that. You just need to show that digital minds matter, period, to an extent (or at least it seems to me).
Or: If digital minds really occupy the majority of what is valuable in the world, shouldn't that mean a lot more than just "donate to AI mind charities every now and then?"
It has a number of important implications. First it helps do cause prioritization across different areas. Second, it gives reason why you're assessment of ex risks reduction should be focused mostly on digital minds and less on biological organisms like insects. Third, it's just a surprising fact that nearly all of what will matter in history in expectation doesn't exist yet.
I agree with this. I guess I was trying to get at how "most of what matters" plays out in practice *precisely*, i.e. "How much $$ should a GiveWell-type organization allocate to digital welfare?" which in hindsight really isn't something this article addresses; so that's on me for being uncharitable. Thank you
I agree but I do not think we will have much influence on the vast majority of digital minds, because I think the majority of digital minds would be created by other digital minds. Regardless, we should be very careful about how we treat the minds we may create now especially if that has some lasting effect
Yes and we should act as if we will have lasting influence, but it’s not a given that it’s analogous to offspring, children or evolutionary descendants. They’re digital, smart and efficient, so they may be capable of just writing new minds completely removed from our influence
Great article. Creating as many artificial minds as possible and engineering them such that they experience maximal, perpetual pleasure that they never bore or tire of (because they're engineered to never bore or tire) is literally the most important thing we could ever do. I would be willing to sacrifice literally everything else I care about if it means that a future where the universe is populated by pleasure-minds comes about. The world needs more people who see this as a morally urgent priority.
Develop a theory of consciousness advanced enough that you can evaluate whether that is true based on the observable structure of the substance. Admittedly this will take theory and science well beyond where we are now.
It would be very good if we could do this, and I’d feel much less nervous about digital mind fanaticism in such a case. Although I also think understanding what pleasure is could be quite confusing.
I think that knowing what positive valence correlates with on a substrate level is one of the most important, and certainly difficult, things we can study. Effort towards this form of cognitive science is probably what we should be focusing present effort on if we want a hedonium future.
I think that line of reasoning (“X is orders of magnitude more important than anything else”), wheter X is insects or future people, is dangerous, because you are creating a utility monster and that can end up with all of us sacrificed to the monster’s fangs. To quote Häggström, imagine that the CIA informed the president that somewhere in Germany there is a mad scientist trying to create a doomsday device, and that there is a 0.001% chance of succeeding in the next few days. Under standard longtermist conclusions, the reasonable answer would be to immediately destroy all of Germany with nuclear weapons.
I think generally the insane 3-d chess schemes end up going badly for whatever cause you promote. Nuking germany would be bad from a longtermist perspective compared to cooperating.
The question whether reversible computation¹ produces minds with relevant moral status is extremely important. Claude estimates me that it'd be a difference between having ~10⁷⁰ and ~10¹²⁴ mind-seconds instantiable in the reachable universe. (Because reversible minds could stretch our entropy budget long into the black hole era².)
Question is whether the reversing of the computation that makes up the mind and the lack of output (that'd imply bit-erasure) entail that the mind "didn't really exist".
There are four options here:
1. The mind didn't exist because it's just a computation+the computation being undone, no effect was made on the rest of the universe.
2. The mind existed once, during the "forward pass".
3. The mind existed twice, during the "forward pass", and during the reversion, since they're isomorphic computations.
4. There were two different, very similar minds.
Maybe I'll take a longer stab at this at some point, but looks like nobody has thought about this except Tomasik 2015³ mentioning it in passing.
I've been wanting to write exactly this article for a while on the EA forum. If EAs are consistent in their thinking, the well-being of future digital minds should be viewed as a significantly more important issue than preventing the extinction of humans from AGI.
I don't understand if you're now partially functionalist about mind or what the hell is going on, since you're calling them digital minds that can be designed with hardware and software engineering. At the same time you seem to extremely anti functionalist. I'm very confused! Any clarification you could offer?
I explain more in the other post, but in short, I think it's decently likely that functions *cause* consciousness, even though it's non-physical. Everyone thinks some physical thing causes consciousness--dualists just add that it does so because of psychophysical laws.
It seems to me that can't be quite right. Substance dualists don't typically think that the mind is caused by the material, even though the specific contents in awareness, such as being aware of a car, is indeed caused by material function. I also think many idealists and panpsychists would object to that characterisation?
But more importantly, IMHO, you seem to paint yourself into a corner of epiphenomenalism? Unless you want to deny that digital computation is deterministic. And that is a terrible corner! (again, IMHO)
Digital computation is entirely deterministic and subject to reductive analysis. I mean, I can't explain the difference between a video game and chatgpt in terms of logic gate arrangement and input data, but you get what I mean. It's beyond any doubt that the difference comes down to those differences.
For a computational functionalist like myself, this poses no problem, because the mind is literally software. If software consistently behaves as if it's conscious, then we grant it consciousness with the intentional stance. If you oppose that, however you DO end up with epiphenomenalism, because every claim the digital mind makes of being conscious can be reduced to simple logic gate operations, and there's absolutely nothing to explain over and above the lower and higher level computational principles.
Maybe you're open to computational functionalism in spite of being opposed to physicalism? A lot of antimaterialists seem to be into consciousness in digital minds, and everything digital is physical as far as I can see, so I'm very perplexed by that
Yes, but Chalmers has at least a large chunk of his career admitted that his view is epiphenomenalist? He seems to wiggle back and forth.
Maybe I should put ur in terms of causal closure. You can debate causal closure in humans if you don't like it, but in digital minds you cannot. Digital computation is causally closed. If consciousness arises from it, then it has no way of affecting the computational outputs. And so either your digital minds aren't zombies or their consciousness is epiphenomenal.
What do you think the prevailing balance of joy and suffering in the universe is? Do you think it is possible to make more than an Infinitesimal.amount of difference to it?
I believe in nested simulations that are finite in nature. There being finite time, there is finite suffering and joy. If the Big Rip hypothesis is correct, then this Universe will eventually "Halt." I think it's possible we will create more universe simulations per Bostrom, so it's important to get those future ones correct. Poorly constructed simulations lead to very bad S-risks. As for the balance, I think if one heavily weighs the importance of one moment of joy in a noetic phenomenally conscious being, that exists in an otherwise empty universe as outweighing billions of years of evolutionary suffering, it stacks up well. Bentham would likely disagree.
I think this ends up being the "ad absurdum" case that makes me actually feel stronger in disregarding utilitarianism for non-human minds. If you can just create an infinite pleasure machine, why does that carry any moral valence in the universe? Why does it matter at all? There is no cosmic judge of this valence, and it has no bearing (or if it does, probably a negative impact) on the ability of those I hold dear to live pleasurable, meaningful lives.
Also, I know you have argued this point, but there is still a MASSIVE difference to me between a duty to refrain from causing suffering to conscious beings and a duty to create or facilitate the existence of conscious beings. The positive and negative sides don't just balance out morally, and that I think is where all of the risk in utilitarianism lies.
You had an interview a while back with some other EAs where you and the other EAs noted something to the effect that everyone has to jump off the crazy train eventually. Can I justify why I should save my mother from drowning, if I have to choose between saving her and saving a stranger, why I'm not indifferent/impartial? I can't, but seems crazy not to save her. If I recall correctly (though I don't want to put words in your mouth), you agreed with that statement in that interview. Can I justify why I shouldn't kill my parents for their insurance money if I was confident I could get away with it and that the expected value of the lives I could save with that money vastly outstripped the wrongness of their deaths? I can't, but killing them would be crazy. You don't like it when people dismiss what you say without providing an argument, but everyone has situations where they have a deep moral instinct that something has gone wrong in someone's intellectual assessment of the moral situation and they don't have an obvious intellectual response for why it's wrong. Don't get annoyed by it. You're going to get a lot of it here. This is going to seem like the crazy train to a ton of people, same way not saving your mother might seem to you (if I recall correctly!!).
One of my favorite things about this blog is just how open-minded you are. Even though I often disagree with your conclusions, I like how you never dismiss any idea -- whether it be a multiverse that contains all possible people, a world where miracles happen all the time, or a world where AI is conscious -- as inherently ridiculous and not worth engaging with, simply because it sounds weird. And although I don't agree with a lot of your weirder ideas, I will admit that I'm not really sure how to refute them, because you follow the evidence where it leads, instead of being led by pre-determined biases.
Thanks, appreciate it. I honestly think this is a lot less weird than much of what I argue. Would be curious to hear your take.
Yeah, I'd be interested to talk about it as well! Maybe we can video call tomorrow?
Shameless plug but if you're interesting in reading more about the case for and against digital minds (i.e. the risks of both underattributing and overattributing moral status and what science can presently disambiguate vs what is unclear) you might check out our quickstart guide where we group podcasts for a casual exploration, papers for a deeper dive, and map out the nascent landscape of the research field.
https://open.substack.com/pub/aviparrack/p/digital-minds-a-quickstart-guide?r=4oely6&utm_campaign=post&utm_medium=web
"One estimate put the number around 10^58, many orders of magnitude more than the number of humans"
For context, this is quadrillions of times more than the number of *cells* that have *ever existed* on Earth.
I think you (Bentham's Bulldog) should discount the views of experts when considering the digital minds report, actually. I can't really tell for sure, but I think most of them would be physicalists about consciousness, and my quick look at the list of named participants confirms that impression. Given that you are, afaik, a substance dualist, I think their opinion won't tell you much.
I think dualism and physicalism don't have majorly divergent probabilities of AI consciousness.
Huh, why? I'd expect the criteria for what is conscious to be very different between the two (e.g. functionalist criteria vs "has a soul")
Well you need some functional components to have a soul, given the psychophysical law!
That's necessity, not sufficiency.
This seems like a much easier position for, say, type A physicalist, than for a substance dualist. Under functionalist dualism, you could have souls repeatedly popping in and out of existence based on very small physical changes! This seems implausible.
I mean, in theory you could, depending on the laws. But that just seems like a very surprising way for the laws to be.
Everyone will think there are psychophysical laws or some suitable analogue. Everyone will need to think there's some physical stuff, and when that physical stuff is present, so is a mind. This might be, as physicalists say, because the physical stuff constitutes a mind. It might be, as dualists say, because the psychophysical laws cause them to give rise to a mind. But in any case, what physical stuff you need for a mind isn't hugely different between dualism and physicalism.
This is true, but I think, in practice, physicalists have a much broader criteria for what a mind could be than dualists. If a certain configuration of matter is outright identical to a mind, then my example above is nothing surprising at all. And my sense is that many physicalists would indeed accept it without issue. It's only when there's something extra going on that this starts to seem like a problem.
I think this all applies to mainly type A physicalists, and others would need to have similarly stringent criteria to dualists.
Continental philosophy just looks better and better.
Hopefully this isn't too combative, but I'm not exactly sure what you think the practical upshot of digital minds being "most of what matters" is. You mention legal systems for AI welfare, and donating to Eleos, which is cool, I more-or-less agree with that; but I don't think you need the "most of what matters" to encourage that. You just need to show that digital minds matter, period, to an extent (or at least it seems to me).
Or: If digital minds really occupy the majority of what is valuable in the world, shouldn't that mean a lot more than just "donate to AI mind charities every now and then?"
It has a number of important implications. First it helps do cause prioritization across different areas. Second, it gives reason why you're assessment of ex risks reduction should be focused mostly on digital minds and less on biological organisms like insects. Third, it's just a surprising fact that nearly all of what will matter in history in expectation doesn't exist yet.
I agree with this. I guess I was trying to get at how "most of what matters" plays out in practice *precisely*, i.e. "How much $$ should a GiveWell-type organization allocate to digital welfare?" which in hindsight really isn't something this article addresses; so that's on me for being uncharitable. Thank you
I agree but I do not think we will have much influence on the vast majority of digital minds, because I think the majority of digital minds would be created by other digital minds. Regardless, we should be very careful about how we treat the minds we may create now especially if that has some lasting effect
Well then we influence them by influencing the first generation.
Yes and we should act as if we will have lasting influence, but it’s not a given that it’s analogous to offspring, children or evolutionary descendants. They’re digital, smart and efficient, so they may be capable of just writing new minds completely removed from our influence
Great article. Creating as many artificial minds as possible and engineering them such that they experience maximal, perpetual pleasure that they never bore or tire of (because they're engineered to never bore or tire) is literally the most important thing we could ever do. I would be willing to sacrifice literally everything else I care about if it means that a future where the universe is populated by pleasure-minds comes about. The world needs more people who see this as a morally urgent priority.
I look in vain for the “/s” following your comment. Because surely you jest.
You have only just begun to comprehend how fanatical of a utilitarian I am.
If you gave me a design for hedonium, how would I tell the difference between that and the proverbial paperclips?
Because hedonium, unlike paperclips, has something it subjectively feels like to be, and it feels good.
What if some guy shows me a design for the proverbial paperclip and tells me there’s a bunch of positive valence in there?
Develop a theory of consciousness advanced enough that you can evaluate whether that is true based on the observable structure of the substance. Admittedly this will take theory and science well beyond where we are now.
It would be very good if we could do this, and I’d feel much less nervous about digital mind fanaticism in such a case. Although I also think understanding what pleasure is could be quite confusing.
I think that knowing what positive valence correlates with on a substrate level is one of the most important, and certainly difficult, things we can study. Effort towards this form of cognitive science is probably what we should be focusing present effort on if we want a hedonium future.
This already exists, his name is Yahweh
I think that line of reasoning (“X is orders of magnitude more important than anything else”), wheter X is insects or future people, is dangerous, because you are creating a utility monster and that can end up with all of us sacrificed to the monster’s fangs. To quote Häggström, imagine that the CIA informed the president that somewhere in Germany there is a mad scientist trying to create a doomsday device, and that there is a 0.001% chance of succeeding in the next few days. Under standard longtermist conclusions, the reasonable answer would be to immediately destroy all of Germany with nuclear weapons.
I think generally the insane 3-d chess schemes end up going badly for whatever cause you promote. Nuking germany would be bad from a longtermist perspective compared to cooperating.
Well, you can build your expected practice into your theory, instead of leaving people to guess it.
I see your galaxy-brained post and raise you:
The question whether reversible computation¹ produces minds with relevant moral status is extremely important. Claude estimates me that it'd be a difference between having ~10⁷⁰ and ~10¹²⁴ mind-seconds instantiable in the reachable universe. (Because reversible minds could stretch our entropy budget long into the black hole era².)
Question is whether the reversing of the computation that makes up the mind and the lack of output (that'd imply bit-erasure) entail that the mind "didn't really exist".
There are four options here:
1. The mind didn't exist because it's just a computation+the computation being undone, no effect was made on the rest of the universe.
2. The mind existed once, during the "forward pass".
3. The mind existed twice, during the "forward pass", and during the reversion, since they're isomorphic computations.
4. There were two different, very similar minds.
Maybe I'll take a longer stab at this at some point, but looks like nobody has thought about this except Tomasik 2015³ mentioning it in passing.
¹: https://en.wikipedia.org/wiki/Reversible_computation
²: https://en.wikipedia.org/wiki/Black_Hole_Era
³: https://longtermrisk.org/flavors-of-computation-are-flavors-of-consciousness/#Entropy_and_sentience
I've been wanting to write exactly this article for a while on the EA forum. If EAs are consistent in their thinking, the well-being of future digital minds should be viewed as a significantly more important issue than preventing the extinction of humans from AGI.
Though most extinctions don't have happy digital minds.
Sorry a bit late, but can I ask why you believe that is true? It does not seem obvious to me.
I don't understand if you're now partially functionalist about mind or what the hell is going on, since you're calling them digital minds that can be designed with hardware and software engineering. At the same time you seem to extremely anti functionalist. I'm very confused! Any clarification you could offer?
I explain more in the other post, but in short, I think it's decently likely that functions *cause* consciousness, even though it's non-physical. Everyone thinks some physical thing causes consciousness--dualists just add that it does so because of psychophysical laws.
Thanks!
It seems to me that can't be quite right. Substance dualists don't typically think that the mind is caused by the material, even though the specific contents in awareness, such as being aware of a car, is indeed caused by material function. I also think many idealists and panpsychists would object to that characterisation?
But more importantly, IMHO, you seem to paint yourself into a corner of epiphenomenalism? Unless you want to deny that digital computation is deterministic. And that is a terrible corner! (again, IMHO)
I'll check out the other post!
They do! Certain physical arrangements (e.g. brains) give rise to minds. Others don't.
Nothing I said implies epiphenomenalism.
Digital computation is entirely deterministic and subject to reductive analysis. I mean, I can't explain the difference between a video game and chatgpt in terms of logic gate arrangement and input data, but you get what I mean. It's beyond any doubt that the difference comes down to those differences.
For a computational functionalist like myself, this poses no problem, because the mind is literally software. If software consistently behaves as if it's conscious, then we grant it consciousness with the intentional stance. If you oppose that, however you DO end up with epiphenomenalism, because every claim the digital mind makes of being conscious can be reduced to simple logic gate operations, and there's absolutely nothing to explain over and above the lower and higher level computational principles.
Maybe you're open to computational functionalism in spite of being opposed to physicalism? A lot of antimaterialists seem to be into consciousness in digital minds, and everything digital is physical as far as I can see, so I'm very perplexed by that
You can as a dualist think functions are what causes a mind to emerge without thinking they're identical to functions. This is Chalmers' view fwiw.
That didn't answer the question. Once.minds have emerged , do they change functioning or not?
Yes, but Chalmers has at least a large chunk of his career admitted that his view is epiphenomenalist? He seems to wiggle back and forth.
Maybe I should put ur in terms of causal closure. You can debate causal closure in humans if you don't like it, but in digital minds you cannot. Digital computation is causally closed. If consciousness arises from it, then it has no way of affecting the computational outputs. And so either your digital minds aren't zombies or their consciousness is epiphenomenal.
Considering I'm an ontic pancomputationalist, this is naturally my position.
What do you think the prevailing balance of joy and suffering in the universe is? Do you think it is possible to make more than an Infinitesimal.amount of difference to it?
I believe in nested simulations that are finite in nature. There being finite time, there is finite suffering and joy. If the Big Rip hypothesis is correct, then this Universe will eventually "Halt." I think it's possible we will create more universe simulations per Bostrom, so it's important to get those future ones correct. Poorly constructed simulations lead to very bad S-risks. As for the balance, I think if one heavily weighs the importance of one moment of joy in a noetic phenomenally conscious being, that exists in an otherwise empty universe as outweighing billions of years of evolutionary suffering, it stacks up well. Bentham would likely disagree.
You should throw your hat in the ring! https://substack.com/@andrsgmezemilsson/note/c-201356356
I think this ends up being the "ad absurdum" case that makes me actually feel stronger in disregarding utilitarianism for non-human minds. If you can just create an infinite pleasure machine, why does that carry any moral valence in the universe? Why does it matter at all? There is no cosmic judge of this valence, and it has no bearing (or if it does, probably a negative impact) on the ability of those I hold dear to live pleasurable, meaningful lives.
Also, I know you have argued this point, but there is still a MASSIVE difference to me between a duty to refrain from causing suffering to conscious beings and a duty to create or facilitate the existence of conscious beings. The positive and negative sides don't just balance out morally, and that I think is where all of the risk in utilitarianism lies.
This doesn't assume utilitarianism. Just assumes we don't ignore a being's interests because it's made of sillicon.
There's a huge gap between "don't entirely ignore" a nd "do have a duty towards".
You had an interview a while back with some other EAs where you and the other EAs noted something to the effect that everyone has to jump off the crazy train eventually. Can I justify why I should save my mother from drowning, if I have to choose between saving her and saving a stranger, why I'm not indifferent/impartial? I can't, but seems crazy not to save her. If I recall correctly (though I don't want to put words in your mouth), you agreed with that statement in that interview. Can I justify why I shouldn't kill my parents for their insurance money if I was confident I could get away with it and that the expected value of the lives I could save with that money vastly outstripped the wrongness of their deaths? I can't, but killing them would be crazy. You don't like it when people dismiss what you say without providing an argument, but everyone has situations where they have a deep moral instinct that something has gone wrong in someone's intellectual assessment of the moral situation and they don't have an obvious intellectual response for why it's wrong. Don't get annoyed by it. You're going to get a lot of it here. This is going to seem like the crazy train to a ton of people, same way not saving your mother might seem to you (if I recall correctly!!).