Rhys Southan
rhysso.bsky.social
Rhys Southan
@rhysso.bsky.social
Philosophy PhD student at the University of Oxford
So really, I think your objection is the goal-first objection, not the delay objection
July 8, 2025 at 12:13 PM
So, there's nothing necessarily irrational or mistaken about having a goal that humans settle Mars and also wanting future people to have that goal. The point is just that there is nothing instrumentally irrational about abandoning that goal, because as soon as you do, it's not your goal anymore
July 8, 2025 at 12:13 PM
I realized I responded to this in a confusing way. What you say here is consistent with the timing problem argument in the paper. The timing problem just entails that if you abandon your goal, you have not done something that violates your goal, since it's not your goal anymore...
July 8, 2025 at 12:11 PM
I think because I have a hedonic-phenomenal notion of desires, and we couldn't assume that a superintelligence will have desires in that sense. In the paper as it is, your objection is closest to the Delay Objection
July 4, 2025 at 3:20 PM
At some point I was thinking about having a potential objection be that we are relying on a very strict concurrentist notion of desire satisfactionism (with the objection being that this is controversial, or at least debatable), but I changed my mind about this
July 4, 2025 at 3:18 PM
So, it's only a problem if you and your future self exist at the same time, which by definition you don't
July 4, 2025 at 2:02 PM
You prefer your future self to have value X concurrently with X being your value, and are indifferent about your future self having value X if X is not your value. If you abandon the value X, your future self will not value X, but this will not go against your value of X, since you no longer value X
July 4, 2025 at 1:58 PM
You prefer others to share your value X while you have the value X, but are indifferent about whether they have the value X when X is not your value. Same with your future self....
July 4, 2025 at 1:57 PM
I had an additional thought after our chat. I wonder if you see wide scope as collapsing into narrow scope, whereas we see narrow scope as collapsing into wide scope
July 4, 2025 at 11:46 AM
Thanks!
July 3, 2025 at 8:26 PM