Infinite Utility

Lexical Priority and Infinite Ethics (0.00 / 0)

The problem with lexical priority rules is that people underestimate the difficulty of doing 'everything we can' to satisfy the first member of the sequence, e.g. Rawls gives a principle of equal liberty lexical priority over his maximin distributive principle. However, one can invest arbitrary amounts into preventing violations of liberty (cops everywhere to protect the integrity of the person against violence, watchers to watch those who watch the watchers, etc) so the distributive principle should drop out entirely.
Likewise, investing in existential risk-reduction is always an option, if only to acquire more information.

Now, consider a civilization that has discovered exotic physics that enable it to circumvent the 1st and 2nd laws of thermodynamics and produce new energy and negentropy. For any finite size and population, all the civilization's efforts should go into reducing existential risk by producing more redundant copies, monitoring the universe, and thinking about risk-related subjects. But then, by induction, our hypothetical super-civilization never produces any utility beyond that involved in an optimal risk-reduction effort (e.g. if colonization probes are more efficient when they are capable of eudaimonia)! To realize the fruits of its labors, the civilization must set some formula to dictate the reallocation of its efforts towards utility production, but for any particular formula, it will always be possible to choose another one with infinitely greater EV.

---

by: Carl Shulman @ Fri May 04, 2007 at 03:34:53 AM CDT

re Lexical (0.00 / 0)

"investing in existential risk-reduction is always an option".  No, it's often an option, perhaps almost always an option, but not always.  We here have our work cut out for us, but, to take an extreme example, someone stuck in jail with no access to the outside world can't really do anything.  And in practice, to reduce existential risk, we'll need to do a lot of quite ordinary things, like stay healthy and make friends who can join our efforts.  If you disagree, I'm listening, but to me fanaticism is often not uncomfortable.  Either way, fanaticism follows from utilitarianism (and many other ethics systems) regardless of any concern for the infinite.

"never produces any utility beyond that involved in an optimal risk-reduction effort".  Yeah, so?  That little amount of utility adds up to an infinite amount if it persists for an infinite amount of time.  True, all this does depend on us experiencing utility while minimizing existential risk.  What's the point in surviving forever if we never enjoy a minute of it?  Alternatively, what is existential risk if not the risk that utility will cease to be experienced? 

This, of course, no practical problem for us, since we will experience utility while reducing existential risk.

Are you proposing trading off an infinite amount of utility for a finite amount of utility?

---

by: Seth Baum @ Fri May 04, 2007 at 15:14:08 PM CDT

[ Parent ]

Selecting an arbitrary quantity of utility or probability of survival (0.00 / 0)

"someone stuck in jail with no access to the outside world can't really do anything."
They can plan possible escapes, preparing themselves in case some outside force (even an unknown force) breaks them out, try to figure out useful ideas, etc.

Incidentally, I advocate allowing oneself a certain proportion of time and resources that is not subject to maximizing calculations (or at least not stringently). Trying to maximize all the time is almost always too psychologically stressful to be a sustained pattern for humans.

"And in practice, to reduce existential risk, we'll need to do a lot of quite ordinary things, like stay healthy and make friends who can join our efforts."
The incidental utility from an optimal risk-reduction strategy for humans is a result of particular psychological features. The most efficient risk-reducing strategy on your formulation might involve hyper-efficient AI entities that do not experience eudaimonia as they go about their activities. http://www.nickbostr... If this turns out to be the case, then one would have to make an arbitrary choice as to when and how much one would reduce the efficiency of the risk-reduction effort to ensure some utility is produced along the way.

Briefly:

1) If you can choose any finite quantity of utility whatsoever, how would you choose the amount? No matter what amount you actually choose, you will be forsaking alternatives with tremendously greater utility.
2) If you could choose any probability of producing infinite utility between 1 and 0, what finite number of 9s would you add after the decimal? No matter what amount you actually choose, you will be forsaking infinite utility.

---

by: Carl Shulman @ Fri May 04, 2007 at 18:55:56 PM CDT

[ Parent ]

infinitesial probability (0.00 / 0)

I think the question that's being raised here is how to trade off a finite amount of expected utility with an infinite amount of utility that has an infinitesimal chance of happening.  The latter situation has an expected utility of infinity divided by infinity.  While the hyperreals seem to be able to handle infinity divided by infinity, I don't think we can estimate these infinities in any useful way.

Quick math: Let A = some infinite number of QALYs.  Then, using hyperreals, A + A = 2A, so if we have a 1/A chance of 2A QALYs occurring, then the expected utility is 2A QALYs / A = 2 QALYs.

The problem with this is in estimating A and 1/A.  In the original use of hyperreals, estimation meant estimating existential risk.  That's fine.  But how is one to estimate how infinitesimal a risk is, compared to how infinite the reward is?  That doesn't seem possible.  If it's not possible, it may be fair to say in this situation, we just pick some finite (non-infinitesimal) level of existential risk arbitrarily.  This is an unsatisfying recommendation, but I can't think of anything better off the top of my head.  Of course, we might never need to resolve this matter, as error and uncertainty may mean there's a limit to how close to optimal we can ever get.  This might not hold for a civilization that's now also got Heisenberg uncertainty beat too, but that's maybe about it.  Either way, I don't think there are any practical recommendations for anyone any time soon coming out of this line of discussion.

...And I too am not intently focused on maximizing utility at every moment, or even every waking moment, but I also think that over the long term, this strategy helps me increase utility more.  We are alas, only human, for now at least.

---

by: Seth Baum @ Sat May 05, 2007 at 10:12:04 AM CDT

[ Parent ]

re Lexical 2 (0.00 / 0)

Here we go again. :)

> Likewise, investing in existential risk-reduction is always an option, if only to acquire more information.

OK, how about this:

Option A: 50% chance of infinite utility, 50% chance of  6*10^18 QALYs.

Option B: 50% chance of infinite utility, 50% chance of 6*10^11 QALYs.

I would say Option A is better than Option B, and I think the causal approach/recommendation 2 lets us make this claim.

Incidentally, the 6*10^18 QALYs figure comes from Total Human Earth Utility on Old Felicifia and 6*10^11 QALYs comes from the same except with humanity going extinct in 100 years.

---

by: Seth Baum @ Sat May 19, 2007 at 17:24:41 PM CDT

[ Parent ]

Bostrom seems to agree (0.00 / 0)

It appears Bostrom already beat me to the punch on this one, go figure.  In Section III of Astronomical Waste, he writes

Therefore, if our actions have even the slightest effect on the probability of eventual colonization, this will outweigh their effect on when colonization takes place. For standard utilitarians, priority number one, two, three and four should consequently be to reduce existential risk. The utilitarian imperative "Maximize expected aggregate utility!" can be simplified to the maxim "Minimize existential risk!".

Astronomical Waste is a more practical paper than the nitpicky, theoretical Infinite Ethics.  It also explicitly ignores matters of the infinite (see footnote 7).  It's nice to see that hyperreals can be used to bring the two papers together towards the same conclusion.

---

by: Seth Baum @ Sun May 06, 2007 at 16:59:52 PM CDT

My own idea (0.00 / 0)

I suggest redefining utility to mean the probability density function the the current definition. This way, positive infinite utility becomes 1, negative infinite utility becomes 0, and average becomes 0.5. Assuming utility to follow natural distribution (except for the infinites) the to versions of utility (with a constant added to one) are proportional to each other as the new definition approaches any given value. Unless you become the emperor of the world, you won't, in all probability, make a big enough difference for the definition to matter, and even then it's debatable.

In short, the new definition is in practice identical to the old, except in cases of ridiculously low probabilities of ridiculously high amounts of utility. It doesn't seem right to just change utilitarianism to how we want it to be, like this, but then again, why is happiness good and sadness bad and not the other way around?

By the way, it is possible to deal with infinitesimal probabilities of infinite utility so long as they both follow a limit by finding the limit of the expected utility as the probability approaches zero. For example, if we call utility u, probability p, expected utility e, and make k an arbitrary real constant, then if u=k/p and e=pu then lim(p->0+)u=+infinity (infinite utility and infinitesimal probability) but lim(p->0+)e=lim(p->0+)pu=lim(p->0+)pk/p=lim(p->0+)k=k. (expected utility is k, which is real) Get it?

---

by: DanielLC @ Sun May 13, 2007 at 22:38:22 PM CDT

[ Parent ]

re idea (0.00 / 0)

1) I suggest redefining utility to mean...  Are you referring to instantaneous utility or total utility?  (cf Total Lifetime Utility for how I use these terms.)

2) In using the PDF, do you mean to use uncertainty in our estimates of utility, or are you simply mapping (-inf.) -> 0, 0 -> 0.5, (+inf.) -> 1?

why is happiness good and sadness bad and not the other way around?  You may like meta-ethics.  Wikipedia's page is a reasonable starting point.

---

by: Seth Baum @ Mon May 14, 2007 at 09:23:37 AM CDT

[ Parent ]

Different Idea (0.00 / 0)

I think I messed up exlaning my idea. But I have a better one anyway. I would suggest dividing by the standard deviation of utility.

Suppose there are two posibilities of the universe, 0 and 1, with equal probability of being the universe, and there are two choices, A and B, as follows:
  Universe 1:
  Utility follows normal distribution and is finite. If you make choice A, utility comes out as three standard deviations lower in utility than expected. If you make choice B, it comes out three standard deviations higher.
  Universe 2:
  Utility can be infinate; specifically, it has a 10% chance of being +infinity and a 10% chance of being -infinity. If you make coice A it would be +infinity, and if you make choice B it would be -infinity. Utility would have a standard deviation of infinity/sqrt(5).
Using my system, 1A would be -3, 1B would be +3, 2A would be +sqrt(5), and 2B would be -sqrt(5). This way choice A would be expected (sqrt(5)-3)/2, and B would be expected (3-sqrt(5))/2. It would then be best to choose B.

I think I meant quite the opposite of PDF. I meant that, for example, if there was a 90% chance of x exceeding utility, x would be 0.9. This might work out better, as I think there might be flaws in the first idea in the comment.

---

by: DanielLC @ Thu May 17, 2007 at 17:45:02 PM CDT

[ Parent ]

re Different Idea (0.00 / 0)

Let me see if I follow this scenario.

First, some quick nomenclature.  I'll use u1cA for universe 1, choice A, etc.  I'll also define X as the finite amount of utility experienced in u1cB.  Finally, E[A] is the expected value of A.

So, the total amounts of utility experienced are:

u1cA: -X
u1cB: +X
u2cA: +inf
u2cB: -inf

Under the conventional framework:

E[A] = 0.5*(-X) + 0.5*(+inf) = +inf
E[B] = 0.5*(+X) + 0.5*(-inf) = -inf

Under your new framework:

E[A] = 0.5*(-3) + 0.5*(+sqrt(5)) = -0.3820
E[B] = 0.5*(+3) + 0.5*(-sqrt(5)) = +0.3820

So this new framework, in the example you gave, recommends choosing an expected negative infinite utility over an expected positive utility.  I don't like it.  In general, I doubt we'll find any satisfying techniques that map infinites to finites, as was done here.

---

by: Seth Baum @ Sat May 19, 2007 at 11:55:25 AM CDT

[ Parent ]

re Different Idea (0.00 / 0)

I seem to have not mentioned an important point. Any possible universe can have its utility added to or multiplied to get an equally likely universe. The added to part is not a problem, as it will make no difference in the final choice of action. The multiplied part, however, is. Here is an example:
...
{
u1cA: +X
u1cB: -X
u2cA: -X
u2cB: +X
}
{
u3cA: +2X
u3cB: -2X
u4cA: -2X
u4cB: +2X
}
...
Grouped like that, E[A] and E[B] look equal, but it could be changed to
...
u1cA: +X
u1cB: -X
}
{
u2cA: -X
u2cB: +X
u3cA: +2X
u3cB: -2X
}
{
u4cA: -2X
u4cB: +2X
...
in which case, it looks like E[A] is twice as good as E[B]. This sort of thing could be used to make any choice look any amount better than any other choice. Using my method, doing what I just showed would be impossible.

Another thought: if I didn't do something like this, couldn't it be argued that I shouldn't try to go for any given infinite utility, such as aleph null QALYs (the amount of happiness if you had positive happiness forever, among other possibilities), because its nothing compared to aleph one?

---

by: DanielLC @ Sat Dec 01, 2007 at 16:10:40 PM CST

[ Parent ]

Re: alephs (0.00 / 0)

if I didn't do something like this, couldn't it be argued that I shouldn't try to go for any given infinite utility, such as aleph null QALYs (the amount of happiness if you had positive happiness forever, among other possibilities), because its nothing compared to aleph one?

It's true that aleph numbers are problematic in this regard because they result in an "arms race" to bigger and bigger infinities. One solution is just to declare a "highest" infinity and optimize probabilities of attaining it--e.g., maximize P(+highest infinity), minimize P(-highest infinity), or do some mix of both.

---

by: Alan Dawrst @ Sat Dec 01, 2007 at 18:17:32 PM CST

[ Parent ]

big oops (0.00 / 0)

A rather large and embarrassing mistake in the original analysis here.  Consider three possible outcomes:

w0: 1,1,0,0,0,...
w1: 1,1,1,1,1,...
w2: 1,1,2,2,2,...

Now consider two possible choices:

c1: 100% chance of w1 happening
c2: 50% chance of w0 happening, 50% chance of w2 happening

Using the same hyperreal approach as above, c1 and c2 come out equivalent.  However, c1 has no existential risk, whereas c2 has 50% existential risk.  Thus my initial conclusion

1. Reduce existential risk as much as you can.  Make any finite sacrifice for any existential risk reduction.

must be revised.  Something along the lines of

1. Maximize long-term average expected utility.  Make any finite sacrifice for any long-term average expected utility increase.

should be used instead.  However, this conclusion handles growth scenarios poorly.  For example,

w3: 1,2,4,8,16,...
w4: 1,3,9,27,81,...

Neither w3 nor w4 have (finite) average utility levels.  Consider:

c3: 100% chance of w3 happening
c4: p chance of w0 happening, (1-p) chance of w4 happening

Here, hyperreals can be used to show that c4 beats c3 for all (1-p) > 1/inf.  For example, if (1-p) = 0.001:

(Matlab.)  Thus, in this case, we'd actually want to choose the option with higher existential risk for any non-zero, non-infinitesimal chance of avoiding the risk.

This observation (minus the uncertainty and existential risk components) follows way back to the "overtaking criterion" in 1960's economics studies of optimal growth- see
Gale, David, 1967.  "On Optimal Development in a Multi-Sector Economy".  Review of Economic Studies, vol. 34, no. 1. (Jan.), pages 1-18.
von Weizsäcker, Carl Christian, 1965.  "Existence of Optimal Programs of Accumulation for an Infinite Time Horizon".  Review of Economic Studies, vol. 32, no. 2. (Apr.), pages 85-104.
Cowen used it recently in his UCLR paper (pdf) (see p.11-12(15-16)).  Indeed, Cowen's

We should make political choices so as to maximize the rate of sustainable economic growth.

works fairly well.  Of course, in a fit of silliness (political correctness?) he revises it to

We should push for sustainable economic growth, but not at the expense of inviolable human rights.

Whatever.  Anyways, let me propose using:

1. Maximize long-term average expected rate of utility growth.

2. If you can't increase long-term average expected rate of utility growth, then maximize long-term average expected utility.

3. If you can't increase long-term average expected utility, increase utility as much as you can.

In practice, our capacity to maintain such growth in instantaneous utility is questionable, especially using the consumption-based assumptions about utility found in the economics papers.  But the more important practical question is the extent to which existential risk reduction efforts are bad for growth or bad for long-term average utility.  For example, space colonization would be a big existential risk reducer and also lead to opportunities to score extraterrestrial utility; friendly AI might be our best protection against unfriendly AI, but it presumably would also lead to even further utility growth; many of the strategies recommended for climate change mitigation would also serve to increase Earth's long-term carrying capacity.  What trade-offs exist between utility and existential risks requires further attention, but the basic decision framework outlined here should work well.  On the other hand, I've been wrong about this before...

For the record, this is not the first time I've made such a glaring error in my basic mathematics.  The first was in Handling Uncertainty back on Old Felicifia, where Gaverick (eventually) persuaded me to recommend maximizing expected utility instead of tolerating some risk aversion with respect to utility.  Hey, this is blogging, not writing textbooks!

---

by: Seth Baum @ Thu Jun 28, 2007 at 19:35:14 PM CDT

Big O (0.00 / 0)

Quick addition: Computer science's Big O notation framework for evaluating algorithm performance should work well for evaluating options under possibly infinite time horizons.

---

by: Seth Baum @ Fri Jun 29, 2007 at 14:51:45 PM CDT

[ Parent ]

Cardinal, not ordinal (0.00 / 0)

Were not dealing with ordinal numbers here. 1+1+1+1+1+...=1+1+2+2+2+...=?₀. This could be modified to 1+1+1+1+1+...=?₀ vs 1+1+?₁+?₁+?₁+...=?₁, but it says finite sacrifice, and going from ?₁ to ?₀ is not finite.

---

by: DanielLC @ Sat Dec 01, 2007 at 16:28:31 PM CST

[ Parent ]

Infinite Utility Spheres (0.00 / 0)

Infinite Spheres of Utility on Philosophy, et cetera.  H/T SIAI.

Now compare the following two variations:

1) Everyone starts off in the blissful sphere. But each day, one more person gets permanently transferred across to the agony sphere, where they reside for the rest of eternity.

2) Everyone starts off in the agony sphere. But each day, one more person gets permanently transferred across to the blissful sphere, where they reside for the rest of eternity.

Which scenario is better?

It's a nice thought experiment, but I'm inclined to file it in the 'not relevant' category.  That is, unless anyone thinks otherwise?  (Note: I haven't read the comments on the two blog posts.)

---

by: Seth Baum @ Fri Nov 09, 2007 at 20:17:54 PM CST

Thoughts (0.00 / 0)

That pdf does not, alas, display for me.

But here are my thoughts on the topic anyway...

Infinities are tricky things.  In quantum electrodynamics infinities can be cancelled to give finite answers.  In mathematics the Banach-Tarski paradox claims that a solid sphere (such as an orange) could be cut into a finite number of solid pieces which could then be fitted together in such a way as to create two oranges (if you had a perfect fractal knife and a lot of patience).

In Utilitarianism, while it might look initially like one needs to think about the knock on effect of an action for the next 20 billion years or so, we can narrow that down.

Firstly, Drake's equation has a factor, which is the expected lifetime of an intelligent species after it first sends a radio message.  In absence of other information, the best guess at this is the same duration up to that point.  Human kind has been around at most for 1.5 million years.

Secondly, once humankind leaves this planet, very little that happened on it will matter much one way or the other.  Time distances things.  How many pieces of music composed before 1000 ad do you listen to?  How much of the biology and medicine from 2000 years ago do you use?  How many of the religions of 3000 years ago are still now believed or even remembered?

Beyond a certain time horizon it becomes impossible to tell whether an action that, in the short term has clearly good or bad effects, will seem the same in retrospect.  Vlad III, ruler of Wallachia in the 1400s, was famed for impaling his opponents on stakes.  A century later, Elizabeth Báthory, a countess in Hungary, bathed in human blood.  Neither of these things seemed good at the time.  500 years on, and millions of people have fun pretending to be Vampires or reading fiction about them.

Under some interpretations of quantum mechanics, there are an infinite number of other universes.  New universes spawn from this one every time a quantum state collapses.  Does that matter to Utilitarianism?  I'd say no.  An ethical decision is not a quantum state.  It is not doomed to go either way 50 % of the time, any more than if you make a bet that you can get 990 heads on tossing 1000 coins you get to win the bet 50 % of the time.

---

by: Douglas.Reay @ Mon Jan 14, 2008 at 20:11:22 PM CST

Many worlds (0.00 / 0)

Under some interpretations of quantum mechanics, there are an infinite number of other universes.  New universes spawn from this one every time a quantum state collapses.  Does that matter to Utilitarianism?  I'd say no.  An ethical decision is not a quantum state.

This is a good question. Even if moral decisions aren't quantum events (is there reason to think they are or aren't?), MWI as a physical theory may have implications for what moral actions we decide to take (though it's not immediately clear what those might be).

---

by: Alan Dawrst @ Tue Jan 15, 2008 at 21:04:04 PM CST

[ Parent ]