Comments on: Competitive Analysis of Goal System Zero http://rhollerith.com/blog/22 A blog about rationality, improving the world and the far future Tue, 07 Jul 2009 15:55:17 +0000 http://wordpress.org/?v=2.9.2 hourly 1 By: Richard Hollerith http://rhollerith.com/blog/22/comment-page-1#comment-97 Richard Hollerith Sat, 06 Sep 2008 18:26:45 +0000 http://rhollerith.com/blog/?p=22#comment-97 Good point, Nick. I retract my argument for the position that GSZ does not suffer from the unbounded-utility problem. Good point, Nick. I retract my argument for the position that GSZ does not suffer from the unbounded-utility problem.

]]> By: Nick Tarleton http://rhollerith.com/blog/22/comment-page-1#comment-96 Nick Tarleton Fri, 05 Sep 2008 20:38:35 +0000 http://rhollerith.com/blog/?p=22#comment-96 ISTM that de Blanc's result, or something close to it, should apply to that situation as well; for an action A, there should be an infinite sequence of hypotheses about the world for which the magnitude of the immediate utility of A increases faster than probability decreases, so E[U(A)] diverges. Also, I assume you wouldn't want to greatly increase creativity now in exchange for a certainty of overwhelming loss in the future, in which case don't you have to refer to outcomes? ISTM that de Blanc’s result, or something close to it, should apply to that situation as well; for an action A, there should be an infinite sequence of hypotheses about the world for which the magnitude of the immediate utility of A increases faster than probability decreases, so E[U(A)] diverges. Also, I assume you wouldn’t want to greatly increase creativity now in exchange for a certainty of overwhelming loss in the future, in which case don’t you have to refer to outcomes?

]]> By: Richard Hollerith http://rhollerith.com/blog/22/comment-page-1#comment-92 Richard Hollerith Fri, 05 Sep 2008 16:37:19 +0000 http://rhollerith.com/blog/?p=22#comment-92 <blockquote>is the idea that a time-indifferent gold-maximizer would not produce gold (unless it was certain it was in a finite universe), but seek to refine its model of reality indefinitely (and thus cooperate with GSZ)?</blockquote> Nick, you got it. <blockquote>doesn’t this mean you would consider the creation of a time-indifferent agent-indifferent gold/happiness/… maximizer just as good as the creation of a GSZ agent?</blockquote> I consider it almost as good. The article gives one reason why it might be suboptimal, and I will repeat the heart of the reason now: "The [happiness maximizer] and [gold maximizer] might fight, and of course a fight has the potential to expend resources that could have gone into maximizing creativity. It is in the interest of GSZ to try to prevent the fight." Is there something about my blog that makes it harder to read than, e.g., Eliezer's blog? If so, I will get a blog of the type (Typepad) Eliezer got. My benefactor Garcia believed that an optimal strategy for maximizing the creativity of the universe was simply at every moment of decision to increase creativity as much as possible. I have always assumed that is true because I cannot imagine some other property of the universe Q such that at some moment of decision, maximizing Q has greater expected utility (under GSZ's definition of utility) than maximizing creativity. If that is indeed true, then obviously the implementor of GSZ will not need the utility of any outcome to rise above all bounds (because it need not refer to eventual outcomes at all; it need refer only to immediate outcomes).

is the idea that a time-indifferent gold-maximizer would not produce gold (unless it was certain it was in a finite universe), but seek to refine its model of reality indefinitely (and thus cooperate with GSZ)?

Nick, you got it.

doesn’t this mean you would consider the creation of a time-indifferent agent-indifferent gold/happiness/… maximizer just as good as the creation of a GSZ agent?

I consider it almost as good. The article gives one reason why it might be suboptimal, and I will repeat the heart of the reason now: “The [happiness maximizer] and [gold maximizer] might fight, and of course a fight has the potential to expend resources that could have gone into maximizing creativity. It is in the interest of GSZ to try to prevent the fight.”

Is there something about my blog that makes it harder to read than, e.g., Eliezer’s blog? If so, I will get a blog of the type (Typepad) Eliezer got.

My benefactor Garcia believed that an optimal strategy for maximizing the creativity of the universe was simply at every moment of decision to increase creativity as much as possible. I have always assumed that is true because I cannot imagine some other property of the universe Q such that at some moment of decision, maximizing Q has greater expected utility (under GSZ’s definition of utility) than maximizing creativity. If that is indeed true, then obviously the implementor of GSZ will not need the utility of any outcome to rise above all bounds (because it need not refer to eventual outcomes at all; it need refer only to immediate outcomes).

]]> By: Nick Tarleton http://rhollerith.com/blog/22/comment-page-1#comment-91 Nick Tarleton Fri, 05 Sep 2008 15:05:53 +0000 http://rhollerith.com/blog/?p=22#comment-91 So a GSZ agent maximizes the ability-to-get-things-done of the universe. Wouldn't it then oppose a gold-maximizer, since a universe full of gold atoms is not maximally able-to-get-things-done? Or is the idea that a time-indifferent gold-maximizer would not produce gold (unless it was certain it was in a finite universe), but seek to refine its model of reality indefinitely (and thus cooperate with GSZ)? If so, doesn't this mean you would consider the creation of a time-indifferent agent-indifferent gold/happiness/... maximizer just as good as the creation of a GSZ agent? Also, what do you think of Peter de Blanc's <a href='http://arxiv.org/abs/0712.4318v1' rel="nofollow">analysis of unbounded utility functions</a>? So a GSZ agent maximizes the ability-to-get-things-done of the universe. Wouldn’t it then oppose a gold-maximizer, since a universe full of gold atoms is not maximally able-to-get-things-done? Or is the idea that a time-indifferent gold-maximizer would not produce gold (unless it was certain it was in a finite universe), but seek to refine its model of reality indefinitely (and thus cooperate with GSZ)? If so, doesn’t this mean you would consider the creation of a time-indifferent agent-indifferent gold/happiness/… maximizer just as good as the creation of a GSZ agent?

Also, what do you think of Peter de Blanc’s analysis of unbounded utility functions?

]]>