Comments on: Drawing the Target around Where the Arrow Landed http://rhollerith.com/blog/31 A blog about rationality, improving the world and the far future Tue, 07 Jul 2009 15:55:17 +0000 http://wordpress.org/?v=2.9.2 hourly 1 By: Roko http://rhollerith.com/blog/31/comment-page-1#comment-83 Roko Wed, 27 Aug 2008 20:34:38 +0000 http://rhollerith.com/blog/?p=31#comment-83 <i>"Azathoth, the blind idiot god shot an arrow, and where the arrow landed determined human nature and the thousand shards of desire. Eliezer seems to claim that the arrow miraculously landed on correct terminal values or on a seed that can be extrapolated by a superintelligence to yield correct terminal values. He seems to support that claim by an appeal to the reader’s natural human desires for, e.g., health, laughter and fun. I agree with Caledonian’s comment that that is drawing the target around where the arrow landed."</i> - yes, indeed. But, in support of that particular place where the arrow landed, it is us. Suppose we can take some set of abstract principles and find that the correct location is somewhere quite significantly different to where the arrow did land; say we find that what's really important is (roughly speaking) creativity, ingenuity, intelligence, variety, co-operation, and expansion to find more space, free energy and matter. What is a human being to do in response to this? Surely we want to move in that direction, but how quickly? I am not personally sure how quickly i would want to move away from the particulars of human existence; fun, family, laughter, sexuality, status, ... all those other things that I enjoy but almost certainly aren't justified on purely general grounds. I suspect that I would find a gradual movement from here to there desirable, as long as at least some of the people I cared about came with me, so to speak. I definitely don't want to remain fully where Azathoth's arrow landed. “Azathoth, the blind idiot god shot an arrow, and where the arrow landed determined human nature and the thousand shards of desire.

Eliezer seems to claim that the arrow miraculously landed on correct terminal values or on a seed that can be extrapolated by a superintelligence to yield correct terminal values. He seems to support that claim by an appeal to the reader’s natural human desires for, e.g., health, laughter and fun.

I agree with Caledonian’s comment that that is drawing the target around where the arrow landed.”

– yes, indeed.

But, in support of that particular place where the arrow landed, it is us.

Suppose we can take some set of abstract principles and find that the correct location is somewhere quite significantly different to where the arrow did land; say we find that what’s really important is (roughly speaking) creativity, ingenuity, intelligence, variety, co-operation, and expansion to find more space, free energy and matter.

What is a human being to do in response to this? Surely we want to move in that direction, but how quickly? I am not personally sure how quickly i would want to move away from the particulars of human existence; fun, family, laughter, sexuality, status, … all those other things that I enjoy but almost certainly aren’t justified on purely general grounds.

I suspect that I would find a gradual movement from here to there desirable, as long as at least some of the people I cared about came with me, so to speak. I definitely don’t want to remain fully where Azathoth’s arrow landed.

]]>