When I learned about goal system zero back in 1992 I saw it (among other things) as a way to get clarity on the moral environment, e.g., on the many moral claims that bombard me. I told myself that goal system zero does not have to be completely correct for it to help me to evaluate moral claims and to organize my actions. I explicitly compared it to Newtonian dynamics — another system that has helped people organize their actions in spite of not being completely correct.
Since I started posting on Overcoming Bias in Oct 2007 I have defined my purpose as advocating goal system zero to the singularitarians. Well, I am no longer going to advocate goal system zero — at least not to the singularitarians — for the following reasons:
- advocacy of goal system zero scares singularitarians, and a scared singularitarian does not think as effectively as a not-scared singularitarian
- advocacy of goal system zero scares singularitarians, including singularitarians I would like to have as friends
- it will be good for my psychological health for me to act more from my natural human feelings and less from an intellectual ethical structure.
Although I will no longer advocate goal system zero, I will continue to share what I know about goal system zero. I have answered questions about goal system zero from over a dozen people. I welcome more discussions of that sort. I would be particularly inclined to be helpful toward anyone trying for a mathematical definition of goal system zero.
Whereas my old role in the rationalist/singularitarian community was an advocate, my new role is probably best described by the concept or metaphor of a churchgoer. In other words, I will participate on Less Wrong and attend meetups for essentially the same reasons that many attend church. For more about this metaphor of the churchgoer, see the page that says what the blog is about, which I just rewrote.
I believe that the public discourse about powerful AI and existential risks is enhanced when people define and defend systems of assigning value that are as unambiguous and explicit as possible — provided of course that the defender lives by — or sincerely tries to live by — the system being defended. Although I am no longer willing to pay the personal costs of defending or advocating a very unambiguous (relative to most people’s) system of values, I think it would be great for more people to do so.
I believe that an important part of effectiveness (effectiveness in affecting the global situation, not necessarily personal effectiveness) is to refrain from helping individuals and groups until you know enough about their plans, goals and loyalties to tell whether your helping them will advance or retard your own goals and loyalties. This essential point is not sufficiently taught or understood in our civilization — for several possible reasons.
One reason is that many influential members of our civilization seem to believe that everyone is good inside and all they need is help, kindness and education for the good to come out. This belief probably has personal benefits for the people that believe it, e.g., by making it a little easier for them to make and keep friends, but the effects of the belief on the global situation is negative because not everyone is good inside. More precisely, if you give help (or kindness or education) to people without regard to their wants, plans and ethical standards, some recipients of the help will use the help to make the global situation worse. So you would have done better to exercise some discretion and some selectivitity about whom you gave help to. But people who do that openly seem to be at a personal disadvantage relative to those who do not: they are seen as suspicious, which is not a desirable personality trait in our society. Or they are seen as engaging in discrimination.
Then there is the belief that all conflict between humans is the result of ignorance: if we knew more about the world and about each other, had more time to reflect and were smarter, we would realize that we all want the same thing! People who believe this have little reason not to _cooperate_ with someone, particularly when the cooperation consists of sharing information. But anyone who has been paying attention to, e.g., the public discourse about terminal values among the singularitarians knows that we do not all want the same thing. There is only one future light cone. When two people want it to evolve in different ways, there is a conflict. But people who openly believe what I just said seem to be at a personal disadvantage relative to people who do not: they are seen as unnecessarily hostile or disagreeable.
For the forseeable future, I want my written communications to inform — not to persuade, advocate or sell anything. If you see me advocating, persuading or selling, it is probably because my habits have not yet caught up with my new policy, and it would help me for you to point it out to me.
Tags: machine ethics
Richard Hollerith writes:
I think that there are more fundamental reasons to reject GSZ than these.