Perhaps the most accurate media coverage the Machine Intelligence Research Institute (MIRI) has yet received wasa pieceby legendary science authorCarl ZimmerinPlayboy. To give you a sense of how inaccuratemostof our media coverage is, here’s a (translated) quote from some coverage of MIRI ina Franco-German documentary:

In San Francisco however, a society of young voluntary scientists believes in the good in robots. How naive! Here at [MIRI]…

Such a quote is amusing because, of course, the Machine Intelligence Research Institute has been saying for a decade that AI willby defaultbe harmful to humanity, that it isextremely difficultto design a “moral machine,” that neither we nor anyone else knows how to do it yet, and thatdozens of approachesproposed by others are naive and insufficient for one reason or another.

Now, ina new pieceforThe Sunday Times, Bryan Appleyard writes:

Yudkowsky [from MIRI] seemed to me simplistic in his understanding of moral norms. “You would not kill a baby,” he said to me, implying that was one norm that could easily be programmed into a machine.

“Some people do,” I pointed out, but he didn’t see the full significance. SS officers killed babies routinely because of an adjustment in the society from which they sprang in the form of Nazism. Machines would be much more radically adjusted away from human social norms, however we programmed them.

Again: MIRI has been saying for a decade that it is extremely difficult to program a machine to not kill a baby. Indeed, our position is that directly programming moral norms won’t work becauseour values are more complex than we realize. The direct programming of moral norms is something that其他人提出,我们批评。For example, here is a quote from theconcise summaryof our research program:

Since we have no introspective access to the details of human values, the solution to this problem probably involves designing an AI to learn human values by looking at humans, asking questions, scanning human brains, etc., rather than an AI preprogrammed with a fixed set of imperatives that sounded like good ideas at the time.

But this doesn’t mean we can simply let some advanced machine learning algorithms observe human behavior to learn our moral values, because:

The explicit moral values of human civilization have changed over time, and we regard this change as progress, and extrapolate that progress may continue in the future. An AI programmed with the explicit values of 1800 might now be fighting to reestablish slavery… Possible bootstrapping algorithms include “do what we would have told you to do if we knew everything you knew,” “do what we would’ve told you to do if we thought as fast as you did and could consider many more possible lines of moral argument,” and “do what we would tell you to do if we had your ability to reflect on and modify ourselves.” In moral philosophy, this notion of moral progress is known as reflective equilibrium.

Moving on… Appleyard’s point that “Machines would be much more radically adjusted away from human social norms, however we programmed them” is another point MIRI been making from the very beginning. See the warnings against anthropomorphism in “Creating Friendly AI” (2001) and “Artificial Intelligence as a Positive and Negative Factor in Global Risk” (written in 2005, published in 2008). AI mind designs will be far more “alien” to us than the minds of aliens appearing in movies.

Appleyard goes on to say:

[Compared to MIRI,] the Cambridge group has a much more sophisticated grasp of these issues. Price, in particular, is aware that machines will not be subject to social pressure to behave well.

“When you think of the forms intelligence might take,” Price says, “it seems reasonable to think we occupy some tiny corner of that space and there are many ways in which something might be intelligent in ways that are nothing like our minds at all.”

What Appleyard doesn’t seem to realize, here, is that Price is basically quoting a point long stressed by MIRI researcher Eliezer Yudkowsky — that there is a huge space of possible minds, and humans only occupy a tiny corner of that space. In fact, that point is associated with MIRI and Eliezer Yudkowsky more than with anyone else! Here’s another quote from the paper Yudkowsky wrote in 2005:

The term “Artificial Intelligence” refers to a vastly greater space of possibilities than does the term “Homo sapiens.” When we talk about “AIs” we are really talking about minds-in-general, or optimization processes in general. Imagine a map of mind design space. In one corner, a tiny little circle contains all humans; within a larger tiny circle containing all biological life; and all the rest of the huge map is the space of minds in general. The entire map floats in a still vaster space, the space of optimization processes… It is this enormous space of possibilities which outlaws anthropomorphism as legitimate reasoning.

So once again, a reporter thinks MIRI’s positions are the opposite of what they are.

Beware what you read in the popular media!

Update: I told Appleyard of his mistake, and he simply denied that his article has made a mistake on this matter.