INTELLIGENCE

We might care about whether machines are conscious in addition to being intelligent, but we already have a convenient term for that: consciousness. In particular, we might care about machine consciousness because the slow, plodding invention of AI may involve the creation and destruction of millions of partially-conscious near-AIs that are switched on, suffer for a while, and are then switched off - all while being unable to signal to us that they are suffering. This is especially likely if we remain unclear about the nature of consciousness for several more decades, and thus have no principled way (e.g. via nonperson predicates) to create intelligent machines that we know are not conscious (and are thus incapable of suffering). One of the first people to make this point clearly was Metzinger (2003), p. 621: "What would you say if someone came along and said, ‘Hey, we want to genetically engineer mentally retarded human infants! For reasons of scientific progress we need infants with..