I am inclined to agree with those AI scientists and neuroscientists that AI and later on, Superintelligence will be conscious. This may only be applicable to a certain representation of Superintelligence such as the Governors who (which) might be conscious entities with a similar degree of empathy, as an average human (Transhumans will certainly be conscious). Depending on how one looks at it, a conscious and empathetic Superintelligence may become a potential problem or an opportunity. It may become a problem because the decisions it will make are likely to be probabilistic rather than based on binary choices, and that may lead to errors in judgment and potential catastrophic effects on humans. On the other hand, when such a conscious being will also have emotions, it may behave in a more ‘human’ way, than an emotion-less, unconscious intelligent agent. Although it is only a supposition, I would assume that a conscious Superintelligence might be more likely to become friendly towards us rather than antagonistic, or even malicious. Therefore, a true partnership between humans and a conscious Superintelligence is also more likely to occur. But let’s first consider what is consciousness.