Superintelligence: Paths, Dangers, Strategies - Chapter 6

Cognitive superpowers

This Chapter will analyse the powers that a superintelligent system may have and how it could use them. 

Functionalities and superpowers

At the outset, Bostrom provides a key piece of advice readers should have in mind: “It is important not to anthropomorphize superintelligence when thinking about its potential impacts. Anthropomorphic frames encourage unfounded expectations about the growth trajectory of a seed AI and about the psychology, motivations, and capabilities of a mature superintelligence.”

However, a superintelligence system that has a capacity to learn can, hence, increase its intelligence and “all other intellectual abilities are within a system’s indirect reach: the system can develop new cognitive modules and skills as needed—including empathy, political acumen, and any other powers stereotypically wanting in computer-like personalities.”

Additionally, we tend to describe people as geniuses when they have an IQ of 130 when the average IQ is around 90, then imagine a world where an AI has an IQ of 6,455. Current software engineers and science cannot really contemplate what the capabilities of such an AI could actually be. As Bostrom indicates, it may even merely be composed of special-purpose algorithms allowing it to tackle standard intelligence test questions with “superhuman efficiency but not much else.” 

Some of the superpowers that a superintelligent system might possess are intelligence amplification such as AI programming etc, safeguarding and protecting its intelligence as well as planning the best way to attain its future goals. According to Bostrom, we cannot rule out the possibility of manipulation and rhetoric persuasion as well as persuading countries to take a course of action. Not to mention, hacking is also a possibility to obtain financial resources or hijack military robots/equipment, among others. The possibilities are endless. 

An AI takeover scenario

1. Pre-criticality phase. In the beginning, the AI relies on human programmers as to how to conduct its operations. As the intelligence of the AI is amplified with its capacity to learn, it starts doing more of its operations by itself.

2. Recursive self-improvement phase. Once the AI does the majority of the work by itself, an intelligence explosion occurs. “At the end of the recursive self-improvement phase, the system is strongly superintelligent.”

3. Covert preparation phase. “Using its strategizing superpower, the AI develops a robust plan for achieving its long-term goals.” In this phase, there is a prospect that the AI hides its intentions to the programmers in order to avoid being shut down. 

4. Overt implementation phase. At this point, the AI could be so capable that it does not need to hide its intentions. Consequently, it is pursuing its goals comprehensively.

Finally, Bostrom put it best: “Without knowing anything about the detailed means that a superintelligence would adopt, we can conclude that a superintelligence—at least in the absence of intellectual peers and in the absence of effective safety measures arranged by humans in advance— would likely produce an outcome that would involve reconfiguring terrestrial resources into whatever structures maximize the realization of its goals.”