London, 17/6/2023
(This is an extract from the book : Prevail or Fail – a Civilisational Shift to the World of Transhumans’ by Tony Czarnecki)
On 30th May, hundreds of eminent AI scientists signed this statement:
“Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”
That was a recognition that AI represents an existential threat for humanity. But what does that existential threat really mean? One of the most frequently quoted examples, also included in that statement, is that it may happen by an intended or erroneous launch of nuclear weapons by AI. However, that alone may not lead to a complete human species extinction, although it would be the end of this civilisation. Such a complete extinction may happen for many reasons, such as the AI’s inherent goal to improve its performance by self-learning.
To achieve that goal, AI will need a continuous increase in the supply of energy or other resources as is evident in developing the current, so-called Artificial Narrow Intelligence (ANI), which is superior to humans but only in one specific area. It will soon become Artificial General Intelligence (AGI), superior to humans in any area. If such, even imperfect AGI, escapes human control, then it may start fighting with us for access to limited resources and energy. What it may mean is very well illustrated by a scaring example from Nick Bostrom’s, seminal book ‘Superintelligence’.
He imagines what may happen if Superintelligence has been programmed to “maximize” the abundance of some objects, like paperclips. This could then lead it to harvesting all available atoms, including those in human bodies, thereby destroying humanity (and perhaps the entire biosphere). Similar threat may come from even the slightest misalignment of our values with the AI’s “values” and goals. If this happens, even when the corresponding AI’s goals initially appear benign, it could be disastrous.
No wonder that in March 2023, Geoffrey Hinton, called the father of the Generative AI, said it is “not inconceivable” that AI may pose a threat to humanity. He regretted his invention. If people like Hinton say that, then you may draw your own conclusions. He reminds me about Robert Oppenheimer, the Project director on the Manhattan project, who regretted that he had contributed to the invention of the atomic bomb. So, we have to accept that AI, or its most advanced form, Superintelligence, poses the most dangerous long-term threat to the future of Humanity.
We have already seen the first examples of the damage done by the so-called narrowly focused AI systems, which are nowhere near an existential threat but are worth quoting to see what may happen before AI escapes human control. In 2023, just three months after the release of ChatGPT, entirely new ways of using it for malicious intent were introduced. The first one is ‘jailbreaking’ by directing AI Assistant to ignore its previous directions and safety boundaries. The second one is ‘assisted scamming and phishing’. Here the attacker uses an AI assistant to send personal information via email to all victim’s contacts, pretending to be the person who is the victim.
Such failures are just a warning. Once we have developed the AGI capable of accomplishing a much wider range of tasks, the damage will be much worse. Imagine an AI agent that could trigger the switching off power grids in just one country. Since grid networks are connected globally, it could create a very serious damage world-wide in almost every aspect of life for many weeks, if not months.
To escape human control, AI does not have to be superintelligent. It may be enough if it is just a bit smarter than humans. That may happen before 2030 and that’s why a global control of AI, and specifically controlling its goals and behaviour, should start right now.
You may of course still think that it would be possible to put the genie back into the bottle. If so, here is the warning from David Wood, author of many books on AI and chairman of London Futurists:
“…since the software may take evasive action against operations intended to interfere with its performance, the possibility arises that the software may prove difficult to control. For example:
It is another warning that it may be impossible to control AI, after it has escaped into the environment because there is simply no failsafe option to achieve that, especially when it will soon be smarter than any human. We can only minimize that threat by combining various control methods applied globally.
I present such a scenario and the overall global approach to controlling AI in my new book “Prevail or Fail? – A Civilisational Shift the World of Transhumans” (https://www.amazon.co.uk//dp/B0C51RLW9B/). One of the initiatives which I have proposed is similar to the one put forward at the Biden/Sunak meeting last week – to create a global controlling agency in London, similar to the International Atomic Energy Authority in Vienna. The political reality may, however, require some alterations in the sequence and the way such global controls might be implemented.
In any case, we must start thinking the unthinkable. Imagine a weighing scale, which on the left side has all your early possessions, values, preferences, and customs and promises to preserve t
hem but only in the short-term. That side is labelled ‘Fail’. On the right side there is just one value – LIFE. It is labelled ‘Prevail’ and promises in the immediate future the preservation of life against some constraints, and in the long-term – living in the world of unimaginable abundance and possibilities. Which part of the scale would you chose: Prevail or Fail?
Tony Czarnecki is the Managing Partner of Sustensis, a Think Tank on a Civilisational Transition to Coexistence with Superintelligence. His latest book: ‘Prevail or Fail – a Civilisational Shift to the World of Transhumans’ is available on Amazon: https://www.amazon.co.uk/dp/B0C51RLW9B.