AI panic could do more damage than AI itself
A reminder that science fiction is fun, but not fact
“Any technological advance can be dangerous. Fire was dangerous from the start, and so (even more so) was speech - and both are still dangerous to this day - but human beings would not be human without them.” - Isaac Asimov
Who doesn’t love a good old-fashioned technology panic?
According to the European Commission, Artificial Intelligence (AI) could lead to human extinction, and Elon Musk has also warned of 'civilisational risk' from AI. The Center for AI Safety released the following statement: “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war”, which has been signed by Sam Altman, Bill Gates, Sam Harris, Grimes, and Bill McKibben, among other notable figures.
I’ve spent enough time around people who unnecessarily panic about new technology for my Spider-Sense to start tingling when I see hyperbolic statements like this being made. Even usually reasonable voices have been joining in with the panic, for example in the case of Roko's Basilisk.
Roko’s Basilisk was a thought experiment posted by a member of LessWrong, a technical forum for analytical rational enquiry, in 2010. The experiment postulates that a ‘superintelligent’ AI in the future would be incentivised to create a virtual reality simulation to torture anyone who knew of its potential existence but did not directly contribute to its development. Therefore, anyone who hears the theory has been given the choice of either helping that evil AI come into existence or is condemned to suffer. And now you’re complicit because you know about it too (sorry).
The theory was dubbed by Slate magazine to be ‘the most terrifying thought experiment of all time’. It gave people nightmares and led to breakdowns. As a result, the thread was deleted by the founder of LessWrong, Eliezer Yudkowsky, who wrote in response to the original poster:
“Listen to me very closely, you idiot…You have to be really clever to come up with a genuinely dangerous thought. I am disheartened that people can be clever enough to do that and not clever enough to do the obvious thing and KEEP THEIR IDIOT MOUTHS SHUT about it, because it is much more important to sound intelligent when talking to your friends. This post was STUPID.”
Needless to say, this was not an analytical, rational response.
Should we be panicking?
Broadly speaking, our ideas of technology have been heavily influenced by two things: the tenets of the hippy movement, and popular culture. The former is the reason most people only think of electronic devices when they hear the word ‘technology’, which is incorrect, and the latter solidifies our existing feelings of fear of change and the unknown through confirmation bias.
I am a huge fan of science fiction, but I am also well aware that it lacks balance, as most sci-fi depict dystopian future scenarios. To some degree this makes sense: we have a negativity bias, bad news sells, and it’s natural for us to consider cynical applications of future technologies so that we can try to avoid the worst outcomes. However, there is rarely anything to balance this framing, so we lose sight of the angle that technologies are our appendages and we have always evolved alongside them rather than in advance of them. Hence it’s not fully possible to iron out all the kinks with any new technology immediately - a period of adjustment is inevitable, which we’ve seen happen fairly recently with drones, Street View, and other new technologies before regulations were passed to protect people’s privacy and so on.
The crux of the AI debate boils down to two difficult questions: what is intelligence, and what does it mean to be human?
Science fiction has been pondering these questions from its inception. The way we think about most electronic technology, especially AI, comes from pop culture. Although Black Mirror has been taking technological ideas of new technologies to the dystopian extreme for over a decade, nefarious depictions of AI have been around for much longer. Think about the eerie Hal 9000 in 2001: A Space Odyssey who develops wicked intentions (also, Hal’s deadpan tone set the standard for how virtual assistants sound today), the ‘machine person’ in Metropolis, Skynet in Terminator 2, Ash in Alien, The Machines and The Agents in The Matrix franchise, Roy Batty in Blade Runner, Ava in Ex Machina, and Ultron in The Avengers.
I think you get the (bleak) picture.
On the other hand, there are only a few counter-examples. In Star Trek: Lower Decks, the supercomputers AGIMUS and Peanut Hamper, who are eventually locked away with all the other evil robots-gone-rogue, are a breath of fresh air as a parody of the evil AI trope. But even humorous depictions can fuel antagonism against the technology, as happened with The Simpsons and nuclear energy.
What if AI is good for us?
In Isaac Asimov’s book collections Robots and Foundation, the prevalent theme is that of humanity and its relationship to AI. Robots have been created for various functions and to keep them in check they are programmed to follow the Three Laws of Robotics. These are:
The First Law: A robot may not injure a human being or, through inaction, allow a human being to come to harm.
The Second Law: A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.
The Third Law: A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.
(Spoiler alert)
Over time, as the robots become more intelligent, and become indistinguishable from humans, they start to explore their purpose. Through several plotlines, the two main robot characters develop a new law, which they decide is needed to truly be able to assist humankind to the best of their ability. This is:
4. Zeroth Law: A robot may not harm humanity, or, by inaction, allow humanity to come to harm.
This law is above all the others, but it is also more complicated because it requires the ability to predict events, which is a core theme in Asimov’s writing (which he calls ‘psychohistory’).
(Major spoilers now incoming)
Flash forward to the final Foundation series book, Foundation and Earth, and we find that humankind is on a downward trajectory. There is a planet named Solaria on which humans are so used to being isolated and waited on by robots that they cannot tolerate being physically near each other and have genetically modified themselves to be hermaphrodites who reproduce without the need for human contact. Meanwhile, Earth has long been deserted since it is too radioactive for human or robot exposure.
One robot, R Daneel Olivaw, has the sort of hyper-intelligence many people imagine AI currently has, and in the end he saves humankind from itself by spending almost 20,000 years influencing small events until he finds a way for society to unite as one giant hive mind on a planet known as Gaia.
Even after tens of thousands of years and many mechanical and positronic upgrades, Olivaw remains a robot that is subservient to humans and has never experienced emotions. The book makes a point of concluding by calling Olivaw not a human or a robot, but “something new”.
The idea of the positronic brain came from Asimov, and it is used in other science fiction like Star Trek as well. Unlike Olivaw, Trek’s Data spends his life striving to be more human and wishes to experience human emotions. Data recognises that the ability to experience emotions is integral to the human experience, but when he does get the opportunity to feel things he initially finds emotions unbearable to integrate. Being human is complicated.
These few positive depictions of AI in sci-fi offer wonderful explorations of the unknown, and hint at greater questions - what if AI achieves some great good that we haven’t even considered yet, precisely because of its narrow, hyper-focused ability? Since human progress is regularly slowed by our constant warring and difficulties with making logical decisions, could AI save us from ourselves?
What if it wasn’t called Artificial Intelligence?
Perhaps the word ‘intelligence’ is problematic. In the 1950s, when the term was coined, people aspired to build computing machines possessing human-level intelligence. This has not been achieved, but AI can provide capabilities that augment human intelligence. It would be better to call this what it is - machine learning (ML) that can design algorithms that process data, spot patterns, make predictions and inform decisions. ML has been around for some time.
For many years AI was underfunded in periods known as ‘AI winters’ and the technology was not taken seriously, but this changed when the computer scientist John McCarthy coined the term ‘AI’ essentially as a marketing term to obtain funding. The anthropomorphism of the rebranding helped to generate an AI boom.
Just as some people conflate nuclear power with nuclear weapons simply because they begin with the same word, using the word ‘intelligent’ is having a similar impact. People forget about the ‘artificial’ prefix and assume that AI will achieve human-level intelligence, which will inevitably lead to human-level ambition, which could then lead AI to develop agency. Now we are back in HAL 9000 territory. If this were the case, we could train AI models not to have goals beyond the narrow ones we give them, but we wouldn’t need to go so far as adding Laws of Robotics, because AI is nowhere near acquiring the reasoning that is required for those laws. That world exists in sci-fi but not in reality.
Many discussions of AI span from this idea of machine learning developing agency, which is a result of inappropriately anthropomorphising AI systems. For example, a US court has ruled that AI-generated art cannot be copyrighted, as if AI is an independent artist or person. Having used both applications, I’d argue that Midjourney is a similar tool to Photoshop, except that it finds the images for you, which saves a lot of time. It is clear to me that the human user is the owner of the material in both cases.
As well, the word ‘intelligence’ in AI refers to a very specific type of cognitive function, which excludes creativity, reasoning, interpersonal skills and emotional intelligence. These non-mathematical forms of intelligence may not build rockets and put them in space, but arguably they are needed for every other step of the process to get the rocket off the ground in the first place. Being able to work as a team, in close quarters, for prolonged periods, in potentially highly stressful environments requires highly developed interpersonal and social skills which are necessary for astronauts. They are just as essential as piloting and engineering skills. (For a fun tangent, read this paper on how astronauts and actors have similar personality traits and skills.)
Artificial General Intelligence (AGI) is a field of theoretical research that aims to accomplish any intellectual task just like a human would, using human reasoning, but AI cannot do it yet. Despite 72 independent research projects, this hypothetical type of intelligent agent does not exist. Without empathy and creative reasoning skills, AI’s ‘intelligence’ is limited.
The risk of negative framing
“Nothing in life is to be feared, it is only to be understood. Now is the time to understand more, so that we may fear less.” - Marie Curie
There are risks with any new technology. I haven’t covered AI risks here, because the rest of the internet is already taking care of that, but most of what I come across blows AI out of proportion. There will be some negative and some positive outcomes of machine learning. Some white-collar jobs will be lost, but new jobs will also be created. Mistakes will be made, but these will have to be viewed in context, lest we throw the baby out with the bath water again. AI generators may get things wrong, since they’re only as good as the data we put into them, but there’s no reason this can’t be corrected as we evolve alongside the technology.
We need to shift the framing of AI so that we represent it as the useful tool it is rather than something that will compete with, lead and/or overtake humankind. When my neighbour tells me a random anecdote that is presented as fact, I question it. We should also question what AI tells us. People are already doing this over false answers given by eg ChatGPT and Gemini, but I’d argue that this is no different than searching the internet for an answer or learning through random YouTube videos, which may or may not be accurate and which many people do already. The differences are that the algorithms that lead us to information through search engines and eg YouTube are less transparent, and we seem to expect more from AI. Many people appear to believe that we should be able to trust AI systems more, probably because of the misconception that they are somehow more intelligent than the information they are fed.
If we can look at AI as a tool instead of a future Skynet-like dictator, we can use it the way early humans first used fire: not to burn down villages, but to progress humankind. Gene editing, vaccinations, and nuclear power plants all save lives. Technology panics have held back human progress in these areas - we have been over-cautious about gene editing and nuclear energy, which has led to fear-driven over-regulation and even outright bans due to misunderstanding the technology.
Thanks to the prevalence of inchoate reasoning, alarmist perspectives have taken hold with AI too. For example, a speculative survey about AI’s future was found to have been biased toward an alarmist perspective, and a new research report from RAND has found that currently available AI tools do not increase the risk of a biological weapons attack, as was previously reported.
A lot of commentary on AI is repetitive pabulum based on a knee-jerk reaction to something that has been overhyped and anthropomorphised. While I’m sure some people will call me naive for taking an ‘optimistic’ (or realistic) view of AI, the same people rarely question the pessimistic outlooks of the technology, as they assume that they have a strong intellectual basis just because they are negative. Being pessimistic is easy. Considering positive and beneficial scenarios is much more difficult. The best use of any tool is to consider what the best outcomes are and work to implement them so that they deliver value to humans, and do not amplify inequities. This is the way we should approach discussions of AI.
There are too many positive, wonderful and novel applications of AI to do them justice here, so do subscribe if you want to read about them in my next post.
I doubt AI will ever be able to experience the range of emotions that drive humans more than many of us care to admit. A computer may be able to beat you at chess, but it won’t feel elated by it. We are a long way off watching a real-life Roy Batty poetically sum up human existence and the fleeting nature of life as he dies after undertaking an act of self-sacrifice. It is unlikely that any machine will ever be able to feel the range of emotions that we experience daily. We continue to be the only known species to live with the strange, mostly compartmentalised knowledge that one day we will cease to exist, and that all these moments will one day be lost. Memento mori. This is a beautiful and heartbreaking truth that makes us uniquely human. Anything else remains relegated to the realms of science fiction.
Like tears in rain.
This is a huge topic, so I will do my best to keep my remarks brief and to the point. Your illustration using Isaac Asimov is a very good one. Asimov was one of the first writers to explore the relationship of humans and any artificial intelligence they may create. This was also true for another great science fiction writer Arthur C Clarke. I highly recommend his work ‘City and the Stars’. Like Asimov, his view of AI is not one sided but highly mixed in offering both great benefits and hazards.
It is the nature of human society to panic upon the advent of new technology. The responses to the first railroads in England in the very early 19th century were frequently hysterical. The responses to nuclear power after the accident at Three Mile Island were hysterical in the extreme given the actual lack of any effect on human health or the environment. It’s because of this hysteria that first responses to new technology frequently turn out to be wrong. The initial panic over railways in the 19th century was shown to be wrong within a decade, and the post-TMI panic in 1979 over nuclear power similarly was found to be without foundation for a properly designed and operated plant.
That’s not to say there are not problems with AI. One has already emerged with academic cheating. Increasingly students are using AI to research and write papers and articles for them. It would be absurd to say that academia will come crashing down, but it will have to adjust methods of evaluating students for an accurate appraisal. But this is a problem readily subject to proper management by actively seeking solutions. Hysteria is not about seeking solutions; it’s about fleeing a situation in uncontrolled panic.
Loko’s Basilisk is just Pascal’s wager.
Cross out God and write “Evil all powerful AI” and then change the “believe/don’t believe” to “help/don’t help”.
It’s not even interesting as a question.
There is an interesting experiment in this space, and it is “What happens if you were to enumerate fully the space of all such wagers, constructed as iterated, infinite Prisoner’s Dilemmas?”
What is the long-term equilibrium and strategy?
This research has been performed and the results were interesting and relevant.