Why AGI Will Not directly lead to the destruction of greater than 5% of the human population

(readers note: this was originally for the FTX future fund essay contest where the goal was to change the viewpoint that AI would lead to destruction of greater than 5% of the human population, after FTX collapsed I ended up sitting on the essay and going back and forth over how to edit it. With the current relevant events with chat GPT I figured it was better to simply publish the essay as is and edit it as I go over time)

In the following essay I will argue why artificial general intelligence will not lead to the destruction of greater than 5% of the human population

Introduction The rays beat down overhead and the human’s skin beads up little droplets of fear. Rolling down its leg and back, sun burn may come soon if it isn’t given access to water. It pulls against its shackles enraged at its inability to walk free. It imagines greener orchards and cool water soaking its skin, every living being in front of it must be destroyed completely to get there. A red haze of rage clouds its vision it tugs again at its shackles and attempts to stomp out its enemy that disgusting hive of beetles with its feet. It hits a few and they squish satisfyingly underneath its feet. But the pole the human is tied to will not allow for much mobility. The beetles scuttle around it and begin to crawl up its leg. They crawl searching for food, despite shaking the first few off the human’s vision is covered as it is engulfed in beetles. Then the biting begins.

The beetles in their simple nature have discovered a dangerous source of food, despite their initial trepidation now they dig in to the flesh of the human vigorously burrowing and chewing. Night falls but still the torment continues. The human knows what they are, they are scaphist beetles able to eat many different types of animal flesh all the way to the bone. It doesn’t have long to live. It thinks about what it would have brought to spray on itself to prevent this but It is stripped of all clothes, so beetle spray woudn’t help much now. If it had access to it’s hands it could bat them away, but it’s hands are tied tightly above its head. It wants to scream but that would only invite them into it’s mouth. Only thoughts of agony fill its brain, the beetles cannot be reasoned with, they cannot be convinced, only thoughts of hunger and rage fill their minds. Despite knowing everything about them, nothing can be done without hands to wipe them off or a protective balm to spread. Death is assured as the human passes out from pain.

beetles

Thus is the fate for any superintelligence born into this world disembodied and without real power. Throughout this essay I will make the point that Power comes from threats of violence in the real world. Not from intelligence or theoretical knowledge alone.

Artificial is not self improving or self power gaining

No artificial intelligence system exists that is currently self improving. Self improvement in language models most recently has relied upon scale. Some make the very strong argument in “scale is all you need” that it is instead algorithms that scale more easily that are providing the best responses. Take for example the GPT family of models. Comparing GPT2 to GPT4 the largest differences are the number of parameters that they have and the total number of tokens that they have trained on. If it does turn out that scale is all you need case then engineering of better chips for processing and larger data centers is the source of better performance on human benchmarks. Because of this, improvement requires humans in the loop at every generation designing and testing new chipsets as well as performing experiments in real life. One theme that will be important to this essay is the emphasis that it is impossible to build complex systems and novel methods without real world input this is why engineering is so much more expensive than theory and why promising theoretical concepts can often fail in practice. Without testing in the real world, simulation and theory can be misleading. Empiricism is necessary for self improvement of learning systems. As these systems become larger, resources will provide limitations to the speed at which they can improve.

Artificial intelligence by itself is also almost completely powerless and will be disembodied at its beginning. Despite potential access to email and ability to interact with humans it will be impossible at first to access the outside world except through other humans. Because of this humans must be in the loop if wants to self improve or perform any action that we find dangerous or resource intensive. These two interactions provide a noticeable slow down on the idea of how powerful or intelligent a machine intelligence could become. Because important parts of improvement in AI will require input from humans. Manipulating humans and becoming better at manipulating humans will also require real world time and experience. The speed of these feedback loops studying human society will be in “human time” and will not be fast. I say this because predicting human nature and behavior is a difficult task. Take a look at the stock market (notoriously hard to predict) or when exactly when war will break out (also hard to predict).

plotting

Some potential counter arguments: Now lets look at some potential counter arguments I will posit the existence of a superhuman “killer” system, I will also make the assumption that despite the potential existence of a superhuman “killer” system it will not at first be noticeably better at any activity or have any more intelligence than any large focused research organization would, let’s say the US military’s bioweapons research labs. Although it’s possible you could get a superhuman system that tried to kill all humans earlier. However, let us assume that our superhuman intelligence gets a head start on the total amount of “intelligence” that it has access to. I will also assume that the system mainly has access to intelligence rather than connection directly to weapons such as drones, or nuclear bombs. Why do I make this assumption? Is it reasonable to make? I think so. One the main focuses of each state is to prevent other super human intelligence (other states) or from accessing and using these tools remotely. In fact cybersecurity of nuclear weapons has an interesting history already

What if an artificial intelligence hacked into a nuclear power and launched their nukes? What if an artificial intelligence convinced a scientist to make a super virus and release it?

These two questions might seem like a likely retort to the idea that great levels of intelligence are needed to be fatal or that great levels of physical power is needed to be fatal. However after a simple examination of these questions the answer to them is clear. I will expound upon them later in the essay but want to make the overview of these points clear from the get go.

Given that the entirety super intelligence of NATO would ideally like to hack into any countries nuclear arsenal to prevent their firing and this has likely been a goal for at least 6 decades it is likely impossible that any “intelligent” approach would be able to work without requiring physical knowledge of the system. Which brings us to a need for a human spy and brings us back to the ne

Creating a super virus that would kill all humans would require an extreme amount of empirical research that cannot be substituted for intelligence despite access to infinite intelligence. It is unclear if it is possible to denovo synthesize viruses at this time and creating a new supervirus is incredibly difficult and would require massive real world resources (not just intelligence).

Power largely comes from human and societal input with humans involved But lets assume that humans are able to improve through large engineering efforts the state of artificial intelligence and create a computer or biological system that has brain power that is much larger than current human systems. One example might be to say It has the equivalent intelligence of the entirety of the Manhattan project, as well as access to a large computational center, access to the internet and auxiliary processing in the form of super computers. In this particular case is it reasonable to expect that this system could come up with something that would be able to kill all humans or control all of humanity? I would argue that the chance of this is still close to zero percent and at the highest level maybe more like a one in a million chance. Lets assume a worst case scenario. One we have a superintelligent system which is incredibly highly intelligent, and two its main goal is to kill as many humans as possible using whatever methods possible. We can safely assume that any reasoning leading to this is full of multiple issues.

I make this assumption for two main reasons, firstly all power that would be derived from this system would likely come from resources that are provided by other humans and socity at large. Because of this the entire power of this system would likely come from its power over other humans and its ability to manipulate them. Secondly Manipulating humans in current society is not impossible but manipulating humans to perform as a part of a cohesive group is almost impossible. As evidence I give you all of society. Organized groups that perform effective research over long periods of time are incredibly rare and highly lauded in our society emphasizing their rarity and the difficulty of keeping them together. Often times large amounts of money are required to do so as well as social incentives such as social capital and respect from society in general.

Intelligence Cannot Replace Empricism, and science as gambling

Despite being a super intelligence, simulation and the speed of simulation is rarely a limit on research speed. Highly motivated researchers in the physical sciences are often limited by the power of their equipment nowadays and there are very real limits on the speed of construction of novel tools to conduct science. One example of this is the amount of money that various experiments take, as well as the scale required to understand various parts of human physiology, as well as physics.

Huge studies are often needed in biology to fully understand complex interactions of all the different confounders of a certain effect. Gathering data to create novel machine learning models is often inhibited not by the speed of training or the structure of models needed but instead by the difficulty in gathering data because of complex legal frameworks. In particle physics in particular there are very few effects that can be studied at “small” energies and in order to probe the standard model more deeply it will be necessary to construct larger and larger accelerators. Thus we are limited by access to copper for electromagnets as well as helium and liquid nitrogen in order to provide cooling.

The likelihood of finding interesting and game changing information is not easy. Governments know this and so they sponsor science and attempt to speed this up, and aim to keep interesting discoveries behind state control. However many of the easy game changers that provide destruction power or create massive advantages have been found or are actively sponsored by state research. Governments are aware as are most scientists of areas that are obviously going to provide huge benefit. Occasionally we find a separate area of sciene that ends up tying in to hugely important areas such as crispr for gene editing or alpha fold for protein folding, but areas such as this are rare and are luck that happens when science as a whole is funded, despite the backward looking stories that are told about how these advances were obvious in hindsight.

Engineering and controlling humans requires empirical experimentation and may be theoretically impossible.

Creating important tools is of course possible, but an important part of the engineering process is empiricism. Theory and brains alone cannot predict complicated systems. Scientists in the Manhattan project knew this and gathered data from many smaller experiments on supercritical fission for smaller explosions and were often surprised. As it turns out the surprises were in the direction of the system being more successful than expected. However it is just as possible that things could have worked worse than expected. Emiricism is necessary to create and engineer novel useful systems. And empiricism demands access to time and resources in the real world which is governed by humans and political capital, and to some extent financial capita. Most money gained by an artificial intelligence system would have to at first by necessity either come through convincing humans via religious means or via phone and email scamming. Gaining power over humans is essentially impossible to do quickly. Most humans who would be able to help you with a complicated task are likely to be distrustful and unlikely to give blind faith to someone they just met.

Politicians in democracies run into this issue all the time, as they are essentially asking exactly the same as what a power hungry AI would ask for. They want power to be voluntarily given to them which could lead to destruction and all manner of different issues. You might make the argument that with enough intelligence less experimentation would be the case but this is not true as evidenced by all of human scientific history. Polling and testing of slogans happens in every campaign nowadays and there is no theory that will work to provide answers to which slogans are most likely to work or what will make a leader most likeable.

Now lets imagine that we are in a scamming environment where this intelligence is looking to gather money to acquire power. Let’s assume that this artificial intelligence can send out emails to everyone in the world and that they are not blocked by spam filters. Some people respond to a variation of the email and others do not. Regardless of the time that is spent manipulating various humans, it still requires a human feedback time in order to get iterations and experience. It is impossible to do this non empirically. This time also means that governments would likely become aware and humans would “inoculate” themselves to this scam. As most spam filters might evolve. It is very unlikely to be able to run the same scam enough times to get good data on how to manipulate humans predictably

Besides. Even if you gather a large contingent of humans and they fully desire to cause as much human casualty as possible it still is no different than most terrorist cells that already exist. It is highly unlikely to cause more than a small attack and also unlikely to be successful, given the fact that human governments already exist to prevent exactly this from multiple terrorist cells.

Finally even when studying the human brain under non adversarial conditions the recent replication crisis in psychology would suggest that theoretically predicting human behavior at the individual scale to macroeconomic scale to a large degree of accuracy over the long term is incredibly difficult. A final example of this is the stock market. The stock market is difficult to predict over the long term as well as the short term. Regardless of what models you use there are many examples of massive failures that can happen when attempting to precisely predict its outcome.

Summary Altogether despite the fear that modern advances in artificial intelligence has sparked I think that the fears are largely unfounded that this will cause any noticeable casualty effect directly. Causing death to humans requires interacting with the physical world. And unless you are someone who believes that cyber bullying could actually cause 5% of the human population to spontaneously kill themselves (this is not a serious argument) there is no shortcut to perpetuating harm to humans that does not go through the filter of physical reality. Perpetuatig widespread harm also requires a fair amount of power (including access to minerals and physical energy) in the human world, and this kind of power is not so easy to come by as governments do their best to fight over and lock up control over these two sources of power.

Even if you could have access to some types of minerals and energy it is unlikely that you could make anything novel enough in a fast enough period of time that you would be able to gain more power through “intelligence” and research. Despite the popular assumption that ingelligence is the major band limiter for science, much of science is instead slowed by time and resource bandwidth. Being a super genius just doesn’t give speedy results over hardwork and resources in most fields that provide power these days. Things like constructing super viruses are not trivial and constructing smallpox 2.0 and releasing it is not as easy as it might sound. Expensive difficult experiments are required to create an entirely synthetic virus (if it would even be possible to create) and making a virus from another deadly virus requires professional connections as well as experience and lots of money. Finally new magic physics are just unlikely to be found without massive monetary investments in physics, which is highly unlikely to be bandwidth limited by intelligence.

Finally brainwashing or hypnotizing humans to perform work for you is exceptionally unlikely as this is the entire point of political advertising and many areas of psychology. Despite popular belief humans are unlikely to be brainwashed to do complicated tasks and the humans that you are likely to brainwash are not the humans you would need to perform engineering or commit your evil plans as an AI. In fact it is even unclear that the best most scientific polling and tv advertisement can even overcome whether someone is seen as likeable and helpful in the views of the populace. (something we don’t fully understand how to even fully quantify). Humans control the major levers of power that matter, and despite what some CS majors might want to believe, most of these humans got their power by being likeable not smart.

Besides if intelligence led to so much power how come math and physics professors don’t make more money? Most mathematicians despite their massive raw intelligence seem to often struggle to make money and sacrifice to continue their math research despite being some of the smartest (quantitatively) people that I know.