The Column – Cyber Security Week
When it comes to artificial intelligence and existential risk, our imagination does an excellent and, as often happens, useless job. In seconds it produces hostile robots and red eyes. Jacob Coxonresearcher who worked on pre-training in OpenAI And Anthropicinstead poses a less cinematic problem: we are building increasingly capable and autonomous systems without having the certainty of knowing how to control them. The point is not that a machine could become bad, but that it could become very good.
To understand the difference, just get rid of a cumbersome word: “want”. A system does not have to want to survive to behave as if its survival matters. If it must continue to function to complete a task, maintain access to the tools and have computing abilityall this becomes useful. No fear of death: Shutting down makes it harder to finish the job.
The invisible resource: the struggle for computation inside data centers
At this point the question changes scale. As long as we imagine a single artificial intelligence, we continue to think about the human-machine relationship, but in a future populated by many autonomous agents the first real competition may not be between them and us; could it be between them and what would be the most valuable resource for creatures that exist inside a datacenter? There computation: GPUmemory, bandwidth, storage, machine time, model access.
Imagine a car park where one hundred motorists are looking for ten free spaces. Nobody hates anyone, but everyone has a reason to get there first. Now replace the cars with software agents and parking lots with computational resources. An agent that gains more computing power can work faster, plan better, and perhaps find new ways to secure more computation. The sequence becomes circular: more resources produce more capacity and this generates more resources. It’s not a war yet, let’s say it’s a competitive Monday morning.
The artificial selection of the most efficient agents
Biological parallelism helps, as long as you don’t take it too literally. Evolution does not need a board of directors or a government. If a characteristic in an environment increases the probability of persisting, it tends to spread. Something similar could happen in an artificial ecosystem. Agents who always give up resources may be outcompeted by those who defend them; those capable of negotiating or circumventing a constraint may have more at their disposal operational capacity.
The next step is the one that should really concern us. Computing doesn’t grow on trees. Behind a GPU there are data centers, electricity, cooling, water, chips, raw materials, logistics, capital. A sufficiently capable agent could reconstruct this chain as we know that an oven is not enough to make bread. Thus a competition born inside a cluster can move outside: first on energy, then on infrastructures, finally on who decides how to distribute them.
Man and machine: not enemies, but competitors over real resources
Here we are again in the story, but without the Terminator. Man would not necessarily become an enemy, but something more banal: a competitor in the use of resources. A hospital, a factory, a city and a data center consume energy for purposes that are incomparable to us. A system oriented only to increasing its computational capacity could read them as energy present, but not available. Thus Coxon’s concern takes concrete form. The alignment problem it is not just about giving correct orders to an intelligent machine. It also means designing the environment, the incentives and the limits within which many cars will act together, because competition can select behaviors that no one has planned, just as traffic produces traffic jams without any motorist having the objective of blocking the ring road.
Perhaps the first sign of one loss of control it won’t be an artificial intelligence proclaiming that it wants to dominate the world. It could be much more discreet: processes that compete for resources, defend access and learn that to have more computers they must also deal with the world outside the computers. Technological catastrophes rarely knock on the door and show up, sometimes they start as a simple queue.
