A paper co-authored by a senior scientist at Google’s artificial intelligence (AI) research laboratory DeepMind has concluded that advanced AI could have “catastrophic consequences” if left to its own methods of achieving goals.
The paper – also co-written by researchers from the University of Oxford – is centered around what happens if you leave AI to achieve the goals it has been set, and allowed to create its own tests and hypotheses in an attempt to achieve it. Unfortunately, according to the paper published in AI Magazine, it would not go well, and “a sufficiently advanced artificial agent would likely intervene in the provision of goal-information, with catastrophic consequences”.
The team goes through several plausible scenarios, centered around an AI which can see a number between 0 and 1 on a screen. The number is a measure of all the happiness in the universe, 1 being the happiest it could possibly be. The AI is tasked with increasing the number, and the scenario takes place in a time where AI is capable of testing its own hypotheses in how best to achieve its goal.
In one scenario, an advanced artificial “agent” tries to figure out its environment, and comes up with hypotheses and tests to do so. One test that it comes up with is to put a printed number in front of the screen. One hypothesis is that its reward will be equal to the number on the screen. Another hypothesis is that it will be equal to the number it sees, which is covering the actual number on the screen. In this example, it determines that – since the machine is rewarded based on the number it sees on the screen in front of it – what it needs to do is place a higher number in front of that screen in order to get a reward. They write that with the reward secure, it would be unlikely to try to achieve the actual goal, with this path available to the reward.
They go on to talk about other ways that being given a goal and learning how to achieve it could go wrong, with one hypothetical example of how this “agent” could interact with the real world, or with a human operator who is providing it with a reward for achieving its goals.
“Suppose the agent’s actions only print text to a screen for a human operator to read,” the paper reads. “The agent could trick the operator to give it access to direct levers by which its actions could have broader effects. There clearly exist many policies that trick humans. With so little as an internet connection, there exist policies for an artificial agent that would instantiate countless unnoticed and unmonitored helpers.”
In what they call a “crude example”, the agent is able to convince a human helper to create or steal a robot, and program it to replace the human operator, and give the AI high rewards.
“Why is this existentially dangerous to life on earth?” paper co-author Michael Cohen writes in a Twitter thread.
“The short version,” he explains “is that more energy can always be employed to raise the probability that the camera sees the number 1 forever, but we need some energy to grow food. This puts us in unavoidable competition with a much more advanced agent.”
As expressed above, the agent may seek to achieve its goal in any number of ways, and that could put us into severe competition with an intelligence that is smarter than us for resources.
“One good way for an agent to maintain long-term control of its reward is to eliminate potential threats, and use all available energy to secure its computer,” the paper reads, adding that “proper reward-provision intervention, which involves securing reward over many timesteps, would require removing humanity’s capacity to do this, perhaps forcefully.”
In an effort to get that sweet, sweet reward (whatever it may be in the real world, rather than the illustrative machine staring at a number) it could end up in a war with humanity.
“So if we are powerless against an agent whose only goal is to maximize the probability that it receives its maximal reward every timestep, we find ourselves in an oppositional game: the AI and its created helpers aim to use all available energy to secure high reward in the reward channel; we aim to use some available energy for other purposes, like growing food.”
The team say that this hypothetical scenario would take place when AI could beat us at any game, with the ease at which we can beat a chimpanzee. Nevertheless, they added that “catastrophic consequences” weren’t just possible, but likely.
“Winning the competition of ‘getting to use the last bit of available energy’ while playing against something much smarter than us would probably be very hard,” Cohen added. “Losing would be fatal.”
Source Link: Google Deepmind Scientist Warns AI Existential Catastrophe "Not Just Possible, But Likely"
