Nick Bostrom frequently highlights the importance of the goal setting process when building a superintelligence. There could be many unintended consequences that lie in wait for us. It looks increasingly likely that AIs will be given an end goal and then given freedom over how they achieve it.
Recent work by researchers at the University of California, Berkeley revolves around setting such a goal and devising algorithms that successfully attain it. “The only thing we say is ‘This is the goal, and the way to achieve the goal is to try to minimize effort,’” the team say.
The process has thus far been tested on relatively simple blocky graphics, such as humanoid shapes. In each iteration thus far, complex behaviors have emerged after a period of learning
This has allowed the machines to undertake relatively complex, repetitive tasks, such as running and swimming. The machines are powered by a neural network that’s trained to control it via information it receives from its environment and past performance.
The concept builds upon the perceived way that humans learn, with children slowly understanding their body and its capabilities as they develop. The approach to learning is undoubtedly interesting, but there remains some concern about leaving AI free to achieve its goal in anyway possible.
For instance, if it’s tasked with going from A to B, what would happen should something get in its way? Whilst humans impart a degree of morality into our decision making, such thinking is not present in AI at the moment.
After all, with the rapid evolution of artificial intelligence, it could quickly do incredible things in the attempt to meet its goal, which if poorly worded could reap significant harm.
Bostrom suggests that this can be averted by giving the AI an overarching goal of friendliness, although even that is not without difficulty.
“How exactly friendliness should be understood and how it should be implemented, and how the amity should be apportioned between different people and nonhuman creatures is a matter that merits further consideration,” he says.