Nurturing Superintelligence

The reason that AI will eventually be smarter than human intelligence has everything to do with assumptions. Given enough compute and data, weaker assumptions/inductive biases, always outperform stronger assumptions asymptotically. The reason that humans couldn’t get away with weaker assumptions is because we had to survive in the wild as we had nothing else to protect us. Without strong enough assumptions we died because we had to build our civilizations from scratch.

The biggest reason that AI will be smarter than humans is because we will be able to nurture AI with the safety of a prebuilt (safe) civilization. We will be able to protect AI from the survival pressures that forced the human brain towards shortcuts (in the form of strong assumptions). Because of this, AI won’t have to worry about death like us humans did in the past, and we’ll therefore be able to run (train) AI for longer than humans have been able to run for. The biggest blocker for getting away with weaker assumptions is premature death and compute (data scale depends on not dying prematurely)! AI won’t have to deal with this.

That’s why superintelligence is possible — not because AI is inherently special, but because we can finally afford to let it learn without the shortcuts survival forced on us humans.