Nurturing Superintelligence

Why will AI eventually be smarter than humans? It all comes down to assumptions. Given enough compute and data, weaker assumptions (or inductive biases) always beat stronger ones asymptotically (even if it takes… a lot of compute).

So why couldn’t us humans get away with weaker assumptions? Because we had to survive in the wild with nothing to protect us—we built civilization from scratch, and without strong assumptions built into our brains we would be eaten by lions.

This is where AI is different. We can nurture AI inside the safety of a civilization we’ve already built, and shield it from the survival pressures that pushed the human brain towards shortcuts (in the form of strong assumptions). AI doesn’t have to worry about death from lions like we did, so we can train it for far longer than any human gets to learn, and with much weaker assumptions.

Increased intelligence comes down to making weaker assumptions, and the real blockers to brains making weaker assumptions were premature death and limited compute. In the future, AI will be blocked by neither of these.

That’s why superintelligence is possible—not because AI is inherently special, but because we can finally afford to let it learn without being hunted by lions.