Artificial Intelligence as Strange Intelligence: Against Linear Models of Intelligence

This paper critiques the linear model of AI progress, introducing "familiar intelligence" and "strange intelligence". It argues that AI intelligence is likely to be strange, combining superhuman capacities with subhuman performance in various domains, and develops a nonlinear model where "general intelligence" is the ability to achieve a broad range of goals in diverse environments, not reducible to a single linear quantity. It discusses implications for adversarial testing of AI capacities, suggesting that errors in seemingly obvious tasks do not necessarily negate outstanding general intelligence.