Parameter Efficiency, Not Scale, Is the True Measure of Intelligence

Abstract The field of artificial intelligence has spent the last several years in a parameter arms race. The tacit assumption—that raw parameter count is a reliable proxy for intelligence—has shaped funding, publication incentives, and public perception. This paper argues that the assumption is both scientifically misleading and practically costly. Intelligence per parameter (IPP), the ratio of capability and generalization to model size (particularly active parameters), offers a more rigorous measure of progress. We develop a framework centered on Pareto efficiency in the performance-parameter space, formalize IPP through explicit mathematical definitions, and show how it can be operationalized with existing benchmark suites. Recent empirical developments through mid-2026, including the Chinchilla scaling laws, the Llama 3 family, DeepSeek’s sparse architectures, the rise of strong sub-10-billion-parameter models, and the July 2026 release of Moonshot’s Kimi K3 (2.8 trillion total parameters with only 104 billion active), demonstrate that the efficiency frontier has shifted sharply leftward. Smaller or sparsely activated systems now match or exceed models with far higher nominal parameter counts on many tasks. We examine the architectural changes driving this shift—Multi-Latent Attention, Mixture-of-Experts routing, extreme quantization, reasoning distillation, and newer mechanisms such as Kimi Delta Attention—and ground the argument in scaling theory and algorithmic information theory. The paper ends with a concrete proposal: treat parameter count as a cost, not a credential, and redesign leaderboards to reward efficiency as aggressively as absolute performance.

کلیدواژه ها:

parameter efficiency ، intelligence per parameter ، scaling laws ، model compression ، ، compute-optimal training ، Kimi K3