Introduces Neural Spectral Capacity (NSC), a closed-form scalar derived from the singular-value spectrum of a network’s weight matrices that scores an architecture’s capacity from its specification alone, with no training or data needed. Its additive layer-wise structure enables NSC-DP, an exact dynamic-programming solver that finds the capacity-maximizing architecture under a resource budget in seconds on a CPU. NSC outperforms parameter/FLOP counts and existing training-free proxies at ranking architectures across seven Transformer and CNN families, discovers a Transformer-XL variant beating a human-designed baseline, and prunes LLaMA-7B to a 5.7B model roughly 5900x faster than the strongest training-free baseline.
