Edge0 is a streaming MoE inference engine that serves large mixture-of-experts models from SSD on consumer hardware by predicting each layer’s expert routing one token ahead with a trained prerouter, so weight reads can be staged before they’re needed. An unmerged recovery LoRA recovers quality lost to int4 quantization and routing replacement. On a single 24GB machine it serves a 35B-parameter MoE at 20 tokens/second within 3GiB of active memory, within a few points of its fp16 teacher, with the framework, checkpoints, and adapters released open source.
