The team describes OreoLook, an open-source AI answer-engine (browser-agent-driven, provider-routed LLM inference) whose local caching stack runs entirely on commodity 8-vCPU CPU hardware. They introduce a three-layer cache — a Redis-backed rolling session-context window with Huffman-compressed disk overflow, a semantic query cache that catches rephrased questions via embedding cosine similarity, and a URL-embedding cache that dedupes embedding computation across sessions — plus a background LRU eviction daemon. In production the system reports an 89.3% aggregate cache hit rate at 0.1ms read latency with only 1.38MB of memory overhead.
