Description

In his article, Charles Frye explores the convergence of large language models (LLMs) and operating system kernels. He underscores the potential of systems metaphors in enhancing LLMs, drawing on a range of research that focuses on pretraining techniques, inference-time speed optimizations, and prompting strategies. These innovations include speculative execution, which expedites LLM inference by predicting certain tokens, and registers, which improve the performance of Vision Transformers by storing intermediate information in uninformative pixels. Frye also covers paged memory and touches upon the potential impact of virtual memory systems on language models, allowing them to access much larger storage.