The sleep framework: A new way for LLMs to retain knowledge in continual learning research.google

Google researchers present a theoretical account of the Sleep framework, which uses memory consolidation and “dreaming” to help large language models learn new information without erasing existing capabilities. They argue that compact knowledge seeding, parameter expansion, varied expert activation and gradient-based filtering make learning updates more informative and stable.