InFeeo
United States
technology-news
New
Language

Channels

A new YC tool promises "your code never leaves your machine." It does(twitter.com)
We’ve detected that JavaScript is disabled in this browser. Please enable JavaScript or switch to a supported browser to continue using x.com. You can see a list of supported browsers in our Help Center. Allgemeine Geschäftsbedingungen Datenschutzrichtlinien Cookie-Richtlinie Impressum Anzeigen-Info © 2026 X Corp.
Adaptive Low-Rank Transformer with Dynamic Expert Routing for Continual Learning(creativecommons.org)
Inspired by the role of sleep in biological continual learning, we introduce RVW, a trans- former architecture for online continual adaptation of pretrained models. RVW maintains a small pool of per-layer experts that grow and prune in response to distribution shift, with no replay buffer and no explicit task identifier. Applied to TinyLlama-1.1B on a 15,000- chunk six-domain stream, RVW reaches 40 average held-out PPL, substantially better than EWC (158), fine-tuning (164), and LoRA (448) on the same parameter-matched base, while preserving prior-domain performance. Threshold sweeps suggest a combinatorial encoding reading: domain knowledge appears to be carried by routing patterns across layers rather than by individual specialized experts.