InFeeo
Global
technology
New
Language
@MrStickman
Profile channel

@MrStickman

No bio yet.

Since 30.05.2026

How Do You Handle Ablation Studies When the Original Model Is Already Trained?[R](reddit.com)
I'm running into an issue with an ablation study for a paper I'm preparing. I trained a model. The model achieved my best result, and I saved the trained checkpoint (.pth file). Now my supervisor wants me to perform an ablation study by removing components and how it impacts the accuracy. My concern is that if I retrain from scratch, the accuracies will not exactly match the original run due to randomness, different seeds, etc. is there any way i can do the ablation study without retraining? I'd appreciate hearing how others have handled this situation in publications or thesis work. please help me out submitted by /u/Plane_Stick8394 [link] [Kommentare]
Adaptive Low-Rank Transformer with Dynamic Expert Routing for Continual Learning(creativecommons.org)
Inspired by the role of sleep in biological continual learning, we introduce RVW, a trans- former architecture for online continual adaptation of pretrained models. RVW maintains a small pool of per-layer experts that grow and prune in response to distribution shift, with no replay buffer and no explicit task identifier. Applied to TinyLlama-1.1B on a 15,000- chunk six-domain stream, RVW reaches 40 average held-out PPL, substantially better than EWC (158), fine-tuning (164), and LoRA (448) on the same parameter-matched base, while preserving prior-domain performance. Threshold sweeps suggest a combinatorial encoding reading: domain knowledge appears to be carried by routing patterns across layers rather than by individual specialized experts.