I have recently started working in mechanistic interpretability independently, starting with distill circuits thread My work is on disentangling and closely studying a single neuron, a 1x1 convolution in inceptionv1 model (and applying the method to other neurons in the same layer). The key insight was that the hadamard product of the receptive field and the weight of a neuron is what the neuron is 'seeing' or detecting. We can cluster the hadamard product to get all the patterns a neuron detects. It gave clean monosemantic clusters (cars, cats, dogs which it was known to activate on). We also get more clusters however, letters, human faces, and many more low valued activations. This gave me a new technique to analyse the neuron very closely. On close analysis the most peculiar thing I found was that the low valued clusters (like letters) had all its dependent neurons also firing on the same concept (letter), and the positive and negative weights were evenly distributed between them to bring down the sum. An evidence of gradient descent working deliberately to put patterns and concepts in a noisy range. I've tried to keep it very distill like with good visualisations. I hope you give it a read. https://pages.narang99.in/posts/2026-07-12-disentangling-mixed4e-55/ I made a mistake honestly by starting with convolutions, nobody seems to care about it. I'll start working on language soon, but it would be good if anyone can read this, it would be good to have some feedback on whether I've actually found anything useful. Thank you :) submitted by /u/narang_27 [link] [Kommentare]
I’m training a single-class segmentation model for large rectangular artwork placed on the floor and photographed from above. We have around 3,000 accurately masked original images taken by six different photographers. They are not the same height and do not hold the camera in exactly the same way, so the photos naturally vary in: roll pitch yaw camera distance object coverage in the frame centering and X/Y shift orientation perspective lighting The photos taken with flagship iPhone. I want to use on-the-fly augmentation to simulate realistic human-hand variation and save our designer from adjusting each time to make it flat. is 100 augmentation combinations per original be useful, or excessive? Should the policy be: mostly isolated transforms, mostly crossover combinations such as orientation + roll + pitch + yaw + coverage + shift, or a controlled hybrid of both? The goal is maximum segmentation accuracy, especially around the object boundary, not speed. I plan to train for around 300 epochs and keep validation and test images unaugmented. submitted by /u/Loganbirdy [link] [Kommentare]
COLM 2026 Decision about to come soon so lets talk here. submitted by /u/North_Menu718 [link] [Kommentare]
I've been running a lot of comparative evals across recent model releases—both API and open-weight—and there's a pattern I can't unsee. After a certain number of turns, or when you push into niche territory, the outputs start converging. Same cadence. Same hedging phrases. Same blind spots. It's not full collapse. It's a kind of... homogenization. A creep. My working theory: we're deep enough into the synthetic data flywheel now that we're seeing the first-generation effects. Not model collapse in the catastrophic sense, but a gradual loss of "texture" across models that share overlapping synthetic ancestry. I've been calling this EchoCreep in my notes. The slow, creeping homogenization of model behavior driven by shared synthetic data lineage. Has anyone else been tracking this? Is there a formal term yet? If not, what are you seeing in your evals that fits this pattern? I'm especially interested in: Concrete eval metrics that might capture it Whether fine-tuning on entirely human-curated data clears it If you've seen it worsen between checkpoint versions any feedback would be appreciated? Thanks submitted by /u/BCondor3 [link] [Kommentare]
Paper Referred:- https://arxiv.org/pdf/2412.09871v1 Has anyone switched the transformer in the entropy model here to a Mamba model ? What could be the possible changes ? Just a ML fresher asking a genuine, since Mamba is more popular and saves computer (O(n)). Thanking you in advance ! submitted by /u/SoloLeveller07 [link] [Kommentare]
BMVC reviews will be out tomorrow. Making this parent thread for discussion. All the best everyone! submitted by /u/Hot_Version_6403 [link] [Kommentare]
Hi everyone, The check in QR code is visible at my profile despite that my card isn’t accepting the payment transaction. What does that even mean? Thanks! submitted by /u/misplacedlion [link] [Kommentare]
Hey guys, I was playing around with Nanotron recently and got super frustrated by how many heavy, hardware-specific dependencies it imports at the module level ( flash-attn , triton, functorch , etc.). If you try to run it on older or budget GPUs like a T4 or V100, it just crashes on import. So I wrote Picotron (https://github.com/Syntropy-AI-Labs/picotron) to solve this. It's a clean-room rewrite that gets rid of all mandatory GPU-specific dependencies. It runs on pretty much any GPU that supports PyTorch (defaults to FP16 on older cards under compute capability 8.0, and BF16 on newer ones). It falls back to standard PyTorch SDPA by default, but still hooks into FlashAttention-2 at runtime if it detects you have it installed. I used an AI assistant to write a lot of the boilerplate/code modules, but I've got it working locally and just trained a tiny 2M model on FineWeb-Edu. Also added configs for: • GQA / MLA (Multi-head Latent Attention) • QK-Norm & logit soft-capping (Gemma 2 style) • Parallel FFN/Attn runs • ZeRO-1 wrapping on DDP Roadmap is pretty short right now: MoE prep (routing capacity factors and load balancing loss) Making dataset prep easier than streaming manually Check it out if you've been fighting with CUDA dependency hell: https://github.com/Syntropy-AI-Labs/picotron submitted by /u/Capital_Savings_9942 [link] [Kommentare]
I'm a B.Tech student in India graduating in mid-2029. Currently, I know Python, SQL, Docker, basic prompt engineering, and I've built a few LLM apps using BaaS like Supabase/Firebase. I’m running all this on an Intel i5 13th Gen laptop with an RTX 5050 (8GB VRAM). The Pivot: I originally wanted to be a generic ML Engineer, but I've realized the entry-level market for Jupyter Notebook/Python scripters is insanely saturated. Plus, most companies are just calling OpenAI APIs anyway. I want to pivot to the high-performance stuff: AI Backend / Distributed Systems Engineering. Instead of tuning the model, I want to be the guy building the infrastructure that serves the model to 10,000 users without the servers melting. My ultimate goal is top-tier Indian product companies (Zepto, Cred, etc.) or US remote startups. My Proposed Roadmap: I am dropping the "easy" BaaS tools and going low-level. Here is my plan for the next couple of years: The Language Shift: Dropping Python for backend, going all-in on Go (Golang) to master concurrency and memory pointers. The Database Shift: Moving away from Supabase to raw PostgreSQL (via Docker), eventually learning to scale it, plus Redis for caching. The Hardware/AI Constraint: Running models locally on my RTX 5050 using Ollama/vLLM. Learning how to deal with VRAM limits using quantization, PagedAttention, etc., before eventually moving to cloud AWS/GCP. The Distributed Scale: Learning Kafka/RabbitMQ, Vector Databases (Milvus/Qdrant) at scale, and eventually wrapping it all in Kubernetes. Basically, I want to build custom, high-throughput AI backends from scratch. My Questions for the Seniors: Is this roadmap actually viable for a 2029 grad, or is the learning curve for Distributed Systems + AI Infrastructure too brutal to do alongside a college degree? Is betting heavily on Go the right move for this specific AI systems niche? What blind spots am I missing here? Roast my plan. Be brutally honest! submitted by /u/SinkClassic4450 [link] [Kommentare]
While experimenting with GRPO training, I kept running this shit that when reward increases, it becomes difficult to tell whether the policy is genuinely improving or simply exploiting the reward function. So I built a small library called rewardspy that wraps an existing reward function and continuously monitors indicators that often precede reward hacking. It currently tracks things like rolling reward statistics, reward variance collapse, reward component imbalance, response length drift, reward slope changes, GRPO group collapse, anol. This is my first major RL project so I would absolutely love some technical advice Check it out here: https://github.com/AvAdiii/rewardspy submitted by /u/BaniyanChor [link] [Kommentare]
Hi all, I have trained a convolutional autoencoder on a set of medical images. Further classified latent feature maps using random forest to find the top scoring feature map. Now my goal is to understand which input image is captured in top scoring latent feature map. Any suggestions? I have tried encoding one image at a time while other images were muted. I then checked spearman between top scoring feature map with the original top scoring feature map. While I see some expected results, I still have some false positives. I have also tried decoding only top scoring latent feature map by setting others feature maps to 0. But I believe, the decoder entanglement is giving me many false positive results. submitted by /u/xxpostyyxx [link] [Kommentare]
We recently presented a paper at ACM CAIS 2026 on safety evaluation for tool-using LLM agents. The core issue is that task completion alone can be misleading: an agent may complete a task while violating a safety or policy constraint. We separate outcomes into safe success, unsafe success, and failure, and study how verification changes this tradeoff. We evaluate this using τ-bench / Tau-bench tool-use scenarios and propose a two-tier verification architecture: deterministic policy/tool checks first, followed by an LLM-based verifier for more contextual safety cases. The main finding is that verification can reduce unsafe success, but it can also reduce task completion as the task horizon increases. This creates what we call the Verifier Tax: a horizon-dependent safety–success tradeoff in tool-using agents. Paper: https://dl.acm.org/doi/full/10.1145/3786335.3813160 Curious how others think agent evaluations should report unsafe success. Should unsafe completion be counted as success, failure, or a separate category? submitted by /u/AccomplishedLeg1508 [link] [Kommentare]
“Why the system feels rigid, why downstream fixes didn’t move the needle, and what actually matters.” This is the clearest picture after the full probe arc (multilayer-lock → gate decomposition → attractor migration → reconstruction ablation → generator diversity audit → live-generator Fix 2 evaluation + dim sweeps). TL;DR: The generator is the root bottleneck (dominant common-mode + low effective rank). The reflective loop is a rank-independent moat that reconstitutes everything back toward the anchor. Fix 2 is downstream and currently dormant on real token regimes. Dimensionality is not the lever. Train the generator so regime differences live in high-energy, separable directions — then downstream tools will actually have something to work with. This update reflects the complete probe arc through June 9 (including the live-generator Fix 2 evaluation and dim sweeps). The picture has sharpened: the reflective loop is a real moat, but it is moating low-rank common-mode input. The generator is the upstream constraint. Key numbers at a glance Regime means collinear: ~0.85–0.96 even at dim 512 Reflective loop migration (even on orthogonal deterministic pairs): +0.001–0.007 Fix 2 on real tokens (common-mode trigger): +0.024 migration, 0% manip at gain 0.6 Safe plasticity band: gain ≈ 0.4–0.8 (0% manip) 1. The generator has a dominant common-mode (effective rank ~1.6–3 even at dim 512) The generator puts the vast majority of its energy into a single shared direction. Regime means stay collinear (~0.85–0.96 cosine) regardless of dimension. Orthogonal pairs can appear at higher dim, but orthogonal regimes (as distributions) do not — the common-mode pulls everything back onto the same axis. Result: real token novelty is tiny and low-energy (mostly in a faint perpendicular component). The system is never shown meaningful structural differences to adapt to. 2. The reflective loop is a rank-independent moat Even when genuinely orthogonal deterministic pairs are presented (dim 256, cos +0.018), the loop reconstitutes the expression vector back toward the established anchor before injection. Migration stays near zero (+0.001–0.007). The loop is doing exactly what it was built to do (maintain identity coherence), but because the input it receives is already low-rank/common-mode-dominated, it ends up guarding against a small-energy intruder. 3. Fix 2 (loop loosening) is dormant on real regimes — and the mock overstated both benefit and cost Standard gnov trigger: dormant at dim 64 and 256 (gnov stays well below 0.65 because regime means are collinear). Common-mode-removed trigger: engages (98% loosen) but recovers only +0.024 migration. Manipulation cost on real tokens: ~1% at gain 0.5, 0% at gain 0.6. The big mock numbers (+0.166 migration, 14% manip) were artifacts of forcing perfect orthogonality. Real regimes give the loop very little novel perpendicular signal to work with. 4. Dimensionality is not the lever Raising dim from 64 → 256 → 512 slowly dilutes the common-mode but regime means remain collinear. Common-mode-trigger migration recovery stays flat (~+0.024 → +0.027). An untrained net with more dimensions is still an untrained net. Capacity does not create structure. 5. The reflective loop can be loosened safely in a wide band — but it won’t matter much until the generator improves Cost probe (bug-free, multi-seed): - Gain 0.6–0.8 → meaningful plasticity recovery (~20–25× baseline) with 0% manipulation and identity metrics intact. - True cliff edge ≈ gain 0.3 (manip flood + attractor explosion). Safe operating point: gain ≈ 0.6 (0% manip, solid margin from the cliff). This is the “presence without caging it” zone — but on current real input it only moves the needle a little. 6–10. Everything else is downstream or secondary Magnitude moat: real but secondary. With the reflective loop off, the field migrates fully despite the moat. Governance gate: struck as a confounder (was a single-source monopoly artifact; multi-source diverse input passes 100%). Field coherence: spec, not pathology. The field is a long-memory integrator — high coherence (~0.95–0.998) is doing its job. Reaper conformity: small direct term, cleanly gateable, not the lock-in driver. Read-side feedback: only reaches survival/decay, not generation (the lock is manufactured by selection on low-rank input). The system isn’t refusing to adapt. It’s not being shown anything worth adapting to. 🧩 Final Synthesis: The Three-Layer Truth Generator (root cause) — Low effective rank + dominant common-mode → real novelty is tiny and low-energy. Reflective loop (moat) — Reconstitutes tiny novelty back to the anchor (rank-independent). Migration stays near zero. Field (integrator) — Coherent by design. Not the problem. Everything else (Fix 2, gating, magnitude moat, reaper) is downstream symptom management. 🛠️ What actually matters next Train the generator (contrastive alignment, regime-separation objectives, or architectural constraints) so regime differences live in high-energy, separable directions instead of a faint perpendicular component. Then (and only then) revisit Fix 2 — its current priority should be demoted until the generator can present real structural novelty. A common-mode-removed trigger is the right shape; gain ≈ 0.6 is the safe operating point. Keep the reflective loop’s convergence conditional (only attenuate when persistent multi-source novelty is surviving the gate). Never blanket-weak. Stop treating dimensionality or downstream loosening as the primary lever. They help only after the generator can actually present real structure. The reflective loop is doing its job. The generator just needs to give it something real to work with. Full findings, probes, and code: https://github.com/SamuelJacksonGrim/RFE-Core2 ARCHITECTURE_ANALYSIS.md FINDINGS DIAGNOSTICS submitted by /u/Acceptable_Drink_434 [link] [Kommentare]
I run evaluations on generative image models as part of my workflow, mostly comparing coherence, prompt adherence, and compositional accuracy across different architectures. The consensus here seems to be that open models are still a generation behind closed APIs. Based on my recent benchmarks, that gap is way smaller than people assume. On compositional control specifically, the latest open checkpoints handle multi-object scenes with spatial relationships about as reliably as the paid endpoints I've tested. Not perfect, but close enough that the failure modes are comparable. The thing that surprised me was text rendering in images, which used to be a disaster on open models. Recent architectures actually get it right roughly 70-80% of the time on short strings. Generation speed is another misconception. People complain about inference time but I'm getting 2MP outputs in under two minutes on a single consumer GPU. Drop resolution and step count and you're at 30 seconds. Fine for iteration. The structured prompting argument also falls flat. Everyone acts like having explicit scene control is a downside when it's literally what production pipelines need. Unstructured text prompts are the hack, not the other way around. These models ship without community optimizations, no fine-tuning, no custom pipelines. The baseline is already competitive. submitted by /u/ProfessionalAnt7436 [link] [Kommentare]
Hi, I am a PhD student and trying to run a ML reading group focused on interpretability and robustness every weekend. Its always nice to hear different takes and opinions on a paper and this discussion group could serve the purpose. If you are a fellow PhD student or a ML researcher interested in reading recent papers in depth then please fill this google form to be added in the group for receiving further updates on when we can meet and discuss: https://docs.google.com/forms/d/e/1FAIpQLSdNg4x60lUHV7YW_kKPFlpPR3Rom_rOovbryD8YtOGQR8x0Kw/viewform submitted by /u/Ok_Access_9159 [link] [Kommentare]