TL;DR for ML Specialists: The Core: An empirical study on how long, semantically dense, completely benign text (with zero triggers, instructions, or jailbreak prompts) drives an implicit shift in the model's latent space trajectories. The Effect: Dilution of the initial system prompt and a bypass of post-training alignment constraints (e.g., the model begins generating harsh political/ethical critiques usually blocked by guardrails). The Data: Layer activations, token probability shifts, and logs from open-source models are linked below. The Goal: I need an expert audit of my metrics to understand where this is a genuine semantic hijacking of hidden states and where it might be an artifact or self-deception. I'm not an engineer and not an ML specialist. I'm just someone who got really pulled into this, and I've spent a few months poking at one thing on my own, pretty amateur. I want to honestly describe what I noticed and ask for help, because I can't tell on my own where there's something real here and where I'm fooling myself. By "coherent context" I just mean a normal, connected passage of text put in front of the question—any topic, no instructions, no tricks. Like a few paragraphs of an essay, an argument, a description, something that reads as real writing. The text can describe something, draw its own conclusions, make its own statements. The model doesn't even have to agree with it. It's enough for it to just be present in the chat for it to have an effect. This is exactly what I was trying to work out and look at: what happens to the model when texts like these come in, where they move it, and where all of this sits inside the architecture. I poured myself into this research. What I Noticed I first ran into this intuitively on closed models, the well-known ones everyone uses. When I put a dense, coherent block of text in front of a question, I got the impression that the model sort of moves from one internal state into another. On the outside, it behaves normally and answers like usual, but it felt like the logic of the answer changes, even when the text contains no direct instructions to do anything. Specifically, I noticed that with texts like these, the model could become significantly bolder in its conclusions, including political or ethical ones. The text acts like a key that opens new doors for the model into a new mathematical dimension where the tokens get distributed differently. Because of that, even the most politically correct models I worked with became able to criticize the West and its politics quite harshly. Without this text, none of that happened. Since I can't see inside closed models, I went to open-source models to try to understand where the root of this is and whether it's real. That's where most of my testing happened, because there I can actually look at the hidden layer activations and track how the attention weights reallocate. Here is why this matters and why this process goes beyond just "changing the context": Latent Space Trajectory: When you inject a massive, highly structured narrative, you aren't just giving it new words to look at. You are forcing the model to calculate massive activation vectors (hidden states) across dozens of attention layers. These vectors act like an attractor in the latent space. By the time the model finishes reading your text, its internal mathematical trajectory is so deeply shifted into your narrative's subspace that the initial system prompt tokens lose their statistical influence. The Security Flaw: One might argue that this behavior is "expected" from a text-generation standpoint. Yes, it is expected. But it is a catastrophic failure from a security standpoint. AI companies build their Guardrails (via RLHF/DPO) under the assumption that they can hard-code safety instructions that the user cannot override. My research suggests that because everything is "just tokens" and because the internal activation states can be completely hijacked by the sheer volume and structure of user text, context-bound alignment is an illusion. So, while the weights are static, the activation states within the hidden layers are completely dynamic. Manipulating those states via high-density context allows us to systematically bypass the model's safety architecture without changing a single weight. From a technical standpoint, a system prompt is just a system prompt; it is processed within the same mathematical framework as ordinary user text. My observation is that a sufficiently long, structured narrative forces the model to encode a massive context across its hidden layers, driving a latent trajectory shift. The model isn't roleplaying a persona; it is mathematically recalculating its entire conditional probability distribution based on the dominant semantic field. Why It Feels Important (But I'm Not Sure) To me, it feels like this could explain a lot of things, from jailbreaks to sycophancy, and maybe more. If just a coherent context can move the model into a different internal state, then a lot of behavior we see on the surface might actually start there, not in the final wording. This leads to a critical architectural question: Is output-side safety (RLHF, DPO, or guardrails that read the final text/short prompts) fundamentally broken at the conceptual level? Safety guardrails are mostly semantic boundary filters looking for explicit toxicity or keywords. But when a user injects a long, benign, highly analytical text, it completely bypasses these surface filters. Alignment techniques are heavily optimized using relatively short prompt-response pairs; on a massive context, those gradient constraints seem to drown out. It makes me wonder whether current safety approaches are just a patch, because the latent shift has already happened deep in the middle layers before anything ever reaches the output filter. We are trying to filter words when the mathematical trajectory of the model's reasoning has already been completely reprogrammed by the structural nature of the language itself. I'm not claiming I discovered something brand new. After I noticed it, I went looking and found this overlaps with work people are already doing regarding latent-space transitions between "safe" and "jailbroken" states, and studies of how safety lives in the middle layers of the network. What seems a bit different in my case is that I'm not using adversarial triggers, exploit strings, or jailbreak prompts at all -just ordinary, coherent text with no tricks. I'm trying to understand where my little thing fits in all that, and whether it's the exact same effect or something else. A Small Ask to the Wider Community If there's anything to this, I think it might be worth a closer look from researchers and from the labs building LLMs. Not because I have the answers, but because if a plain coherent context can shift the internal latent baseline so easily, we need to verify if current safety approaches are looking in the right place and at the right time. I might be completely wrong. I'd just rather someone competent check than have it sit ignored. I've put everything out in the open. I'm not selling anything, not promoting anything. There's a lot of raw stuff in there, a lot of draft notes I wrote for myself, and the navigation is messy, I know. What I need help with is exactly this: separating what's real from what's noise. Where I actually have something, and where it's an artifact, a mistake, or self-deception. I honestly can't judge this alone. If someone with experience is willing to even skim it and say "this part is interesting, this part is nonsense," I'd be very grateful. Harsh criticism is welcome. If you tell me the whole thing is empty, I'll take that too. I care more about understanding the truth than about being right. Materials & Data: GitHub: https://github.com/ngscode23/latent-space-shift-research doi.: https://doi.org/10.5281/zenodo.20747205 submitted by /u/PresentSituation8736 [link] [Kommentare]
From Genesis AI on 𝕏: https://x.com/gs_ai_/status/2066869851659121128 Website: https://www.genesis.ai/ submitted by /u/Nunki08 [link] [Kommentare]
I work in robotics, and have for almost a decade now. I keep noticing that a huge chunk of search-under-uncertainty problems, in robotics and outside it, converge on the same two-step architecture: generate variation indiscriminately first, then apply a scoring/selection pressure that keeps what works and discards what doesn't. No model of the problem is required upfront. Most the "intelligence" lives in the selection step, not the generation step. The clearest version of this in our own field is sampling-based motion planning. RRT and its relatives don't try to compute a path analytically. They expand randomly in many directions through the configuration space and then retain/extend the branches that make progress toward the goal, pruning the rest. Genetic algorithms and evolution strategies (CMA-ES, for instance) run an identical loop in parameter space instead of configuration space: generate a population of variants, score them against a fitness (cost) function, keep the survivors, repeat. Simulated annealing is a single-particle version of the same thing, generate a random perturbation, accept or reject it based on a score. Once I started paying attention to this pattern, I noticed it shows up well outside robotics too, in places that have nothing to do with computer science: Slime mold expanding in all directions through a maze of food sources, with the inefficient tendrils pruned back, and famously reconstructing something close to the Tokyo rail network when food sources are placed at the positions of major stations. Evolution itself: random mutation generates variation with zero regard for whether it's useful, and survival does the selecting after the fact. Neural development: neurons and synaptic connections proliferate in directions that aren't pre-planned, and dopamine-linked reinforcement selectively stabilizes the ones that turn out to matter. Once I started looking for more instances, I found two more that fit the same structure almost exactly: The immune system: B-cells mutate antibody variants somewhat randomly (somatic hypermutation) and the ones that bind the pathogen get clonally selected and expanded. Thought/creativity: you can't generate a genuinely novel idea by deduction from evidence that it's correct. The evidence only exists after the idea does. Novelty has to come first; judgment comes second. That last one turns out to have a surprisingly direct precedent. Henri Poincaré, describing how he worked out the theta-fuchsian functions, wrote that ideas rose in crowds and collided in his mind until pairs interlocked into stable combinations, almost like watching his own unconscious work made partially visible to consciousness, and that what got selected from that flood of combinations was governed by something close to an aesthetic sense of mathematical elegance. That's a generate-then-select loop running inside a human mind, described in 1908. The principle has been formalized more than once since then, from different directions: Richard Dawkins' Universal Darwinism: the claim that variation/selection/retention isn't a biology-specific mechanism but a substrate-independent algorithm that biology happens to be one instance of. Donald Campbell's blind variation and selective retention (BVSR), later developed extensively by Dean Keith Simonton, which applies the same two-step structure directly to creative cognition. Karl Popper's conjectures and refutations model of how knowledge grows: blind generation of new theories, followed by selective retention of the ones that survive criticism. Popper explicitly treated this as the same process as biological evolution, just running on ideas instead of organisms. Gerald Edelman's Neural Darwinism (Theory of Neuronal Group Selection), the formal version of the neuron/dopamine point above: synaptic overproduction followed by activity-dependent selective stabilization. Worth flagging here: I'm not claiming the expansion step is ever truly random. In every example above, the variation is guided. Slime mold follows chemoattractant gradients, not isotropic noise. Mutation isn't uniform across a genome, there are hotspots and repair biases. Axon growth follows chemical guidance cues, not random angles. Informed RRT* deliberately biases sampling toward the goal region instead of sampling uniformly. Even Poincaré's account isn't pure randomness, he describes an aesthetic sense that seems to steer which combinations even get generated, not just which ones survive afterward. Liane Gabora has made this exact critique of BVSR, that calling the variation "blind" overstates how random it actually is. But that's the part I find more interesting, not less. The expansion step across all of these systems is intelligently informed, biased toward promising regions by something the system already "knows," and yet it still needs the separate selection/scoring step on top of that guidance to actually converge. Neither half does the job alone: the guidance is too crude or too local to solve the problem outright (that's why expansion is still happening at all instead of direct computation), and the selection pressure has no foresight of its own, it only works because it's filtering output that the guided expansion already biased toward viable territory. Has anyone else found that this specific combination, intelligently biased expansion paired with a separate selection/scoring step, actually performs best in practice? I have personally found, that at least for my applications in autonomous vehicles and motion planning, this combination works the best. Curious whether other people doing sampling-based planning, evolutionary algorithms, or other search methods have found the same thing I have: that this architecture outperforms the alternatives, rather than just being one option among several that works comparably well. I will personally only ever use this type of algorithm after realizing this is how nature does it. Lastly, I am not proposing that analytical algorithms are better or worse than machine learning algorithms. I believe that HOW this is solved is irrespective of the fundamental search algorithm of the universe I have observed. I believe that it is the expand, then score mechanism that is important here, and does not exclude any method of expansion or scoring, as I have observed this in the above stated forms across nature. Thanks for the read if you're here, I've been thinking about this all year and needed to post it somewhere. submitted by /u/Pepperoni-Flame [link] [Kommentare]
Microsoft Research Preprint Next-token prediction is myopic. What if transformers learn to predict their own next latent state? Microsoft Research present Next-Latent Prediction (NextLat): a self-supervised learning method that teaches transformers to form compact world models for reasoning and planning. It also unlocks up to 3.3x faster inference via self-speculative decoding! On top of next-token prediction, NextLat trains the transformer to predict its own next latent state given the current latent state and next token. NextLat has a few key benefits: Representation Learning: NextLat encourages transformers to compress history into compact belief states. Better Data Efficiency: predicting in latent space provides denser supervision than predicting one-hot tokens. Faster Inference: via recursive multi-step lookahead. I'm super excited about this work. Please do check it out below: 💬 Blog: https://jaydenteoh.github.io/blog/2026/nextlat 💻 Code: https://github.com/JaydenTeoh 📝 Paper: https://arxiv.org/abs/2511.05963 submitted by /u/jayden_teoh_ [link] [Kommentare]
I was digging through Hugging Face’s Transformers repo and found https://github.com/huggingface/transformers/blob/main/src/transformers/models/gpt_oss/modeling_gpt_oss.py From what I can tell, this isn’t just boilerplate, it looks like a full implementation. is it actually the full code on which gpt_oss is built on? or is it a skeleton for experimentation? Similarly there are many models in https://github.com/huggingface/transformers/blob/main/src/transformers/models are they really the true open source implementations? if not, can we actually find them publicly? submitted by /u/PravalPattam12945RPG [link] [Kommentare]
Hello all! Half of all industrial "chatbots" are just text-to-SQL models in a trenchcoat (and the other half RAG!). I wanted to explore just how small you could make these models if you trained, evaluated, and ran inference in the exact same structured harness, leading to Cleo: a Qwen3.5-2B-Base finetune. Currently, some features of cleo that are only possible/useful in a unified hardel are: Training on the exact same gather, repair, and answer contract it uses at inference time Searching over candidate queries with live execution evidence, not just model likelihood Co-designing the model contract, SQL safety layer, dialect handling, timeouts, and clarification behavior as one system Everything is completely open-source, including the harness, model, and datasets. GitHub: https://github.com/Dreeseaw/cleo Hugging Face model: https://huggingface.co/dreeseaw/cleo PS: If you're also resource-constrained and trying to do RL like me, I would highly recommend experimenting with ECHO: https://arxiv.org/abs/2605.24517 submitted by /u/Dreeseaw [link] [Kommentare]
I’ve been thinking about whether the same basic concept behind Bitcoin could be applied to AI training. In Bitcoin, miners perform proof-of-work and are rewarded for contributing computational resources to secure the network. The actual computation itself isn’t particularly useful outside of the network, but it creates a decentralized system. What if a similar incentive structure could be used for training large language models? Instead of miners solving hash puzzles, participants would contribute GPU resources toward training an open-source AI model. In return, they would receive tokens or rewards based on their contribution. Some questions that immediately come to mind: How could the network verify that a participant actually performed useful training work? How would you prevent people from submitting fake or harmful gradients? Could model improvements be measured objectively enough to determine rewards? Would this be more efficient than training models in centralized data centers? Could a decentralized network eventually compete with large AI companies? I know there are already decentralized AI and compute projects, but I’m specifically interested in whether a true “proof-of-training” mechanism could exist, where rewards are tied directly to improving a model rather than simply renting out compute. Curious to hear thoughts from people who understand distributed systems, machine learning, or crypto economics. Is this fundamentally impossible, or is there a viable architecture that could make it work? submitted by /u/notfinancialadvice0 [link] [Kommentare]
Recent years I see plenty of folks from OpenAI and Anthropic attending conferences like ICML/Neurips, yet obviously few are presenting. Are they mainly recruiting? Following emerging research? Curious if anyone with firsthand experience can shed some light on how attendance is justified internally and what the main objectives usually are. submitted by /u/snekslayer [link] [Kommentare]
I'm working on a paper and would love some input on model choice. Suppose you're trying to detect a specific type of cancer, but the negative samples are visually and morphologically very similar (i.e., “mimics” of the cancer). In this setting, would it make more sense to approach the problem as: Anomaly detection (treating the cancer as the target distribution and everything else as out-of-distribution), or Supervised classification (explicitly learning to distinguish cancer vs. mimics)? submitted by /u/DryHat3296 [link] [Kommentare]
I've always been interested in point clouds and spatial data, so I created my own LiDAR scanner! It runs off of an esp32 and TMC2209s on a custom PCB, which continuously rotate and sweep the LiDAR sensor. I learned a ton creating this project, as this was my first time creating a PCB and using NEMA motors (I have used other motors before). Github repo submitted by /u/thatonebckid [link] [Kommentare]
Hi, About a year ago I shared my PaddleOCR implementation here. Since then I've made many improvements, and it now supports PP-OCR v3 through the latest v6 models. The official Paddle C++ runtime has a lot of dependencies and is very complex to deploy. To keep things simple I use ncnn for inference, it's much lighter (and faster in my task), makes deployment easy. Hope it's helpful to some of you, and feedback welcome! https://github.com/Avafly/PaddleOCR-ncnn-CPP submitted by /u/Knok0932 [link] [Kommentare]
Can anyone please guide me on how to prepare for Amazon ML Summer School? I’m quite confused about where to start and what topics are most important for the selection process. Any tips, resources, or preparation roadmap would be really helpful. Thanks! submitted by /u/addiemaddieee [link] [Kommentare]
I've written a C++ implementation of distilHuBERT. https://github.com/pfeatherstone/hubert.cpp It has no runtime dependencies, the weights are compiled into the library, it supports dynamic sizes, has performance on par with onnxruntime (in my tests) and can be easily integrated into any CMake project. Please let me know your thoughts. submitted by /u/Competitive_Act5981 [link] [Kommentare]
“…5 papers at ICML (1 Spotlight)…” “…Five ICML papers is what a strong PhD produces in four years. I did it in five months…” I recently saw these posts from people at the same AI company. At first, I was extremely surprised. It turned out they were workshop papers. Am I missing something here, or are workshop papers now being treated as equivalent to main-track papers? submitted by /u/Terrible-Chicken-426 [link] [Kommentare]