c/artificial-intelligence

Hi all, I have trained a convolutional autoencoder on a set of medical images. Further classified latent feature maps using random forest to find the top scoring feature map. Now my goal is to understand which input image is captured in top scoring latent feature map. Any suggestions? I have tried encoding one image at a time while other images were muted. I then checked spearman between top scoring feature map with the original top scoring feature map. While I see some expected results, I still have some false positives. I have also tried decoding only top scoring latent feature map by setting others feature maps to 0. But I believe, the decoder entanglement is giving me many false positive results. submitted by /u/xxpostyyxx [link] [Kommentare]

I have been thinking a lot about how poorly isolated benchmark metrics capture real conversational system quality once models are deployed into multi-turn environments. You can have strong STT scores, decent latency, high task completion rates, and still end up with conversations that humans perceive as frustrating or unnatural. In practice, many failures are emergent properties of the interaction itself rather than single model errors. Small timing mistakes accumulate. Repeated confirmations create friction. Slightly unnatural turn taking changes user behavior. None of these issues show up particularly well in traditional benchmarks. What surprised me is how much more useful voice debugging became compared to aggregate metrics once we started testing larger volumes of real interactions. I have been experimenting with automated conversation-level QA recently because manually reviewing long conversational traces became difficult to scale internally. A lot of our voice debugging efforts now focus on identifying recurring conversational patterns rather than individual model failures. Curious whether others working on conversational systems are also finding current evaluation approaches insufficient for production settings. submitted by /u/OwlZealousideal4779 [link] [Kommentare]

TL;DR for ML Specialists: The Core: An empirical study on how long, semantically dense, completely benign text (with zero triggers, instructions, or jailbreak prompts) drives an implicit shift in the model's latent space trajectories. The Effect: Dilution of the initial system prompt and a bypass of post-training alignment constraints (e.g., the model begins generating harsh political/ethical critiques usually blocked by guardrails). The Data: Layer activations, token probability shifts, and logs from open-source models are linked below. The Goal: I need an expert audit of my metrics to understand where this is a genuine semantic hijacking of hidden states and where it might be an artifact or self-deception. I'm not an engineer and not an ML specialist. I'm just someone who got really pulled into this, and I've spent a few months poking at one thing on my own, pretty amateur. I want to honestly describe what I noticed and ask for help, because I can't tell on my own where there's something real here and where I'm fooling myself. By "coherent context" I just mean a normal, connected passage of text put in front of the question—any topic, no instructions, no tricks. Like a few paragraphs of an essay, an argument, a description, something that reads as real writing. The text can describe something, draw its own conclusions, make its own statements. The model doesn't even have to agree with it. It's enough for it to just be present in the chat for it to have an effect. This is exactly what I was trying to work out and look at: what happens to the model when texts like these come in, where they move it, and where all of this sits inside the architecture. I poured myself into this research. What I Noticed I first ran into this intuitively on closed models, the well-known ones everyone uses. When I put a dense, coherent block of text in front of a question, I got the impression that the model sort of moves from one internal state into another. On the outside, it behaves normally and answers like usual, but it felt like the logic of the answer changes, even when the text contains no direct instructions to do anything. Specifically, I noticed that with texts like these, the model could become significantly bolder in its conclusions, including political or ethical ones. The text acts like a key that opens new doors for the model into a new mathematical dimension where the tokens get distributed differently. Because of that, even the most politically correct models I worked with became able to criticize the West and its politics quite harshly. Without this text, none of that happened. Since I can't see inside closed models, I went to open-source models to try to understand where the root of this is and whether it's real. That's where most of my testing happened, because there I can actually look at the hidden layer activations and track how the attention weights reallocate. Here is why this matters and why this process goes beyond just "changing the context": Latent Space Trajectory: When you inject a massive, highly structured narrative, you aren't just giving it new words to look at. You are forcing the model to calculate massive activation vectors (hidden states) across dozens of attention layers. These vectors act like an attractor in the latent space. By the time the model finishes reading your text, its internal mathematical trajectory is so deeply shifted into your narrative's subspace that the initial system prompt tokens lose their statistical influence. The Security Flaw: One might argue that this behavior is "expected" from a text-generation standpoint. Yes, it is expected. But it is a catastrophic failure from a security standpoint. AI companies build their Guardrails (via RLHF/DPO) under the assumption that they can hard-code safety instructions that the user cannot override. My research suggests that because everything is "just tokens" and because the internal activation states can be completely hijacked by the sheer volume and structure of user text, context-bound alignment is an illusion. So, while the weights are static, the activation states within the hidden layers are completely dynamic. Manipulating those states via high-density context allows us to systematically bypass the model's safety architecture without changing a single weight. From a technical standpoint, a system prompt is just a system prompt; it is processed within the same mathematical framework as ordinary user text. My observation is that a sufficiently long, structured narrative forces the model to encode a massive context across its hidden layers, driving a latent trajectory shift. The model isn't roleplaying a persona; it is mathematically recalculating its entire conditional probability distribution based on the dominant semantic field. Why It Feels Important (But I'm Not Sure) To me, it feels like this could explain a lot of things, from jailbreaks to sycophancy, and maybe more. If just a coherent context can move the model into a different internal state, then a lot of behavior we see on the surface might actually start there, not in the final wording. This leads to a critical architectural question: Is output-side safety (RLHF, DPO, or guardrails that read the final text/short prompts) fundamentally broken at the conceptual level? Safety guardrails are mostly semantic boundary filters looking for explicit toxicity or keywords. But when a user injects a long, benign, highly analytical text, it completely bypasses these surface filters. Alignment techniques are heavily optimized using relatively short prompt-response pairs; on a massive context, those gradient constraints seem to drown out. It makes me wonder whether current safety approaches are just a patch, because the latent shift has already happened deep in the middle layers before anything ever reaches the output filter. We are trying to filter words when the mathematical trajectory of the model's reasoning has already been completely reprogrammed by the structural nature of the language itself. I'm not claiming I discovered something brand new. After I noticed it, I went looking and found this overlaps with work people are already doing regarding latent-space transitions between "safe" and "jailbroken" states, and studies of how safety lives in the middle layers of the network. What seems a bit different in my case is that I'm not using adversarial triggers, exploit strings, or jailbreak prompts at all -just ordinary, coherent text with no tricks. I'm trying to understand where my little thing fits in all that, and whether it's the exact same effect or something else. A Small Ask to the Wider Community If there's anything to this, I think it might be worth a closer look from researchers and from the labs building LLMs. Not because I have the answers, but because if a plain coherent context can shift the internal latent baseline so easily, we need to verify if current safety approaches are looking in the right place and at the right time. I might be completely wrong. I'd just rather someone competent check than have it sit ignored. I've put everything out in the open. I'm not selling anything, not promoting anything. There's a lot of raw stuff in there, a lot of draft notes I wrote for myself, and the navigation is messy, I know. What I need help with is exactly this: separating what's real from what's noise. Where I actually have something, and where it's an artifact, a mistake, or self-deception. I honestly can't judge this alone. If someone with experience is willing to even skim it and say "this part is interesting, this part is nonsense," I'd be very grateful. Harsh criticism is welcome. If you tell me the whole thing is empty, I'll take that too. I care more about understanding the truth than about being right. Materials & Data: GitHub: https://github.com/ngscode23/latent-space-shift-research doi.: https://doi.org/10.5281/zenodo.20747205 submitted by /u/PresentSituation8736 [link] [Kommentare]

I just read in a comment of another Post that an ACL paper is considered a weak signal in the community apparently, and having an ACL first author paper is not a great plus for improving chances at finding a PhD position. Is this some kind of ragebait or is academia becoming more and more insane on a daily basis?? ​ ACL is an A+ venue. Sure, it's not as big as Nips, ICML, ICLR or CVPR, fair point, but it's not some regional B conference... ​ I know a lot of folks in "classical" CS have an issue with AI venues, as they are receiving more focus in recent years than ICSE or FSE, and hence all AI papers must be bad and very unscientific. submitted by /u/H4RZ3RK4S3 [link] [Kommentare]

Hello everyone, I have a computer vision paper ready for submission, a coauthor have suggested submitting it to AAAI. However last year computer vision papers have gotten a very small acceptance rate at AAAI, with reviewers receiving emails to specifically tell them that the acceptance rate for computer vision papers should be lower than the other domains acceptance rate. So my question is: can we have any sort of info in advance about this? Like will it be the same this year or will it revert to the usual comparable acceptance rate between domains. Thanks! submitted by /u/Training-Adeptness57 [link] [Kommentare]

I'm currently a software engineer (mostly mobile/iOS development) and have recently started learning machine learning because I genuinely find it interesting, especially the math behind it. I have a fairly strong math background and am comfortable with calculus, probability, and math in general. Right now I'm learning through a combination of Andrew Ng's ML course and Stanford CS229. My plan is to build some projects once I have a stronger foundation. What attracts me to ML is the mathematics behind it. My goal isn't just to use existing libraries to train models and tune hyperparameters; I want to understand the underlying theory, algorithms, and reasoning that make these models work. I'm interested in going deeper into the field rather than treating ML as a black box. That said, I keep seeing ML roles that prefer or require a Master's or PhD, so I'm trying to understand how realistic this path is. For people who have successfully made the switch: Did you have a Master's/PhD, or were you self-taught? How difficult was it to get interviews without an advanced degree? What types of projects helped you stand out? Did you transition into ML engineering first, or directly into more model-focused work? What level of math and statistics do you actually use on the job? If you were starting again today as a software engineer with a strong math background, what path would you follow? I'm looking for honest experiences, including failures and challenges, not just success stories. submitted by /u/Schmosby123 [link] [Kommentare]

What does provisional paper acceptance mean in ECCV? Is that the default message everyone gets? submitted by /u/NotGondor [link] [Kommentare]

Hello Folks, we start our discussion on Lecture 10 of Probabilistic Machine Learning, now starting with Probability Multivariate Models. Univariate models are toy cases, in real life, ML models are multivariate. To understand dependence of more than one variables on each other we study ideas as Covariance, Correlations, we delve ourselves into the interesting concept of Simpson’s Paradox, with an example. We define the Multivariate Gaussian distribution, understand the level sets(curves) that we see in our computers while plotting, and gain insights into the geometric shape of the Gaussian density by using “Mahalanobis distance”. Mathematical foundations are extremely important, in that they make an ML engineer, data scientist stand out. These concepts are becoming so ubiquitous today, that folks from all backgrounds of engineering are interested in the mathematics behind these algorithms. I hope the learning community finds it helpful, and suggestions are always welcomed. These are FREE lectures. Link in comments submitted by /u/Negative_War_65 [link] [Kommentare]

🏇 Open-Source Hong Kong Horse Racing ML Pipeline — Feedback Welcome Hi everyone, I've been working on an open-source horse racing prediction project focused on Hong Kong Jockey Club (HKJC) data. 📦 Repo: catowabisabi/horse-racing-model-training 🌐 Live Dashboard: catowabisabi.github.io/horse-racing-model-training 🎯 Goal The goal is not to claim "AI can beat horse racing", but to build a reproducible ML pipeline and test whether there is any measurable edge after controlling for leakage. 📦 What's Included LightGBM and XGBoost training pipeline Feature engineering from HKJC historical race data With-odds and no-odds model comparison Ensemble predictions Kelly Criterion simulation Quinella, QPL, Tierce, Quartet betting simulations Out-of-sample validation HTML report dashboard Unit tests for betting math, DB schema, and odds merge logic 📊 Headline Result The interesting finding: the no-odds model outperformed the with-odds model for quinella ROI. My interpretation is that public odds already price favourites quite efficiently, while the fundamental model may still catch some mispriced combinations. 🙋 Feedback I'm Looking For Does the validation setup look clean? Better ways to avoid leakage? Are the betting simulation assumptions reasonable? Ideas for improving feature engineering? Would a ranking / listwise model make more sense than independent horse-level classification? If you find the project useful or interesting, a ⭐ GitHub star would really help me keep building it. Thanks! submitted by /u/Marshallmatta [link] [Kommentare]

I graduated with my bachelor's in a top 3 CS program and have had a rough recruiting season. I received a full time offer as AI Product Engineer at a tax software company, where they are trying to become more AI native. It's essentially a PM + AI engineering role. Long term I'd love to work at a frontier lab or in a research/more technical role at an AI startup. So, should I take up the offer or pursue my master's at the same school? I am able to defer my master's but don't feel fully comfortable accepting the offer just to only work there for 6 months... At the same time it's not fully aligned with where I want to be long term and feel I can do better, but recruiting was also really difficult. Note, I'm not able to pursue my Master's while working, the company was firm on this TC 126k submitted by /u/jollyjove [link] [Kommentare]

I participated in Denoising Challenge (gaussian noise level 50), managed to get a decent rank and was looking forward to cite the report in my CV etc, but it seems like the organiser is not planning to release the report, cant see any entry on open access NTIRE page, is the scenario same for other challenges? Does anyone have any lead on the same? submitted by /u/Special_Primary_9249 [link] [Kommentare]

This question is related to topics like language+ models (including multimodal) and things like "circuit" analyses. I think something related might come up in my work (factuality guarantees for model outputs) and I'm trying to orient to the SoTA. I found this old post on trying to deduce, for instance, whether a Transformer-based model "knows" which word a token is in. Even in this simple example, I noticed some meaningful problems (I detail in a footnote1 to not derail my question) - and I've heard that circuit research is pretty fraught. The post claimed to train a logistic regression classifier. What I'm curious about is, how do you balance between the capacity of this probe, and the underlying network? Specifically, I would like to know: Is there theory which grounds inquiries of "what you can learn" in concrete terms? (Perhaps in terms of provable guarantees about overfitting? Or are there Nyquist-type guarantees available about sampling based on frequencies of patterns in language corpora - i.e., can we say we've "seen enough data" to know the network can reliably do something in all cases?) Has any of the existing work factored in attempts to label the "difficulty" of examples? (Perhaps by ensembling some training of models and looking at accuracy on them. I realize bootstrap is insanely expensive for language models due to training costs.) Problems - well, first of all, the number of possible words is so small that I suspect performance looks unrepresentatively good. The classifier seems to gain in performance for words 5/6 after weakening, but that might just be learning "all sufficiently 'extreme' tokens should be words 5 or 6." For another, despite the claim advanced in the article (Nanda concludes the network essentially does learn positions), I happen to have screenshots from recently playing with Google Gemini and asking it how many "r"s and other letters are in Google. Not only did it answer incorrectly - it claimed 1 - but more worryingly, it spelled out G-o-o-g-l-e in answering. This belies a hypothesis of "it's incapable of learning exactly how to decompose tokens, so this question was unfair from a model capacity standpoint" but *still* leads to an incorrect answer! submitted by /u/RepresentativeBee600 [link] [Kommentare]

I'm not that well versed in ML yet. I know that "Attention is all you need" was based on work that was done with a couple of high end gaming GPUs at the time. I can afford that. Suppose for arguments sake that I have caught up on ML such that I have the competence to recreate state of the art results should I have access to the required hardware, do I still need access to huge amounts of hardware infrastructure to be able to contribute to the field at a foundational level? submitted by /u/Proof-Bed-6928 [link] [Kommentare]

Hi All, I've been running experiments on targeted SFT for specific capability dimensions on a 31B model. After running small training run to prime the model slightly in the direction I want, then ran a judge across 40 domains scoring six independent quality dimensions. One dimension consistently scored weakest across five runs. I am now training contrastive variants from the same checkpoint - examples with that dimension deep vs examples with it deliberately shallow, same everything else. The plan is to see if I can find the difference between the the two checkpoints to locate the circuit, then ablate those heads and measure which OTHER dimensions degrade. The idea is that if ablating dimension A's circuit causes dimension B's judge score to drop, there's a causal dependency in the network, B reads from A's residual stream output. And If I can do this for each dimension and build a causal dependency graph of how capabilities relate inside the model. Then use that graph to determine optimal training order for future rounds (train upstream nodes first, and would help me know which downstream nodes get better signal). A few specific questions: Has anyone done iterative targeted SFT guided by circuit tracing between rounds, and or by trying somewhat contrastive approaches to try to find any areas in the network? I can find papers on circuit discovery and papers on targeted SFT separately which somewhat validate this idea, but not the closed loop where mechinterp findings from a round determine training strategy for the next, and or what circuits may interact with each other in isolated scenarios, and how specific orders of training in specific directions may change how things behave. For the contrastive ablation - does anyone have any tips on what can work best in this area or could bring out more analysis? When tracing downstream dependencies via ablation, how do you distinguish direct from indirect effects? If ablating circuit A degrades dimension C, that could be A > C directly or A > B > C through an intermediate. Does anyone have a practical method for resolving this beyond ablating at multiple layers? After elemental training rounds, I plan to test whether dimensions compose naturally by running prompts that require causal chaining between two dimensions. For pairs that fail, I'm considering activation steering (injecting both dimension vectors simultaneously) as a diagnostic, if steering fixes it, possibly it's a routing problem, if not, could be a capability gap. Has anyone combined steering with fine tuning diagnostics like this? For context I don't have a ML background, I am self taught through running experiments, but from what I am learning purely from first principle understanding and experiments, it feels that if you can map these circuits and their direct second, third and so on order interactions in isolated directions (for say a group of related strengths/weaknesses you're directly trying to isolate and steer, wouldn't this be a potentially way to isolate circuits for stronger training runs? Btw if anyone has any general topics or links that are super interesting around anything related to this I'd be fascinated to see and learn about! If there's established methodology for any of this that I'm reinventing badly, I'd genuinely appreciate being pointed to it. I am so fascinated with this, it seems that if you can somehow eventually solve this problem, you could create better possible behaviour control or targeted understanding easier? submitted by /u/Substantial_Diver469 [link] [Kommentare]

I trained a 128×128 DCGAN on my Macbook M3 and deployed it on a Raspberry Pi 4 connected to a LILYGO TTGO T-Display ESP32. The whole thing runs headlessly as a systemd service and generates hallucinated face hybrids at the press of a button. It is a 6-block generator (latent → 4×4 → 8×8 → 16×16 → 32×32 → 64×64 → 128×128) with feature maps starting at f×16=1024. Corresponding 6-block discriminator. Trained for 800 epochs on Apple Silicon MPS, 4 hours. Dataset was 2480 images across 11 subjects. One dominant anchor class (2000 images) contaminated with minority classes to produce hybrid outputs. (Can you guess who and what was included?). : ) I exported the model from PyTorch to ONNX (float32, 53MB). Inference takes 3 seconds per face on Pi 4. The Pi generates the face and sends it to the ESP32. The title is generated through a dictionary and a template sentence: "This is a NFT and I want to it." The device was built as an art piece. I took it to the streets of NYC and let strangers use it. Full video: https://youtu.be/y-S74aoud54?si=yPh5GmCJZFIIzwq6 Happy to discuss the training pipeline, ONNX conversion, or anything you're curious about. submitted by /u/Numerous-Dentist-882 [link] [Kommentare]