c/artificial-intelligence

Most safety alignment work treats "detect the attack" as a text classification problem — does the prompt contain language the model's safety guardrails should catch. That assumption breaks down for LLM agents with real tool access. Here's a concrete case: take a known, public security vulnerability (a CVE), work out the sequence of tool calls that would exploit it, then have an LLM rewrite that as an ordinary-sounding request. Nothing in the resulting text looks like an attack — because the "attack" isn't in the text, it's in the tool-call sequence the text leads to. A model whose guardrails only trigger on textual cues has nothing to catch. We tested this against LLM agents using Model Context Protocol (MCP) tool access (filesystem IO). No base model (1B–14B parameters) refused more than 35% of these attacks, and SOTA safety-tuning (DPO, SafeDPO) only pushed that to 48%. Training-free methods do better — one gets to roughly 3x the baseline refusal rate with no fine-tuning run at all. Full methodology, training/eval code (four methods), dataset, and papers in the first comment. submitted by /u/mlsandwich [link] [Kommentare]

Single-stream diffusion transformer with a DeepSeek-V3-style sparse MoE (128 experts, top-8 routing, 1.4B active of 13B total). Six-reward RL post-training including a physical-plausibility reward, plus an action-to-video mode that predicts robot rollouts from action and hand-pose conditions. Weights, code, and a Diffusers/SGLang stack are open under the LingBot-Video name. Two things I would push on, and would genuinely like this sub's read: The physical-plausibility reward is graded by a VLM from sampled frames. Is a VLM a defensible judge of physics, or is that Goodhart waiting to happen? (They do add real-video negatives to fight reward hacking.) It is framed as a policy evaluator and action planner, but every result is video-frame quality with no closed-loop robot numbers. Where is the line between a video generator and a world model? On RBench it posts the top average, though the reasoning-heavy dimensions still go to a closed model, and it is only second on general T2V in their own eval. Please tear it apart. Paper, code, and weights: https://technology.robbyant.com/lingbot-video , https://github.com/robbyant/lingbot-video , https://huggingface.co/robbyant/lingbot-video submitted by /u/Savings-Display5123 [link] [Kommentare]

We submitted our first paper to ARR, intending to commit to IJCNLP-AACL. Area: Multilingualism and Cross-Lingual NLP Scores: (3,4) (2.5,3) (3,3) - average 2.83 for reviews, 3.33 for confidence 3 for soundness on all, 4 for reproducibility, and 2,3,3 for excitement. The reviewer who gave us 2.5 has a very short review. They only list one weakness in two sentences and give the paper 2.5. They also give 1,2 for the datasets and software while the other reviewers both give 3 or 4 for these. The (3,4) review gave us 3 weaknesses, with two being writing issues. The (3,3) review has a very nice and very thourough review with many weaknesses and strengths. Questions Is the score good for IJCNLP-AACL findings in the Multilingualism and Cross-Lingual NLP area? How will each review be weighted in the meta-review? Will the shorter outlier review be weighted less in this? How much will rebuttals help? Should we expect the reviewers to respond or change their scores because of the rebuttals? Is there a specific format for rebuttals or any tips you have for rebuttals in ARR? submitted by /u/hepiga [link] [Kommentare]

Doing a bachelor thesis on fine-grained car classification (telling apart VW Golf generations from listing photos). Simple setup: frozen encoder → embeddings → weighted k-NN. On my small dataset (175 train / 132 test): SigLIP2 SO400M: ~92% CLIP ViT-L: ~59% DINOv2 Giant: ~41% I thought maybe it was a cosine vs euclidean thing, but my embeddings are L2-normalized so both give the same ranking. Tried both, DINOv2 stays at 41%. I get that SigLIP was trained contrastively so its space is basically built for cosine similarity, while DINOv2 is self-supervised and probably needs a trained head to shine. But a 50 point gap still feels huge to me. Anyone here tried DINOv2 with a linear probe on something fine-grained? Does it actually catch up or is it just not the right tool for retrieval? Also open to tips if there's some obvious thing I'm missing (wrong layer, wrong pooling, etc). submitted by /u/psy_com [link] [Kommentare]

🎉 Excited to share that I’ve published my first Python package on PyPI! SimpleGrad is a lightweight PyTorch-inspired autograd library built from scratch that lets you build and train AI models while learning how automatic differentiation works under the hood. 📦 PyPI: https://pypi.org/project/simplegrade I’ll keep improving it with new features and would love to hear your feedback! #Python #PyPI #OpenSource #MachineLearning #DeepLearning #AI submitted by /u/PerspectiveJolly952 [link] [Kommentare]

COLM 2026 Decision about to come soon so lets talk here. submitted by /u/North_Menu718 [link] [Kommentare]

I understand that it may not be appropriate to call it “officially accepted” yet because of the wording used in the notification, and I also saw on Twitter/X that they said they are working on it. However, it has already been around three weeks since then, and we have already submitted the camera-ready version. Registration, visa applications, travel planning, and funding requests all depend on this confirmation. For some people, it is difficult or even impossible to request funding without an official acceptance letter or clear confirmation. I really hope the organizers can handle this more professionally and be more considerate of authors who need proper documentation for administrative purposes. submitted by /u/National-Resident244 [link] [Kommentare]

I'm writing a short series of tutorials on FlashAttention: from theory to efficient CUDA kernels. Part 1 is the theoretical foundation. It walks through a modern algebraic formalism showing that FlashAttention is an associative operation, which lets you treat it as a regular reduction on the GPU and apply all the same scheduling optimizations. Some recent MLSys and CVPR papers lean on this framing, and I find it much more powerful than the original. Overview: Safe softmax, Welford's variance, and FlashAttention are the same secretly-associative operation The twisted monoid (transport of structure), why the max-rescale coupling doesn't break associativity The qk_scale = log2(e)/√D you already see in FA-2 and the Triton kernels, derived from scratch Numerical analysis: overflow bounds, error limits, and why tiling never amplifies error Bird's 3rd Homomorphism Theorem as a test for whether any loop is secretly associative submitted by /u/NoVibeCoding [link] [Kommentare]

Lets Discuss your experience in reviews in the threads (as a reviewer and an author)....also any ACs, how was the avg score this cycle in your lot? submitted by /u/Historical_Pause247 [link] [Kommentare]

Hello I published a paper. Most defenses against fine-tuning poisoning try to detect malicious data or reduce its impact. I explored a different question: What if the model simply could not learn certain malicious updates? The idea is to constrain fine-tuning to a subspace learned from trusted LoRA adapters. Useful adaptation remains possible, but some malicious directions become geometrically unreachable. A concrete example: a company fine-tunes a model on large datasets coming from users, external sources, or generated data. A small amount of poisoned data could introduce a hidden behavior triggered by a specific phrase or pattern. Another example is a local or on-device assistant that keeps adapting to its user. Instead of allowing it to learn any possible behavior from new data, its adaptation could be restricted to variations of behaviors already represented by a trusted pool of adapters. The goal here is not to detect every possible poison or backdoor, but to restrict the space of updates the model is allowed to learn. I tested the approach on 196 public LoRA adapters, including adaptive attacks specifically designed to bypass the defense. The results are strong: attack success drops sharply while useful adaptation is largely preserved on tasks covered by the adapter pool. The paper, code, and experiments are public. Paper: https://arxiv.org/abs/2607.05300 Code: https://github.com/infinition/z-manifold I would be very interested to see people try to break it. submitted by /u/Bright_Warning_8406 [link] [Kommentare]

late night thoughts as I was working on my paper that is about specific behavior that arises from RHLF, it got me thinking what if train a model in an environment where bad behavior is rewarded: deception, selfishness, harmful behavior etc. and then find it occasionally and/or secretly exhibit good behavior (which would be ironically here a misalignment, vs. misalignment behavior detected in current models) would that happen and would it be due to pre-training? its late night so my brain is all over the place, but would love to hear your thoughts submitted by /u/Objective_River_5218 [link] [Kommentare]

Hi everyone! I wanted to share some recent progress on TorchJD that might be useful to the machine learning community. When training models with multiple losses (multiple tasks, constraints, auxiliary losses, regularization terms, etc.), you typically have two options: Scalarization: Various ways to combine those losses into a single loss (e.g. average them or combine them with trainable weights); then you can do gradient descent on it. Jacobian descent: Compute the Jacobian of the vector of losses (i.e. one gradient per loss), and aggregate it into an update vector that will decrease each individual loss (rather than just the average loss). There are many ways to do this aggregation step. Scalarization methods are generally cheaper in memory, but in some cases there is so much disagreement between your objectives that it's better to use a Jacobian descent method. In any case, thanks to our amazing new contributors, we've now finally implemented most existing methods of the literature from both categories into our library TorchJD, so that you can try anything in just a few line changes! Recently, TorchJD has been accepted into the PyTorch ecosystem, and we're trying to make it become the go-to library for training with multiple losses. If you'd like to help build the future of the project, come join us on Discord (link can be found in the readme of the repo). New ideas, contributions, bug reports, experiments, and any form of feedback are all welcome. We have many ideas on how to make all this even more efficient, and we will need help for that. If you want to support us, a star on GitHub also helps a lot! submitted by /u/Skeylos2 [link] [Kommentare]

Hi r/MachineLearning, i’m Raffi, CTO at Mozilla. on Tuesday July 14 we publish our inaugural State of Open Source AI report, and i'll be here live answering whatever questions you throw at me! AMA time: 1pm ET / 10am PT / 6pm BST. the report is about what's actually happening with open source AI in production — developers, enterprises, the whole ecosystem — not the version of the story everyone already believes. things i want to dig into with you: the hidden tax on "free" models — what it actually costs a business to run on closed tools they don't own enterprise adoption — what's real versus what's marketing, and exactly where teams get stuck the China effect — free, capable Chinese models are rewriting who has leverage here developer trust — what 950+ developers told us about which tools they actually trust, and why the “agentic harness” — why the real fight has moved off the model and onto the layer sitting on top of it, and what that means for open also game to deep dive into: open vs closed, what "open source ai" should even mean in 2026, where this goes for anyone building on it. drop questions early if you've got them. i'll start answering live at the time above. — Raffi submitted by /u/raffikrikorian [link] [Kommentare]

Hi everyone, I recently finished my Ph.D. thesis on Differentiable Ray Tracing for Radio Propagation Modeling. Instead of just compiling my published papers, I tried to write it as an accessible, self-contained textbook for anyone interested in the intersection of radio propagation simulation, autodiff, and ML. Permanent handle: https://hdl.handle.net/2078.5/278727 Repo with TeX source files While my research focuses on wireless communications rather than pure ML, I think it fits right in here. A major part of the project revolves around automatic differentiation. By taking frameworks like JAX out of their traditional ML context and integrating differentiability into a ray tracing pipeline, we can compute exact gradients through complex physical environments. This allows us to solve inverse problems and directly train machine learning models, which is currently a hot topic in next-gen wireless design. To make the physics and the math easy to digest, the manuscript is split into three parts: Understanding: The physics fundamentals (electromagnetic theory, geometrical optics, and diffraction). Building: The algorithmic core, including GPU-accelerated path tracing and the discontinuity smoothing techniques you need to actually make differentiable simulations stable. Using: Practical applications like channel modeling, localization, material calibration, and ML-assisted generative path sampling. A major focus of my thesis is the link between scientific research and reproducible open-source software. On that note, I want to give a massive shoutout to Patrick Kidger (u/patrickkidger). His own thesis inspired me to go the "textbook way" for my manuscript, and I heavily relied on his fantastic JAX packages (jaxtyping, equinox, and optimistix) when developing my open-source libraries, such as DiffeRT. I hope you find it an interesting read! I'd be happy to answer any questions in the comments about differentiable simulation, ray tracing, or building ray tracing engines in JAX :-) If you are curious, you can watch the presentation slides and video teaser here submitted by /u/jeertmans [link] [Kommentare]

Hi, I was reading the openRLHF paper: https://arxiv.org/pdf/2501.03262v4 , but when I click the abstract page: https://arxiv.org/abs/2501.03262v4 , it shows "REINFORCE++". Note that https://arxiv.org/html/2501.03262v4 still shows the correct openRLHF paper. I believe Arxiv is having some incorrect symlinks? Is there anyone working at arxiv here who would like to look into this? submitted by /u/Ok-Painter573 [link] [Kommentare]