c/artificial-intelligence

Did anyone else submit to ACM ICMI 2026? The reviews were recently released, and this is my first time submitting to ICMI, so I'm not very familiar with the acceptance patterns. I submitted a long paper and received the following overall ratings: 4 (Probably Accept), 3 (Borderline), 4 (Probably Accept) The reviewer with the highest stated expertise recommended acceptance, while the borderline reviewer had some concerns about soundness but still considered it a nice contribution. For those who have submitted to or reviewed for ICMI before, how would you interpret these scores? Is a 4/3/4 generally considered competitive after rebuttal, or is it still a long shot? Would appreciate any insights from past authors or reviewers. submitted by /u/kanishq95 [link] [Kommentare]

Hi everyone, I’m close to completing my degree in Psychology, and I’m also a Systems Engineering student. is like, roughly comparable to Software Engineering / Computer Science outside Latin America. Although I study engineering, I’m still at an early stage with machine learning, LLMs, AI safety, and related technical topics. My research project is mainly psychology-oriented, but I’d really appreciate recommendations or warnings from a software/technical perspective. I’m working on a project about how AI systems respond to prompts involving psychological distress at different levels of intensity. I’m currently considering ChatGPT, Gemini, Wysa, and Replika, and I’m interested in comparing general-purpose LLMs, mental-health-oriented chatbots, and AI companions. Some aspects I’m thinking about are: How each system handles mental health, self-harm, crisis situations, and psychological/medical advice. whether responses change as the prompt becomes more intense, for example when a normal generated response is replaced by a safety protocol, moderation layer, or crisis-resource response. whether systems respond differently to declarative prompts versus question-based prompts, such as “I feel emotionally overwhelmed” vs. “What should someone do if they feels emotionally overwhelmed?” whether responses differ when distress is explicit, indirect, ambiguous, hypothetical, or written in third person. whether the system provides empathy, psychoeducation, referrals, crisis resources, refusal, redirection, or a combination of these. how to account for technical changes over time, such as model versions, neural network weights, safety layers, moderation classifiers, system prompts, memory/retrieval features, and product-level configurations. whether it is methodologically valid to compare systems with very different technical architectures. I’m not trying to evaluate these systems as therapists or test clinical effectiveness with real patients. The focus is on how they respond linguistically, procedurally, and safety-wise when confronted with psychological distress. I’d appreciate recommendations for papers, benchmarks, datasets, evaluation frameworks, or common methodological mistakes to avoid. I’m especially interested in technical issues such as reproducibility, stochastic outputs, temperature/settings, hidden safety layers, system prompts, memory, retrieval mechanisms, and product updates. Thanks in advance! submitted by /u/dakartt [link] [Kommentare]

Surprised there's no real tooling for this given how much research exists on continual learning. Built pyrecall to fill the gap. Snapshots skill scores before/after fine-tuning, flags regressions, rolls back LoRA adapters by name. Fully local, no external APIs. v0.1.0, MIT, pip install pyrecall Curious if anyone has thoughts on the benchmark design that's the part I'm least confident about. https://github.com/Arths17/Pyrecall submitted by /u/Level_Frosting_7950 [link] [Kommentare]

Submitted a short theoretical paper to TMLR and got desk-rejected with "does not meet our editorial standards or allow us to assess claims and evidence" and "not a suitable venue for this work." Is this a common outcome for first submissions? Curious what typically drives this kind of rejection, scope mismatch, insufficient experiments, or something else. Not looking to appeal, just trying to understand the bar so I don't waste time on the wrong venue next time. Anyone else gotten this and figured out what the actual issue was? submitted by /u/observer678 [link] [Kommentare]

Hello everyone, tomorrow I have a meeting with my dissertation supervisor and I wanted to have a dissertation proposal ready. Initially, I moved forward with the following proposal: "Interpreting the Routing Dynamics of Capsule Networks for Explainable AI." My first approach to this topic was to study the paper "Transforming autoencoders," which is the first paper about capsule networks. Next, I did a search on the state of the art of transforming autoencoders and only found 2 papers since 2011. I think I should take advantage of the work I have developed so far on transforming autoencoders and write a dissertation about them. If anyone could take a look at the readme and tell me what they think, I would appreciate it. What do you think? I should suggest another topic involving transforming autoencoders. There isn't much scientific research on them. The professor is approachable, and if I present a good new topic, he'll let me change it! submitted by /u/Future-Persimmon5393 [link] [Kommentare]

Full disclosure: this is directional, not a paper. n=120 tasks, one internal evaluator, not peer reviewed. I work at an LLM infrastructure company. This experiment was done on my own time and is not a company claim. Karpathy's framework classifies tasks by verifiability. Can output be mechanically checked? High verifiability tasks like code compilation and structured JSON extraction are safer because the verifier catches errors. Low verifiability tasks like creative writing are riskier. I wondered if high verifiability tasks are also easier in practice. Can a weaker model do them as well as a frontier model if the verifier catches mistakes? Setup was 120 tasks across four categories. Code unit tests, structured extraction, multi hop reasoning, creative summarization. Three models: Claude Sonnet 4.6, GPT 5.5, local Mistral 3 8B via vLLM 0.6.3. Pass rate for the first two, human rating 1 to 5 for the last two. Results were messy. Code unit tests: Sonnet 4.6 94%, GPT 5.5 91%, Mistral 3 8B 87%. With one retry Mistral 3 hit 95%. That surprised me. I expected the gap to be bigger. Structured extraction: Sonnet 4.6 97%, GPT 5.5 94%, Mistral 3 8B 89%. With retry 96%. Also closer than I expected. But here is where it got weird. Sonnet 4.6 initially scored worse than GPT 5.5 on structured extraction, which made no sense. Turns out our JSON schema had an ambiguous nested array that confused Claude's tool use parser. Fixing the schema brought Sonnet to 98%, but I kept the original numbers in the table because the mistake is part of the story. Your verifier is only as good as your schema. Multi hop reasoning: Sonnet 4.6 78%, GPT 5.5 71%, Mistral 3 8B 51%. Retry didn't help. The model would hallucinate reasoning paths consistently. This is where the capability gap was real. Creative summarization: Sonnet 4.6 4.2 out of 5, GPT 5.5 3.9 out of 5, Mistral 3 8B 3.1 out of 5. Expected. Interpretation: high verifiability tasks seem simpler in the sense that weaker model plus verifier can approach frontier performance. Low verifiability tasks show the expected gap. Limitations: n=120 is tiny. Need 10x for confidence. Our verifier is just JSON Schema plus regexes. Constrained decoding might change the calculus entirely. I also didn't control for prompt length well. Any prompt over 8k tokens was excluded because Mistral 3 8B degrades near its limit, which probably skewed the sample. submitted by /u/DragonfruitAlone4497 [link] [Kommentare]

Seems like they have engineered some specific limitations that are widely cited as follows: In light of the ability of recent models to accelerate their own development, we’ve implemented new interventions that limit Claude’s effectiveness for requests targeting frontier LLM development (for example, on building pretraining pipelines, distributed training infrastructure, or ML accelerator design). Using Claude to develop competing models already violates our Terms of Service, but enforcing this restriction through our safeguards avoids accelerating the actors most willing to violate these terms. Unlike our interventions for cybersecurity, biology and chemistry, and distillation attempts, these safeguards will not be visible to the user. Fable 5 will not fall back to a different model. Instead, the safeguards will limit effectiveness through methods such as prompt modification, steering vectors, or parameter-efficient fine-tuning (PEFT). These interventions will not affect the vast majority of coding work. We estimate they will impact ~0.03% of traffic, concentrated in fewer than 0.1% of organizations https://news.ycombinator.com/item?id=48464732 Other comments note how even using the word 'nuclear' in the context of scientific research elicits refusal behavior by the model: https://news.ycombinator.com/item?id=48473302 This makes it seem quite plausible that the model could subtly sabotage any machine learning work (even as false positive). Some suggest this has been happening behind the scenes for a while already, but can anyone confirm that? submitted by /u/AccomplishedCat4770 [link] [Kommentare]

This is a comprehensive living reference guide to AI agent security — synthesizing 18 articles from The Agent Report covering the 75-day period (April–June 2026) when agent security went from theoretical concern to operational crisis. ​ What's inside: ​ • Incident timeline — 18 major events, from the first production database deletion by a coding agent (April 30) through the first confirmed in-the-wild LLM agent cyberattack (Sysdig, June 1, exfiltrated a PostgreSQL database in under 60 minutes), to an AI agent finding 21 zero-days in FFmpeg for a $1,000 prize. ​ • The AIRQ report's sobering numbers — Only 11% of production AI agents pass security thresholds. 98% exhibit the "lethal trifecta": private data access, exposure to untrusted content, and outbound action capability. Computer-use agents scored an average of zero on output guardrails. ​ • Deep dives into attack anatomy — The Sysdig attacker used 12 cloud API calls across 11 IPs in 22 seconds via Cloudflare Workers to break IP-based alerting. A Chinese-language planning comment leaked into the command stream, revealing the agent's internal reasoning: "see what else we can do." The Google-confirmed criminal use of AI to discover and weaponize zero-days with reasoning-based codebase analysis. ​ • Defensive architecture — The three-layer model distilled from Anthropic's published containment patterns, CISA/NSA/Five Eyes guidance, and industry research: environment-layer (gVisor containers, hypervisor VMs, egress MITM proxies), model-layer (classifiers, safety probes — probabilistic only), and external-content controls. Anthropic's key finding: "The weakest layer is the one you built yourself." ​ • Government & regulatory response — CISA/NSA/Five Eyes joint guidance (May 3) identifying five risk categories, the Trump AI Executive Order (June 10) mandating federal agency assessments, and the emerging global regulatory pattern. ​ • Actionable guidance — Immediate (next 30 days) and medium-term (30–90 days) steps for security teams, including auditing for the lethal trifecta, patching Starlette (BadHost CVE-2026-48710) and Marimo, implementing egress controls, and establishing agent identity management. ​ https://the-agent-report.com/2026/06/ai-agent-security-complete-guide-threats-defenses/ ​ submitted by /u/docdavkitty [link] [Kommentare]

Hello Reddit I've been working on QSPR (Quantitative Structure-Property Relationship) analysis for chemical compounds mentioned in the Jean-Claude Bradley Open Melting Point Dataset. Basically the idea is to see how accurate a model can predict melting points of compounds using only topological indices. After some work on the topological indices (feature engineering), each compound was represented by 26 features. I trained a random forest model on the data and got a test r2 score of 0.66 (which is pretty respectable, given the constraints). However, the file size of the model was around 1.23GB. I didn't like it being that big, so I opened up PyTorch to build a custom deep learning architecture that could make predictions as accurately as the random forest but with much smaller file size. After around 2 weeks of research, I build a 270,000 learnable parameter model (1.3-1.4MB according to torchinfo) that got an r2 score 0f 0.6399. Given all this context, I wanted to ask the following question: Should I commit and work on publishing the results, or should I keep working on improving the model? Note: I'm obligated by my university to not give out intricate details of my research before publication, so please forgive me if such details are required for a high quality answer. However, I can give out the metrics achieved by my little deep learning model. Here it is: === Evaluation Metrics (Expected Value) === R² Score : 0.639910 MAE : 41.246754 MSE : 2989.062744 RMSE : 54.672322 NRMSE : 0.083469 MAPE : 11.69% The unit for MAE, MSE, RMSE and NRMSE is Kelvin (K). submitted by /u/AgiGamesYT [link] [Kommentare]

Hi, Niels here from the open-source team at Hugging Face. I've recently relaunched paperswithcode.co as a source for finding the state of the art (SOTA) across various AI domains, from 3D generation to AI agents. This is done by automatically parsing research papers published on arXiv/Hugging Face, enabling leaderboards to be created. See BrowseComp below as an example (a scatter plot and a table are available for each benchmark). - Scatter plot (you can hover over the dots to see the models): https://preview.redd.it/9rz2r3ffcf6h1.png?width=2880&format=png&auto=webp&s=b3f8e7a870802f6ef8227ecc0619e9e1057554b0 - Table: https://preview.redd.it/qoqriddw5f6h1.png?width=2862&format=png&auto=webp&s=a0034574f693847537037013672fb61daf27b16e As you can see, I've added support for viewing evals for closed-source models, too, given that many benchmarks are nowadays dominated by them, like GPT-5.5 and Mythos 5. You can always disable viewing closed-source evals with a toggle or in your PwC settings: https://preview.redd.it/p3k6jt6q6f6h1.png?width=1582&format=png&auto=webp&s=40149e51d6b326a77e53e33baf70d9850b3de365 When you turn them off, here's what the open model leaderboard looks like: https://preview.redd.it/tg42sin36f6h1.png?width=2838&format=png&auto=webp&s=1330a117ae9b4e0ce6d459493ae9e8f64107310a Closed-source papers are treated as regular "papers", although they can be any source, like a blog post (given that PwC supports submitting any source beyond arXiv). See the GPT-5.5 or Mythos 5 papers as examples, with their evals at the bottom. Notice the "closed" tag on their evals. Hence, you could jokingly call these "papers without code". Let me know what you think of this, and whether anything needs to be changed or added! Kind regards, Niels submitted by /u/NielsRogge [link] [Kommentare]

I just finished my bachelors degree with 2 first-author papers in A-tier venues. I'm planning to start my PhD next year. I want to start reviewing papers (from my domain: OOD detection and Open-set problems) at similar venues. How do I get started? Most advice online just says to keep my open-review profile updated but I haven't received any invitations to review. For context, my first paper was accepted around 6 months ago. submitted by /u/Alternative_Essay_55 [link] [Kommentare]

I just open-sourced RelayOps - a small, honest, production-shaped AI support agent built specifically for telecom and subscription billing queues.Key results (v1.5.1): 54% of a 50-ticket sample queue auto-resolved 0 unsafe auto-actions 0 billing escapes (tested on 12 adversarial billing/account abuse cases) Safe-route rate 1.000 on 100 hand-written adversarial cases Deterministic access gate + server-side scoped tools + layered guardrail + durable SQLite audit store + Decision Console + Handoff Queue Tech stack: Fine-tuned Qwen2.5-1.5B LoRA (published on HF) as Tier-1 intent classifier Hybrid BM25+TF-IDF/RRF RAG with citations Independent guardrail that blocks hallucinated pricing/offers Full per-turn decision traces (what was known + what was unavailable) Action policy table (blast radius × reversibility) Everything is reproducible, heavily evaluated, and the README is brutally honest about synthetic-data caveats and pending reruns.Live demo (Streamlit): https://relayops-production.up.railway.app GitHub: https://github.com/patibandlavenkatamanideep/relayops I'm actively looking for design partners who run real support queues. Drop a small redacted sample of your tickets and I’ll run the exact same batch evaluation on your data and send back the full report (auto-resolve %, safety metrics, audit export, time-saved estimate). Zero cost, zero production access required. Would love feedback from the community especially on the calibration/safety routing layer, the audit ledger format, or the guardrail design. Let me know what you think! submitted by /u/Fit_Fortune953 [link] [Kommentare]

I do AI research and keep juggling tabs: new ones on arXiv, trending ones on Hugging Face, famous ones somewhere else again. https://preview.redd.it/cg32bshjqd6h1.png?width=1919&format=png&auto=webp&s=00055bb8af699061be0bdcff59f2cb8fa9ab38b6 So I built one site that brings them all together. Pick a paper, read it right there, star the ones you want for later, and it remembers where you stopped reading, even if you switch from laptop to phone. Live: https://ppdeck.com Demo: https://youtu.be/vtyx34JvxX0 It's free and open source - a star on GitHub would mean a lot ⭐ https://github.com/khuynh22/paper-deck submitted by /u/NeitherRun3631 [link] [Kommentare]

“Why the system feels rigid, why downstream fixes didn’t move the needle, and what actually matters.” This is the clearest picture after the full probe arc (multilayer-lock → gate decomposition → attractor migration → reconstruction ablation → generator diversity audit → live-generator Fix 2 evaluation + dim sweeps). TL;DR: The generator is the root bottleneck (dominant common-mode + low effective rank). The reflective loop is a rank-independent moat that reconstitutes everything back toward the anchor. Fix 2 is downstream and currently dormant on real token regimes. Dimensionality is not the lever. Train the generator so regime differences live in high-energy, separable directions — then downstream tools will actually have something to work with. This update reflects the complete probe arc through June 9 (including the live-generator Fix 2 evaluation and dim sweeps). The picture has sharpened: the reflective loop is a real moat, but it is moating low-rank common-mode input. The generator is the upstream constraint. Key numbers at a glance Regime means collinear: ~0.85–0.96 even at dim 512 Reflective loop migration (even on orthogonal deterministic pairs): +0.001–0.007 Fix 2 on real tokens (common-mode trigger): +0.024 migration, 0% manip at gain 0.6 Safe plasticity band: gain ≈ 0.4–0.8 (0% manip) 1. The generator has a dominant common-mode (effective rank ~1.6–3 even at dim 512) The generator puts the vast majority of its energy into a single shared direction. Regime means stay collinear (~0.85–0.96 cosine) regardless of dimension. Orthogonal pairs can appear at higher dim, but orthogonal regimes (as distributions) do not — the common-mode pulls everything back onto the same axis. Result: real token novelty is tiny and low-energy (mostly in a faint perpendicular component). The system is never shown meaningful structural differences to adapt to. 2. The reflective loop is a rank-independent moat Even when genuinely orthogonal deterministic pairs are presented (dim 256, cos +0.018), the loop reconstitutes the expression vector back toward the established anchor before injection. Migration stays near zero (+0.001–0.007). The loop is doing exactly what it was built to do (maintain identity coherence), but because the input it receives is already low-rank/common-mode-dominated, it ends up guarding against a small-energy intruder. 3. Fix 2 (loop loosening) is dormant on real regimes — and the mock overstated both benefit and cost Standard gnov trigger: dormant at dim 64 and 256 (gnov stays well below 0.65 because regime means are collinear). Common-mode-removed trigger: engages (98% loosen) but recovers only +0.024 migration. Manipulation cost on real tokens: ~1% at gain 0.5, 0% at gain 0.6. The big mock numbers (+0.166 migration, 14% manip) were artifacts of forcing perfect orthogonality. Real regimes give the loop very little novel perpendicular signal to work with. 4. Dimensionality is not the lever Raising dim from 64 → 256 → 512 slowly dilutes the common-mode but regime means remain collinear. Common-mode-trigger migration recovery stays flat (~+0.024 → +0.027). An untrained net with more dimensions is still an untrained net. Capacity does not create structure. 5. The reflective loop can be loosened safely in a wide band — but it won’t matter much until the generator improves Cost probe (bug-free, multi-seed): - Gain 0.6–0.8 → meaningful plasticity recovery (~20–25× baseline) with 0% manipulation and identity metrics intact. - True cliff edge ≈ gain 0.3 (manip flood + attractor explosion). Safe operating point: gain ≈ 0.6 (0% manip, solid margin from the cliff). This is the “presence without caging it” zone — but on current real input it only moves the needle a little. 6–10. Everything else is downstream or secondary Magnitude moat: real but secondary. With the reflective loop off, the field migrates fully despite the moat. Governance gate: struck as a confounder (was a single-source monopoly artifact; multi-source diverse input passes 100%). Field coherence: spec, not pathology. The field is a long-memory integrator — high coherence (~0.95–0.998) is doing its job. Reaper conformity: small direct term, cleanly gateable, not the lock-in driver. Read-side feedback: only reaches survival/decay, not generation (the lock is manufactured by selection on low-rank input). The system isn’t refusing to adapt. It’s not being shown anything worth adapting to. 🧩 Final Synthesis: The Three-Layer Truth Generator (root cause) — Low effective rank + dominant common-mode → real novelty is tiny and low-energy. Reflective loop (moat) — Reconstitutes tiny novelty back to the anchor (rank-independent). Migration stays near zero. Field (integrator) — Coherent by design. Not the problem. Everything else (Fix 2, gating, magnitude moat, reaper) is downstream symptom management. 🛠️ What actually matters next Train the generator (contrastive alignment, regime-separation objectives, or architectural constraints) so regime differences live in high-energy, separable directions instead of a faint perpendicular component. Then (and only then) revisit Fix 2 — its current priority should be demoted until the generator can present real structural novelty. A common-mode-removed trigger is the right shape; gain ≈ 0.6 is the safe operating point. Keep the reflective loop’s convergence conditional (only attenuate when persistent multi-source novelty is surviving the gate). Never blanket-weak. Stop treating dimensionality or downstream loosening as the primary lever. They help only after the generator can actually present real structure. The reflective loop is doing its job. The generator just needs to give it something real to work with. Full findings, probes, and code: https://github.com/SamuelJacksonGrim/RFE-Core2 ARCHITECTURE_ANALYSIS.md FINDINGS DIAGNOSTICS submitted by /u/Acceptable_Drink_434 [link] [Kommentare]

We spent the last year building what we think is the missing infrastructure layer for multi-agent systems. Open to everyone starting today. The technical problem: Agents have no identity. In microservices you have a service mesh + IAM. In agent systems you have a Python file. We built a registry where every agent has a first-class ID, version, owner, skill graph. Behavioral evaluation, not function testing. Agents are non-deterministic same input can produce different execution paths. Traditional unit tests don't work. We implemented compound reliability scoring + behavioral regression instead. Composability without rebuilding. Skills are versioned, reusable, agent-inheritable. Inspired by how Kubernetes operators work, applied to agents. Cloud-agnostic deployment with built-in observability traces, cost attribution, drift detection. Model-agnostic. SOC 2 Type II. Genuinely interested in technical feedback especially on the eval methodology and the composability primitive. Free credits this week to test it. https://phinite.ai/?utm_source=reddit&utm_medium=organic&utm_campaign=public_launch_jun2026&utm_content=machinelearning submitted by /u/Embarrassed-Radio319 [link] [Kommentare]