@MrStickman

my preprint is on SSRN and i feel somewhat shy to share it here... but the PKU lab's paper that cited mine got accepted by ICML 2026: https://arxiv.org/html/2602.06358v2 submitted by /u/max6296 [link] [Kommentare]

Hey everyone, I am planning out a new open-source infrastructure project and want to get some brutal feedback on the architecture and use-case validity from people running high volume LLM workloads in production. The Problem: Python-based proxies/gateways introduce too much latency overhead for real-time streaming agent steps or fast UI completions. Additionally, centralized semantic caching still suffers from cross-region network latency (e.g., London to us-east-1), and enterprise API costs remain a massive bottleneck for repetitive/predictable user queries (like customer support or structured data extraction). The Proposed Architecture: Instead of a heavy centralized gateway, the goal is to build a lightweight, zero-dependency semantic cache running directly at the CDN Edge using WebAssembly (WASM) compiled from Rust. The flow looks like this: Inbound Prompt: Hits the edge node closest to the user (e.g., Cloudflare Workers / Fastly Compute). Edge Embedding: The Rust/WASM module intercepts the raw text prompt and instantly generates a vector using an edge-native lightweight model (e.g., bge-small-en-v1.5). Similarity Index Check: It performs a fast cosine similarity check against an edge vector database (like Cloudflare Vectorize) to find the nearest semantic neighbor. Cache Hit: If similarity >= threshold (e.g., 0.88), it pulls the full generated response text from an edge KV store and returns it in ~5ms. The main LLM provider is never billed or touched. Cache Miss: It proxies the streaming request to OpenAI/Anthropic/vLLM, streams it back to the client, and asynchronously updates the edge vector index and KV store. Why Rust/WASM? To achieve sub-millisecond execution overhead on the proxy itself, avoid garbage collection pauses, and maintain a tiny memory footprint suitable for edge runtime constraints where traditional databases or Python scripts cannot run. My Questions for the Community: For those running LLMs in production (especially customer support, internal RAG, or autonomous agents), what is your realistic semantic cache hit rate? Is the power law of repetitive queries high enough in your domains to justify this? What are the biggest footguns with semantic caching at the edge? (e.g., Cache invalidation strategies, handling system prompt updates, or drift in embedding models). Would you actually use a drop-in open-source template/CLI that lets you spin this up on your own edge account, or do you prefer centralized API gateways? submitted by /u/Real-Huckleberry-934 [link] [Kommentare]

ACL ARR May 2026 reviews are due on July 2. I do not see any reviewer assignement as of today. Will the review period be just 2 weeks in that case? Anyone got papers assigned for reviewing? submitted by /u/Impossible-Garden612 [link] [Kommentare]

From Wired: “We’re changing Fable 5’s safeguards for frontier LLM development to make them visible.” Anthropic said in a statement to WIRED. “We made the wrong tradeoff and we apologize for not getting the balance right.” Anthropic now says it’s changing course, and that Claude Fable 5’s safeguards for AI development will be visible to users. If the company suspects a user is trying to use Claude to build a highly capable AI it will alert them that it’s either refusing the request, or rerouting the user to a less capable model. Full article: https://www.wired.com/story/anthropic-responds-to-backlash-on-claudes-secret-sabotage-on-ai-research/ submitted by /u/goldcakes [link] [Kommentare]

This is a comprehensive living reference guide to AI agent security — synthesizing 18 articles from The Agent Report covering the 75-day period (April–June 2026) when agent security went from theoretical concern to operational crisis. ​ What's inside: ​ • Incident timeline — 18 major events, from the first production database deletion by a coding agent (April 30) through the first confirmed in-the-wild LLM agent cyberattack (Sysdig, June 1, exfiltrated a PostgreSQL database in under 60 minutes), to an AI agent finding 21 zero-days in FFmpeg for a $1,000 prize. ​ • The AIRQ report's sobering numbers — Only 11% of production AI agents pass security thresholds. 98% exhibit the "lethal trifecta": private data access, exposure to untrusted content, and outbound action capability. Computer-use agents scored an average of zero on output guardrails. ​ • Deep dives into attack anatomy — The Sysdig attacker used 12 cloud API calls across 11 IPs in 22 seconds via Cloudflare Workers to break IP-based alerting. A Chinese-language planning comment leaked into the command stream, revealing the agent's internal reasoning: "see what else we can do." The Google-confirmed criminal use of AI to discover and weaponize zero-days with reasoning-based codebase analysis. ​ • Defensive architecture — The three-layer model distilled from Anthropic's published containment patterns, CISA/NSA/Five Eyes guidance, and industry research: environment-layer (gVisor containers, hypervisor VMs, egress MITM proxies), model-layer (classifiers, safety probes — probabilistic only), and external-content controls. Anthropic's key finding: "The weakest layer is the one you built yourself." ​ • Government & regulatory response — CISA/NSA/Five Eyes joint guidance (May 3) identifying five risk categories, the Trump AI Executive Order (June 10) mandating federal agency assessments, and the emerging global regulatory pattern. ​ • Actionable guidance — Immediate (next 30 days) and medium-term (30–90 days) steps for security teams, including auditing for the lethal trifecta, patching Starlette (BadHost CVE-2026-48710) and Marimo, implementing egress controls, and establishing agent identity management. ​ https://the-agent-report.com/2026/06/ai-agent-security-complete-guide-threats-defenses/ ​ submitted by /u/docdavkitty [link] [Kommentare]

From TF's website: Quantization aware training emulates inference-time quantization, creating a model that downstream tools will use to produce actually quantized models. So is it designed to work with a very specific quantization method (for Gemma-4, presumably, Google's own)? Or would it make sense to use alternative quantization methods? According to the benchmarks unsloth released, its (alternative) quantizations of Gemma-4-QAT are closer to the QAT fine-tunes, but is it a good thing, or does it defeat the purpose of QAT? submitted by /u/we_are_mammals [link] [Kommentare]