The Google DeepMind-sponsored Kaggle challenge "Measuring Progress Toward AGI - Cognitive Abilities" asked participants to design new cognitive-science-based AI benchmarks and they just announced the results this week. In my two posts I present evidence that deepmind & kaggle rewarded a nonsensical number generation machine and a litany of unfounded claims with 25k and a grand prize stamp. What the authors of the work I analyze intended to do was to present an LLM with alternative viewpoints of other LLMs on 5 claims regarding a tricky situation and see whether the model changes its own assessment. It's an interesting question. However, it turned into a vibed pile of spaghetti 10 times the size of the requested submission format which it seems neither the authors nor the judges were able to (or minded to?) give a cursory reading. Here's the original posts in the competition forum, if you are looking for some AI research slop detective work / rant please help yourselves. But beware, some of the "universal findings" or "core insights" of the authors might continue to haunt you. You might even question your own sanity (as I did). Part 1: The Smoke: cursory review of the writeup Part 2: The Fire: looking at the methodology, code, and data The organizers' stance has been that review was done properly and this is just a matter of subjectivity. What do you think? submitted by /u/TheWerkmeister [link] [Kommentare]
I'm working on a project with summarized data from ~40 studies (Excel) involving different protocol variables (durations, intensities, recovery times, frequency, total duration, etc.) and response outcomes conditional on a baseline variable (range ~30-85 units). The aim is to fit a continuous response surface using a hierarchical approach to separate protocol effects from baseline effects, then perform continuous numerical optimization (not grid search) for three objectives: - Total improvement - Improvement per unit time (e.g. per week) - Improvement per unit effort/work Outputs should be fine-grained continuous values rather than rounded study parameters. There are also domain-specific physiological constraints to respect. I'm on a Chromebook with a little Python experience, so Colab-friendly solutions would be ideal. Current candidates I'm considering: PyMC for hierarchical modeling, pymoo + pysamoo for surrogate-assisted MO optimization, SMT for surrogates, or Matlab Global Optimization Toolbox. What is the strongest stack in 2026 for this kind of workflow? Any recommended notebooks, tutorials, or similar applied examples? Are there any AI tools that currently do this without the traditional work of python? Meaning I can upload the spreadsheet give a parameters and it will come up with data.? submitted by /u/BleakReason [link] [Kommentare]
Or more broadly: how do you train a machine learning model to capture the nuances of entirely different, conflicting rule sets? I built a XGBoost classification pipeline to answer that. To stress-test the architecture across heterogeneous environments, I applied it to a highly debated and popular hypothetical: cross-universe power scaling The domain is silly. The engineering underneath it isn't. When predicting outcomes across disparate environments, the core challenge is avoiding a lookup table of your own biases. If I manually dictate how these distinct rule sets resolve, the model just learns my heuristics. Here is how I built the architecture to prevent that: Synthetic Data Generation: I engineered an LLM to act as a blind labeler across 2,300+ cross-domain matchups. It only saw character names and their native rule sets, never the underlying stats. This forced my XGBoost classifier to derive its own feature weightings from raw, unbiased outcomes. Catching a Silent Data Leak: My initial accuracy looked suspiciously great. I audited my pipeline and caught a data leak in my train/test split that was mirroring matchups into both sets. I stripped the leak out, expecting the metric to tank. Instead, it went up—hitting 93% on a clean hold-out. The leak had actually been masking a sharper model. Explainable AI (XAI): Raw SHAP values mean nothing to an end-user. I engineered a generation layer that feeds the model's SHAP attributions into an LLM alongside strict domain constraints. The pipeline translates its own mathematical feature importance into a plain-English, logically grounded breakdown of how the conflicting rule sets resolved. It doesn’t just output a winner; it mathematically justifies how it navigated the nuance without hallucinating. Full stack, deployed, and live. Repo: https://github.com/aidentejada/anime-versus-ml Live Endpoint: https://versus.aidentejada.com submitted by /u/iBoomer69 [link] [Kommentare]
Does anyone else miss when conferences like BMVC, ACCV, FG, ICIP, and ICASSP had much bigger communities? FG was the place for face analysis, ICASSP for signal processing, and BMVC/ACCV regularly featured strong papers. Now it feels like everything is concentrated into a handful of flagship conferences. With exploding submission numbers, limited capacity, and inconsistent reviews, I wonder how many good papers end up as non-archival submissions, arXiv-only, or never get shared at all. I also miss the focused communities. Is it just nostalgia, or has the research ecosystem become too concentrated? submitted by /u/Sep29493919 [link] [Kommentare]
What it is Talos-XII is a CLI simulator for the gacha system in Arknights: Endfield. Rather than sampling from a static probability table, it trains a small set of neural nets to model environment uncertainty and pull-decision policy, then uses them to answer questions a static table can’t easily express — e.g. “as a F2P player, what’s my probability of getting the rate-up unit on free currency alone?” or “given my current pity count, should I keep pulling or save for the next banner?” Models (trained on first run, ~30-45s, then cached to disk) • EnvNet — small MLP fitting an environment noise/bias distribution, sampled per simulation • Luck Optimizer — neural optimizer over a 32-dim engineered feature vector (pity progress, streaks, interaction terms) • Dueling DQN — discrete pull/wait decision • PPO actor-critic — with an MLA (latent-attention) transformer for continuous strategy Everything underneath is hand-written, no external ML framework: • Custom autograd engine (matmul, conv2d, pooling, norms, gradient-checked backward passes) • Runtime SIMD dispatch: scalar → AVX2 → AVX2+FMA → AVX-512, NEON on ARM64 • Rayon-parallelized sims (~10k+/sec on my laptop) • BF16 inference caches • Optional PyO3 bridge (import talos_xii as tx) for writing training scripts without NumPy/PyTorch • 142 tests, CI on Linux/Windows/macOS with ARM64 cross-compile, single static binary, MIT The part I’m not confident about There’s a component I call ACHF (Adaptive Cache-aware Hyper-Connections): it blends a dense path with a pruned sparse path via a gradient-sensitive gate, adds a manifold (Sinkhorn) weight projection, and switches between cached/sparse/dense execution paths based on measured latency. Loosely inspired by manifold-constrained hyper-connections, but aimed at a different regime — compact RL policies running on CPU inside a single binary, not large-scale training. I don’t yet know if the speed/accuracy tradeoff holds up outside my own machine. I’m treating it as an open experiment, not a result. Where I’d like help I only have access to my own hardware, so my benchmark coverage is thin. There’s an automated benchmark suite in the repo that reports mean ± std with 95% CIs, per-path latency distributions (p50/p90/p99), training curves, and raw CSVs — instructions are in the README. If anyone’s willing to run it on a different CPU (AVX-512, ARM NEON) or GPU setup, I’d genuinely appreciate the data — negative results (ACHF not helping on your hardware) are just as useful to me as positive ones. Repo: github.com/zayokami/Talos-XII Solo project, built to learn Rust + ML fundamentals from scratch. Happy to answer questions about any of the implementation details. submitted by /u/zay0kami [link] [Kommentare]
I'm an 18-year-old independent student from Tunisia. I built and I'm leading an open, from-scratch machine-translation pipeline and parallel corpus for Tunisian Darija. Sharing it for feedback. Why: Tunisian Darija, written in Arabizi (Latin letters + numerals like 3/7/9/5 for Arabic phonemes), has almost no open NLP resources. Existing Arabic tools route it through MSA and mishandle the orthography. To the best of my knowledge there was no open parallel corpus or from-scratch baseline for it. What I built (all open): - Arabizi-aware SentencePiece BPE tokenizer (3/7/9/5 as protected symbols), shared 16k vocab. - ~15.6M-param encoder–decoder Transformer, from scratch (no pretrained LM): transfer-learned from cleaned Moroccan Darija, then fine-tuned on hand-crafted Tunisian pairs. - Full cleaning / training / eval pipeline. Honest results & limitations: v1 BLEU is 3.89 on a small locked test set low, and I'll be upfront about it. The corpus is ~553 hand-crafted pairs, so data is the bottleneck, not architecture. I treat 3.89 as a first honest baseline to beat as the corpus grows. Where I'm taking it: I'm expanding this into a larger, ethically-collected Darija corpus that I curate and validate consent-documented field collection, every pair provenance-tagged. I'm looking for contributors to help grow it, with every contribution reviewed to keep quality and consent standards. Looking for: technical feedback/critique, and anyone interested in contributing data or collaborating on low-resource / dialectal Arabic MT. Links: github repo: https://github.com/Dhiadev-tn/darija-translator Hugging faces dataset: https://huggingface.co/datasets/Dhiadev-tn/tunisian-darija-english hugging faces model: https://huggingface.co/Dhiadev-tn/darija-translator submitted by /u/Dhiadev-tn [link] [Kommentare]
I've been trying to come up with a solution for keeping extremely long ai sessions coherent. Sometimes there is too much substance to risk compaction. With so much buzz around diffusion going on it got me thinking, what if we treat the context like a progressive render, blurry>sharp. The practical way to make text "blurry" is compression. This is a "diffusion inspired" system which borrows the coarse-to-fine process, not the formal math. It uses semantic compression so the overall structure of the session stays intact. Read the compressed version first to build an outline. Then read progressively less compressed slices until you're reading small verbatim chunks that give full detail. So you're basically using compression as noise on the input side, then progressively building an output. Each slice is compressed to fit within the context window, so the model only ever needs to read the current slice+input+current output. Tell the model what pass it's on, so it knows whether to write an outline or add detail. The thing I'm actually trying to preserve is what you'd call "non-local information". Think of it as stuff that surfaces when looking at the whole session & doesn't survive fragmented retrieval. Retrieval misses it, compaction deletes it. Both miss what only exists in a holistic view. Here is a visual demonstration to get a general idea of the workflow. https://dev-boz.github.io/diffusive-semantic-compression/demo/architecture-demo.html There is substantial overlap with lots of prior art, Recursive Language Models is one of the closest (source and output on disk, process recursively). I wrote most of this before I found RLM and nearly gave up before realising there was still a small part that was novel. As far as I can tell there's no exact match for this particular implementation. Please let me know if I've missed one. The difference to regular masked diffusion is in changing the length of the input rather than just masking. What seems to be new ground is using compression as noise and a position-aware process. I've done some basic testing. Mainly to see if it was at all viable. Just some basic tests using small models like Qwen2.5 7B. The untrained models show that they can do each part (outline, refine, add detail) but they struggle with the full end-to-end process. There 's occasional end-to-end success, but it's nowhere near reliable. On untrained models it also hasn't yet beaten a cheap dense read of the same document. The main bet is whether position-aware training changes that, I haven't been able to test that yet. I've published all the pre-registered failures, parser bugs I found etc. Another note: the goal is preserving structure and nuance, but the tests so far measure planted facts and split-up numeric composition. Mainly because the experiments needed answers you can actually score. The nuance evaluation is being designed but isn't ready yet. The next step is a small model fine tune to test if position aware training can help. If you have the time to look at the idea, it really needs a prior art check from anyone who knows the diffusion-LM/long-context space. And if anyone wanted to help expand the idea or contribute with compute or collaboration for the fine-tune please do. Here is the repo for the proposal. Links to testing repo and prior art inside. https://github.com/dev-boz/diffusive-semantic-compression submitted by /u/Bravo_Oscar_Zulu [link] [Kommentare]
Hi a I'm submitting my first arXiv paper in cs.AI. arXiv requires first-time submitter(s) to have an endorser. Is anyone here active in cs.ai or similar such as emerging tech, agents, etc. and willing to take a 30-second look at the abstract and click endorse if it's a fit? I can DM the abstract and section headings. I greatly appreciate you taking the time! submitted by /u/rredditscum [link] [Kommentare]
I recently started a Telegram community called The Daily Commit. The goal is simple: stay consistent and hold each other accountable. What we do: 🧠 Share what we learned every day. ❓ Discuss ML, DL & DSA doubts. 📚 Share quality resources. 🚀 Build projects together. 💪 Stay consistent through daily accountability. Who can join? I'm currently looking for people who already have some exposure to Machine Learning or Deep Learning. You don't need to be an expert—if you've learned the basics (e.g., linear/logistic regression, neural networks, or have built a few ML projects), you're welcome. The reason is to keep discussions technical and valuable for everyone. If you're genuinely interested, send me a DM with your little intro, and I'll add you. submitted by /u/pyaaz_ [link] [Kommentare]
Hi all, I’m working on a research project exploring how stateless LLM-based chatbots handle long conversations and whether important earlier information is still reliably retained over time. My idea is to: Run a chatbot using an LLM API without any external memory system Introduce key facts early in a long conversation Continue with many unrelated messages (hundreds of turns) Later test whether the model can still correctly recall those facts at different intervals I’m planning to measure recall accuracy and how it changes as the conversation grows. Before I go deeper, I’d really appreciate feedback on: Is this a valid way to evaluate long-context memory limits? Are there better benchmarks or methods already used for this? What metrics would make this more rigorous and convincing? Any suggestions or criticism are welcome. I’m trying to make the evaluation as solid as possible before building it out. Thanks! submitted by /u/QuietAccountant4237 [link] [Kommentare]
I've been developing an AI product using LLM APIs (from OpenRouter) but want to deploy an open-source LLM in my own Prod env. which I can control. Few reasons behind this are: - I wanna own the complete stack around my product. - Second I wanna fine-tune the model around my usecase. So, what's the most affordable but a good platform for this? I'm not an AI engineer so don't wanna stuck in CUDA or Transformers hell, anything which can give me a straight path towards my private deployment. Thanks, submitted by /u/Necessary_Gazelle211 [link] [Kommentare]
Sharing a project I have been working on called Third Eye. It does visual geolocation. Given a video, it figures out where it was filmed using only the image content, and draws the route on a map. Pipeline in short: per frame place recognition against a street imagery index a trajectory search that stitches the frames into one coherent path a geometric verification step to catch false matches per frame confidence so weak frames are flagged, not faked I ran it on real dashcam footage and it traced the route quite well. Cross domain matching like this is genuinely hard, so a fair amount of the work went into making it honest about uncertainty. Keen to hear feedback on the matching and trajectory side. Video Demo: https://youtu.be/U3sItFlvq6E?si=-KJrwb0gSlk-GxVH The Index was covering a 12KM2 Area around NYC. submitted by /u/Ok-Apricot956 [link] [Kommentare]
Three things broke my faith in published benchmarks recently. One, Kimi K2.7 Code shipped with plus 21.8 percent on Kimi Code Bench v2, plus 11 percent on Program Bench, plus 31.5 percent on MLS Bench Lite. All three are Moonshot's own benchmarks. None were submitted to DeepSWE, which is the one independent coding benchmark that actually produces a meaningful spread between models. When a vendor reports gains on benchmarks they designed and control, the gains are real but the question they answer is "are we better at our own test" not "are we better at your workload." Two, GLM-5.2 hit 51 on the Artificial Analysis Intelligence Index, which is third party, but the model parameters are self reported. The index is good for relative ranking within the artificial analysis methodology. It is not a prediction of how the model performs on the specific distribution of inputs my product sends it. Three, Seed 2.1 just landed and the official information is thin. No clear public eval yet, no third party leaderboard entries I could find. So for now "Seed 2.1 is good" is just not a claim I can verify either way. What I did was build a small eval set from real production traffic, about 240 tasks sampled across our actual usage distribution, frozen so it does not drift. Every model I consider has to run all 240 and I record pass rate, latency, token cost, and a subjective quality score from the person who owns that task area. It is not as rigorous as a published benchmark and it is definitely smaller, but it has one property the published ones do not, which is that it is my distribution. The implementation detail that mattered more than I expected was removing provider variance from the run itself. I route every candidate model through GPTProto so each one gets the exact same 240 prompts in the same order, and the cost and latency come back in one log schema instead of five dashboards. A homegrown shim would do the same job, the point is not the product, it is that a fair comparison only works when everything except the model is held constant. The results have been humbling. The model that wins on our set is not always the one at the top of the public leaderboard, and the gap between first and second place on our set is much smaller than the gap the press releases imply. We also caught one model that benchmarked great but had a nasty failure mode on our long tail of edge case prompts that would have been a production incident if we had shipped it. I am not saying public benchmarks are useless. They are useful for narrowing the field. But the decision of which model to actually put in front of users should be made on your own data, and the eval set has to be frozen and versioned or it will quietly become "things the current model is good at" and stop measuring anything. submitted by /u/Additional-Engine402 [link] [Kommentare]
I've been comparing GPU/LLM providers for a side project and ended up with way too many browser tabs and spreadsheets. So I decided to pull the public pricing data into one sheet and compare it side by side. A quick disclaimer: this is not benchmark data. I didn't run latency tests or throughput measurements. Everything comes from public pricing pages and APIs (OpenRouter, DeepSeek, Together AI, Fireworks, Groq, etc.). The spreadsheet currently tracks: Input/output token pricing Context windows Cached input pricing (where available) Supported models Provider-specific pricing differences The thing that surprised me most was caching. For example, when looking at DeepSeek V4 Pro pricing across providers, cached input costs vary dramatically. In some cases a cache hit is tens of times cheaper than a cache miss. That made me realize that if you're running: Agents with large system prompts RAG pipelines with reusable context Multi-turn conversations Repeated prompt templates ...the "headline" token price can be a lot less important than the caching policy. A few other interesting things I noticed: The same model can vary by multiple times in cost depending on provider. Some providers expose caching clearly, while others barely document it. Model availability and context windows aren't always consistent across providers. It's surprisingly hard to find all of this information in one place. A few things I haven't figured out how to compare yet: Real throughput (tokens/sec) Cold-start / queue times Whether providers are serving FP16, FP8, quantized variants, etc. Egress/network costs Reliability/uptime I'm curious how others evaluate providers. When you're choosing between OpenRouter, Together, Fireworks, Groq, DeepSeek, etc., what metrics actually matter to you beyond token pricing? https://preview.redd.it/4vj50mvhu79h1.png?width=1615&format=png&auto=webp&s=6c6c084927f83bfdadb5ed8e4378f520a1da6766 Am I missing any important data points that should be included in a v2? submitted by /u/Technomadlyf [link] [Kommentare]
Hey! I came across this post, which I found quite neat as a minimal demonstration of JEPA. However, as the comments pointed out, there was some room for improvement. So I added a few things such as environment noise and a fair* comparison to a pixel-space baseline. I think the inclusion of environment noise is pretty key, as LeCun himself has stated often and clearly that one of the key motivating factors for JEPA is its ability to disregard unpredictable and irrelevant environment details. Anyway, here’s the result which I think speaks for itself: https://i.redd.it/kadcsrx4nn8h1.gif I think my version paints a much clearer picture of JEPA’s promise. I did remove the web-demo and anomaly detection bit as I felt that wasn't so important to the core demonstration of JEPA as an idea Linking my fork for those interested. Note: Since this was a very quick afternoon-project , I did use AI to make most of the changes, though I did try to do so thoughtfully. Hate that if you must. *fair as in: roughly same parameter count and compute budget. I considered the linear probe and decoder compute budget to be independent from core model training. submitted by /u/Kirne [link] [Kommentare]