c/artificial-intelligence

Suppose you’re a PhD advisor in machine learning. Your student has been in the program for 4 years, has done solid work, and has a coherent thesis direction but they haven’t published in an A*ML venue or top journal. No NeurIPS/ICML/ICLR/CVPR/etc., and no equivalent top venue in their subfield either but 3 First author A level paper. Would you still support them graduating if the thesis itself is solid? submitted by /u/Hope999991 [link] [Kommentare]

Hi internet friends, I recorded a workshop about building your own LLM without any math / ML prerequisites. It covers everything from machine learning fundamentals, deep neural networks, transformer architecture, and pre/post-training. The only prerequisite is being comfortable with learning through code & excel examples. Sampling Large Language Models Reverse Engineering Large Language Model Perceptrons: wx+b Activation Functions: ReLU, GELU, SwiGLU GPU Coding: PyTorch, torch.compile(), fused kernels, CUDA, Triton MLPs/FFNs: Multi-input, Multi-Layer Perceptrons, Feed-Forward Networks Loss Functions: Residual errors, RMSE, Cross Entropy, Loss Landscapes Backpropagation: Training loops, Optimizers, Learning Rate, Batch Size Saving & Loading Models Initialization: Kaiming, Glorot Residuals: Addition, Scaling, Gated, Concatenation Normalization: Pre-norm vs. Post-norm, RMSNorm, BatchNorm, LayerNorm Regularization: Dropout, Gradient Clipping, Weight Decay SoftMax Tokenizers: By Character, By Word, BPE, SentencePiece Embeddings: Absolute vs. Learned, Sinusoidal vs. RoPE Attention: MHA, GQA, MQA, MLA Transformers Pre-training: Data Sources, Datasets, HTML Cleaning, Quality Filtering, Sharding Evaluation: Leaderboards, Benchmarks, Verifiers vs LLM-as-Judge Instruction Tuning: Alpaca & Other Formats, Self Instruct, Capabilities Reinforcement Learning: Policy Optimization, SimPO What We Didn't Cover: Scaling Each section has slides teaching the concepts, followed by excel-by-hand developing intuition for the math, and then coding examples. The goal is able to grok all parts of modern LLM development. We did this workshop in-person in San Francisco last month and hopefully the spaciousness of watching online works for everyone. If don't like watching videos, you can get the slides and exercises and work self-paced. submitted by /u/JustinAngel [link] [Kommentare]

I've been working through the internals of LLM inference and writing up what I learn as an open, in-progress handbook. Just wrapped another chapter on GPU execution and memory internals: why a GPU sits mostly idle during inference, how the memory hierarchy gates throughput, and where the real bottlenecks live. Added mermaid diagrams for the architecture pieces so the flow is easier to follow than a wall of text. It's a personal learning project, still growing chapter by chapter. I'd value feedback or corrections from anyone who's run inference in production, where my mental model breaks down is exactly what I want to find. Issues and PRs welcome. github.com/harshuljain13/llm-inference-at-scale submitted by /u/YouFirst295 [link] [Kommentare]

A paper currently trending on paperswithcode.co in the "Anomaly Detection" category is DVD-JEPA. https://i.redd.it/r6fd8n3d4f8h1.gif Here is the short summary: Most attempts to learn a world model from video try to predict the next frame pixel-by-pixel, and drown in detail that is fundamentally unpredictable. JEPA (Joint-Embedding Predictive Architecture, LeCun 2022) makes a different bet: predict the representation of the future, not the pixels, and let the encoder discard whatever it cannot predict. DVD-JEPA is the smallest honest demonstration of that idea we could build. The "world" is a DVD logo bouncing in a 16×16 box. A context encoder, an EMA target encoder, and a latent predictor are trained — with no labels and no decoder — to predict the next observation in a 32-dimensional representation space. We then show three things: It learned the world. A linear probe recovers the logo's exact (y, x) position from the frozen 32-d latent to within 0.73 px — though it was never given a coordinate. It can dream (once you add a decoder). Bolt an optional decoder onto the frozen latents and roll the predictor forward: it renders a correct future-frame video of the bounce, including wall reflections, for ~20 steps before latent drift sets in. It is useful. Run it as a 1-step predictive monitor, and the prediction error becomes an anomaly signal: inject a teleport and surprise spikes 88× over baseline, on the right frame. The whole thing runs client-side in your browser — the trained MLPs are re-implemented in ~40 lines of JavaScript. It is a joke, and it is also a correct, working instance of the architecture behind I-JEPA, V-JEPA, and V-JEPA 2. Find the paper, HF model, and project page here: https://paperswithcode.co/paper/98361 submitted by /u/NielsRogge [link] [Kommentare]

Hey everyone, I’ve been building an end-to-end Air Quality (PM2.5) forecasting pipeline for 4 countries (US, UK, India, Australia) using 1.6M+ rows of OpenAQ and NASA weather data. The problem i hit (the variance trap): My V7 model was a standard stateless Gradient Boosting Regressor. It worked great for low-variance regions (like the US), but in highly chaotic environments (like India and the UK), the model was mathematically failing. When I calculated the MASE (Mean Absolute Scaled Error), it was > 1.0. Literally, a naive carryover guess was outperforming my ML model because the model couldn't anticipate sudden momentum shifts. the fix (Horizon aligned architecture): Instead of falling into the recursive snowball trap (where day 1 error compounds into day 30), I completely decoupled the horizons. I engineered strict autoregressive lag vectors aligned specifically to the target horizon (h=1, 7, 14, 30). Injected a 3-day rolling volatility matrix that ends precisely at the inference boundary to prevent data leakage. Result: MASE dropped strictly below 1.0 globally Even at a 30-day horizon, the model maintains a 57% predictive accuracy over the chaotic thermodynamic baseline. The stack: backend pipeline : Python, Pandas (for the memory matrix), scikit-learn, FastAPI. frontend : Next.js 16 (App Router), Tailwind v4, Recharts. Deployment: Vercel with automated GitHub CI/CD sync. (currently pushing updates manually afetr every test, so the site is actually static will automate it later) I'm currently using scikit-learn GBR, but but my immediate next step is to rip it out and rewrite the core engine using Xgboost or LightBGM to handle the sparse temporal features better. If any MLOps or Data Engineers here have advice on scaling XGBoost for multi-horizon forecasting without exploding the compute, I’d love to hear it. Roast my architecture, the repo is public. live URL : https://global-aq-intelligence.vercel.app/ github: https://github.com/divyanshailani/global-aq-intelligence-pipeline submitted by /u/Divyanshailani [link] [Kommentare]

In our #ICML2026 position paper we argue a dynamical systems perspective is needed to drive time series (TS) modeling forward: https://arxiv.org/abs/2602.16864 Essentially all time series in nature and engineering come from some underlying dynamical system (DS), mostly chaotic for complex systems, and acknowledging this helps to address many open problems. Dynamical systems reconstruction (DSR) goes beyond mere forecasting and gives us an understanding of the dynamical rules that underlie observed time series. This in turn may enable true out-of-domain generalization and predicting a system’s long-term behavior, something current TS models cannot do. In the paper, we compare a variety of custom-trained and recent foundation models for TS and DSR w.r.t. short- & long-term forecasting. Specifically, we suggest: 1) Put a focus on DSR-specific training techniques and objectives in TS model training, such as generalized teacher forcing (https://proceedings.mlr.press/v202/hess23a.html). These will enable capturing long-term statistical properties and dynamical structure, and at the same time help massively reducing parameter load and complexity of TS models. Proper training is more important than model architecture! 2) Pretrain TS models on simulations from dynamical systems, rather than on artificially created time series functions. These will yield much more natural priors for real-world TS. Chaotic systems in particular contain a rich temporal structure and many timescales (often an infinite skeleton of unstable periodic orbits of any period). 3) Move away from transformers, back to modern RNNs. DS are defined by recursions in time. By ignoring this and potentially further coarse-graining signals, transformers lose essential dynamical information, making them generally incapable of capturing a system’s dynamical rules. This is evidenced by their failure to forecast a DS’ long-term statistical or geometrical structure. 4) Address the hard problems in TS modeling: Topological shifts (https://proceedings.mlr.press/v235/goring24a.html). Although in itself tricky, the really hard problem in TS forecasting is not so much mere out-of-distribution shifts, but changes that drive a system across tipping points or into different dynamical regimes, where the vector field topology changes. 5) DS properties like attractors or bifurcations are universal – acknowledging this in TS modeling will give a kind of mechanistic and transferable understanding of TS properties that is independent from specific (physical, medical, …) domain knowledge. It therefore also pays off to put a focus on mathematically tractable and interpretable models. With a great team of shared-first & co-authors, Christoph Hemmer, Charlotte Doll, Lukas Eisenmann & Florian Hess! submitted by /u/DangerousFunny1371 [link] [Kommentare]

as per the title, how to access books3 dataset for research purposes? submitted by /u/xolmnyc [link] [Kommentare]

Recommend some of the best modern books about probability and/or statistics to help you get that probability intuition or mindset needed to excel at ML, from beginning to advanced or separated please! submitted by /u/c_carav_io [link] [Kommentare]

I apologize for such an amateur question if someone is offended ​ I just finished my 2nd year of degree. Well, the degree was a bit slow and I did the ML course this semester as well but being a Third World Country and stuff, it doesn't really matter cause I didn't learn antg of value from them ​ I've been studying ML myself for 5-6 months, but I skipped the last 2 months cause of some issues and I've failed to get that motion back so I need a little bit of advices as where to continue ​ I know python of course and I've learned many ML algorithms, all supervised and what you'd call easy. I have understood their general concepts and maths but never went in deep. I did them in practical as well. Made a very few projects. ​ Now, I'm confused what should I learn next, I feel unsupervised learning isn't really my thing or I wouldn't be able to do it so can I just skip that? And idk what's next, so what is it? I've thought of learning Agentic AI as well but I can't do that until I'm satisfied with myself that I completely know ML and I can work on professional level. ​ And if you've any resources to learn from, certifications etc as well. I'd really appreciate it. Again I apologize for really rookie questions. submitted by /u/Negative-Guard-4487 [link] [Kommentare]

I'm a recent CS grad trying to break into ML engineering, and I just finished the first version of a side project I've been working on. Posting it here because I want people who know this space better than me to poke holes in it. The idea started from that feeling every dev has had, where you commit something and a second later your stomach drops because you think you just pushed an AWS key. Server side scanning catches that eventually, but only after it's already in your git history. Local tools like gitleaks run before that, which is the right idea, but they're pure pattern matching, so anything that isn't a known secret format slips through and they say nothing about whether the code itself is risky. Piping your diff to a cloud LLM just trades that problem for a different one. So I built a git hook that runs three checks on a commit, all on device. A fast Rust regex pass catches known secret formats and blocks the commit if it finds one. A small classifier running on the Neural Engine through CoreML catches riskier patterns that don't have a fixed string to match, things like shell=True in a subprocess call or disabled TLS checks, and also blocks if it fires. A small local LLM (Qwen2.5-Coder, 1.5B, running through MLX) reads the diff more like a human reviewer and flags things like injection risks or dead code, but it never blocks anything, just leaves notes. I kept it that way on purpose, since a false positive that blocks your commit kills trust fast, but a false positive that's just a comment costs nothing. Biggest weaknesses right now: it's Apple Silicon only since CoreML and MLX are both Apple specific, and the classifier is trained on a fairly small dataset so I wouldn't call it bulletproof yet. Repo's here if anyone wants to dig into the code: https://github.com/stalzkie/local-forge Mainly curious whether the three layer split makes sense to people who do this for a living, and what risky code patterns I might be missing for the classifier. submitted by /u/StalWrites [link] [Kommentare]

Months ago, I got my first maintenance project. Before this, I had only built new solutions from scratch and maintained my own code. But maintaining someone else's system feels completely different. ​ ​It’s a prescriptive recommendation system that uses XGBoost models and Differential Evolution for optimization. The problem is that everything is in a single repository: raw data ingestion, transformations, model training, reporting, the optimization engine, post-processing, and MUCH more. The only thing outside the repo is the frontend website. To me, it looks like a massive, super complicated monolith. ​ ​After almost 3 months, I still find new "patches" (quick fixes) every single day. Every time I do, I have to re-learn how the system works. The documentation is very generic and a total mess; it mixes the original design with patches from the two maintenance teams that came before me. I’ve checked some of the docs, but definitely not all of them, because there are about 50 long markdown files. ​ ​Have you ever dealt with a prescriptive system like this? How do you survive? Honestly, I’m debating whether to just quit or keep patching the code however I can until the project ends—even though I know that’s not the right way to do things. ​ submitted by /u/DescriptionBorn153 [link] [Kommentare]

Hi All! I have developed a research optimizer (QQN Quadratic Quasi-Newton) and published a paper on it where I am able to, but I would really like to make the algorithm itself easily available to the community for evaluation. I have a Rust, Java, and Javascript implementations, but these are built with my own learning frameworks around them (or Tensorflow.js for the last), so I need to port it to something with wider usage. Tensorflow.js seems to lack a central place for optimization algorithms, and also doesn't seem super widely used? I checked out argmin (rust) but it looks like there has been no dev activity on it for about 8 months. I don't want to invest the time porting to a project that might have the same issue! This space is always changing and hard to keep up with, so I thought you guys would have some good ideas. Also, I'm looking for something close-to-metal and strongly typed. Thank you for your time! PS: Apologies if this qualifies as a "career question" - I posted after no small amount of internal debate! submitted by /u/Kooky-Bit8706 [link] [Kommentare]

I was pondering on this question and decided to dive deep into torch.compile. It was a lot of fun learning about operator fusion as the central idea behind torch.compile. So I created a tiny version of torch.compile in 500 lines of python and a notebook showing how this works: https://github.com/purohit10saurabh/tinytorchcompile Let me know if you find this interesting! 🙂 submitted by /u/Other-Eye-8152 [link] [Kommentare]

I maintain cuTile Rust and just posted the paper "Fearless Concurrency on the GPU." As more GPU code gets AI-generated, the bottleneck moves from writing it to trusting it. cuTile Rust lets you write or generate GPU kernels whose memory safety and data-race freedom are verified by the compiler, through Rust's ownership and borrow checking. You get those guarantees by construction. It's a tile-based programming model that lowers to CUDA Tile IR, carrying Rust's ownership model across the launch boundary. You partition a mutable output into disjoint mutable sub-tensors, pass inputs as shared references, and write tile kernels with single-threaded semantics that the compiler maps to thread blocks. End to end, we built Grout, a Qwen3 inference engine, on cuTile Rust with Hugging Face. At batch-1 decode it reaches 171 tok/s for Qwen3-4B on an RTX 5090 and 82 tok/s for Qwen3-32B on a B200, competitive with vLLM and SGLang. Batch-1 decode is memory-bandwidth-bound, and Grout's throughput is consistent with our HBM roofline analysis. Many of Grout's kernels still use the unsafe path today, but they can be migrated to safe variants, providing a verifiable target for generated kernels. We've started a collection of such kernels in the cutile-kernels crate in the repo. If this is your thing, contributing safe variants helps grow a library of safe, high-performance kernels that future kernel synthesis can draw from. On the kernel side, the safety is effectively free. On a B200 the safe GEMM is within 0.3% of a hand-written low-level version (~92% of dense f16 peak), and element-wise hits ~7 TB/s, matching cuTile Python within measurement noise. Some additional caveats worth noting: Grout is batch-1 with a small set of supported models (a research case study, not a drop-in server), it's NVIDIA-only (lowers to Tile IR), and GEMM still slightly trails cuBLAS at some sizes. - Paper: https://arxiv.org/abs/2606.15991 - Code: https://github.com/nvlabs/cutile-rs - Grout: https://github.com/huggingface/grout Hope you enjoy the paper and learn something new! Happy to answer any questions :) submitted by /u/Exciting_Suspect9088 [link] [Kommentare]

Hi! We just released a paper where we study “Rosetta Neurons”: universal neurons across different neural networks, and their relationship to scaling laws, specialization, and monosemanticity. Would love to kick off a discussion and get the community's thoughts. Main Findings: We find that the universal Rosetta Neurons scale as a sublinear power law: larger models have more of them, but they occupy a shrinking fraction of all neurons. They also become more selective/monosemantic and more specialized with scale. We can use a single Rosetta Neuron to filter data for continued pretraining and nearly match oracle data filtering. Paper: https://arxiv.org/abs/2606.03990 Summary thread: https://x.com/_AmilDravid/status/2062959617941074069?s=20 Code: https://github.com/avdravid/rosetta-neuron-scaling Project page: https://avdravid.github.io/rosetta-neuron-scaling/ https://preview.redd.it/sus4wqc9g38h1.png?width=1806&format=png&auto=webp&s=4aac2b2209779cb05e1c73cdaadac860318f0162 submitted by /u/avd4292 [link] [Kommentare]