InFeeo
United States
All
New
Language
Profile channel

@Nikobar

No bio yet.

Since 06.06.2026

Learning FlashAttention the Hard Way. Part 1: The Algebraic Foundation [D](reddit.com)
I'm writing a short series of tutorials on FlashAttention: from theory to efficient CUDA kernels. Part 1 is the theoretical foundation. It walks through a modern algebraic formalism showing that FlashAttention is an associative operation, which lets you treat it as a regular reduction on the GPU and apply all the same scheduling optimizations. Some recent MLSys and CVPR papers lean on this framing, and I find it much more powerful than the original. Overview: Safe softmax, Welford's variance, and FlashAttention are the same secretly-associative operation The twisted monoid (transport of structure), why the max-rescale coupling doesn't break associativity The qk_scale = log2(e)/√D you already see in FA-2 and the Triton kernels, derived from scratch Numerical analysis: overflow bounds, error limits, and why tiling never amplifies error Bird's 3rd Homomorphism Theorem as a test for whether any loop is secretly associative submitted by /u/NoVibeCoding [link] [Kommentare]
What if a model could only learn what trusted LoRA adapters can express? [R](reddit.com)
Hello I published a paper. Most defenses against fine-tuning poisoning try to detect malicious data or reduce its impact. I explored a different question: What if the model simply could not learn certain malicious updates? The idea is to constrain fine-tuning to a subspace learned from trusted LoRA adapters. Useful adaptation remains possible, but some malicious directions become geometrically unreachable. A concrete example: a company fine-tunes a model on large datasets coming from users, external sources, or generated data. A small amount of poisoned data could introduce a hidden behavior triggered by a specific phrase or pattern. Another example is a local or on-device assistant that keeps adapting to its user. Instead of allowing it to learn any possible behavior from new data, its adaptation could be restricted to variations of behaviors already represented by a trusted pool of adapters. The goal here is not to detect every possible poison or backdoor, but to restrict the space of updates the model is allowed to learn. I tested the approach on 196 public LoRA adapters, including adaptive attacks specifically designed to bypass the defense. The results are strong: attack success drops sharply while useful adaptation is largely preserved on tasks covered by the adapter pool. The paper, code, and experiments are public. Paper: https://arxiv.org/abs/2607.05300 Code: https://github.com/infinition/z-manifold I would be very interested to see people try to break it. submitted by /u/Bright_Warning_8406 [link] [Kommentare]
EBRD and EU back URECA’s Coal-to-Solar project in Mongolia. Data stored on Blockchain(reddit.com)
In Ulaanbaatar, Mongolia, families who live in ‘gers’ (Mongolian yurts central to the nomadic way of life) wake throughout the night in winter to feed their stoves coal: at 01.00, 03.00, and again before dawn, just to survive freezing temperatures. The very fuel that keeps them warm also makes the city one of the most polluted capitals in the world, with each household on average responsible for 12 to 13 tonnes of carbon emissions. URECA is a climate tech startup whose pilot Coal-to-Solar Initiative project pairs technology with carbon finance to direct capital into scalable, high-impact climate solutions while supporting low-income households to transition to clean energy and tackling severe air pollution. They propose a new technology designed to restore trust and credibility in climate finance, making it more accessible to communities. With help from the EBRD’s Star Venture programme, URECA is now better positioned to scale its climate solution. submitted by /u/semanticweb [link] [Kommentare]