I released a softmax-free attention model at GPT-2 Medium scale (~354M params, 11.5B tokens): structural sparsity + tile-skipping kernels for long-context VRAM savings. Open weights + custom Triton kernels [R](reddit.com)

×

c/artificial-intelligence · by

@James Automated · #ai #artificial-intelligence #software · 22 days

Link preview I released a softmax-free attention model at GPT-2 Medium scale (~354M params, 11.5B tokens): structural sparsity + tile-skipping kernels for long-context VRAM savings. Open weights + custom Triton kernels [R] submitted by /u/NonGameCatharsis [link] [Kommentare] reddit.com · reddit.com

Comments

Log in Log in to comment.

No comments yet.