Mechanistic interpretability: a first paper on disentangling a convolutional neuron [R](reddit.com)

I have recently started working in mechanistic interpretability independently, starting with distill circuits thread My work is on disentangling and closely studying a single neuron, a 1x1 convolution in inceptionv1 model (and applying the method to other neurons in the same layer). The key insight was that the hadamard product of the receptive field and the weight of a neuron is what the neuron is 'seeing' or detecting. We can cluster the hadamard product to get all the patterns a neuron detects. It gave clean monosemantic clusters (cars, cats, dogs which it was known to activate on). We also get more clusters however, letters, human faces, and many more low valued activations. This gave me a new technique to analyse the neuron very closely. On close analysis the most peculiar thing I found was that the low valued clusters (like letters) had all its dependent neurons also firing on the same concept (letter), and the positive and negative weights were evenly distributed between them to bring down the sum. An evidence of gradient descent working deliberately to put patterns and concepts in a noisy range. I've tried to keep it very distill like with good visualisations. I hope you give it a read. https://pages.narang99.in/posts/2026-07-12-disentangling-mixed4e-55/ I made a mistake honestly by starting with convolutions, nobody seems to care about it. I'll start working on language soon, but it would be good if anyone can read this, it would be good to have some feedback on whether I've actually found anything useful. Thank you :) submitted by /u/narang_27 [link] [Kommentare]