Neural networks have always been a tricky subject to understand. Deep neural networks are beyond the scope of most people. They consist of multiple neurons and which are used for various and diverse applications in the industry.
But these multiple hidden neurons is what has given them the ‘black box’ stigma.
In a blog post, researchers at DeepMind have explained how they went about understanding and judging the performance of a neural network by deleting individual neurons one by one, as well as in groups. The researchers developed image classification models and then removed several neurons. Then they measured how each deletion affected the outcome of the model.
According to DeepMind, their findings yielded two outcomes:
In the above image, the top most neuron (which is greyed out) has been deleted. On DeepMind’s blog post, you can delete each neuron and see how it affects the output results.
You can read DeepMind’s official research paper on the topic here. They are scheduled to present this next month at the International Conference on Learning Representations (ICLR).
Deep neural networks have been hard to interpret so this is a nice start towards demystifying them by one of the leading research companies.Their results imply that individual neurons are much less important than we would have initially thought.
I highly encourage you to go through their blog post and the research paper to understand each step that was taken in performing this.
Lorem ipsum dolor sit amet, consectetur adipiscing elit,
Hey Pranav , Has there been any research put in finding a generalized neural network which can help in any type of classification (audio , image , text) ??
Yes there has been such research going on. You can check out the arXiv paper "One model to learn them all"