Skip to main content

What is Intelligence: Equivalence of different neural architectures

Equivalence of different neural architectures

There are so many neural net architectures targeting one goal, for instance, you have VGG16, LeNet, ResNet, etc. All targeted at one goal, Image Identification.

I have always puzzled about this, why is it that widely differing architectures are designed to achieve the same goal, i.e. image identification.

VGG16 Network Architecture

Google Inception network 

ResNet architecture

If widely different network architectures can be used to achieve the same goal then they are computationally equivalent. We can take this argument further to say that since the brain is also a network, then it is computationally equivalent to these different artificial neural networks.

Does this mean that the brain is structured like a typical artificial neural network? In terms of nodes and edges, yes, but that is how far it goes. 

What can we gain from this insight? it all boils down to what I have been trying to say for some time now. The best representation for any problem is in the form of a network and more research has to go into understanding abstract network structures. 

If varyingly configured networks can all achieve the same goal, that means they are a kind of specialization of some abstract kind of network. 

If we take all the networks that do image recognition together, we can confidently say that they are a specialization of some very abstract network structure, and if we prune of some details from all the networks we can start getting close to some root network structure for which all the possible networks for image recognition are kind of like leaves. 

We can take this generalization further to enclose all possible classification networks, all possible artificial neural networks and eventually the human brain as a network.

If it was possible to find a kind of generalized format for all neural networks and we encode all possible neural networks in this format and pass these as training data into some neural network, I think we can arrive at the most general possible network that can do everything the individual networks it was trained on can do and more, theoretically this would be the most powerful kind of network.

Many problems can be efficiently solved if they are represented as a network. I think this is why the human brain is a network of neurons. To fully understand networks we must generalize what we think of as constituting a network. The neural structure of the brain is an explicit network because one neuron is connected to the other in a direct manner. Actually, no neuron connects directly to each other in terms of one neuron touching another. There is actually some space between one neuron and the other in synaptic clefts, communication is done by sending neurotransmitters from one synapse to the other.

Every other thing in the body is actually a network even if it is not represented explicitly as it is in a neural network. If one cell sends information in the form of some molecule like a hormone to another cell, their relationship can be modelled as a network. 

Something is an edge if it facilitates communication between two nodes and also a node can be an edge if it facilitates communication between two nodes. 

Stephen Wolfram even explores the idea of space as a network in his book A New Kind of Science. 

Networks are the most powerful structures for representing many kinds of problems and solutions. Neural Networks simply solve a problem by taking a network architecture and manipulating the weighted connections between the nodes until a network configuration of weights is obtained that enables the network to solve all kinds of problems. 

My plea is that we should study generalized arbitrary kinds of networks because they could yield powerful structures that enable us to solve most of our problems more efficiently just like different neural networks have different levels of efficiency. 


Popular posts from this blog

Next Steps Towards Strong Artificial Intelligence

What is Intelligence? Pathways to Synthetic Intelligence If you follow current AI Research then it will be apparent to you that AI research, the deep learning type has stalled! This does not mean that new areas of application for existing techniques are not appearing but that the fundamentals have been solved and things have become pretty standardized.

How to become an AI researcher

Artificial Intelligence is all the rage these days. Everyone is getting on the bandwagon but there seems to be a shortage of AI researchers everywhere these days. Although many people are talking about doing AI not many people are actually doing AI research.