Skip to main content

What is Intelligence: The Wright Brothers


Every AI researcher should meditate on the invention of the aeroplane by the Wright brothers because what we intend to create is no different or less fundamental than mechanical powered flight. It is educative to watch the history of man’s attempt to replicate the flight of birds. The painful aspect of it is that non of the ingenious inventors including no one less than Leonardo DaVinci himself could foresee that the internal combustion engine was the single key to unleashing mechanical powered flight and not the attempt to replicate the flight of birds.

If the early dreamers of powered flight had learnt the art of abstraction, the first question they would have asked themselves is, what components of a bird that are really essential to produce flight? Rather than think in terms of flapping wings they would have abstracted away from the details of the wing implementation, and focus more on the root need for wings. This abstraction process is similar to what deep learning does.

If we had a deep learning network that can return some kind of meaningful data about what is going on in every layer of the network and if the deep learning system can go a bit beyond abstracting away just visual patterns, then inputting the image of a bird in flight would bring about the design proposition for some kind of aircraft!

The blood vessels, bones and feathers that make up the wings would have been abstracted to something that was flapping invariant. The body of the bird with its hollow bones would be abstracted to the hollow cabin under pressure. The head would be the control centre, that is the cockpit. And the internal metabolism of the bird in all its complexity would have been abstracted away simply as a source of power that generates thrust powerful enough to lift the plane against gravity.

This didn’t happen until the Wright brothers and for more than 2000 years man dreamt of flight attained nothing more than gliders or ornithopters. It was not until the invention of the internal combustion engine for a completely different reason that the road to powered flight was paved and then the ingenuity of the Wright brothers applied this invention as a component/module required to power their system. The rest of the history of flight was merely optimizing the initial invention of the Wright brothers.

My fear about the timeline for the appearance of Strong AI is that I hope we are not in the pre-steam engine days in our journey! Then it will take lots of creative leaps to arrive at the “engine” of AI. Another question is, is this engine a physical one like some kind of hardware, for example, quantum computing or is the engine more of software ingenuity? I think this engine is neither purely a software nor a hardware problem but some combination of both. Although I think the software aspect is more of low hanging fruit and is easily accessible by any researcher with modest means and some ingenuity.

Hardware and of course lots of data are the drivers of deep learning based AI system, in fact, they wouldn't be possible without enormous computational resources. But we have to avoid being tricked into thinking that the trend will continue in that direction linearly. No different than people who saw the rise of the rail industry imagined a future where rails would be coming out of individual doorsteps but then came flight. I doubt that scaling alone will lead us to Strong AI. We will need new or revamped algorithms, then maybe hardware will follow to make our algorithms work.

The sad thing about many in the AI research community is that they are replicating the ornithopter (one of the conceptions of Leonardo da Vinci of a flying machine). The saddest of all is those that think we are going to achieve artificial intelligence by replicating the exact structure of the human brain. The argument goes like this: if the brain possesses intelligence then replicating it will result in an intelligent thing. Of course, this replication will be done in software and the hope is that once it is done then intelligence lives on the computer! This is no different from the enthusiasm of the Human Genome Project, the argument was this: sequence the human genome and we will have a flat ladder of nucleotides that will tell us all the secrets of the human organism. This ambition was met with severe disappointment when the project was completed, it turned out there are fewer genes than expected and lots of non-coding regions. The project itself was necessary at least we have turned away from that hype.

The brain replicators are also vying on the same kind of proposition, sometimes because a project looks gigantic it does not mean that achieving it will mean anything. The human genome project has given us a lot of insight into the human organism, but it didn't give us the cure to all diseases. The replication of a brain in the tiniest detail does not give us “intelligence” but it will give us a lot of insight into the functioning of the human brain and might help in the diagnosis and treatment of brain diseases.

In my opinion deep learning is like the steam engine powered by coal! It works at the task of pattern recognition but it does so at the huge expense of energy. We will eventually need some kind of gasoline-powered engine of the deep learning world to reduce the energy usage of a deep learning system. By energy usage, I am mostly talking about two major variables of “energy”. One is the actual electricity that is required to run the enormous arrays of GPUs that are needed to train deep learning models. The second one is the enormous amounts of data that is also required to prevent the model from not just memorizing its input, known as overfitting, to really learning a generalization of the input or the features.

Even with deep learning, we might be trying to copy too much of the physical structure of the brain. Although “Neural Networks” have nothing to do with real neurons in the brain, they strongly follow the principle of Hebbian learning which postulates that as a group of neurons learn new information, they encode this knowledge in the synaptic strengths. Although we know that this is an oversimplification of what is actually going on in the brain. There are other people thinking of different kinds of learning by simplifying other structures in the brain like dendrites. Another group is proposing a microtubule-based theory of consciousness.

There are so many physical structures in the brain from which we can “borrow” and base our theories on and many will be successful if we do the simplification properly by preserving the most important qualities which require us to differentiate between a core structure that is important for cognition and a mere artefact.

Deep learning which is based on synaptic activity has been super successful but we should not fall into the trap that this is all there is to the brain. The deep learning system enables us do good pattern recognition at great cost, but this is only one piece of the architecture that will lead us to full synthetic intelligence on human-engineered hardware, not the whole picture.


Popular posts from this blog

Next Steps Towards Strong Artificial Intelligence

If you follow current AI Research then it will be apparent to you that AI research, the deep learning type has stalled! This does not mean that new areas of application for existing techniques are not appearing but that the fundamentals have been solved and things have become pretty standardized.

Software of the future

From the appearance of the first programmable computer, the software has evolved in power and complexity. We went from manually toggling electronic switches to loading punch cards and eventually entering C code on the early PDP computers. Video screens came in and empowered the programmer very much, rather than waiting for printouts to debug our programs, we were directly manipulating code on the screen.

At the edge of a cliff - Quantum Computing

Quantum computing reminds me of the early days of computer development (the 50s - 60s) where we had different companies come up with their different computer architectures. You had to learn how to use one architecture then move to another then another as you hoped to explore different systems that were better for one task than they were for another task.