Skip to main content

What is Intelligence: Invention as Feature Extraction


When a human agent encounters some problem it wants to solve, it usually invents a solution to that problem using its faculties. Most of the time this invention is usually a tool or a process that enables the agent to achieve its goals.

The primary motivator for invention is the desire for ease and expedience which is embedded deep inside the root code of human agents.

If a human agent roaming the wilderness constantly encounters a fruit it cannot bite open, it invents hitting it on some solid surface to crack it. If it encounters an animal, probably freshly killed by another creature, and it wants to pry some flesh to eat it will usually look for some object with the property of being sharp to cut through the flesh.

It could take that sharp stuff, probably flint and get something to tie it to a stick and thus form a spear from which it can hunt its own animals from a distance. To add ease to its hunting it could observe the flexing and forceful return to normal of some plant stem and see that as a key to inventing a bow and thus a means to project a small spear (arrow) towards an animal by merely using muscle more efficiently in flexing the bow and combining its strength with that existing in the flexible piece of wood, attain its goal of obtaining animals for food with ease.

But how does a human know about these pre-existing properties of materials that it can find in its environment, like wood, rope and flint? It finds out the properties of individual materials through experimentation which is forged by curiosity. See some broken flint on the floor with a sharp edge and eventually testing for sharpness by controlled pressure on its own skin could lead an agent to discover that this sharpness could serve the purpose of cutting meat.

Seeing a crawling plant that is sturdy in the jungle will lead the agent towards understanding that ropes are possible and in combination with a straight stick and flint could form a formidable hunting weapon that will give it dominion over its environment.

The ability to combine these things powered by its imagination because, it has done some kind of feature extraction on the gross input data it encountered in its environment, like the flint stone, the sturdy rope and the straight stick. The imagination plays on the features and produces new imagery of a spear and by picturing this spear fly into an animal and cause injury and death, the agent realizes that it should go ahead to implement a physical spear in the physical world.

The imagination is like some internal simulation environment. At one stage of the imagination process, the outer world is represented as is but is capable of being infinitely modified beyond what the sturdy outside world will allow. At a deeper stage of the imagination, features extracted by layers of the abstraction process are combined in novel ways to produce new structures which can be clothed in more physically realizable images as new things that never existed independently.

When the agent encounters these items in the world using its senses, like sight, for seeing it and touch for testing what looks sharp for actual sharpness. The mind is extracting features from these input data. In the case of the flint, two features that could be extracted are sharpness and its tendency to be beaten into shape by hitting layers off at a certain angle.

Combine these features with the feature of sturdy straightness observed from some standing thin plant or some thin branch of a tree that is holding some heavy foliage and the flexibility of the rope. The imagination cooks up an image of a spear and its power, and the agent implements some physical version.

As more features are extracted from the raw input data in its environment, the agent goes along to invent tools that will ease its life. Eventually inventing some rudimentary shelter to shelter it from the elements and animals.

Whenever something new is invented, the new object becomes input for the feature extraction systems of the mind running on the brain. The feature extraction system extracts features and combines these features with other features obtained from other objects and according to need new objects are invented like the bow and arrow invented from observing the spear and other things and combining them in a form that reduces effort.

We must remember that by feature extraction I mean the typical process of transferring once set of inputs into another representation. In this case of invention, the representation is usually of features of the input data that are invariant for the purposes of achieving the goals for which the agent is optimized for like its survival, not just the features that enable identification like our current CNNs can obtain.

While our current Convolutional Neural Networks seek invariances like translational, that is a cat is a cat independent of where it occurs in an image, etc. The kinds of features an agent optimized for survival will seek are things like will this piece of wood I am about to use to cross this ditch remain sturdy despite my weight?

The very simple invention algorithm can be extended in numerous ways to account for modern sophisticated inventions like space shuttles and computers but it is still the same root process of feature extraction. How the features are combined by the imagination to achieve an object that is useful in achieving certain goals is a function of creativity and is very wild but not necessarily random.

Stuff like neural architecture search is doing the kinds of things our imaginations are doing when they are trying to achieve a creative goal of invention. Although neural architecture search is currently limited to the search for neural network architectures and other optimization parameters, the same searching can be implemented in systems searching for creative solutions to problems. This is the foundation of artificial creativity.

The mind is at it's most primitive level simply about honing in on one contingent blob of useful stuff and connecting it with another blob to create new stuff that can be used as a blob in other configurations. What is considered a blob of useful stuff is usually obtained by trial and error, but I doubt if this trial and error is a random thing. Rather there is some kind of proposed blob of stuff generator, that generates stuff via some random looking process. If this blob producer produces consistently better blobs it is used as the primary blob-of-useful stuff generator, but that doesn't stop the mind from experimenting on other possible generators.


Popular posts from this blog

Next Steps Towards Strong Artificial Intelligence

If you follow current AI Research then it will be apparent to you that AI research, the deep learning type has stalled! This does not mean that new areas of application for existing techniques are not appearing but that the fundamentals have been solved and things have become pretty standardized.

Software of the future

From the appearance of the first programmable computer, the software has evolved in power and complexity. We went from manually toggling electronic switches to loading punch cards and eventually entering C code on the early PDP computers. Video screens came in and empowered the programmer very much, rather than waiting for printouts to debug our programs, we were directly manipulating code on the screen.

At the edge of a cliff - Quantum Computing

Quantum computing reminds me of the early days of computer development (the 50s - 60s) where we had different companies come up with their different computer architectures. You had to learn how to use one architecture then move to another then another as you hoped to explore different systems that were better for one task than they were for another task.