What is Intelligence: Electronic Computers

ELECTRONIC COMPUTERS

With the availability of electricity, an agent charged with solving a sudoku puzzle by applying the greatest amount of intelligence possible will opt for the construction of a mechanical system that goes beyond the manual application of labour towards one where motive power is supplied by the agency of electricity.
The agent might still choose to use their manual mechanical systems but rather than using human or animal muscle to power the system, the agent will use electricity to supply motive force. This will allow the agent speed things up because electricity will allow the rotors in the system to move faster, at least just fast enough so that they don’t fly out of the system. With muscle power, the machine is limited to just how much power the human or beast moving it can generate, while with electricity the limitation is on how tolerant the mechanical system is to rapid infusions of electrical power.

Also, we have to know that there are other things like heat generated in a system with moving parts, caused by friction, will limit the speed of operation but applying intelligence the agent will go ahead to design a cooling system to cool down the mechanical parts allowing it to do more calculations at higher speeds.

If the agent is creative enough and has had experience with certain materials the agent will start considering the elimination of moving and parts and replacing them rather with parts that just hold states long enough so that they can be manipulated. The power provided by electricity is kind of limitless if the agent goes beyond merely using it for supplying motive power. The agent could see from experiments elsewhere that certain substances hold their charge for a while even when the supply of electricity is cut off. These materials which can be set in a certain arrangement will become a form of storage for the agent through which the agent can hold a state.

The agent will also need the means to switch the state holding device on or off because this gives it basic control over the states and the agent will find a means of reading off the states that the system holds.

So basically we can imagine that the agent discovers a simple arrangement of materials where two metal plates are separated by some space and when this arrangement is supplied electricity, it can hold that electric charge for some time even when the electricity supply is cut off. This is basically a capacitor and using its creative capacities the agent realizes that it has found a means of holding memory beyond paper and pen. Although paper and pen are much more permanent means of holding memory, this new system can be automated much better than paper and pen.

More fundamentally the agent will discover that a switching mechanism for turning that storage on and off will give it the ability to quickly create a memory and erase it. With paper and pen or rather a pencil, the agent understands that erasing what has been written on paper is limited by the need for muscle power.

In the earliest design of the switching mechanism the agent might have to use a mechanical toggle to switch its storage on and off and although this still depends on muscle it is way faster than the use of the eraser. It is not much of a stretch of intelligence to find a way to automatically switch the storage on and off rapidly.

The capacitor and the switch form the fundamentals of memory and control, from this point on replicating this basic unit and finding means of synchronizing their actions will lead to a larger system and with time the agent will not only advance beyond the paper and pen system but also gain the ability to build other abstractions beyond what paper and pen could ever offer.

If a state can be held by means of electricity, then a system can be designed to hold many states at once using the same electric power. This is an electronic extension of the paper and pen system.

In a paper and pen-based system, we would write out the symbols that pertain to the data and also write out other symbols that pertain to operators and then using some basic procedure that we also represent with other symbols we would apply the operator symbols to the operand symbols and produce results. This is basically computation at its most primitive level. We could do the same computation on the earth using sticks and writing on dirt, but the availability of paper or even a blackened wooden slate and chalk would still work but would be messy.

Having access to electricity gives us an economical way of doing computations very fast and if we shift the application of electricity from merely providing motive power for some gear systems to holding a state in a capacitor, we reduce the amount of electricity we use even more.

Computation is the ability to change the memory-held states in a systematic fashion such that if a certain instruction is passed unto the computation system, it transforms the state of the system in a highly predictable way. The is the basis of computation on paper and on electronic systems or on any other system we will find in the future.

Now, all though we have been focusing on memory and the ability to control that memory using a switch. On paper and pen, memory simply means writing out what we want to compute on the paper and the process of changing one thing to the other simply used more paper to transform symbols. To be economical we could erase symbols we no longer needed in our transformation process but in all these, there was no explicit way of issuing instructions to the system. This is because we were using the paper and pen as an extension of our own mind in a very direct manner. The paper and pen are dead systems because they have no motive force of their own. But when we bring electricity in then we have new lifeblood to power a different kind of system, we can now build a system and power it to perform operations on its own using electricity without our intervention.

This is the first step towards real automation. We could build an automated writing system that did not require our presence by using some mechanical system powered by human or beast or even electricity but building a more efficient system that uses electricity gives us a real chance at true automation because it takes away the need for the mechanical application of force by muscle.

Since we now have an automated system it would do the agent a lot of good if it could just issue out instructions to the system and then wait while the system uses electricity to crunch out the details. The two components of computation are memory and control. The memory part we have achieved using a capacitor and the primitive control of the capacitor can be achieved by a vacuum tube or better still a transistor but the higher level control of the entire system will be done by some kind of programmability which allows us to issue instructions into the system.

This is similar to the symbols for data and symbols for operators set up in the paper and pen system. The operators are like instructions for performing certain manipulations on the symbols for data.

If we have a sequence of symbols like 1 + 4, the symbols 1 and 4 are data while the + symbol is an operator that performs an operation on the data items to produce a result which is 5. When doing this on paper, we could perform the operation mentally and only write the result or we could do it explicitly on paper using some procedure.

A procedure for doing addition could be simple as using counting. In the previous example if we have some primitive device for doing counting like our own ability to say 1, 2, 3,… while pointing to objects, we can mechanize the addition process by writing:

1 + 4

1 + 1111 (* remembering that 4 is made up of 4 1s *)

Removing the plus sign

11111

If we count by pointing at each 1 and not repeating any previous count we come up with the answer

5

This simple procedure can be fully automated if we could somehow construct a mechanical counting system beyond our brains in some explicit medium. The entire + operation encapsulates all the activities that involve adding with counting as a subprocess of the addition process.

If we were to transfer these operations into an electronic system, we would build some primitive circuitry for counting probably using a 2 symbol alphabet like 1 and 0 to represent the decimal symbol system that we use on paper.

Using a reduced alphabet is necessary because remember that our memory system has only two states, on or off which we could use to represent 1 and 0. If we have four cells to represent the first operand (data item we want to operate on) and four cells to represent the second operand and provided we have a simple method of performing addition on the binary symbol system for numbers we could tap off the information for the two sets of four operand cells and perform the addition in some temporary memory location which we call a register and write out the results in another set of four cells.

The operator, in this case, is the instruction we give to the system to perform the desired operation like addition and this operator logic can be built into circuitry using a system of memory and control switches that are connected in a certain manner to produce certain results.

In summary, rather than representing the system on paper like we have done earlier or encoded the “rules” that enables us to arrive at a solution in some gear and crank mechanical system, we build an electrical system for holding and manipulating memories. This device is what is called an electronic computer.

If the agent has access to this system it will go ahead and transfer its sudoku solution procedure unto the electronic medium so that it could avoid paper wastage and much application of muscle energy in the crank and gear system. With access to an electronic computer the agent has more flexibility and could start building out new iterations of the solution procedures until maybe it could hit on a goldilocks procedure by representing the sudoku problem as a constraint satisfaction problem and building the equivalent solver to solve it.

This will require some system of representing the problem in a form that is adapted to this medium and writing a “program” to manipulate the representation until a solution “state” is arrived at. A program here would be a sequence of instructions that manipulate the memory and switches in a fixed repeatable order.

The agent will have to use the symbols and controls of this new world of electronic computing to represent the problem. This will deviate a lot from the paper or the mechanical crank world because now the agent is dealing with a system with many limitations but nonetheless a system whose capacity for processing is faster and much more efficient than the paper/pen arrangement or the mechanical crank system.

After due representation in this electronic world, the way to the solution is using another set of symbols which encode instructions to the system which manipulate the representation until a solution is reached.

Initially, when operating in this world the agent may not come up with the most sophisticated procedure for solving sudoku. The earlier attempts might be very naïve and just a fair improvement over solving the problem on paper.

The agent will initially attempt to duplicate the paper world by building a program that is not different from a raw simulation of solving sudoku on paper, even the mechanical crank system must have been a raw representation of the sudoku on the paper solution.

If the agent has some math skills then it would have proceeded to solve the sudoku problem using equations but we are assuming raw intelligence with very little priors. If the agent was in possession of math skill which it would have developed independently of the current sudoku problem that it is currently faced with, it could have come up with a sophisticated solution and thus any manipulation on paper would be with math symbols and not naively replicating the raw physical structure of sudoku.

If the agent went in the path of math, it would solve it on paper and then proceed to build a mechanical crank system for doing the math and then solve it on the system. And if the agent built an electronic system, it will be a system used to represent a problem in mathematical form and manipulate the system using mathematical rules until a solution is arrived at.

But in this exploration, we are just observing the evolution of an agent that might just possess raw intelligence which might be composed of only an ability to recognize the state of a system visually and the ability to generate a new state (mentally) that obeys the constraints of a particular task the agent is faced with.

The reason why the agent tasked with solving a sudoku problem would not go straight on towards a mathematical solution is that the mathematical nature of the game is not very obvious. The game is more pictorial because first of all, you are dealing with a board with certain numbers filled in some slots. All the agent sees is board cells with symbols and those without and it is tasked with filling those squares without symbols following some constraints given by the game designer.

If the problem the agent was given was one of calculating the consequences of performing certain operations on numerical symbols, assuming the meaning of the symbols and operations is explained to the agent, the agent would go with paper and pen calculations, if that is available or just do the thing on earth by scratching with a stick.

When the agent has mastered calculation with paper and pen it could proceed to automate its task even further because agents are naturally lazy and want to reduce as much effort as is needed to achieve a goal. The agent could go ahead and build a machine for doing its calculation like an abacus. With more insight, the agent could invent variables or place holder symbol for representing other symbols in a general way so as to abstract beyond the specifics of any calculation towards some generalized representation.

From such generalization, the agent would then go beyond the abacus to building a generalized symbol manipulator, which is the kind of machine Babbage and others envisaged to build. From there, if presented with the wonderful motive power of electricity, the agent would go ahead to build some kind of electronic symbol manipulator which is essentially a computer.

The instructions that make it possible to compute things on an electronic computer are very simple, the complexity of modern computers might make us lose sight of this simplicity, but the entity/agent tasked with Sudoku for the first time and possesses intelligence will construct a much simpler electronic system. It will most likely end up with some kind electromechanical system where switching is manipulated by hand to encode the problem and then the computation is done by further manipulating other switches till a solution is read out by probably reading out some output encoded in a form that makes it easy to design an expedient electromechanical system.

Now, this entity could improve the electromechanical system by applying intelligence and reducing the total amount “mechanical” actions needed to solve the problem due to its inbuilt laziness factor. At the highest level the entity could come up with a system that just requires a keyboard to enter the characters that encode the solution to the problem into the system and a whole series of automatic transformations go-ahead to transform the data into some representation close to the machine till it can now manipulate the internal electric circuits to produce the solution to the problem.

We must always keep in mind that the pen and paper, the gear and crank mechanical device, the electromechanical computer, the purely electronic computer, etc. into any future computation devices based on any kind of motive power are all memory and manipulation aids that enable the intelligent human being to encode its systematically arrived at solution to the problems it encounters.

Whenever an entity wants to reduce the amount of effort required to achieve a particular goal, it uses intelligence. If the entity asks itself how can I do this in the easiest way possible with the least application of effort, it summons the powers of its intelligence to “invent” a solution which is usually in the form of some “algorithm” or recipe for how to achieve that goal using improved methods, algorithm here is another name for procedure.

If the entity likes a particular fruit and decides that the distance to the fruit is too much and it decides that it wants to find a way to reduce that distance, it goes ahead and invents agriculture. Using observation, it could notice that the fruit it ate a week ago is germinating around its residence and it then discovers that things can grow away from their main source, which is the tree in this case. Over the years this entity observes that seedling grow to become a tree. It contemplates the process that must have brought it close to home and remembers from memory that it actually threw a previously consumed food item there.

Following the coping algorithms of the organism it might decide that if it wants more of that particular fruit it could run an “experiment” to eliminate the influence of “gods” in the arrival of the tree in its vicinity, it could go eat the fruit and throw the seed which it had found from experiments were not very tasty, most seeds aren’t tasty by design. This time around the sowing will be a conscious event with an intention to have more trees around. Once the entity sees the first sprouting of the seedling, it is sure that agriculture works and it can go ahead and start optimizing this newly invented art of agriculture until it becomes a full-blown science.

Waiting for trees to bear fruit is quite boring and dissatisfying. Out in the wild, you could just pluck a fruit already made on a tree. You didn’t need to plant it because you just found it out there and you enjoyed the fruit. But on discovering agriculture you realize that growing fruit is a slow process and it is natural to want to speed up this process due to built-in impatience in the agent. It is in attempting to speed up a process that we have some understanding about, that we summon higher and higher powers of intelligence to come to bear.

Without advanced biotechnology which will eventually be invented in the future when different agents in different localities working on solving different problems that their environment has presented to them use their basic intelligence procedure to come up with different kinds of intellectual systems finally share these innovations with each other leading to new combinations of innovations that might lead to some speed up in agriculture due to the engineering of advanced species, the agent is limited to the basic system of agriculture it has acquired.

Since the agent is as far away from any real biotechnology as the sun is far from the moon, the agent uses its basic intellectual capacity to recognize that certain areas were producing young trees that looked better overall. Not knowing what was directly responsible for this the agent goes ahead to cluster the seeds in this area of fertile growth until overcrowding overwhelms the young tree and by providence the agent sees that trees that were planted on soil with a particular colour and that didn’t have neighbours close by generally did better, these result might come from a long period of painful experimentation and providential accidents.

If an agent has applied intelligence successfully in this agricultural setting, it stores the knowledge of experimentation mentally in some abstract formalism and transfers that knowledge into the domain its sudoku problem.

Because it had experimented with agriculture and at least found out that planting in an area with deeper coloured soil produced better plants, it will have the definite understanding that it can always do better than what is readily available and with that understanding it might start tinkering through naive algorithm after algorithm and try to keep improving them until it arrives at something like a constraint satisfaction problem solver. If the same agent faced with an agricultural problem is now faced with a sudoku problem.

In the world of electronic computation where results can be obtained quickly, the agent will be able to iterate faster through different solutions than it would have been able to do with its agriculture system. This is because the computer declares results faster and it can see where it failed and use its intelligent process to device other means of arriving at the results faster than before. With a CSPS solving the sudoku problem at about 0.1 seconds, the agent enters a plateau of innovation and then it realizes that it might need external input from the abstract structures learnt by other agents if it ever needs to go beyond the CSPS, no different from the way it will need other agents’ knowledge to advance its biotechnology.

Intelligence is the most generalized data structure plus algorithm ever, it is actually a meta-algorithm because its purpose is to produce algorithms. If researchers keep this in mind, they can understand what they encode in their programs when they are trying to build artificial intelligence.

Earlier on I said that general intelligence is not some high-level program abstraction that tries to mimic the effect of intelligent processes like we were doing in the early days of AI research. General intelligence lies in the core data structure of the human mind, which is simply the NETWORK and all the algorithms that operate on this network structure.

The closest we have come to Artificial Intelligence is with the network of programs that we call Deep Neural Networks, this in some sense is reminiscent of what general intelligence looks like and just like the primitive machine instructions on the circuitry of computer systems look so different from the higher level abstractions that are used to create the fancy software that we use. The base level of what General Intelligence truly looks like is not some fancy high-level algorithm like we have been chasing all along but rather a very primitive system of nodes and edges alongside routines for altering the state of those nodes and edges.

Intelligence is not some sophisticated elegant system that we can see and say, hey! This is intelligence but rather it is as primitive as the meaningless (to unlooking human beings) activities that go on in an atom, a molecule or any substructure of nature.

We should have been looking at nature all along, how can simple stupid looking things like atoms become flamingos? Where is the underlying intelligence? We will see more of this discussion later on.

The early days of artificial intelligence, where GOFAI (Good Old-Fashioned Artificial Intelligence) was bae (before anyone else), involved programmers trying to do specialized programs to emulate tasks that will usually take human intelligence to do.

The near infinite capacity of computers to simulate any kind of system is what misled us initially in our earlier attempts at AI. If the computer could simulate any world, including worlds that could not possibly exist in the real world because it is just like a pen and paper and can provide us with a medium for conceiving impossible worlds, why don’t we go ahead and simulate human intelligence?

The proposition looked so natural nobody bothered to question if it was the right path to follow, we just forged right on to start replicating individual aspects of human cognition and hope that someday by some mysterious process the computer would just get up and start thinking.

We were dealing with the multiple-heads of a hydra-monster of a problem and not the root problem, we didn’t even know what the root problem of intelligence was because first of all we were handicapped by the reliance on our poor definition of the word intelligence and rather than get that the root and solve the problem we were focused on the leaves, the attributes of intelligence that was demonstrated in human action.

The field of mathematics should have inspired our efforts if we could look at it from the right dimension. Mathematics is built on foundations called axioms, and using an axiom system a proposition (Theorem) in mathematics could be verified through systematic proof.

If we had some inspiration from math we would have tried to invent a primitive axiom system for intelligence, and maybe we would have ended up discovering the power of network structures and gone ahead to build stuff like Deep Neural Networks earlier. Actually there was something about perceptrons which were invented by Frank Rosenblatt but because of the wild claims made about it, he had just discovered the machine code for intelligence but thought he had discovered the web browser, people tore the perceptron apart through rigorous research and came up with proofs about its inability to solve the XOR (exclusive OR) problem, research into the nature and capability of networks was delayed until much later.

This reminds us of humility and not making excessively wild claims when we make discoveries because most of the time discoveries are like open doors leading to a world of possibility and development. We don’t know the stage of solution our intuitions actually represent, we might be in the days of machine code toggled back and forth with electromechanical switches when we just come upon an idea.

This doesn’t mean we shouldn’t share our ideas, we should share our ideas but be humble about their immediate impact, time will reveal other aspects of our ideas that we cannot see currently as an individual agent participating in only local problem-solving.

The AI guys who were doing GOFAI stuff did not realize that what they were actually demonstrating with their programs was actually their own intelligence. The biggest invention in the field of creating algorithms that mimic human intelligence is deep learning. This is because it is the first-time human intelligence has created a generalized tool for doing in a mechanical way that which the human mind does in its default mode.

Deep learning architectures can be viewed like assembly language in computer programming abstraction, and just like fancy software like web browsers can be built from such simple fundamentals like machine language, assembly language and higher level languages. A cognitive application can be built from the primitives of deep learning which is just a collection of nodes and edges, essentially a network.

Deep learning is a specialization of general intelligence. If we view things from this angle we can see that everything fits in properly. Things like Deep Reinforcement learning, Generative Adversarial Networks, Recurrent Neural networks, Autoencoders, or any other application of Networks is just a specialization. Backpropagation is an algorithm that modifies the properties of a network, there could be other algorithms. The deepest level of intelligence, which we can call general intelligence is not really about any special kind of algorithm but more about representation. Once a problem can be represented in a particular form, it is only a matter of finding the right algorithms to manipulate that representation until we find a solution.

If we treat the most generalized representation for all kinds of problems which are networks, explicitly like it is done in neural networks rather than implicitly in most traditional programming we have a general intelligence that we have always sort after. It is only a matter of building other subsystems that transform any kind of problem into this generalized network representation and then manipulate the representation in a standard way to produce solutions.

Even whole programs like operating systems can be modelled as an interconnection of nodes and edges, some nodes containing functions and some containing data. Let's not get trapped by a mere specialization as we have with deep learning based on backpropagation and investigate deeply into networks because they hold the key. More research effort should be channelled into studying the properties of arbitrary networks, much like Stephen Wolfram attempts in his book A New Kind of Science. In that Scenario, Stephen tries to represent space as a Network. It is this kind of low-level primitive research that AGI researches should be doing, we don’t know what we may discover when we tinker with the lowers level of possibility.

We talked about coping algorithms that use simple pattern-recognize-generate-actions algorithms. This is the most basic algorithm that the human being possesses. It is so simple yet so powerful that even if the human being is developing an advanced algorithm for solving some task, the only thing it possesses to make that happen is the simple recognize pattern and perform action sequence. This is the algorithm of algorithms so therefore the discovery of deep learning is the most important leap ever in the entire field of algorithms.

Do not imagine that deep learning is the intelligence of a human being, it is only one part of the picture, a specialization of the most general case of arbitrary networks, yet it is powerful enough that it has enabled us to solve many problems for which it would be hideously difficult to hand code a solution an artefact of the days of GOFAI.

Remember that even though a human being has to recognize patterns, which deep learning does for us, it still has to generate meaningful action. This meaningful action can be as simple as recognizing food (pattern recognition phase) find a way to reach it and eat it (action phase).

Even this action phase can be complicated in itself and can involve a series of complex algorithms for initiating movement in a particular direction and angles. But concentrating on algorithms for generating mechanics is not very informative in the creation of the kind of generally intelligent agents that humanity dreams about. Rather we want a kind of disembodied system that resides in some artificial substrate like a computer that we can use to solve any kind of goal.

Our coping algorithms can be used for simple things like finding food or it can be applied to a task as sophisticated as recognizing positions on a chess board and generating a meaningful action which is, of course, the gameplay. This simple ability extends to even many complex abilities like: recognize pattern integrity in an experiment and develop a theory that says this activity is a law of nature.

We have been too focused on the action generation as the intelligence side of the coin and that preoccupation has prevented us from fully appreciating the power of the recognition phase, which required developing an appropriately general representation for solving pattern recognition problems codified in the discipline of deep learning. With deep learning, we are now able to develop a system that automatically performs the pattern recognition phase by extracting features out of the input representation and transforming this representation through various stages till we arrive at another representation that represents a cognized pattern. If we push this idea of representation a little bit we can even find a general way of essentially searching representation space for actions rather than hardcoding these actions. Deep reinforcement learning is a specialization of network systems that essentially allows us to do this search for a representation that encodes rational action.

The GOFAI people of the past focused all their attention on solving the problem of intelligence by building systems that acted intelligently when faced with certain problem scenarios. This was a practical approach that enabled the researchers to demonstrate fancy programs to the public. This approach put AI research in a dead end because after all the fancy toy programs have been demonstrated and displayed “interesting” properties, it was time to bring in return on investment because AI is a very expensive endeavour.

It was during this phase that the GOFAI researchers realized that they were dealing with leaves of the problem, not even a branch or an even close to the trunk talk less of the roots of the problem. This disappointment led to the disillusionment in AI research that led to the AI winter.

Backpropagation is the resurrector of modern AI activity. The passage of time allowed hardware capabilities to develop to the state that it was now possible to run backpropagation successfully, the AI winter was over and we entered spring, summer and now the autumn of backprop based AI aka deep learning. The over-enthusiasm about core deep-learning and its endless adaptations without looking into some basic theory work, the worship of only data and heavy duty computation without thinking about basic research and general cleverness will bring us into another AI winter sooner than we think.

The entire effort of GOFAI guys in hand coding systems to navigate complex environments has been surmounted by the deep learning guys with deep reinforcement learning. Vanilla reinforcement learning is involved with rewarding or punishing an agent in an environment for either making the right choices or wrong choices and optimizing the agent for the goal of gaining greater rewards. The agent eventually formulates a policy (action sequence) for acting in a way that brings in the most reward. If the policy is represented as a policy network, then using deep learning we can search for an optimal policy. Using this simple system we can implement many of the older AI systems that are not strictly dependent on perception alone, those that require that some agent generates some rational action.

Going back to our sudoku agent we can see that having reached the plateau of intelligence with the invention of CSPS, the agent progresses beyond this level using reinforcement learning. Meaning that rather than try to develop an algorithmically heavy approach to solving Sudoku such as Constraint Satisfaction, the agent just reformulates the problem and a Reinforcement learning problem and start evolving better and better policies that will enable it to achieve its goal in shorter timeframes and with the least effort.

Eventually, the agent invents self-play where a subsystem of its own design proposes sudoku games and it tries to generate many general policies for solving sudoku till it reaches a point in its evolution of intelligence that it can find any of 6,670,903,752,021,072,936,960 Sudoku game solutions that can ever be found.

The bewildering complexity of the human intelligence is simply due to the fact that there are so many patterns stored in the human mind and so many stored action algorithms that when the human being is choosing to do a particular task it is hard to separate the recognition step and the action step. To add to the already existing complexity, the human mind can use the most abstract generalization of the pattern of a bird in nature, to aid its imagination in synthesizing a completely new idea like artificial flight configuration.

This creative aspect of the mind is currently been approximated by a network called Generative adversarial networks, this kind of network uses a system of a generator paired with a discriminator to generate things that have less and less deviation from reality. But the aspect of the human mind we are interested in dissecting when it comes to abstracting concrete stuff away from the details has little with trying to generating super realistic data. It is more about tearing data to pieces and rearranging back in a new configuration after mixing it with data from other sources to solve a problem.


Comments

Popular posts from this blog

Software of the future

This powerful CEO works remotely, visits the office only a few times a year

At the edge of a cliff - Quantum Computing