Skip to main content

The power of symbols

A symbol is a representation of some underlying complexity. It compresses information by allowing us to deal with a more compact representation of some underlying structure than the whole structure itself.



The human mind is at best a symbol processing machine because it maps symbols to some underlying chunk of data to ease management when doing thinking.

Without this ability to represent chunks of data with symbols we wouldn't even be able to communicate efficiently via language. If two entities understand the meaning of a symbol then they can communicate with this symbol rather than the actual data the symbol maps to.

We can talk about apples by simply mentioning the sound apple or writing it down with character symbols rather than dealing with an actual apple in nature. This efficiency of representation is at the core of what we do in our thinking. When you are thinking you are not dealing with things in nature but the symbolic representation of these things.

To be used efficiently a symbol should be expressed with fewer bits than the data it represents. By data, I mean the full description of the thing we want to represent by the symbol. For instance, if we could come up with a complete description of an apple, then that description is the data that is mapped to the symbol of the character sequence "apple" or the sound of apple.

Theoretically the complete description of an apple is perfectly equivalent to the most general description of an apple, actually, we do not really need to "describe" the apple because our attempts to describe the apple in any language would be akin to creating a symbolic representation of the apple.

We must assume that the perfect "description" of the apple is the apple itself, or the most generalized apple possible, which of course is not physically realizable. Anyway when we think or communicate we are passing on symbols back and forth. Symbols can be communicated with some exactness but the data represented by the symbol can be interpreted in a variety of ways between communicating entities and even within the mind of a thinker, this means that the way you think of an apple at this moment will be different from the way you think about it at the next moment. What persists is the symbol but the data usually varies slightly.

We must also know that despite being a compressed representation of some underlying data, this is just a mapping between symbol and data and it is not always possible to generate the full data from its compressed representation as a symbol. But as far as there is a system of assigning symbols to data which makes sure that there is no duplication of symbols pointing to the same data, then we can take the symbol to be a full representation of the data even though the symbol does not save on space by performing real compression of the underlying data.

Our process of thinking is some kind of symbolic manipulation, we don't work with data itself but models of these data. These models are symbolic representations of the data albeit possessing some real compression, i.e. The models exist for more than just an arbitrary mapping between symbol (model) and data, the models behave like the data to a high degree of precision because the compression process of the mind conserves structure even without caring more about space-saving.

The data hitting our nervous system is raw and repetitious, but the symbols we think about are more definite than the massive electrical currents running from our senses to our brains when we observe something.

The symbols we generate in our minds are a transformative representation of the raw data we receive through our senses. We think with these symbols not with the raw data we are receiving from the outside. This is why deep-learning is so powerful, it is currently our most powerful attempt at representing data within powerful models, weight-matrices.

In our minds we can create higher-level symbols from base symbols, enabling us to go from simple things like some symbolic representation of a physical apple to the apple logo which symbolically represents other things that are not directly observable as atomic agglomerations in nature.

Using the base hardware circuitry of our neurons, we can create higher and higher levels of abstractions that represent high level concepts than we can rawly observe in nature, and we can combine these symbols to create new symbols and perform all manner of operations on the symbols, this is what we do when we are thinking.

Comments

Popular posts from this blog

Virtual Reality is the next platform

VR Headset. Source: theverge.com It's been a while now since we started trying to develop Virtual Reality systems but so far we have not witnessed the explosion of use that inspired the development of such systems. Although there are always going to be some diehard fans of Virtual Reality who will stick to improving the medium and trying out stuff with the hopes of building a killer app, for the rest of us Virtual Reality still seems like a medium that promises to arrive soon but never really hits the spot.

Next Steps Towards Strong Artificial Intelligence

What is Intelligence? Pathways to Synthetic Intelligence If you follow current AI Research then it will be apparent to you that AI research, the deep learning type has stalled! This does not mean that new areas of application for existing techniques are not appearing but that the fundamentals have been solved and things have become pretty standardized.

New Information interfaces, the true promise of chatGPT, Bing, Bard, etc.

LLMs like chatGPT are the latest coolest innovation in town. Many people are even speculating with high confidence that these new tools are already Generally intelligent. Well, as with every new hype from self-driving cars based on deeplearning to the current LLMs are AGI, we often tend to miss the importance of these new technologies because we are often engulfed in too much hype which gets investors hyper interested and then lose interest in the whole field of AI when the promises do not pan out. The most important thing about chatGPT and co is that they are showing us a new way to access information. Most of the time we are not interested in going too deep into typical list-based search engine results to get answers and with the abuse of search results using SEO optimizations and the general trend towards too many ads, finding answers online has become a laborious task.  Why people are gobbling up stuff like chatGPT is not really about AGI, but it is about a new and novel way to rap