You need to jump dimensions to solve certain problems

 

In the image caption above you see the solution to the equation given in the first line in the second line. A simple looking innocuous equation doesn't have a solution in the real line. In order to solve it, we have to jump into the imaginary line. 

I am always amazed by this equation because it informs us of certain deep realities that we might not usually contemplate when we are going about finding the solutions to other problems we encounter in daily life.

While the typical definition of dimensionality involves the length of a vector, I think jumping from the reals to the imaginary is a dimensional jump in some sense.

So why is it important to put this in mind? There are many scenarios where one can solve a problem in the same plane where the problem is found like there are many equations for which there is a solution in the real line. But one must be vigilant about those problems that require a dimensional jump because every now and then they will crop up in all kinds of systems.

The main motivation for this post is because I think the stagnation we are experiencing in deep learning research despite inventions like GPT-3 requires a massive dimensional shift to overcome. 

We might need to use the tools of pure mathematics to really understand more of what deep learning systems are doing and thus make the needed jump or else we will eventually use the whole planet as a computer to run very weak algorithms.