The attempt to reverse engineer the human brain as a means to achieve Artificial Intelligence is an exercise in futility. Because of the abundance of computing power, modern researchers seem to be less and less creative than their counterparts in the early times of the 20th century. Everything these days seem to be about getting more data, cleverness and creativity seem to have taken the back seat as brute force approaches appear to be the best solution to every problem.
Where are the Alan Turings, the Kurt Gödel and the Shannons? The urge to publish or die is turning out low quality research paper after paper. To solve the current problem of AI we have to dig deep into our creative selves and stop worshipping data, look at algorithms like back propagation, they were invented in the 70s, what are we inventing now? Constantly tweaking the creative fruits of the past. How does something like Wolfram Alpha work? Mountains of innovative algorithms working together, it computes like the brain but it is not based on reverse engineering the brain or anything like that, it is the work of pure creative research.
This is the direction we should take; deep learning is doing great things but let’s not be stuck on any kind of method or technique because we think it must be a network to be able to learn. We should explore far and wide, we should try out new models, new approaches, even if in the end we still find networks maybe a more efficient way to learn would be better than spending weeks training some deep network.