Creating AGI roughly falls into two camps: sticking with current approaches to AI and extending them to greater scale, or striking out in new directions that have not been as extensively explored.
The dominant form of AI is the “deep learning” field within machine learning, where neural networks are trained on large data sets. Given the progress seen in that approach, such as the progression of OpenAI’s language models from GPT-1 to GPT-2 to GPT-3 and GPT-4, many advocate for staying the course.
Kurzweil, for example, sees AGI as an extension of recent progress on large language models, such as Google’s Gemini. “Scaling up such models closer and closer to the complexity of the human brain is the key driver of these trends,” he writes.
To Kurzweil, scaling current AI is like the famous Moore’s Law rule of semiconductors, by which chips have gotten progressively more powerful. Moore’s Law progress, he writes, is an instance of a broad concept coined by Kurzweil, “accelerating returns.” The progress in Gen AI, asserts Kurzweil, has shown even faster growth than Moore’s Law because of smart algorithms.
Programs such as OpenAI’s DALL*E, which can create an image from scratch, are the beginning of human-like creativity, in Kurzweil’s view. Describing in text an image that has never been seen before, such as, ” A cocktail glass making love to a napkin,” will prompt an original picture from the program.
Also: Generative AI is the technology that IT feels most pressure to exploit
Kurzweil views such image generation as an example of “zero-shot learning”, when a trained AI model can produce output that is not in its training data. “Zero-shot learning is the very essence of analogical thinking and intelligence itself,” writes Kurzweil.
“This creativity will transform creative fields that recently seemed strictly in the human realm,” he writes.
But, neural nets must progress from particular, narrow tasks such as outputting sentences to much greater flexibility, and a capacity to handle multiple tasks. Google’s DeepMind unit created a rough draft of such a flexible AI model in 2022, the Gato model, which was followed the same year by another, more flexible model, PaLM.
Larger and larger models, argues Kurzweil, will even achieve some of the areas he considers deficient in Gen AI at the moment, such as “world modeling”, where the AI model has a “robust model of how the real world works.” That ability would allow AGI to demonstrate common sense, he maintains.
Kurzweil insists that it doesn’t matter so much how a machine arrives at human-like behavior, as long as the output is correct.
“If different computational processes lead a future AI to make groundbreaking scientific discoveries or write heartrending novels, why should we care how they were generated?” he writes.
Again, the authors of the DeepMind survey emphasize AGI development as an ongoing process that will reach different levels, rather than a single tipping point as Kurzweil implies.
Also: 8 ways to reduce ChatGPT hallucinations
Others are skeptical of the current path given that today’s Gen AI has been focused mostly on potentially useful applications regardless of their “human-like” quality.
Gary Marcus has argued that a combination is necessary between today’s neural network-based deep learning and the other longstanding tradition in AI, symbolic reasoning. Such a hybrid would be “neuro-symbolic” reasoning.
Marcus is not alone. A venture-backed startup named Symbolica has recently emerged from stealth mode championing a form of neuro-symbolic hybrid. The company’s mission statement implies it will surpass what it sees as the limitations of large language models.
“All current state of the art large language models such as ChatGPT, Claude, and Gemini, are based on the same core architecture,” the company says. “As a result, they all suffer from the same limitations.”
The neuro-symoblic approach of Symbolica goes to the heart of the debate between “capabilities” and “processes” cited above. It’s wrong to do away with processes, argue Symbolica’s founders, just as philosopher Searle argued.
“Symbolica’s cognitive architecture models the multi-scale generative processes used by human experts,” the company claims.
Also skeptical of the status quo is Meta’s LeCun. He reiterated his skepticism of conventional Gen AI approaches in recent remarks. In a post on X, LeCun drew attention to the failure of Anthropic’s Claude to solve a basic reasoning problem.
Instead, LeCun has argued for doing away with AI models that rely on measuring probability distributions, which include basically all large language models and related multimodal models.
Instead, LeCun pushes for what are called energy-based models, which borrow concepts from statistical physics. Those models, he has argued, may lead the way to “abstract prediction”, says LeCun, allowing for a “unified world model” for an AI capable of planning multi-stage tasks.
Also: Meta’s AI luminary LeCun explores deep learning’s energy frontier
Chalmers maintains that there may be “greater than 20% probability that we may have consciousness in some of these [large language model] systems in a decade or two.”
