A new
New kind of AI uses a fresh approach to reasoning — researchers say it costs up to 11 times less to run than a leading OpenAI model
A transformer model forms its responses to user queries by looking at the full prompt simultaneously and then predicting what the next word in the sequence of its reply should be. It does this word by word, using natural language to effectively verbalize a linear train of thought in the background. Transformers’ reasoning also functions sequentially, meaning they have to work through each stage of a problem in a strict linear order.
These models have significant advantages over earlier architectures, which would often forget the start of an input by the time they reached the end. However, transformer architectures can struggle with longer or more complex prompts, as the computational complexity of evaluating the prompt increases quadratically — meaning that doubling the length of an input uses four times as much processing power.
AI model usage is measured on a per-token basis, with a token representing any data fragment (equivalent to roughly four characters of text) that the AI has to ingest or output. Because more complex prompts require longer trains of thought with multiple steps, processing and responding to these queries can burn through significant amounts of tokens.
AI’s next generation?
Conventional transformer-based token generation is prone to causing memory bottlenecks, as AI re-reads every previous word in the conversation with every new word generated. Eventually, this will clog up the memory in the graphics processing units (GPUs) used for AI operations.
Because of this, scaling AI reasoning has become an expensive computational challenge. Pathway’s post-transformer approach changes how the AI’s memories of a conversation and the relationship between pieces of information are stored and processed. It replaces text logs with new tools, including an improved short-term memory and a mechanism that allows it to work through problems without consuming tokens.
Transformer-based models retain prompts and interaction histories as a long string of numerical values representing the text of requests. That string then expands as new tokens are added through processing the request. BDH-CQ uses numerical arrays to represent the underlying rules and contextual patterns of a task, using numbers to track relationships between chunks of information rather than defining them in text.
These arrays represent vectors — directional information that points to another point on a theoretical map stored inside the GPU’s memory as part of the training data, implanted during the model’s creation. The scientists said in the study that this allows the model to process complex abstract reasoning without increasing its memory footprint or computational cost.
To execute tasks, BDH-CQ implements a “latent reasoning engine” as its internal workspace. Using numbers to represent the different elements of a prompt or problem, it carries out a series of iterative recurrent loops to determine the best answer to return based on the prompt. The model takes the output of the last loop, assesses how the result could be improved based on its training data, and feeds back the previous output as the starting point for the next iteration. It repeats this for a pre-set number of loops, with each iteration theoretically closer to the desired outcome.
To tackle more complex problems requiring more thinking time, BDH-CQ can execute more loops. This increases the time taken, but the amount of memory and computational power consumed does not scale with more attempts — in theory, the model would consume a consistent proportionality of memory and power running 200 loops as it would running 20 loops. Standard transformer models, by contrast, achieve extra thinking time by generating long chains of written text tokens, which exponentially consumes GPU memory and computing power across an AI cluster.
Related stories
- AI found a weakness in one of the world’s most studied encryption systems — is your data under threat?
- AI may accelerate scientific progress — but here’s why it can’t replace human scientists
- AI could soon think in ways we don’t even understand, increasing the risk of misalignment — scientists at Google, Meta and OpenAI warn
The model’s ARC-AGI-1 benchmark results have been independently verified and reproduced by prominent researchers in the AI field, including NYU researcher Richard Zhong, and Łukasz Kaiser, a co-author of the seminal 2017 paper “Attention Is All You Need,” which introduced the concept of transformers within large language models.
“I’ve followed Pathway closely and replicated their ARC-AGI-1 results myself,” Kaiser said in a statement. “Pathway shows that model architecture, not just scale, can drive the next leap in AI reasoning.”
Pathway plans to scale the BDH architecture up to 600 billion parameters and apply its vector-based reasoning to more challenging benchmarks, such as ARC-AGI-2 and ARC-AGI-3, as well as develop a fully-fledged large language model (LLM) based on the technology, which would provide a basis for building text-based chatbots. The company hopes the technology can be applied to complex reasoning problems in sectors such as cybersecurity incident response and industrial operations.
