“Training” Isn’t Learning: Demystifying AI

This article is the third in my On Building Intelligence series, which aims to foster a realistic understanding of AI’s potential and limits. Of particular importance is how conflating technical AI terms and everyday language can obscure emerging risks and opportunities. Here, I focus on one such conflation: treating neural network “training” as if it were human-like “learning.” I hope to show why precision in terminology matters—especially for leaders in business and government making high-stakes decisions.

The term “learning” has been applied to neural networks since their inception. Prior to the advent of transformer based generative models, the term of art for neural network systems was “Machine Learning”. Within the industry, data scientists use the term “training” for the way in which data is used to create a neural network based model, but for the most part those same data scientists are just as likely to conflate this with “learning”.  So what does AI training entail and how does it differ from human learning?

AI Model Training

Digital neural networks are trained using a curated data set. Machine Learning type models are usually designed to accept a flow of data from a selected source and produce a prediction or classification. For these models, the training data is typically a previously collected set of data from that source for which a clear result is known. In contrast, transformer based, generative models draw  from many sources based on the goals of the training. For so-called Foundational Models, the very large LLMs such as ChatGPT that are intended to be responsive to a broad range of queries, this curation is a massive exercise in collection from a very broad variety of sources. ChatGPT and its brethren don’t autonomously “read the internet” to learn, instead, automated systems and human teams assemble the training data. The volume of this data, and the effort involved, are enormous. ChatGPT 4 was trained on a dataset equivalent to all of the books published in the last 100 years. The important thing here is that  the curation is done prior to training and the data doesn’t change as training proceeds. The data is applied iteratively to the model, with the error in the output being fed back to the input until the model converges on producing “correct” answers. These may be correct predictions or classifications for Machine Learning models, or correctly reproducing the input for LLMs. What makes this iterative is that though the training data doesn’t change, negative feedback passes the error back to the input. Iteration ends when the state of the system converges. Once trained, the neural network is deployed, the inference phase, where it is available to infer responses to queries. During the inference phase, the neural network doesn’t change.

Human Learning

Human learning, by contrast, is self-curating and recursive. Unlike digital neural networks, we don’t rely on a static dataset or stop once a certain output is achieved. New information is continuously absorbed from direct experiences, which we refine and reinterpret over time.

There is a form of model training referred to as Reinforcement Learning that is used in many autonomous systems and has some characteristics of recursive learning. I’ll come back to that later, for now let's focus on human learning.

To clarify, consider how a human brain acquires information and learns. What follows is not intended as a full accounting of that process, but should illuminate key aspects.

Human memory is not like machine memory, it’s not a simple, addressable data storage mechanism. Memory and information processing in the brain are deeply intertwined, and share a common neural substrate. This is true of digital neural networks as well, but the way in which the two obtain and instantiate information are quite different. In the human brains, memory and information processing operate in two distinct and interoperating modalities. These are often referred to as Episodic and Semantic memory, and they operate in physically separate parts of the brain, the hippocampus and neocortex, respectively.  Semantic Memory is, in a sense, pure information. When I answer a question, for example, about the height of Mount Everest (29,000 ft), that is Semantic Memory at work. My writing this article is an exercise of my Semantic Memory and associated Semantic Reasoning. Episodic memory, by contrast, is about recording experience, things that happened to YOU. It is concerned with the subjective experience of the world and your ability to recall, not just what happened, but how it felt. I would argue that the entire concept of “Qualia”, the brain’s expression of experience through a form of re-experience (e.g. when I recall the cup of coffee I had this morning and can feel its heat and taste) is rooted in Episodic memory structures. For the purposes of this discussion, I’m less concerned with exactly how each of these work on their own  and more concerned with how these two memory modalities work together to enable human learning and cognition that is distinct from “training”.

How Episodic and Semantic Memory Work Together

Here’s an overview of how this works:

• From Raw Experience to Curated Knowledge: Humans learn through raw sensory input—vision, hearing, touch, taste, smell. Each experience is recorded in an episodic format, capturing rich contextual cues. This initial recording  can be seen as a self-generated “curation” step. We pay attention to (and thus encode) some details more than others—based on salience, emotional impact, and personal relevance.
• Semanticization of Episodic Memory: Over time, the brain “distills” episodic details into semantic knowledge that can be recalled independently of the original context. Memory consolidation (particularly during sleep) supports this transition, transforming episodic traces into more robust semantic structures that can be recalled independently of the original context
• Open Ended Semantic Updates: Instead of an iterative optimization with a final convergence point, semantic knowledge is updated continuously, over a lifetime, as experiences accumulate and prior beliefs are tested and revised. The process is never complete, because we are always open to new information that might refine or overhaul our existing semantic frameworks.
• Recursive Reinforcement through New Experiences and Reflection: Unlike a fixed training dataset, human learning is inherently recursive. We continue to gather new experiences (fresh episodic data) while simultaneously refining our semantic knowledge. Each new episode can confirm, challenge, or augment existing semantic structures. This dynamic interplay means the line between “training” and “inference” is continually blurred: as we deploy our semantic knowledge in real-world tasks, we simultaneously gather new experiences that reshape that knowledge. The creation of new semantic knowledge is itself an episodic experience, as is the act of recalling and applying semantic knowledge. We remember that we learned something, and what we thought before we learned it. This means that semantic knowledge is subject to review and reflection and can continue to evolve even absent new raw data.

Together, these mechanisms demonstrate why human cognition remains open-ended and adaptive in ways AI training does not.

Self Reflection and Metacognition

As noted, one reason humans can capitalize on this deep recursion is our capacity for self-reflection and metacognition:

1. When we use semantic knowledge, we can monitor (consciously or subconsciously) how well it serves our goals.
2. If we detect a mismatch—errors, inefficiencies, confusion—that discrepancy itself becomes an experience we can learn from.
3. We then adjust our future approach or refine our mental models.

Consider the role of “Doubt” in human cognition as an example of how self-reflection and metacognition play out. Doubt can be thought of as a self generated error term that can be applied tentatively and experimentally, with both the doubt and pre-doubt learning available for A-B testing until the doubt is resolved. This ‘error term’ is more than a numerical mismatch; it’s a qualitative, subjective sense that something may be wrong. It’s common in literature to encounter a character who engages in colorful arguments with themselves in order to work through some doubt, considering first the one hand, and then the other hand. During these arguments, the character simultaneously holds multiple semantic constructs in his or her mind. This rich internal dialogue relies on episodic memory to track and evaluate each ‘version’ of the network.  A purely semantic cognition has no capacity for doubt.

Contrasting Training and Human Learning

Here’s how each core feature of human learning stands in contrast to current neural network training:

1. Externally Curated Datasets vs. Self-Generated Experiences: Data (e.g., text corpora, labeled images) is pre-collected, pre-processed, cleaned, and tokenized. The network does not experience the world directly; instead, it passively receives a curated feed of examples.
2. Iterative Optimization vs. Recursive Updates: The network parameters are adjusted iteratively until the model converges on a state that (hopefully) generalizes well to new data. Training is often a single large-scale phase, after which the model is “deployed.” Although in some newer paradigms models can be fine-tuned or updated, there is still a strong separation between “training time” and “inference time.”
3. Lack of “Personal” Episodic Traces: Digital neural networks do not store episodic memories in the human sense; they store learned weights and representations optimized across the training corpus. While some architectures attempt to simulate episodic memory with external systems (e.g., memory-augmented neural networks, retrieval-based transformers), the notion of personally contextualized, time-stamped, emotional, or self-referential memory is absent or only very loosely approximated.
4. Static vs. Dynamic Experience: Most large-scale AI models are trained with a static dataset before deployment. Humans, by contrast, are continuously immersed in real-time experiences that become new episodic memories. Models can be re-trained or fine-tuned on updated data, but each re-training cycle is usually a discrete iteration rather than an ongoing, inseparable process from day-to-day “existence.”

These distinctions highlight why neural networks, however sophisticated, remain fundamentally limited compared to the flexible, self-aware learning of humans.

A Closer Look at Reinforcement Learning

As mentioned earlier, Reinforcement Learning (RL) is used in many autonomous systems and has some characteristics of recursive learning. It is a paradigm in which agents learn by receiving feedback (rewards or penalties) from actions taken in an environment. Like human learning, RL models can update their state throughout their lifecycle.  

RL frameworks allow agents to adapt by trial-and-error, updating internal parameters when outcomes diverge from predictions. Although this seems more dynamic than static supervised learning, it still relies on iterative updates rather than true episodic-to-semantic recursion. The agent lacks a higher-order mechanism to restructure experiences into conceptual knowledge, relying instead on raw reward signals and a replay buffer that stores past decisions. These stored experiences lack the contextual richness and self-referential quality of human episodic memory.

One other note about RL models is that they tend to be dramatically smaller than transformer based neural networks. There are a number of reasons for this but it is not the case that it is simply a design decision. The barriers to using RL at scale are very significant. The main issues are:

• Data collection (environment interactions) is expensive and slow.
• Learning stability is fragile with large, complex models and non-stationary data.
• Compute requirements grow quickly when you combine large models with the need for millions of environment samples.

So, while RL techniques may eventually apply to large scale models, there are inherent difficulties, the costs may be enormous and the effective results less impressive than expected.

Conclusion: Why Precision Matters

Information stored in current digital neural network models is purely semantic in nature and, with the exception of RL models, a rigid separation is maintained between “training time” and “inference time.” Episodic understandings, and the recursive interplay between Episodic and Semantic knowledge, are completely absent. The self-reinforcing dance that characterizes human cognition  and is one reason humans can continually adjust and deepen our knowledge base is not available.  For Human brains, the act of using knowledge is inseparable from the act of generating new experiences, which in turn reshapes that knowledge. The question of whether a digital neural network architecture that possesses this ability is possible is an open one, and not likely to be answered by incremental improvements in AI technology. It remains essential to distinguish the engineering term “training” from genuine “learning” to avoid overestimating AI’s capabilities—and underestimating the power of human cognition. Failing to do so can warp expectations, undercut strategic planning, and ultimately skew the conversation about AI’s true potential and risks.

#AI #AGI #Artificial_Intelligence #Neural_Network