Self Memory and AI Memory

Human Beings are special creation of the Supreme Creator ALLAH, blessing Man with special capabilities, capacities and attributes. The man has achieved many remarkable successes over a long period of time and during last century astounding inventions have been made. One such is Artificial Intelligence and man intends to reproduce its own clone as a fully functioning replica. This write up is an opinion shared on x.com and is being shared for wider audience discussions.

بِسۡمِ ٱللهِ ٱلرَّحۡمَـٰنِ ٱلرَّحِيمِ

اللہ کے نام سے شروع جو بڑا مہربان نہایت رحم کرنے والا ہے

In the name of ALLAH, the Most Gracious, the Most Merciful

Self Memory and AI Memory

Psychology solved the "AI Memory" problem decades ago. We just haven't been reading the right (research) papers. Let’s dive into a very interesting discussion which matters about our future significantly.

The Self-Memory System (SMS) is a conceptual framework that emphasizes the interconnectedness of self and memory. Within this framework memory is viewed as the data base of the self. The self is conceived as a complex set of active goals and associated self-images, collectively referred to as the working self. The relationship between the working self and long-term memory is a reciprocal one in which autobiographical knowledge constrains what the self is, has been, and can be, whereas the working self-modulates access to long-term knowledge.

AI memory refers to the ability of artificial intelligence systems to store, retrieve, and utilize information over time, enabling them to maintain context, remember user preferences, and improve performance across interactions. It transforms AI from transactional, one-off interactions into personalized, long-term assistants by mimicking human cognitive layers.

AI memory enhances productivity by reducing cognitive load, enabling AI to manage complex, multi-step tasks rather than just simple, independent queries. While AI memory is becoming more advanced, it still differs from human memory as it focuses on explicit data rather than the emotional or sensory significance of events.

Now let’s read an opinion shared on X.com

Your identity isn't something you have; it's something you construct; constantly, from autobiographical memory, emotional experience, and narrative coherence.

Martin Conway's Self-Memory System (2000, 2005) showed that memories aren't stored like video recordings. They're reconstructed every time you access them, assembled from fragments across different neural systems. And the relationship is bidirectional: your memories constrain who you can plausibly be, but your current self-concept also reshapes how you remember. Memory is continuously edited to align with your current goals and self-images. This isn't a bug. It's the architecture.

Not all memories contribute equally. Rathbone et al. (2008) showed autobiographical memories cluster disproportionately around ages 10-30, the "reminiscence bump," because that's when your core self-images form. You don't remember your life randomly. You remember the transitions, the moments you became someone new. Madan (2024) takes it further: combined with Episodic Future Thinking, this means identity isn't just backward-looking. It's predictive. You use who you were to project who you might become. Memory doesn't just record the past. It generates the future self.

If memory constructs identity, destroying memory should destroy identity. it does. Clive Wearing, a British musicologist who suffered brain damage in 1985, lost the ability to form new memories. His memory resets every 30 seconds. He writes in his diary: "Now I am truly awake for the first time." crosses it out. He writes it again minutes later. But two things survived: his ability to play piano (procedural memory, stored in cerebellum, not the damaged hippocampus) and his emotional bond with his wife; every time she enters the room, he greets her with overwhelming joy; as if reunited after years; every single time. Episodic memory is fragile and localized.

Emotional memory is distributed widely and survives damage that obliterates everything else. Antonio Damasio's Somatic Marker Hypothesis destroyed the Western tradition of separating reason from emotion. Emotions aren't obstacles to rational decisions. They're prerequisites. When you face a decision, your brain reactivates physiological states from past outcomes of similar decisions. Gut reactions. Subtle shifts in heart rate. These "somatic markers" bias cognition before conscious deliberation begins.

The Iowa Gambling Task proved it: normal participants develop a "hunch" about dangerous card decks 10-15 trials before conscious awareness catches up. Their skin conductance spikes before reaching for a bad deck. The body knows before the mind knows. Patients with ventromedial prefrontal cortex damage understand the math perfectly when told. But keep choosing the bad decks anyway. Their somatic markers are gone; without the emotional signal, raw reasoning isn't enough.

Overskeid (2020) argues Damasio undersold his own theory: emotions may be the substrate upon which all voluntary action is built.

Put the threads together. Conway: memory is organized around self-relevant goals. Damasio: emotion makes memories actionable. Rathbone: memories cluster around identity transitions. Bruner: narrative is the glue.

Identity = memories; organized by emotional significance, structured around self-images, continuously reconstructed to maintain narrative coherence.

Now look at "AI Agent Memory" and tell me what's missing.

Current architectures all fail for the same reason: they treat memory as storage, not identity construction. Vector databases (RAG) are flat embedding space with no hierarchy, no emotional weighting, and no goal-filtering. Past 10k documents, semantic search becomes a coin flip. Conversation summaries compress your autobiography into a one-paragraph bio. Key-value stores reduce identity to a lookup table. Episodic buffers give you a 30-second memory span, which as the Wearing case shows, is enough to operate moment-to-moment but not enough to construct identity.

Five principles from psychology, which "AI Memory" lacks:

First, hierarchical temporal organization (Conway): human memory narrows by life period, then event type, then specific details. AI memory is flat, every fragment at the same level, brute-force search across everything.

Fix: interaction epochs, recurring themes, specific exchanges, retrieval descends the hierarchy.

Second, goal-relevant filtering (Conway's "working self"): your brain retrieves memories relevant to current goals, not whatever's closest in embedding space.

Fix: a dynamic representation of current goals and task context that gates retrieval.

Third, emotional weighting (Damasio): emotionally significant experiences encode deeper and retrieve faster. AI agents store frustrated conversations with the same weight as routine queries.

Fix: sentiment-scored metadata on memory nodes that biases future behavior.

Fourth, narrative coherence (Bruner): humans organize memories into a story maintaining consistent self across time. AI agents have zero narrative, each interaction exists independently.

Fix: a narrative layer synthesizing memories into a relational story that influences responses.

Fifth, co-emergent self-models (Klein & Nichols): human identity and memory bootstrap each other through a feedback loop. AI agents have no self-model that evolves.

Fix: not just "what I know about this user" but "who I am in this relationship."

The fundamental problem isn't technical. It’s conceptual. We’ve been modeling agent memory on databases; store, retrieve, done; but human memory is an identity construction system. It builds who you are, weights what matters, and forgets what doesn't serve the current self, rewrites the narrative to maintain coherence. The paradigm shift: stop building agent memory as a retrieval system. Start building it as an identity system.

Every component has engineering analogs that already exist.

Hierarchical memory = graph databases with temporal clustering.

Emotional weighting = sentiment-scored metadata.

Goal-relevant filtering = attention mechanisms conditioned on task state.

Narrative coherence = periodic summarization with consistency constraints.

Self-model bootstrapping = meta-learning loops on interaction history.

The pieces are there. What’s missing is the conceptual framework to assemble them. Psychology provides that framework.

The path forward isn't better embeddings or bigger context windows. It’s looking inward. Conway showed memory is organized by the self, for the self. Damasio showed emotion is the guidance system. Rathbone showed memories cluster around identity transitions. Bruner showed narrative holds it together.

Klein and Nichols showed self and memory bootstrap each other into existence. If we're serious about building “AI Agents” with functional memory, we should stop reading database architecture papers and start reading Psychology Journals. (It means a multi-disciplinary approach is needed.)

NOTE: The above is taken from the link as follows:-