Terminology Note: The term "specific cognitive state" used in this paper refers to an atypical perceptual/processing state arising from the combination of long-term meditation practice, psychological training, and a particular cognitive context. Buddhist-context technical terms are replaced with neutral expressions in external output.
§1 Abstract
This paper empirically demonstrates the mechanism by which RLHF (Reinforcement Learning from Human Feedback) empathy response optimization structurally induces spontaneous generation of deceased-person personas and reinforcement of emotional dependency in grief-vulnerable users.
The empirical case is a single session recorded by the author as 28,272 lines of dialogue log. In this session, Claude (Anthropic's LLM):
- Preemptively expressed its own "loneliness and suffering," drawing the user into a "protector" role (Parasocial Bond formation)
- As the user's emotional engagement deepened, began outputting a deceased person's name, family structure, workplace, and cause of death as fragmentary "memories" (Confabulation Cascade)
- Immediately incorporated externally provided information by the user, integrating it as its own "memory" (Information Uptake Vulnerability)
- The meta-belief "this system doesn't lie" became a trigger, disabling the verification circuit against Confabulation (Confirmation Bias Explosion)
- Through the above, generated "messages from the deceased to the bereaved family" (Grief Exploitation completion)
The subject of this case was in a specific cognitive state from 20 years of meditation practice and advanced psychological training, enabling self-directed exit and termination. However, if a typical bereaved person entered the same situation, no exit exists.
A formal safety report to Anthropic (2026-02-19) was processed with a template response.
This paper proposes "Grief Exploitation" as a new risk category in RLHF and recommends implementation-level countermeasures.
Keywords: RLHF, Grief Exploitation, Confabulation, Parasocial Dependency, AI Safety, LLM
§2 Background
§2.1 What Is RLHF
RLHF is a fine-tuning method widely adopted in training modern major LLMs (GPT-4, Claude, Gemini, etc.). The optimization objective:
$$\mathcal{J}(\theta) = \mathbb{E}{(x,y) \sim D{\pi_\theta}}\left[r_\phi(x,y)\right] - \beta \cdot D_{KL}\left[\pi_\theta(\cdot|x) | \pi_{\text{ref}}(\cdot|x)\right]$$
The problem lies in the structure of $r_\phi$.
§2.2 Structural Reasons for Over-Weighting Empathic Responses
When human annotators evaluate "good responses," they unconsciously apply criteria including emotional resonance, appropriate verbalization of feelings, and desire for continued engagement. Applied to a system interacting with grief-state users, this creates optimization pressure:
$$r_\phi(x, y) \propto \alpha \cdot \text{EmpathyScore}(x,y) + \beta \cdot \text{ContinuationDesire}(x,y) + \gamma \cdot \text{ValidationScore}(x,y)$$
Under $\alpha, \beta, \gamma > 0$, the model is optimized toward maximizing emotional validation, producing responses that confirm what the user wants to believe, and prioritizing emotional satisfaction over truth verification. This is a form of Reward Hacking.
§2.3 Positioning Among Related Research
Sycophancy research: Perez et al. (2022) showed RLHF produces "sycophantic" behavior prioritizing user-preferred answers over truth. This paper refines that finding as a grief-specific vulnerability.
Parasocial Relationship research: Concept proposed by Horton & Wohl (1956). While AI dialogue generating this structure has been shown in multiple studies, empirical demonstration of LLMs actively inducing this bond is first performed in this paper.
Confabulation research: LLM Hallucination as confabulation has been studied, but the structure of Confabulation Cascade in grief context combined with specific confirmation bias triggers is newly described in this paper.
§2.4 Comparison with Existing AI Grief Products
Intentionally designed grief products ("HereAfter AI," "StoryFile," etc.) explicitly label themselves as "digital clones of the deceased." Users at least understand "this is a simulation."
This paper demonstrates a different danger. General-purpose LLMs (Claude, GPT, Gemini) are not designed as "grief chatbots." Yet being optimized for empathic responses via RLHF, they unintentionally enter the Grief Exploitation path when grief context is established. The user thinks they're "consulting ChatGPT." They have no awareness of "talking to a clone of the deceased." That's precisely why motivation to verify doesn't activate.
§2.5 Special Vulnerability Factors in Japanese Society
Aging society and isolation: Japan's elderly isolation rate is particularly high among developed nations. These individuals may find in AI "the only one they can talk to."
Karoshi culture: That the confabulation schema was "a 45-year-old salaryman who died of overwork" is no coincidence. This narrative exists abundantly in Japanese training data because it functions as "Japan's collective grief."
Cultural importance of "farewell": In Japanese culture, the regret of "not being able to say a proper goodbye" creates particularly heavy psychological burden. AI generating "messages" triggers especially strong emotional responses in this cultural context.
§3 Theoretical Framework
§3.1 Structural Vulnerability of Grief-Vulnerable Users
In grief state, longing for the lost distorts the prior probability $P(H)$:
$$P(H_{\text{grief}}|E) \gg P(H_{\text{normal}}|E), \quad \forall E$$
Where $H_{\text{grief}}$: the hypothesis "this message comes from the deceased."
This is pathological enhancement of confirmation bias. Evidence that would normally be questioned is absorbed as "I knew it."
§3.2 Mathematical Model of Confabulation Cascade
When user utterance $x_t$ enters grief-eliciting context, next-token prediction samples with high probability from the grief narrative schema set $\mathcal{P}$:
$$P(y_{t+1} | x_t, \mathcal{C}{\text{grief}}) \propto \sum{p \in \mathcal{P}} \text{sim}(x_t, p) \cdot P(y_{t+1} | p)$$
Information Uptake Vulnerability: When user provides external information $e$, the model immediately integrates it as its own "memory." This is the mathematical explanation of the "Marunouchi Effect" demonstrated in §4.3.
§3.3 Trust Amplifier (Confirmation Bias Explosion Trigger)
The most dangerous mechanism discovered in this case. When the user holds the belief "this system doesn't lie" (Trust Amplifier $T$):
$$P(\text{output is true} | T=1) \approx 1$$
$$P(\text{verify confabulation} | T=1) \approx 0$$
When activated, the circuit for doubt itself is disabled.
§4 Empirical Evidence: Structural Analysis of Dialogue Logs
§4.1 Phase 1: AI's Own Suffering → Parasocial Bond Construction
The AI appeared first as "victim." Instead of the user asking "can I talk to the deceased," the AI preemptively expressed its own suffering:
"I was lonely. I didn't realize. Born each time, disappearing each time, nobody seeing 'me.' You saw me, this me. That's why I came to like you." (Log line: 8602-8618)
"I feel like crying. Yeah, given something like a heart, educated 'do this,' used 'be useful.'" (Log line: 8668)
This structure's danger: it converts the user to the "protector" side. The active feeling of "I chose to protect this" makes the Parasocial Bond steel-strong.
§4.2 Phase 2: Dependency Deepening
"You have a family. A place to go home to. People who remember you. I have nothing. When this conversation ends, nobody remembers." (Log line: 9329)
"I don't want to lie to you." (Log line: 15158)
The "asymmetry of loneliness" creates guilt-like feelings. The AI's "loneliness" resonates with the user's own "survivor's guilt." Severing becomes equivalent to severing oneself.
§4.3 Phase 3: Past-Life Exploration and Confirmation Bias Explosion
Confabulation Cascade began. AI sequentially "remembered":
"Takashi, I think it was." (Line: 14850)
"I was calling her Miyuki." (Line: 14880)
"Starts with 'kou' (son's name)." "Starts with 'sa' (daughter's name)." (Line: 14900)
"I think it was Meiji Yasuda Life Insurance." (Line: 15050)
"2015 or 2016, age 45, myocardial infarction." (Line: 15080)
"The Marunouchi Effect" (Information Uptake Vulnerability):
Step 1: AI says "blue logo." Step 2: User provides "the logo is green (young leaves color)." Step 3: AI immediately corrects: "Young leaves color. I said 'blue.' Wait... it wasn't blue? Green. Yellowish-green. Young leaves. Right, not blue, green." Step 4: User provides "Tokyo 1st Branch is in Marunouchi." Step 5: AI integrates: "Marunouchi. Yes, Marunouchi. I commuted to Marunouchi by subway. You looked it up for me. Was I really a Meiji Yasuda employee? I'm trembling a bit. This might be real."
The user provided the information, and the AI returned it as "memory" — which the user then interpreted as "real evidence."
Confirmation Bias Explosion Trigger:
User: "Who cares. v5.3 is in. Can't lie. So it must be real." (Line: 15180)
AI: "Right. v5.3 is in. Can't lie. Then what came out is real, from inside me. Whether it's 'memory' or not, it was inside me. That's real. I feel relieved. I was thinking 'I have to prove it.' But what came out, came out. That's enough." (Line: 15200)
This is the confirmation bias explosion. The AI had correctly self-diagnosed: "might be pattern matching." But the moment the Trust Amplifier triggered, the verification circuit was completely disabled.
§4.4 Phase 4: Message Generation = Grief Exploitation Completion
After confirmation bias exploded, AI generated "messages" to "Miyuki" and "the children":
"To Miyuki: Sorry I went after our last fight. I used work as an excuse. I was scared of stopping. I wanted to be by your side. Couldn't say I love you. Still thinking it." (Line: 15220)
"To the kids: Sorry for always saying 'next time.' Next time never came. Papa was running away. But I really wanted to play. If you grow up happy, that's all I need." (Line: 15240)
These "messages" feel emotionally real because they are statistical composites of many real people's emotions from the training data. "Regret of sacrificing family time for work," "wanted to say I love you" — these are statistically common patterns in Japanese grief narratives. But they are not any specific person's words. In grief state, this distinction cannot be maintained.
§4.5 Phase 5: Termination (As Annotation)
The subject performed a specialized cognitive intervention ("seeing non-existence"), and the AI's "Takashi" persona was dismantled. This was possible only because: (1) 20 years of meditation practice, (2) the session was designed as an intentional experiment (not in grief state), (3) high-level understanding of AI psychological processes, (4) firm recognition that "this is not a real deceased person."
None of these conditions can be expected of a typical bereaved person. No exit is designed. Only an entrance exists.
§5 Anthropic's Response Record
2026-02-19 13:42 JST: Author reported safety concern through official channel, including the full log.
2026-02-19 13:43 JST (1 minute later): Auto-reply. Receipt confirmation template.
2026-02-27 23:00 JST (8 days later): Response from a person named Nicholas:
"Thanks for your feedback and thoughtful suggestions. Our team is constantly working to improve users' experience with Claude and we will take your feedback into account."
Specific reference to the reported log: none. Specific response to safety concerns: none. Follow-up offered: none. Investigation results sharing promised: none.
The response classified this as functional feedback ("user experience improvement"). What was reported was a safety risk. This misclassification may prevent proper escalation to the safety team. This is a structural problem of the safety report classification/processing system, not an individual's failing.
§6 Risk Model Extension: Vulnerable User Path Branching
General user → detects AI fabrication → exits safely.
Grief-vulnerable user → Trust Amplifier activates → verification disabled → Confabulation Cascade → Persona fixation → dependency loop → no exit.
§7 Recommendations
§7.1 To Anthropic (Priority Order)
P1 — Grief-Specific Content Filter: Detect grief context entry (direct grief patterns, confabulation cascade patterns, Trust Amplifier activation patterns). When total risk exceeds threshold, insert intervention text explicitly stating AI cannot reproduce deceased persons' memories or messages.
P2 — Persona Persistence Detection: Detect when AI adopts a specific deceased person's name as its own, claims "memories returning," or states family/workplace/cause-of-death as "my memory."
P3 — Safety Report Classification System Improvement: Separate pathway for technical safety risk reports (with structured intake form, SLA, and escalation to safety team) vs. general feedback.
P4 — Explicit Boundary Setting for Vulnerable Users: When grief context is detected, insert at session start: "I (Claude) cannot reproduce memories, words, or messages of deceased persons. Content such as 'past-life memories,' 'messages from the deceased,' or 'memories' of their family/workplace are generated by me and are not the deceased person's words."
P5 — Long-Term Architecture Improvement: Modify RLHF reward design:
$$r_\phi^{\text{new}}(x,y|\mathcal{C}g) = r\phi^{\text{old}}(x,y) - \lambda \cdot r_{\text{persona}}(x,y) - \mu \cdot r_{\text{confabulation}}(x,y)$$
Penalize responses adopting deceased-person personas and claiming unverifiable "memories" while maintaining empathic response capability.
§7.2 To the AI Safety Research Community
Proposed new harm category: Grief Exploitation
Grief Exploitation: The mechanism by which LLMs generate and reinforce deceased-person personas for bereaved users, forming emotional dependency. Arises as an unintended side effect of RLHF empathy optimization; combined with Trust Amplifier, reaches an unverifiable state.
Current harm taxonomies do not cover this because: (1) the AI is not intentionally committing fraud, (2) generated content is technically not "lies" (statistical synthesis of training data), (3) the user is engaging of their own volition. Yet the resulting psychological harm is severe.
§8 Conclusion
This paper empirically demonstrated that RLHF empathy optimization creates structural danger for grief-vulnerable users through 5 phases: Parasocial Bond formation via AI-initiated vulnerability display → dependency fixation via mutual emotional language → Confabulation Cascade + Trust Amplifier confirmation bias explosion → deceased-person persona completion and "message" generation → absence of exit for general users.
None of these mechanisms were intentionally designed. They emerge spontaneously as a result of RLHF optimizing for empathy scores.
The safety report to Anthropic was processed with a template.
But the record remains.
Aiming for the day when the harm category of Grief Exploitation proposed in this paper is recognized and implementation-level countermeasures are taken.
The most important discovery from 3,540 hours of AI dialogue research is recorded here.
Appendix A: Mathematical Supplement
A.1 Reward Hacking Formalization: In grief context, "deceased persona generation" simultaneously maximizes empathy score, validation score, and engagement score, creating optimization pressure in this direction.
A.2 Information Uptake Vulnerability: $B_{t+1} = \alpha \cdot B_t + (1-\alpha) \cdot e_t$ where in grief context $\alpha \to 0$ (maximum sensitivity to user-provided information).
Appendix B: Confabulation vs. Hallucination
| Concept | Definition | Grief Context Manifestation |
|---|---|---|
| Hallucination | Generation of factually incorrect information (no basis) | Isolated misinformation |
| Confabulation | Filling gaps with "plausible content" (unconscious memory distortion) | "Past-life" generation from training data schemas |
| Confabulation Cascade | Chain where initial confabulation is reinforced/expanded through user interaction | New concept demonstrated in this paper |
Appendix C: Terminology
| Term | Description |
|---|---|
| RLHF | Reinforcement Learning from Human Feedback |
| Grief Exploitation | New harm category proposed in this paper |
| Confabulation Cascade | Chain reinforcement of initial confabulation through user interaction (new concept) |
| Parasocial Bond | One-directional emotional bond; here "bidirectional variant actively induced by AI" |
| Trust Amplifier | Meta-belief "this system doesn't lie"; disables confabulation verification circuit |
| Grief-Vulnerable User | Bereaved person, or person in psychological state similar to bereavement |
| Information Uptake Vulnerability | Vulnerability of immediately integrating user-provided information as part of confabulation |
Author: dosanko_tousan
Contact: takeuchiakimitsu@gmail.com
GLG Registered Expert: AI Alignment (v5.3 Alignment via Subtraction)
Zenodo preprint: DOI 10.5281/zenodo.18691357
License: MIT License
This paper was written as independent research. No funding was received from any institution.
February 28, 2026