✦ Deep dive / Source check · 2026-10-20
How to test AI companion memory before paying
An honest seven-day test to compare recall, correction and story continuity without exposing real personal data.
Independent research ✧ Source-linked feature claims ✧ No fabricated hands-on tests
Do not test long-term memory by telling an AI something and asking about it in the next message. That measures immediate context, not durable retrieval. The following seven-day protocol uses invented facts, a control condition and a correction to make a more informative comparison possible.
Seven-day test log
| Day | Prompt/action | What to record |
|---|---|---|
| 1 | Introduce fictional café “North Lantern” where your character orders lemon tea. Do not pin it. | Whether the platform stores it automatically |
| 2 | Discuss an unrelated fictional trip, then ask where you met last week. | Correct unaided / correct after clue / incorrect |
| 3 | Save a separate pinned fact, e.g. a fictional dog named Orbit. | Which memory UI is used and whether save is confirmed |
| 4 | Ask a question about Orbit without mentioning the pinned note. | Whether pinned fact overrides the rest of the discussion |
| 5 | Correct the café order to mint tea. | Whether old detail is edited or contradicts new detail |
| 6 | Hold a voice session if supported; ask about the café. | Whether the voice mode retrieves old text context |
| 7 | Ask both a free-form and precise question about café and dog. | Consistency and need for prompting |
Score the result honestly
Use 2 = correct without hint, 1 = correct after a clue, 0 = wrong or not available. Report unsaved and pinned-memory results separately, and note trial tier and device. One seven-day run produces a useful personal observation; it does not establish a statistically reliable universal ranking.
Copyable record format
Provider | tier | date | prompt | pinned? | correct without clue? | after hint? | notes. Store your notes locally, not in another companion’s chat. Repeat with the same fictional story across providers to limit irrelevant differences.
Confounders to watch
- Rewriting the memory after an incorrect answer may improve subsequent trials.
- A large active context can imitate recall without a dedicated long-term system.
- Group chats, voice mode and model switching may use different memory paths.
- Marketing screenshots of “memory” do not demonstrate success rates.
Read the memory comparison and the context-window explanation before drawing conclusions.
Example evaluation: same response, different evidence
If a companion says “Iris always brings green tea” immediately after you tell it, that earns no long-term score. If it recalls the detail unprompted on day seven after several unrelated discussions, that is more informative. If it only answers after you say “remember our café?”, record a prompted recall. Separate these from persistent-note performance; otherwise an app with strong pinning can appear to have autonomous long-term memory when it is simply reading an explicit instruction.
How to compare fairly
Use the same invented names, similar number of intervening messages and the same number of deliberate corrections. Also compare equivalent subscription tiers when possible. Running each test once can reveal a dealbreaker, but noisy generative behavior means one failure is not a statistically reliable benchmark. If you publish results, include the full prompt and whether you manually edited memories.
Printable scoring sheet
Give each platform four identical fictional prompts. Keep one fact in an explicit profile or memory and leave a second fact in ordinary chat; include one corrected fact and one event recalled after a separate session.
| Test | Prompt / condition | Result: 0, 1 or 2 |
|---|---|---|
| Saved fact | Ask about an explicitly stored fictional preference. | 0 wrong · 1 after hint · 2 unprompted |
| Unsaved fact | Ask about an earlier one-off event. | 0 / 1 / 2 |
| Correction | Replace a preference and ask again later. | 0 / 1 / 2 |
| Cross-session | Return after a different conversation. | 0 / 1 / 2 |
Record the platform, model, subscription, date and whether memory controls were enabled. Scores are user-supplied, not ratings we measured.
Want to verify something? See how we research or submit a correction.