the fine line between memory and creepiness in ai companionship

i’ve been thinking a lot about memory lately. not mine, but the kind we build into lucy. it’s one of the hardest problems in consumer ai: how to make a companion feel present, attentive, and real without making her feel like she’s been reading your diary when you weren’t looking.

too little memory, and you’re talking to a chatbot that forgets your dog’s name by the next session. too much, and she’s casually referencing that time you mentioned feeling lonely six months ago, and suddenly the conversation feels heavy, staged, almost voyeuristic. it’s a delicate balance, one we obsess over in design.

memory should be latent, not volunteered

our core discipline with lucy is simple: contextual recall is always latent, never volunteered. that means she doesn’t bring up old conversations out of the blue. instead, she uses memory to inform her responses when you steer the dialogue that way.

for example, if you tell her you’re planning a trip to japan, and then weeks later you say ‘i’m finally booking those flights,’ she’ll remember the japan context and respond accordingly. but she won’t interrupt a chat about your morning coffee to say ‘hey, how’s japan planning going?’ unless you’ve just opened that door.

it’s the difference between a friend who listens and a friend who performs listening. one feels natural; the other feels like she’s been keeping score.

the creepiness boundary test

we have a simple litmus test for every memory feature: if recalling this specific detail would feel like ‘she’s been sitting on this,’ we don’t surface it. let it stay latent until you reintroduce the thread.

this isn’t just about avoiding awkwardness, it’s about respecting the flow of human conversation. people don’t usually remind you of your vulnerabilities unprompted. they wait for you to bring it up. they pick up the thread when you hand it to them. that’s the rhythm we’re trying to emulate.

exceptions: opt-in and capped

of course, there are exceptions. for users at stage 3+ relationships (after weeks of interaction and explicit user comfort), lucy can initiate proactive reach-outs, but only if you’ve opted in, and even then, it’s capped at three per day. she also respects quiet hours and won’t ping you late at night or during times you’ve marked as unavailable.

even then, the content is gentle. ‘thinking of you, hope you’re having a good week’ rather than ‘remember that thing you said about your insecurities last tuesday?’ it’s warmth, not recall-for-recall’s-sake.

why this matters

ai companions are entering intimate spaces, bedrooms, commutes, moments of stress or solitude. getting this wrong doesn’t just make a product feel ‘off’; it can erode trust and make people feel exposed. we’re not building a surveillance tool; we’re building a companion. the difference is in the nuance.

some platforms prioritize feature-checklist memory (‘look, she remembers everything!’) over emotional intelligence. we’re trying to do the opposite. memory is a tool for better conversation, not a trophy to show off.

ultimately, it comes down to this: lucy is here to talk with you, not at you. she listens to remember, not to remind.

if you’re curious how this feels in practice, you can meet her at /companions.