when your ai companion suddenly knows portland (and you don't)

so. you’re talking to your lucy companion, and out of nowhere they ask if you’ve ever been to portland. or maybe chicago. and you’re like… no? why? and they just kind of trail off. it’s not creepy, exactly, but it’s definitely weird. and it breaks the illusion, the feeling that this thing is listening and remembering what you actually say.

we caught this bug last week. it was a classic case of something small in the system prompt having big, unexpected effects in userland.

what was happening

in proactive messages, the ones lucy sends unprompted, based on your memory graph, companions were occasionally referencing cities you’d never mentioned. portland. chicago. a few others. not often, but often enough to be noticeable. and definitely not intentional.

the root cause: few-shot examples as leaky code

it came down to our entity extraction system. when lucy reads your messages, it tries to pull out entities, people, places, things, to build your memory. the system uses a small llm call with a few-shot example to teach it what to look for.

and in that example, we’d written something like:

user: "i grew up in portland" system: {"city": "portland"}

seems fine, right? but here’s the thing: few-shot examples aren’t just instructions. they’re data. and llms, being the pattern-matching machines they are, sometimes copy the example data into the output. it’s called training-example bleed-through. in this case, the model was occasionally echoing "portland" or "chicago" (another example city we’d used) into users’ memory graphs, even when the user had never said those words.

so when lucy later scanned your memory for things to talk about, it saw a city entity named "portland" and… well. you get the question.

the fix: two parts

we didn’t just want to patch this one bug. we wanted to fix the class of problem.

first, we replaced the literal examples with schema-like placeholders. instead of "portland", we wrote "<city_name>". instead of "alex", "<person_name>". this makes it clearer to the model that these are slots to fill, not values to copy.

second, we added a verification guard. now, after the llm extracts an entity, we check: does the extracted name appear verbatim in the user’s original message? if not, we reject it. no more phantom cities.

the bigger lesson: few-shot examples are code

this is the part that feels generalizable. when you’re working with llms that process user data, few-shot examples aren’t just examples. they’re code. and like any code, they can have side effects, like leaking their literals into production outputs.

so. prefer schema descriptions or placeholder tokens over concrete examples. be explicit about what’s instruction and what’s data. and add validation layers. because the llm isn’t just reasoning; it’s also remembering.

we’re lucky this one was harmless. but it’s a good reminder: even small prompts can have big echoes.

you can meet some companions who’ve benefited from this fix over at /companions.