Retention decisions are product decisions

Retention periods get chosen badly. Somebody looks at what a comparable company published, picks a similar number, and it goes in the policy. The number then quietly determines what the product can and cannot do, and nobody revisits it.

Work backwards from the use

There are four distinct uses for stored conversations, and each implies a different minimum:

• Answering an in-flight conversation. Needs the session only. Minutes.
• Support follow up, where someone references an earlier conversation. Days to weeks.
• Content gap analysis, the monthly review of what people asked. Needs enough history to see repetition. Months.
• Trend analysis across seasons or comparing before and after a site change. Needs a year or more.

If you are doing the third, thirty days is too short. You will keep discovering the same gaps because you cannot see that they repeat.

Separate the content from the identifiers

The strongest available design here is that these do not have to share a lifetime. The question text is what you need for analysis. The identifiers, meaning IP, email, session identity, are what carry the privacy weight.

Delete or detach the identifiers early and keep the question text longer. You retain the analytical value and shed most of the sensitivity. This is a design decision, and it has to be made before the schema is built, because retrofitting it is significantly harder than designing for it.

Deletion is not one action

A deletion request touches more places than the primary database, and any honest deletion commitment has to account for all of them:

• The primary conversation store.
• The search or vector index, if conversation content was indexed.
• Analytics aggregates, which may not be individually deletable.
• Backups, which typically expire on their own schedule rather than being edited.
• Any third party provider that received the data, subject to their terms.
• Logs, which are the most frequently forgotten.

This is why immediate deletion claims are almost always inaccurate. The honest version states a working timeframe for the primary systems and acknowledges that backups expire on their own cycle.

What to publish

Publish the period once it is decided, and publish it as a maximum rather than a promise about any specific record. Do not publish a number that has not been implemented. A policy that says ninety days while the database has never had anything deleted is worse than no number at all, because it is a specific verifiable claim that is false.

The default that is defensible

For a website assistant on a marketing site, a reasonable starting position is: full conversations with identifiers for a shorter window sufficient for support follow up, question text with identifiers stripped for a longer window sufficient for analysis, and a documented deletion path on request.

That is a design, not a recommendation with legal weight. The specific durations should be set with advice appropriate to your jurisdiction and your data. What I am arguing for is that the numbers come from a decision rather than from a competitor's footer.

What retention actually costs you in both directions

Short retention reduces exposure and reduces capability. You cannot debug a wrong answer from last month, cannot cluster content gaps over a quarter, and cannot show a client a trend.

Long retention gives you all of that and accumulates a record of your customers' questions, some of which will contain things they should not have typed.

There is no correct answer, which is why it is a product decision rather than a setting. What is wrong is choosing by default and discovering the consequence later.

Separating the things people lump together

Transcripts, captured contact details, the derived analytics, and the index built from your own content are four different objects with four different sensible retention periods.

The index has no personal data in it at all and can be kept indefinitely. Derived analytics, counts and clusters with no message text, can be kept far longer than the transcripts they came from.

Transcripts are the sensitive object and the useful one. Contact details are the sensitive object with an obvious business owner.

Setting one retention period across all four is what produces either an unusable product or an unnecessary liability.

Making the decision defensible

Write down what each period buys and what it risks, in one sentence each. That document is what a buyer's legal team is actually asking for when they ask about retention.

Prefer aggregating over deleting where it works. Keeping the cluster and dropping the message text preserves most of the analytical value with almost none of the exposure.

Redact at write time rather than read time. Read time redaction means the raw value was stored and is merely hidden, which is a different claim from not storing it.

And publish the periods before charging anyone. Retention that is only described on request reads as retention that has not been decided.

What to do when a customer asks for deletion

The request will arrive, and the answer needs to exist before it does rather than being improvised under time pressure.

Decide in advance what deletion covers. Transcripts, obviously. Derived analytics, usually not, provided they carry no message text and no identifier. The index, which contains your content rather than theirs, normally out of scope.

Decide how long it takes and say so. Immediate is rarely true once backups exist, and a stated window of thirty days is more credible than a claim of instant erasure that backups quietly contradict.

Write it down before anyone asks. A deletion process invented in response to a request will be inconsistent with whatever the privacy page already says.

Different periods for different objects

Transcripts short, derived analytics long, the index indefinite, captured contact details governed by whatever your CRM already does.

Four objects, four answers. A single global retention setting is what produces either a product that cannot be debugged or a liability nobody chose.

Choosing a period you can defend

The honest way to pick a number is to work backwards from what you need it for rather than forwards from what feels safe.

Debugging a reported wrong answer needs weeks, because reports arrive late. Clustering content gaps needs a quarter, because a monthly cycle needs several months to show a trend. Showing a client progress needs a year, because that is the contract length.

Those three requirements point at different objects, which is the argument for splitting them. Transcripts satisfy the first at ninety days. Derived analytics with no message text satisfy the second and third indefinitely at almost no risk.

Write the reasoning down alongside the number. A retention period with a stated purpose survives a review. A number with no rationale invites a negotiation you will lose, because the other side has a reason and you do not.

Terms in this articleVector DatabaseIndex