Where your website content goes when you train an assistant

Before any policy language is written, somebody should be able to draw where content goes. Most teams cannot, and that is why privacy pages end up vague or, worse, confidently wrong.

The path, step by step

1. A crawler fetches your pages, or you upload files.
2. Text is extracted and split into chunks.
3. Each chunk is sent to an embedding model, which is usually a third party API.
4. The returned vectors are stored, along with the chunk text, in an index.
5. A visitor asks a question. The question is sent to the embedding model.
6. Matching chunks are retrieved from the index.
7. The question and the chunks are sent to a language model, usually a third party API.
8. The answer comes back and is stored in the conversation log.

The three crossings that matter

Steps three, five and seven leave your infrastructure. Everything else is internal. Those three crossings are what any honest privacy statement has to describe.

Notice what is being sent at each. Step three sends your content. Step five sends the visitor's question. Step seven sends both together. That last one is the widest exposure and it is the one people describe least accurately.

Two separate data subjects

This is the distinction that gets collapsed and should not be. There are two kinds of data here with different owners and different sensitivities:

**Your content.** You published it, usually publicly. Sending it to an embedding API is low sensitivity for a public marketing site. It gets more complicated if you upload internal documents.

**Visitor questions.** These come from a third party who did not choose your vendors. They can contain personal information, and visitors routinely type things into chat boxes that they would not put in a form. The sensitivity here is materially higher and it is not yours to be relaxed about.

The questions this forces

Once the path is drawn, a set of questions become unavoidable, and each one needs a real answer before it can appear on a website:

• Which providers receive data at each crossing?
• Is provider training on submitted data disabled, and is that contractual or a setting?
• How long does each provider retain what it receives?
• How long do you retain conversation logs?
• Which geography does processing happen in?
• What is the deletion path, and how long does it take end to end?
• Who can see conversation logs internally?

Why unknown is an acceptable answer, publicly

Many teams treat these as questions that must be answered before launch, so they invent answers or copy a competitor's. Both are worse than saying the work is in progress.

A published statement that says which providers you use and that retention terms are being finalised is verifiable and honest. A statement claiming a specific retention period nobody has confirmed is a claim that will be checked eventually, and being wrong about data handling is a different category of wrong from a marketing exaggeration.

Applying the same standard to ourselves

Since this is our own site, the same standard applies to us. Creobot opens to teams in order. Our model providers are not finalised, our retention periods are not finalised, and our subprocessor list is not published yet. We are not claiming any certification, and we will publish specifics before anyone is charged rather than approximating them now.

Draw the diagram before you promise anything

There are five places content moves and most teams can only name two.

Your pages are read by a crawler and the text is extracted. That is the first copy.

The text is split and embedded, and the vectors plus the passage text are stored in an index. Second copy, and it persists.

At question time, the retrieved passages are sent to a model provider along with the question. Third movement, and it is the one that crosses a boundary you do not control.

The answer comes back and is stored in the transcript. Fourth copy, now containing both your content and the visitor's words.

Any analytics or error monitoring in the path may see fragments of the above. Fifth, and the one that is almost always forgotten in a data map.

The questions this forces you to answer

Does the model provider retain the prompt. This depends on their terms rather than yours, and it is the single most common thing vendors summarise incorrectly.

Where is the index hosted, and does that satisfy any residency requirement your customers have.

How long do transcripts live, given they now contain both your content and visitor input.

What happens on deletion. Removing a page from your site does not remove it from the index, and does not remove it from transcripts where it was already cited.

Explaining it to a buyer without overclaiming

Describe the movement, name the parties, and state what is not settled. A buyer with any technical literacy will trust that far more than a reassuring paragraph.

Avoid the constructions that sound like commitments and are not. Data is handled securely. We take privacy seriously. Industry standard practices. None of these is falsifiable, which is precisely why they read as evasion to the people who matter.

If a signed agreement does not exist yet, say that it is planned and when. A buyer can work with a timeline. They cannot work with a template presented as a policy.

Keeping the diagram current

A data map is accurate on the day it is drawn and drifts every time someone adds a tool.

Tie it to a trigger rather than a calendar. Any new provider in the request path, any new analytics or monitoring service, any change of model provider updates the map and the subprocessor list at the same time.

Review it before any security questionnaire rather than during one. Answering from a map you know is current is a different conversation from reconstructing it under deadline.

The copy people forget

Error monitoring. A logged exception frequently carries the request payload, which on a chat endpoint means the visitor's question and sometimes the retrieved passages.

It is a genuine copy of the data in a genuine third party system, and it almost never appears on a data map because nobody thinks of it as storage.

Writing the map so someone else can check it

A data map that only its author understands fails at the moment it is needed, which is usually during a security review conducted by someone who was not there when it was drawn.

Write it as a list of movements rather than a diagram of systems. Content moves from the site to the crawler. Extracted text moves to the index. Passages and the question move to the model provider. The answer and the question move to the transcript store. Fragments may move to error monitoring.

For each movement name the destination, whether it crosses an organisational boundary, and how long the data persists there. Five rows, and it answers most of a security questionnaire directly.

Then keep it next to the subprocessor list rather than in a separate document, because the two go out of date together and will only be updated together if they are in the same place.

Terms in this articleEmbeddingChunkingIndex