Batch ingest
POST /events/batch
- Accept a batch, not one call per click
- Drop repeats by client_event_id
- Refuse past 100 events per customer per minute
- Refuse past a queue depth of 10,000
HyperPersona · team of four · agentic pipeline on AWS Bedrock
A recommender that has to prove what it just said
Four of us built it. It reads shopping behaviour, writes one personal offer, and then a second pass checks every claim in that offer against the source data and deletes anything it cannot find.
A confident model is not a correct one
Ask a language model to write a personal offer and it will write a good one. It will name a brand the customer likes, mention a discount, and land on a product. The brand is usually right. The discount is often invented.
That is not a small flaw in a shop. A made-up percentage is a promise the business has to honour or explain away, and it is exactly the kind of detail a model produces most fluently, because a sentence with a number in it reads better than one without.
And most of what it reads is noise
The second problem is upstream. The system learns from a stream of shopping events, and the first version ran every one of them through the full pipeline: embed it, ask a model to pull facts out of it, embed each fact, store them all.
A purchase deserves that. A page view does not, and a route change fires one, so a single customer clicking around generates dozens. Worse, the model does answer: it extracts facts like the customer viewed a page, the path was /cart. Those land in the same store as the real signal and compete with it at retrieval time.
So the cheap fix and the good fix turned out to be the same fix. Spending less also meant storing less rubbish, which meant retrieving better.
How we got there
We spent a while trying to make the model stop inventing discounts. Three passes, in the order they happened.
Explore
Told it not to make things up
Realise
A prompt is a request, not a rule
Insight
Read the draft back against the source
Do not ask a model to be right. Check whether it was.
Two decisions, at opposite ends
Everything else follows from those. At the front, events are sorted into three tiers before anything expensive happens, and the sorting is a list rather than a judgement. At the back, no draft is the output: a separate pass reads it against the source and rewrites what it cannot support.
In the middle sit the pieces that make both possible. Three kinds of memory, so the cheap tier still has somewhere to go. Recency-weighted ranking, so what the model reads is the current version of the customer rather than every version.
One event, three possible fates
Consent runs before the sorting, because an event we are not allowed to keep should not be sorted, stored, or spent on either.
Privacy gate
worker/src/agents/tools/privacy_tool.py
- No consent record at all means blocked, not defaulted
- Consent carries scopes; this needs personalization
- PII is redacted before the text goes anywhere
noise
Dropped
page_view, profile edits, UI state
- Marked aggregated on the spot
- Never enqueued, so no model ever sees it
Costs nothing
and cannot pollute retrieval
low signal
Embedded only
skip_low_signal_tool.py
- One embedding call, no fact extraction
- Lands in behavior-embeddings so search can still find it
- Rolled into a session summary later
Kept, not thought about
recoverable if it turns out to matter
high signal
Full pipeline
purchase, add_to_cart, return, search
- Analyzer pulls atomic facts out of the event
- Each fact carries a polarity: likes, neutral, avoids
- Every fact is embedded and stored on its own
Worth the money
the events that say what someone wants
The sorting is two frozen sets in one file with the reasoning written beside them. There was a pull toward letting an agent decide which events mattered, and it would have been the wrong call: a list is cheaper, testable, and readable by the next person who adds an event type.
Three tiers in, three kinds of memory out
The tiers are not arbitrary. Each feeds a different store, and the three stores answer different questions when a recommendation is asked for.
What each store is for
What I saw
What it meant
customer-facts
Short declarative statements about the person, extracted by the analyzer: owns Salomon X Ultra, interested in trail running. The expensive tier writes here, and only the expensive tier.
behavior-embeddings
The events themselves, embedded but never interpreted. Cheap to write, and still findable by search if the question comes up later.
session-summaries
One document standing in for a run of quiet events, so a long browse costs one summary rather than forty extractions.
Three ways of remembering, because remembering everything the same way is how you end up unable to find anything.
One recommendation, end to end
What happens between the request and the sentence that comes back.
The API hands it off
Recommendations are a job, not a request. FastAPI takes it, queues it, and returns; the worker picks it up. Nothing waits on a model with a socket open.
server/src/routes/recommend.py
The supervisor decides the order
Two implementations behind one Protocol, picked at startup. A hand-rolled one that runs a fixed order, and one that hands the tools to an agent and lets it choose. Same trace output either way, so they can be compared.
worker/src/agents/recommend_supervisor.py
Retrieve from all three stores
Facts, behaviours and session summaries are pulled by nearest-neighbour search against the request context. The cheap tier earns its keep here: those events were never interpreted, but they are still searchable.
worker/src/agents/tools/recommender_tool.py
Rank, and catch contradictions
Similarity times recency on a 45 day half life. Anything under 0.12 is cut, near-duplicates collapse, top six survive. Topics holding both a positive and a negative fact are flagged as conflicts.
shared/ace_ranking.py
Draft the offer
One sentence, and the prompt forbids inventing numbers: if it cannot point at a source line for a price or a discount, it must leave the number out. Conflicts come through labelled, with an instruction to weight the recent side.
worker/src/agents/tools/recommender_tool.py
Verify it, claim by claim
A second model gets the draft alongside the real source lines and checks each concrete claim. Every claim supported means VALID and the draft passes untouched. One unsupported means a rewrite that drops it and keeps the rest.
worker/src/agents/tools/verifier_tool.py
Why the verifier gets the whole source
The obvious way to build a checker is to hand it the draft and a summary of what went in. That does not work, and it is worth saying why: a checker that only knows six facts were used cannot check anything. It can only agree.
So the verifier is given the actual fact, behaviour and summary lines the recommender drew from, in the same structured layout. It is not asked whether the draft seems reasonable. It is asked, for each product name, brand, price, discount and feature, to find the source line that supports it.
The recommender and the verifier get separate model clients on purpose. They are different jobs: one wants to write well within a boundary, the other wants to be unforgiving about a boundary. Sharing one client made it too agreeable with itself.
The contradiction case
Every fact carries a polarity when it is extracted: the customer wants this, is neutral about it, or avoids it. A return event produces a negative fact as readily as a purchase produces a positive one.
That makes contradictions detectable. If one topic holds both a positive and a negative fact, it is flagged and the more recent one wins. Somebody who loved a brand two hundred days ago and said they were done with it last week is not both, and averaging them would produce a recommendation nobody wants.
What the sorting is worth
The gap between the cheap path and the full one, per event.
$0.0056
Full analyzer pass
embed, generate, embed each fact
$0.000001
Cheap path
one embedding call
~5,600x
Between them
on the events that carry no signal
Source · the team's own per-call estimates, written in the skip_low_signal_tool docstring. Not measured against a bill.
Those are estimates from the code, not a reading off an invoice, and the page should say so plainly. The ratio is the point either way: the difference between the two paths is not a percentage, it is orders of magnitude. That is why the sorting happens at the front door rather than as an optimisation of the pipeline behind it.
The limits at the door
Setting
Why
Per customer
100 events a minute
one runaway client cannot starve everyone else
Queue depth
10,000 jobs
refuse early rather than fall over late
Duplicates
client_event_id
a retried batch is not counted twice
Source · server/src/config.py
What building it taught us
What I saw
What it meant
The cost problem and the quality problem had one fix
We went looking for a way to spend less and found the cheap events were also the ones making retrieval worse. That does not usually happen, and it is worth noticing when it does.
Not everything in an agentic system should be agentic
The event policy wanted to be a model and should be a list. Frozen sets in one file are cheaper, testable, and someone can read them and know the answer.
A checker needs the evidence, not a summary of it
Handing the verifier counts instead of the actual source lines turned it into something that agreed with everything. Verification is only verification if the thing can fail.
Consent had to come before the sorting
It is tempting to gate at the expensive step, since that is where the risk feels like it lives. But an event we were never allowed to keep should not be stored cheaply either.
The interesting decisions were about where to put the thinking, not how much of it to do.
Where it is still rough
Three things worth fixing, all visible in the code today.
Still open
The cost figures are estimates in a docstring, not measurements. The honest version wires per-job spend into the trace log, so the ratio becomes something the system reports rather than something a comment claims.
Still open
Polarity is decided twice, and the second time by word lists: a fact counts as negative if it contains a negation word and no positive word. That misses sarcasm, comparatives and anything phrased sideways, and it feeds the conflict detection that decides which of two contradictory facts survives.
Still open
Near-duplicate facts collapse on a key made from their first four meaningful words. Fast, and fine on what we had, but two genuinely different facts that open the same way get treated as one topic.
What HyperPersona came down to:
01
Sort before you spend
Three tiers decided by a list at the front door, so the expensive pipeline only ever sees the events that say what somebody wants.
02
Remember three ways
Stated facts, raw embedded behaviour, and rolled-up summaries, each fed by a different tier. One store for everything is how retrieval stops working.
03
Never ship the first draft
A second pass checks every concrete claim against the real source lines and rewrites what it cannot support. Fluent and correct are not the same thing.