POST /v1/assistant/chat/{site_id}
Models: Claude Sonnet (answer) · Haiku (guardrails, follow-ups, title)
Multi-tenant: client → brand → site
Runs on: AWS · Kubernetes
Evaluation: 2 datasets · 14 evaluators · 3 experiments
// POST /v1/assistant/chat/{site_id} { "user_id": "anon_reader_1", "message": "What electric SUVs has Brand Alpha reviewed recently?", "conversation_id": null, // new thread "site_id": "brand_alpha_us", "url": "https://www.brand-alpha.example/cars/electric-suv-review/", "content_id": "0868deca-…", // CMS article id "page_title": "Cars of the Year: an electric SUV review" }
// response: server-sent events data: {"type":"text","data":{"text":"Brand "}} data: {"type":"text","data":{"text":"A "}} … one per token data: {"type":"end","data":{ "response": "Brand Alpha recently reviewed …", "conversation_id": "50a19952-…", "title": "Electric SUV Reviews", "model_used": "claude-sonnet-4-6", "guardrail_triggered": null, "recommendations": {"related":[{"title":"Electric SUV review", …}], "query_index":[]}, "topics": ["Luxury EV Comparison","Electric Vehicle Performance"], "follow_up_suggestions": ["How does it compare to other luxury SUVs?", …], "intent": "content_question" }}
| ID | Format | Created by | Scope | Enters code at |
|---|---|---|---|---|
| user_id | caller string | widget | one reader / device | ChatRequest.user_id |
| conversation_id | UUID4 | service, if not sent | one chat thread | ChatRequest.conversation_id |
| site_id | slug (brand_alpha_us) | brand config | one brand / locale | ChatRequest.site_id |
| registry site id | UUID | site registry | one brand / locale | brand JSON → state |
| content_id | UUID (CMS) | CMS | the article on the page | optional, for related articles |
| message_id | ULID | service | one stored message | Message.message_id |
| article id | UUID (CMS) | content search index | one article | recommendation cards |
GET /v1/assistant/users/{user_id}/conversations: the "past chats" list (api.conversations.get)GET /v1/assistant/conversation/{conversation_id}: one thread (api.conversation.get)DELETE /v1/assistant/conversation/{conversation_id}: the reader deletes a chat (api.conversation.delete)Message, one chat line: conversation_id (PK), message_id (SK, ULID), role, content, created_at, recommendations, topics, follow-ups, model_name, guardrail, ttl.
Conversation, one summary row per thread (the "past conversations" list): user_id (PK), conversation_id (SK), site_id, created_at, last_message_at, LLM-generated title, last-message preview, message_count, ttl.
Why models: they serialize items for DynamoDB, validate data before the write, fill in defaults, and give one definition of a message and a conversation. Writes happen in persist_conversation(), a background task after the graph finishes.
| Datadog metric | Type | What it measures |
|---|---|---|
| query_received | counter | every POST to the chat route |
| conversation_started | counter | first message of a new thread |
| conversation_deleted | counter | DELETE conversation |
| response_latency | gauge (ms) | end to end, request in to response out |
| llm_latency | gauge (ms) | round trip to the AI gateway |
| tool_call | counter | each graph tool call, tagged by tool |
| search_query_index_latency | gauge (ms) | query_index call, only when the tool runs |
| search_related_latency | gauge (ms) | related-articles call, only when the tool runs |
| guardrail_triggered | counter | input topic block, grounding fail, language pattern, hallucination |
| error | counter | unhandled errors by source (gateway, search, DynamoDB, Valkey, config); feeds the error-rate SLOs |
.github/workflows/main.yml checks out the code and installs the deploy CLI.BUILD_ENV=DEV) and run ruff (lint) and pytest inside it. Tests gate the release.ai-assistant:<git-sha>.k8s/deployment.yml, fills in the env variables, rolls out pods and health-checks them.