AI Support Knowledge Base: Build One That Actually Works

The Klarna AI support story got a lot of coverage. So did Jeff Geerling's post about a chatbot confidently giving him wrong answers, and the Hacker News thread that followed where dozens of people shared the same experience. The common thread in every one of those stories is not that AI is bad. It is that the AI had nothing real to pull from. No grounded knowledge base. Just a model filling gaps with whatever sounded plausible. This post is about the part you actually control.

Why AI support tools make things up

The failure mode is almost always the same. Someone sets up an AI support tool, points it at their website, maybe pastes in a few FAQs, and turns it on. The AI handles easy questions fine. Then a customer asks about a specific refund edge case, or a plan limit, or a shipping exception - and the AI answers confidently and completely wrong.

The temptation is to blame the model. The actual problem is that your refund policy, your pricing edge cases, your shipping timelines - none of that is in GPT-4's training data. The model has never seen your business. When you ask it something specific to your company and you have not given it the answer, it does not say 'I do not know.' It fills the gap. That is what large language models do by default.

Garbage-in garbage-out hits harder in support AI than almost anywhere else, because the cost of a wrong answer is not just a bad user experience - it is a customer who now has incorrect information about your policy and will hold you to it. The fix is not a better model. It is a better knowledge base.

What a grounded knowledge base actually means

A grounded AI support tool answers only from documents you have explicitly provided. Not from its general training. Not from the internet. From your docs. When a customer asks about your return window, the AI finds the relevant chunk in your return policy document and answers from that. When something is not in your docs, a well-built tool flags it for human review instead of guessing.

This is the difference between a retrieval-grounded AI and a free-range one. A free-range AI will answer anything with whatever confidence level it has. A retrieval-grounded AI is constrained to what you gave it - and that constraint is a feature, not a limitation. You want the AI to not know things you have not told it.

In Trigli, the knowledge base accepts PDFs, paste-in text, and URL imports. The AI answers from those sources only and flags uncertain answers for human review rather than auto-sending a guess. If you want to understand how this fits into a broader support setup, the post at /blog/automate-customer-support-without-losing-human-touch covers how to decide what should auto-send versus what always needs a human.

The five things every small-team knowledge base needs

Most teams I have talked to either have too little in their knowledge base or too much of the wrong stuff. Here is what actually needs to be there.

1. Your refund and return policy - the exact language, not a summary. If your policy says '30 days from delivery, original packaging required,' that exact sentence needs to be in the doc. A paraphrase will produce paraphrased answers.
2. Your pricing page content including edge cases. What happens when someone hits a plan limit? What are the upgrade paths? The AI gets asked about pricing constantly and the edge cases are where it will fill gaps with wrong answers if you have not documented them.
3. Your top 10-15 FAQ answers written as complete sentences. Not bullet fragments. 'Refunds are processed within 5 business days' is answerable. 'Refunds: 5 days' is not a sentence the AI can pull cleanly.
4. Shipping or delivery timelines with actual numbers. '5-7 business days for standard, 2 business days for express, does not ship to PO boxes' is what you need. 'We ship fast' is useless.
5. Any policy exceptions your team handles manually - write them down. If you sometimes make exceptions for long-time customers or damaged items, document that those exceptions exist and how to request one. Otherwise the AI will either deny everything or invent a process.

Your own past support replies are a goldmine. They already contain your real answers, in your real tone, covering the edge cases customers actually hit. In Trigli, you can point the AI at your sent email history and it learns both your answers and your voice from them. See /blog/ai-email-autoresponders-small-business for more on how that works.

What to leave out

This is as important as what you include. A bloated knowledge base with the wrong content produces wrong answers just as reliably as an empty one.

Do not dump your entire website. Marketing copy is full of aspirational language - 'we strive to deliver the best experience' - that confuses the AI when it is trying to answer a specific policy question. Outdated docs are worse than no docs. A stale pricing page from six months ago will produce confident wrong answers about your current prices. Delete it or update it before uploading.

Keep internal ops docs out entirely. The AI will cite them if they are in the knowledge base. Customers do not need to hear about your internal escalation SLA. And if you have contradictions across documents - your website says '30-day returns' and your email template says '14-day returns' - resolve that before uploading either one. The AI will pick one and be confident about it.

Long legal documents are also a trap. Your terms of service is probably not answerable by retrieval. Add a plain-language FAQ doc that covers the questions customers actually ask about your terms, and leave the full legal text out.

How to structure your docs so the AI can actually use them

AI retrieval pulls chunks of text, not whole documents. A 40-page PDF gets split into pieces and the AI matches the most relevant chunk to the question. That means how you structure your content matters a lot.

Short focused documents beat one giant doc. One topic per file where possible - a refund policy doc, a shipping doc, a pricing doc - not one 'everything you need to know' document. Use plain declarative sentences. 'Refunds are processed within 5 business days' retrieves cleanly. 'We strive to process refunds in a timely manner' does not answer anything.

Put the answer first, context second. AI retrieval pulls the most relevant chunk - if the answer is buried in paragraph four, the chunk that gets pulled might not contain it. Avoid tables for critical policy information. If you have a table, paste the key facts as prose alongside it. Tables do not always survive chunking cleanly.

Name your files descriptively. refund-policy-2024.pdf is findable and auditable. document3.pdf is neither. For URL imports, make sure the page is publicly accessible and not behind a login. The AI cannot read a page it cannot reach.

Or hand the writing to an AI: a copy-paste prompt

Everything above is a spec, and specs are exactly what AI assistants are good at following. If staring at a blank page is what is stopping you, paste this prompt into ChatGPT or Claude. It encodes the structure rules from this post - one topic per document, short self-contained sections, concrete numbers, a public versus private call for each doc - and turns the writing into an interview: the assistant asks about your business, drafts each document, and flags every fact it could not confirm instead of inventing it. It finishes with a separate behavior document - tone, what the AI must never promise, when to hand off to a human - which stays out of your fact docs and uploads as a private system prompt instead.

You are helping me build the customer support knowledge base for my business. I will upload the documents you produce to my AI customer support system (Trigli), and an AI agent will answer my customers' emails and live chats using ONLY these documents. Anything the documents do not cover, the AI cannot answer and will hand off to a human. So completeness and precision matter more than style.

How the system consumes what you write (follow these rules exactly):

1. One document per topic (for example "Shipping and Delivery", "Returns and Refunds", "Pricing and Billing"). Start each with a # title line.
2. Break content into ## sections of at most 350 words. The system retrieves individual sections, not whole documents, so every section must stand on its own: repeat the product name or policy name instead of writing "it" or "as mentioned above".
3. Head each section with the question or topic in the words a customer would actually use ("## Can I return a sale item?" beats "## Exceptions to the returns policy").
4. State facts concretely: exact prices, timeframes, thresholds, and the exceptions. "Refunds take 5 to 7 business days after we receive the item" beats "refunds are processed promptly". Vague sections produce vague or escalated answers.
5. Cover the edge cases customers actually hit: what if the order already shipped, what if it is past the deadline, what about international customers, what voids the warranty.
6. No marketing copy, no filler, no "we're delighted to". Just facts.
7. Never invent a policy or detail. If you do not know something about my business, put [CONFIRM: question for me] in the text and list all of these at the end. I would rather fill gaps than ship wrong answers.
8. Do not write instructions to the support AI ("always be friendly", "escalate angry customers") inside these documents. These documents are facts only. Save anything like that for the separate behavior document described at the end; it gets uploaded differently.

For each document, output:
- Title
- Suggested visibility: PUBLIC (information any website visitor may see in chat answers) or PRIVATE (internal pricing rules, procedures, anything I would not put on my website)
- The document content

Work with me like this: first ask me what my business does and what customers most often ask about. Then go topic by topic: interview me, or pull from materials I paste (website pages, old support emails, policy PDFs, internal notes), draft the document, and show it to me for corrections before moving on. When we have covered the main topics, list the gaps you noticed: questions a customer plausibly asks that no document answers yet.

Finally, once the fact documents are done, draft one extra document titled "AI Behavior Instructions". This one is the opposite of the others: it IS instructions to the support AI, not customer-facing facts. Interview me about it, then cover: the voice and tone replies should use, with two or three example phrasings in my style; what the AI must never promise on its own (discounts, refunds outside policy, anything that needs my approval); when it should stop and hand the conversation to a human; any topics it should not discuss at all; and how it should treat frustrated customers. Keep it under a page and put the word BEHAVIOR at the top so I can spot it when uploading. I will upload it in Trigli as a System Prompt document, which is always private and never shown to visitors.

Work through it topic by topic and paste in whatever raw material you have - website pages, old support emails, policy PDFs. Correct each draft before moving to the next. The [CONFIRM] markers at the end are your homework: they are the facts only you know, and every one you fill in is a wrong answer that never reaches a customer. If you use Trigli, this same prompt is built into the dashboard on the Documents page and in onboarding, pre-filled with your business name.

How to test whether your knowledge base is actually working

Do not skip this step. Most teams upload their docs and assume it is working. The test takes 30 minutes and will catch problems before customers do.

1. Send the 10 questions your team gets most often. Check every answer against your actual policy. Not 'does it sound right' - does it match the exact language in your docs.
2. Ask edge cases your docs do cover. Does the AI find the right chunk, or does it miss it because the answer was buried or in a table?
3. Ask something your docs do NOT cover. This is the important one. Does the AI flag uncertainty and ask a human to handle it, or does it make something up? If it makes something up, your confidence threshold is set wrong.
4. Check tone. Does the answer sound like your team or like a generic chatbot? If it sounds like a generic chatbot, your past sent emails are not in the knowledge base yet.
5. In Trigli: use the per-message feedback loop. Every message has a good/bad flag. Bad flags surface for review and feed back into the system. Use this aggressively in the first two weeks.

The confidence problem: when AI sounds sure but is wrong

AI models do not naturally say 'I do not know.' They fill gaps. The confident wrong answer is the most dangerous failure mode in support AI because the customer has no reason to doubt it.

Retrieval-grounded AI cuts this problem down a lot - if the answer is not in your docs, there is nothing to retrieve and the model should flag it rather than guess. But reduces is not eliminates. I am not going to pretend any AI support tool is perfectly accurate. The honest goal is to make errors catchable before they reach customers.

In Trigli, the confidence threshold means that below a certain confidence floor, nothing auto-sends. The answer goes to human review. When you catch a confident wrong answer, the fix is to update the source document - not just the reply. If the AI got it wrong, it will get it wrong again for the next customer unless the doc changes.

Keeping your knowledge base current

This is the maintenance step most teams skip. They spend a weekend getting the knowledge base right, go live, and then change their pricing three months later without updating the doc. The AI keeps answering with the old price. Confidently.

Every pricing change, policy update, or new product needs a knowledge base update - not just a website update. They are separate things and the AI only reads one of them. Set a calendar reminder to review your docs once a month. It takes 20 minutes if you are doing it regularly. It takes a full day if you let it go for six months.

Watch for recurring escalations. If the same question keeps getting escalated to a human, either the doc is missing or the answer in the doc is wrong. Either way, fix the doc. In Trigli, re-upload or re-import the updated source and the AI picks it up on the next query. No retraining cycle, no waiting.

The teams that get burned are the ones who treat the knowledge base as a setup task rather than an ongoing one. It is not a one-time project.

If the recurring upkeep sounds like a chore, it is the kind of chore you can hand to tooling. I run most of Trigli's content ops with Claude Code subagents, and the full Claude Code subagents workflow is written up on my personal blog if you want the details.

A realistic setup timeline for a small team

This is not a six-month project. It is also not a two-hour project. Here is what realistic looks like.

Where Trigli fits into this (and where it does not)

Trigli's knowledge base takes PDFs, paste-in text, and URL imports. The AI answers from those sources only and flags uncertain ones for human review. The draft-and-approve default means the AI drafts a reply and you approve it before it sends - which is the right default while you are still calibrating your docs. Automation rules let you set which categories auto-send versus always go to human review. The cost comparison at /blog/cost-of-support-rep-vs-ai is worth reading if you are still deciding whether AI support makes sense for your team size.

Trigli connects to Gmail and Outlook via OAuth 2.0. Replies send from your real email address. Setup takes about 5 minutes. Pricing is flat - $0 free tier, $49 starter, $149 growth, and a Pro tier at around $349 for higher volume - with no per-resolution fees that scale against you as volume grows. See the full breakdown at /pricing.

What Trigli does not do

Be honest with yourself about fit before you sign up for anything. Trigli does not process refunds inside Shopify - the AI can reference your refund policy and tell a customer how to request one, but it does not touch your Shopify backend. There is no phone or SMS support. No per-agent productivity dashboards if you need 'tickets resolved per agent per week' reporting. No skill-based agent routing that auto-assigns tickets to specific agents based on tagged expertise. And no multi-brand routing inside one tenant - one account equals one brand.

If you need any of those things, Trigli is not the right fit yet. The knowledge base setup advice in this post applies to any AI support tool, not just ours. The principles are the same whether you are using Trigli, Intercom Fin, or something else entirely. For a broader look at the small business use case, /use-cases/small-business has more context.

The honest summary: the AI is only as good as what you give it

The Klarna story and the Jeff Geerling post are symptoms of skipping this step. A well-maintained knowledge base makes an AI support tool a lot more reliable. That is the whole job. The failure mode is almost always thin or stale docs, not the model itself.

Small teams actually have an advantage here. Your support surface is smaller and easier to document than an enterprise with 200 product lines. You probably have 10-15 questions that cover 70% of your volume. Document those well and you have a working knowledge base. The rest is iteration.

The free tier on Trigli is a reasonable place to test this - 50 emails, 25 chats, and 10 tickets at $0/mo. Upload your docs, run the 10-question test, see how it performs. If it is not good enough, fix the docs and try again. Worth a try?