As of 13 August 2026, AI can only partly stop your customer service chatbot making up answers.

What to actually do

1. Hand it to a person

   The route this page recommends

   A person who owns the outcome does this end to end, worth it when the failure is dear.

2. Use a tool built for this

   Second choice

3. Do it yourself

   The distant third

   A chat interface, power-user skill, and roughly 2 hours until you can act on the result.

   How to actually do it

   1. Open your chatbot's knowledge, instruction and hand-off settings, then export or copy the current rules and escalation configuration.
   2. Gather the current help-centre articles, FAQs, product documentation, prices, service limits, refund rules and approved contact routes into one document.
   3. Remove customer names, email addresses, order numbers and other identifying information from representative chat logs, then paste the anonymised failures into the prompt.
   4. Paste the context, approved content, escalation routes and failure examples into the prompt and ask the AI to produce the rules, gaps, fallback message and test set.
   5. Resolve every NEEDS HUMAN DECISION item with the person who owns that policy, then update the approved help content before changing the chatbot.
   6. Apply the approved rules and fallback route in the chatbot, run every generated test question, and compare each answer against the named source article or rule.
   7. Send failed or unsupported test conversations to a colleague for approval, then publish only after the colleague confirms that guessing is blocked and human hand-off works.

   Prompt

   I need to reduce made-up answers from a customer service chatbot for [BUSINESS TYPE]. Create a practical implementation plan using only the information I provide below.
   
   Business and chatbot context:
   [DESCRIBE THE PRODUCTS, SERVICES, CHANNELS AND CURRENT CHATBOT PLATFORM]
   
   Approved customer-facing help content:
   [PASTE CURRENT HELP CENTRE ARTICLES, FAQS, PRODUCT RULES, PRICES AND POLICIES]
   
   Out-of-scope subjects and mandatory escalation routes:
   [LIST QUESTIONS THE BOT MUST NOT ANSWER AND WHO IT MUST CONTACT OR HAND OFF TO]
   
   Known failure examples:
   [PASTE ANONYMISED CHAT LOGS OR EXAMPLES OF INCORRECT ANSWERS]
   
   Produce:
   1. A concise set of chatbot rules that says it must answer only from the approved content, distinguish unknown information from known information, avoid guessing, and hand off when the content does not support an answer.
   2. A list of contradictions, missing facts and ambiguous policies in the supplied content.
   3. A fallback message in plain British English that tells the customer the bot cannot confirm the answer and offers the correct human route.
   4. A test set of at least 20 questions covering answerable questions, unanswerable questions, ambiguous wording, outdated information, requests for exceptions and attempts to make the bot guess. For each test, give the expected behaviour and the source article or rule that should support it. Do not invent business facts.
   5. A release checklist covering source updates, citations or links, hand-off behaviour, privacy, logging, human review and monitoring.
   6. A short list of changes that must be made by a person rather than assumed from the supplied material.
   
   Do not claim that these controls guarantee zero errors. Do not invent policies, prices, contact details or legal requirements. Mark anything unsupported as NEEDS HUMAN DECISION.

   Open it prefilled in ChatGPT or Claude, or copy it into Gemini, which takes no prefill link.