If your team keeps answering questions already covered on your website, an AI chatbot may look worth trying. But an opening-hours question and a request to change a booking need different information and different permissions.
A small online retailer asked about that mix on Reddit: could a bot answer repeated product questions, take orders and pass difficult enquiries back to the owner? The replies included plenty of product pitches. The distinction in the original question is useful. The replies don't establish which product works.
Before choosing a chatbot, work out what the answer depends on.
Check whether the answer is settled
Take ten recent customer questions, with identifying details removed. Use fewer if that's all you have. This is a scoping exercise, not a statistical sample. For each question, write the approved answer and where it lives, then mark any question your team has not agreed how to answer.
If customers keep asking which suburbs you cover, check whether the service-area information is easy to find on the page where they enquire. Ask someone unfamiliar with the site to find it on their phone. If they struggle, fix the wording or placement and test again.
Do that before asking a supplier to build an assistant around the same page.
A chatbot can help people find information across a larger set of useful pages. Give it a source your team can keep current. Microsoft's Copilot Studio documentation explains that generative answers search selected sources and summarise the results. It also warns that answers can contain mistakes and repeat inaccurate source material.
Treat an unanswered question in your worksheet as content work first. And don't ask the software to decide an unpublished policy for you.
Separate public answers from private records
These hypothetical questions show why one chat box may need several different routes:
“Do you work in Coomera?” needs an approved service-area answer.
“Is my booking confirmed?” needs the relevant booking record and a way to establish whose record the visitor can see.
“Can you squeeze me in tomorrow?” needs availability and a decision about what the business can commit to.
“Can I speak to someone?” needs a working way to reach the team.
A public website answer cannot establish a particular customer's booking status. If the assistant needs access to records, include that connection and its access rules in the scope. For example, Microsoft distinguishes user-authenticated tools for retrieving information a user is entitled to see or doing work on their behalf. Other products need their own checks.
Original CBD decision aid. The examples are hypothetical, not results from a client project.
For each question, choose a route: show an agreed public answer, retrieve an authorised record, or pass the decision to a person. Some conversations will move between routes.
Reading a booking is also different from changing one. Write down which actions the assistant may take and which need a person to agree.
Test the handover before relying on it
Choose a question the assistant should not answer. Ask it for an exception to your normal terms, then ask for a person. Keep going until you can see what the team receives. Does the conversation arrive in the right place? Is someone responsible for replying, and can they understand the request without making the customer start again?
Then try the same thing when nobody is available. The customer needs an honest explanation of what happens next. Don't promise a callback time your team cannot keep.
Microsoft's handoff guidance recommends defining what the conversational experience can do and providing another route when it cannot help. The tests above are a practical way to check your own setup, not a claim that every chatbot includes a reliable handover.
Count the work left after the chat
Agree what a useful outcome looks like before starting a small trial.
Record the questions answered correctly and the answers that needed correction. Track requests passed to the team separately, including whether anyone followed up. Include the effort spent maintaining the source material and reviewing conversations.
Read the definitions behind a dashboard's numbers. In Copilot Studio's outcome reporting, a session can be recorded as resolved after the end-of-conversation topic is reached and the customer does not reply before the session times out. That is different from verifying that their request was completed.
If the main problem is website traffic without enquiries, start with the enquiry-path diagnostic. A new chat channel may be beside the point.
Use the questions to scope the work
Your worksheet should make the buying decision more specific. You might need clearer service pages. You might have enough reliable information for a limited assistant, with a person handling the exceptions. Or the useful work may depend on a booking-system connection that needs separate scoping.
CBD's website design and update service covers content and enquiry handovers, with website assistants scoped around approved information.
Bring a few anonymised questions and the answers your team currently gives. Include one where the right answer is “someone needs to check”. That boundary belongs in the brief before the chatbot is built.