Chatbot for business: how to build an AI chatbot that answers from your documents

Technologies
Services
Date
  • 20/11/2026

Many companies already have a chatbot on their website, and many end up removing it. The ones built on buttons and fixed answers fall short as soon as someone asks something nobody planned for. The ones that simply plug in a language model answer everything, including what they do not know, and sometimes make up a returns policy or a price.

What a business needs is something else: a chatbot that answers only from its own knowledge, says where each answer comes from and hands the case to a person when it does not know. This guide explains how a chatbot like that works, what companies use it for and what to consider before launching one.

A chatbot that answers from the company's documents and cites the source

What kind of chatbot does a business need

How it answersWhen it fitsThe risk
Rule-based chatbotMenus and hand-written answersVery frequent, closed questions (opening hours, order status)Falls short with any open question, and every answer has to be maintained
Generic AI chatbotA language model with what it knows from the internetIdeas, drafts, personal useConfidently answers things that are not about your company, or are not true
AI chatbot with RAGSearches the company’s knowledge base and writes only from what it findsCustomer service, support, internal questionsDepends on the knowledge being organised and up to date
AI agentBesides answering, it takes actions in other systemsMulti-step processes (creating an order, opening a ticket)Needs permissions, limits and supervision

For most business uses, the starting point is the third one. AI agents come later, once the chatbot answers well and you want it to act too.

What companies use an AI chatbot for

  • Customer service on the website: questions about products, shipping, terms or services, at any time and in several languages.
  • Internal chatbot: the team asks about procedures, policies, technical documentation or internal rules, and gets the answer with its source document.
  • Technical and after-sales support: manuals, data sheets and solved tickets, to give the first answer or prepare the case for a technician.
  • Sales: questions about the catalogue, compatibility or which product fits, handing over to a sales rep when the conversation calls for it.
  • Onboarding: what a new employee asks in their first weeks, without always depending on a colleague.

The same engine can answer on the website, on the intranet or in a messaging channel. What changes is the entry point, not the knowledge.

How a RAG chatbot works

RAG (retrieval-augmented generation) means that, before answering, the chatbot searches the company’s information and writes from what it has found.

  1. Validated knowledge base. You gather the documents that count as a source: FAQs, product sheets, terms, manuals, procedures. Someone in the company decides what goes in and what does not.
  2. Indexing. The documents are split into chunks and stored in a vector database (for example, PostgreSQL with pgvector), which can search by meaning and not only by exact words.
  3. Retrieval. For each question, the system finds the chunks about the same thing.
  4. Writing. The language model writes the answer using only those chunks.
  5. Citing the source. The answer says which document it comes from, so the person asking can check it.
  6. Review and handover. If there is not enough information, the chatbot says so and passes the conversation to a person.
How a RAG chatbot works: validated documents, search, answer with its source and handover to a person

We explain the pieces behind it (data, model, code, control) in more detail in our guide to AI architecture.

How to stop it from making up answers

A business chatbot cannot improvise. What works is layering the control:

  • Input filter: detects off-topic questions, attempts to manipulate the chatbot or personal data it should not receive.
  • Writing only from approved content: if the answer is not in the knowledge base, it does not answer with what the model “thinks”.
  • A second model that reviews every answer before showing it: it checks that the answer is backed by the sources and promises nothing the company does not offer.
  • Handover to a person when information is missing or the case calls for it (a complaint, a quote, a customer’s data).
  • A set of test questions run before every change, to know whether the chatbot got better or worse.
  • Conversation logs: they show what people ask and what has no answer, which tells you what to add to the knowledge base.

Data, privacy and AI provider

  • Where the data lives. Hosting in the EU and personal data handled under the GDPR.
  • No training models on your data. The company’s information is used to answer, not to improve a third party’s model.
  • Provider independence. If the system does not depend on a single language model, you can switch providers without rebuilding it when a better or cheaper one appears.
  • Who maintains it. Company knowledge changes; someone must be able to update the base without going through a developer, for example from a content management system.

Off-the-shelf tool or custom chatbot

Off-the-shelf toolCustom chatbot
Set-upFast, with templatesNeeds a project, starting with a pilot
KnowledgeUpload documents or link your websiteValidated sources, with the structure and permissions you need
IntegrationsWhatever the tool includesWith your ERP, CRM, catalogue or ticketing system
ControlWhatever the tool allowsRules, review and handover designed for your case
Data and providerThe vendor’sYou decide where it is hosted and which model it uses
Fits whenSimple questions and low volumeYour own knowledge, several systems, or control and privacy requirements

Common mistakes

  • Loading every company document without reviewing it: the chatbot will also repeat what is outdated or contradictory.
  • Not planning what happens when it does not know the answer.
  • Launching it to every customer at once, with no pilot and no test questions.
  • Measuring only how many conversations there are, not how many are solved well.
  • Forgetting to maintain it: products, prices and policies change, and the knowledge base has to change too.

How to get started

  1. Pick one specific use: customer questions about shipping and returns, or internal questions about one procedure.
  2. Gather and review the sources for that use, and decide who maintains them.
  3. Prepare the test questions with the correct answers.
  4. Run a pilot with a small group and review the conversations.
  5. Measure how many questions it solves, how many it hands over and what is missing from the base.
  6. Extend it to other uses or channels once the first one works.

AI chatbots at Omitsis

We build chatbots with governance and full control: they answer only from the knowledge the company has validated, cite the source, go through three layers of control and keep the data in the EU, with our own service in Node.js that does not depend on any one language model. We connect them to the systems you already use, as part of our AI solutions for business.

If you are thinking about a chatbot for your website or your team, tell us about your case. And if you are looking for someone to build it, our guide on what an AI integrator does and how to choose one may help.

Tell us about your project and we’ll help you make it happen

Whether you already have a defined plan or just an initial idea, we’ll work with you to find the right path.

With offices in Barcelona, we collaborate with clients all over the world.

Omitsis
Balmes 76, Principal 2
08007 Barcelona
(+34) 931 720 287

Which services are you interested in?

Responsible for the data: Omitsis Consulting SL. Purpose of the data: Commercial communication and project management. Data storage: Database hosted at Omitsis Consulting S.L. (EU). Rights: At any time you can consult, modify or delete your information.