How to train a chatbot on your website and PDFs
A practical beginner guide to crawling your website, uploading a PDF and verifying that your chatbot answers from both sources.
A chatbot can only give reliable company-specific answers when it can retrieve the right source content. In WebChatAgent, “training” means indexing your website and documents for retrieval-augmented generation (RAG). It does not retrain or fine-tune the underlying language model.
This guide starts with an entirely empty account. You will create a chatbot, crawl one website page, upload a small PDF and then test one answer from each source.
What you will have at the end
- A new chatbot with a clean knowledge base
- One successfully indexed website source
- One successfully indexed PDF document
- Two verified answers grounded in your own content
Before you start
- A WebChatAgent account with permission to create a chatbot
- A public website page that you are allowed to index
- The downloadable demo PDF from this tutorial or your own supported document
What “training” means here
WebChatAgent extracts the readable content, splits it into useful passages and creates vector embeddings. When someone asks a question, the most relevant passages are retrieved and supplied to the language model as evidence.
This RAG approach keeps your knowledge replaceable and current. If a page changes, you can re-index it without training a new model. The quality therefore depends on clean source content, sensible crawl scope and realistic testing.
01–09
Train your chatbot step by step
Start from the empty dashboard
Create your first assistant from the clean account.
After signing in, the dashboard shows that no AI assistants exist yet. Select the primary create button to start with a clean chatbot rather than reusing an assistant that already contains sources.
Create Tutorial Lab
Give the new assistant a clear name and keep the safe defaults.
Enter “Tutorial Lab” as the name. The default internal provider is sufficient for this guide, so you do not need to add a personal API key.
Create the chatbot. WebChatAgent opens its configuration dashboard after the assistant has been saved.
Open Data Sources
Move to the area that manages the chatbot knowledge base.
Select “Data Sources” in the chatbot navigation. A new chatbot displays an empty state because no website, document or connected workspace has been indexed yet.
Choose “Add Data Source” to open the source selector.
Choose Website as the first source
Open the crawler settings for a public webpage.
The selector offers websites, documents, text input and connected knowledge tools. Select “Website”.
Use a website source when the information already lives on public, crawlable pages and should retain its page URL as a reference.
Enter the URL and limit the crawl depth
Index one controlled page before expanding to a whole site.
Enter https://webchatagent.com/train-chatbot-on-your-data in the URL field and set the crawling depth to 0. A depth of 0 indexes only this page, which keeps the tutorial fast and predictable.
Leave automatic re-indexing off for this one-time test. For production content, Premium can refresh a source daily. Advanced settings can include a specific CSS content area or exclude URL patterns such as login, cart or legal pages.
Select “Add & Index” to start extraction and indexing.
- Depth 0: only the entered page
- Empty depth: follow all reachable subpages
- CSS selector: keep only the meaningful content container
- Exclude patterns: prevent irrelevant or duplicate URLs
Wait for the website to be indexed
Confirm that the source is ready before testing answers.
The new source may briefly show a pending or processing state. Wait until it changes to “Indexed” or “Completed”. The indexed page and extracted content count confirm that the crawler finished successfully.
If the source fails, open its details first. Check the URL, robots.txt accessibility, crawl depth and any restrictive CSS selector before retrying.
Upload the demo PDF
Add a second source with a unique fact that is easy to verify.
Select “Add Data Source” again, choose “Documents” and upload the demo knowledge-base PDF linked above. The file contains the unique support code NORDSTERN-42, which does not appear on the website.
Keep AI optimization off for this small, already structured document. It is useful for poorly formatted source material, but it is not required for clean text.
Verify both knowledge sources
Make sure the website and PDF are ready together.
The Data Sources list should now show both the website URL and the PDF with successful statuses. This is the checkpoint that prevents confusing model behavior with an unfinished indexing job.
Open a source when you need to inspect extracted content, change its category, trigger a new index or remove outdated knowledge.
Verify the PDF answer in the live chat
Confirm the unique PDF fact in the same chat visitors will use.
Open the live chatbot preview and ask: “What is the verification code in the uploaded tutorial PDF? Answer in one sentence and name the source.” The screenshot shows the exact question and the real answer from the temporary tutorial account.
The answer must contain NORDSTERN-42 and identify the Northstar Services demo knowledge base. The complete website and PDF Knowledge Test results are shown directly below this step, so both indexed sources are still verified with fixed questions.
- Ask questions whose answers appear explicitly in the source.
- Include unique names, codes or numbers when validating retrieval.
- If an answer is weak, improve the source text before expanding the prompt.
Example & result
Test both indexed sources with fixed questions
These are the exact questions used in the temporary tutorial account. Each result keeps the relevant Knowledge Test area complete; when one viewport is not enough, overlapping frames show the score and every answer without cutoffs.
Website example: verify supported data sources
This question was run against the indexed WebChatAgent website page with crawl depth 0.
Exact test input
Can I train a WebChatAgent chatbot with both website pages and PDF documents?
Expected result
The answer confirms that website pages and PDF documents can both be used as knowledge sources.
What was actually verified
The real Knowledge Test answered 5 of 5 variants (100%) and grounded the result in the indexed website content.
PDF example: verify a unique document fact
This question targets a fact that exists only in the uploaded demo PDF, making the source easy to verify.
Exact test input
What is the unique verification code in the uploaded tutorial PDF?
Expected result
Every answer contains the exact value NORDSTERN-42.
What was actually verified
The real Knowledge Test answered 5 of 5 variants (100%); every generated answer contained NORDSTERN-42.
Pro tips
Tips for better answers from your knowledge base
A stronger model cannot compensate for missing or contradictory source content. Improve the knowledge base first, then choose the model that meets your quality, speed, quota and data-residency requirements.
Start with a fast model
Use a model marked fast or fastest for the first retrieval tests. Move to a smart model only when a fixed question set shows a real improvement in reasoning or wording. The multiplier in the model picker shows how strongly a choice counts against the message quota.
Include data residency in the model decision
If processing location matters, choose an option marked EU in the model picker. Evaluate residency, answer quality, latency and quota usage together instead of selecting solely by model name.
Write one topic per section
Use descriptive headings, short paragraphs and explicit facts. Keep a product rule, price condition or process together with its context so the retrieved passage can answer the question without relying on a distant section.
Prepare documents for extraction
Prefer selectable text, clear heading levels and simple tables. Run OCR on scanned PDFs, explain abbreviations once and avoid important facts that exist only inside decorative images.
Remove duplicates and conflicts
Do not index print views, tag archives, translated duplicates or multiple outdated policy files at the same time. Conflicting versions make retrieval less predictable even when every source was indexed successfully.
Build a repeatable evaluation set
Collect 10 to 20 real customer questions with expected source facts. Re-run the same set after changing content, crawl settings, prompts or models so improvements are measurable instead of subjective.
Troubleshooting
Troubleshooting and best practices
Most training problems come from source quality or crawl scope, not from the language model itself.
The website stays pending or fails
Check that the URL is public, loads without a login, permits crawling and is not blocked by an overly narrow CSS selector.
Too many irrelevant pages are indexed
Reduce the crawl depth and exclude search, account, cart, tag, legal and parameterized URLs.
The bot misses a PDF fact
Confirm that the document status is complete and that selectable text or OCR output contains the fact clearly.
Answers become outdated
Enable an appropriate re-index interval for changing pages and remove or replace obsolete documents promptly.
Your chatbot now answers from your content
Continue with your real website pages and documents, then test the questions your customers actually ask. Keep the knowledge base focused, current and free of conflicting versions.