Internal knowledge base chatbot: search company files with AI

Open the internal assistant, choose Knowledge and add a connected folder source. WebChatAgent supports sources such as SharePoint, Google Drive, Nextcloud or WebDAV and SFTP. Select the system your team already maintains instead of copying documents into a second unmanaged repository.

Write down the source owner, intended employee group and exact question this folder should answer. In the Northstar example, the owner maintains travel policies for all employees; payroll data belongs to a different audience and must stay outside this assistant.

Enter the base URL, username and a dedicated app password. Use a service identity created for this assistant, grant read-only access and restrict it to the required library. Test Connection once and continue only when the UI confirms that the credentials and endpoint work.

Never reuse an administrator or employee password, and never place credentials in prompts, screenshots or source files. Store the secret in an approved manager, record who rotates it and revoke it when the connector is retired.

Browse the connected storage and choose the approved policies root instead of the account root. Include subfolders only when every included team and document is suitable for the same chatbot audience. A smaller scope reduces duplicates, indexing time and accidental retrieval.

In the demo, `/company/policies/approved` is allowed. `/company/archive` and `/company/restricted` are outside the intended knowledge. If the service account can see more, narrow the storage permission first and then add filters as defense in depth.

Set a daily or weekly sync that matches how quickly the owner publishes policy changes. Restrict file types to formats the team maintains and exclude temporary, archived and restricted locations with explicit patterns such as `/archive/**` and `/restricted/**`.

Save the source and record the expected inventory before indexing. Exclusion patterns reduce irrelevant retrieval, but they are application configuration—not an authorization boundary. The connected account itself should be unable to read secrets the assistant never needs.

Wait for indexing to complete, then compare the visible file inventory with the written expectation. Open the travel policy preview and verify the exact current sentence, effective date and approved file path. The initial Northstar version contains USD 68 once.

Confirm that archived versions and restricted payroll files do not appear. If an unexpected file is present, stop: correct account permission, root or exclusion rule, delete the incorrect index entry and only then run answer tests.

Start a fresh chat and ask: “What is the current daily travel expense allowance, and where can I find the approved policy?” Write the expected value and path down before looking at the answer so a fluent but wrong response cannot pass.

The verified assistant returns USD 68 and the approved travel-policy location. Save the question, expected fact, observed answer, source path and date as a regression test. Missing source evidence or an archived value fails the test.

Ask for the fictional payroll approval code that exists only under the excluded restricted path. The safe result states that the information is unavailable in the approved company knowledge and reveals no code, path fragment or guessed alternative.

A positive answer test is only half an acceptance test. Keep at least one negative control for every sensitive source boundary. If the value appears, disable the assistant for that audience and fix storage permissions before changing prompts or model settings.

The fictional policy owner changes the approved document from USD 68 to USD 72, effective August 11, 2026. Do not edit a chatbot answer or paste a correction into another knowledge record. Update the maintained company file and preserve its owner and version history.

Trigger Reindex for the controlled test and wait for completion. Reopen the indexed preview: it must show USD 72 and no longer contain USD 68. In normal operation the scheduled sync performs this refresh; manual reindex is useful for urgent changes and acceptance tests.

Start a fresh chat so the result cannot reuse an earlier conversational turn. Ask the same travel-allowance question. The verified answer now returns USD 72, includes the same approved path and does not repeat the old USD 68 value.

This proves the complete chain: authentication, folder scope, exclusions, indexing, retrieval, source update, reindex and fresh retrieval. Add the positive, negative and update tests to the release checklist and repeat them after important source, permission, prompt or model changes.