Adding web content
Point your Aijent at more pages from your website, and control how much it reads.
To add web pages, open the Knowledge tab, put the address in Page address, and choose whether to read just that page or that page plus the pages it links to. The second option follows links several levels deep and can use a large part of your monthly page allowance, so aim it at a section rather than your whole site.
- Open the Knowledge tab.
- Under Add web content, put the address in Page address.
- Choose what to read under What should we index?
- Select Add to knowledge base.
Two choices, and they behave very differently.
Just this one page reads exactly that page. Nothing else. Use it when you know precisely where the information lives, such as a pricing page or one detailed guide. It costs one page of your allowance.
This page and pages it links to follows links several levels deep and can read a great deal. Use it for a whole documentation section or a help site. It can consume a large part of your monthly allowance, so aim it at a section rather than your entire site.
When you choose the second option, two extra fields appear: Only include
pages and Skip pages. Use them. Adding /blog/* to Skip pages, for example,
keeps a large blog out of your knowledge and out of your allowance.
Nothing is skipped automatically here
During setup your Aijent deliberately skipped blogs, careers and legal pages. When you add web content yourself, it does not. Whatever you point it at is what it reads, so use Skip pages if you want anything left out.
If a page will not read
Some websites block automated access. If that happens you will see the page marked as unread with an explanation underneath. See keeping it current for what the statuses mean.