More people now ask an AI assistant questions like “Where is a food pantry open today in Ames?” The assistant answers from pages its search crawler has read. A few settings decide whether your directory is part of that answer.
Check that your site is visible
- In WordPress, go to Settings > Reading and make sure Discourage search engines from indexing this site is not checked.
- Keep Search engine structured data on under Resources > Settings > General > Advanced. Each resource page then tells search tools its hours, phone number, and the date its information was last checked.
Allow the AI search crawlers
These crawlers read pages so their assistants can answer questions and link back to you:
| Crawler | Used by |
|---|---|
OAI-SearchBot | ChatGPT search |
Claude-SearchBot | Claude search |
PerplexityBot | Perplexity |
They are allowed unless something blocks them. If your site has a custom robots.txt, or an SEO plugin that edits it, make sure it does not block them. To allow them explicitly, add:
User-agent: OAI-SearchBot
Allow: /
User-agent: Claude-SearchBot
Allow: /
User-agent: PerplexityBot
Allow: /Search crawlers are different from crawlers that collect data to train AI models, such as GPTBot and ClaudeBot. You can block training crawlers and still allow search crawlers.
Check your host and firewall
Some hosts, firewalls, and CDNs block AI crawlers no matter what robots.txt says. For example, Cloudflare has a setting to block AI bots. If you use one, check its bot settings and allow the search crawlers above.
Keep listings fresh
Search tools prefer information that looks current. Each resource page shares its Info last verified date, so update it whenever you confirm a listing. Pro’s freshness reminders can help.
Let assistants search your directory directly
Assistants can also connect to your directory and search it live. See Connect AI assistants.
Still stuck? Email us