AIRankWPHow to check which AI crawlers visit your WordPress site
Short answer: your server access log knows. Search it for the bot names, count the hits, and check a few IPs against the vendor's published ranges. Ten minutes if you know where the log lives. This post shows you where, what to grep for, and how to spot a fake.
"Is ChatGPT reading my site?" is the question behind half the emails about AI SEO. The honest answer is not in Google Analytics and not in Search Console. It is in the access log, the boring text file your web server writes for every request. Let us go look at it.
Step 1: find the log
Where it lives depends on your host:
- cPanel hosts: Metrics > Raw Access, download the current log.
- Nginx on a VPS: usually
/var/log/nginx/access.log. - Apache:
/var/log/apache2/access.logor a per-site variant. - Managed WordPress hosts: Kinsta, WP Engine and friends expose logs in their dashboards; some make you ask support.
If you are on Cloudflare, remember it sits in front of your server. Your origin log only shows requests Cloudflare let through, which is actually useful: it tells you what got past the firewall.
Step 2: grep for the names that matter
The current names, straight from OpenAI's bot documentation and Perplexity's bot guide:
- OAI-SearchBot - ChatGPT search
- GPTBot - OpenAI training
- ChatGPT-User - a real person asking ChatGPT to open your page
- PerplexityBot and Perplexity-User - Perplexity's index and live fetches
- ClaudeBot, Claude-SearchBot, Claude-User - Anthropic's three
- Google-Extended shows up in some logs for specific checks, though normal Googlebot does the Search work
On a Linux box the check is one line:
grep -Ei 'OAI-SearchBot|GPTBot|ChatGPT-User|PerplexityBot|ClaudeBot' /var/log/nginx/access.log | awk '{print $1, $12, $13, $14}' | sort | uniq -c | sort -rn | head -20
The count per bot per IP is the interesting part. One hit from OAI-SearchBot on your best guide is worth more than fifty hits on your tag pages.
Step 3: check the bot is real
Anyone can write "GPTBot" in a User-Agent header. It takes one curl command, so scrapers do it constantly. Two checks separate real from fake:
- Reverse DNS. Take the IP and run
host 1.2.3.4. Real OpenAI crawlers resolve back to openai.com domains. Real Perplexity bots resolve to perplexity.ai. Random hosting providers in the result mean fake. - Behavior. Real crawlers fetch robots.txt first and space their requests. A "GPTBot" pulling 300 pages in a minute is neither polite nor OpenAI.
Step 4: decide what the numbers mean
A few patterns we see in audits:
- Search bot hits but no citations: normal. Being read is step one; being quoted depends on whether your content is the best answer available.
- ChatGPT-User or Perplexity-User hits: someone asked the assistant about you or your topic and it opened your page live. That is the closest thing to a warm lead a log file can show you.
- Zero AI bot hits on an established site: check your robots.txt and firewall before assuming nobody cares. Something may be blocking them.
The easier way
This is the part where we mention our own plugin, so take it with the appropriate grain of salt: AIRankWP's AI traffic tab counts assistant referrals and self-reported bot visits for you, off by default, with the caveat that User-Agents can lie and caches can hide requests. The log file is still the ground truth. The plugin just saves you the grep.
FAQ
How far back should I look? A month is a good window. AI crawling is bursty; a single day tells you nothing.
Should I block the fakes? Rate-limiting abusive requests is fine. Blocking by User-Agent alone is theater, since fakes rotate names. Block by behavior or verified IP ranges.
What if my host refuses to give me logs? Ask support for raw access logs once. If the answer is no, Cloudflare in front of the site gives you bot analytics on the free plan.