---
title: "How to Track GPTBot, ClaudeBot, and PerplexityBot | HitKeep"
description: "Track AI crawler fetches from GPTBot, ClaudeBot, PerplexityBot, and related AI user agents by forwarding server-side request logs into HitKeep."
canonical: "https://hitkeep.com/guides/tracking/track-gptbot-claudebot-perplexity/"
---

# How to Track GPTBot, ClaudeBot, and PerplexityBot

HitKeep tracks GPTBot, ClaudeBot, PerplexityBot, and other AI crawler fetches when you forward server-side request records into the AI fetch ingest endpoint.

Browser JavaScript is not enough for this job because most AI crawlers do not run `hk.js`.

Managed option

## Connect a scoped AI crawler feed.

Forward only known crawler request records, then verify fetches and errors in the AI Agents Crawlers tab.

[Choose Business annual](https://cloud.hitkeep.eu/signup?utm_source=hitkeep_docs&utm_medium=ai_visibility&utm_campaign=agency_pilot&utm_content=ai_crawler_tracking&plan=business&billing=annual)[Read AI fetch ingest guide](https://hitkeep.com/guides/tracking/ai-fetch-ingest/)

## What to collect

Forward records from the layer that can see crawler requests. Each record should include the requested path, hostname, status code, user agent, and optional response metadata such as content type, bytes served, and response time.

- CloudFront standard logs.
- nginx or Caddy access logs.
- App server middleware.
- Load balancer logs.
- A static-site edge pipeline.

## User agents to watch

| Family | Common user agents |
| --- | --- |
| OpenAI | GPTBot, ChatGPT-User |
| Anthropic | ClaudeBot, Claude-Web |
| Perplexity | PerplexityBot |
| Google AI | Google-Extended, GoogleOther, Google-Safety |
| Apple | Applebot-Extended |
| Meta | meta-externalagent, meta-externalfetcher |
| Amazon | Amazonbot |

Do not treat this table as a consent or robots policy. It is only a reporting filter for requests you already observe at the server or edge.

## Send records to HitKeep

Create a scoped API client for the site, then post matching crawler requests to the AI fetch endpoint.

```
POST /api/sites/{id}/ingest/ai-fetch
```

## Minimum record shape

| Field | Why it matters |
| --- | --- |
| Hostname and path | Groups fetches by the site and page requested |
| User agent | Identifies the AI crawler family |
| Status code | Shows crawler errors and blocked pages |
| Content type | Separates HTML pages, assets, and feeds |
| Response time and bytes | Highlights slow or unexpectedly heavy responses |

## Verification checklist

- Generate or wait for matching crawler requests in your logs.
- Confirm the forwarder only sends AI crawler families.
- Open **AI Agents**, switch to the **Crawlers** tab, and check fetch volume by assistant family.
- Filter by status code to find 404, 403, 429, and 5xx responses.
- Compare fetched paths with AI-referred human visits after enough time has passed.
- Keep one saved report for crawler fetch volume and one for AI referral visits so clients can see that the two signals answer different questions.

## How this differs from robots.txt

`robots.txt` tells crawlers what you allow or disallow. HitKeep reporting tells you what matching AI crawler user agents actually requested and how your site responded.

Use both. Keep robots rules and crawler policies in the site configuration, then use HitKeep to verify crawler access, errors, response types, and later AI-referred visit patterns.

## What appears in the dashboard

AI fetch records are shown in the **Crawlers** tab of the **AI Agents** page. HitKeep groups requests by assistant, operator family, resource type, path, status code, response time, and bytes served. The browser tracker separately records human visits from AI referrers. Correlation only works when both signals exist.

## Related

- [AI Fetch Ingest](https://hitkeep.com/guides/tracking/ai-fetch-ingest/)
- [AI visibility reporting for SEO agencies](https://hitkeep.com/use-cases/ai-visibility-seo-agencies/)
- [AI crawler logs vs AI referral traffic](https://hitkeep.com/use-cases/ai-crawler-logs-vs-ai-referral-traffic/)
- [CloudFront AI crawler tracking](https://hitkeep.com/guides/tracking/cloudfront-ai-crawler-tracking/)
- [AI Visibility Dashboard Reference](https://hitkeep.com/guides/analytics/ai-visibility/)

[Previous CloudFront AI crawler tracking](https://hitkeep.com/guides/tracking/cloudfront-ai-crawler-tracking/)[Next Period-over-period comparison](https://hitkeep.com/guides/analytics/comparison/)