CloudFront AI crawler tracking uses standard logs in S3, a Lambda forwarder, and HitKeep AI fetch ingest.
Use this path when a client site is served by CloudFront, S3, or a static-site host that can write CloudFront-style logs.
Recommended setup
This setup records server-side crawler fetches. It does not replace the browser tracker. Keep hk.js installed so HitKeep can also see AI-referred human visits, goals, events, and automatic interactions.
CloudFront -> S3 standard logs -> Lambda parser -> HitKeep AI fetch ingestWhat to configure
HitKeep full AWS guide includes an example Lambda forwarder and the required environment variables.
- A HitKeep site for the domain.
- A scoped API client token for that site.
- CloudFront standard logging enabled.
- An S3 bucket for logs.
- A Lambda function that filters known AI crawler user agents.
What to forward
Do not send request bodies or form data. The AI fetch endpoint is for request metadata that explains crawler visibility.
- Requested path.
- Hostname.
- Status code.
- Content type.
- Response time.
- Bytes served.
- User agent.
How to verify
Correlation needs time because it compares fetched paths with later AI-referred human visits.
- Open AI Visibility in HitKeep.
- Check that fetch volume appears for the site.
- Filter by assistant family.
- Look for 4xx or 5xx errors.
- Wait for normal hk.js traffic before reading the correlation report.