AI Search, Crawler, Bot Analytics

Track visits from AI search engines, crawlers, and bots like ChatGPT, Claude, and Perplexity to understand how AI platforms access your website

Promptmonitor allows you to track AI search engines, crawlers, and bots visiting your website. This provides you with insights into traffic from AI platforms like ChatGPT, Claude, and Perplexity.

AI Analytics Setup Guide

Server-side analytics tracks AI agents, crawlers, and all other bots that don't run JavaScript. To enable AI traffic tracking, add PromptMonitor tracking to your server middleware.

You can do this using our REST API or a Next.js middleware.

REST API

Use this endpoint from your server middleware:

Endpoint

POST https://analytics.promptmonitor.io/v1/visits/track

Headers

Content-Type: application/json

Body

{
  "website_id": "your-website-id",
  "request_path": "/current/path",
  "request_method": "GET",
  "request_headers": {
    "user-agent": "User-Agent header from request",
    "referer": "Referer header from request",
    "host": "Host header from request"
  }
}

Implementation Tips

  • Add this call in your server middleware to cover all pages

  • Make the call non-blocking to avoid adding latency

  • Handle errors silently so they don't break your pages

Example with cURL

curl -X POST https://analytics.promptmonitor.io/v1/visits/track \
  -H "Content-Type: application/json" \
  -d '{
    "website_id": "your-website-id",
    "request_path": "/",
    "request_method": "GET",
    "request_headers": {
      "user-agent": "GPTBot/1.0",
      "referer": "",
      "host": "example.com"
    }
  }'

Next.js

Create a middleware.js file at the root of your Next.js project:

// middleware.js
import { NextResponse } from 'next/server';

export function middleware(request) {
  // Skip static assets
  if (request.nextUrl.pathname.match(/\.(js|css|png|jpg|jpeg|gif|ico|svg)$/)) {
    return NextResponse.next();
  }

  // Track the visit (non-blocking)
  fetch('https://analytics.promptmonitor.io/v1/visits/track', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      website_id: 'your-website-id', // Get from Web Analytics page
      request_path: request.nextUrl.pathname,
      request_method: request.method,
      request_headers: {
        'user-agent': request.headers.get('user-agent'),
        'referer': request.headers.get('referer'),
        'host': request.headers.get('host')
      }
    })
  }).catch(() => {
    // Fail silently to avoid breaking your app
  });

  return NextResponse.next();
}

export const config = {
  matcher: '/((?!api|_next/static|_next/image|favicon.ico).*)',
};

Save this file and deploy. The middleware will automatically track all AI bot visits.

AI Crawlers, Bots that we track

  • ChatGPT: OpenAI's conversational AI and search

  • Claude: Anthropic's AI assistant

  • Perplexity: AI-powered search engine

  • Gemini: Google's AI platform

  • DeepSeek: Advanced AI search

  • Others: Various AI crawlers and bots

Debugging

Not seeing AI traffic?

  • Verify your website_id from the Web Analytics page

  • Add a console.log to confirm middleware is running

  • Check that the API endpoint is accessible from your server

Need Help?

If you have any questions, kindly check the Web Analytics overview, or: