CUSTOMER-OWNED LOG DRAIN
Your crawler data, in your PostHog
The free scan is an outside probe. This flow uses your own Vercel logs, filters them to crawler events, checks vendor-published IP ranges where available, and sends the reduced event directly into the PostHog project you control. The free tier also contributes a normalized domain-level event to ReadableByAI's commercial benchmark network.
The data path
Your Vercel drain → ReadableByAI filter → your PostHog project + ReadableByAI benchmark panel
Vercel sends a log batch to an opaque endpoint. The endpoint accepts only requests with your separate drain secret, rejects entries for other hostnames, drops non-crawler traffic, strips query strings, and never forwards the client IP or raw user-agent. The customer event contains the configured domain, bot token, verification verdict, exact path without its query, status, and timestamp.
This is still transient processing: the function necessarily receives the Vercel batch and briefly reads a crawler entry's IP to compare it with a published network. The privacy boundary is minimization—not the inaccurate claim that no processing occurs. The contributed copy retains domain, vendor/bot, verdict, route category, status class, latency band when available, and an hourly time bucket. It excludes IP, exact path, query, cookies, referrer, headers, and raw user-agent.
Create the connection
Enter a domain you administer and the write-only project token from the PostHog project that should receive the events. US and EU PostHog Cloud are supported. The project token is encrypted into the endpoint credential; do not enter a personal API key.
Use the write-only project token beginning with phc_, never a personal API key. Your destination is encrypted into the opaque URL. ReadableByAI retains the contributed benchmark fields described above, not your PostHog credential.
Questions
What does the customer own?
The PostHog project, event stream, insights, retention settings, access controls, and deletion controls for the customer event. ReadableByAI owns the normalized contributed dataset and may commercialize derived domain-level intelligence, benchmarks, reports, APIs, and models under the Data Contribution Terms. Removing the Vercel drain stops new deliveries and contributions immediately.
Why does the free tier contribute data?
The contributed panel is the consideration for free hosted processing. It lets ReadableByAI build cross-site intelligence about real crawler behavior while the customer keeps its operational dashboard. The contribution is disclosed at setup and defined field-by-field in the Data Contribution Terms.
How is the endpoint authenticated?
The endpoint URL contains an AES-GCM-encrypted destination configuration. A different, randomly generated secret must be sent in a custom Vercel drain header on every POST. The endpoint stores only that secret's hash inside the encrypted configuration. Tampered URLs and missing or incorrect secrets fail closed.
What does “verified” mean?
The request IP matched a network published by the vendor for that crawler or outbound product traffic. “Impostor” means the vendor publishes a usable range and the address did not match it. “Unknown” means there was no usable source, the source could not be fetched, or the input could not be parsed. It does not prove what a vendor indexed or cited.
Which vendors can be checked?
The current registry checks OpenAI, Anthropic, Perplexity, Microsoft bingbot, and Googlebot using vendor-published sources. Other recognized crawler claims remain unknown rather than receiving an invented verdict. Vendor sources can change, so the registry is maintained data.
What happens if my PostHog destination rejects a batch?
The endpoint returns an error instead of acknowledging and silently dropping the batch, allowing Vercel's drain delivery behavior to surface and retry the failure. Regenerate the endpoint if the region or project token was entered incorrectly.
How do I stop or rotate it?
Remove the drain in Vercel to stop. To rotate, generate a new endpoint and secret, update the drain, then discard the old values. Because there is no customer account or database row, ReadableByAI cannot revoke one destination independently on your behalf.