The Crawl Price Index ("the Service") publishes measurements of how the web responds to AI crawlers — block rates, observed pay-per-crawl prices, paywall signals, per-country breakdowns, and their movement over time. Headline figures are published freely. The full dataset, per-country editions, API access, and trend history ("the Dataset") are provided to paying subscribers ("Terminal") under the licence below.
Headline figures shown on the public site are licensed CC-BY-4.0: free to quote and republish with attribution to "The Crawl Price Index (crawlpriceindex.com)". We encourage citation.
Subscription to Terminal grants you a single-subscriber, non-exclusive, non-transferable licence to access and use the Dataset for your own internal business or editorial purposes.
You may publish limited quotations and derived insights from the Dataset with attribution (e.g. citing a figure in an article), provided you do not reproduce the Dataset itself or a substantial extract of it.
Your API key is personal to your subscription and rate-limited. Automated bulk mirroring, scraping designed to reconstruct the full Dataset for redistribution, or sharing a key across organisations is a breach of this licence and may trigger rate-limiting, suspension, or termination.
Terminal is billed monthly via Stripe. VAT is handled by our payment provider as merchant of record where applicable. You may cancel at any time; access continues until the end of the paid period and your key is deactivated on lapse.
Measurements reflect what public robots.txt files and crawler-facing HTTP responses disclosed at scan time. robots.txt figures record stated preferences, which sites may or may not enforce. Observed prices change without notice. The Service is provided "as is", without warranty of accuracy, completeness, or fitness for a particular purpose. It is not legal, financial, or licensing advice.
To the maximum extent permitted by law, our liability arising from your use of the Service is limited to the amount you paid in the preceding three months.
We may update these terms; material changes will be dated here. Questions, licensing enquiries, or bulk/enterprise access: hello@crawlpriceindex.com.