Microsoft Clarity has added another new AI metric, this one is called AI Scrape-to-Referral insights. It puts a single metric on the AI scrape activity against referral traffic. The purpose is to give you insight into whether it is worth having that AI bot crawl your site, and whether there is enough return and value to keep allowing that bot to scrape your website.
What is this metric. Microsoft wrote, “With the new AI Scrape-to-Referral Ratio card and direct links into session recordings, you can evaluate the tradeoff between content extraction and traffic return, then investigate how AI-referred visitors behave once they arrive.”
There is a new AI Scrape-to-Referral Ratio card which compares AI scrape activity against referral traffic in a single metric, helping you understand whether AI platforms are returning value in the form of visits rather than only extracting content.
Here is a graphic Microsoft uploaded of this metric:
This seems to say that this AI tool sent 41 referrals to your website, and if you do the math, it has a 6,000 scrapes-to-1-referral ratio based on its crawl activity related to referral activity.
More metrics. The Bot Analytics report now shows these metrics, as Microsoft said:
- AI Scrape-to-Referral Ratio card: Compare AI scrape activity against referral traffic in a single metric, helping you understand whether AI platforms are returning value in the form of visits rather than only extracting content.
- Operator-level breakdown: Review a ranked list of referring operators to identify which AI sources are sending traffic back and which ones may have high scrape activity with limited referral return.
- Mapped-domain coverage safeguards: Ratios are calculated only across correctly mapped domains, helping prevent misleading rollups when CDN and Clarity coverage do not fully align.
- Clearer handling of coverage gaps: When your AI bot data and referral traffic data cover different domains, Clarity provides context so you understand how domain coverage may affect the Scrape-to-Referral Ratio.
- Direct path to session recordings: Click from referral insights into the Recordings tab with filters already applied for the selected AI referral source, making it easier to validate attribution and inspect real user journeys.
- Better traffic quality analysis: Move beyond top-line referral counts to see whether AI-referred visitors scroll, engage, convert, or leave quickly, so you can compare which AI sources are driving valuable traffic.
How do you see it. Microsoft posted these steps to access this data for your site(s):
- Open Bot Analytics in the Clarity AI Visibility Dashboard.
- Review the AI Scrape-to-Referral Ratio card to understand overall referral return and operator-level performance.
- Check the coverage context to confirm your mapped domains are aligned across your connected data sources.
- Select a referral source from the card to open session recordings filtered for
- Watch recordings to evaluate visit quality, including engagement patterns and meaningful actions such as purchases, sign-ups, or form fills.
- Use those insights to compare AI sources and refine how you measure the value of AI visibility for your site.
Why we care. Many site operators ask, is it worth allowing AI crawlers to access, scrape, and use my site’s content? Well, here is a new metric that puts a number on that figure, allowing you to be more informed about which AI crawlers you allow and which AI crawlers you disallow.
What is the ratio you are aiming for when making that decision? Well, that depends on the site, the business, and the purpose of the website.
