The AI Readiness Index
Version 9.0 — specification Effective 2026-09-16.
License: This document and the tests it defines are published under CC BY-SA 4.0. Copy it, adapt it, build your own scanner from it — for any purpose, free, forever — as long as you credit TEN7 as the creator of the AI Readiness Index and license what you build the same way. The scanner that computes a score from this document is separate, proprietary software. Full terms at readinessindex.io/license.
What this is
This document defines what a website has to do to score 100 on the AI Readiness Index. It is written for two readers at once: the engineer building the scanner in this repository, and the consultant explaining a score to a client.
Every check has an ID, a weight, a detection method, and a scoring rubric. If a check is not described here, the scanner should not score it.
We built this because "is your site ready for AI" is currently an unanswerable question. Vendors assert things. Nobody measures. Our answer is a number you can reproduce, defend line by line, and re-run in six months to show what changed.
The Index measures one thing: can machines find, read, trust, and use this website? It does not measure design quality, marketing effectiveness, or whether the content is any good. A beautiful site can score 12. A plain one can score 94.
How to read a score
A result has three parts, and all three matter.
The score (0–100). A single weighted number across eight dimensions.
The gate status. Two checks are gates. If a site blocks AI crawlers in robots.txt or rejects them at the CDN, gated is true — a separate fact reported beside the score, not a ceiling on it. The score still says how well the site is built; the gate says whether any of it is reachable today. A perfectly marked-up site that returns 403 to ClaudeBot is not badly built, it is invisible, and those are different problems with different fixes. Blending them into one capped number made a well-built gated site indistinguishable from a badly-built one — see the changelog for 4.0.
The automated coverage. The percentage of applicable weight the scanner verified on its own. Some tests — log retention, a monthly prompt panel, whether schema is generated from fields or hardcoded in a template — cannot be seen from outside. Those are scored from attestation and flagged. A score of 82 with 91% automated coverage is a much stronger claim than 82 with 64%.
Report all three together. A bare number invites arguments we don't need to have.
Grade bands
| Score | Grade | What it means |
|---|---|---|
| 90–100 | Exemplary | Machine-legible by design. Remaining work is optional or emerging. |
| 75–89 | Strong | Solid fundamentals with specific, addressable gaps. |
| 60–74 | Adequate | Findable and readable, but leaving significant value unclaimed. |
| 40–59 | Weak | Partially legible. Real content is not reaching answer engines. |
| 0–39 | Not ready | Structural problems. Start with retrievability before anything else. |
| Any | Gated | A separate fact, not a grade. AIR-1.1 or AIR-1.2 failed: AI crawlers cannot reach the site, so none of the score above is reaching one today. Before 4.0 this capped the total at 40, which blended two facts into a number comparable to neither. |
grade is always the band the real, uncapped total falls in — a gate failure never changes it. gated is the separate boolean the report checks to decide whether to lead with the access problem, so it can say "Strong at 81.2, and gated — none of that is reaching an assistant today" rather than either hiding the score behind the word Gated or letting it stand alone with no warning attached.
The scoring model
Bands
Every check scores on a 0–4 band. The check's contribution is weight × (band ÷ 4).
Unless a test says otherwise, use the default coverage rubric against the sampled page set:
| Band | Condition |
|---|---|
| 4 | Present and correct on ≥95% of applicable sampled pages |
| 3 | Present and correct on 75–94% |
| 2 | Present on 40–74%, or present everywhere but with correctness errors |
| 1 | Present on 1–39% |
| 0 | Absent, or present but broken everywhere |
Coverage percentages are weighted, not raw page counts. Each money page (see Sampling, step 4) counts as 2 in both the numerator and the denominator; every other sampled page counts as 1. Ten money pages in a 250-URL sample therefore score as a 260-page sample. A failure on /apply costs twice what the same failure on a 2019 press release costs, which is the reason the client names those URLs in the first place.
Tests that are inherently site-wide and binary (a file exists, a header is set) define their own bands in the test entry.
Points
| # | Dimension | Points | Tests |
|---|---|---|---|
| 1 | First contact | 28 | 13 |
| 2 | Rendering and extraction | 15 | 6 |
| 3 | Structured data and the entity graph | 14 | 8 |
| 4 | Crawl, index and licensing hygiene | 14 | 10 |
| 5 | Authorship, provenance, freshness | 8 | 6 |
| 6 | Agent interfaces | 7 | 7 |
| 7 | Alternate representations | 7 | 5 |
| 8 | Measurement and instrumentation | 7 | 7 |
| 9 | Unscored leading indicators | 0 | 6 |
| Total | 100 | 68 |
Dimensions 1, 2 and 3 (First contact, Rendering and extraction, and Structured data and the entity graph) carry 57 of the 100 points — just over half. First contact is heaviest by construction: it is the highest-signal test pulled from six other dimensions, not a new editorial judgment about what matters most.
Dimension 8 carries only 7 of 100 points despite being the most commercially valuable work we do. Weight here reflects what a scanner can observe from outside a site, not what the work is worth. Track measurement maturity in the engagement, not in this number.
Dimension 9 carries none of the 100 — by design, not omission. It holds every leading-tier test (see Versioning and change control, 4.0), grouped on its own rather than scattered through the dimensions that would otherwise host them, so a site's score can never be quietly diluted by a practice almost nobody has built yet. They are still run and still reported.
Applicability and N/A
Each test declares an applicability rule:
always— scored on every site.conditional— scored only if a precondition holds. State the precondition in the check.vertical— scored only for a declared client vertical (higher ed, healthcare, nonprofit, government, research).
A conditional test states its precondition in one of two forms, and never in both:
- A named site fact — a single named predicate over collected evidence, such as
sample_contains_tablesorsite_has_search. Predicates are code, not expressions in a config file, so each one is unit-tested and each one names itself in the N/A reason an auditor reads. - A dependency on another check — written as
requires: AIR-4.1 ≥ 2. A dependency is satisfied when the named check scores band 2 or higher. Below band 2 the foundation is broken, and measuring what sits on top of it produces a finding nobody can act on: telling a client their malformedllms.txtis well cached is noise. A dependency that is itself N/A cascades N/A to everything that depends on it.
Five checks carry a dependency: AIR-4.2 → AIR-4.1, AIR-7.3 → AIR-7.2, AIR-7.5 → AIR-1.10 or AIR-7.2, AIR-6.2 → AIR-6.1, AIR-8.4 → AIR-8.3.
Facts the scanner cannot observe
Six checks are conditional on a business fact no crawler can determine: AIR-4.1 and AIR-4.3 (licensing and access posture), AIR-5.3 (YMYL content), AIR-5.5 (research and policy publishing), AIR-5.6 (original asset production), and AIR-6.8 with AIR-6.9 (named critical flows). AIR-3.3 needs a declared vertical.
These come from an engagement profile supplied per client — never from inference. A fact not declared is N/A with the reason not_declared_in_engagement_profile. The scanner does not sniff a site for medical keywords and decide it is YMYL. A guess inside a number we ask a client to defend line by line is worse than a blank row, and a blank row is an easy conversation: we did not assess this because you have not told us yet.
A pre-sales scan run before any client contact will therefore leave those seven tests unassessed. The score renormalizes correctly around them.
A test ruled N/A is removed from the denominator. Recompute:
score = (Σ earned_points ÷ Σ applicable_weight) × 100
Never score an N/A check as zero. Penalizing a museum for having no Physician markup makes the Index useless.
Record every N/A decision with its reason in the result file. An auditor must be able to see what we chose not to measure.
Gates
Two checks are gates: AIR-1.1 and AIR-1.2.
If either scores 0 or 1, set gated: true and record which one in gate_failures. Nothing else changes: total is computed exactly as in Scoring above, from every applicable check including the gate checks themselves, and grade is the band that real total falls in.
Before 4.0, a gate failure capped the final score at 40 (total = min(ungated_score, 40)) and forced grade to Gated. That produced one number standing for two different facts — how well the site is built, and whether any of it is reachable today — and made them impossible to tell apart: a gated site with excellent structure scored identically to a gated site with none. Reporting gated and total as two separate facts says both; capping said neither.
Verification mode
Every test declares how its band was determined:
auto— the scanner determined the band from evidence it collected.auto-partial— the scanner produced a strong signal, but a human must confirm. Heuristics that can be fooled live here.manual— not observable from outside. Scored from a client attestation, or leftnulland excluded.
Automated coverage counts auto-partial at half weight, because a strong signal awaiting human confirmation is genuinely half-verified:
automated_coverage = (Σ auto_weight + 0.5 × Σ auto_partial_weight) ÷ Σ applicable_weight
The half is not arbitrary. Counting auto-partial at zero puts a ceiling of 0.83 on every possible run, because the always-applicable auto-partial checks are 15.75 points that never leave the denominator. Counting it in full reports 0.96 for a site nobody has confirmed anything about, and erases the distinction the mode exists to draw.
Never let a manual check silently default to 4. An unanswered attestation scores null and drops out of the denominator, and the result records it as unverified.
Attestation against an auto-partial check
Four auto-partial checks have a top band that asks something no scanner can see. This document says so in each case, and each has a different observable ceiling because their band tables differ:
| Test | Unattested ceiling | What the attestation confirms |
|---|---|---|
| AIR-9.3 | 3 | That IndexNow submissions fire on publish, not on a schedule |
| AIR-1.10 | 3 | That llms.txt is generated from the content model rather than hand-maintained |
| AIR-3.1 | 3 | That JSON-LD comes from mapped fields, confirmed with the build team |
| AIR-8.7 | 3 | That sitemaps are submitted and alerts reach a named person |
Each of these four now has a band 3 that describes exactly what a scanner can see, so the attestation lifts 3 to 4 rather than jumping a gap. A test whose band table skips a value cannot express mostly right, and mostly right is the state most real sites are in.
An attestation may raise the ceiling on these four. It never sets a band and never lowers one: the scanner scores what it observed, and a supplied attestation permits that observed band to rise as far as the evidence supports. The result records which points came from a claim rather than a probe, and the report prints them as attested.
Without an attestation the test scores at its observable ceiling and the report says what would be needed to go higher. A client who did the work can prove it; a client who says nothing earns nothing.
Every other auto-partial check is fully observable. It carries that mode because the heuristic can be fooled — AIR-4.3 compares a signed request against an unsigned one, AIR-6.3 calls a live tool — not because it needs a client to vouch for it.
Tests that need a previous run
Four checks are defined against change over time: AIR-4.8 (is lastmod a real signal or deploy noise), AIR-2.4 (do anchor IDs survive a deploy), AIR-5.4 (do dates move only when content moves), and AIR-6.9 (do critical-flow selectors churn).
On a first run there is nothing to compare against. Score the observable half, cap the band at 3, set confidence: medium, and record stability_unconfirmed in the evidence. AIR-5.4 already words its band 3 this way; the other three follow the same rule.
Supplying a previous run directory as a baseline lifts the cap and permits band 4.
Every ceiling — this one, the attestation ceiling above, and a test's own declaration that it could not see further — records the reason it applied. A client told a number was held below the evidence is owed the sentence explaining what would lift it. This makes the re-audit a first-class input to the score rather than a report-writing exercise, and it gives the client a visible, earnable reason the number rises next quarter.
How the audit runs
Sampling
Most tests score against a sampled page set, not the whole site. Build the sample like this:
- Always include: the homepage,
/robots.txt,/sitemap.xml(and every child sitemap),/llms.txt,/.well-known/probes, and every URL in the site's primary navigation. - Stratify by content type. Use the sitemap index, URL patterns, or a declared content-type map. Take up to 10 URLs per type.
- Add depth. Include at least 5 URLs three or more levels deep, to catch templates the homepage never exercises.
- Add the money pages. The client names up to 10 URLs that matter most — programs, services, donate, apply, contact. These are weighted double in coverage math.
- Cap the crawl. Default 250 URLs. Configurable. Record the actual count.
Minimum viable sample is 25 URLs. Mark the run low_confidence only when the sample is under 25 and more URLs were discovered than were sampled. A twelve-page nonprofit site that was crawled in full is a complete measurement, not a weak one.
The representation sub-sample
Four checks want an extra request against every sampled URL: AIR-7.2 ({path}.md), AIR-7.4 (Accept: text/markdown), AIR-7.5 (cache behavior), and AIR-4.4 (X-Robots-Tag on non-HTML). At the default 250-URL cap and one request per second, that is over 800 requests and roughly fifteen minutes of politeness delay stacked on top of the base crawl, against infrastructure that often belongs to a hospital.
Those four tests run against a representation sub-sample: 25 URLs by default, drawn from the main sample, always including every money page. The size is configurable and the selected URLs are recorded in the run metadata, so the sub-sample is reproducible across audits. Twenty-five pages resolves a rubric whose finest distinction is 75% against 95%.
Fetching
Fetch every sampled URL twice:
- Raw — plain HTTP GET, no JavaScript, with a declared TEN7 audit user-agent.
- Rendered — a headless browser with JavaScript enabled.
The difference between those two responses is the entire evidence base for AIR-2.1 and AIR-2.2, which together carry 9 points. Do not skip the second fetch.
Separately, probe a small set of URLs with each AI crawler user-agent to feed AIR-1.2. There is no other way to measure what the edge does: robots.txt states an intention, a CDN rule states a fact, and the two disagree often enough that a readiness score which skipped the question would be worth less than the awkwardness of asking it.
Say plainly what that means, because it is the part an implementer will be asked about. The wire-level User-Agent on those requests carries another company's product token — GPTBot, ClaudeBot, and the six others in the table below. Three constraints make that measurement rather than impersonation, and an implementation that drops any of them is doing something else:
- The requests still identify you. Send your own
Fromheader and anX-Audit-Operatorheader naming your crawler on every probe. An operator inspecting the whole request, rather than the user-agent alone, can see who it was. - Probe only URLs you were already allowed to fetch. The probe runs over the sample, and the sample is filtered by
robots.txtfirst. A probe must never reach, under somebody else's name, a page you declined to reach under your own. - Publish the behavior before you do it, at the URL in your own user-agent string, alongside how to opt out. Disclosure volunteered reads differently from disclosure discovered.
The agent classes
The Index groups the ten agents three ways. AIR-1.1 evaluates all ten against robots.txt. AIR-1.2 probes only the eight that actually issue requests.
| Class | Agents | AIR-1.1 | AIR-1.2 probe |
|---|---|---|---|
| Answer-serving | OAI-SearchBot, ChatGPT-User, PerplexityBot, Claude-User |
yes | yes |
| Training and corpus | GPTBot, ClaudeBot, CCBot, Bytespider |
yes | yes |
| Directive-only | Google-Extended, Applebot-Extended |
yes | no |
Google-Extended and Applebot-Extended are not crawlers. They are opt-out tokens that Google and Apple honor in robots.txt; no request ever arrives carrying either string. Blocking them is a rights decision with no fetch behavior attached, which is why they count in AIR-1.1 and cannot be probed in AIR-1.2. A user-agent matrix that lists them is describing a crawler that does not exist.
Where a band anchor says major agents, it means the four answer-serving agents plus GPTBot and ClaudeBot.
Politeness
Respect Crawl-delay. Default to 1 request per second with 4 concurrent workers. Back off on 429 and 503, honoring Retry-After where it is given, and abandon a host that keeps refusing rather than retrying into a wall. Cache every response to disk so re-scoring never re-crawls.
Consent
Two different questions, and conflating them is how a scanner ends up doing something indefensible.
Whose site may you scan? A client audit runs with the client's agreement, recorded before the crawl. A survey of a public population — see Populations, below — runs without asking each site first, on the same footing as any other published measurement of the public web. What is not acceptable in either case is scanning under a consent you did not obtain. Record which basis a run used, in the run itself.
What may you fetch once you are there? robots.txt, and specifically the group addressed to your own crawler. Name your crawler in your user-agent string, publish a page at the URL that string points to, and stop when a site names you and disallows you.
A scanner may reasonably decide that a blanket User-agent: * disallow does not address it — a * rule is aimed at the anonymous scrape, and a named crawler with a published policy, a rate limit and a working opt-out is not that. If you make that choice, publish it on your crawler's own page and honor Crawl-delay under * regardless, because a rate limit is not an exclusion. What is not defensible is making the choice quietly.
Populations
A survey scores a defined population and publishes the aggregate. Two rules keep it a measurement rather than an accusation.
Cite the population. Record the source list, its publication date and its URL. "How did you pick these" is the first question anyone asks, and the answer has to be a citation rather than a judgment call. Use sources you may lawfully redistribute, and claim no ownership over somebody else's list.
Publish the version and the date. Every result records which version of this Index it was scored against and the date that version took effect. Without both, an old number silently starts meaning something new the next time the Index changes.
Reproducibility
Store the raw evidence — headers, HTML, extracted JSON-LD, screenshots — in a run directory keyed by timestamp. A score you cannot reproduce next quarter is an opinion, not a measurement.
Result schema
The scanner writes one result.json per run. Everything else (Markdown report, HTML dashboard, remediation backlog) renders from this file.
{
"schema_version": "1.0",
"index_version": "1.0",
"site": {
"name": "Example University",
"base_url": "https://example.edu",
"vertical": "higher-ed",
"cms": "drupal-10"
},
"run": {
"id": "2026-09-01T20-14-33Z",
"started_at": "2026-09-01T20:14:33Z",
"finished_at": "2026-09-01T20:41:02Z",
"scanner_version": "0.1.0",
"urls_sampled": 187,
"representation_sample": 25,
"low_confidence": false,
"baseline_run_id": null,
"extraction_method": "readability-density",
"extraction_version": "1"
},
"score": {
"total": 61.4,
"gated": false,
"gate_failures": [],
"grade": "Adequate",
"applicable_weight": 94.5,
"earned_points": 58.0,
"leading_adopted": 2,
"leading_total": 6,
"earnable_points": 88.0,
"automated_coverage": 0.87
},
"dimensions": [
{
"id": 3,
"name": "Rendering and extraction",
"weight": 20.0,
"applicable_weight": 20.0,
"earned_points": 9.5,
"score_pct": 47.5
}
],
"checks": [
{
"id": "AIR-2.1",
"title": "Primary content is present without JavaScript",
"dimension": 3,
"weight": 6.0,
"applicability": "always",
"applicable": true,
"na_reason": null,
"verification": "auto",
"band": 1,
"earned_points": 1.5,
"confidence": "high",
"summary": "Raw HTML contains 22% of rendered text on average across 187 URLs.",
"evidence": [
{
"url": "https://example.edu/admissions",
"raw_text_chars": 812,
"rendered_text_chars": 9440,
"coverage": 0.086
}
],
"remediation": {
"severity": "critical",
"effort": "large",
"summary": "Server-render the decoupled front end.",
"detail": "Program and admissions templates ship an empty shell..."
}
}
],
"remediation_backlog": [
{
"rank": 1,
"check": "AIR-2.1",
"points_available": 4.5,
"severity": "critical",
"effort": "large"
}
]
}
Schema notes
site.cmsis best-effort, from thegeneratormeta tag and response headers. It is nullable and nothing scores against it.- A dimension with no applicable checks reports
score_pct: null, not0. Zero means measured and failed. - Rounding happens once, at the end. All arithmetic runs unrounded; the values written to this file are rounded to one decimal for display. The per-check
earned_pointstherefore will not sum exactly toscore.earned_points, and neither is wrong. Do not "fix" this by rounding intermediates — that reintroduces the drift the single rounding step exists to prevent. extraction_methodandextraction_versionrecord how main-content text was isolated. When extraction improves, old scores stay interpretable because the method that produced them is on the record.
Remediation ranking
Sort the backlog by points available (weight − earned_points), then by effort ascending, then by check ID ascending. The last key is not cosmetic: two runs of the same evidence must produce the same backlog in the same order, or the re-audit comparison is unreadable.
A check appears in the backlog only if it is applicable, has a band, and has points available above zero. N/A checks, unattested manual checks, and checks already at band 4 are not remediation items.
Use three severities — critical, major, minor — and three effort sizes — small, medium, large. Effort is a property of the check, defined in this document. Severity is derived: gate failures and any check scoring 0 or 1 with weight ≥3.0 are critical; band ≤2 with weight ≥1.5 is major; everything else is minor.
Band 1 on a heavy check is a critical finding. A site whose program pages ship 22% of their content scores AIR-2.1 at band 1, not 0, and calling that major alongside a missing caption on a table understates it to the person deciding what to fund.
The client-facing output is not the score. It is the ordered list of what to fix and what each fix is worth.
The tests
Each entry gives: points, applicability, verification mode, effort, what the test measures, how to detect it, what evidence to keep, and the band rule when it differs from the default coverage rubric.
#1 · First Contact · 13 tests · 28 pts
Points 28
What the free, single-page quick scan can see. These are the thirteen tests that ask no crawl budget and no second page — reachability, the first heading, the entity graph's spine, one organizational fact. They were originally scattered across six other dimensions; grouping them here means the number on the home page and the weight in this document are finally the same claim. Heaviest by construction, not by design — it is the sum of the highest-signal test from each of the others.
AIR-1.1 — No blanket robots.txt blocks on AI crawler user-agents
- Points: 5 · Gate · Applicability: always · Verification: auto · Effort: small
- Measures: Whether
robots.txtdisallows the crawlers that feed answer engines. - Detect: Parse
robots.txt. Evaluate each ofGPTBot,OAI-SearchBot,ChatGPT-User,ClaudeBot,Claude-User,PerplexityBot,Google-Extended,Applebot-Extended,CCBot,Bytespideragainst the rule set. ResolveUser-agent: *fallbacks correctly, including the longest-match group rule. - Evidence: Raw
robots.txt, the parsed rule tree, and a per-agent allow/deny verdict for/and for three sampled content URLs. - Bands: anchored on which agents are blocked, not how many.
- 4 — every listed agent may fetch content paths.
- 3 — one training or directive-only agent blocked; every answer-serving agent allowed.
- 2 — two or more training or directive-only agents blocked, up to and including all of them; every answer-serving agent allowed.
- 1 — exactly one answer-serving agent blocked.
- 0 —
Disallow: /for*, or two or more answer-serving agents blocked.
- Why the gate turns on the answer-serving class: the Index measures whether machines can find, read and use a site. An answer engine blocked at
robots.txtcannot answer a question about the client, which is the failure the gate exists to catch. A training block is a rights posture, and this document declines to judge rights postures — AIR-1.4 scores any combination ofyesandnoat 4 for exactly that reason. A university that blocksGPTBotand welcomesChatGPT-Useris reachable, and gating it would be measuring our approval rather than machine reach. - Consequence: blocking every training crawler costs 2 of the 5 points and does not gate. Blocking a single answer-serving agent gates.
- Absent robots.txt: a 404 scores 4. Nothing is disallowed, so nothing is blocked. This is correct and frequently queried; a missing file is not a failure of this test.
- Note: A deliberate, documented block is a business decision, not a defect. Record it, score it honestly, and say in the report that the client chose it. The number should reflect machine reach, not our approval.
- Fix: Remove inherited blocks. Keep only directives the client can explain.
AIR-1.2 — CDN and WAF do not silently reject AI crawlers
- Points: 5 · Gate · Applicability: always · Verification: auto · Effort: medium
- Measures: Whether the edge returns real content to AI user-agents, regardless of what
robots.txtpermits. - Detect: Request 10 sampled URLs with each of the eight fetching AI user-agents — the answer-serving and training classes defined under Fetching.
Google-ExtendedandApplebot-Extendedare excluded; they issue no requests. Record status code, response size, and whether the body matches the baseline fetch. Flag 403, 429, 503, interstitial challenge markup, and bodies more than 40% smaller than baseline. - Evidence: A user-agent × URL matrix of status codes, sizes, and challenge detection.
- Bands: 4 — all agents get 200 with full content. 3 — one agent rate-limited but eventually served. 2 — one agent blocked or challenged. 1 — several blocked. 0 — most blocked.
- Fix: Audit Cloudflare bot settings, Pantheon AGCDN rules, Fastly VCL, and origin WAF. Verify by re-running this probe, not by reading configuration.
AIR-1.3 — No interactive challenges on read-only paths
- Points: 1 · Applicability: always · Verification: auto · Effort: small
- Measures: Whether anonymous GETs on browse, search, listing, and detail pages return content without a challenge.
- Detect: Look for Turnstile, reCAPTCHA, hCaptcha, and JS-challenge signatures in raw responses on non-form URLs. Detect meta-refresh interstitials and challenge cookies.
- Evidence: Per-URL challenge detection with the matching signature.
- Fix: Scope challenges to POST endpoints and authenticated routes.
AIR-1.4 — Content Signals declared in robots.txt
- Points: 1 · Applicability: always · Verification: auto · Effort: small
- Measures: Whether the site states a machine-readable position on
search,ai-input, andai-trainusing Cloudflare's Content Signals Policy. - Detect: Parse
Content-Signal:lines. Validate syntax (key=yes|no, comma-delimited) and check for contradiction withDisallowrules. - Bands: 4 — all three signals declared, syntactically valid, consistent with
robots.txt. 3 — declared but one signal omitted. 2 — declared with a syntax error or an internal contradiction. 1 — a partial or malformed attempt. 0 — absent. - Note: Any combination of
yes/noscores 4. We are measuring that a deliberate position exists, not which position it is. - Fix: Run the policy conversation with the client, then add the line.
AIR-1.5 — Human-readable AI policy comment in robots.txt
- Points: 1 · Applicability: always · Verification: auto-partial · Effort: small
- Measures: Whether a person reading
robots.txtfinds a plain-language statement of the organization's position. - Detect: Extract
#comment blocks. Require ≥120 characters of prose and at least two policy terms (AI,training,crawl,license,permission). - Bands: 4 — substantive block present and consistent with the machine signals. 2 — a comment exists but is thin or boilerplate. 0 — none.
- Fix: Write four sentences above the signals. Journalists and counsel read this file.
AIR-1.6 — No stray noarchive or nosnippet directives
- Points: 1 · Applicability: always · Verification: auto · Effort: small
- Measures: Whether the site suppresses the excerpting that answer engines depend on.
- Detect: Parse meta robots tags and
X-Robots-Tagheaders across the sample fornoarchive,nosnippet, andmax-snippet:0. - Bands: 4 — none present. 2 — present on a minority of pages. 0 — present site-wide.
- Fix: Remove them. They are almost always left over from a legacy SEO module.
AIR-1.7 — Every page emits a valid, self-consistent canonical
- Points: 2 · Applicability: always · Verification: auto · Effort: small
- Measures: Presence and correctness of
rel=canonical. - Detect: For each sampled page, extract the canonical and resolve it. Flag: missing, multiple, cross-host, pointing at a redirect, pointing at a non-200, or pointing at the homepage from a deep page.
- Fix: Emit a self-referencing canonical by default and correct the exceptions.
AIR-1.8 — Semantic HTML structure
- Points: 2 · Applicability: always · Verification: auto · Effort: medium
- Measures: Whether document structure carries meaning or only styling.
- Detect: Per page, require exactly one
<main>, at least one<article>or<section>wrapping the primary content, and a<nav>for primary navigation. Compute a div-to-semantic-element ratio in the content region and flag pages above 12:1. - Fix: Replace structural divs with
article,section,nav,aside,figure, anddl.
AIR-1.9 — One h1 and a non-skipping heading hierarchy
- Points: 2 · Applicability: always · Verification: auto · Effort: small
- Measures: Whether headings segment the page into topics a model can follow.
- Detect: Count
h1elements. Walk the heading tree and flag skipped levels, empty headings, and headings used purely for styling (no following content). - Bands: Default coverage rubric, where a page passes only if it has exactly one non-empty
h1and no skipped levels. - Fix: Move the site name out of
h1. Fix skips in the templates, not the content.
AIR-1.10 — /llms.txt exists and is generated from the content model
- Points: 2 · Applicability: always · Verification: auto-partial · Effort: medium
- Measures: Presence, validity, and freshness of
/llms.txt. - Detect: Fetch
/llms.txt. Validate against the llmstxt.org structure: an# H1title, an optional blockquote summary, and##sections of Markdown links. Resolve a sample of the linked URLs. Compare listed URLs against the sitemap to estimate coverage and staleness. Detect the Drupalllms_txtmodule by response headers or known output patterns. - Bands: 4 — valid, links resolve, coverage tracks the sitemap, generation confirmed. 3 — valid and current but hand-maintained. 2 — present with broken links or clear staleness. 1 — present but malformed. 0 — absent.
- Fix: On Drupal, use the
llms_txtmodule with token-driven menus. A hand-written file is stale within a quarter.
AIR-1.11 — A single linked entity graph with @id references
- Points: 2 · Applicability: always · Verification: auto · Effort: medium
- Measures: Whether schema forms one coherent graph or dozens of disconnected assertions.
- Detect: Extract all JSON-LD across the sample. Build a graph of nodes and
@idreferences. Measure: what fraction of entity nodes carry an@id— value objects such asListItem,PostalAddress,ContactPoint,EntryPointandBreadcrumbListare excluded, because they have no identity and nothing should ever reference them; whetherOrganizationandWebSitehave stable canonical@idURIs reused site-wide; whetherWebPagenodes referenceWebSiteandWebSitereferencesOrganization; and how many duplicate definitions of the same entity exist. - Evidence: The resolved graph, orphan node count, duplicate entity count.
- Bands: 4 — canonical
@ids reused, chain intact, no duplicates. 3 — chain intact with minor duplication. 2 —@ids present but inconsistently reused. 1 —@ids rare. 0 — isolated blobs with no references. - Fix: Define canonical
@idURIs once and reference them everywhere.
AIR-1.12 — sameAs links to external authority records
- Points: 2 · Applicability: always · Verification: auto · Effort: small
- Measures: Entity disambiguation — whether a model can be certain which organization this is.
- Detect: Extract
sameAsfromOrganizationandPersonnodes. Classify each URL by authority: tier 1 (Wikidata, ROR, NPI, IRS EIN, ORCID), tier 2 (Wikipedia, Crunchbase, LinkedIn, Candid), tier 3 (social profiles). Resolve each URL and confirm it returns 200 and references the organization back where possible. - Resolution is opt-in. Those hosts are not the client's, they are not on the allowlist, and requesting them tells a third party which site is being audited. Classification from the markup alone separates a controlled identifier from a Facebook page; without resolution the test stops at band 3 and says why.
- Bands: 4 — at least one tier-1 identifier plus two others, all resolving. 3 — tier-1 present, some links stale. 2 — tier-2 and tier-3 only. 1 — social profiles only. 0 — no
sameAs. - Fix: Claim the Wikidata item. For universities, ROR is free and takes a day.
AIR-1.13 — About, Contact, and organizational detail are complete
- Points: 2 · Applicability: always · Verification: auto · Effort: medium
- Measures: Whether a model can resolve who this organization actually is.
- Detect: Locate About and Contact pages. Score presence of: founding date, leadership names, postal address, service area,
contactPointentries differentiated by purpose, and public legal identifiers (EIN, ROR, DUNS). Check both the visible page and theOrganizationnode. - Bands: Fraction of those seven facts present in both HTML and schema: 4 — ≥6. 3 — 5. 2 — 3–4. 1 — 1–2. 0 — none.
- Fix: Thin About pages are a persistent weakness for nonprofits and institutions. Do not leave the key facts in an annual-report PDF.
#2 · Rendering and Extraction · 6 tests · 15 pts
Points 15
Most AI crawlers do not execute JavaScript. This dimension asks whether the facts exist in the server response, in a structure a parser can follow.
AIR-2.1 — Primary content is present without JavaScript
- Points: 6 · Applicability: always · Verification: auto · Effort: large
- Measures: The gap between the raw HTTP response and the hydrated DOM. This is the single heaviest test in the Index.
- Detect: For every sampled URL, extract main-content text from the raw fetch and from the rendered fetch. Compute
coverage = raw_chars ÷ rendered_charsafter normalizing whitespace and stripping nav, header, and footer regions. Average across the sample and report the per-template distribution. - Evidence: Per-URL character counts, coverage ratio, and the first 500 characters of each version for spot-checking.
- Bands: 4 — mean coverage ≥0.95. 3 — 0.80–0.94. 2 — 0.50–0.79. 1 — 0.20–0.49. 0 — below 0.20.
- Edge cases: Coverage above 1.0 happens when JavaScript removes content that the raw response contained. Clamp the per-URL value to 1.0 for banding and keep the unclamped ratio in the evidence. A URL whose rendered text is empty is excluded from the mean and the exclusion is recorded — it is a fetch problem, not a coverage measurement.
- Note: Report the worst-performing template by name. "Your program pages ship 8% of their content" lands harder than a mean.
- Fix: Server-render or statically render. For decoupled Drupal this is the largest line item in most remediations.
AIR-2.2 — Progressive disclosure content ships in the initial HTML
- Points: 3 · Applicability: conditional — the site uses tabs, accordions, load-more, or infinite scroll · Verification: auto · Effort: medium
- Measures: Whether tabbed, collapsed, or lazily paged content exists before interaction.
- Detect: Identify disclosure widgets by ARIA roles (
tab,tabpanel,regionwitharia-expanded),<details>, and common class patterns. For each, check whether the panel body is populated in the raw HTML. Detect infinite scroll by watching for XHR content growth on scroll in the rendered fetch, and check for a paginated fallback. - Bands: 4 — every panel populated in raw HTML; infinite scroll has crawlable pagination. 3 — one pattern incomplete. 2 — panels populated on some templates only. 1 — most content loads on interaction. 0 — all disclosure content is fetched on demand.
- Fix: Render all panels and hide with CSS. Give infinite scroll a paginated fallback.
AIR-2.3 — Real data tables with header cells
- Points: 2 · Applicability: conditional — the sample contains tables · Verification: auto · Effort: medium
- Measures: Whether tabular facts — tuition, hours, comparisons, fees — are machine-parseable.
- Detect: For each
<table>, require<th>withscope, a<caption>, and<thead>/<tbody>. Separately flag layout tables (noth, presentational attributes, single row or column). - Bands: 4 — every data table complete, no layout tables. 3 — minor omissions such as missing captions. 2 — header cells present without scope. 1 — tables present with no header cells. 0 — layout tables carrying data.
- Fix: Fix the tables that carry facts first. Convert layout tables to CSS.
AIR-2.4 — Stable, human-readable anchor IDs on section headings
- Points: 2 · Applicability: always · Verification: auto-partial · Effort: medium
- Measures: Fragment-level citability — whether a model can deep-link a passage rather than a page.
- Detect: For headings below
h1, check for anid. Classify each as slug-like (derived from heading text) or unstable (section-3,block-a7f2c1, framework hashes). Compare IDs across two runs to detect churn. - Bands: 4 — ≥95% of subheadings carry slug-like IDs that are stable across runs. 3 — 75–94%. 2 — IDs present but a majority are generated or unstable. 1 — sparse. 0 — none.
- First run: cap at 3 without a baseline run. Stability cannot be asserted from one observation. See Tests that need a previous run.
- Fix: Derive IDs from heading text, not render order.
AIR-2.5 — Alt text on content images, empty alt on decorative
- Points: 1 · Applicability: always · Verification: auto · Effort: medium
- Measures: Whether images have a textual representation.
- Detect: For every
<img>, record presence ofalt, its length, and whether the image sits inside a content region or a decorative one. Flag missingalt, filename-as-alt, and long alt on obviously decorative images. - Fix: Audit existing content, then make the field required in the media entity so the problem stops recurring.
AIR-2.6 — Key facts exist as HTML text, not only in images or PDFs
- Points: 1 · Applicability: always · Verification: auto-partial · Effort: large
- Measures: Whether high-value facts are trapped in a scan, an infographic, or a PDF.
- Detect: Identify pages whose visible text is thin relative to their images or that link to a PDF as the primary content. Extract PDF text and look for numbers, dates, and prices that appear nowhere in the site's HTML. Requires review before scoring.
- Bands: 4 — no high-value facts found only in binaries. 2 — some found only in PDFs. 0 — core facts (tuition, hours, deadlines, contact) exist only in binaries.
- Fix: Publish an HTML equivalent. Keep the PDF as a download, not as the source of truth.
#3 · Structured Data and the Entity Graph · 8 tests · 14 pts
Points 14
An Organization block and some FAQ markup is commodity work. A linked, externally anchored entity graph is not — the graph's two entry points, @id coverage and sameAs, moved to First contact because they are cheap to check on one page; what is left here is the depth behind them.
AIR-3.1 — JSON-LD is generated from mapped fields, not hardcoded
- Points: 2 · Applicability: always · Verification: auto-partial · Effort: medium
- Measures: Whether schema follows the content or rots on the next content type change.
- Detect: Heuristics only. Compare JSON-LD values against visible field values on the same page — a mapped implementation matches. Flag identical literal values repeated across pages of the same type where the visible content differs. Detect Schema.org Metatag output patterns on Drupal. Confirm with the build team.
- Bands: 4 — values track content across every sampled page of a type, and the build team confirms the mapping. 3 — values track content on every sampled page, generation unconfirmed. 2 — mixed, with some literals. 0 — clear hardcoding, or values contradict the visible page.
- Fix: Map Drupal fields to schema properties through configuration so editors maintain it.
AIR-3.2 — BreadcrumbList on every page below the homepage
- Points: 2 · Applicability: always · Verification: auto · Effort: small
- Measures: Whether a page declares where it sits in the information architecture.
- Detect: Validate
BreadcrumbListstructure and confirm the trail matches the URL path or menu position. Flag trails that are a single item or that contradict the path. - Fix: Generate from the menu, not by hand.
AIR-3.3 — Vertical-specific schema types
- Points: 5 · Applicability: vertical — scored against the client's declared vertical only · Verification: auto · Effort: large
- Measures: Whether the site models the things it actually contains, using the right types.
- Detect: From the declared vertical, load the expected type set and check coverage against matching content types:
- Higher ed:
Course,EducationalOccupationalProgram,CollegeOrUniversity,EducationEvent - Healthcare:
MedicalOrganization,Physician,MedicalCondition,MedicalWebPage - Nonprofit:
NGO,DonateAction,Grant,FundingScheme - Government:
GovernmentOrganization,GovernmentService,Legislation - Research:
Dataset,DataCatalog,ScholarlyArticle
- Higher ed:
- Bands: Fraction of expected types present and valid on the content that warrants them: 4 — ≥90%. 3 — 70–89%. 2 — 40–69%. 1 — 1–39%. 0 — none.
- Note:
DatasetandDataCatalogare badly underused. Research organizations routinely hold catalogd data with no machine-readable description at all. - Fix: Pick the applicable set deliberately. Do not implement types the content does not support.
AIR-3.4 — FAQPage and QAPage on genuine question-and-answer content
- Points: 1 · Applicability: conditional — the site has real Q&A content · Verification: auto · Effort: small
- Measures: Correct markup on genuine Q&A — and the absence of fabricated FAQ blocks.
- Detect: Validate
FAQPageandQAPagestructure. Cross-check that every marked-up question appears in the visible page text. Flag pages where FAQ markup exists with no corresponding visible content. - Bands: 4 — real Q&A marked up, nothing fabricated. 2 — partial coverage. 0 — absent where warranted. Cap at 1 if fabricated FAQ blocks are detected, regardless of coverage.
- Fix: Mark up what exists. Do not invent questions to earn the markup — it is a spam pattern and it degrades the page for people.
AIR-3.5 — isAccessibleForFree, about, and mentions with entity references
- Points: 1 · Applicability: always · Verification: auto · Effort: medium
- Measures: Access status and topical entity linkage.
- Detect: Check for
isAccessibleForFreeon substantive content. Checkaboutandmentions, and classify their values as@idreferences or bare strings. - Bands: 4 — access declared and subject entities referenced by identifier. 2 — properties present with string values only. 0 — absent.
- Fix: Entity references are how a model decides a page is about a subject rather than merely containing the word.
AIR-3.6 — Product, Offer, and Service with real prices
- Points: 1 · Applicability: conditional — the site sells, charges, or offers something · Verification: auto · Effort: medium
- Measures: Whether commercial facts are marked up and current.
- Detect: Validate
Offerstructure. Compareprice,priceCurrency, andavailabilityagainst the visible page. Flag any mismatch and anypriceValidUntilin the past. - Bands: 4 — accurate and matching the page. 2 — present but drifting from visible values. 0 — absent, or demonstrably stale.
- Note: Stale pricing in schema is worse than none. A model will quote it with confidence.
- Fix: Source the schema value from the same field the page renders.
AIR-3.7 — SearchAction on the WebSite node
- Points: 1 · Applicability: conditional — the site has search · Verification: auto · Effort: small
- Measures: Whether site search is exposed as a callable pattern.
- Detect: Extract
potentialAction/SearchActionfrom theWebSitenode. Execute the URL template with a test term and confirm real results come back. - Bands: 4 — declared and the template returns results. 2 — declared but the template fails. 0 — absent.
- Fix: Small piece of the agent-interface story, and it costs almost nothing.
AIR-3.8 — Schema validation runs in CI
- Points: 1 · Applicability: always · Verification: manual · Effort: medium
- Measures: Whether broken markup fails the build instead of reaching production.
- Detect: Not observable from outside. Attested by the build team, ideally with a link to the pipeline definition.
- Bands: 4 — validation gates merges on all covered content types. 2 — validation runs but does not block. 0 — none.
- Fix: This is the test that separates an engineering firm from an SEO boutique. Schema rots silently otherwise.
#4 · Crawl, Index and Licensing Hygiene · 10 tests · 14 pts
Points 14
Two questions that used to live in separate dimensions turn out to be one: what has the organization said an AI crawler may keep, and does the crawler's budget reach real pages instead of drowning in facets and duplicate URLs. Licensing (RSL, Web Bot Auth, X-Robots-Tag) is a rights posture; index hygiene (canonicals, sitemaps, pagination) is plumbing. Both gate nothing and both are cheap to get right, which is why they share a home rather than two three-point dimensions apiece.
AIR-4.1 — RSL licensing document published and referenced
- Points: 1 · Applicability: conditional — the client holds licensable archives (journals, museums, associations, research bodies) or has opted into a licensing posture · Verification: auto · Effort: medium
- Measures: Whether machine-readable licensing terms exist under RSL 1.0 and are discoverable.
- Detect: Fetch
/license.xmland any path referenced fromrobots.txt. Validate against the RSL 1.0 schema. Check for an HTTPLink:header on content responses. - Bands: 4 — valid document, referenced from both
robots.txtand headers. 3 — valid, referenced from one. 2 — present but invalid. 1 — referenced but missing. 0 — absent. - Fix: Decide terms with the client first. The XML is the easy half.
AIR-4.2 — RSL terms propagated to feeds and schema
- Points: 1 · Applicability: conditional — requires AIR-4.1 to pass · Verification: auto · Effort: small
- Measures: Whether licensing travels with the content through every channel, not just the well-known file.
- Detect: Check RSS and Atom feeds for licensing elements. Check sampled JSON-LD for a
licenseproperty on licensable types. - Fix: Add the license reference to feed generation and to the schema field mapping.
AIR-4.3 — Web Bot Auth verification configured
- Points: 1 · Applicability: conditional — the client wants selective agent access rather than open or closed · Verification: auto-partial · Effort: large
- Measures: Whether the edge distinguishes cryptographically verified agents from unsigned scrapers.
- Detect: Probe
/.well-known/http-message-signatures-directoryif the site operates its own agents. Otherwise send a signed and an unsigned request and compare treatment. - Bands: 4 — verified agents pass, unsigned agents claiming an AI UA are challenged. 2 — signature headers accepted but not acted on. 0 — no differentiation.
- Fix: Configure edge bot rules on
Signature-Agentrather than user-agent strings.
AIR-4.4 — Correct X-Robots-Tag on non-HTML resources
- Points: 1 · Applicability: conditional — the site serves PDFs, documents, media, or JSON endpoints · Verification: auto · Effort: small
- Measures: Whether indexing directives reach resources that cannot carry a meta robots tag.
- Detect: HEAD every sampled PDF, document, media file, and API endpoint. Record
X-Robots-Tag. Flagnoindexon substantive content and absent directives on endpoints that should be excluded. - Bands: 4 — every non-HTML type carries a deliberate, correct value. 2 — mixed or partial. 0 — absent everywhere, or substantive PDFs marked
noindex. - Fix: Set headers by path pattern at the CDN or web server.
AIR-4.5 — Missing pages return 404 or 410
- Points: 2 · Applicability: always · Verification: auto · Effort: small
- Measures: Soft 404s — "not found" content served with a 200 status.
- Detect: Request 10 deliberately invalid URLs under real path prefixes. Flag 200 responses. Separately, flag sampled pages whose body matches a not-found template or falls below a minimum content threshold while returning 200.
- Evidence: Probe URLs with status codes and body fingerprints.
- Fix: Return real status codes. Use 410 for content that is permanently gone.
AIR-4.6 — Redirects resolve in a single hop
- Points: 1 · Applicability: always · Verification: auto · Effort: small
- Measures: Redirect chain depth and loops.
- Detect: Follow redirects without auto-resolution and count hops per sampled URL and per internal link target.
- Bands: 4 — no chain exceeds one hop. 3 — a few two-hop chains. 2 — chains of three or more. 1 — widespread chains, meaning both more than 10% of the sample and at least three URLs. 0 — loops present.
- Fix: Flatten the redirect table so every source points at its final destination.
AIR-4.7 — Faceted, calendar, and parameterized URLs are controlled
- Points: 2 · Applicability: conditional — the site has search, filtering, or a calendar · Verification: auto-partial · Effort: medium
- Measures: Whether the crawlable URL space is finite and predictable.
- Detect: Crawl listing pages two levels deep with parameters followed. Measure URL growth rate and count distinct parameter combinations. Check whether those URLs are disallowed,
noindex, or canonicalized to the unfiltered listing. - Bands: 4 — parameter space bounded by robots rules,
noindex, or canonicals. 3 — mostly controlled with gaps. 2 — partially controlled. 1 — minimal control. 0 — unbounded expansion observed. - Fix: Combine disallow patterns,
noindexon parameterized views, and canonicals back to the base listing.
AIR-4.8 — Sitemap lastmod reflects real content changes
- Points: 3 · Applicability: always · Verification: auto-partial · Effort: medium
- Measures: Whether
lastmodis a freshness signal or deploy noise. - Detect: Parse all
lastmodvalues. Flag clustering — more than 60% of URLs sharing a single date, or timestamps identical to the minute across content types. Cross-check against on-pagedateModifiedwhere present. Re-run across two audits to confirm. - Bands: 4 — values are distributed and match on-page dates. 3 — mostly accurate with some clustering. 2 — heavy clustering. 1 — all identical. 0 — absent.
- First run: cap at 3 without a baseline run. See Tests that need a previous run.
- Fix: Wire
lastmodto the node's changed timestamp, not to cron.
AIR-4.9 — Sitemap index split by type, with media sitemaps
- Points: 1 · Applicability: always · Verification: auto · Effort: small
- Measures: Whether sitemap structure makes crawl coverage diagnosable.
- Detect: Fetch
/sitemap.xml. Confirm it is an index with child sitemaps. Check for image and video sitemaps where the site has substantial media. Validate against the sitemap schema and the 50k URL / 50MB limits. - Fix: Generate per-content-type children and submit the index to both search consoles.
AIR-4.10 — Paginated series are coherently signaled
- Points: 1 · Applicability: conditional — the site has paginated listings or multi-page articles · Verification: auto · Effort: small
- Measures: Whether page 2 and beyond are reachable and correctly related to page 1.
- Detect: Identify paginated series by URL pattern. Check for
rel=next/rel=prev, a view-all page carrying the canonical, and whether deep pages are reachable by crawl. - Fix: Prefer a view-all canonical where page weight allows.
#5 · Authorship, Provenance, Freshness · 6 tests · 8 pts
Points 8
Trust signals, expressed as structured data rather than as vibes.
AIR-5.1 — Author entity pages with credentials
- Points: 1 · Applicability: always · Verification: auto · Effort: medium
- Measures: Whether authors resolve to real, credentialed entities.
- Detect: Follow
authorreferences to their target pages. RequirePersonmarkup with a canonical@id, plus at least two ofjobTitle,affiliation,knowsAbout,sameAs. ResolvesameAs(ORCID, LinkedIn, institutional profile). - Bands: 4 — every recurring author has a resolvable entity page with verified identifiers. 3 — most do. 2 — author pages exist without markup or credentials. 1 — author names only. 0 — no author entities.
- Fix: A byline string asserts nothing. A resolvable entity with credentials does.
AIR-5.2 — Bylines linked to author entities
- Points: 2 · Applicability: always · Verification: auto · Effort: small
- Measures: Whether substantive pages are signed, and whether the schema
authoris an@idreference rather than a repeated string. - Detect: For each substantive page, check for a visible byline and an
authorproperty. Classify the value as@idreference or literal. - Bands: 4 — visible byline plus
@idreference on ≥95%. 3 — 75–94%. 2 —authorpresent as a bare string. 1 — sparse. 0 — unsigned. - Fix: Where the author is a department, name the department and model it as an entity.
AIR-5.3 — reviewedBy on YMYL content
- Points: 1 · Applicability: conditional — the site publishes medical, legal, or financial content · Verification: auto · Effort: medium
- Measures: Whether expert review reaches the markup.
- Detect: On YMYL pages, check for
reviewedByreferencing a credentialedPerson, pluslastReviewedwhere the type supports it, plus a visible reviewer statement. - Fix: Most healthcare and legal clients already run review workflows. The reviewer's identity simply never reaches the page.
AIR-5.4 — datePublished and dateModified are accurate
- Points: 2 · Applicability: always · Verification: auto-partial · Effort: medium
- Measures: Whether the freshness signal is honest.
- Detect: Extract both dates from schema and visible markup. Flag: missing dates,
dateModifiedbeforedatePublished, future dates, and clustering that suggests a deploy stamp. Compare against sitemaplastmod(AIR-4.8) and against the previous audit run to confirm dates only move when content moves. - Bands: 4 — dates present, internally consistent, and stable across a deploy with no content change. 3 — present and consistent, stability unconfirmed. 2 — present but clustered or contradicting the sitemap. 1 — one date only. 0 — absent.
- Fix: Wire both to the node's own timestamps. A deploy must not move them.
AIR-5.5 — citation markup on primary-source references
- Points: 1 · Applicability: conditional — the site publishes research or policy content · Verification: auto · Effort: medium
- Measures: Whether sourcing is machine-readable.
- Detect: Check for
citationon research and policy pages. Resolve DOIs where present. Compare against the count of outbound links to primary sources in the body. - Fix: Credibility that a parser cannot see does not count.
AIR-5.6 — C2PA Content Credentials on original assets
- Points: 1 · Applicability: conditional — the client produces original photography, primary research, or documents where authenticity matters · Verification: auto · Effort: large
- Measures: Whether provenance survives into the delivered file.
- Detect: Download sampled original images and PDFs and read C2PA manifests. Confirm the manifest survives the image derivative pipeline — this is where it usually breaks.
- Bands: 4 — valid manifests on originals and derivatives. 2 — present on originals, stripped by the pipeline. 0 — absent.
- Fix: Scope this deliberately. It needs signing infrastructure.
#6 · Agent Interfaces · 7 tests · 7 pts
Points 7
The move from "a site agents can read" to "a site agents can use." Low weight, high differentiation — most sites score zero here, and that is currently fine. Treat this dimension as an opportunity map rather than a deficiency list.
AIR-6.1 — NLWeb exposed as an MCP server with an ask method
- Points: 1 · Applicability: conditional — requires AIR-9.5 · Verification: auto · Effort: medium
- Detect: Attempt an MCP handshake. Confirm the
askmethod is discoverable and returns correct answers.
AIR-6.2 — OpenAPI specification for public APIs
- Points: 1 · Applicability: conditional — the site has a public or semi-public API · Verification: auto · Effort: small
- Detect: Look for the spec at conventional paths and from
/.well-known/. Validate it, and confirm it is versioned and linked from the site.
AIR-6.3 — /.well-known/ discovery entries for agent capabilities
- Points: 1 · Applicability: conditional — the site exposes any agent capability · Verification: auto · Effort: small
- Detect: Enumerate
/.well-known/for MCP, NLWeb, OpenAPI, andhttp-message-signatures-directory. Validate each returns parseable metadata.
AIR-6.4 — Form fields carry label, name, and autocomplete
- Points: 1 · Applicability: conditional — the site has public forms · Verification: auto · Effort: medium
- Measures: Whether an agent can tell what each field is for.
- Detect: For every public form field: a
<label for>or equivalent accessible name, a meaningfulnameattribute (rejectfield_1,input3), and a validautocompletetoken where one applies (given-name,email,postal-code,tel). A token applies where the field type implies a purpose —email,tel,url,password. A search box or a free-text field has no sensible token and is not marked down for lacking one. - Bands: Fraction of fields passing all three: 4 — ≥95%. 3 — 75–94%. 2 — 40–74%. 1 — 1–39%. 0 — none.
- Fix: Same change satisfies WCAG. Easy joint justification.
AIR-6.5 — No captchas on browse, search, or filter interactions
- Points: 1 · Applicability: always · Verification: auto · Effort: small
- Detect: Submit search and filter forms and check for challenge responses. Complements AIR-1.3, from the application side rather than the edge.
AIR-6.6 — Critical flows complete without JS-only interactions
- Points: 1 · Applicability: conditional — the client names critical flows · Verification: auto-partial · Effort: large
- Measures: Whether register, donate, apply, or find-a-clinician can be completed through standard form submissions and real URL transitions.
- Detect: For each named flow, walk it with JavaScript disabled. Flag
div-based controls with click handlers, wizards with no addressable step URLs, and modal-only paths. - Bands: 4 — every flow completes with real URLs at each step. 2 — flows start but cannot complete. 0 — flows are JS-only.
- Fix: Run this as a joint engineering and UX audit.
AIR-6.7 — Stable selectors on critical-flow elements
- Points: 1 · Applicability: conditional — requires AIR-6.6 flows to be named · Verification: auto · Effort: medium
- Detect: Collect selectors for the key elements in each flow. Classify as stable (
id,data-*, semantic) or fragile (hashed utility classes, build-generated IDs). Diff across two runs to detect churn. - Bands: 4 — stable and unchanged across runs. 2 — mixed. 0 — hashed classes only.
- First run: cap at 2 without a baseline run, since band 4 is defined entirely by cross-run stability. See Tests that need a previous run.
- Fix: Treat these selectors as a contract that survives redeploys.
#7 · Alternate Representations · 5 tests · 7 pts
Points 7
Clean, low-noise versions of the content alongside the HTML.
Read this before you sell it: evidence that llms.txt is actually consumed by the major models remains thin, and Google has said it does not use it. We include the dimension because it is cheap, it forces good content hygiene, and being early costs nothing. /llms.txt itself moved to First contact — it is checked on one page — so what is left here is the fuller Markdown-companion apparatus. We weight the pair at 20 of 400 points for the same reason the old spec weighted it low: do not let a report imply this is a ranking lever.
AIR-7.1 — /llms-full.txt where full-text inclusion is appropriate
- Points: 1 · Applicability: conditional — the client permits full-text inclusion; N/A if Content Signals declare
ai-train=no· Verification: auto · Effort: small - Measures: Presence of the expanded variant.
- Detect: Fetch
/llms-full.txt. Confirm it inlines content rather than linking. Cross-check against AIR-1.4 for contradiction. - Bands: 4 — present and consistent with the licensing posture. 2 — present but contradicts declared signals. 0 — absent where appropriate.
- Fix: Gate this on the licensing decision, not on convenience.
AIR-7.2 — Markdown companion for every canonical page
- Points: 2 · Applicability: always · Verification: auto · Effort: medium
- Measures: Whether a clean
.mdrepresentation exists alongside each page. - Detect: For each sampled content URL, request
{path}.md. Require a 200 withtext/markdownortext/plain, content that materially matches the page body, and no navigation chrome. - Fix: Generate from the same source as the HTML. Strip nav, promos, and cookie banners.
AIR-7.3 — link rel=alternate advertises the Markdown version
- Points: 1 · Applicability: conditional — requires AIR-7.2 to pass · Verification: auto · Effort: small
- Measures: Discoverability of the alternate representation.
- Detect: Parse
<link rel="alternate" type="text/markdown">from the head and resolve the href. - Fix: Add it to the head template. A file nobody can find is not much use.
AIR-7.4 — Accept: text/markdown content negotiation
- Points: 2 · Applicability: always · Verification: auto · Effort: medium
- Measures: Whether the canonical URL serves Markdown on request, with correct cache headers.
- Detect: Request sampled URLs with
Accept: text/markdown. Confirm the response type and thatVary: Acceptis set. Re-request through the CDN to confirm the variants are cached separately. - Bands: 4 — negotiation works and
Vary: Acceptis correct. 2 — negotiation works butVaryis missing, which risks serving Markdown to browsers. 0 — not supported. - Fix: Getting
Varywrong is worse than not doing this at all. Verify at the edge.
AIR-7.5 — Alternate representations are edge-cached
- Points: 1 · Applicability: conditional — requires AIR-1.10 or AIR-7.2 · Verification: auto · Effort: small
- Measures: Whether generated representations hit the CDN rather than origin.
- Detect: Request
/llms.txtand sampled.mdURLs twice. Read cache-status headers and compare response times. - Fix: Set TTLs and wire invalidation to the same cache tags as the source content.
#8 · Measurement and Instrumentation · 7 tests · 7 pts
Points 7
The proof layer. Commercially, this is the most valuable work in the whole checklist — and almost none of it is visible from outside a site, which is why it carries only 12 of 400 points. Do not read a low score here as unimportant. Read it as unmeasurable by scanner.
AIR-8.1 — GA4 channel group for AI assistant referrers
- Points: 1 · Applicability: always · Verification: manual · Effort: small
- Measures: Whether the client can see traffic from
chatgpt.com,perplexity.ai,claude.ai,copilot.microsoft.com, andgemini.google.com. By default it lands in Direct and disappears. - Detect: Analytics presence is detectable from the page. The channel group is not — attest it, ideally with a screenshot.
- Bands: 4 — group live with a saved report. 2 — analytics present, no grouping. 0 — no analytics.
AIR-8.2 — Server-side tagging captures stripped referrers
- Points: 1 · Applicability: always · Verification: manual · Effort: medium
- Detect: A server-side endpoint may be inferable from network requests. Reconciliation against client-side numbers must be attested.
AIR-8.3 — Access log retention with a queryable store
- Points: 1 · Applicability: always · Verification: manual · Effort: medium
- Measures: Whether crawler logs survive long enough to show a before-and-after window. Many hosts discard them in days.
- Bands: 4 — ≥12 months retained and queryable by user-agent, path, status, date. 2 — retained but not queryable. 0 — discarded.
AIR-8.4 — Scheduled AI crawler activity report
- Points: 1 · Applicability: conditional — requires AIR-8.3 · Verification: manual · Effort: medium
- Measures: Whether crawler behavior is monitored continuously rather than audited once. A crawler that stops appearing usually means somebody reintroduced a block.
AIR-8.5 — Fixed prompt panel run monthly across models
- Points: 1 · Applicability: always · Verification: manual · Effort: medium
- Measures: Whether answer-share is tracked on a stable panel of brand, category, and comparison prompts.
- Bands: 4 — fixed panel, baseline stored, monthly runs comparable. 2 — ad-hoc tests. 0 — none.
- Note: The panel has to stay fixed. Tuning prompts between runs destroys the trend line.
AIR-8.6 — Extraction-fidelity baseline captured
- Points: 1 · Applicability: always · Verification: manual · Effort: medium
- Measures: Whether a before-and-after fidelity score exists — feed the page to a model, ask it the client's key questions, score the accuracy.
- Note: The scanner can generate this itself in a later version. Until then, attest it. The before-and-after delta is the best single-slide value demonstration we have.
AIR-8.7 — Search Console and Bing Webmaster Tools verified
- Points: 1 · Applicability: always · Verification: auto-partial · Effort: small
- Detect: Look for
google-site-verificationandmsvalidate.01meta tags, or DNS TXT verification records. Whether anyone reads the alerts must be attested. - Bands: 4 — both verified, sitemaps submitted, alerts routed to a person. 3 — both verified; submission and alert routing unconfirmed. 2 — one verified. 0 — neither.
- Note: Bing matters more than its search share suggests. Its index feeds several AI products, and clients frequently have Search Console with no Bing property at all.
#9 · Unscored Leading Indicators · 6 tests · 0 pts
Points 0
Every leading-tier test, gathered in one place rather than scattered through the dimensions that would otherwise host them (see Versioning and change control, 4.0). Each measures a practice with essentially no 2026 adoption — scoring it would subtract the same points from every site, which moves nobody relative to anybody. Run and reported like every other test. Worth nothing toward the hundred until that changes.
AIR-9.1 — HowTo on procedural content
- Points: 1 · Leading · Applicability: conditional — the site has procedural content · Verification: auto · Effort: small
- Measures: Whether step sequences are explicit rather than inferred from prose.
- Detect: Identify procedural pages by ordered lists with step-like language in headings. Check for
HowTowith orderedHowToStep. - Fix: Application walkthroughs, enrollment steps, and permit processes are exactly what people ask assistants about.
AIR-9.2 — speakable on summaries and ledes
- Points: 1 · Leading · Applicability: always · Verification: auto · Effort: small
- Measures: Whether the page flags its own best short answer.
- Detect: Extract
speakablewithcssSelectororxpath. Resolve the selector against the DOM and confirm it matches 1–3 elements containing 40–400 characters. Flag selectors that match the whole body or nothing. - Bands: 4 — resolves to a genuine summary passage. 2 — present but resolves too broadly. 0 — absent.
- Fix: Point it at the lede, not the article.
AIR-9.3 — IndexNow fires on publish and update
- Points: 1 · Leading · Applicability: always · Verification: auto-partial · Effort: small
- Measures: Whether the site pushes change notifications instead of waiting to be crawled.
- Detect: Look for an IndexNow key file at the site root (a 32–128 character hex filename returning its own key). Confirm the key resolves. Whether submissions actually fire on publish requires attestation.
- Bands: 4 — key file present, valid, and submission confirmed. 3 — key file present and valid, submission unconfirmed. 2 — a key file is referenced but does not resolve or does not contain its own key. 0 — absent or not discoverable.
- Limitation: the key file's name is the key, and the key is not published. A scanner can only find it where
robots.txtor a conventional path advertises it, so a zero here means not discoverable rather than absent. Score it honestly and say which. - Fix: Hook submission into the publish workflow rather than a schedule.
AIR-9.4 — Editorial policy page referenced via publishingPrinciples
- Points: 1 · Leading · Applicability: always · Verification: auto · Effort: small
- Measures: Whether the site documents how content is produced, reviewed, corrected, and funded.
- Detect: Look for
publishingPrincipleson theOrganizationnode and resolve it. For publishers, also checkcorrectionsPolicyanddiversityPolicy. - Fix: Write the page that describes the process the client already follows.
AIR-9.5 — NLWeb endpoint over the site's schema.org data
- Points: 1 · Leading · Applicability: always · Verification: auto · Effort: large
- Detect: Probe known NLWeb paths and
/.well-known/entries. If found, send a representative question and check for a schema.org-formatted JSON response. - Bands: 4 — endpoint answers correctly and is grounded in site content. 2 — endpoint responds but answers poorly. 0 — absent.
AIR-9.6 — Domain MCP server over real content APIs
- Points: 1 · Leading · Applicability: always · Verification: auto-partial · Effort: large
- Measures: Whether the client's structured data — course catalog, provider directory, grant database, program finder — is callable as documented tools.
- Detect: Discover from
/.well-known/. Enumerate tools, check each has a description and typed parameters, and call one read-only tool against live data. - Bands: 4 — documented, typed tools returning live data, with auth and rate limiting. 3 — tools present but thinly documented. 2 — a generic query endpoint rather than narrow tools. 0 — none.
- Note: The most defensible item in the whole Index. A well-described tool with three parameters beats a generic endpoint.
What 100 means
A site scoring 100 has, at minimum:
- Every AI crawler reaching real content at the edge and in
robots.txt, with a deliberate, documented licensing position. - A clean, finite URL space with honest freshness signals.
- Every substantive fact present in the server response, in semantic markup, with stable anchors a model can cite.
- Generated
llms.txtand Markdown companions, cached at the edge. - One linked entity graph, anchored to external authorities, validated in CI so it cannot rot.
- Credentialed authors, honest timestamps, machine-readable sourcing.
- Documented agent endpoints and flows an agent can actually complete.
- Instrumentation that proves all of it, running on a schedule.
Almost no site will score 100, and that is the point. The Index is a target to move toward, not a bar to clear. A client who moves from 41 to 72 in one engagement has a number they can take to their board — which is worth considerably more to them than a perfect score they will never reach.
What the Index deliberately does not measure
We keep this list public because leaving it out would be dishonest.
- Whether the content is any good. Structure, not substance.
- Whether the models actually cite the site. That is answer-share monitoring (AIR-8.5), a different instrument on a different cadence.
- Traditional SEO performance. Rankings, backlinks, and Core Web Vitals are real, and they are not this.
- Accessibility conformance. Several tests overlap with WCAG, happily. This is not a WCAG audit and must not be sold as one.
- Security posture. We probe bot handling at the edge. We do not test the site's defenses.
Versioning and change control
The Index version is independent of the scanner version. Both appear in every result file.
Bump the minor version for new tests, clarified detection, or corrected band anchors. Bump the major version for any weight change, gate change, or scoring-model change — anything that makes two scores non-comparable.
Never silently re-weight. A client's trend line is the most valuable thing the Index produces, and one quiet weight change destroys it. When a major bump lands, re-score the last run under both versions and show the difference.
Changelog
9.0 — 2026-09-16
A new dimension, not a new score. Every leading-tier test — measured
and reported since 4.0, never scored — moves out of the dimension it
happened to be filed under and into its own: #9 · Unscored Leading
Indicators, declared at 0 of the 100 points, last in dimension order.
Nothing about why a leading test doesn't count changes; what changes is
that it no longer sits inside a dimension whose own heading implies every
one of its tests carries weight. A reader adding up a dimension's own rows
now gets that dimension's own declared total, every time — before this,
seven of those rows didn't, because they were leading tests hiding in
plain sight in dimensions 3, 4, 5, and 6.
- Six tests move, ten more renumber to close the gaps they leave.
checks.yamlstays grouped and numbered by dimension (6.0's own rule), so a check moving dimension is a check getting a new id:- AIR-3.5 (HowTo) → AIR-9.1, AIR-3.6 (speakable) → AIR-9.2. The four that followed them close the gap: AIR-3.7 → AIR-3.5, AIR-3.8 → AIR-3.6, AIR-3.9 → AIR-3.7, AIR-3.10 → AIR-3.8.
- AIR-4.11 (IndexNow) → AIR-9.3. Last in its dimension; nothing behind it to renumber.
- AIR-5.7 (editorial policy) → AIR-9.4. Also last; same reason.
- AIR-6.1 (NLWeb endpoint) → AIR-9.5, AIR-6.3 (domain MCP server) → AIR-9.6. The seven that followed close the gap: AIR-6.2 → AIR-6.1, AIR-6.4 → AIR-6.2, AIR-6.5 → AIR-6.3, AIR-6.6 → AIR-6.4, AIR-6.7 → AIR-6.5, AIR-6.8 → AIR-6.6, AIR-6.9 → AIR-6.7.
- The one dependency naming an id that moved is updated to match: the test now numbered AIR-6.1 (formerly AIR-6.2) required AIR-6.1 before this change; it requires AIR-9.5 now, the new number for the same test it always depended on.
- No weight, band, or score changes. Every test kept its own weight; every dimension holding only core tests kept its own declared total, because removing a test that never contributed to it changes nothing about the sum. A site scored under 8.0 and the same site scored under 9.0 report identical numbers everywhere except which id and dimension two handfuls of findings print under — the same guarantee 6.0 made for the same reason.
- A related bug, caught while building this: a dimension's own
applicable/earned points included any leading test inside it. Scoring
itself was never affected —
score_run's own total already excluded leading tests — but_dimension_resultsdidn't apply the same exclusion, so a dimension holding one could show a share-earned number diluted by a test that was never part of its denominator. Dimension 9 would have inherited this as a dimension that could show nonzero earned points against a declared weight of zero — negative points "left," on the one dimension where that number most needs to read as exactly zero. Fixed in the same change: leading tests are now excluded from a dimension's own applicable and earned totals, everywhere they're computed, not only in the overall score.
8.0 — 2026-09-16
A convention change, not a scoring one, bumped major because it touches
the one canonical dimension label everywhere it's used. A dimension
heading now reads #1 · First Contact · 13 tests · 28 pts rather than
13 checks · 28% weight — the dimension's own weight, shown as points,
not re-expressed as a percent of the total (every other number on the
site already reads as points, and the percent was the odd one out). The
site's own word for what it audits is "test" now, not "check" — the nav,
the footer, the scan pages. Nothing is re-scored: a check's id, weight,
and band are exactly what they were; only what the page calls it changed.
7.0 — 2026-09-15
A major bump on both the letter and the spirit of the rule. Seventeen checks change weight, and for the nine that were at zero this is not accounting — a check worth zero points cannot move a score, and now every one of them can. Re-score before comparing a 7.0 result to anything earlier.
- No core check weighs zero. The nine checks 5.0 rounded to zero on the
100-point scale —
AIR-5.5,AIR-5.6,AIR-6.4,AIR-6.5,AIR-7.4,AIR-8.1,AIR-8.2,AIR-8.6,AIR-8.7— each now carry at least 1 point, funded by moving weight rather than by widening the scale. A check in the Index is a claim that something is worth measuring; a check worth nothing was a claim the number itself contradicted. - Three dimensions could not fund their own floor. Agent interfaces (9 checks) and Measurement and instrumentation (7 checks) each totaled fewer points than they have checks — mathematically, some of them had to be zero. Both dimensions' declared weight rises to match their own check count (5 → 7, and 3 → 7), every check in each now weighing exactly 1. Alternate representations funds its own zero-weight check from within its existing 5, plus 2 more for the same reason (5 → 7), landing at 1/2/1/2/1 across its five checks rather than the flat 1 the other two dimensions reach.
- First contact funds the difference: 36 → 28. The 8 points the other
three dimensions needed come from here, not from Structured data or
Crawl, index and licensing hygiene — First contact is still by far the
heaviest dimension, and the two checks it gates on (
AIR-1.1,AIR-1.2) keep their weight of 5 each unchanged; the reduction is absorbed by the other eleven, heaviest first, none dropping below 1. - Dimensions are still declared in descending weight order — see Weights — which is why Alternate representations lands at 7 rather than the 5 its own checks would otherwise need: dimension 7 must weigh at least as much as dimension 8, and dimension 8's own floor is 7.
6.0 — 2026-09-11
A major bump, on the letter of the versioning rule rather than its spirit, same as 5.0. No check's detection, band anchor, gate, or weight moved. What changed is how a check is named.
- Every check is renumbered so its id prefix names its own dimension.
AIR-1.1throughAIR-1.13are Dimension 1,AIR-2.1throughAIR-2.6are Dimension 2, and so on throughAIR-8.7— this reverses 5.0's decoupling (above), which let a check's id and itsdimensionfield disagree. checks.yaml is now grouped and ordered by dimension, and_validate_idsenforces the ascending order that follows from it. - The id prefix itself changes, from
AR-toAIR-. Every check id, every anchor (#air-1.1, not#ar-1.1), and every mention of a check anywhere the site or this document names one, uses the new prefix. - A dimension has one canonical label, used everywhere one is named:
#N · Title Case Name · N checks · W% weight— for example,#1 · First Contact · 13 checks · 36% weight. The methodology page's own dimension headings and the checks list both follow it now. - Nothing is re-scored. A site scored under 5.0 and the same site scored under 6.0 differ only in which id and dimension label each finding is printed under, not in any check's verdict, weight, or band.
5.0 — 2026-09-10
A major bump, on the letter of the versioning rule rather than its spirit. Nothing here changes what a check detects or how a band is assigned — no detection logic moved, no band anchor moved, no gate moved. What changed is the accounting underneath the checklist, which is exactly the kind of change the rule says must never happen silently.
- Weights are integers, on the same 100-point scale. Every dimension
total and every check weight is now a whole number, and the eight
dimension totals still sum to exactly 100. That forces nine checks —
AIR-7.4,AIR-5.5,AIR-5.6,AIR-6.4,AIR-6.5, and four of the five checks in Measurement and instrumentation — to round to zero: each was already worth less than half a point, and a 100-point scale has no smaller unit to give them. A check worth zero points cannot move a score. That is a known, accepted cost of staying on 100 rather than moving to a finer-grained scale, not an oversight — recorded here rather than left for someone to discover. The leading tier's own weights (measured, still never scored) round the same way, with a floor of 1 rather than 0: they describe a real practice worth naming even at the bottom of the scale. (Fixed in 7.0, below — every core check now floors at 1 too, by moving weight rather than by widening the scale.) - A dimension is a field, not a naming convention. The registry used to
validate that a check's id prefix matched its
dimension—AR-2.3(nowAIR-1.7) had to live in dimension 2. That coupling is gone. A check's id is permanent from the day it is written; which dimension it is grouped under can move, and now does. (Reversed in 6.0, below — the id prefix is coupled to the dimension again, this time by renumbering the check rather than by validating against it.) - First contact is a new dimension: exactly the 13 checks the free quick
scan runs, pulled out of six different dimensions they used to sit in.
The quick scan's own weight and the number this document states for
Dimension 1 are now the same claim — they used to be two different sums
that happened to disagree.
AIR-4.1–AIR-4.4(RSL, Web Bot Auth,X-Robots-Tag) — what was left of the old Dimension 1 once its quick-scan checks moved out — folded into the old Dimension 2, renamed Crawl, index and licensing hygiene, so the Index still has 8 dimensions, not 9. - Dimensions are ordered by weight, descending; checks within a dimension
the same way, gates pinned first. Both orders are computed from the
registry, never a hand-maintained list — see Weights, and every
### AIR-x.ysection in this document, for the result. - Nothing is re-scored. Every scan already on record keeps its
4.0stamp and renders exactly as it did the day it ran; the Census stays on 4.0 until it is next run.docs/AI-READINESS-INDEX.md's own rule above — re-score before comparing across a major bump — still applies to anyone who wants to line up a 4.0 result next to a 5.0 one; nothing here does that automatically, because nothing here changed what a re-score would find. A site scored under 4.0 and the same site scored under 5.0 differ only by integer rounding, not by any check's verdict.
4.0 — 2026-09-08
A major bump, and the versioning rule applies in full. Any site scored under 3.x must be re-scored under 4.0 before its number is compared to anything.
- A gate failure no longer caps the score.
totalis the site's real, uncapped score, computed exactly as in Scoring;gatedandgate_failuresare reported beside it as a separate fact, never blended into it. The old model produced one number standing for two facts and comparable to neither — a gated site with excellent structure scored identically to a gated site with none.ungated_score,ungated_grade, and theGatedgrade are gone rather than kept as duplicates of whattotalandgatedalready say between them. See Gates. - A leading tier exists for checks with essentially no 2026 adoption. Six checks
move out of the hundred points entirely — scoring them would subtract the same
points from every site, which moves nobody relative to anybody. They are still run
and still reported, as
leading_adoptedofleading_total(adopted meaning band 2 or better), so a site gets credit for being early without it changing anyone's score.
3.0 — 2026-09-07
A major bump, and the versioning rule applies in full. Any site scored under 2.x must be re-scored under 3.0 before its number is compared to anything.
- The crawler obeys the
robots.txtgroup addressed to it. Until now the scanner readrobots.txtonly to score it, and crawled regardless of what it said about the scanner itself. It now honors aDisallowaddressed to its own product token, and a site that turns it away is recorded as exactly that —robots.txtis still fetched, so AIR-1.1 still resolves and the gate still answers, while every other check reports insufficient evidence. This changes which URLs are sampled, and therefore what a site can score, which is why the version is 3.0. - A blanket
User-agent: *disallow does not stop the crawler, and the reasoning is now published rather than implied.Crawl-delayunder*is honored regardless. See Consent. - The AIR-1.2 probe is documented rather than glossed. The behavior has not changed — the probe has always carried other crawlers' product tokens — but the specification previously said we "do not pretend to be somebody else's bot", which was not an accurate description of the wire. It now states what the requests carry and the three constraints that make it measurement rather than impersonation. No score moves; the honesty of the document does.
- A survey population is a first-class concept, with the sourcing and citation rules a published measurement needs. See Populations.
- Every result carries the Index's effective date alongside its version, so an old report cannot silently start meaning something new.
2.0 — 2026-09-01
A major bump, and the versioning rule applies in full. The one-time exemption taken at 1.1 is spent. Any site scored under 1.x must be re-scored under 2.0 before its number is compared to anything.
- AIR-1.1's bands are anchored on agent class rather than agent count, and the
gate now turns on the answer-serving class. Blocking every training crawler costs 2
of 5 points and does not gate; blocking one answer-serving agent does. Under 1.x the
table could not tell
PerplexityBotfromCCBot— both scored band 3 — which meant a site invisible to an answer engine and a site excluded from a corpus were graded identically. This changes which sites are gated, which is why the version is 2.0. criticalseverity widens to band 0 or 1 on checks weighing 3.0 or more. A 6-point check at band 1 is not amajorfinding sitting alongside a missing table caption. Ordering and points are unchanged; only the label moves.- AIR-4.11, AIR-3.1 and AIR-8.7 gain a band 3, and their unattested ceilings rise from
2 to 3 to match. Their tables previously skipped 3, so with a ceiling at 2 the whole
observable range was
{0, 2}and there was no way to say mostly right.
1.2 — 2026-09-01
Refinements found while implementing all 68 checks. None changes a weight or a gate. Two change how a band is reached and are recorded here so a re-score is explicable.
@idcoverage in AIR-1.11 counts entity nodes only. AListIteminside aBreadcrumbList, aPostalAddresson anOrganizationand aSearchAction'sEntryPointare value objects: they have no identity and nothing should reference them. Counting them penalized a site for marking breadcrumbs up correctly.- AIR-1.12 resolution is opt-in. Band 4 requires
sameAslinks to resolve, which means requesting Wikidata, ROR and LinkedIn — hosts the client never allowlisted, and hosts that learn which site is being audited. The scanner classifies by authority tier from the markup alone and declares an observed ceiling of band 3 until told to fetch. - AIR-6.6 expects an
autocompletetoken only where the field type implies one (email,tel,url,password). A search box has no sensible token and must not be marked down for lacking one. - "Widespread" in AIR-4.6 needs a count as well as a share. One three-hop chain on a two-page site is a finding, not a systemic failure.
- Every band ceiling now states its reason in the result and in the report: awaiting a baseline run, awaiting confirmation from the client, limited by the band attested, or needing review by a person. A client told a number was held down is owed the reason.
- AIR-4.11's detection method is weaker than this document implied. The IndexNow key file is named after the key, and the key is not published, so a third party can only find it where the site advertises it. A negative result is not discoverable, which is weaker than absent, and the check says so.
- AIR-5.6 is not implemented. Reading C2PA manifests means parsing image and PDF bytes. The check reports that it had no evidence and leaves the denominator rather than scoring a site zero for a measurement nobody took.
1.1 — 2026-09-01
Twelve gaps found while building the first scanner against 1.0. Several of these would ordinarily force a major bump, because they change how a score is computed. They are landing as a minor bump because no site has ever been scored under 1.0 — the scanner did not exist yet, so there is no trend line to break. This exemption applies once. Every change after this one follows the rule above.
- Automated coverage now counts
auto-partialat half weight. Under the previous wording the metric could not exceed 0.83, while this document's own example result showed 0.87 — arithmetically unreachable. - Attestations may lift an
auto-partialband ceiling, never set or lower a band. Four checks (AIR-4.11, AIR-1.10, AIR-3.1, AIR-8.7) had a top band this document itself says requires attestation, and no mechanism to supply one. - Cross-run checks cap at band 3 on a first run (AIR-6.9 at 2) and lift with a supplied baseline. AIR-4.8, AIR-2.4 and AIR-6.9 previously required a second audit with no definition of what run one does.
- The ten AI agents are sorted into three classes. AIR-1.1's bands referenced categories the document never defined, and AIR-1.2 asked us to probe two user-agents that no crawler ever sends.
- Money-page double weighting is defined as counting 2 in both sides of the coverage rubric. Previously asserted and never operationalized.
- A gated run grades as
Gatedand gains anungated_gradefield. The bands table offered two labels for the same run. - "Requires AIR-X.Y to pass" means band ≥ 2, with N/A cascading to dependents. Previously undefined against a 0–4 scale.
- Business-fact applicability comes from a declared engagement profile, never inference. Undeclared scores N/A with a reason.
- A 25-URL representation sub-sample bounds AIR-4.4, AIR-7.2, AIR-7.4 and AIR-7.5, which together would otherwise triple the request count against a client's origin.
confidenceis an enum:high,medium,low.- AIR-2.1 coverage clamps to 1.0 and excludes empty renders; a missing
robots.txtscores AIR-1.1 band 4;score_pctisnullwhere nothing applies; backlog ties break on check ID;site.cmsis best-effort and nullable. - Rounding happens once, and the file records that per-check points will not sum exactly to the total.
Appendix — Test index
| Test | Points |
|---|---|
| AIR-1.1 | 5 |
| AIR-1.2 | 5 |
| AIR-1.3 | 1 |
| AIR-1.4 | 1 |
| AIR-1.5 | 1 |
| AIR-1.6 | 1 |
| AIR-1.7 | 2 |
| AIR-1.8 | 2 |
| AIR-1.9 | 2 |
| AIR-1.10 | 2 |
| AIR-1.11 | 2 |
| AIR-1.12 | 2 |
| AIR-1.13 | 2 |
| AIR-2.1 | 6 |
| AIR-2.2 | 3 |
| AIR-2.3 | 2 |
| AIR-2.4 | 2 |
| AIR-2.5 | 1 |
| AIR-2.6 | 1 |
| AIR-3.1 | 2 |
| AIR-3.2 | 2 |
| AIR-3.3 | 5 |
| AIR-3.4 | 1 |
| AIR-3.5 | 1 |
| AIR-3.6 | 1 |
| AIR-3.7 | 1 |
| AIR-3.8 | 1 |
| AIR-4.1 | 1 |
| AIR-4.2 | 1 |
| AIR-4.3 | 1 |
| AIR-4.4 | 1 |
| AIR-4.5 | 2 |
| AIR-4.6 | 1 |
| AIR-4.7 | 2 |
| AIR-4.8 | 3 |
| AIR-4.9 | 1 |
| AIR-4.10 | 1 |
| AIR-5.1 | 1 |
| AIR-5.2 | 2 |
| AIR-5.3 | 1 |
| AIR-5.4 | 2 |
| AIR-5.5 | 1 |
| AIR-5.6 | 1 |
| AIR-6.1 | 1 |
| AIR-6.2 | 1 |
| AIR-6.3 | 1 |
| AIR-6.4 | 1 |
| AIR-6.5 | 1 |
| AIR-6.6 | 1 |
| AIR-6.7 | 1 |
| AIR-7.1 | 1 |
| AIR-7.2 | 2 |
| AIR-7.3 | 1 |
| AIR-7.4 | 2 |
| AIR-7.5 | 1 |
| AIR-8.1 | 1 |
| AIR-8.2 | 1 |
| AIR-8.3 | 1 |
| AIR-8.4 | 1 |
| AIR-8.5 | 1 |
| AIR-8.6 | 1 |
| AIR-8.7 | 1 |
| AIR-9.1 | 1 |
| AIR-9.2 | 1 |
| AIR-9.3 | 1 |
| AIR-9.4 | 1 |
| AIR-9.5 | 1 |
| AIR-9.6 | 1 |
68 tests · 100 points