How the Citability Index is calculated

The Citability Index is a score from 0 to 100 that measures whether a website can be read, understood and cited by a generative model.

The Citability Index is calculated from 28 checks grouped into four weighted layers. This page documents every one of them: what it checks, how it is checked, what threshold counts as a pass, what weight it carries and why it matters to a model.

Version

Version 0.1.0-borrador, published on 2026-08-02. Methodology changes are logged below and previous scores are not recalculated: every report states which version generated it.

The four layers and their weights

Discoverability: 25 · 7
Content accessibility: 30 · 7
Semantic structure: 30 · 9
Entity authority: 15 · 5

The 28 checks

Discoverability

robots.txt reachable and parseable

robots_exists

What it checks: Checks that /robots.txt exists, returns 200 and contains recognizable directives.
How it is checked: https://yourdomain/robots.txt is requested and parsed for lines in field: value form. A 200 that returns HTML does not count as a robots.txt.
Passing threshold: Passes with 200 and at least one valid directive. A 404, a server error or HTML served with 200 all fail.
Weight: 2 (Discoverability)
Why it matters to a model: robots.txt is the first file any crawler requests. Without it, each agent decides on its own what to do with the site, and whoever publishes the site does not control that decision.

Explicit rules for AI crawlers

robots_ai_rules

What it checks: Checks whether robots.txt declares its own rule for GPTBot, ClaudeBot, PerplexityBot, Google-Extended, CCBot and Bytespider.
How it is checked: A User-agent block is looked up for each agent. Allowing and denying both count: what is measured is that a decision has been declared, not which one.
Passing threshold: Passes with all six declared. Partial with one or more but not all. Fails with none.
Weight: 3 (Discoverability)
Why it matters to a model: An explicit Allow communicates intent; a missing rule communicates neglect. Several crawler operators treat sites that name them differently, because naming them shows somebody thought about it.

AI crawlers are not blocked by accident

robots_not_blocking

What it checks: Checks that no AI crawler is disallowed from the site root.
How it is checked: robots.txt is evaluated per agent using the standard rule: the longest pattern wins, and Allow wins ties.
Passing threshold: Passes if no AI agent is blocked from /. Blocking all of them is partial, since it may be deliberate. Blocking only some fails.
Weight: 3 (Discoverability)
Why it matters to a model: A model cannot cite what it cannot crawl. Blocking a subset of agents is almost always an inherited rule nobody reviewed rather than a decision, and the effect is disappearing from those engines.

Sitemap declared and reachable

sitemap_declared

What it checks: Checks that a valid XML sitemap exists and that robots.txt announces it.
How it is checked: The Sitemap directive in robots.txt is read and, failing that, /sitemap.xml is tried. The file must return 200 and contain a urlset or a sitemapindex.
Passing threshold: Passes if the sitemap is valid and declared in robots.txt. Partial if reachable but unannounced. Fails if missing or not valid XML.
Weight: 2 (Discoverability)
Why it matters to a model: The sitemap is the full list of what a site considers important. Without it, a crawler only finds what is linked from wherever it entered, and deep pages may never be discovered.

Modification dates in the sitemap

sitemap_fresh

What it checks: Checks what share of the sitemap's URLs declares a recent lastmod date.
How it is checked: loc tags and parseable lastmod tags are counted, and the percentage carrying a date within the last twelve months is calculated.
Passing threshold: Passes above 50% of URLs with a recent lastmod. Partial with some. Fails with no dates at all.
Weight: 1 (Discoverability)
Why it matters to a model: A crawler on a budget uses lastmod to decide what to re-read. A sitemap without dates forces it to treat everything alike, so what changed yesterday gets re-read whenever, not when it matters.

llms.txt file

llms_txt

What it checks: Checks whether a /llms.txt exists in Markdown with a heading and links.
How it is checked: https://yourdomain/llms.txt is requested and checked for not being HTML, having at least one Markdown H1 and at least one link.
Passing threshold: Passes with H1 and links. Partial with only one of the two. Fails if missing or returning an HTML page.
Weight: 2 (Discoverability)
Why it matters to a model: llms.txt is a Markdown index meant to let an agent understand what the site is about and where to start, without inferring it from the navigation menu. It is cheap to maintain and still uncommon.

Link discovery headers

link_headers

What it checks: Checks whether the home page sends an HTTP Link header with rel canonical or alternate.
How it is checked: The Link header of the home page response is inspected for rel="canonical" or rel="alternate".
Passing threshold: Passes with either present. Fails with no Link header.
Weight: 1 (Discoverability)
Why it matters to a model: A crawler that only reads headers, without parsing HTML, gets the canonical and the alternate versions of the page before downloading the body. For an agent walking thousands of URLs, that signal resolves duplicates without spending an extra request.

Content accessibility

Render ratio

render_ratio

What it checks: Measures what share of the visible text already exists in the initial HTML, before JavaScript runs.
How it is checked: Text is extracted from the raw HTML and from the page after loading it in a real browser until networkidle, both normalized by stripping whitespace, and then divided.
Passing threshold: Passes at a ratio of 0.80 or above. Partial between 0.50 and 0.79. Fails below 0.50.
Weight: 4 (Content accessibility)
Why it matters to a model: Most AI crawlers do not run JavaScript. They see the page as it leaves the server, and if the content is injected afterwards they find a nearly empty document and have nothing to cite. This is the check most websites fail.

Main content without JavaScript

main_content_in_html

What it checks: Checks that the H1 and a substantive paragraph exist in the raw HTML.
How it is checked: The downloaded HTML is searched, without executing anything, for an H1 with text and at least one paragraph of twelve words or more.
Passing threshold: Passes with H1 and paragraph. Partial with an H1 but no paragraph. Fails with no H1.
Weight: 3 (Content accessibility)
Why it matters to a model: The H1 and the first paragraph are what a model uses to decide what the page is about. If they arrive empty, the rest of the content is irrelevant: the page does not enter the answer because nobody knows what it discusses.

Markdown content negotiation

markdown_negotiation

What it checks: Checks whether the server returns Markdown when the client asks for it.
How it is checked: The home page is requested with the header Accept: text/markdown and the response is checked for 200 with content-type text/markdown.
Passing threshold: Passes with 200 and content-type text/markdown. Fails otherwise.
Weight: 1 (Content accessibility)
Why it matters to a model: An agent that receives Markdown skips parsing HTML and does not drag menus, footers or cookie notices into the passage it cites. It is a cheap advantage and still rare.

Clean redirect chain

status_and_redirects

What it checks: Checks that reaching the home page does not require several hops and that the final destination is https.
How it is checked: Redirects are followed one by one, recording every URL, and loops are detected by comparing the chain travelled.
Passing threshold: Passes with one hop or none and an https destination. Partial with two hops. Fails with more than two, with a loop, or if the destination is not https.
Weight: 2 (Content accessibility)
Why it matters to a model: Every hop is one more request and one more chance to lose headers, cookies or the crawler itself. Some clients stop following the chain before the end, and for them the site simply does not respond.

Time to first byte

ttfb

What it checks: Measures how long the server takes to start responding.
How it is checked: Timed from opening the request to the arrival of the first response headers of the home page.
Passing threshold: Passes below 800 ms. Partial between 800 and 1600 ms. Fails above 1600 ms.
Weight: 1 (Content accessibility)
Why it matters to a model: Response time shapes how many pages an agent crawls per visit. A slow server does not prevent a page from being cited, but it reduces how many pages get read at all.

Same content for bots and for people

bot_parity

What it checks: Compares the text served to a browser with the text served to a crawler.
How it is checked: The home page is requested twice, once with a browser user-agent and once with PrimafonteBot's, and the length of the visible text extracted from each response is compared.
Passing threshold: Passes below 10% difference. Partial below 30%. Fails above that.
Weight: 3 (Content accessibility)
Why it matters to a model: Serving a bot a different version of the page is cloaking, whether or not whoever configured the cache knows it. The model indexes what it receives, not what a person sees, and the gap turns into wrong citations or none.

The crawler is not blocked

waf_block

What it checks: Checks that the site does not answer a bot user-agent with a block or a captcha.
How it is checked: The home page is requested with PrimafonteBot's user-agent and the status code and response text are inspected for browser-verification notices.
Passing threshold: Passes with 200 and no captcha. Partial with another non-blocking code. Fails with 401, 403, 429, or a captcha served with 200.
Weight: 3 (Content accessibility)
Why it matters to a model: A firewall that blocks bots by default also shuts out AI crawlers. A 200 that serves a captcha blocks just as effectively as a 403, and it additionally fools anyone who only looks at the status code.

Semantic structure

Structured data on the home page

jsonld_present

What it checks: Checks that the home page includes at least one parseable JSON-LD block.
How it is checked: script tags of type application/ld+json are counted in the HTML and each one is parsed.
Passing threshold: Passes with at least one parseable block. Fails with no blocks, or if none parses.
Weight: 2 (Semantic structure)
Why it matters to a model: Structured data is the only part of a website written to be read by a machine rather than interpreted. Without it, a model has to infer from the prose who publishes the page and what it covers.

Valid structured data

jsonld_valid

What it checks: Checks that the JSON-LD blocks parse and that the declared types carry their required fields.
How it is checked: Every block is parsed, @graph structures are flattened, and each known type is checked for the fields without which it identifies nothing, such as name on Organization or headline on Article.
Passing threshold: Passes with no broken blocks and no missing required fields. Partial with one or two gaps. Fails with a block that does not parse or more than two gaps.
Weight: 2 (Semantic structure)
Why it matters to a model: A block with one stray comma does not exist for the model reading it: it is discarded whole and silently, so the site loses the signal without noticing. And a type declared without its name is decorative markup, because it omits the very fact that justified declaring it.

Entity and business types declared

jsonld_entity_types

What it checks: Checks that the site declares who publishes it and what it offers.
How it is checked: The declared types are searched for an entity type such as Organization or Person, and for one describing the offering such as SoftwareApplication, Product, Service or Article.
Passing threshold: Passes with both present. Partial with only one. Fails with neither.
Weight: 2 (Semantic structure)
Why it matters to a model: A model needs to resolve two separate questions: who these people are and what they sell. Declaring only one leaves half the answer open to interpretation, and that is where wrong descriptions come from.

Heading hierarchy

heading_hierarchy

What it checks: Checks that there is a single H1 and that no level is skipped.
How it is checked: All headings are extracted in document order, H1s are counted, and skips are detected, such as going from H2 to H4 with no H3.
Passing threshold: Passes with exactly one H1 and no skips. Partial with two H1s or up to two skips. Fails with no H1, more than two, or several skips.
Weight: 2 (Semantic structure)
Why it matters to a model: Headings are the document's outline. A skipped level leaves a block hanging off nothing, and the chunking a model performs in order to cite passages leans on exactly that structure.

Direct answers after each heading

direct_answers

What it checks: Checks whether the first paragraph following each H2 answers that heading.
How it is checked: For each H2 the first following paragraph is taken and scored: the 40 to 80 word band scores one point, the 25 to 120 range scores half, and starting with a connector that depends on the previous paragraph scores zero.
Passing threshold: Passes at an average score of 60% or above. Partial from 30%. Fails below that.
Weight: 3 (Semantic structure)
Why it matters to a model: A model does not cite pages, it cites passages. A heading followed by a paragraph that answers it produces a fragment that stands on its own; followed by a transition or an image, it produces nothing citable.

Paragraphs that stand outside their context

chunk_independence

What it checks: Checks what share of the paragraphs makes sense without having read the previous one.
How it is checked: Paragraphs of twenty words or more are taken and their first sentence is searched for a pronoun or deictic with no antecedent, such as it, this, here or the above.
Passing threshold: Passes with 50% or more independent paragraphs. Partial from 25%. Fails below that.
Weight: 3 (Semantic structure)
Why it matters to a model: A passage is citable only if it makes sense on its own. A paragraph opening with that is fixed by forces the model to drag in the previous paragraph, and as soon as the fragment is trimmed it loses meaning and stops working as a source.

Title and meta description length

title_meta

What it checks: Checks that the title and meta description fit the space they are given.
How it is checked: The character length of the home page title and of the meta description content attribute are measured.
Passing threshold: Passes with a title of 30 to 60 characters and a description of 110 to 160. Partial if only one is in range. Fails if neither is, or if they are missing.
Weight: 2 (Semantic structure)
Why it matters to a model: The title and meta description are the summary a site writes about itself, and both search engines and models reuse them verbatim when presenting it. A truncated title or an absent description hand that summary to whoever reads it.

Frequently asked questions marked up

faq_markup

What it checks: Checks that visible frequently asked questions, where present, are marked up as FAQPage.
How it is checked: Question-shaped headings and mentions of frequently asked questions are detected in the text, and a JSON-LD FAQPage block is looked for.
Passing threshold: Passes with a visible FAQ that is marked up. Fails with a visible FAQ left unmarked. With no visible FAQ the check does not apply and does not score.
Weight: 2 (Semantic structure)
Why it matters to a model: The FAQ block is the most cited part of a website, because every question is already a self-contained fragment. Marking it up makes that explicit instead of leaving it to inference.

Descriptive anchor text

internal_links

What it checks: Checks what share of the internal links uses text that says nothing about the destination.
How it is checked: Internal links on the home page are collected and their text compared against a list of generic anchors such as read more, here or click here.
Passing threshold: Passes below 10% generic anchors. Partial below 25%. Fails above that. With fewer than five internal links it does not score.
Weight: 2 (Semantic structure)
Why it matters to a model: Anchor text is one of the signals a model uses to understand what the destination page covers before visiting it. A read more contributes none, and wastes the link as a signal.

Entity authority

Brand name consistency

name_consistency

What it checks: Checks that the brand is written the same way in JSON-LD, in Open Graph and in the title.
How it is checked: The name declared in JSON-LD is taken as the reference and compared with og:site_name and with the title's segments, ignoring case, accents and punctuation.
Passing threshold: Passes if every available source agrees. Partial if one disagrees. Fails if all disagree. With a single source it does not score.
Weight: 3 (Entity authority)
Why it matters to a model: A model has to resolve the brand to one entity. Three variants of the name on the same page produce three candidates, and the authority that should accumulate on one gets split across all three.

Description consistency

description_consistency

What it checks: Checks that the meta description, the Open Graph one and the JSON-LD one say the same thing.
How it is checked: They are compared pairwise using the Dice coefficient over words, ignoring case and accents, and the worst-scoring pair decides.
Passing threshold: Passes at a similarity of 0.80 or above. Partial from 0.50. Fails below that. With a single description it does not score.
Weight: 3 (Entity authority)
Why it matters to a model: The description is the sentence a model will use to introduce the site when it cites it. Two different versions force a choice, and the one chosen is not always the one the publisher wanted.

Identifiable company or author page

about_page

What it checks: Checks that a page exists explaining who is behind the site.
How it is checked: Links whose target or text points at an about, about us, company or team page are looked for, failing which an entity declared in JSON-LD with url or @id.
Passing threshold: Passes with a link to the page. Partial with only an entity declared in JSON-LD. Fails with neither.
Weight: 2 (Entity authority)
Why it matters to a model: The about page is where a model confirms there is a real entity behind the site. Its absence does not prevent citation, but it lowers confidence in content that cannot be attributed to anyone.

Readable contact details

contact_verifiable

What it checks: Checks that there is an email or a phone number a machine can read.
How it is checked: An email or phone number is looked for in the visible text and in mailto and tel links, failing which in the email and telephone fields of the JSON-LD.
Passing threshold: Passes with contact details in the visible text or in a link. Partial if only declared in JSON-LD, or if the phone appears without a link. Fails with neither.
Weight: 2 (Entity authority)
Why it matters to a model: A model does not fill in forms. Contact details that only exist behind a form, inside an image or obfuscated in JavaScript cannot be cited or verified, and they are one of the signals that mark out a real business.

External profiles declared with sameAs

sameas_links

What it checks: Checks how many external profiles the site declares through the sameAs property.
How it is checked: All sameAs values across the JSON-LD blocks are collected. If there are none, the HTML is checked for links to known profiles.
Passing threshold: Passes with two or more sameAs entries. Partial with one, or with two profiles linked but undeclared. Fails with none.
Weight: 3 (Entity authority)
Why it matters to a model: sameAs is what lets a model tie the site to every other trace of the same entity: LinkedIn, GitHub, a company register. Without that link, each trace resolves as a separate entity and none accumulates enough signal to get cited.

What this index does not measure

The Citability Index does not measure the quality of your content, your reputation, your inbound links or whether a model mentions you today. It measures whether the technical and structural possibility of being cited exists at all.

Known limitations

Generative engines do not publish the internal weights they use to select sources. The Citability Index measures observable, reasoned signals, not a formula reproduced from any specific model.
Results vary between engines. A site can be perfectly readable to one crawler and still fall outside another's answers, on selection criteria that are not technical.
Several of the standards being checked are still being defined. The clearest case is llms.txt: there is a de facto convention, not a settled specification, and its weight may change.
Checks run against the home page and the resources it declares. A site may have internal sections that behave differently from its front page.
The render ratio is measured with a browser that does execute JavaScript, while each AI crawler decides on its own whether and how far to execute it. The ratio indicates risk, not the exact behaviour of any specific engine.

Changelog

0.1.0-borrador · 2026-08-02
First public version of the methodology. Thresholds are provisional and will be frozen in version 1.0, after calibration against a sample of 200 websites. Until then, scores may vary between scans.