AI crawler access

Free website diagnostics

Check your website’s access, structure and content

Inspect public pages for crawler rules, readable HTML, structured data and content signals. Review the evidence and decide what needs attention.

No sign-up required · Evidence you can inspect · No citation guarantees

Browse all tools · View a sample report

Start with crawler access

This check inspects crawler access rules for one URL. It does not run every tool.

Initial HTTP responses only; no JavaScript browser rendering.

Scope of this check

Inspect robots.txt and page directives for the submitted URL.

Permission in robots.txt is not proof that a real crawler can pass every network or security control.

No result yet

Enter a public URL to run this check.

Choose a check for your task

llms.txt check

Inspect optional llms.txt files and their referenced links.

Claims & sources

Inspect source markup near claim candidates and relevant authorship or date signals.

Brand identity

Compare the names, descriptions and identity signals declared on the submitted page.

Page comparison

Compare two to five pages using the same checks and inspect their differences.

Technical baseline

Record a dated snapshot of six technical categories with supporting observations.

From a URL to a clear next step

  1. Enter a public page URL and choose the relevant check.
  2. Review observations, evidence and limits.
  3. Make relevant changes, then rerun the check to verify them.

Questions about these checks

Does a good result guarantee an AI citation?

No. These tools inspect selected website signals; they do not predict AI recommendations or citations.

Does LaComfe change my website?

No. Checks are read-only. Any draft or recommendation must be reviewed and applied by you.

Does a check cover my whole website?

Usually it covers one submitted page and a limited set of resources. Each report states its actual scope.

Is llms.txt required?

No. It is optional and experimental. Focus first on access, readable content and accurate information.

Can I check private pages?

No. Submit public URLs only. Do not include credentials or sensitive information.