I audited my own SaaS for AI agents. Half my content was invisible.

I audited my own SaaS site for AI agents. Half my content was invisible.

A solo founder's wake-up call about the silent web crisis nobody warned us about.

I spent 30 years as a civil engineer in Kharkiv. I sized rebar. I checked rebar twice. The first rule of structural engineering is: if you can't measure it, it isn't there.

So when I started building software two years ago, I assumed the rule still held. If my website worked, it worked. If Google indexed it, it indexed. Done.

Then I ran an audit on my own site.

What I expected to find: nothing

I run guardlabs.online. We've been around 8 months. We're not Stripe — but we're not a single-page hobby project either. Multiple landing pages, half a dozen products, three languages, real customers.

I wrote a quick auditor to check what AI agents — ChatGPT, Claude, Perplexity, Grok, the dozens of B2B agents scraping the web in 2026 — actually see when they look at our site.

I expected one or two yellow warnings.

What I actually found: 12 red flags

Here's the slightly edited output:

✗ /llms.txt — missing (AI agents have no idea what this site is about)
✗ robots.txt — no rules for GPTBot/ClaudeBot/PerplexityBot
✗ /.well-known/ai-plugin.json — missing (no manifest for agent integration)
✗ OpenAPI spec — missing (your API is invisible to agents)
✗ RSS / Atom feed — missing (agents can't subscribe to updates)
✗ Sitemap.xml hreflang — broken on 3 language pairs
✗ JSON-LD Organization — missing
✗ JSON-LD SoftwareApplication — missing
✗ Content-Type — text/html on JSON API responses
✗ Open Graph — incomplete on 4 product pages
✗ Meta description — missing on /agent-ready, /audit, /care
✗ canonical URLs — wrong on /es/* pages

My own product page — the one selling AI-readiness as a service — was failing 12 AI-readiness checks.

I sat with that for a minute.

The thing nobody tells you in 2026

Here's what's changed in the last 18 months and almost nobody acted on:

The traffic that discovers your site is no longer just Googlebot. It's a swarm:

  • GPTBot (OpenAI) — indexes content for ChatGPT answers
  • ClaudeBot (Anthropic) — same for Claude
  • PerplexityBot — Perplexity's real-time search
  • CCBot (Common Crawl) — feeds dozens of LLMs
  • Google-Extended — Google's Gemini-specific crawler
  • Bytespider — TikTok / Doubao AI
  • Applebot-Extended, Bingbot, MJ12bot, AhrefsBot, and 30+ less polite scrapers

Each of these makes a slightly different request. Each respects slightly different signals. Most don't respect robots.txt at all. Some require an llms.txt (a new file specifically for them). Some look for .well-known/ai-plugin.json. Some need OpenAPI specs to interact with your product. Some only follow JSON-LD schema.

In 2024, you had to please one crawler well (Googlebot) and you got 90% of your discovery traffic.

In 2026, you have to please eight crawlers reasonably and you'll get maybe 60% of your discovery traffic. Mess up one and you lose ~5-15% of that.

My site was messing up eight.

Why this is the silent crisis

Nobody's writing about this because nobody can see it.

Your Google Analytics still shows visitors. Your Search Console looks fine. Your uptime monitor shows 99.97%. Everything technically works.

But a quiet trend is happening underneath: when a user asks ChatGPT "recommend a good website-monitoring SaaS" — does ChatGPT know your site exists? Does it understand what you do? Can it pull pricing? Can it tell the user how to sign up?

If your llms.txt is missing, the answer is "probably no". If your JSON-LD is missing, "definitely no" on pricing. If your robots.txt blocks GPTBot, you literally don't exist in the answer.

Search is bifurcating. Half your future customers are asking AI agents instead of typing into Google. The half you don't see in Analytics.

The painful part: it took me 30 seconds to find out

The whole audit ran in 30 seconds. It just looks at signals: file presence, headers, structured data, sitemap shape, content-type. Boring engineering. No magic.

I wrote it as a tool for our customers. I never thought to run it on our own site.

This is the funny thing about building reliability tools — you spend so much time pointing the flashlight at other people that you forget to point it at yourself.

What I did about it (the boring fix list)

In one weekend I:

  1. Wrote /llms.txt with a sentence about each product
  2. Added explicit User-agent: GPTBot / ClaudeBot / PerplexityBot rules in robots.txt
  3. Wrote /.well-known/ai-plugin.json pointing at our public API
  4. Generated OpenAPI spec from our Flask routes (FastAPI users — yours is automatic)
  5. Added JSON-LD Organization and SoftwareApplication blocks on landing pages
  6. Fixed Open Graph on 4 product pages
  7. Fixed canonical URLs on Spanish pages
  8. Set Content-Type correctly on JSON endpoints
  9. Added an Atom feed for the blog
  10. Re-checked hreflang in sitemap

Total time: ~6 hours over a weekend. Cost: $0.

AI-Readiness score went from 38/100 to 92/100.

I don't yet have numbers on what this changed in actual AI agent traffic — that's a 6-month story. But I know one thing: I went from "invisible" to "discoverable" in a weekend.

The free tool

Because I felt slightly stupid finding all this on my own site, I made the auditor free for anyone:

guardlabs.online/audit — type your domain, get the 10+ checks in 30 seconds. No signup. Email optional (if you want the PDF report).

I'm not selling anything in this post. The audit is free, the fixes are public, the standards are open. If you find issues, fix them yourself — most are 10-line additions to your repo.

If you do want help, we have a fix-it-for-you service (/agent-ready, one-time) and an ongoing monitoring product (/care/, $240/year) — but those are entirely optional. The audit is the gift.

The civil engineer's parting thought

In construction, we have a saying: the load you don't calculate is the one that kills the building.

In 2026, the discovery channel you don't audit is the one that kills your growth.

Your site might be fine. It's also likely missing 6-12 things you didn't know existed two years ago. Take 30 seconds and find out.

Stas builds reliability tools at GuardLabs from Kharkiv, Ukraine. The free audit lives at guardlabs.online/audit.

🔍 Run the free audit on your site → · No signup, 30 seconds.

Комментарии

Отправить комментарий