AI SEO Agent: What It Actually Catches, From Running One

Most writing about AI SEO agents is speculative. This is not. Over the past few weeks we pointed an agent at this site, with live Search Console, GA4 and Ahrefs access, and worked through a backlog of articles one at a time. What follows is what it found, what it got wrong, and the setup that made the difference, with the actual numbers.

The short version: the wins did not come from writing. They came from the agent checking things a human reviewer assumes are fine.

The finding that paid for the whole exercise

One of our pages targets “amazon ppc tools”. It had been live for months. Search Console showed it ranking position 51 for that exact phrase, while sitting at position 6.5 for the generic “ppc tools” it was never written for.

The agent fetched the rendered page and counted the headings:

h1 count: 0

The theme prints the post title as an h2, so unless an article supplies its own h1, the page ships without one. Nobody notices, because the page looks completely normal in a browser. We found the same thing on two more pages, one of which takes 816 impressions a month and ranks in the top five for several commercial queries while converting none of them.

That is the category of problem agents are good at: boring, invisible, and consistent. A person auditing those pages reads the content and moves on. An agent that fetches the live HTML and counts tags does not get bored on page three.

Two tools, two answers, and only one of them true

A competitor review page on this site reports like this:

Page	Ahrefs organic/mo	GA4 sessions/mo
how-to-find-who-you-follow-on-amazon	1,370	851
amazon-influencer-program-requirements	92	199
collabstr	886	84

Ahrefs puts the third page at 886 visits a month. GA4, which counts what actually happened, says 84. That is a factor of ten, on the page we were about to describe as a traffic asset.

Neither number is wrong, exactly. One is a model and one is a measurement. The point is that a workflow reading a single source never sees the gap, and an agent holding both open flags it in the same pass. We would have quoted the wrong figure in a sales document otherwise.

Splitting a page’s demand by intent

The clearest single analysis the agent ran took one page’s 414 queries and sorted them by what the searcher actually wanted:

Intent	Queries	Impressions	Weighted position
Generic PPC, not Amazon	162	1,405 (53%)	15.4
Amazon-specific	251	1,162 (44%)	31.5
Irrelevant	1	82 (3%)	27.9

The page ranked twice as well for queries it could not satisfy as for the ones it existed to answer. Zero clicks from 2,649 impressions, and now an obvious reason why.

No keyword tool produces that table, because the classification is a judgement about meaning. That is the part worth handing to a language model, and it is reproducible: the same split on a different page showed 86% of impressions were about joining a programme while the page was selling an agency service.

Where the agent was wrong, which matters more

Three failures from the same run, because a list of wins is not useful on its own.

It nearly published a statistic that does not exist. A draft carried “90% of Amazon private label sellers source from Alibaba.” Checked against Jungle Scout’s published seller research, the figure is not there, and the direction is arguably wrong: their 2025 report describes China sourcing falling while US sourcing roughly doubled. The sentence read perfectly and was invented.

It overstated a finding. Checking a tool we had reviewed, every URL on the vendor’s domain including the login page returned a redirect to a third party. The first conclusion was “this product is dead”. The accurate version is that an acquirer bought the domain and the product has not formally shut down. Those are different claims and only one of them was defensible in public.

It misread the existing setup. It reported a page as having no structured data and prepared to add some. The page already emitted FAQPage and BlogPosting through the SEO plugin. Adding more would have created duplicate, conflicting schema, which is worse than the problem it was solving.

The pattern across all three: the agent was confident, fluent and wrong, and in each case the fix was checking a primary source rather than reasoning harder. Which is the single most useful thing to know about running one.

The setup that made it work

The difference between an agent that produces this and one that produces generic advice is almost entirely the data it can reach.

Layer	What it provides	Why it matters
Search Console API	Query and page data, paginated past the UI’s limits	The only source for what you actually rank for. The 25,000-row page limit means the UI hides most of it
GA4 API	Sessions, engagement, channel	The reality check on every third-party estimate
A keyword tool API	Volume and difficulty	Search Console cannot tell you about queries you do not rank for
Live page fetches	The rendered HTML	Where the h1 problem was found. What is served is not always what the CMS stores
Write access to the CMS	Applying the fix	Otherwise you have a report, not a change

Those connections are MCP servers, which is the layer that lets an agent reach a system at all. The method it follows is a skill, and the two are often bundled together as a plugin. Public SEO skills worth reading before writing your own include seo-audit, ai-seo, programmatic-seo, site-architecture and schema in the marketingskills collection.

The guardrails that mattered more than the prompts

Every change went through a script that refused to write if a check failed. Specific refusals that fired during the run and prevented real damage:

• Edit anchors must match exactly once. If the text an edit targets appears zero or two times, nothing is written. This caught several near-misses where a replacement would have hit the wrong section.
• Schema questions must exist as visible copy. An FAQ added to structured data but not to the page is a guideline violation; the script blocked it.
• Snapshot before every write. Every edit wrote the previous version to disk first. Cheap, and the reason nothing needed recovering.
• No publish by default. New articles are created as drafts and only go live on an explicit flag, so an automation bug cannot publish.

If you take one thing from this: an agent with write access and no refusals is not an SEO agent, it is an outage waiting for a reason.

Where an agent differs from the SEO tool you already pay for

Most teams considering this already have a crawler and an SEO suite, so the question is not whether an AI SEO agent is useful in the abstract. It is what it adds to a stack that already exists.

Three genuine differences, and one thing that is not a difference:

• It reads the page as prose. A crawler checks whether a title exists and how long it is. An agent can tell you the title promises a comparison and the page delivers a definition, which is the mismatch that actually suppresses a ranking and the one no rule-based check has ever caught
• It answers questions you did not pre-configure. Crawlers report against a fixed list. You can ask an agent something specific to this week’s problem and get an answer about it
• It explains rather than flags. “Missing H1” is a flag. “This page has no H1 because the theme prints the title as an H2, so every article on the site has the same problem” is a diagnosis, and the second one gets fixed

What is not a difference: accuracy of the mechanical checks. A crawler is better at those, it is faster, it is cheaper and it does not invent anything. Replacing one with the other is a mistake in either direction, and the sensible setup runs both for what each is good at.

There is a fourth difference worth naming because it decides whether any of this is usable in practice. A crawler produces the same output every run, which means you can diff this month against last month and see what changed. An agent does not guarantee that: ask the same question twice and the wording, the ordering and sometimes the emphasis will differ. For tracking regressions over time that is a genuine weakness, and it is the main reason to keep the deterministic tool rather than replace it. Use the agent for questions and the crawler for monitoring.

What an AI SEO agent cannot do for you

The useful version of this comparison names the limits as clearly as the capabilities, because the failures are predictable and they are not the ones people expect.

• It cannot decide what the page is for. An agent will tell you a title is weak. It cannot tell you whether the page should exist, which is the decision that actually moves traffic
• It does not know your business. Which products carry margin, which queries bring buyers rather than readers, which competitor you cannot afford to lose to. Every recommendation is made without that context unless you supply it
• It cannot see what you have not given it. Without Search Console or analytics access, an agent is reasoning from the page alone, which is the least informative source available about how that page performs
• It will be confidently wrong sometimes. The failure mode is not an error message. It is a plausible recommendation with no flag on it, which is why a human reads the output before anything ships

So the honest framing is narrow and still valuable: an agent compresses the mechanical half of an audit from hours into minutes, and leaves every judgement call exactly where it was. That is a real saving on the half of the work that was never the interesting half.

It is worth being precise about who that saving reaches, because it is not distributed evenly. Somebody who already knows what good looks like gets an assistant that drafts the boring part and a reviewer’s eye on the output. Somebody who does not gets fluent recommendations with no way to tell the good ones from the plausible ones, which is a worse position than having no tool at all, because the output carries an authority it has not earned. The tool amplifies judgement rather than supplying it.

How to run one without creating work

The common failure is not a bad agent. It is an agent producing more findings than anyone will ever action, which converts a tool into a backlog.

1. Scope it to a question, not a site. “Audit everything” returns a list nobody reads. “Which of these forty pages has a title that does not match what it ranks for” returns something you can act on this week
2. Give it your own data. Search Console impressions and positions turn generic advice into prioritised advice, because the agent can then rank findings by what they are worth rather than by what it noticed
3. Cap the output deliberately. Ten findings you will fix beat two hundred you will not
4. Re-run on a schedule rather than continuously. Monthly is enough for most sites. Continuous monitoring mostly detects your own changes

One practical note on tooling: much of this now runs through agent frameworks with their own documentation and conventions, and the official docs for the agent platform you choose are worth reading before you build anything, because the constraints you design around are usually in there rather than in a blog post.

What it is worth, honestly

An agent did not invent strategy here. Every genuinely good decision in the run came from reading data a person could have read, and the agent’s contribution was doing it on every page without getting bored, and noticing when two sources disagreed.

What it is bad at is knowing when to stop. It will optimise a page that should be merged, write a section for a query that does not deserve one, and present a plausible number it has no source for. The work is still yours. The tedium is not.

Frequently asked questions