AI SEO Agent: What It Actually Catches, From Running One

Most writing about AI SEO agents is speculative. This is not. Over the past few weeks we pointed an agent at this site, with live Search Console, GA4 and Ahrefs access, and worked through a backlog of articles one at a time. What follows is what it found, what it got wrong, and the setup that made the difference, with the actual numbers.

The short version: the wins did not come from writing. They came from the agent checking things a human reviewer assumes are fine.

The finding that paid for the whole exercise

One of our pages targets “amazon ppc tools”. It had been live for months. Search Console showed it ranking position 51 for that exact phrase, while sitting at position 6.5 for the generic “ppc tools” it was never written for.

The agent fetched the rendered page and counted the headings:

h1 count: 0

The theme prints the post title as an h2, so unless an article supplies its own h1, the page ships without one. Nobody notices, because the page looks completely normal in a browser. We found the same thing on two more pages, one of which takes 816 impressions a month and ranks in the top five for several commercial queries while converting none of them.

That is the category of problem agents are good at: boring, invisible, and consistent. A person auditing those pages reads the content and moves on. An agent that fetches the live HTML and counts tags does not get bored on page three.

Two tools, two answers, and only one of them true

A competitor review page on this site reports like this:

Ahrefs puts the third page at 886 visits a month. GA4, which counts what actually happened, says 84. That is a factor of ten, on the page we were about to describe as a traffic asset.

Neither number is wrong, exactly. One is a model and one is a measurement. The point is that a workflow reading a single source never sees the gap, and an agent holding both open flags it in the same pass. We would have quoted the wrong figure in a sales document otherwise.

Splitting a page’s demand by intent

The clearest single analysis the agent ran took one page’s 414 queries and sorted them by what the searcher actually wanted:

The page ranked twice as well for queries it could not satisfy as for the ones it existed to answer. Zero clicks from 2,649 impressions, and now an obvious reason why.

No keyword tool produces that table, because the classification is a judgement about meaning. That is the part worth handing to a language model, and it is reproducible: the same split on a different page showed 86% of impressions were about joining a programme while the page was selling an agency service.

Where the agent was wrong, which matters more

Three failures from the same run, because a list of wins is not useful on its own.

It nearly published a statistic that does not exist. A draft carried “90% of Amazon private label sellers source from Alibaba.” Checked against Jungle Scout’s published seller research, the figure is not there, and the direction is arguably wrong: their 2025 report describes China sourcing falling while US sourcing roughly doubled. The sentence read perfectly and was invented.

It overstated a finding. Checking a tool we had reviewed, every URL on the vendor’s domain including the login page returned a redirect to a third party. The first conclusion was “this product is dead”. The accurate version is that an acquirer bought the domain and the product has not formally shut down. Those are different claims and only one of them was defensible in public.

It misread the existing setup. It reported a page as having no structured data and prepared to add some. The page already emitted FAQPage and BlogPosting through the SEO plugin. Adding more would have created duplicate, conflicting schema, which is worse than the problem it was solving.

The pattern across all three: the agent was confident, fluent and wrong, and in each case the fix was checking a primary source rather than reasoning harder. Which is the single most useful thing to know about running one.

The setup that made it work

The difference between an agent that produces this and one that produces generic advice is almost entirely the data it can reach.

Those connections are MCP servers, which is the layer that lets an agent reach a system at all. The method it follows is a skill, and the two are often bundled together as a plugin. Public SEO skills worth reading before writing your own include seo-audit, ai-seo, programmatic-seo, site-architecture and schema in the marketingskills collection.

The guardrails that mattered more than the prompts

Every change went through a script that refused to write if a check failed. Specific refusals that fired during the run and prevented real damage:

• Edit anchors must match exactly once. If the text an edit targets appears zero or two times, nothing is written. This caught several near-misses where a replacement would have hit the wrong section.
• Schema questions must exist as visible copy. An FAQ added to structured data but not to the page is a guideline violation; the script blocked it.
• Snapshot before every write. Every edit wrote the previous version to disk first. Cheap, and the reason nothing needed recovering.
• No publish by default. New articles are created as drafts and only go live on an explicit flag, so an automation bug cannot publish.

If you take one thing from this: an agent with write access and no refusals is not an SEO agent, it is an outage waiting for a reason.

What it is worth, honestly

An agent did not invent strategy here. Every genuinely good decision in the run came from reading data a person could have read, and the agent’s contribution was doing it on every page without getting bored, and noticing when two sources disagreed.

What it is bad at is knowing when to stop. It will optimise a page that should be merged, write a section for a query that does not deserve one, and present a plausible number it has no source for. The work is still yours. The tedium is not.

Frequently asked questions