How SEO Teams Stop Guessing Their AI Search Stratégies: A Testing Method

Tuesday morning, a client calls. He’s invested $8,000.

$8,000 into content built for AI. Articles stuffed with schema, cross-citations, entity markup.
Result after 4 months: 3 mentions. Not 3 per page. 3 across the entire site.
And the worst part? Zero repeatability. The client tells me: « I don’t know what worked, why it worked, or how to do it again. »

This case is not isolated. Since language models invaded search, I see a new SEO team every week that made a big investment, got a spike in citations… then nothing.

The wall hits everyone the same way: you can’t test cleanly. The A/B test that worked for 10 years on SERPs doesn’t apply to LLMs. No stable control group. No isolatable variable. Responses change from query to query, session to session, model version to model version.

Loren Baker puts it brutally in a Search Engine Journal article:

« You can’t run a clean A/B test on an LLM. There’s no way to split-test a model’s response the way you’d split-test a title tag. »

So most teams read weak signals. A traffic spike from Gemini? They credit the last published article. A jump in Perplexity citations? Check the box « AI search OK ».
It creates an illusion of control.
Until the quarterly review.
Then, no hard evidence. Just guesswork.

The killer question: how do you test without tests?
That’s where the DOSE framework comes in. The one I share with Guillaume Attias at BMO Academy. The central pillar here is Control. Getting measurable grip on a channel that feels slippery.

Running A/B tests on an LLM is like measuring the color of wind

First: each LLM has its own crawlers, its own citation rules, its own signals.
What generates a citation in ChatGPT has nothing to do with Perplexity. None of these models share a stable « SERP » where you could isolate one variable.

Mark Traphagen, VP Product Marketing at seoClarity, says in the same SEJ piece: you need an AI control group without a true split testing. A test structure that isolates what moves, even if the platforms don’t allow pure split purification.

So what do teams that don’t guess do?
They build a three-layer system.

1. Choose which prompts to track.
Not all of them. Only the ones that deliver measurable signal. Classify them: transactional, informational, navigational. Pair them. Weight them.
One of my clients defined 47 precise prompts in their sector (project management SaaS software). Not 46, 47. We mapped the frequency of each prompt across ChatGPT, Claude, Gemini. We kept the 17 that produced a citation in at least one model. That’s the baseline we tracked for 8 weeks.

2. Create a control group without split testing.
On the same site, take pages not optimized for AI. Same domain authority, same depth, same content type—but with no added semantic layer. They become the « bare » organic performance baseline.
When you apply a modification to the « test » pages, you compare citation evolution against that control group. No statistical purge, but a net, repeatable, documented differential.

3. Layer proprietary data on top.
Google lifts the veil with new AI Visibility data in Search Console, but it’s partial. It covers neither ChatGPT nor Perplexity. So you build your own dashboards, cross-referencing crawl data from tools like ZipTie, server logs, and manual citation reports from each platform. The magnitude becomes reliable.

This triptych changes the game. You stop the wet-finger guess and enter a test engineering process. That’s when Control, in the DOSE sense, solidifies.

Voici le protocole en 6 étapes que j’applique avec mes clients. Chaque étape est documentée et reproductible, sans recours aux A/B tests classiques.

The 6 steps of my AI Search testing methodology

When I presented this method to a client, he said: « This is algebra. »
No.
It’s architecture.
A 6-step structure that applies regardless of CMS, industry, or catalog size.

Concrete example: a 945-page site in lab equipment.
Baseline: 11 citations across 40 target prompts.
After 2 test cycles: +42% net citations. Not an average. Raw measurement, page by page, prompt by prompt.
All without a single A/B test. No magic tool. Pure discipline.

The $15,000 mistake I see every week

Another client, a pure-play e-commerce player, decided to measure everything. All prompts. All models. All pages.
$15,000 invested in tracking tools, real-time dashboards, external consulting.
Three months later, the team was drowning in 27,000 data points.

No decision was possible.
The signal was buried in noise.

I see this trap everywhere. Wanting to track « everything, all the time » goes against Control. The human brain needs readable patterns, not a metric frenzy.
The solution: narrow the scope. Fewer prompts, but qualified prompts. One change per cycle. One leading indicator.

The other pitfall: forgetting that ChatGPT and Perplexity don’t respond to the same signals. What boosts citations in one may be ignored by the other. Better to keep a separate matrix, never aggregate results into one score.
And above all, never skip a control step. The rush to see fast results is the grave-digger of measurement.

Control is the ability to say « this action produced exactly this delta, on this model, for these 7 prompts. » Nothing more.
And that’s already huge.

What separates a testing program from a one-shot

I have two curves in front of me.
One from a site that tested once, got a spike, then stalled.
One from a site that integrated the methodology into a 6-week cycle.

The first looks like an EKG.
The second looks like a staircase.

Three factors make the difference:

• Documentation. Every test is logged with the variable, duration, citation delta by platform. Without it, the team starts from scratch every quarter.
• Alignment with semantic clustering. An AI citation doesn’t appear by accident. It often comes from authority built through a semantic cluster. My clients with star-pattern linking get more stable citations on niche prompts. This isn’t correlation: I’ve seen it on 12 projects.
• Iteration pace. 14 days, not more. A longer cycle dilutes variable effect; a shorter one doesn’t give bots time to recrawl and models to relearn.

Those who stabilize the AI Search channel aren’t the ones who get it right on day one.
They’re the ones who document their failures.
Counterintuitive, but it checks out.

That’s the heart of Control: knowing how to redo what worked, without banking on planetary alignment.

Apply this framework to your own ecosystem

You can start tomorrow, zero extra budget.
Take 10 conversational prompts that drive traffic or conversions. The ones your sales team hears in meetings.
Record current citations on ChatGPT, Perplexity, Claude.
Wait a week without touching anything.
Then modify one thing only on the corresponding pages—an FAQ block, a Wikidata entity, an authority citation.
Watch for 10 days.
Compare to your control group.
You have your first delta.

Repeat.
After 3 cycles, you’ll have more reliable data than any off-the-shelf agency report.

The breakthrough is when the SEO team shifts from hope to protocol.
Done with « I hope we get cited on this query. »
Welcome to « I know which modification increases citations on this prompt basket, and I can redo it. »

I’m not selling you the method. I’m showing you the pages.
And the pages don’t lie.

C’est le constat que je dresse en clôture : la majorité des équipes n’ont aucun framework de test reproductible. Sans cela, il est impossible de savoir ce qui a réellement généré une hausse de citations.

You’ve tried measuring AI Search. How did you isolate what really works?

As I write this, 4 out of 5 teams have no structured testing framework.
They navigate by feel, with dashboards that lump everything together without isolating anything.

And you?
When your director asks « Show me what generated that citation spike in March, » what do you say?
A finger pointed at a graph?
Or a documented playbook?

If you don’t have that structure, what are you basing your next move on?

Stéphane Jambu

SEO & AI Engineer

I build growth systems / AI / Neuroscience | 650+ clients · 80 LinkedIn testimonials · 30 years of expertise · 15 years of systems running without me.

Follow on LinkedIn