Will Critchlow
banner
willcritchlow.bsky.social
Will Critchlow
@willcritchlow.bsky.social
330 followers 37 following 180 posts
Founder and CEO at SearchPilot. Previously founded Distilled (acquired by Brainlabs). Interest in basketball, whisky, cooking, auction theory and of course organic search.
Posts Media Videos Starter Packs
And there is a key extra variable in GEO experiments when compared to SEO tests:
GEO vs. SEO is potential ragebait. I'm more interested in understanding what's going on.
I really enjoyed the conversation I had with @aleyda.bsky.social about ecommerce search in an AI-first world, and, for those who prefer to listen rather than watch, the audio is now available on the Business Class Lounge podcast feed.

Go grab it wherever you get your podcasts - it's a good one!
Testing capabilities are a *requirement* for doing effective ecommerce SEO at scale. I don't see why GEO is going to be any different.

Spoiler: you can test these hypotheses using SearchPilot!
Almost everything you see about how to do better in AI search is speculation. Guesswork. Predictions.

We need to move from react mode to control mode. We need to go from predicting the future to experimenting to discover it.
[Spoiler: SearchPilot has released the ability to run experiments against shopping / merchant center data, as well as against impressions alongside clicks. Stop trying to predict the future, experiment to discover it.]
Join my conversation with @brodieclark.bsky.social who's tracked more details of this stuff than anyone I know, and think about how you can optimise in this environment.
The move from PLPs (category pages) ranking to PDPs (product pages), plus the inclusion of product "grids" taken from merchant center / organic shopping results all mean that any kind of month-over-month or year-over-year tracking could show wild trends in all directions.
While we're all very excited about AI, there's another search change that is arguably MORE dramatic for ecommerce and retail in the short term: Google's changing approach to product search.
@ryanlucht.com was kind enough to share his experiences from the field of CRO and user testing where they have been addressing these questions for longer than even we have in the SEO testing world.

There's a clip above, and here's the full conversation:
www.searchpilot.com/resources/bl...
SEO testing lessons from more mature experimentation programs - a conversation with Ryan Lucht
Discover lessons from CRO to SEO testing with Ryan Lucht. Learn how to build experimentation culture, prove impact, and scale search growth.
www.searchpilot.com
The core answer is that if it were that simple, you wouldn't have needed the test to spot the improvement, but the deeper answer is in the communication of uncertainty.
We do a load of hard statistical work to tease out the signal, but then sometimes hear "but why can't I see the x% uplift in before and after numbers".
A challenge of testing is that detecting small, single-digit % improvements is hard. Doing so repeatedly can be transformational to business performance, but each individual one can get lost in the noise.