effects of split testing on amazon

The Data-backed Answer: How Does Split Testing Effect Your Amazon Ranking?

andrew browne Split Testing Best Practices Leave a Comment

Sharing is caring!

There is a rumor in the Amazon seller community that there is a penalty each time you update your listing, in that you will see a drop in sessions or ranking after updating a listing. People have concerns that they will hurt their rankings for their main keywords.


Now that Splitly has been live and kicking for some months, we have a lot of data from our customers tests, which we can analyse to see if there’s any truth to this claim.


We compared the total number of sessions before and after tests are started, to see how running a split test might effect them.


The results are show below:

updated listing daily on amazon

This chart shows the relationship between sessions before a test and after a test. As you can see, the number of sessions is relatively unchanged. The small discrepancy lies within a very small error range. The sessions increase with the number of days, as more days are being included. Later they decrease, as less tests can be used to get data.


Overall sessions before is very close to sessions after, with sessions after being slightly improved in parts. We are not sure if this is due to Amazon rewarding listings for being regularly updated, or our customers choosing good variants to test.


However either way, we can be sure that on average split testing is not hurting sellers rankings at all. As long as you comply with Amazons policies – you have next to zero risk of your rankings being hurt.


Now here is the upside: when you test the keywords in your Product Title, you can find new pockets of Amazon customers. And these customers could bring you more clicks, conversions, and sales!


Here is a snapshot of a Splitly test from a user who was looking to increase their daily sessions by including their brand name in the Product Title:

title text for increased sessions

There is a significant jump in their daily sessions (the bar graph in the upper left), which was their primary goal in running the test.


You may see varying opinions around the web on whether to include a brand name in the Product Title. Is it the right move to include it, or omit it? The answer completely depends on your listing, brand, and target audience. How do you find out what you should do? Yep, you guessed correctly: test it!


The moral of the story is that daily updates have absolutely no negative effect on your organic Amazon ranking. Moreover, if you are not running tests on a daily basis, you are continuing to miss opportunities to improve your listing and profit potential. If you have any questions about this, please drop them in the comments section below so I can answer them directly!

Sharing is caring!

Leave a Reply

Your email address will not be published. Required fields are marked *