4 Ways To Split Test For Low Volume Amazon Products

greg mercer Split Testing Best Practices Leave a Comment

Sharing is caring!

I like to think of split testing like pro athletes think about weight lifting: a necessary routine in order to maintain high performance. Do you think LeBron James stops lifting weights, even though he is a freak athlete who is already in the 0.01% of athletes? Nope, he keeps training. And he has been training since he was a little kid, before any NBA dreams became a reality. Likewise for Amazon and ecommerce sellers: in order to reach peak performance (ie the level of multi-million dollar annual sales), disciplined training and improvement begins with split testing. Regimented, consistent, split testing.


But what happens if you are a new Amazon seller, or don’t have a lot of traffic, sales, or conversion history? This is a moot point, because there are always things that you can test, and ways to structure your tests, to get valuable insights.


In this post, I want to review some tests that you can do regardless of how many sessions or conversions your amazon listing gets on a daily basis.


Before we get started, what exactly do I mean by “low volume”? According to Brian Eisenberg, a conversion rate expert, you can consider anything less than 5 to 10 conversions per week to be “low volume”.


Here are four ideas on how you can split test your product, even if you have low volume:


Idea #1. Easy Wins: Price


If you are just launching your product, there is a lot of room to optimize. So I think it is best to start with the “easy wins”—those tweaks that can be easily made and also have big impacts. I think the biggest impacts can be made by trying the different prices, and trying different main feature images.


But note: don’t test both of these at the same time, because then that would be a “multivariate test”, and you would not be able to identify which split test or adjustment was responsible for which change.


If you test a different price, the metrics you will want to monitor will be overall conversions, conversion rate, and profit. If you think about the customer psychology for a moment, you can see why this is a high-impact, quick win test that you can run. If you price your product on the lower end, then you may be appealing to a mass of price-conscious consumers. You may get more sales and convert more visitors into customers, but perhaps your overall profits are down. Or alternatively, you can go up-market and premium pricing. Though you may get fewer sales, do you have a greater overall profit?


You can see here in this example that I have been testing two different price points in Splitly. Once I created this test, and entered in my fixed and variable costs, Splitly automatically rotates the pricing on my product so that I get even exposure to each price point.


price test results


So here’s the nice part about having the automated statistical calculations of Splitly: I can easily tell how certain the statistics of a particular test will be, and I get an estimated time for the test to be completed.




So you can see that this product doesn’t necessarily get a lot of sales (about 3 sales per day), and I still have to keep testing before I can get a conclusive takeaway, but thus far there are some good indicators that a change in pricing can have a significant impact on profits.


Like I said, even without a lot of traffic or sales, start with pricing because it can have the largest immediate impacts.


Idea #2: Lower Your Confidence Interval


One compromise you can make with your split tests when you have low Amazon traffic is to lower your confidence interval. Statisticians use the term confidence level to indicate the level of uncertainty associated with a set of data. For example, you could choose to have your test run with a 90% confidence interval, which means that if you ran the same test over and over again, you could expect that the same result would occur 90% of the time.


Ideally, you want a higher confidence level to improve the certainty of your test. However, if you have less traffic or conversions, you can lower your confidence interval to 80% or so (Splitly will automatically calculate the interval for you, so don’t worry about the complicated calculation).


With a lower confidence interval, you are basically running fewer tests to derive your conclusions, and you are making more assumptions in the data. If you are aware of these limitations, and account for them in your takeaways, you can still run meaningful tests on a smaller data set.

Idea #3: Be Patient, Test Longer


The logical course of action to increase your data sample is to simply wait longer, and collect more data. This does require patience, and once you make the decision to wait longer, don’t peek….ok, go ahead and look at the test while it’s running, but don’t change anything and let the data accumulate.


Although your test may not be complete, checking in on the progress of your split test is part of the fun and excitement. Here’s an example of a test result, before any conclusions can be drawn. But you see that all of the most important metrics, like sales, conversion rate, sessions, profits, and more are tallied up on a daily basis.


As you run tests for a longer period of time, no matter how little traffic you get on a daily basis, you will see more numbers accumulate in this dashboard. Be patient, test longer.


Idea #4: Increase Traffic


I don’t like to mix traffic sources while running a test, as organic and paid traffic can have different behavior and qualities. But I do like to generate more reviews for my product, and boost it higher on Amazon’s search engines.


This means that I will intermittently pause a test, run some promotions on Review Kick, and then resume a test after I have gathered more reviews. This process may take a few days, but once the test is back up and running, I will have more reviews and presumably more traffic.


I find that this strategy of one step back (pausing a campaign to get more reviews), leads to two steps forward (more traffic and sales to the page, and better test results).




If you follow these four steps to initiating split tests for your Amazon product, you will quickly start optimizing your listing and improving your sales. Do you think a young Lebron James just instantly became a physical specimen without a workout regimen?


becoming-lebron-james-disney-header - from si

image from sports illustrated

No, he worked like hell, put himself through rigorous tests and improved incrementally every day. If you do the same with your Amazon listing, you will also be a force to be reckoned with in your respective niche. Work hard now, reap Amazon glory forever!

Sharing is caring!

Leave a Reply

Your email address will not be published. Required fields are marked *