Directly Speaking

In really tough times, we have to be smarter marketers. We have to find ways to do more with less. This means that creating valid, insightful tests is more important than ever. Here are some of the questions that require testing to answer:

*How will our rolling 12-month revenue per customer with an opt-in email be impacted by a reduction in catalog contacts?
*What is the incremental value of mailing catalogs to customers who only shop via web or at retail?
*Should we be emailing our lapsed customers with an opt-in email address every week (multiple times each week), or can we save on email costs by reducing the number of email contacts to this group?
*What is the incremental profit per customer with and without different types of promotions?
*What is the long term impact of the promotions we offer today?

All of these questions can be answered by setting up the right tests. Here are a few key rules:

1. Understand your expected response when you are selecting your segment sizes- you need enough orders to be statistically valid.
2. Only compare like groups. We all know this goes without saying. The segments in the test must be equal representations of the customer group in question (a random nth).
3. Only compare like segment sizes. Your segments must be the same size- comparing the results of a 25K test cell against a 200K control cell is not valid.
4. Don’t try to test too many variables at once. For example, using the same segments to measure the incremental value of mailing catalogs to web only customers AND creating a test to optimize your catalog/email contact strategy is hard to accomplish in one mailing unless you have large segment sizes.
5. Focus on creating a few, meaningful tests. For example, if you know you will always mail catalogs to your 0-12 month customers even if they have an email on file, then only test limiting catalogs to your lapsed customers or low dollar customers. If you know you would not give up mailing catalogs to your older customers unless they have an opt-in email address on file, then make sure you are only testing a segment with opt-in email addresses.
6. When testing promotions, consider the open use of the coupon code on the web and how you will account for those sales. When you look at the subsequent value of customers acquired with a promotion, be sure to include the initial costs of the offer.
7. Create sustainable control groups that last over a season (at least) and understand the prior contact history of customers in these control groups when you are evaluating results.


These are basic rules for tests that we are all aware of, but as we position ourselves to learn more and optimize our businesses in this economy, it is good to remember!