#5 How to Run and Analyze Tests
0 likes240 views
This document provides tips and guidance on testing in direct marketing. It emphasizes that testing allows marketers to know what works rather than guessing. It outlines various elements that can be tested, from email subject lines and envelopes to prices and incentives. The benefits of testing include making businesses safer by avoiding failures and allowing prediction of results. A step-by-step process for running tests is described, involving selecting groups, promotions, comparing and monitoring results. Repeating tests is advised to gain reliable insights. Various online and offline testing examples are also included.
#5 How to Run and Analyze Tests
- 1. Know, Don’t Guess Tests - How to Run & Analyze Them? Direct Marketing tips by Mediapost Hit Mail #5
- 2. Why tests are SO important? The answer is simple. It’s all about knowledge. In direct marketing you know what works and what doesn’t, so you don’t have to guess, nor to rely on someone else’s opinion. There are two major activities that bring successful results in DM: testing and building a continuing relationship with your customers. By testing you will be sure you are not losing money (which, in fact, is what you have to worry first of all).
- 3. What could be tested in DM? In fact, almost everything: Sender/ Subject line for an e-mail campaign; New outer envelope of a mailing; New brochure for your product; Different piece of copy in the promotion; Various order options; Different benefits for the customers; Different prices of your product/ service; The best time to call, send an e-mail or questionnaire; The product you will be offering; Feedback incentives etc. Almost any question you have in direct marketing can be answered cheaply and quickly by a test.
- 4. Testing benefits Failure proof: Tests can protect you from mistakes and thus make your business safer. You can hardly fail. When it’s over: you can predict your business results with quite high level of confidence. Tests save a lot of money: if it doesn’t work, you don’t use it, but you are safe in terms of profitability, because you didn’t spend a lot; if it works, you start using it, and you even try to make it better, thus getting optimal results. Control: You have a full control on your business, because you know exactly what are you doing and what is too risky for you.
- 5. Step by step: – how to run tests 1. Choose a variable to test. It could be price, insert, sender/subject e-mail combination, benefit, brochure, approach etc. 2. Select two similar groups from your database (A and B); A is your control group, B is the test group; in terms of quantity A=B; 3. Define which promotion you will use for A (an existing and working one) and B (the new one for the test); 4. Send the control promotion to group A and the test to group B at the same time within a single campaign; 5. When results come, compare A to B to define the initial winner; 6. Keep monitoring the results after that to define the final winner; 7. When you know the winning promotion, re-test it at least once. 8. After that you will know what works better for you.
- 6. Repetition is crucial: Don’t rely on results from a single test. Repeat tests at least once, preferably twice. Direct marketing is a business for cautious people.
- 7. Some on- and offline test examples A/B split: half of the mailing has one piece of copy and the other one a different one. This is very close to statistical perfection. A minus is that you can only test two to four things at a time. Split-run inserts: thus you can test different media and find which will work best for you. Or you can test different creative. Flexibility is a plus here. A drawback is that initial printing costs are usually very high. Split-run mailings: here you can test new databases before renting them, new approaches before going for them, and different segments before choosing which offer/ product to send to them. Sender/ Subject test (on an e-mail campaign): it could increase your open rate dramatically if properly used. A single word added to the subject line could literally double your results. And it’s amazingly cheap.
- 8. When reading the results: Be careful - initial results may mislead you. Monitor your test results with utmost attention on the value of the sales you make and the customers you attract. Every media has its own response pattern. The phone, obviously, gets instant response. E-mails are much faster than direct mail. Don’t rely on the very initial results; be patient and wait as long as it is needed. The least reliable result is a low response to a small mail quantity. Tests involving small numbers and modest rates are a complete waste. Make your test group large enough in order to get reliable results.
- 9. S IX Our five testing tips: 1. Always test new databases with proven existing promotional materials. 2. Every time you test a new approach, make sure you have your best available offer against it. This is your “control group”. 3. If you want to test higher intensity impact have a control group that you mail as usual and test group that gets mail more frequently. 4. Don’t test two variables simultaneously. For example two different benefits and two different prices. It is extremely important. If you do so, you will never be sure which one made the real difference in results. 5. The cheapest way to test is to use an already existing regular communication (a monthly statement for example) with your customers. In this case you can use even some existing printed inserts with no additional postage costs. 6. Bonus tip: Only test a factor that really matters for you.
- 10. A point to remember If you are testing carefully, you don’t have to worry at all about creative flair. You will know. You won’t guess. Guesswork brings hidden surprises, as it is shown here: And your customers will tell you what works by voting with their own money, at no additional costs for you.
- 11. More to come: 1. Why Do You Need Your Own Database? 2. How to Build a Database? 3. How to Manage Your Database? 4. How to Segment Your Customers? 5. Tests - How to Run and Analyze Them? 6. The Deliverability Problem 7. Personalization 8. Online & SMS Promotions 9. Useful Direct Marketing Tips 10. How to Launch Your CRM Program? 11. The Advantages of Direct Marketing (Summary)
- 13. Our contacts: 9 Sveti Sedmochislenitsi Str., Sofia 1421, Bulgaria Phone: (+359) 2 962 86 29 Fax: (+359) 2 962 86 29 Mobile: (+359) 885 360 715 E-mail: office@mediaposthitmail.bg Web: www.MediapostHitMail.bg just one more...
- 14. Hello, It’s time to become more practical with the fifth presentation in the “Direct Marketing Tips” set. I hope it is useful and you liked it. Please, share your opinion on it with us via e-mail at office@mediaposthitmail.bg, or call us (+359 885 360 715). In case you’ve missed the previous ones, this way you will get all the presentations in our set as soon as they are available. Thanks! Hristo Radichev Country Manager, Mediapost Hit Mail P.S. Please, send this file to a friend or colleague who you think would like these materials, or send us their contacts. They'll get a polite invitation from us (which they can decline) and we can help them build their own direct marketing strategy.