Comments on: DOE, Rae, Me, Mu$ic to Your Ear Part I – DOE and Test & Learn http://blog.aspenms.com/2010/01/27/doe-rae-me-muic-to-your-ear-part-i-%e2%80%93-doe-and-test-learn/ Built to Sell. Engage. Retain Fri, 07 May 2010 16:33:31 +0000 http://wordpress.org/?v=2.9.2 hourly 1 By: Michael Woo http://blog.aspenms.com/2010/01/27/doe-rae-me-muic-to-your-ear-part-i-%e2%80%93-doe-and-test-learn/comment-page-1/#comment-19429 Michael Woo Sat, 01 May 2010 17:44:09 +0000 http://blog.aspenms.com/?p=379#comment-19429 howdy there, i just came across your web portal on google, and i must say that you express exceptionally well via your web portal. i am really taken by the method that you write, and the subject is great. anyhow, i would also love to know whether you would love to exchange links with my web portal? i will be to the great extent than willing to reciprocate and put your link off in the blogroll. waiting for your reply, i would like to convey my appreciation and enjoy your day! howdy there, i just came across your web portal on google, and i must say that you express exceptionally well via your web portal. i am really taken by the method that you write, and the subject is great. anyhow, i would also love to know whether you would love to exchange links with my web portal? i will be to the great extent than willing to reciprocate and put your link off in the blogroll. waiting for your reply, i would like to convey my appreciation and enjoy your day!

]]> By: Nathaniel Lin, PhD http://blog.aspenms.com/2010/01/27/doe-rae-me-muic-to-your-ear-part-i-%e2%80%93-doe-and-test-learn/comment-page-1/#comment-13515 Nathaniel Lin, PhD Tue, 02 Feb 2010 20:49:55 +0000 http://blog.aspenms.com/?p=379#comment-13515 I agree that to achieve incrementality by holding a representative random sample from the "treatment" is hard to sell to the business stakeholders and sometimes impossible to justify the opportunity cost due to the size of the revenue risk as you have rightly pointed out! This however should not stop one from assessing the incremental impacts of marketing "treatment". It is, in fact a great segue way to my next blog on the use of Propensity Score Matching (PSM) to assess the incremental impacts when no random hold out sample is available or even possible. Rather than trying to remove the sample biases by creating different control groups, PSM allows one to remove the biases by using a single propensity score. The PSM method is still relatively novel to marketers but is gaining popularity for the very reasons that you have alluded to! I agree that to achieve incrementality by holding a representative random sample from the “treatment” is hard to sell to the business stakeholders and sometimes impossible to justify the opportunity cost due to the size of the revenue risk as you have rightly pointed out!

This however should not stop one from assessing the incremental impacts of marketing “treatment”. It is, in fact a great segue way to my next blog on the use of Propensity Score Matching (PSM) to assess the incremental impacts when no random hold out sample is available or even possible. Rather than trying to remove the sample biases by creating different control groups, PSM allows one to remove the biases by using a single propensity score. The PSM method is still relatively novel to marketers but is gaining popularity for the very reasons that you have alluded to!

]]> By: Chris Clegg http://blog.aspenms.com/2010/01/27/doe-rae-me-muic-to-your-ear-part-i-%e2%80%93-doe-and-test-learn/comment-page-1/#comment-13238 Chris Clegg Thu, 28 Jan 2010 20:52:18 +0000 http://blog.aspenms.com/?p=379#comment-13238 Well said. I find the incrementality the hardest part. It is hard for a Client, while committed to the idea of testing the outcomes, to hold back on what everyone believes to be a great plan and risk revenue for the sake of the test. I often find myself managing this by using multiple controls to establish baselines thus allowing me to identify the incremental impact. Sometimes if I segment the control groups I can identify the other factors I would have tested and compare performance among controls in light of the true test (marketing target group). Way off base or a reasonable compromise? Welcome your thoughts. Chris @portma Well said. I find the incrementality the hardest part. It is hard for a Client, while committed to the idea of testing the outcomes, to hold back on what everyone believes to be a great plan and risk revenue for the sake of the test.

I often find myself managing this by using multiple controls to establish baselines thus allowing me to identify the incremental impact. Sometimes if I segment the control groups I can identify the other factors I would have tested and compare performance among controls in light of the true test (marketing target group).

Way off base or a reasonable compromise? Welcome your thoughts.

Chris
@portma

]]>