<\/span>Consider ease of use<\/span><\/h2>\nThe first part of the answer has to do with the functionality of the ESP. If you want to be testing more, then you have to be able to do more tests. This means ease of use becomes critical when evaluating your email service provider options.<\/p>\n
<\/span>Tests based on data and segmentation<\/span><\/h2>\nIf you wanted to call on other information such as the data in your CRM system or your web analytics, how easily can you integrate that data with the data in the ESP for segmentation purposes? How easy is it to map outside data into the ESP?<\/p>\n
Ideally you want the CRM and email data integrated<\/a> so it is at your fingertips. And you want the segmentation to be as easy as drag-and-drop. The more segmenting you can quickly<\/strong> do (emphasis here on the quickly), the more optimizing you\u2019ll be doing on those segments \u2014 and therefore even more segmenting as you break groups into sub groups and then subs of those sub groups.<\/p>\n<\/span>Automation of the test process<\/span><\/h2>\nConsider the ease of automating different part of the tests\u2014or conversely the difficulty of doing so. Some ESP platforms will automatically send a winner for you. You set up the A\/B test to a small portion of your list with criteria that determines the winner. The platform will automatically send the winner to the whole list\u2026without you having to lift a finger. Some ESPs offer a split testing wizard<\/a> that can automate much of the setup for you.<\/p>\n<\/span>Number of variants<\/span><\/h2>\nMany email marketers take the term A\/B testing very literally and only use 2 versions in their A\/B tests. We recommend testing more variations, if your list size allows for it while keeping tests statistically significant. But does your ESP let you test four or eight or even 16 different versions at the same time? With ease?<\/p>\n
What about block-level testing that lets you test only a small part of an email rather than create different versions? Does the platform allow for multivariate testing?<\/p>\n
<\/span>Evaluating ease of use<\/span><\/h2>\nI can\u2019t stress enough the importance of ease of use because if it\u2019s not easy to do, you won\u2019t do it. Here\u2019s one quick way to evaluate it: Ask the ESP how long it takes to set up a subject line test (or how many steps). Or better yet, ask for a test drive and discover for yourself how seamless or clunky the process is in real life.<\/p>\n
Then there\u2019s the actual testing. When something is easy to do and understand, you\u2019ll do it more. That\u2019s true for testing and optimizing as well: The easier it is for you to test and understand results, the more testing you will do and the more you will be able to optimize your email marketing effectiveness. Now, about understanding the results\u2026<\/p>\n
<\/span>Understanding the results: email analytics<\/span><\/h2>\nThe other part of this answer is the optimizing part: being able to understand the results of all of your testing, and not just understanding, but being able to dig even deeper into the results.<\/p>\n
For example, this screenshot from eMVision<\/a> shows open rates at glance, by time of day, day of week, and day of month. With one click, a marketer can delve into the details of any of this data.<\/p>\n![\"emvision-emailtesting\"](\"https:\/\/www.emailvendorselection.com\/wp-content\/uploads\/2016\/04\/emvision-emailtesting.jpg\")
<\/p>\n
This sounds easy, but is it? No, because most analytics don\u2019t work this way. When it comes to email analytics, the industry seems to be lagging behind a bit in what it offers marketers, namely that the analytics tools are not yet providing the capability needed. According to a Salesforce study on the state of email analytics<\/a>, survey respondents cite the following challenges:<\/p>\nRelying on manual processes to get all the data in one view (53%) is one of the greatest challenges enterprises face today. Additional factors driving enterprises to integrate more data sources into their analytics applications include finding that too much data is left unanalyzed (53%), spending too much time updating spreadsheets (52%), and analysis is performance by business analysts, not end users of the data (50%).<\/em><\/p>\n![\"marketing-analytics-painpoints\"](\"https:\/\/www.emailvendorselection.com\/wp-content\/uploads\/2016\/04\/marketing-analytics-painpoints.jpg\")
<\/p>\n
Email analytics (a part of Business Intelligence) is important because it\u2019s one thing to be able to test A vs. B but something else altogether to be able to identify segments, patterns and trends. Whether the trends are good or bad, you want to be able to act on them, taking advantage of the good trends to do more of the same and taking action to stop the bleeding when the trend is one that\u2019s quickly spiraling downward.<\/p>\n
However, you also need a big picture view because it could be an upswing in one of your metrics coincides with a downward trend in another, meaning you didn\u2019t improve after all. If you get more opens with subject line A, but more click throughs with subject line B, you need to know this. (And this is also why you need to know what you are testing for before you start.)<\/p>\n
In addition\u2014as with the testing\u2014ease of use is critical because you won\u2019t do the analytics if it\u2019s not easy to make sense of the data before you.<\/p>\n
<\/span>Solving multiple marketing challenges<\/span><\/h2>\n
If you wanted to call on other information such as the data in your CRM system or your web analytics, how easily can you integrate that data with the data in the ESP for segmentation purposes? How easy is it to map outside data into the ESP?<\/p>\n
Ideally you want the CRM and email data integrated<\/a> so it is at your fingertips. And you want the segmentation to be as easy as drag-and-drop. The more segmenting you can quickly<\/strong> do (emphasis here on the quickly), the more optimizing you\u2019ll be doing on those segments \u2014 and therefore even more segmenting as you break groups into sub groups and then subs of those sub groups.<\/p>\n Consider the ease of automating different part of the tests\u2014or conversely the difficulty of doing so. Some ESP platforms will automatically send a winner for you. You set up the A\/B test to a small portion of your list with criteria that determines the winner. The platform will automatically send the winner to the whole list\u2026without you having to lift a finger. Some ESPs offer a split testing wizard<\/a> that can automate much of the setup for you.<\/p>\n Many email marketers take the term A\/B testing very literally and only use 2 versions in their A\/B tests. We recommend testing more variations, if your list size allows for it while keeping tests statistically significant. But does your ESP let you test four or eight or even 16 different versions at the same time? With ease?<\/p>\n What about block-level testing that lets you test only a small part of an email rather than create different versions? Does the platform allow for multivariate testing?<\/p>\n I can\u2019t stress enough the importance of ease of use because if it\u2019s not easy to do, you won\u2019t do it. Here\u2019s one quick way to evaluate it: Ask the ESP how long it takes to set up a subject line test (or how many steps). Or better yet, ask for a test drive and discover for yourself how seamless or clunky the process is in real life.<\/p>\n Then there\u2019s the actual testing. When something is easy to do and understand, you\u2019ll do it more. That\u2019s true for testing and optimizing as well: The easier it is for you to test and understand results, the more testing you will do and the more you will be able to optimize your email marketing effectiveness. Now, about understanding the results\u2026<\/p>\n The other part of this answer is the optimizing part: being able to understand the results of all of your testing, and not just understanding, but being able to dig even deeper into the results.<\/p>\n For example, this screenshot from eMVision<\/a> shows open rates at glance, by time of day, day of week, and day of month. With one click, a marketer can delve into the details of any of this data.<\/p>\n This sounds easy, but is it? No, because most analytics don\u2019t work this way. When it comes to email analytics, the industry seems to be lagging behind a bit in what it offers marketers, namely that the analytics tools are not yet providing the capability needed. According to a Salesforce study on the state of email analytics<\/a>, survey respondents cite the following challenges:<\/p>\n Relying on manual processes to get all the data in one view (53%) is one of the greatest challenges enterprises face today. Additional factors driving enterprises to integrate more data sources into their analytics applications include finding that too much data is left unanalyzed (53%), spending too much time updating spreadsheets (52%), and analysis is performance by business analysts, not end users of the data (50%).<\/em><\/p>\n Email analytics (a part of Business Intelligence) is important because it\u2019s one thing to be able to test A vs. B but something else altogether to be able to identify segments, patterns and trends. Whether the trends are good or bad, you want to be able to act on them, taking advantage of the good trends to do more of the same and taking action to stop the bleeding when the trend is one that\u2019s quickly spiraling downward.<\/p>\n However, you also need a big picture view because it could be an upswing in one of your metrics coincides with a downward trend in another, meaning you didn\u2019t improve after all. If you get more opens with subject line A, but more click throughs with subject line B, you need to know this. (And this is also why you need to know what you are testing for before you start.)<\/p>\n In addition\u2014as with the testing\u2014ease of use is critical because you won\u2019t do the analytics if it\u2019s not easy to make sense of the data before you.<\/p>\n<\/span>Automation of the test process<\/span><\/h2>\n
<\/span>Number of variants<\/span><\/h2>\n
<\/span>Evaluating ease of use<\/span><\/h2>\n
<\/span>Understanding the results: email analytics<\/span><\/h2>\n
<\/p>\n<\/p>\n<\/span>Solving multiple marketing challenges<\/span><\/h2>\n
![\"issues-marketing-departments\"](\"https:\/\/www.emailvendorselection.com\/wp-content\/uploads\/2016\/04\/issues-marketing-departments.jpg\")
<\/p>\n