Improve Your Email Marketing

Subscribe to the StrongMail Advisor today:

We value your privacy. We will not rent or sell your email address.

StrongMail Webinar Series

Webinar Questions & Answers

Focus: Email Metrics and Bounce Management

Event Date: 03/27/2007

Presenters:

Dave Lewis, eec Bounce Project Co-Chair / VP of Market Development, StrongMail Systems

Deirdre Baird, eec Deliverability Roundtable Chair / President & CEO, Pivotal Veracity

Moderator:

Jeanniey Mullen, eec Founder / Executive Director of Worldwide Email Marketing, OgilvyOne

Summary

More than 650 industry professionals participated in StrongMail's March 27, 2007 webinar on email metrics and bounce management. The interactive presentation explored recent survey research from the Email Experience Council (eec) Deliverability Roundtable that was compiled and analyzed in a white paper entitled, "The State of Email Metrics and Bounce Management." During the course of the presentation, many questions were submitted to the panelists by webinar attendees.

Those questions have been captured and answered by Dave Lewis, a 20-year direct marketing veteran, email marketing expert, and active member of several industry associations. The questions have been divided into four categories for you to explore. For more information on the eec survey data referenced in the Q&As, you can access the executive summary here.

Explore the Question & Answer Resource

Get started exploring the various topics by clicking on the categories and questions below.

Click on a question to reveal the answer. Some questions have been edited for clarity, and redundant questions have been removed. Please contact StrongMail Systems with any comments or questions: 800-971-0380.

Submit Your Own Questions

We hope that you use this resource to engage in the dialogue and also evaluate how your organization uses email metrics and bounce management. Additionally, we encourage you to pose new questions to . Dave will respond to each inquiry, and relevant questions will be added to this active resource.

Email Metrics & Standards

1. What percent use 'delivered' rather than total 'mailed' to calculate their open rate?

According to the eec survey, 50% of ESPs calculate open rates by dividing unique opens by delivered. 17% calculate it by dividing unique open rates by total mailed and 4% by total opens divided by total mailed. Mailers showed greater consensus with 73% calculating open rates by dividing opens by total delivered. However, in the case of both mailers and ESPs, the open calculation is based on a 'delivered' metric for which there's little commonality or visibility.

Close answer

2. Which metric should be used for calculating the ROI on 'delivered' emails?

In the webinar, we referred to 'delivered' being the starting point for any ROI calculation. What we meant is that without delivered email there can be no open, click, conversion – and hence, no ROI.

In estimating ROI on 'delivered' email, you should use 'unique records sent less total failures.' Including records that weren't delivered for whatever reason will distort your ROI estimate.

The following simple formula can be applied to estimate the dollar value of each delivered email:
Click Rate X Online Conversion Rate X Average Sale Size = Online Revenue/Delivered Email

To determine the total dollar value of your delivered email, you'll want to multiply the above by your total delivered email. Again, you should use 'unique records sent less total failures' in the calculation.

Close answer

3. Do most companies calculate open rates based on mailed or delivered?

While 73% of mailers and 50% of ESPs calculate 'opened' as a percent of 'delivered' email, there are significant minorities that use other methodologies. Additionally, there is variation in how companies determine 'delivered' in the calculation and indications that many don't capture all the data or interpret it correctly, so the resulting metric is suspect.

Close answer

4. Do you recommend that a company use one type of measurement versus another for open rate?

The purpose of the eec survey was to determine the state of email metrics and measures. Given the wide variation found, we believe that a cross-industry initiative/discussion is the appropriate next step in examining various metric methodologies and recommending standards and applications.

As stated in the webinar, different metrics are appropriate for different business applications. For example, as a measure of subject line effectiveness in getting recipients into the envelope, 'unique opens divided by delivered' is probably the best metric. However, for examining list effectiveness you'd probably want to use 'unique opens divided by mailed.'

Close answer

5. I've found that if someone does not open an email in a new window in Outlook then it does not count as opened even though it was viewed. Is that common?

Yes. Viewing email in the preview panel of Outlook is one of several limitations to the 'opened' metric. Others relate to the type of email sent, such as plain text versus HTML, and factors on the receiving end, such as image/link suppression. In such cases, it may be impossible for your deployment system to detect that an email has been opened. This can result in underestimating the number of actual opens and a lower open rate. To partially rectify this problem, some systems auto-generate an open should a click occur.

Close answer

6. How do images in email impact the calculation of open rate?

The suppression of images in email and other factors may make it impossible for your deployment system to detect that an open has occurred. This can result in underestimating the number of actual opens and a lower open rate. Auto-generating an open when a click occurs only partially addresses the issue.

Close answer

7. What about opens you get when a consumer views the HTML email on the webpage link? Is anyone including this in their metrics? What are the trends of consumers viewing through the webpage link versus the email itself?

Typically, when a recipient clicks through to a hosted version of an email, the same level of tracking is not available. With a standard email, your email deployment system generally tracks clicks by link and by recipient. It's unusual to find that level of tracking in the hosted version of an email, since everyone goes to the same page and sees the same content. You can get aggregate information, but not user-specific or unique information.

With regard to inclusion of this data in your reporting, most mailers who implement a hosted version of an email track those results in a separate cell or mailing within the campaign. Others use their web analytics platform to track the performance of the hosted version.

At this time, we're not aware of any studies that have evaluated the use of hosted versions of email or consumer trends in viewing hosted versus standard versions. Effective use of this strategy will likely vary based on the email behavior of different segments, the content of the email itself, and whether recipients have images and/or links turned off in their email application.

Close answer

8. While we wait for the standardization of metrics to become a reality, what should we use as a benchmark for delivery, open, click and bounce rates? How can we see where our email campaigns fall against the industry average?

There's no good answer here so long as variation in metric methodologies exist without clear labeling. The lack of reliable industry benchmarks is one of the most compelling reasons for standardization. The best approach for the present is to know how your own core metrics are calculated, seek to compare your own results over time and benchmark your performance to others who you're certain use the same methodology. That's the only way to ensure a valid, apples-to-apples comparison.

Close answer

9. Is there an initiative underway to standardize email metrics and bounce management practices? How much buy-in can you reasonably expect from the major ESPs?

There are no industry initiatives to standardize metrics underway at the present time. We envision a cross-industry group to address the issue as the logical next step. Buy-in from the major ESPs is expected. After all, it's certainly not in the best interest of the ESPs or mailers who are producing good results to perpetuate our metrics mess.

Close answer

10. Is the eec going to recommend or develop an industry standard on how metrics should be calculated?

To date, our objective on the eec has been to raise industry awareness about the email metric and bounce management issues. In large measure that objective has been achieved. As mentioned in the whitepaper and webinar, the next step is to form a cross-industry group to examine different metric methodologies and their applications, recommend consistent definitions, calculations and labels, and ultimately propose standards. It's expected that the eec will play a role along with other industry trade groups, constituencies and thought leaders.

Close answer

Back to topics

Bounce Management

1. As a marketer, I rely on my IT department and ESP to manage bounces for me. Why should I care about the particulars of how my bounce management system operates?

There's nothing wrong with reliance on your IT department and ESP to manage the actual application of bounce data. However, as an email marketer, it's important that you understand how "delivered" is defined and how your bounce management system works in meeting that definition. While others may act on your bounce management rules, you should be the one defining them and overseeing their consistent application. As with other forms of direct marketing, the keys to success often lie in the details, and the particulars of bounce management represent some of the essential details of successful email marketing.

As discussed in the webinar, there are three good reasons why you as a marketer should know how your bounce management system works. They are:

  1. Performance evaluation. Bounce management provides crucial performance data on your use of the email channel for campaigns, products and brands.
  2. List management. The data provided is central to keeping your list clean and to maintaining or restoring communications with customers.
  3. Practice improvement. Your system can furnish a wealth of data for diagnosing issues with your marketing practices (data capture, targeting, etc.) and for taking the corrective action that will ensure both a good reputation and better deliverability.

Close answer

2. Please review the requirements of a good bounce management system again. Why are they important?

As we discussed in the webinar, there are different ways to calculate "delivered." But regardless of the calculation used, the reliability of your answer depends on the accuracy and completeness of the data that goes into it. And your bounce management system is the source of that data.

In assessing the integrity of a bounce management solution, there are five factors to consider:

  1. Capture all data streams. Make sure that the system captures all data streams – both synchronous and asynchronous bounces. A synchronous bounce occurs at the time of the SMTP transaction and takes the form of a code and text message indicating the reason for non-delivery. An asynchronous bounce occurs after the SMTP transaction and takes the form of a "bounce email" that's received along with other inbound emails you get as a result of your mailing. This second data stream can represent a sizable portion of your bounces, particularly if mailing to corporate domains or smaller ISPs. Unfortunately, this is the data stream that's often overlooked or incompletely captured by most bounce management systems.
  2. Correctly interpret data. The right solution will be able to process the incoming bounce data across the data streams and correctly interpret the myriad of inconsistent messages received from ISPs and other domains. This capability doesn't come out of a recipe book. It requires interrogating and comparing both the codes and text messages and applying confidence codes to different ISPs based on the reliability of the data received. Correct interpretation takes continual testing and tuning.
  3. Organize (normalize) data. No one can cope with hundreds of different bounces messages, so the next step is to normalize the data and organize it into logical categories, such as permanent ('hard') bounce, transient ('soft') bounce, block and technical failure. A good system will then map failures into sub-categories, such as 'unknown user' under permanent bounce. Identifying the reasons for failure is crucial to your ability to diagnose the underlying causes and take the corrective actions that will improve your bottom-line results.
  4. Make data actionable. Once organized, the system should generate reports that make the data actionable in addressing the causes of failures. The reporting should be user-friendly and available near real-time so you can easily and quickly take action to hygiene your list, adjust your targeting and modify other practices to improve the deliverability of your email. Ideally, the reports will feature pop-up definitions of the bounce categories and sub-categories with prompts on the likely causes and corrective actions you should consider. Your reporting system should also feature intuitive drilldowns that allow you to use it as a diagnostic tool in examining failures by the categories or the domains that are important to you. You should then be able to quickly cross-reference your failure data with other key metrics on the same campaign – open, click, unsubscribe and complaint rates. Finally, your system should enable you to pull up sample records to validate the data mapping and the conclusions you've drawn from it.
  5. Be continually updated. Even the best conceived bounce management system will degrade rapidly in our ever-changing environment. Therefore, this last criteria is an important one – it must contain a "future proofing" provision to stay abreast of changing ISP bounce codes and messages

With a bounce management system that meets these requirements, you'll be in a position to properly evaluate your performance, manage your list and improve your practices – all of which translate into better bottom-line results.

Close answer

3. What typical percentage of bounces comes back coded as 'other' (bounce reason not known)?

There's no definitive answer to this question given the wide variation in how bounce management systems capture, interpret and report bounce data. However, 'other' and 'unknown' reasons should not be viewed as synonymous. The term 'other' should apply to bounces where the reason is known but where the incidence is low or they don't cleanly fit into predefined categories. On the other hand, 'unknown' should refer to bounces where the reason can't be determined based on current rules. In both cases, records in the 'other' and 'unknown' categories should be continually examined and remapped as appropriate. Should the combined incidence of 'other' and 'unknown' bounces exceed 3-5% or total bounces, it's time to take a hard look at your bounce management system.

Close answer

4. How can you act on the bounce data from the ISPs if you can't always trust the accuracy of what they're sending you?

The reliability of bounce data both within and between ISPs is a legitimate concern that was raised by both mailers and ESPs in the eec survey. However, this concern cannot become an excuse for senders not making good use of the data that is provided, leveraging bounce management systems that allow them to 'see' the data, and agreeing on common definitions for the actions to be taken. The eec survey found these to be significant problems as well.

There are initiatives underway at the Messaging Anti-Abuse Working Group (MAAWG), an industry trade group principally comprised of ISPs and other receivers, to address the consistency of email delivery notices and policies. Sender organizations, such as the Email Sender & Provider Coalition (ESPC), as well as many individual companies are engaged with the ISPs on this issue as well. While improved ISP transparency is exceedingly important, it's important to recognize that it will take time to adopt and implement the appropriate changes.

In the meantime, senders must make best use of the bounce data they get from the ISPs today. This means you need a bounce management capability that enables you to differentiate between failures due to undeliverable records, such as 'bad address,' and those due to practices the ISPs find unacceptable, such as 'spam block.' Visibility is the key to diagnosing the cause of deliverability problems and to driving corrective actions. The last thing you want to do is retain 'bad' records while dropping 'good' ones, or to re-mail into an ISP without rectifying the practice problem that may have precipitated a block.

In terms of invalidating 'bad' records, a good bounce management system will assign confidence codes to the bounce data received from different ISPs. You should utilize this data in building your re-try and invalidation rules. Applying a universal 3- or 5-bounce rule is not the best strategy; doing nothing is a recipe for disaster.

Close answer

5. As someone who sends out email on others' behalf, what can I do if clients continually give me records that produce high 'hard' bounce rates? How can I clean their lists to avoid bounces and protect my reputation as a sender?

You've got a tough problem that's common to many in your position, including ESPs. You're assuming the risk for the practices of your clients. Unfortunately, many clients expect their provider to solve their deliverability problems even though they may be of their own making. While you can provide informed guidance and tools, the hard truth is that you can't solve their practice problems for them. What you can do is implement processes to prompt greater ownership and accountability, take measures to protect your own reputation, and ensure good results for those clients who do adhere to industry best practices.

As a first step, you'll want to segment your bounces, and particularly permanent ('hard') bounces, by client so you can assess relative list quality and isolate those clients whose lists are most problematic. Of course, you'll then want to encourage those clients with particularly dirty lists to exercise tighter controls with their own list sources and points of data capture, such as registration pages. Aside from identifying clients with weak list management practices, you'll likely find correlations to other list-related problems, such as spam traps, closed accounts or stale records, which can precipitate complaints or blocks that impact the deliverability of your clients' lists as well as your own reputation.

Obviously, correcting the practice deficiencies that produce bounces at their source is your best bet. Second best is identifying and correcting problems before you send and reap the consequences in your delivery rate and reputation. There are email address hygiene software packages and services from companies like Return Path and FreshAddress that allow you to identify and fix or suppress some bad addresses. You should consider applying them to all new lists received from your clients. While you can't determine some problems in advance of sending, your address hygiene results can be very informative. A high incidence of bad addresses is a danger sign that other problems may be lurking in a list as well.

Knowing the quality of your clients' lists, you can further mitigate your sending risk by assigning individual IP addresses to clients or by grouping them into risk categories and segregating their IPs and sub-domains. This approach minimizes the 'common fate' syndrome you may be suffering from today and makes clients more accountable for their own practices. It also enables you to set up sending rules associated with the risk profile of individual clients or groups of clients, such as throttling and re-try rates. This can be extended to how you utilize the real-time data coming from your bounce management system, such as failures or complaints relative to known ISP blocking thresholds, to manage the sending process. Lastly, you'll want to closely examine your bounce results by client to understand the reasons, underlying causes and appropriate corrective actions. At minimum, you should be removing permanent ('hard') bounces, such as bad addresses and unknown users, from your clients' lists and taking steps to ensure they're not subsequently re-introduced in the new lists you receive from them.

Implementing these strategies won't be easy. You'll need to modify your own processes as well as the expectations and, ultimately, behaviors of your clients. To get these jobs done, you'll need a sophisticated deployment engine with differentiated sending capabilities and a smart bounce management system. While not trivial, these may be minor issues compared to the bottom-line consequences of progressively worsening delivery rates for both you and your clients.

Close answer

6. What are the specific error codes and messages to look for in developing a bounce management system? Are there particular resources where I can get the details needed?

The interpretation of error codes and messages is only one of the components to consider in evaluating or building a bounce management system. Bounce information for the major ISPs can often be found in the postmaster sections of their websites. The protocols for the different error codes and messages are also published (see RFCs 2821 and 3463). However, the number of codes/messages and their inconsistent adherence across thousands of ISPs and other domains make the accurate interpretation of bounce data difficult. Additionally, a good bounce management system will be continually tuned and updated to accommodate the new and changed codes/messages.

Given how integral bounce management is to deliverability and the enormity of the undertaking, you should probably consider a commercially available solution, such as StrongMail, or that of an ESP. Use the bounce management requirements outlined in the webinar as your yardstick for assessing different solutions. Don't attempt to build your own unless you're prepared to make a significant initial investment and an ongoing commitment to maintaining it.

Close answer

7. Why does the industry continue to use the terms 'hard' and 'soft' bounces?

Given the confusion and different definitions applied to these terms, it's probably best for the industry to adopt new nomenclature. However, even new nomenclature, such as 'permanent' bounce, doesn't totally rectify the problem since ISP use of 'permanent' bounce codes and messages can signify many things. For instance, some 'permanent' bounces, such as a spam block, tell the sender that the record should not be re-tried until the underlying practice problem has been addressed, whereas other permanent bounces, such as an unknown user, signify that the record is undeliverable and should be corrected or replaced.

This suggests that the real issue runs deeper than nomenclature. Senders must have visibility into the exact bounce reasons and build their rules for correction action based on those reasons.

Close answer

8. What are you defining as a "soft" bounce?

The term 'soft' bounce is one of the terms that should probably be changed. But whether the term is 'soft' or 'transient' bounce, it refers to a temporary condition, such as 'mailbox full.' However, it's important to note that some bounces of this nature are precursors of a permanent condition, so records that repeatedly 'soft' bounce should probably be addressed in your invalidation rules.

Close answer

9. The challenge with providing detailed 'soft' bounce data to marketers is their ability to assimilate and act on it. For instance, how do you deal with 'grey listing' soft bounces?

We'd agree. The trick in building a good bounce management system is in aggregating bounce data into logical and actionable categories. 'Grey listing' is a good example. That particular type of transient 'soft' bounce is used by ISPs to combat the 'pump and dump' strategy of spammers. Legitimate email senders will generally retry their 'soft' bounces during a deployment whereas spammers often don't. Should a marketer see a high incidence of 'grey listing' bounces on a failure report, it may signify a retry logic or ISP relations issue – both things that are actionable by the marketer.

Close answer

Back to topics

Best Practice Questions

1. What suggestions do you have to boost open rates? What's a good test size and how often should you test?

As with all other forms of direct marketing, testing should be an integral part of your email marketing program, and the email medium is particularly well suited to testing since you can quickly and cost-effectively read the results. The impact of timing, targeting, subject lines, content and links are all items you'll want to test relative to their impact on your open, click and conversion rates. Just be sure not to test so many variables that you can't read the results. The size of your test depends on your current open, click or conversion rates, and how many records you'll need to mail to see a statistically significant difference.

Ultimately, your open rate is an indicator of how inclined recipients are to get into the email envelope. As such, the factors that most directly impact your open rate relate to your list, subject line and 'From' address (brand). When it comes to email, some of the tricks you may use with direct mail don't apply or are even counter productive. In a recent Email Sender and Provider Coalition (ESPC) study of consumer email behavior, it was found that 80% base their decision to delete or hit the spam button based on whether they recognize the sender of the email and can infer its purpose from the subject line and From address. And they make these decisions quickly and en masse without opening individual messages. Aside from targeting the right people, this suggests that your subject line should be clear and concise and that both the subject line and From address should reinforce your brand, particularly if it's known to recipients. Testing in these areas is where you should start in your efforts to improve open rates.

Close answer

2. Do you suggest mailing bad-address customers to find out their correct or updated email address?

Absolutely not! There's nothing in the bounce message from the ISPs that will tell you the new email address and re-mailing bad addresses is a sure way trigger blocking at some point. Your best action is to remove bad addresses from your list and seek to programmatically fix them if due to syntax errors or find replacement records through email appends, reverse appends or offline communications. If you're experiencing a high incidence of bad addresses, you'll want to investigate your list sources to isolate and correct the cause. For instance, you may find that simple controls or edits on your registration page may rectify the problem. This is another example of where knowing the exact reason for failure can help you take the appropriate corrective action.

Close answer

3. Would it be beneficial to remove email subscribers from a mailing list after several soft bounce occurrences?

Your objective should be to restore communication as rapidly as possible. This not only helps your bottom line, but avoids spam complaints that may result from a long lapse in communication. This means that some permanent ('hard') bounces, such as bad addresses, should be removed immediately if you're confident in the accuracy of the ISP reporting the failure. But it also means that all bounces should be subject to removal at some point if they continue to bounce for the same reason. So if you can't address the underlying cause of a bounce, such as a spam block, you should consider tagging the record, removing it from your active mailing list and attempting to find a replacement address. There's little point in continuing to re-mail records that you know will bounce.

The same principle applies to transient ('soft') bounces. However, you may want to set the rules for removal based on how frequently you turn your list. For example, if you turn your list every week and hit a 'mailbox full' bounce, you may want to allow 4 or more bounces for the same reason before affecting removal. On the other hand, if your list turn is monthly, you'll want to consider a lower tolerance since the customer may not remember you if you wait 4 months before initiating remedial action.

Close answer

4. Do people tend to turn images on manually on a per email basis or do they add mailers to their safelist?

We know of no definitive research in this area. Our belief is that consumers engage in both behaviors. Adding a mailer to their 'safelist' or address book is probably a function of their level of engagement with a specific brand and whether they've been prompted to do so at a point where the value exchange makes sense.

Close answer

5. How do you differentiate between different types of permanent ('hard') bounces, say spam block versus bad address?

The data provided by the ISPs generally enables you to make the distinction between different types of permanent ('hard') bounces. However, accurate interpretation requires referencing both the codes and text messages and assigning some confidence factor to the ISP providing the data. Of course, interpretation is just one component of a good bounce management system. You'll also want to be sure your system captures all the data sources, normalizes the data and organizes it into logical categories, and provides the reporting that allows you to act on the different types of permanent bounces you may be experiencing. Lastly, be sure your system is continually tuned and updated since the ISP codes and messages are subject to change.

Close answer

6. What does it mean when customers see our email and hit the spam button? Why don't they just opt-out from receiving the email? Each time someone hits the spam button it increases our spam percentage, which is frustrating.

When customers hit the spam button they're rescinding their permission for you to send them email. Of course, that recession can take many forms. They can also unsubscribe or simply delete your email (no open or click activity). It's important that you monitor these indicators individually and collectively to assess the satisfaction of your customers with the content you're sending them.

Interestingly, in its study of the email behavior of consumers, the Email Sender and Provider Coalition (ESPC) found that only 20% use the spam button in lieu of unsubscribing. Trust in the unsubscribe function is actually very high, and over 80% of consumers use it to stop receiving email they'd previously requested from companies. However, an equal number said they base their decision to hit the spam/junk button or delete email on the subject line and 'From' address, and they make this decision quickly and en masse without opening individual messages. This behavior suggests that email marketers need to give special attention to both items, particularly the visibility of their brand if they've got a pre-existing relationship.

If you're experiencing a high incidence of spam complaints, you should examine how engaged customers are with your brand. You'll likely find that your spam complaints as well as unsubscribes are coming from those customers who are least engaged (no or low open and click activity). Your strategy should be to either engage these customers with content relevant to their needs or remove them from your list. Analysis on spam complaint activity has consistently shown a direct correlation between the level of engagement and propensity to complain – the lower the engagement, the higher the risk. In email marketing the 'holy grail' isn't the number of customers on your list. It's the number of engaged customers on your list.

Close answer

7. Why do ISPs 'silently delete' email?

The term 'silent delete' refers to the practice of some ISPs and other receiving domains to delete email without providing any bounce or other notice to the sender. It's predicated on the belief that by returning bounce data to spammers they're better equipped to continue their abusive practices. Unfortunately, spammers aren't the only ones impacted by the 'silent delete.'

'Silent delete' is a controversial practice in both the sender and receiver communities. Many of the larger ISPs, such as AOL and Yahoo, have discontinued the practice, but others continue to use it unabated. The practice is particularly prevalent among smaller ISPs and, as B2B marketers know, at corporate domains. The quarantine function of many corporate spam filters that intercepts email between the gateway and inbox and automatically deletes it, with or without recipient review, is a form of 'silent delete.'

There are several strategies you can employ to detect a 'silent delete.' First, organize your delivery data by the ISPs and domains that are most important to you. (This is a function that a good bounce management system should enable you to do.) Examine your delivery, open and click rates. If you see reasonable delivery rates but no/very low open and click rates, that's a good indication that 'silent delete' may be occurring. Second, on major ISPs, look at the mailbox monitoring data from your deliverability service provider (DSP), such as Pivotal Veracity, Return Path or Habeas, and compare it to the delivery stats from your bounce management system. If you see a number of 'missing' seeds from your DSP but reasonable delivery from your bounce management system, that too may be an indication of 'silent delete.'

In advocating a discontinuance of the 'silent delete' practice, the place to start is at home. Check out what your own corporate spam filter is doing, and suggest a change if you find that your company is engaging in the practice. Of course, you'll also want to contact the ISPs and other domains that 'silently delete' your mail and let them know what you think of the practice. And you should let your voice be heard at Messaging Anti-Abuse Working Group (MAAWG) and other industry groups that are examining this practice.

Close answer

8. Does MSN/Hotmail "silently delete" email?

Email that Microsoft's models score as spam is subject to 'silent delete.'

Close answer

Back to topics

Solution and Service Providers

1. In evaluating ESPs, how would I determine if they're on a blacklist? Is there any government agency that they are required to register with to become an official service provider? Can this be verified by a potential customer?

Like virtually all legitimate senders of email, it's likely that an ESP will be listed on one or more minor blacklists, but the impact on the deliverability of your email will probably be negligible. Unless an ESP has a stable of particularly bad players as clients, their reputation should not directly impact you. What can impact you is if you're sharing an IP address with other ESP clients. In that instance, their blacklisting or other reputation demerits could affect the deliverability of your email.

The Federal Trade Commission (FTC) has issued rules under CAN-SPAM that ESPs and other email senders must adhere to in sending email. However, there is no governmental agency that licenses ESPs, and we don't believe that such regulatory oversight is warranted.

Close answer

2. We recommend that clients remove all hard bounces from their lists. If a spam block is considered a hard bounce, and it can be addressed, then it's not really permanent, is it? So should we remove all hard bounces? How many ESPs identify bounces by their specific reason?

You are correct. Different types of permanent ('hard') bounces warrant different actions. As indicated in the webinar, a 'spam block' suggests that the record shouldn't be retried until the underlying practice problem is addressed; whereas another form of permanent bounce, such as 'bad address,' indicates that the record in undeliverable and should be removed from the list. The decision to remove all permanent bounces would depend on whether the cause of the bounce can be addressed within a reasonable period of time. In any event, it's important to tag records differently to reflect the different types of bounces involved and the corrective actions to be taken.

The eec survey indicates that 25% of ESPs don't break out permanent 'hard' bounces from transient 'soft' bounces, and nearly 40% can't isolate bounce reasons. Whether these shortcomings come from their data capture and/or reporting systems are not known.

Close answer

3. How many ESPs actually track bounce data to the level of the specific reason? Is it reasonable to ask them to provide us this level of detail or is it too costly for them to do so?

According to the eec survey, about 60% of ESPs capture the specific reasons for bounces. However, some only provide this data on a special reporting basis. Given the importance of more granular data, it's certainly not unreasonable for you to ask for it. If your ESP can't accommodate your request, it might be time to investigate alternative providers or solutions. Use the requirements outlined in the webinar as your yardstick for assessing different bounce management solutions.

Close answer

4. Is it possible to create a document that outlines how each ESP calculates their metrics? That would make it easier for clients to make comparisons.

The email industry needs greater transparency across the board – from the ISPs that receive email to the ESPs and others who send it. To facilitate valid comparisons, ESPs, as well as the providers of solutions for in-house use, should disclose their methodologies and clearly label their metrics. While comprehensive documentation of ESP practices is unlikely any time soon, some service and solution providers are already moving in this direction. The demands of clients, such as you, will accelerate the trend.

Close answer

5. What's your opinion of using email verification software to clean-up email lists to improve overall deliverability?

While we don't endorse specific products, there are good tools available that allow you to fix common email address errors, such as syntax, and/or suppress obviously undeliverable records. Typically, this entails comparison of your records to the general or ISP-specific protocols for acceptable email addresses and programmatic corrections. Be sure to test the results before incorporating such products into your overall process. You'll want to be sure that programmatic fixes don't introduce other errors into your list. Also, if you're experiencing a high incidence of bad addresses, you'll want to investigate the underlying cause. You may have a problem with data capture on your registration page or with another list source.

Close answer

6. Do ESPs have the ability to actually evaluate the number of emails read? How does the preview pane impact that capability?

ESP and other email deployment solutions do not have the ability to determine if email is actually read. The 'opened' metric is used as a proxy for email read. The ability of recipients to review email without opening in a preview pane is a limiting factor in the 'opened' metric.

Close answer

7. How are the differences in email metrics reconciled by the web analytic companies in their online marketing resource management systems, such as Omniture's Genesis?

Ominture's Genesis is a product that facilitates the integration of various data sources into a single platform. It doesn't seek to standardize the data or reconcile the differences in metric calculations. Having your key stats readily viewable in Genesis may highlight the differences if you're sensitive to the metrics issue or lead to erroneous conclusions if you're not.

Close answer

8. Do you have any resources where we can learn how to develop or assess a bounce management system?

StrongMail's whitepaper, "Getting Smart about Bounce Management," is a good place to start in understanding the bounce management process and criteria for assessing different solutions.

Close answer

9. As an ESP, my company absolutely manages bounce rates. Is it fair to say that if others aren't doing so, they're probably missing other key analytics as well?

Different ESPs have strengths and weaknesses in different areas. However, a weakness in bounce management must be seen as a serious shortcoming given the current environment of email. Whether that weakness extends to other areas is speculation.

Close answer