Test Before You Trust: Black Box Email Tools

Test Before You Trust: Black Box Email Tools

Remember that game MASH? It told you how certain major areas of your life would end up. What is your spouse's name? How many kids are you going to have? What kind of car will you drive? The game was certainly entertaining and fun, but not intended to be taken too seriously nor provide accurate information.

In the email marketing world, there are some tools out there that have about the same level of accuracy as MASH, but are unfortunately given much more credit than deserved by those selling them.

What do these tools do for senders?

In any good email delivery problem investigation, the tools should be trying to solve problems in one of three areas:
  1. How you collect email addresses
  2. The right content/frequency/recipient group to send to (as well as when and why)
  3. When to know when to stop sending to recipients to help preserve your reputation
With any tools you use (or are assessing to use) in any of these areas, take note of the “grades” or “names” they give to portions of your content, recipient group, or similar. Then test the proprietary results of those tools against your own raw stats around opens, clicks, spam reports, blocks/bounces, and unsubscribes.

Why you need to be wary

You may find that these tools could give two “grades” to different portions of your email program that actually perform very similarly in terms of raw metrics. Now you know to merge those groups instead of treating them differently as the tool may have suggested.

Remember, most of these tools are trying to take multiple customer problems and synthesize them to a one-size-fits-all user interface or solution. It may be applicable to use with your program, but maybe not the way it is “out of the box.”

How to use these tools

The best ways to use these tools is to talk with your ESP to make sure you can tag their “results” categories in a way that can tie to your actual full sending volume and resulting statistics that show after a campaign. This will let you truly test the results of these tools against real-life scenarios.

SendGrid allows you to use our API along with our categories and unique arguments in a way to tag the segments that are given certain “results” from the tool you’re using in tandem with us as your ESP.
If you don’t use SendGrid, make sure you have a similar mechanism to build on top of the pipeline you send through.
The main area you want to drill down on is to ensure that your “anticipated user reaction” is in line with what your theory was on each segment of your campaign that day/week/month.

An example of this is that you expect recipient addresses that you collect from your website and your app to perform the same.


Find tools that help you know if your theory is right. If the results are written in how the tool “talks” more than the raw insight you’re looking for, make it work for you.

Some of the tools out there promise a lot. But, be careful to not be sold on them because they use new buzzwords or scare tactics. As long as you can tweak the tools to tell that story, they may be useful...even if it isn’t the way they were originally built.

For more information on how to improve your email deliverability rates, check out SendGrid’s 2019 Deliverability Guide.

Happy Sending!

Recommended For You

Most Popular

Send With Confidence

Partner with the email service trusted by developers and marketers for time-savings, scalability, and delivery expertise.