Confirming Why SendGrid Built Its Own MTA

A number of people, both inside SendGrid and out, have asked me about our decision to build our own message transfer agent (MTA) versus using one of the commercial products that exist in the market. In light of recent market news concerning the merger of two commercial MTA vendors, I thought it would be a good time to share my thoughts on this.

Back in the day

In the very beginning, SendGrid used a modified version of Postfix, one of the most popular open source MTAs, to receive and deliver email. This worked pretty well until we started running into some of its scaling limitations. Postfix is a really great MTA for most cases, but at the scale we were running, it was really starting to hate life. For instance, we had a customer who had a viral invitation campaign that saw their traffic grow from 10k emails per day to 10M in the course of just a few days. How we were using it, combined with the traffic patterns our customers were seeing, was putting a serious hurt on us.

Development of KAMTA

To alleviate the issues we were having we decided to investigate developing our own MTA. The work on this predated our series A funding round, and since at that time we were purely running the system from our profits, we couldn’t really afford to buy a scalable commercial MTA.

We had a lot of experience in what the MTA needed to do, and decided to develop one ourselves to be both scalable and integrated into the other systems needed for a multi-tenant cloud email IaaS. This way, our MTA wouldn’t have the bottlenecks we had been experiencing. When we beta tested it with our main reseller customer at the time the performance was so much better than we had been experiencing that he said “You should call it the“Kick Ass MTA.” And we did. It solved a major bottleneck and allowed us to get back to developing the rest of our system without worrying about how our MTA could scale.

Build vs. buy

Eventually we did receive funding, and at that time we considered replacing our own KAMTA with a commercial “off the shelf” MTA. While good products were available, we felt that delivering email was so core to our business that we did not want to give up our ability to customize that essential component of our infrastructure. It didn’t really make sense to rip out a piece of our system that was running fine to then figure out how to integrate with some new product. The benefits of a commercial MTA were outweighed by the time, cost, and risk to integrate. We did not want to sacrifice the freedom of owning our entire tech stack.

Was this a good call?

As I said at the beginning, people still ask if this was the right decision. I can say with complete certainty that as a cloud email infrastructure service, it was. Owning our own tech stack has been a major contributor to a number of innovations that we have been able to achieve in the past few years, including:

• Working with Google as the first ESP to integrate with Gmail’s spam complaint feedback loop, which helps us identify who’s sending unwanted mail.
• Creating Enforced TLS, allowing customers to choose to help push the requirement of encryption on the email community.
• Being one of the leaders in the cloud email industry to enable opportunistic TLS for all customers, which increases the security/privacy of messages as they’re transferred.

The opportunistic TLS work is one where the advantage of owning our own MTA really shows. As discussed in a previous blog post we were able to test our code against over 1.8 million different mail servers on the internet to ensure that when we turned it on for our customers it would work as expected the first time, resulting in over 90% of our traffic being encrypted.

After we released this feature one of our competitors did the same, but then wrote a blog post of how they were manually verifying domains and that only 25% of their traffic was being encrypted at that point. As an aside, and completely unrelated, about 25% of our traffic goes to Gmail, who was the only mailbox provider making a big deal about TLS at the time.

And if being able to innovate wasn’t enough, we have recently seen a ton of changes in the commercial MTA industry, with the number one company buying out number two, and launching a competing cloud service. I’m extremely glad we are not in a position in which we compete with our vendor for our core technology.