Use a Mongo REST Interface to Store SendGrid Event Data


Posted on

Perhaps one of SendGrid’s most useful features (other than email sending, of course) is our Event Webhook. This webhook notifies you whenever an email is sent, opened, marked as spam, and clicked through, among a variety of other events. Notification is all well and good, but the real power comes in storing and analyzing the events. You can do this with third party services like Keen.io or store it yourself for slightly more power.

Our event webhook POSTs JSON documents to any URL, this sounds like a job for…

This is a Job For A Mongo REST Interface

A Mongo REST Interface!

Mongo has a number of community built RESTful interfaces and hosted solutions. In this post I’ll go through a few of them, but know that there are many more ways to store your data with Mongo.

If you want to get started storing your events quickly, with minimal configuration, skip down to MongoLab.

Python (Self Hosted)

Kule is Python’s preferred Mongo/REST implementation.

Kule will accept POST requests made to /{collection} (e.g. /sendgrid). Once up and running, just direct our Event Notification App (found in your account under Apps) to POST to your collection’s URL.

With that, you should be good to go! However, before you get too excited, you’ll need to make sure both your target database and collection exist (otherwise you’ll get application errors). Additionally, Kule is wide open, meaning that anyone can access your database, luckily Kule is easily configurable, so you can harden your installation.

Ruby (Self Hosted)

DrowsyDromedary is the preferred REST implementation for Mongo in Ruby.

Out of the box, this interface expects POST requests to be made to /{db}/{collection} (e.g. /events/sendgrid). However it expects only one document per request (i.e. {“this”:["is", "cool"]} and our webhook sends multiple [{"this" : "is"}, {"so" :"cool"}] by default. However, a previous version of our webhook only sent one document per request, so we can get this running out of the box, by making a POST request.

POST https://sendgrid.com/api/filter.setup.json
POST Data
api_user=your_sendgrid_username&api_key=your_sendgrid_password&name=eventnotify&processed=0&dropped=1&deferred=1&delivered=1&bounce=1&click=1&unsubscribe=1&spamreport=1&url=http://your.server.com/events/sendgrid&version=2

With that, you should be good to go, your events will be stored in Mongo! There are some caveats though. You’ll need to make sure both your database and collection have been created, otherwise you’ll get errors and your data will not be stored. Out of the box DrowsyDromedary is wide open, exposing the whole of your Mongo instance, the project’s maintainer has a number of recommendations for locking down the installation. Additionally, this uses our old webhook, which unfortunately is deprecated. If you plan on processing much data it’s likely better to write your own wrapper (or use a third party service like Keen.IO). However, for initial data collection, this method should work great.

MongoLab (Hosted)

MongoLab isn’t the only Mongo Cloud Provider with a REST API, however, it’s the only one with an API that plays nice with our webhook out of the box.

To get started, sign up for a MongoLab account (it’s free) and create a database. Go find and make note of your API Key (found under your Account User). Finally, direct our Event Notification App (found in your account under Apps) to the following URL, changing out the necessary parameters:

https://api.mongolab.com/api/1/databases/<database-name>/collections/<collection-name>?apiKey=<api-key>

Your events will be stored from here on out! Congratulations!


No matter how you do it, storing your SendGrid Event data is highly recommended, it gives you the most granular way to analyze your event data. Even if you don’t think you’ll be using it, it’s good to store it, as you never know when that might change. Luckily, storing it will cost you next to nothing. If you opt to use Mongo, you can store a bunch of data for free using MongoHQ, or any Mongo REST Interface combined with Heroku and any of their Mongo hosts.

Get out there and start storing your data! If you want more general background on use cases, case studies of successful integrations and a technical overview, download the Event Webhook Guide.


Nick Quinlan is a SendGrid Developer Evangelist based out of San Francisco. He works to make developers lives easier by providing them with the help they need to grow their ideas. Give him a shout, @YayNickQ.

Nick Quinlan on Twitter

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>