Process SendGrid Events in Real-time with TempoDB Martyn Davies May 23, 2013 Product, Technical // SUMMARIES ?> If you’re a SendGrid customer you will no doubt have taken a look around your statistics dashboard a few times. It’s great isn’t it? Being able to get such a granular view of what’s happening with the email you send through us. I’m fascinated by statistics and analytics so when I stumbled upon Techstars Cloud graduate company TempoDB recently I had to check them out. What is TempoDB? TempoDB is a database designed specifically for storing time series data. This data could be anything really but it’s better used for little pieces of information like thermostat temperatures, network latency or heart rates. Perfect for storing data over extended periods of time, or where the storage requirement is very, very frequent. There is so much potential here, particularly for the new generation of ‘Internet of Things’ connected devices that will be emerging over the next 10 years; a supposed 50 billion. How it relates to SendGrid Hmm, which part of our system can give you data over periods of time? If you don’t know, you haven’t dug deep enough into our awesome docs. The Event API can spit out a tonne of data to wherever you tell it. We show it to you on your Statistics Dashboard, but there’s no reason why you couldn’t also store the same data elsewhere for further, different, or customised analysis. TempoDB is perfect for this. Digging the ‘Event Tunnel’ To test it out, I wrote a small server using Node.js that would tunnel data from the SendGrid Event API through to a TempoDB database as the events come through in real time. The full code is below, but here are the steps it takes: Includes Node.js HTTP and QueryString libraries (Lines 1-2) Includes the TempoDB Node.js library (Line 3) Grabs whatever the Event API sends us and write it out to a time series on our TempoDB (Line 11-27) Listens to HTTP server listens for POST requests. If it gets one it hands the data off to be stored on TempoDB. If the request is not POST then we just kill it, 405 style. (Lines 30-55) The full code, hosted on Github: That’s all it takes to pipe your SendGrid events, like clicks, opens or unsubscribes, into TempoDB. Try it out If you just want to keep a count of the events coming out of your account in a store then you could deploy this code right now, either to Heroku or Nodejitsu or directly to your own server. You might want to tweak it a little once you’ve seen it in action to better suit your specific need. Once you have the server running somewhere publicly accessible, you’ll need to enable the Event Notification app and change the ‘POST Event URL’ setting to the new URL for the TempoDB tunnel… and that’s it – watch the data flow! Then what? That’s really up to you. I would suggest just leaving it initially to build up a decent picture of data you’ll need a few days. After that it’s well worth exploring some of the visualisation options TempoDB offer in their dashboard to give you an idea of what can be done. In part two of this look at TempoDB, I’ll show you how to extract and analyse some of the data into a beautiful (maybe) daily digest email. In the meantime, why not take a look at your own SendGrid stats? If you’re not already digging in, you’re really missing out.