Automatically Submit Data From Received Emails to an HTML Form Using Node and PhantomJS


January 09, 2017
Written by
SendGrid Team
Contributor
Opinions expressed by Twilio contributors are their own

Automatically Submit Data From Received Emails to an HTML Form Using Node and PhantomJS

SendGrid’s Inbound Parse Webhook feature is extremely powerful, but its use cases are diverse and wide-reaching. This post is an example of how SendGrid’s Community Development team uses this tool internally to automate HTTP API calls from incoming emails. Perhaps it will spark some ideas for how you can use the Inbound Parse Webhook for your business.

Our Situation

The SendGrid Accelerate program helps startups worldwide by providing mentorship and resources, and many startups fill out the form on our website to get in touch with us. When a startup submits the form, they are automatically added to our database, and our team receives an automated email so that we can get in touch with them.

We partner with many top-tier accelerators that have internal tools that automate the process of contacting us so that the startups don’t actually have to fill out the form on our website. In these cases, we receive an automated email from the accelerator with information about the startup so that we can connect with them.

However, this is a bit disjointed from a process standpoint. While it is easy for these startups to get in touch with us, their information does not enter our database because the startups never actually fill out the form on our website. We needed an automated way of entering that information into our database without making the accelerators’ or startups’ lives any more difficult.  To make matters more challenging, we do not have an API to explicitly communicate with our database.

Enter Inbound Parse and PhantomJS                         

We love automation, and we also like making it simple for amazing startups to get in touch with us, so we used SendGrid’s Inbound Parse Webhook to solve this problem. Our solution was a Node/Express app that parses incoming emails for relevant information, and then automates the process of filling out the form on our website with the headless WebKit PhantomJS.  With this app in place, it is no different to our team than if startups fill out the form on our website themselves.

Implementation

The implementation of this app is extremely simple. First, we set up an Inbound Parse Webhook endpoint under Settings -> Inbound Parse. With that in place, the contents of any email sent to the domain we specified will automatically be POSTed to the URL endpoint chosen. In our implementation, we chose the endpoint processEmail and created a route to handle the requests.

[script name="processEmail.js" username="devchas" script_id="4a8ba2082127652e48dc488456e94af8"/]

Because the POST request sends the data as mutli-part form data, we need to use middleware to convert the request data into an object we can work with. We chose to work with an npm package called multer because of its simplicity and ease of use. Another option we considered was multiparty. We then pass the request to a function called ParseEmail that parses the body of the email, retrieves the relevant startup information from the email and passes the data to a worker that enters the information into our database.

If the app is unable to parse the emails to get the relevant fields, the app sends an email to our team using SendGrid’s Mail Send API with the original contents of the email, or the fields variable. This app uses SendGrid’s Node.js library to do so. This way our team still receives an email even if there is some sort of error in the parsing process.

In the event we successfully parse the email, we send an object to the worker with the relevant startup info (startupMsg.data) and the original message (startupInfo.originalEmail). The worker then uses a PhantomJS script to submit the startupMsg.data into the form on our website. In the event of an error in the database entry process, we use startupInfo.originalEmail to send our team an email with the original email contents, letting us know there was an error. That way, we’re covered no matter what because we don’t want any startups slipping through the cracks.

[script name="startJob.js" username="devchas" script_id="bc404b0e1ea79bc721b4f8bd418b95c6"/]

Our team has found this to be an extremely easy way to automate a task to keep us streamlined and organized using the Inbound Parse Webhook. We’d love to hear how you’re using this tool as well! Contact us here or share on our Facebook or Twitter page.

Recommended For You



Most Popular


Send With Confidence

Partner with the email service trusted by developers and marketers for time-savings, scalability, and delivery expertise.