On April 8, the SendGrid account of a Bitcoin-related customer was compromised and used to send phishing emails.
We initially believed that this account takeover was an isolated incident and worked with our customer to help them recover control of their account and minimize the damage of the attack.
After further investigation in collaboration with law enforcement and FireEye’s (Mandiant) Incident Response Team, we became aware that a SendGrid employee’s account had been compromised by a cyber criminal and used to access several of our internal systems on three separate dates in February and March 2015.
These systems contained usernames, email addresses, and (salted and iteratively hashed) passwords for SendGrid customer and employee accounts. In addition, evidence suggests that the cyber criminal accessed servers that contained some of our customers’ recipient email lists/addresses and customer contact information. We have not found any forensic evidence that customer lists or customer contact information was stolen. However, as a precautionary measure, we are requesting a system-wide password reset. Because SendGrid does not store customer payment cards we do know that payment card information was not involved.
Upon discovery, we took immediate actions to block unauthorized access and deployed additional processes and controls to better protect our customers, our employees, and our platform.
Beginning today, and in line with standard practice, we are requesting that all of our customers reset their passwords to all of their SendGrid account access points. For more information on how to reset your password click here.
For the approximately 600 customers who have custom DKIM keys for sending mail, we are requesting that you generate new DKIM keys through our interface and update your DNS records to reflect the change. If you use custom DKIM keys, you will receive a separate email with instructions. For more information click here.
In addition to changing your password, SendGrid also recommends taking the following actions to ensure your account is protected:
Two-Factor Authentication: We encourage all of our customers to enable two-factor authentication, which can effectively prevent unauthorized logins. For instructions on how to do this, please click here.
Guard your credentials: Avoid publishing your credentials to public source code repositories. Cyber criminals use automated scripts to search for published credentials and can exploit them quickly.
Use unique, random passwords: Even though SendGrid salts and iteratively hashes passwords, we recommend that all passwords be unique, randomly generated and stored in a password manager.
We are committed to the development of new features that will improve the security of our platform. This includes:
API Keys: Our engineering team is working to expedite the release of API keys, which will permit our customers to use keys instead of username/password to send mail through our system programmatically, further reducing security exposure. The API keys are in open beta. Find out more here: https://sendgrid.com/docs/User_Guide/Account/api_keys.html
Customer Access Controls: Our engineering team is also expediting the release of IP allow listing, which will permit customers to authorize specific IP ranges to interact with their SendGrid account’s control panel, further reducing security exposure.
Enhanced Two Factor Authentication: Engineering is working on enhancements to our two factor authentication system which will support additional authentication methods and also work for customers who assign multiple credentials to an account.
We realize that email delivery is an essential part of our customers’ regular course of business and we sincerely apologize for all the inconvenience this has caused. Security is a priority to us at SendGrid and we will continue to work hard to earn your trust by making every effort to deliver a secure service.
We value our relationship with you and if you have any additional questions, we encourage you to contact us at support@sendgrid.com. For more information about our security practices and privacy policies visit https://sendgrid.com/privacy.
Q: Was my account compromised? Were the email lists in my account compromised?
A: Identifying the perpetrator(s) of cyber attacks is difficult. While we cannot rule out the possibility that customer lists or customer contact information was stolen, we have no forensic evidence indicating that it was. As a proactive and preventative measure, we’re working with all of our customers to reset their passwords.
Q: What does this mean for our email lists? Should we refrain from sending emails?
A: While you do not need to refrain from sending emails, we recommend that you reset your password and enable two factor authentication. We also recommend that all passwords be unique, randomly generated and stored in a password manager.
Q: Was there any other personal information that the cyber criminal has access to?
A: Evidence suggests that the cyber criminal accessed servers that contained some of our customers’ contact information. However, we have not found any forensic evidence that customer lists or customer contact information was stolen. We have verified that no financial, credit card or payment information was accessed, since SendGrid does not process or store any of that information.
Q: What are you doing to prevent future attacks?
A: Upon discovery, we took immediate actions to block all unauthorized access and deployed additional processes and controls to better protect our customers, our employees, and our platform. We have been working in collaboration with law enforcement and FireEye’s (Mandiant) Incident Response Team to thoroughly investigate this incident and are taking a number of additional actions to increase our system security. The first step is to work with our customers to ensure they have taken all the appropriate precautions to protect themselves.
We are also developing new features that will improve the security of our platform including API keys, IP allow listing and enhanced two factor authentication.
