I joined SendGrid in July 2013 as the company’s first Product Management (PM) leader. At that point, the company was four years old, with 150 employees and an impressive revenue growth rate. Installing and optimizing PM at a rapidly scaling company is hard, so I’d like to share some of our learnings as we tackled various challenges, in hopes that it benefits the broader PM and technology community.
When I joined SendGrid, we used Google Analytics to track activity on our marketing site and the front end of our Customer Portal. We also had the ability to search Splunk logs and do SQL queries to gather information around product usage. But the PM team didn’t regularly consume Google Analytics data, and deeper queries of Splunk and SQL were infrequent and tactical. In short, we weren’t making many decisions based on product usage data, which is something we needed to change.
The specific data we needed to improve our decision making included:
Time series product usage data at the use case level. For example, how many times was our domain authentication feature used week over week?
Usage data organized around customer experience funnels. For example, what are the first five steps we expect most new customers to take in the product, and what are drop off levels at each stage of the setup process?
Usage data combined with customer data such as package type and MRR. What features do high MRR users tend to use? What is the “happy path” through the product for highly engaged users?
Do you have access to the product usage data you need to make good decisions?
When building a new feature or product, it’s critical to set success criteria in advance, so you can gauge success or failure of your investment. Once success criteria are set, then you know what to monitor and can include monitoring requirements in your project definition. Coming up with measurable and business relevant success criteria is challenging. I recommend How to Measure Anything as a strong read on the general topic.
SendGrid’s products have both UI and API surfaces for most every feature. As we learned, there are an abundance of tools for UI product usage tracking, but relatively few options for API product usage tracking. Here are usage tracking tools we’ve used to this point.
Segment for storing API “events.” Our engineering team had to take the step to write code in our product to send off an event to Segment each time a specified user action was taken via API. Segment then normalizes the data and facilitates turnkey integration with tons of other products for data analysis and visualization.
KissMetrics for real-time dashboarding of our usage tracking data. KissMetrics is integrated with Segment and building dashboards is quite straightforward and quick.
SendGrid has a team called Enterprise Data Operations (EDO). The EDO team is in the process of revamping our data warehouse. They are gradually pulling various data sources into the warehouse, and then building dashboards for functional teams to visualize business metrics important to them. For the PM team, this means that our product usage tracking events (from Segment) will be pulled into the data warehouse. We then use a product called Looker to visualize events on top of the data warehouse. Once this is fully rolled out, we will have a view of product usage data that is tied in with customer details like revenue, volume, and package details. Hallelujah! The promised land!
Product usage tracking often gets de-prioritized in favor of the next big feature, but that’s a mistake. Having accurate and relevant product usage tracking helps improve decisions and deliver product experiences your customers will love and value. To get there, you need to determine what additional usage data is required, define your product success metrics, select your tools, and tie it together with business data.
I hope you took something from this post, and best of luck getting the product usage tracking data you need! Stay tuned for Part 5, which will cover the topic of “Investing in Customer Validation.”
I joined SendGrid in July 2013 as the company’s first Product Management (PM) leader. At that point, the company was four years old, with 150 employees and an impressive revenue growth rate. Installing and optimizing PM at a rapidly scaling company is hard, so I’d like to share some of our learnings as we tackled various challenges, in hopes that it benefits the broader PM and technology community.
