Workflow 2.0

This is a conversation we had with our Engineering, Product, and Design (EPD) organization. We are publishing it as we believe it’s important to our customers and fundamental to our open source approach. You can join the conversation on GitHub.

Lately, I am spending a lot of my time thinking about Sentry and its core developer story. I also consider why we haven’t been able to overcome the main challenges we recognized years ago. Part of this may be a symptom of growing the company - or a symptom of doing too much - but it goes deeper than that. So like any good engineer, I set out to root cause this problem.

Let’s pick a starting point. For me, this has always been our customers. Listening to customers - just like listening to employees, or sitting in sales demos - hits you with the recognition that Sentry can’t articulate a compelling product story.

What is the story? Why don’t we dive in a bit more…

Your team is shipping a great new feature this week. Part of that feature relies on a big change that you landed yesterday. The change has gone live. The feature is broken.

When we build a product at Sentry, we tackle a problem that is emotionally valuable to our customers. For us, that customer is the developer, and there are a few problems they are experiencing in this story:

• They may not have known the feature was going live, or even went live.
• The feature is broken, and it might be caused by their change.
• The feature may have been broken for hours at this moment, meaning a significant outage.

These line items are fundamental problem statements posed to a developer. Sentry has chosen to solve these problems, and thus you must ask yourself, what is the compelling story of Sentry as it relates to that problem? What could it be? This is the real question we need to be asking ourselves.

Let me give you my take on what it could be, given the context of Sentry.

You installed Sentry. Your code was released - Sentry let you know that. Sentry also let you and your team know that there were 3 new errors detected from your release, and it did this within minutes of the code going live. You turn off the feature flag which enables the feature, ship a fix, and Sentry helps you verify that the error is no longer happening.

This is the core problem that we set out to solve, but we’re not quite there yet. Our early product addressed part of this problem in a roundabout way by building 90% of that workflow and relying on the concept of a “new error” to be timely and contextual. But even then, it wasn’t addressing the entirety of that core developer situation above. The one that creates real frustration and angst for millions of developers daily. That is, we would send out a notification whenever a new error was seen, which in many situations happened to coincide with a recent change being deployed.

Unfortunately, we haven’t gone back yet and actually completed that mission. We instead settled into a lifecycle of optimization and edge cases - let’s call it duct tape - along the way. While that development has helped us learn and evolve, it hasn’t driven the level of experience and emotional response we’re after. To tackle this we’re going to start peeling off those layers of duct tape and complete the original vision.

So with this in mind, we’re going to do something we’ve not done at this company before. We’re going to pivot a large number of EPD’s resources towards realizing this goal. Just as we aligned cross-team investments around Dynamic Sampling and Hybrid cloud, we will be doing the same with Issues. For many of you, this means that we’re asking you to throw away any current roadmap you have and contribute to the problem at hand.

Why would we do this you ask? Why change a good thing?

Because we are looking to create great outcomes, not simply optimize the bottom line. As our investors mentioned yesterday in the panel, it’s not about focusing on things like revenue, or market conditions, it’s about building a product that customers love and will continue to love. We have a real-life Field of Dreams situation, so let’s leverage that.

To do that we’re going to revisit the core Issues product. While there are many things customers love about it, there are two critical areas that have been left unsolved. Those two areas identified are how we’re going to set our targets, and we’re going to codename this project as Workflow 2.0:

• Deliver an experience that ensures a developer is informed about relevant issues after a new release of their software.

• Expand the Issues product to cover additional current and future concerns, including bringing first-class performance issues.

To deliver on this we will be asking teams to design and develop a plan to tackle the aforementioned workflow challenges. This is going to be an uncomfortable exercise for many teams - especially as we just went through planning - but it is needed for us to move the product forward. We will then execute on those plans over the remainder of this year. Our objective is to deliver substantial, incremental value towards this problem on a monthly basis, rather than our typical quarterly time boxes. Velocity here is just as important as correctness.

Over the next two weeks, we will be building a design and plan to execute on the project. Expect an email from Ian and Vlad shortly on the next steps for the broader EPD org. If you’re in Emerging Tech, Ben will help ensure future investment aligns here.

As far as outcomes go — measuring success is going to be subjective here. We will know we’ve nailed it when customers tell us they have an amazing experience — when they convince us we’ve solved it. We will not be focusing on if we’re seeing ARR growth, or if we’re seeing more Business plan adoption. The only metrics we will be focused on are engagement and retention: are people using Sentry, and are they continuing to use Sentry?

Do you have opinions to share? Share them on GitHub.