Workflow Improvements: Resolution and Silence

One of our biggest challenges at Sentry is the signal to noise ratio. Errors are unpredictable in so many ways. Even with the large amounts of structured information we gather, it’s still sometimes hard to understand what matters and, more importantly, who it matters to. While we continue to work towards solutions to this problem, we’re constantly looking for ways to improve the day-to-day workflow of Sentry. That workflow primarily focuses on new issues, issues that are more predictable and often caused by a recent change to the application. As part of that, we’re expanding on two concepts in Sentry: resolution and silence.

Silencing Issues

The option to ignore an issue in Sentry has existed for many years. It lets you tell Sentry that you don’t want to see updates for a given exception or at the very least don’t want to see them right now. We started off by just allowing you to permanently ignore the issue and later followed up with giving you duration-based triggers. For example, you might not care about a database error right now as you’re performing maintenance, but if it happens again after 24 hours it’s probably important. This helped in many situations, but it was still not handling one big case: impact.

Our latest addition to silencing focuses on letting you ignore issues until an impact threshold is hit. This currently includes two core subjects: the number of occurrences and unique users affected. Additionally each subject lets you target a rate of impact or a delta. For example, you might just want to know when an event affects 100 users, but alternatively you might want to focus on if it happens again another 10,000 times. All of these triggers are now available via issue actions.

Resolution Within Releases

Resolution is a huge part of Sentry. It tells us intent: “I’ve resolved this issue and don’t expect it to occur again.” This lets Sentry intelligently understand when an issue is a true regression. It’s an important part of any triage workflow and holds us accountable for ensuring that when an error is resolved it’s truly resolved. Just like the silence options inside of Sentry, resolution is an old concept that’s still evolving. We again started with simply letting you say “this is resolved and shouldn’t happen again.” That worked for a while, but we quickly found ourselves wanting to tell Sentry that it was “about to be resolved.”

This workflow is something you’re already familiar with in issue trackers: you make a change and reference the open issue. When that change gets landed, the issue is marked as resolved. To combat that, we initially added “Resolved in Next Release.” It’s a fairly unintelligent mechanism which instructs Sentry to treat the issue as resolved unless it’s seen in a release that’s newer than the most recent.

That works great, but it’s not as accurate as we’d like. So we’ve now added two additional options: resolved in current release, and resolution in an explicit release. These are simply extensions of what we’re doing before, but allow you an even greater degree of correctness in the workflow.

These are still stopgap measures, as what you really want is for Sentry to be commit-aware. Fortunately, we’ve also tackled that with commit annotations. Commit annotations will require you to configure our new deploy tracking, but they ensure absolute correctness. The best part is it’s a workflow you’re already familiar with, and it works just like any other issue tracker:

billing: fix novat option

Fixes SENTRY-3R7

Sentry’s able to correlate this commit with a deploy, and we’ll automatically deal with resolution once that changeset has been shipped.

Forever Evolving

We think these are huge steps forward in managing issue triage, but as with everything we’re always looking at what’s next. We intend to provide full integration with code reviews (e.g. Pull Requests on GitHub) as well as even better visibility into triaged issues. Some of that’s already showing up in the UI. We’re exposing more details about the status of an issue when it’s been resolved or silenced, but also looking at pushing this information into your favorite third-party tools.

Stay tuned!