Using machine learning to focus on issues that matter

We’ve been exploring new approaches to make Sentry issues more actionable. Read on to learn how we’re using machine learning to improve issue creation (i.e. grouping) and alerts, with as little configuration as possible, so you can focus on fixing the most important issues.

Using machine learning to dedupe Sentry Issues

For those not familiar with Sentry, a Sentry Issue is a collection of error or performance events aggregated in one single report. When Sentry creates an issue, that step is also referred to as grouping and is critical to having an actionable issue feed. Grouping starts with fingerprinting, or setting a unique identifier for an event based on its characteristics. Events with the same fingerprint are then grouped together.

While this works well most of the time, there are edge cases that can create erroneous groups. For example, updating your SDK version, renaming a function, or adding an extra newline can result in new issues created under our current fingerprinting approach. If new issues are created when they should be a part of an existing one, it can result in more alerts and an even busier issue stream. This is where machine learning comes in - we’re using a powerful embeddings model to build a semantic representation of error stack traces. This representation enables us to efficiently compare errors and automatically identify instances where issues should be grouped together.

While we’ll still be fingerprinting every new event received. If it’s a new hash, we’ll apply another step where the machine learning grouping algorithm determines if there’s an existing issue that this event is semantically identical to. If so, we’ll map it to the existing issue.

In internal testing, we’re seeing a 30-50% reduction in issue creation and far fewer grouping mistakes. While these grouping improvements represent a meaningful step forward in making issues higher quality and therefore more actionable, the next step of the issue workflow, alerts, is another important area to improve.

More actionable alerts and issues with machine learning

For every new project, Sentry sets a default alert rule to alert on every new issue created. This alert rule works well initially, especially for projects with lower error volume where every new issue could be meaningful. However, at scale, this alert rule becomes noisy when not every new issue is important to triage. While users can try to fix noisy alerts by adding filters to their alert rules we want Sentry to work “out of the box”.

We built a new issue priority model that analyzes incoming errors and identifies the ones that you are unlikely to engage with, based on prior actions on similar issues. These issues will be automatically tagged as “low” priority, and will not trigger new issue alerts or appear in your feed by default. In some cases, lower-priority issues can become bigger problems if they spike in volume. We’ve added safeguards where we’ll automatically update the priority of those issues to high priority if we detect escalating volume.

This priority model will first be applied to new issue alerts since those tend to be the most noisy. Over time, we’ll apply this priority model across the product, including the issue stream. As Sentry becomes more intelligent at filtering out the noise, each alert and issue viewed is a higher signal, which means your team can spend more time working on what matters – noticing the theme? ;)

Sentry is just getting started with machine learning

These new features are meant to make Sentry more actionable and easier to use because less time spent debugging means more time building great user experiences.

We are being thoughtful and deliberate about how we incorporate AI and machine learning features and products in Sentry, and understand the data privacy and security concerns related to using this technology. We’ll strive to always be clear on what data we’re using and how, and provide transparent consent mechanisms in the product.

As we look forward to bringing other AI/ML-enabled features to Sentry, your feedback is invaluable. Please sign up to get early access to these new features, and join us on Discord to share your ideas and feedback.